WO2022262474A1 - 变焦控制方法、装置、电子设备和计算机可读存储介质 - Google Patents

变焦控制方法、装置、电子设备和计算机可读存储介质 Download PDF

Info

Publication number
WO2022262474A1
WO2022262474A1 PCT/CN2022/091747 CN2022091747W WO2022262474A1 WO 2022262474 A1 WO2022262474 A1 WO 2022262474A1 CN 2022091747 W CN2022091747 W CN 2022091747W WO 2022262474 A1 WO2022262474 A1 WO 2022262474A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
camera
magnification
switching
scale
Prior art date
Application number
PCT/CN2022/091747
Other languages
English (en)
French (fr)
Inventor
胡刚
Original Assignee
北京旷视科技有限公司
北京迈格威科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京旷视科技有限公司, 北京迈格威科技有限公司 filed Critical 北京旷视科技有限公司
Publication of WO2022262474A1 publication Critical patent/WO2022262474A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/67—Focus control based on electronic image sensor signals
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/60—Rotation of whole images or parts thereof
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/62—Control of parameters via user interfaces
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/67—Focus control based on electronic image sensor signals
    • H04N23/675—Focus control based on electronic image sensor signals comprising setting of focusing regions
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/69—Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming

Definitions

  • the present disclosure relates to the technical field of image processing, and in particular, to a zoom control method, device, electronic equipment, and computer-readable storage medium.
  • Optical zoom is produced by changing the positions of the lens, object and focus.
  • the angle of view and focal length will change, and the farther scene will become clearer, making people feel that the object is gradually changing. progressive.
  • Digital cameras usually rely on optical lens structures to achieve zooming, but in mobile devices such as mobile phones, due to the insufficient thickness of a single lens, optical zooming of a single lens cannot be achieved, and usually rely on dual-camera or multi-camera modules to achieve zooming.
  • the switching of the camera will be involved; in related technologies, the module can be calibrated in advance, and the images before and after switching can be corrected through the calibration parameters; It cannot truly reflect the positional relationship between the lenses, resulting in image jumps during lens switching, affecting the zoom effect.
  • the present disclosure provides a zoom control method, device, electronic device and computer-readable storage medium, so as to ensure a smooth and stable zoom effect even when the module is aging or collided.
  • Some embodiments of the present disclosure provide a zoom control method, the method is applied to a device configured with a first camera and a second camera; the method includes: acquiring a first image corresponding to the first camera and a second image corresponding to the second camera ; When the user input magnification is in the switching magnification interval, according to the matching relationship between the first image and the second image, determine the translation amount between the first image and the second image, and the relative positional relationship between the first camera and the second camera; According to the focal length of the first camera and the second camera, and the scale of the first image, determine the switching magnification; process the first image according to the translation amount and relative positional relationship, so that when the user input magnification reaches the switching magnification, the first image and the second Images have the same scale and field of view.
  • the above step of acquiring the first image corresponding to the first camera and the second image corresponding to the second camera may include: acquiring the initial images captured by the first camera and the second camera; The image cropping coefficient of the camera; according to the image cropping coefficient of the first camera, the initial image collected by the first camera is clipped to obtain the first image; according to the image cropping coefficient of the second camera, the initial image collected by the first camera is clipped , to get the second image.
  • the above-mentioned step of determining the translation amount between the first image and the second image according to the matching relationship between the first image and the second image may include: determining a specified area from the first image; wherein, the specified area includes the first image Focus area or central area; determine the matching area of the specified area in the second image; determine the translation amount according to the position of the specified area in the first image and the position of the matching area in the second image.
  • the above-mentioned step of determining the relative positional relationship between the first camera and the second camera according to the matching relationship between the first image and the second image may include: obtaining multiple pairs of matching feature points from the first image and the second image; Multiple pairs of matching feature points are used to determine the relative positional relationship between the first camera and the second camera.
  • the method may further include: compressing the scale of the first image and the scale of the second image to a preset scale range; adjusting the compressed The scale of the first image and the second image, so that the scale of the first image and the second image match.
  • the above step of obtaining multiple pairs of matching feature points from the first image and the second image may include: dividing the first image into a plurality of grid areas according to preset grid parameters; A feature point is extracted from the second image, and the feature points matching each feature point are obtained from the second image to obtain multiple pairs of matching feature points.
  • the above-mentioned step of determining the relative positional relationship between the first camera and the second camera based on multiple pairs of matching feature points may include: inputting multiple pairs of matching feature points into the epipolar constraint formula to obtain an essential matrix;
  • the essential matrix determines the relative positional relationship between the first camera and the second camera; the relative positional relationship includes a rotation relationship.
  • the method may further include: setting an error equation Using the line alignment of the first image and the second image as a constraint condition, through the preset optimization algorithm, the value of R wt is calculated when the value of the error equation CostF is the smallest; where, R wt is the optimized rotation relationship; i represents The i-th feature point; n represents the total amount of feature points; y ti represents the y coordinate value of the i-th feature point in the second image; Represents the y coordinate after the matching point of the i-th feature point in the first image is aligned with the i-th feature point in the second image; K t represents the internal parameters of the second camera; Represents the inverse matrix of the internal parameter matrix of the first camera; U w is the first image.
  • the above step of determining the switching magnification according to the focal length of the first camera and the second camera, and the scale of the first image may include: determining the initial magnification according to the focal length of the first camera and the second camera; based on the scale of the first image, Adjust the initial magnification to obtain the switching magnification.
  • the above step of determining the initial magnification according to the focal lengths of the first camera and the second camera may include: determining the ratio of the focal length of the second camera to the focal length of the first camera as the initial magnification.
  • the above step of adjusting the initial magnification based on the scale of the first image to obtain the switching magnification may include: calculating the switching magnification through the following formula: Among them, switchLevel is the switching magnification; ft is the focal length of the second camera; fw is the focal length of the first camera; s is the scale value of the first image; tScale is the image cropping factor of the second camera; fw is the focal length of the first camera; ft is the focal length of the second camera; wScale is the image cropping factor of the first camera.
  • the above-mentioned relative positional relationship may include: a rotational relationship between the first camera and the second camera; the step of processing the first image according to the translation amount and the relative positional relationship may include: performing rotation processing on the first image according to the rotational relationship to obtain the intermediate image; performing translation processing on the intermediate image according to the translation amount to obtain the processed first image.
  • the above-mentioned step of rotating the first image according to the rotation relationship to obtain the intermediate image may include: calculating and obtaining the intermediate image through the following formula: Among them, I′ W is the intermediate image; (u 0 , v 0 ) is the coordinates of the center point of the first image; switchLevel is the switching magnification; userLevel is the magnification input by the user; the warpping length is the difference between the switching magnification and the lowest magnification of the second camera; ⁇ z is the rotation angle on the z latitude in the rotation relationship; is the inverse matrix of H ot ; I W is the first image.
  • the above step of performing translation processing on the intermediate image according to the translation amount to obtain the processed first image may include: determining the translation matrix according to the translation amount and the user input magnification; performing translation processing on the intermediate image based on the translation matrix to obtain the processed image first image.
  • the switching magnification interval may include: the minimum value of the magnification range of the second camera to the maximum value of the magnification range of the first camera;
  • switching the magnification interval may include: the maximum value of the magnification range of the second camera to the minimum value of the magnification range of the first camera.
  • a zoom control device which is set on a device configured with a first camera and a second camera; the device includes: an image acquisition module configured to acquire a first image corresponding to the first camera The second image corresponding to the second camera; a parameter determination module configured to determine the difference between the first image and the second image according to the matching relationship between the first image and the second image when the user input magnification is in the switching magnification interval The amount of translation, and the relative positional relationship between the first camera and the second camera; the magnification determination module is configured to determine the switching magnification according to the focal length of the first camera and the second camera, and the scale of the first image; the image processing module , configured to process the first image according to the translation amount and the relative positional relationship, so that when the magnification input by the user reaches the switching magnification, the first image and the second image have the same scale and field of view.
  • Still other embodiments of the present disclosure provide an electronic device, and the electronic device may include: a processing device and a storage device; a computer program is stored in the storage device, and the computer program executes the above zoom control method when the processed device is run.
  • Still other embodiments of the present disclosure provide a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processing device, the steps of the above-mentioned zoom control method are executed.
  • the zoom control method, device, electronic device, and computer-readable storage medium described above first obtain the first image corresponding to the first camera and the second image corresponding to the second camera; Matching relationship with the second image, determine the translation between the first image and the second image, and the relative positional relationship between the first camera and the second camera; according to the focal length of the first camera and the second camera, and the first image Determine the scale of switching magnification; process the first image according to the translation amount and relative positional relationship, so that when the magnification input by the user reaches the switching magnification, the first image and the second image have the same scale and field of view.
  • This method determines parameters such as the translation amount between the images and the relative positional relationship between the cameras through the matching relationship between the first image and the second image, and processes the first image based on these parameters; at the same time, through the focal length of the camera and the image Determine the scale of the switching magnification; so that when the user input magnification reaches the switching magnification, the processed first image and the second image have the same scale and field of view, realizing smooth switching of displayed images and improving the zoom effect; In addition, Since this method does not require calibration parameters, it can still guarantee a smooth and stable zoom effect even when the module is aging or collided.
  • FIG. 1 is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure
  • FIG. 2 is a flow chart of a zoom control method provided by an embodiment of the present disclosure
  • FIG. 3 is a schematic diagram of a magnification range of a dual camera provided by an embodiment of the present disclosure
  • FIG. 4 is a schematic diagram of another dual camera magnification range provided by an embodiment of the present disclosure.
  • FIG. 5 is a flowchart of a zoom control method using a dual-camera module as an example provided by an embodiment of the present disclosure
  • FIG. 6 is a schematic structural diagram of a zoom control device provided by an embodiment of the present disclosure.
  • Artificial Intelligence is an emerging science and technology that studies and develops theories, methods, technologies and application systems for simulating and extending human intelligence.
  • the subject of artificial intelligence is a comprehensive subject that involves many technologies such as chips, big data, cloud computing, Internet of Things, distributed storage, deep learning, machine learning, and neural networks.
  • computer vision is specifically to allow machines to recognize the world.
  • Computer vision technology usually includes face recognition, liveness detection, fingerprint recognition and anti-counterfeiting verification, biometric recognition, face detection, pedestrian detection, target detection, pedestrian detection, etc.
  • the dual-camera module usually chooses a wide-angle lens with an 80-degree field of view and a 40-degree telephoto lens arranged left and right; among them, the magnification is between 1x-2x, and the wide-angle lens is used to enlarge; when the magnification reaches 2x , switch to the telephoto lens, when the magnification is greater than 2x, use the telephoto lens to zoom in.
  • the lens switching method due to the influence of the manufacturing process of the lens and the module, when switching the lens at 2x magnification, there will often be obvious image content jumps, such as a large translation of the same image content area.
  • one method in the related art is to adopt the hard cutting method, that is, to increase the precision of optical axis control only in the process of model production without doing software algorithm processing.
  • the disadvantage of this method is that it increases the The production difficulty of the model reduces the good rate of the product, but still does not solve the problem of image content jumping when the camera is switched.
  • Another way is to correct the alignment method.
  • the image is stereoscopically corrected. This method can realize the image alignment when the baseline is aligned with the known camera distance; the disadvantage of this method is that once the module occurs Collision, aging, etc., the relationship between the cameras will change. The original calibration parameters are still used to correct the image, and there will be jumps in the image content.
  • the last method is feature point detection alignment.
  • the disadvantage of this method is that the stability of feature points and the effective detection area are uncontrollable, that is, feature points may not be extracted in the region of interest, which leads to image alignment in this way. The effect is also not reliable.
  • embodiments of the present disclosure provide a zoom control method, device, electronic equipment, and computer-readable storage medium.
  • This technology can be applied to security equipment, computers, mobile phones, cameras, tablet computers, vehicle terminal equipment, etc.
  • this technology can be implemented by using related software and hardware, which will be described through embodiments below.
  • an example electronic device 100 for implementing the zoom control method, apparatus, electronic device, and computer-readable storage medium of the embodiments of the present disclosure is described with reference to FIG. 1 .
  • the electronic device 100 includes one or more processing devices 102, one or more storage devices 104, an input device 106, an output device 108, and may also include one or more images
  • the acquisition device 110, these components are interconnected via a bus system 112 and/or other forms of connection mechanisms (not shown). It should be noted that the components and structure of the electronic device 100 shown in FIG. 1 are only exemplary, not limiting, and the electronic device may also have other components and structures as required.
  • the processing device 102 may be a gateway, or an intelligent terminal, or a device including a central processing unit (CPU) or other forms of processing units with data processing capabilities and/or instruction execution capabilities, and may control other devices in the electronic device 100. Data of the components are processed, and other components in the electronic device 100 can be controlled to perform desired functions.
  • CPU central processing unit
  • Storage device 104 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory.
  • the volatile memory may include random access memory (RAM) and/or cache memory (cache), etc., for example.
  • Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, and the like.
  • One or more computer program instructions can be stored on the computer-readable storage medium, and the processing device 102 can execute the program instructions to implement the client functions (implemented by the processing device) in the following embodiments of the present disclosure and/or other desired Function.
  • Various application programs and various data such as various data used and/or generated by the application programs, can also be stored in the computer-readable storage medium.
  • the input device 106 may be a device used by a user to input instructions, and may include one or more of a keyboard, a mouse, a microphone, a touch screen, and the like.
  • the output device 108 may output various information (eg, images or sounds) to the outside (eg, a user), and may include one or more of a display, a speaker, and the like.
  • the image acquisition device 110 can acquire preview video frames or image data (such as images to be corrected or recognized images), and store the acquired preview video frames or image data in the storage device 104 for use by other components.
  • preview video frames or image data such as images to be corrected or recognized images
  • each device in the exemplary electronic device for implementing the zoom control method, apparatus, electronic device, and computer-readable storage medium may be integrated or distributed, such as the processing device 102,
  • the storage device 104, the input device 106 and the output device 108 are integrated into one body, and the image capture device 110 is set at a designated position where pictures can be collected.
  • the electronic device can be realized as an intelligent terminal such as a camera, a smart phone, a tablet computer, a computer, a vehicle terminal, a video camera, and the like.
  • This embodiment provides a zoom control method, which is applied to a device equipped with a first camera and a second camera; A module composed of one or more cameras; the above-mentioned first camera and the second camera are two cameras in the module; as shown in Figure 2, the method includes the following steps:
  • Step S202 acquiring a first image corresponding to the first camera and a second image corresponding to the second camera;
  • the above-mentioned first image and second image may be the initial images collected by the first camera and the second camera respectively, or may be images after the initial image has been processed; for example, in terminal devices such as mobile phones, the IFE(Image Front End, image front-end processing), IPE (Image processing engine, image processing engine) and other hardware modules; the above-mentioned first image and second image can be images output after IFE processing; wherein, the first image is collected by the first camera The output image after the initial image is processed by the IFE; the second image is the output image after the initial image collected by the second camera is processed by the IFE.
  • the device displays the first image; the purpose of this embodiment is to determine a magnification switching point when the user input magnification is constantly changing, and when the user input image reaches the magnification switching point, the displayed image will be changed from the first image Switch to the second image; at the same time, within a certain magnification range before reaching the magnification switching point, the first image is processed, so that when the first image is switched to the second image, the image can transition smoothly without image content jumping Phenomenon.
  • the above-mentioned first camera may be a secondary camera, and the first camera may be a wide-angle lens with a relatively large field of view; the above-mentioned second camera may be a main camera, and the second camera is a telephoto lens.
  • the magnification input by the user is generally low, and at this time, the zooming is implemented through the first camera, and at the same time, the image displayed by the device is the first image.
  • the second camera is switched to achieve zooming, and the image displayed by the device is the second image.
  • the above-mentioned first camera is a telephoto lens; the second camera is a wide-angle lens; in the initial state, the magnification input by the user is relatively high, and at this time the zoom is realized through the first camera, and at the same time, the image displayed by the device is the first image .
  • the magnification input by the user keeps decreasing, at a certain magnification switching point, the second camera is switched to achieve zooming, and the image displayed by the device is the second image.
  • Step S204 when the magnification input by the user is in the switching magnification range, according to the matching relationship between the first image and the second image, determine the translation amount between the first image and the second image, and the relative position of the first camera and the second camera relation;
  • the magnification range of the camera is determined by the hardware of the camera; after the module is installed, the magnification range of the first camera and the magnification range of the second camera can be determined;
  • the magnification ranges of the two cameras have a certain overlapping range, and when the magnification input by the user reaches the overlapping range, the calculation of the above-mentioned translation amount and relative position relationship starts.
  • the above switching magnification range can be understood as the overlapping range of the magnification range of the first camera and the magnification range of the second camera.
  • the magnification input by the user is in the switching magnification range, the user input magnification belongs to both the magnification range of the first camera and the magnification range of the second camera.
  • the matching relationship between the first image and the second image can specifically be the matching relationship of a specific region between the first image and the second image, and can also be the matching relationship of specific feature points; based on these matching relationships, determine the first image and the second image
  • the amount of translation between the two images, and the relative positional relationship between the first camera and the second camera; the relative positional relationship here may be one or more of positional relationships such as rotation relationship and xyz coordinate relationship.
  • module parameters such as focal length, and various parameters such as internal parameters and external parameters of the first camera and the second camera may also be involved in the calculation.
  • Step S206 determining the switching magnification according to the focal lengths of the first camera and the second camera, and the scale of the first image
  • the switching magnification can be directly obtained through the focal length of the first camera and the second camera; but if the module is aging or a collision occurs, the focal length of the camera may change, and then the original calibration focal length The determined switching magnification will be inappropriate, and the phenomenon of image content jumping will still occur when the camera is switched at this switching magnification. Based on this, in this embodiment, when determining the switching magnification, not only the focal lengths of the first camera and the second camera need to be considered, but also the scale of the image, so as to improve the accuracy of switching the magnification.
  • the initial value of the switching magnification may be determined through the focal lengths of the first camera and the second camera, and then based on the scale of the first image, the initial value of the switching magnification may be adjusted to obtain the final switching magnification.
  • the initial value of the switching magnification may also be adjusted based on the scales of the first image and the second image at the same time to obtain a final switching magnification.
  • Step S208 process the first image according to the translation amount and relative positional relationship, so that when the magnification input by the user reaches the switching magnification, the first image and the second image have the same scale and field of view.
  • the amount of translation here can be understood as the same image content, the difference between the positions of the first image and the second image;
  • the relative position relationship can be understood as the position of the first camera in the coordinate system with the second camera as the origin of the coordinate system,
  • the relative positional relationship usually includes a rotational relationship, a translational relationship, and the like.
  • the translation amount and relative position relationship can reflect the difference between the first image and the second image, and the first image can be processed through the translation amount and relative position relationship so that the first image and the second image have the same scale when switching magnification and field of view, that is, it can be understood that the first image is the same as the second image.
  • the magnification range in order to achieve a smooth transition of the image, when the magnification range is close to but not reached, the magnification range can also become a warping interval; the user can enter the magnification into this interval, and the first image can be processed.
  • the specific parameters for processing the translation amount and relative position of the first image should also change.
  • these parameters can form a linear function with magnification as an independent variable, so that the user input magnification is in the warping interval , the processed first image gradually approaches the second image, and when the magnification input by the user reaches the switching magnification, the scale and field angle of the first image and the second image are the same.
  • the display image is switched from the first image to the second image, there is no obvious content jump in the image.
  • the magnification input by the user reaches the switching magnification, the second image is displayed.
  • the above zoom control method first acquires the first image corresponding to the first camera and the second image corresponding to the second camera; when the user input magnification is in the switching magnification range, according to the matching relationship between the first image and the second image, determine the first The amount of translation between the image and the second image, and the relative positional relationship between the first camera and the second camera; according to the focal length of the first camera and the second camera, and the scale of the first image, determine the switching magnification; according to the translation amount and The relative positional relationship processes the first image so that when the magnification input by the user reaches the switching magnification, the first image and the second image have the same scale and field of view.
  • This method determines parameters such as the translation amount between the images and the relative positional relationship between the cameras through the matching relationship between the first image and the second image, and processes the first image based on these parameters; at the same time, through the focal length of the camera and the image Determine the scale of the switching magnification; so that when the user input magnification reaches the switching magnification, the processed first image and the second image have the same scale and field of view, realizing smooth switching of displayed images and improving the zoom effect; In addition, Since this method does not require calibration parameters, it can still guarantee a smooth and stable zoom effect even when the module is aging or collided.
  • This embodiment focuses on the manner of acquiring the first image and the second image. Firstly, the initial images captured by the first camera and the second camera are obtained; and the image cropping coefficients of the first camera and the second camera are determined according to the magnification input by the user.
  • common cameras include an ultra-wide-angle lens with a field of view of 120 degrees, an 80-degree wide-angle lens, a 45-degree telephoto lens, and a 25-degree periscope lens;
  • the combination, and the combination of the telephoto lens and the periscope lens are common combinations of dual camera modules.
  • the above-mentioned user input magnification is usually input by the user of the device operating a related control or button of the device.
  • the device is usually provided with a magnification conversion library, and the user inputs the magnification into the magnification conversion library, and then the above-mentioned image cropping coefficients of the first camera and the second camera can be output.
  • the magnification conversion library can determine the user input magnification as the image cropping factor of the first camera; calculate the image cropping factor of the second camera through the following formula:
  • tScale is the image cropping factor of the second camera
  • fw is the focal length of the first camera
  • ft is the focal length of the second camera
  • wScale is the image cropping factor of the first camera.
  • both tScale and wScale have a certain range; among them, when tScale is less than 1, set tScale equal to 1.0; and, when wScale and tScale are greater than 2.0x, both set to a maximum value of 2.0, the maximum value depends on The maximum magnification depends on the cropping of the IFE module; if it exceeds the maximum value, it will easily lead to blurred images; of course, according to the different processing capabilities of the device, the above maximum value can be smaller or larger.
  • wScale is generally reduced to increase the redundancy of the field of view by about 10 degrees, so as to avoid the rotation and translation of the first image. After translation processing, black borders appear.
  • the solid line with arrows represents the magnification range of the first camera
  • the dotted line with arrows represents the magnification range of the second camera
  • black dots represent The position point of switching magnification
  • the warping interval is between the minimum value of the magnification range of the second camera and the switching magnification range
  • the dual camera is between the minimum value of the magnification range of the second camera and the maximum value of the magnification range of the first camera interval.
  • the above-mentioned dual-camera interval can also be understood as a magnification switching interval.
  • the switching magnification interval includes: the minimum value of the magnification range of the second camera to the maximum value of the magnification range of the first camera .
  • the solid line with arrows represents the magnification range of the first camera
  • the dotted line with arrows represents the magnification range of the second camera
  • black dots represent The position point of switching magnification
  • the maximum value of the magnification range of the second camera to the switching magnification is the warping interval
  • the maximum value of the magnification range of the second camera to the minimum value of the magnification range of the first camera is a dual camera interval.
  • the above-mentioned dual-camera interval can also be understood as a magnification switching interval.
  • the switching magnification interval includes: the maximum value of the magnification range of the second camera to the minimum value of the magnification range of the first camera.
  • the initial image collected by the first camera is cropped to obtain the first image; according to the image cropping coefficient of the second camera, the initial image collected by the first camera is cropped , to get the second image.
  • the clipping operation can be realized through the aforementioned IFE model, and the specific clipping method can be center clipping or other clipping methods.
  • the translation amount between the first image and the second image can be determined through the following steps 30-34:
  • Step 30 determining a specified area from the first image; wherein, the specified area includes a focus area or a central area of the first image;
  • the focus area of the first image can be understood as the image area where the focus point of the first image is located, and the scale of the image area can be preset; the user can click on the screen to determine the click position as the focus point, thereby determining the focus area, the designated area at this time may be the focus area. If the user does not determine the focus point, the central area of the first image can be determined as the specified area, and the scale of the central area can also be preset.
  • Step 32 determining the matching area of the designated area in the second image
  • the matching area that matches the specified area can be obtained from the second image by means of image recognition or feature point matching; the image content of the matching area in the second image is usually the same as that of the specified area in the first image.
  • Step 34 Determine the translation amount according to the position of the designated area in the first image and the position of the matching area in the second image.
  • the positions of the first camera and the second camera are different, the positions of the same image content in the first image and in the second image are also different; the position of the specified area in the first image, and the position of the matching area in the second image
  • the position can be represented by coordinates, and the above translation can be obtained by calculating the coordinates.
  • the translation amount can be calculated every 10 frames of images, and the frequency of calculating the translation amount can also be set to other values.
  • the frequency of calculating the translation amount is relatively low. High, mainly because the translation amount of each scene is different, and the translation amount calculated at a higher frequency can obtain the translation amount of the current scene in time.
  • the relative positional relationship between the first camera and the second camera can be determined through the following steps 40-46:
  • Step 40 the scale of the first image and the scale of the second image are compressed to a preset scale range;
  • the resize function can be used to compress the image, for example, compressing an image with a size of 1000*750 into an image with a size of 500*375; Compression can improve the performance of extracting feature points.
  • Step 42 adjusting the scales of the compressed first image and the second image, so that the scales of the first image and the second image match.
  • the operation of this step may also be called image scale alignment.
  • the image scale alignment operation may be performed according to the focal lengths of the first camera and the second camera, and the image cropping coefficients of the first image and the second image.
  • only the scale of the first image may be adjusted, and the calculation method of the scale s of the first image is as follows:
  • tScale is the image cropping factor of the second camera
  • fw is the focal length of the first camera
  • ft is the focal length of the second camera
  • wScale is the image cropping factor of the first camera.
  • Step 44 obtaining multiple pairs of matching feature points from the first image and the second image
  • a certain number of feature points can be selected from the first image, and then the feature points matching each feature point can be found from the second image by means of feature point matching; in order to make the distribution of feature points more uniform, a
  • the first image is divided into multiple grid areas; a feature point is extracted from each grid area, and a feature point corresponding to each feature point is obtained from the second image.
  • Matching feature points get multiple pairs of matching feature points.
  • multiple grid areas can be obtained by dividing the grid into equal parts.
  • feature extraction algorithms such as ORB (abbreviation of Oriented FAST and Rotated BRIEF, where FAST and BRIEF are both algorithm names) are used.
  • Step 46 based on multiple pairs of matching feature points, determine the relative positional relationship between the first camera and the second camera.
  • E is an essential matrix
  • F is a fundamental matrix
  • t is a translation matrix, which represents the translation relationship between the first camera and the second camera
  • R is a rotation matrix, which represents the rotation relationship between the first camera and the second camera
  • K is the internal parameter matrix of the first camera
  • K T is the transpose matrix of K
  • K -1 is the inverse matrix of K
  • x 1 K -1 p 1
  • p 1 is a feature point in the first image
  • p 2 is the phase of p 1 in the second image
  • x 2 K -1 p 2
  • the essential matrix E can be obtained by calculating the above-mentioned epipolar constraint formula through eight pairs of matching feature points, and then use SVD (Singular Value Decomposition, singular value decomposition) and triangular reconstruction principle to decompose to obtain the external parameters, that is, the above-mentioned rotation matrix R and the translation matrix t.
  • SVD Single Value Decomposition, singular value decomposition
  • the rotation matrix R among the external parameters is mainly used.
  • the default principal point is the center point of the image; focal length and principal point are internal parameters of the camera.
  • R wt is the optimized rotation relationship
  • i represents The i-th feature point
  • n represents the total amount of feature points
  • y ti represents the y coordinate value of the i-th feature point in the second image
  • K t represents the internal parameters of the second camera
  • U w is the first image.
  • the above optimization algorithm can be the Lederberg-Marquardt optimization algorithm, so as to optimize the accurate parameter R, and convert R into an angle ( ⁇ x , ⁇ y , ⁇ z ) output .
  • the images are corrected by means of image matching to obtain mutually matched images, and there is no obvious problem of image rotation and translation before and after image switching, so that the images can transition smoothly.
  • This embodiment focuses on the calculation method of the switching magnification, specifically, the switching magnification can be determined through the following steps 50-52:
  • Step 50 determine the initial magnification according to the focal lengths of the first camera and the second camera;
  • the focal length of the first camera is fw and the focal length of the second camera is ft
  • the ratio ft/fw of the focal length of the second camera to the focal length of the first camera can be determined as the initial magnification.
  • Step 52 Adjust the initial magnification based on the scale of the first image to obtain a switching magnification.
  • the switching magnification can be understood as, under the switching magnification, the scale and field angle of the first image and the second image are consistent, and when the user input magnification is the switching magnification, the displayed image is switched from the first image to the second image.
  • the switching magnification can be calculated by the following formula:
  • switchLevel is the switching magnification
  • ft is the focal length of the second camera
  • fw is the focal length of the first camera
  • s is the scale value of the first image, after scale matching, the scale values of the first image and the second image are the same
  • tScale is the image cropping factor of the second camera
  • fw is the focal length of the first camera
  • ft is the focal length of the second camera
  • wScale is the image cropping factor of the first camera.
  • An accurate switching magnification can be calculated through the above method, and switching images under the switching magnification can make the viewing angles of the images before and after switching consistent, so that the switching images can transition smoothly.
  • the relative positional relationship here mainly includes: the rotation relationship between the first camera and the second camera; in this embodiment, the method of first rotating and then translating is mainly used to process the first image, including the following steps 60 and 62:
  • Step 60 performing rotation processing on the first image according to the rotation relationship to obtain an intermediate image
  • the rotation relationship here represents the rotation relationship of the first camera with respect to the second camera, based on the rotation relationship to process the first image, the field angle of the first image can be corrected to match the state of the second image; specifically,
  • the intermediate image can be calculated by the following formula:
  • I′ W is the intermediate image
  • (u 0 , v 0 ) is the coordinates of the center point of the first image
  • switchLevel is the switching magnification
  • userLevel is the magnification input by the user
  • the warpping length is the difference between the switching magnification and the lowest magnification of the second camera
  • ⁇ z is the rotation angle on the z latitude in the rotation relationship
  • I W is the first image.
  • I′ W H rotate *I W ; the inverse matrix of H ot among them can move the image center to the origin of the coordinate system, and the H ot matrix can move the rotated image back to the image center; H r is the image two-dimensional rotation formula.
  • Step 62 performing translation processing on the intermediate image according to the translation amount to obtain the processed first image.
  • the amount of translation here can be understood as the total amount of translation.
  • the total amount of translation needs to be evenly distributed in the warping interval, that is, within the warping interval, there is a linear change between the user input magnification and the translation amount.
  • the magnification is at the maximum value, that is, the magnification is switched, and the translation amount reaches the maximum, that is, the above-mentioned total translation amount.
  • a translation matrix is determined according to the translation amount and the magnification input by the user; based on the translation matrix, translation processing is performed on the intermediate image to obtain the processed first image.
  • the translation matrix in switchLevel is the switching magnification; userLevel is the magnification input by the user; the warpping length is the difference between the switching magnification and the lowest magnification of the second camera; T is the translation amount; t x is the value of the x dimension of t; t y is the value of the y dimension of t value.
  • the userLevel in is equal to wScale at this time, which is the image cropping coefficient of the first camera;
  • the translation amount T is the translation amount under the switching magnification switchLevel, and is passed The amount of translation can be converted to wScale.
  • H algo H shift *H rotate .
  • the cropping process may be performed on the first image by an image processing engine IPE module, and the cropped first image may be displayed.
  • the IPE module usually performs image processing such as noise reduction, color processing, and detail enhancement on the first image, so as to improve the display effect of the first image.
  • the calculation formula of the clipping area above can be applied to the user input magnification in the double-shot interval shown in Figure 3 or Figure 4.
  • the magnification other than the double-shot interval that is, in the single-shot interval
  • Figure 5 shows the flow of the zoom control method taking a dual-camera module as an example;
  • the image data stream in Figure 5 includes the initial images captured by the aforementioned first camera and the second camera, and the IFE module is based on the image cropping coefficient wScale and tScale to adjust the scale of the initial image to obtain the aforementioned first image and second image;
  • the rotation is calculated by the method of the aforementioned embodiment Angle, translation and switching magnification; then process the image according to the rotation angle and translation, and calculate the cropping area Rect according to the switching magnification, translation and userLevel. If the magnification input by the user does not belong to the dual-camera area, the clipping area Rect is calculated according to the magnification input by the user userLevel itself; the calculated clipping area input value IPE module is used for further processing of the first image.
  • a zoom control solution without calibration parameters is proposed. Calculate the accurate rotation angle and switching magnification based on the image matching method. When switching the magnification, the field of view angles of the images corresponding to the two cameras can achieve a smooth transition of the image, which has the same user requirements as the zoom control scheme based on calibration parameters. experience.
  • the basic idea of this solution is: first obtain the basic information of the dual-camera module, including focal length, pixel size, image resolution, and module assembly position parameters (Tx, Ty, Tz). Then calculate the initial switching magnification according to the basic information of the module. Since the magnification is calculated by the module parameters, there is a certain error. Then, the scheme of rotating and translating the image with a large field of view to the image with a small field of view is adopted. In the dual-camera coexistence interval, the multi-threading method is used to calculate the accurate switching magnification of the current module, the image rotation alignment angle and the translation amount. Finally, the image smooth switching scheme can be realized according to the above calculation data.
  • the above zoom control method does not require calibration parameters of the module production line, and the calculation method is simple and reliable, which greatly reduces the production cost of the module, and the performance of the zoom control is good, and can be applied to various terminal equipment such as high-end mobile phones, as well as dual-camera, triple-camera, etc.
  • the optical smooth switching scheme of multi-camera and even multi-camera modules has promoted the promotion of multi-camera zoom solutions.
  • the above-mentioned embodiment uses a dual-camera module as an example to describe the zoom control method. Since a multi-camera module is also composed of dual-camera, the multi-camera module can also apply the zoom control method in the foregoing embodiments.
  • FIG. 6 the device may include:
  • the image acquisition module 60 may be configured to acquire a first image corresponding to the first camera and a second image corresponding to the second camera;
  • the parameter determination module 62 may be configured to determine the translation amount between the first image and the second image according to the matching relationship between the first image and the second image when the user input magnification is in the switching magnification interval, and the first camera The relative position relationship with the second camera;
  • the magnification determination module 64 may be configured to determine the switching magnification according to the focal lengths of the first camera and the second camera, and the scale of the first image;
  • the image processing module 66 may be configured to process the first image according to the translation amount and relative positional relationship, so that when the magnification input by the user reaches the switching magnification, the first image and the second image have the same scale and field of view.
  • the above-mentioned zoom control device first acquires the first image corresponding to the first camera and the second image corresponding to the second camera; when the user input magnification is in the switching magnification interval, according to the matching relationship between the first image and the second image, determine the first The amount of translation between the image and the second image, and the relative positional relationship between the first camera and the second camera; according to the focal length of the first camera and the second camera, and the scale of the first image, determine the switching magnification; according to the translation amount and The relative positional relationship processes the first image so that when the magnification input by the user reaches the switching magnification, the first image and the second image have the same scale and field of view.
  • This method determines parameters such as the translation amount between the images and the relative positional relationship between the cameras through the matching relationship between the first image and the second image, and processes the first image based on these parameters; at the same time, through the focal length of the camera and the image Determine the scale of the switching magnification; so that when the user input magnification reaches the switching magnification, the processed first image and the second image have the same scale and field of view, realizing smooth switching of displayed images and improving the zoom effect; In addition, Since this method does not require calibration parameters, it can still guarantee a smooth and stable zoom effect even when the module is aging or collided.
  • the above-mentioned image acquisition module can also be configured to: acquire the initial images captured by the first camera and the second camera; determine the image cropping coefficients of the first camera and the second camera according to the magnification input by the user; , the initial image captured by the first camera is cropped to obtain the first image; according to the image cropping coefficient of the second camera, the initial image captured by the first camera is cropped to obtain the second image.
  • the above-mentioned image acquisition module can also be configured to: determine the user input magnification as the image cropping factor of the first camera; calculate the image cropping factor of the second camera through the following formula:
  • tScale is the image cropping factor of the second camera
  • fw is the focal length of the first camera
  • ft is the focal length of the second camera
  • wScale is the image cropping factor of the first camera.
  • the above-mentioned parameter determination module can also be configured to: determine the specified area from the first image; wherein the specified area includes the focus area or central area of the first image; determine the matching area of the specified area in the second image; according to the specified The location of the region in the first image, and the location of the matching region in the second image, determine the amount of translation.
  • the above parameter determination module can also be configured to: obtain multiple pairs of matching feature points from the first image and the second image; determine the relative positional relationship between the first camera and the second camera based on the multiple pairs of matching feature points .
  • the above device also includes an image preprocessing module, which can be configured to compress the scale of the first image and the scale of the second image to a preset scale range; adjust the scale of the compressed first image and the second image, so that the scale of the first image The scales of the first image and the second image are matched.
  • an image preprocessing module which can be configured to compress the scale of the first image and the scale of the second image to a preset scale range; adjust the scale of the compressed first image and the second image, so that the scale of the first image The scales of the first image and the second image are matched.
  • the above-mentioned parameter determination module can also be configured to: divide the first image into a plurality of grid areas according to preset grid parameters; extract a feature point from each grid area, and obtain a feature point corresponding to Each feature point matches the feature point, and multiple pairs of feature points that match each other are obtained.
  • the above parameter determination module can also be configured to: input multiple pairs of matching feature points into the epipolar constraint formula to calculate the essential matrix; based on the essential matrix, determine the relative positional relationship between the first camera and the second camera ; The relative positional relationship includes a rotational relationship.
  • the above device also includes a parameter optimization module, which can be configured to: set the error equation Using the line alignment of the first image and the second image as a constraint condition, through the preset optimization algorithm, the value of R wt is calculated when the value of the error equation CostF is the smallest; where, R wt is the optimized rotation relationship; i represents The i-th feature point; n represents the total amount of feature points; y ti represents the y coordinate value of the i-th feature point in the second image; Represents the y coordinate after the matching point of the i-th feature point in the first image is aligned with the i-th feature point in the second image; K t represents the internal parameters of the second camera; Represents the inverse matrix of the internal parameter matrix of the first camera; U w is the first image.
  • a parameter optimization module which can be configured to: set the error equation Using the line alignment of the first image and the second image as a constraint condition, through the preset optimization algorithm, the value of R
  • the above magnification determination module can also be configured to: determine the initial magnification according to the focal lengths of the first camera and the second camera; and adjust the initial magnification based on the scale of the first image to obtain the switching magnification.
  • the above magnification determination module may also be configured to: determine the ratio of the focal length of the second camera to the focal length of the first camera as the initial magnification.
  • the above magnification determination module can also be configured to: calculate the switching magnification through the following formula:
  • switchLevel is the switching magnification
  • ft is the focal length of the second camera
  • fw is the focal length of the first camera
  • s is the scale value of the first image
  • tScale is the image cropping factor of the second camera
  • fw is the focal length of the first camera
  • ft is the focal length of the second camera
  • wScale is the image cropping factor of the first camera.
  • the above-mentioned relative positional relationship includes: the rotation relationship between the first camera and the second camera; the above-mentioned image processing module also includes: performing rotation processing on the first image according to the rotation relationship to obtain an intermediate image; and translating the intermediate image according to the translation amount processing to obtain the processed first image.
  • the above-mentioned image processing module can also be configured to: calculate and obtain the intermediate image through the following formula:
  • I′ W is the intermediate image
  • (u 0 , v 0 ) is the coordinates of the center point of the first image
  • switchLevel is the switching magnification
  • userLevel is the magnification input by the user
  • the warpping length is the difference between the switching magnification and the lowest magnification of the second camera
  • ⁇ z is the rotation angle on the z latitude in the rotation relationship
  • I W is the first image.
  • the above-mentioned image processing module may also be configured to: determine a translation matrix according to the translation amount and the magnification input by the user; and perform translation processing on the intermediate image based on the translation matrix to obtain the processed first image.
  • the switching magnification interval may include: the minimum value of the magnification range of the second camera to the maximum value of the magnification range of the first camera;
  • the magnification interval may include: the maximum value of the magnification range of the second camera to the minimum value of the magnification range of the first camera.
  • This embodiment also provides an electronic device, which may include: a processing device and a storage device; a computer program is stored in the storage device, and the computer program executes the above zoom control method when the processed device is run.
  • This embodiment also provides a computer-readable storage medium.
  • a computer program is stored on the computer-readable storage medium.
  • the steps of the above-mentioned zoom control method are executed.
  • the zoom control method, device, electronic device, and computer program product of a computer-readable storage medium provided by the embodiments of the present disclosure include a computer-readable storage medium storing program codes, and the instructions included in the program codes can be used to execute the preceding method
  • the specific implementation may refer to the method embodiment, and details are not repeated here.
  • connection should be interpreted in a broad sense, for example, it can be a fixed connection or a detachable connection , or integrally connected; it may be mechanically connected or electrically connected; it may be directly connected or indirectly connected through an intermediary, and it may be the internal communication of two components.
  • installation should be interpreted in a broad sense, for example, it can be a fixed connection or a detachable connection , or integrally connected; it may be mechanically connected or electrically connected; it may be directly connected or indirectly connected through an intermediary, and it may be the internal communication of two components.
  • the computer software product is stored in a storage medium, including several
  • the instructions are used to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in various embodiments of the present disclosure.
  • the aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disc, etc., which can store program codes. .
  • the present application provides a zoom control method, device, electronic device, and computer-readable storage medium; wherein, the method includes: when the user input magnification is in the switching magnification range, through the first image and the second image The matching relationship between the images, determine the translation amount between the images and the relative positional relationship between the cameras and other parameters, and process the first image based on these parameters; at the same time, determine the switching magnification through the focal length of the cameras and the scale of the image; thus allowing the user to input
  • the magnification reaches the switching magnification
  • the processed first image and the second image have the same scale and field of view, which realizes the smooth switching of the displayed image and improves the zoom effect; in addition, because this method does not require calibration parameters, the aging of the module Even in the event of a collision or collision, a smooth and stable zoom effect can still be guaranteed.
  • the zoom control method, device, electronic device and computer-readable storage medium of the present application are reproducible and can be used in various industrial applications.
  • the zoom control method, device, electronic device, and computer-readable storage medium of the present application may be used in the technical field of image processing.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Studio Devices (AREA)

Abstract

本公开提供了一种变焦控制方法、装置、电子设备和计算机可读存储介质；其中，该方法包括：当用户输入倍率处于切换倍率区间时，通过第一图像和第二图像之间的匹配关系，确定图像之间的平移量和摄像头之间的相对位置关系等参数，基于这些参数处理第一图像；同时，通过摄像头的焦距和图像的尺度，确定切换倍率；从而使用户输入倍率到达切换倍率时，处理后的第一图像和第二图像具有相同的尺度和视场角，实现显示图像的平滑切换，提高了变焦效果；另外，由于该方式无需标定参数，在模组老化或碰撞的情况下，依然能够保证平滑稳定的变焦效果。

Description

变焦控制方法、装置、电子设备和计算机可读存储介质
相关申请的交叉引用
本申请要求于2021年06月18日提交中国国家知识产权局的申请号为202110682766.2、名称为“变焦控制方法、装置、电子设备和计算机可读存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本公开涉及图像处理技术领域，尤其是涉及一种变焦控制方法、装置、电子设备和计算机可读存储介质。
背景技术
光学变焦是通过镜头、物体和焦点三方的位置发生变化而产生的，当成像面在水平方向运动的时候，视角和焦距就会发生变化，更远的景物变得更清晰，让人感觉物体逐渐递进。数码摄像机通常依靠光学镜头结构实现变焦，但在手机等移动设备中，由于单个镜头的厚度不足，无法实现单个镜头的光学变焦，通常依靠双摄或多摄的模组实现变焦。在变焦的过程中，会涉及到摄像头的切换；相关技术中，可以预先对模组进行标定，通过标定参数对切换前后的图像进行校正；但是，如果模组发生碰撞、老化等情况，标定参数就不能真实反映镜头之间的位置关系，导致镜头切换过程中图像发生跳变，影响变焦效果。
发明内容
本有鉴于此，本公开提供了一种变焦控制方法、装置、电子设备和计算机可读存储介质，以在模组老化或碰撞的情况下，依然能够保证平滑稳定的变焦效果。
本公开的一些实施例提供了一种变焦控制方法，方法应用于配置有第一摄像头和第二摄像头的设备；方法包括：获取第一摄像头对应的第一图像和第二摄像头对应的第二图像；当用户输入倍率处于切换倍率区间时，根据第一图像和第二图像的匹配关系，确定第一图像与第二图像之间的平移量，以及第一摄像头和第二摄像头的相对位置关系；根据第一摄像头和第二摄像头的焦距，以及第一图像的尺度，确定切换倍率；根据平移量和相对位置关系处理第一图像，以使用户输入倍率到达切换倍率时，第一图像与第二图像具有相同的尺度和视场角。
上述获取第一摄像头对应的第一图像和第二摄像头对应的第二图像的步骤，可以包括：获取第一摄像头和第二摄像头采集的初始图像；根据用户输入倍率，确定第一摄像头和第二摄像头的图像裁剪系数；根据第一摄像头的图像裁剪系数，对第一摄像头采集的初始图像进行裁剪，得到第一图像；根据第二摄像头的图像裁剪系数，对第一摄像头采集的初始图像进行裁剪，得到第二图像。
上述根据用户输入倍率，确定第一摄像头和第二摄像头的图像裁剪系数的步骤，可以包括：将用户输入倍率确定为第一摄像头的图像裁剪系数；通过下述算式，计算第二摄像头的图像裁剪系数：tScale＝fw/ft*wScale；其中，tScale为第二摄像头的图像裁剪系数；fw为第一摄像头的焦距；ft为 第二摄像头的焦距；wScale为第一摄像头的图像裁剪系数。
上述根据第一图像和第二图像的匹配关系，确定第一图像与第二图像之间的平移量的步骤，可以包括：从第一图像中确定指定区域；其中，指定区域包括第一图像的焦点区域或中心区域；确定指定区域在第二图像中的匹配区域；根据指定区域在第一图像中的位置，以及匹配区域在第二图像中的位置，确定平移量。
上述根据第一图像和第二图像的匹配关系，确定第一摄像头和第二摄像头的相对位置关系的步骤，可以包括：从第一图像和第二图像中获取多对相互匹配的特征点；基于多对相互匹配的特征点，确定第一摄像头和第二摄像头的相对位置关系。
上述从第一图像和第二图像中获取多对相互匹配的特征点的步骤之前，方法还可以包括：将第一图像的尺度和第二图像的尺度压缩至预设尺度范围；调整压缩后的第一图像和第二图像的尺度，以使第一图像和第二图像的尺度相匹配。
上述从第一图像和第二图像中获取多对相互匹配的特征点的步骤，可以包括：按照预设的网格参数，将第一图像划分为多个网格区域；从每个网格区域中提取一个特征点，从第二图像中获取与每个特征点相匹配的特征点，得到多对相互匹配的特征点。
上述基于多对相互匹配的特征点，确定第一摄像头和第二摄像头的相对位置关系的步骤，可以包括：将多对相互匹配的特征点输入至对极约束公式中，计算得到本质矩阵；基于本质矩阵，确定第一摄像头与第二摄像头之间的相对位置关系；该相对位置关系包括旋转关系。
上述基于本质矩阵，确定第一摄像头与第二摄像头之间的相对位置关系的步骤之后，方法还可以包括：设置误差方程
Figure PCTCN2022091747-appb-000001
将第一图像和第二图像行对齐作为约束条件，通过预设的优化算法，计算得到当误差方程CostF的值最小时，R wt的值；其中，R wt为优化后的旋转关系；i代表第i个特征点；n代表特征点总量；y ti代表第二图像中第i个特征点的y坐标值；
Figure PCTCN2022091747-appb-000002
代表第一图像中第i个特征点的匹配点与第二图像中第i个特征点对齐后的y坐标；K t代表第二摄像头的内参数；
Figure PCTCN2022091747-appb-000003
代表第一摄像头的内参数矩阵的逆矩阵；U w为第一图像。
上述根据第一摄像头和第二摄像头的焦距，以及第一图像的尺度，确定切换倍率的步骤，可以包括：根据第一摄像头和第二摄像头的焦距，确定初始倍率；基于第一图像的尺度，调整初始倍率，得到切换倍率。
上述根据第一摄像头和第二摄像头的焦距，确定初始倍率的步骤，可以包括：将第二摄像头的焦距与第一摄像头的焦距的比值，确定为初始倍率。
上述基于第一图像的尺度，调整初始倍率，得到切换倍率的步骤，可以包括：通过下述算式，计算得到切换倍率：
Figure PCTCN2022091747-appb-000004
其中，switchLevel为切换倍率；ft为第二摄像头的焦距；fw为第一摄像头的焦距；s为第一图像的尺度值；
Figure PCTCN2022091747-appb-000005
tScale为第二摄像头的图像裁剪系数；fw为第一摄像头的焦距；ft为第二摄像头的焦距；wScale为第一摄像头的图像裁剪系数。
上述相对位置关系可以包括：第一摄像头与第二摄像头之间的旋转关系；根据平移量和相对位置关系处理第一图像的步骤，可以包括：根据旋转关系对第一图像进行旋转处理，得到中间图像；根据平移量对中间图像进行平移处理，得到处理后的第一图像。
上述根据旋转关系对第一图像进行旋转处理，得到中间图像的步骤，可以包括：通过下述算式，计算得到中间图像：
Figure PCTCN2022091747-appb-000006
其中，I′ W为中间图像；
Figure PCTCN2022091747-appb-000007
(u 0，v 0)为第一图像的中心点坐标；
Figure PCTCN2022091747-appb-000008
switchLevel为切换倍率；userLevel为用户输入倍率；warpping长度为切换倍率与第二摄像头的最低倍率的差值；θ z为旋转关系中z纬度上的旋转角度；
Figure PCTCN2022091747-appb-000009
为H ot的逆矩阵；I W为第一图像。
上述根据平移量对中间图像进行平移处理，得到处理后的第一图像的步骤，可以包括：根据平移量和用户输入倍率，确定平移矩阵；基于平移矩阵对中间图像进行平移处理，得到处理后的第一图像。
上述根据平移量和相对位置关系处理第一图像的步骤之后，方法还可以包括：确定处理后的第一图像的裁剪区域Rect＝[(w-w/c]/2,(h-h/c]/2,w/c,h/c]；其中，w为第一图像的宽度；
Figure PCTCN2022091747-appb-000010
userLevel为用户输入倍率；wScale为第一摄像头的图像裁剪系数；h为第一图像的高度；根据裁剪区域对第一图像进行裁剪处理。
可选地，当用户输入倍率逐渐放大至切换倍率时，切换倍率区间可以包括：第二摄像头的倍率范围的最小值至第一摄像头的倍率范围的最大值；当用户输入倍率逐渐缩小至切换倍率时，切换倍率区间可以包括：第二摄像头的倍率范围的最大值至第一摄像头的倍率范围的最小值。
本公开的另一些实施例提供了一种变焦控制装置，装置设置于配置有第一摄像头和第二摄像头的设备；装置包括：图像获取模块，配置成用于获取第一摄像头对应的第一图像和第二摄像头对应的第二图像；参数确定模块，配置成用于当用户输入倍率处于切换倍率区间时，根据第一图像和第二图像的匹配关系，确定第一图像与第二图像之间的平移量，以及第一摄像头和第二摄像头的相对位置关系；倍率确定模块，配置成用于根据第一摄像头和第二摄像头的焦距，以及第一图像的尺度，确定切换倍率；图像处理模块，配置成用于根据平移量和相对位置关系处理第一图像，以使用户输入倍率到达切换倍率时，第一图像与第二图像具有相同的尺度和视场角。
本公开的又一些实施例提供了一种电子设备，电子设备可以包括：处理设备和存储装置；存储装置上存储有计算机程序，计算机程序在被处理设备运行时执行如上述变焦控制方法。
本公开的再一些实施例提供了一种计算机可读存储介质，计算机可读存储介质上存储有计算机程序，计算机程序被处理设备运行时执行如上述变焦控制方法的步骤。
本公开实施例至少带来了以下有益效果：
上述变焦控制方法、装置、电子设备和计算机可读存储介质，首先获取第一摄像头对应的第一图像 和第二摄像头对应的第二图像；当用户输入倍率处于切换倍率区间时，根据第一图像和第二图像的匹配关系，确定第一图像与第二图像之间的平移量，以及第一摄像头和第二摄像头的相对位置关系；根据第一摄像头和第二摄像头的焦距，以及第一图像的尺度，确定切换倍率；根据平移量和相对位置关系处理第一图像，以使用户输入倍率到达切换倍率时，第一图像与第二图像具有相同的尺度和视场角。
该方式通过第一图像和第二图像之间的匹配关系，确定图像之间的平移量和摄像头之间的相对位置关系等参数，基于这些参数处理第一图像；同时，通过摄像头的焦距和图像的尺度，确定切换倍率；从而使用户输入倍率到达切换倍率时，处理后的第一图像和第二图像具有相同的尺度和视场角，实现显示图像的平滑切换，提高了变焦效果；另外，由于该方式无需标定参数，在模组老化或碰撞的情况下，依然能够保证平滑稳定的变焦效果。
本公开的其他特征和优点将在随后的说明书中阐述，并且，部分地从说明书中变得显而易见，或者通过实施本公开而了解。本公开的目的和其他优点在说明书、权利要求书以及附图中所特别指出的结构来实现和获得。
为使本公开的上述目的、特征和优点能更明显易懂，下文特举较佳实施例，并配合所附附图，作详细说明如下。
附图说明
为了更清楚地说明本公开具体实施方式或相关技术中的技术方案，下面将对具体实施方式或相关技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图是本公开的一些实施方式，对于本领域技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1为本公开实施例提供的一种电子设备的结构示意图；
图2为本公开实施例提供的一种变焦控制方法的流程图；
图3为本公开实施例提供的一种双摄像头的倍率范围的示意图；
图4为本公开实施例提供的另一种双摄像头的倍率范围的示意图；
图5为本公开实施例提供的一种以双摄模组为例的变焦控制方法流程；
图6为本公开实施例提供的一种变焦控制装置的结构示意图。
具体实施方式
为使本公开实施例的目的、技术方案和优点更加清楚，下面将结合附图对本公开的技术方案进行清楚、完整地描述，显然，所描述的实施例是本公开一部分实施例，而不是全部的实施例。基于本公开中的实施例，本领域技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本公开保护的范围。
近年来，基于人工智能的计算机视觉、深度学习、机器学习、图像处理、图像识别等技术研究取得 了重要进展。人工智能(Artificial Intelligence，AI)是研究、开发用于模拟、延伸人的智能的理论、方法、技术及应用系统的新兴科学技术。人工智能学科是一门综合性学科，涉及芯片、大数据、云计算、物联网、分布式存储、深度学习、机器学习、神经网络等诸多技术种类。计算机视觉作为人工智能的一个重要分支，具体是让机器识别世界，计算机视觉技术通常包括人脸识别、活体检测、指纹识别与防伪验证、生物特征识别、人脸检测、行人检测、目标检测、行人识别、图像处理、图像识别、图像语义理解、图像检索、文字识别、视频处理、视频内容识别、行为识别、三维重建、虚拟现实、增强现实、同步定位与地图构建(SLAM)、计算摄影、机器人导航与定位等技术。随着人工智能技术的研究和进步，该项技术在众多领域展开了应用，例如安防、城市管理、交通管理、楼宇管理、园区管理、人脸通行、人脸考勤、物流管理、仓储管理、机器人、智能营销、计算摄影、手机影像、云服务、智能家居、穿戴设备、无人驾驶、自动驾驶、智能医疗、人脸支付、人脸解锁、指纹解锁、人证核验、智慧屏、智能电视、摄像机、移动互联网、网络直播、美颜、美妆、医疗美容、智能测温等领域。
以双摄模组为例，双摄模组通常选择视场角在80度的广角镜头和40度的长焦镜头左右排列；其中，倍率在1x-2x之间，采用广角镜头放大；倍率达到2x时，切换到长焦镜头，倍率大于2x时，采用长焦镜头放大。上述镜头切换方式中，受到镜头和模组的制造工艺影响，在2x倍率时切换镜头往往会存在较明显的图像内容跳变，如相同图像内容的区域发生较大的平移。
为了解决上述问题，相关技术中的一种方式是采用硬切方法，就是不做软件算法上的处理，只在模型生产过程中，加大光轴管控精度，该方式的不足之处在于增加了模型的生产难度，降低了产品的优良率，但依然没有解决在摄像头切换时的图像内容跳变的问题。另一种方式是校正对齐方法，通过标定方法，对图像进行立体校正，该方式可以实现基线上下对他和已知摄像头距离情况下的图像对齐；该方式的不足之处在于，一旦模组发生碰撞、老化等，摄像头之间的关系就会发生变化，依然采用原始的标定参数校正图像，还会出现图像内容的跳变，另外，难以知道对焦距离的情况下，图像对齐难度依然很大。最后一种方式是特征点检测对齐，该方方式的不足之处在于，特征点的稳定性和有效检测区域不可控，即感兴趣区域不一定能提取到特征点，导致通过该方式进行图像对齐的效果也不可靠。
基于此，本公开实施例提供了一种变焦控制方法、装置、电子设备和计算机可读存储介质，该技术可以应用于安防设备、计算机、手机、摄像机、平板电脑、车辆终端设备等多种带有摄像装置的设备中，该技术可以采用关软件和硬件实现，下面通过实施例进行描述。
实施例一：
首先，参照图1来描述用于实现本公开实施例的变焦控制方法、装置、电子设备和计算机可读存储介质的示例电子设备100。
如图1所示的一种电子设备的结构示意图，电子设备100包括一个或多个处理设备102、一个或多个存储装置104、输入装置106、输出装置108，另外可以包括一个或多个图像采集设备110，这些组件通过总线系统112和/或其它形式的连接机构(未示出)互连。应当注意，图1所示的电子设备100的组件 和结构只是示例性的，而非限制性的，根据需要，电子设备也可以具有其他组件和结构。
处理设备102可以是网关，也可以为智能终端，或者是包含中央处理单元(CPU)或者具有数据处理能力和/或指令执行能力的其它形式的处理单元的设备，可以对电子设备100中的其它组件的数据进行处理，还可以控制电子设备100中的其它组件以执行期望的功能。
存储装置104可以包括一个或多个计算机程序产品，计算机程序产品可以包括各种形式的计算机可读存储介质，例如易失性存储器和/或非易失性存储器。易失性存储器例如可以包括随机存取存储器(RAM)和/或高速缓冲存储器(cache)等。非易失性存储器例如可以包括只读存储器(ROM)、硬盘、闪存等。在计算机可读存储介质上可以存储一个或多个计算机程序指令，处理设备102可以运行程序指令，以实现下文的本公开实施例中(由处理设备实现)的客户端功能以及/或者其它期望的功能。在计算机可读存储介质中还可以存储各种应用程序和各种数据，例如应用程序使用和/或产生的各种数据等。
输入装置106可以是用户用来输入指令的装置，并且可以包括键盘、鼠标、麦克风和触摸屏等中的一个或多个。
输出装置108可以向外部(例如，用户)输出各种信息(例如，图像或声音)，并且可以包括显示器、扬声器等中的一个或多个。
图像采集设备110可以采集预览视频帧或图片数据(如待矫正图片或识别图片)，并且将采集到的预览视频帧或图像数据存储在存储装置104中以供其它组件使用。
示例性地，用于实现根据本公开实施例的变焦控制方法、装置、电子设备和计算机可读存储介质的示例电子设备中的各器件可以集成设置，也可以分散设置，诸如将处理设备102、存储装置104、输入装置106和输出装置108集成设置于一体，而将图像采集设备110设置于可以采集到图片的指定位置。当上述电子设备中的各器件集成设置时，该电子设备可以被实现为诸如相机、智能手机、平板电脑、计算机、车载终端、摄像机等智能终端。
实施例二：
本实施例提供一种变焦控制方法，该方法应用于配置有第一摄像头和第二摄像头的设备；该设备可以配置由第一摄像头和第二摄像头组成的双摄模组，也可以配置由三个或更多摄像头组成的模组；上述第一摄像头和第二摄像头为模组中的两个摄像头；如图2所示，该方法包括如下步骤：
步骤S202，获取第一摄像头对应的第一图像和第二摄像头对应的第二图像；
上述第一图像和第二图像可以为分别由第一摄像头和第二摄像头采集的初始图像，也可以为初始图像经过处理后的图像；例如，在手机等终端设备中，通常配置由IFE(Image Front End，图像前端处理)、IPE(Image processing engine，图像处理引擎)等硬件模块；上述第一图像和第二图像可以为IFE处理后输出的图像；其中，第一图像为第一摄像头采集的初始图像经过IFE处理后的输出图像；第二图像为第二摄像头采集的初始图像经过IFE处理后的输出图像。
在初始状态下，设备显示第一图像；本实施例的目的在于，当用户输入倍率不断变化时，确定一个 倍率切换点，当用户输入图像到达该倍率切换点时，将显示图像由第一图像切换为第二图像；同时，在到达倍率切换点之前一定的倍率范围内，对第一图像进行处理，使得第一图像切换至第二图像时，图像可以平滑过渡，不出现图像内容跳变的现象。
一个具体的示例中，上述第一摄像头可以为副摄像头，第一摄像头可以为广角镜头，具有较大的视场角；上述第二摄像头可以为主摄像头，第二摄像头为长焦镜头。在初始状态下，用户输入倍率通常较低，此时通过第一摄像头实现变焦，同时，设备显示的图像为第一图像。当用户输入倍率不断增加时，在某个倍率切换点下，切换第二摄像头实现变焦，同时设备显示的图像为第二图像。
另一个示例中，上述第一摄像头为长焦镜头；第二摄像头为广角镜头；在初始状态下，用户输入倍率较高，此时通过第一摄像头实现变焦，同时，设备显示的图像为第一图像。当用户输入倍率不断降低时，在某个倍率切换点下，切换第二摄像头实现变焦，同时设备显示的图像为第二图像。
步骤S204，当用户输入倍率处于切换倍率区间时，根据第一图像和第二图像的匹配关系，确定第一图像与第二图像之间的平移量，以及第一摄像头和第二摄像头的相对位置关系；
摄像头的倍率范围由摄像头的硬件决定；模组安装完毕后，第一摄像头的倍率范围与第二摄像头的倍率范围即可确定；且，为了实现稳定变焦，通常设置第一摄像头的倍率范围和第二摄像头的倍率范围具有一定的重合范围，当用户输入倍率到达该重合范围时，即开始计算上述平移量和相对位置关系。上述切换倍率区间可以理解为第一摄像头的倍率范围和第二摄像头的倍率范围的重合范围。当用户输入倍率处于切换倍率区间时，该用户输入倍率同时属于第一摄像头的倍率范围和第二摄像头的倍率范围。
第一图像和第二图像的匹配关系，具体可以为第一图像和第二图像之间特定区域的匹配关系，也可以为特定特征点的匹配关系；基于这些匹配关系，确定第一图像与第二图像之间的平移量，以及第一摄像头和第二摄像头的相对位置关系；这里的相对位置关系可以是旋转关系、xyz坐标关系等位置关系中的一种或多种。在确定平移量和相对位置关系的过程中，可能还会涉及到焦距等模组参数，以及第一摄像头和第二摄像头的内参数、外参数等各类参数参与计算。
步骤S206，根据第一摄像头和第二摄像头的焦距，以及第一图像的尺度，确定切换倍率；
当摄像头的焦距标定的较为准确时，通过第一摄像头和第二摄像头的焦距，可以直接得到切换倍率；但如果模组老化，或者发生碰撞，摄像头的焦距可能会变化，再通过原始的标定焦距确定出的切换倍率，就会不合适，在该切换倍率切换摄像头，依然会出现图像内容跳变的现象。基于此，本实施例中在确定切换倍率时，不仅需要考虑第一摄像头和第二摄像头的焦距，还需要考虑图像的尺度，从而提高切换倍率的准确性。
一种具体的方式中，可以通过第一摄像头和第二摄像头的焦距，确定切换倍率的初始值，然后基于第一图像的尺度，对切换倍率的初始值进行调整，得到最终的切换倍率。另外，也可以同时基于第一图像和第二图像的尺度，对切换倍率的初始值进行调整，得到最终的切换倍率。
步骤S208，根据平移量和相对位置关系处理第一图像，以使用户输入倍率到达切换倍率时，第一图 像与第二图像具有相同的尺度和视场角。
这里的平移量可以理解为相同的图像内容，在第一图像和第二图像的位置之差；相对位置关系可以理解为以第二摄像头为坐标系原点，第一摄像头在坐标系下的位置，该相对位置关系通常包括旋转关系、平移关系等。平移量和相对位置关系可以反映第一图像和第二图像的差异，通过平移量和相对位置关系对第一图像进行处理，可以使第一图像与第二图像在切换倍率时，具有相同的尺度和视场角，即，可以理解为第一图像与第二图像相同。
在实际实现时，为了实现图像的平滑过渡，在接近但未到达切换倍率的倍率范围时，该倍率范围也可以成为warping区间；用户输入倍率进入该区间内，即可开始对第一图像进行处理，随着用户输入倍率的变化，处理第一图像的平移量和相对位置关系的具体参数也应当变化，例如，这些参数可以形成一个以倍率为自变量的线性函数，使得用户输入倍率在warping区间时，处理后的第一图像逐渐接近第二图像，当用户输入倍率到达切换倍率时，第一图像与第二图像的尺度和视场角均相同。此时，当显示图像从第一图像切换至第二图像时，图像不存在明显的内容跳变。当用户输入倍率到达切换倍率时，显示第二图像。
上述变焦控制方法，首先获取第一摄像头对应的第一图像和第二摄像头对应的第二图像；当用户输入倍率处于切换倍率区间时，根据第一图像和第二图像的匹配关系，确定第一图像与第二图像之间的平移量，以及第一摄像头和第二摄像头的相对位置关系；根据第一摄像头和第二摄像头的焦距，以及第一图像的尺度，确定切换倍率；根据平移量和相对位置关系处理第一图像，以使用户输入倍率到达切换倍率时，第一图像与第二图像具有相同的尺度和视场角。
该方式通过第一图像和第二图像之间的匹配关系，确定图像之间的平移量和摄像头之间的相对位置关系等参数，基于这些参数处理第一图像；同时，通过摄像头的焦距和图像的尺度，确定切换倍率；从而使用户输入倍率到达切换倍率时，处理后的第一图像和第二图像具有相同的尺度和视场角，实现显示图像的平滑切换，提高了变焦效果；另外，由于该方式无需标定参数，在模组老化或碰撞的情况下，依然能够保证平滑稳定的变焦效果。
实施例三：
本实施例重点描述第一图像和第二图像的获取方式。首先，获取第一摄像头和第二摄像头采集的初始图像；根据用户输入倍率，确定第一摄像头和第二摄像头的图像裁剪系数。
对于双摄模组而言，常见的摄像头包括视场角为120度的超广角镜头、80度的广角镜头、45度的长焦镜头和25度的潜望式镜头等；其中的超广角镜头和广角镜头的组合、以及长焦镜头和潜望式镜头的组合是常见的双摄模组的组合。上述用户输入倍率通常由设备的用户操作设备的相关控件或按钮输入。设备中通常设置有倍率转换库，用户输入倍率输入至该倍率转换库，即可输出上述第一摄像头和第二摄像头的图像裁剪系数。
作为一种实施方式，该倍率转换库可以将用户输入倍率确定为第一摄像头的图像裁剪系数；通过下 述算式，计算第二摄像头的图像裁剪系数：
tScale＝fw/ft*wScale；
其中，tScale为第二摄像头的图像裁剪系数；fw为第一摄像头的焦距；ft为第二摄像头的焦距；wScale为第一摄像头的图像裁剪系数。
这里需要说明的是，tScale和wScale均具有一定的范围；其中，当tScale小于1时，设置tScale等于1.0；并且，当wScale和tScale大于2.0x时，均设置为最大值2.0，该最大值取决于IFE模块的裁剪的最大倍率；如果超过该最大值，则容易导致图像模糊不清；当然，根据设备的处理能力的不同，上述最大值可以更小或更大。
另外，当用户输入倍率进入warping区间后，因为需要对第一图像进行旋转和平移处理，此时一般会将wScale变小，增加10度左右的视场角冗余，避免第一图像进行旋转和平移处理后，出现黑边现象。
如图3所示，当第一摄像头为广角镜头，第二摄像头为长焦镜头时，带箭头的实线代表第一摄像头的倍率范围，带箭头的虚线代表第二摄像头的倍率范围；黑点代表切换倍率的位置点；第二摄像头的倍率范围的最小值至切换倍率之间，为warping区间；第二摄像头的倍率范围的最小值至第一摄像头的倍率范围的最大值之间，为双摄区间。
上述双摄区间也可以理解为倍率切换区间，当所述用户输入倍率逐渐放大至切换倍率时，该切换倍率区间包括：第二摄像头的倍率范围的最小值至第一摄像头的倍率范围的最大值。
如图4所示，当第一摄像头为长焦镜头，第二摄像头为广角镜头时，带箭头的实线代表第一摄像头的倍率范围，带箭头的虚线代表第二摄像头的倍率范围；黑点代表切换倍率的位置点；第二摄像头的倍率范围的最大值至切换倍率之间，为warping区间；第二摄像头的倍率范围的最大值至第一摄像头的倍率范围的最小值之间，为双摄区间。
上述双摄区间也可以理解为倍率切换区间，当用户输入倍率逐渐缩小至切换倍率时，该切换倍率区间包括：第二摄像头的倍率范围的最大值至第一摄像头的倍率范围的最小值。
确定图像裁剪系数之后，根据第一摄像头的图像裁剪系数，对第一摄像头采集的初始图像进行裁剪，得到第一图像；根据第二摄像头的图像裁剪系数，对第一摄像头采集的初始图像进行裁剪，得到第二图像。裁剪操作可以通过前述IFE模型实现，具体的裁剪方式可以是中心裁剪，也可以使其他裁剪方式。
实施例四：
本实施例重点描述图像之间的平移量以及摄像头之间的相对位置关系的计算方式。首先，可以通过下述步骤30-步骤34确定第一图像与第二图像之间的平移量：
步骤30，从第一图像中确定指定区域；其中，该指定区域包括第一图像的焦点区域或中心区域；
具体的，第一图像的焦点区域可以理解为第一图像的对焦点所在的图像区域，该图像区域的尺度可以预先设置；用户可以在屏幕上点击，将点击位置确定为对焦点，从而确定焦点区域，此时的指定区域可以为焦点区域。如果用户没有确定对焦点，则可以将第一图像的中心区域确定为指定区域，中心区域 的尺度也可以预先设置。
步骤32，确定指定区域在第二图像中的匹配区域；
可以通过图像识别或特征点匹配的方式，从第二图像中获取与指定区域相匹配的匹配区域；第二图像中的匹配区域的图像内容通常与第一图像中的指定区域的图像内容相同。
步骤34，根据指定区域在第一图像中的位置，以及匹配区域在第二图像中的位置，确定平移量。
由于第一摄像头和第二摄像头的位置不同，相同图像内容在第一图像和在第二图像中的位置也就不同；指定区域在第一图像中的位置，以及匹配区域在第二图像中的位置，均可以通过坐标表示，通过坐标计算，即可得到上述平移量。
需要说明的是，当用户输入倍率进入图3所示的双摄区间后，可以设置每10帧图像计算一次平移量，计算平移量的频率也可以设置为其他值，通常计算平移量的频率较高，主要是考虑到每个场景的平移量都不同，较高频率的计算平移量，可以及时获取到当前场景的平移量。
作为一种实施方式，可以通过下述步骤40-步骤46确定第一摄像头和第二摄像头的相对位置关系：
步骤40，第一图像的尺度和第二图像的尺度压缩至预设尺度范围；可以采用resize功能对图像进行压缩，比如，把1000*750大小的图像压缩成500*375大小的图像；对图像进行压缩可以提高提取特征点的性能。
步骤42，调整压缩后的第一图像和第二图像的尺度，以使第一图像和第二图像的尺度相匹配。该步骤的操作也可以称为图像尺度对齐，在具体是现实，可以根据第一摄像头和第二摄像头的焦距，以及第一图像和第二图像的图像裁剪系数进行图像尺度对齐操作。一种具体的实现方式中，可以仅调整第一图像的尺度，第一图像的尺度s的计算方式如下：
Figure PCTCN2022091747-appb-000011
其中，tScale为第二摄像头的图像裁剪系数；fw为第一摄像头的焦距；ft为第二摄像头的焦距；wScale为第一摄像头的图像裁剪系数。
步骤44，从第一图像和第二图像中获取多对相互匹配的特征点；
可以先从第一图像中选择一定数量的特征点，然后再通过特征点匹配的方式，从第二图像中寻找每个特征点相匹配的特征点；为了使特征点的分布更加均匀，一种具体的实现方式中，按照预设的网格参数，将第一图像划分为多个网格区域；从每个网格区域中提取一个特征点，从第二图像中获取与每个特征点相匹配的特征点，得到多对相互匹配的特征点。例如可以通过网格等分的方式划分得到多个网格区域，在每个网格区域中，使用ORB(Oriented FAST and Rotated BRIEF的简称，其中的FAST和BRIEF均为算法名称)等特征提取算法，提取一个较为稳定的特征点，然后使用LK(Lucas–Kanade的简称)光流方法在第二图像中追踪每个特征点的匹配点，然后再采用RANSAC(Random Sample Consensus，随机抽样一致性)方法提纯匹配点。
步骤46，基于多对相互匹配的特征点，确定第一摄像头和第二摄像头的相对位置关系。
作为步骤46的一种实施方式，确定相对位置关系需要采用对极约束公式，该对极约束公式可以表示为：E＝t ∧R,F＝K TEK -1,
Figure PCTCN2022091747-appb-000012
其中，E为本质矩阵，F为基础矩阵，t为平移矩阵，表征第一摄像头和第二摄像头之间的平移关系；R为旋转矩阵，表征第一摄像头和第二摄像头之间的旋转关系；K为第一摄像头的内参数矩阵；由于第一图像和第二图像进行了图像尺度对齐操作，因而第一图像和第二图像的尺度一致，所以这里的第一摄像头的内参数的初始值可以设置为和第二摄像头的内参数一致。K T为K的转置矩阵，K -1为K的逆矩阵；x 1＝K -1p 1；p 1为第一图像中的一个特征点，p 2为第二图像中与p 1相匹配的特征点；x 2＝K -1p 2；
Figure PCTCN2022091747-appb-000013
为x 2的转置矩阵；
Figure PCTCN2022091747-appb-000014
为p 2的转置矩阵。
然后，将多对相互匹配的特征点输入至对极约束公式中，计算得到本质矩阵；基于该本质矩阵，确定第一摄像头与第二摄像头之间的相对位置关系，该相对位置关系具体可以为旋转关系。具体的，可以通过八对相互匹配的特征点通过上述对极约束公式计算得到本质矩阵E，然后采用SVD(Singular Value Decomposition，奇异值分解)和三角重建原理分解得到外参数，即上述旋转矩阵R和平移矩阵t。
本实施例中，主要用到了外参数中的旋转矩阵R，为了使R得值更加准确，这里还需要对R进行进一步地优化。假设校正后的第一图像和第二图像的尺度均为s，第二摄像头的焦距为ft，则，此时第一摄像头的焦距修正为f′ w＝f t/s。默认的主点均为图像中心点；焦距和主点均属于摄像头的内参数。
首先，设置误差方程
Figure PCTCN2022091747-appb-000015
将第一图像和第二图像行对齐作为约束条件，通过预设的优化算法，计算得到当误差方程CostF的值最小时，R wt的值；其中，R wt为优化后的旋转关系；i代表第i个特征点；n代表特征点总量；y ti代表第二图像中第i个特征点的y坐标值；
Figure PCTCN2022091747-appb-000016
代表第一图像中第i个特征点的匹配点与第二图像中第i个特征点对齐后的y坐标；K t代表第二摄像头的内参数；
Figure PCTCN2022091747-appb-000017
代表第一摄像头的内参数矩阵的逆矩阵；U w为第一图像。
上述
Figure PCTCN2022091747-appb-000018
也可以称为第一摄像头向第二摄像头的对齐公式；上述优化算法可以为Lederberg-Marquardt优化算法，从而优化出准确的参数R，将R转换成角度(θ x,θ y,θ z)输出。
上述方式中，通过图像匹配的方式对图像进行校正，得到相互匹配的图像，可以在图像切换前后，不出现明显的图像旋转和平移的问题，使得图像可以平滑过渡。
实施例五：
本实施例重点描述切换倍率的计算方式，具体可以通过下述步骤50-步骤52确定切换倍率：
步骤50，根据第一摄像头和第二摄像头的焦距，确定初始倍率；
假设在相同的分辨率下，第一摄像头的焦距为fw，第二摄像头的焦距为ft，则可以将第二摄像头的焦距与第一摄像头的焦距的比值ft/fw，确定为初始倍率。
步骤52，基于第一图像的尺度，调整初始倍率，得到切换倍率。
切换倍率可以理解为，在该切换倍率下，第一图像和第二图像的尺度和视场角均一致，当用户输入倍率为切换倍率时，将显示图像由第一图像切换至第二图像。具体可以通过下述算式，计算得到切换倍率：
Figure PCTCN2022091747-appb-000019
其中，switchLevel为切换倍率；ft为第二摄像头的焦距；fw为第一摄像头的焦距；s为第一图像的尺度值，经尺度匹配之后，第一图像和第二图像的尺度值相同；
Figure PCTCN2022091747-appb-000020
tScale为第二摄像头的图像裁剪系数；fw为第一摄像头的焦距；ft为第二摄像头的焦距；wScale为第一摄像头的图像裁剪系数。
通过上述方式可以计算出准确的切换倍率，在该切换倍率下切换图像，可以使切换前后的图像的视场角一致，使得切换图像可以平滑过渡。
实施例六：
本实施例重点描述根据平移量和相对位置关系处理第一图像的具体实现方式。这里的相对位置关系主要包括：第一摄像头与第二摄像头之间的旋转关系；本实施例主要采用先旋转后平移的方式，处理第一图像，共包括下述步骤60和步骤62：
步骤60，根据旋转关系对第一图像进行旋转处理，得到中间图像；
这里的旋转关系表征了第一摄像头相对于第二摄像头的旋转关系，基于该旋转关系处理第一图像，可以将第一图像的视场角校正至与第二图像相匹配的状态；具体的，可以通过下述算式，计算得到中间图像：
Figure PCTCN2022091747-appb-000021
其中，I′ W为中间图像；
Figure PCTCN2022091747-appb-000022
(u 0，v 0)为第一图像的中心点坐标；
Figure PCTCN2022091747-appb-000023
switchLevel为切换倍率；userLevel为用户输入倍率；warpping长度为切换倍率与第二摄像头的最低倍率的差值；θ z为旋转关系中z纬度上的旋转角度；
Figure PCTCN2022091747-appb-000024
为H ot的逆矩阵；I W为第一图像。
其中，令
Figure PCTCN2022091747-appb-000025
上述算式也可以表示成：I′ W＝H rotate*I W；其中的H ot的逆矩阵可以将图像中心移至坐标系的原点，H ot矩阵可以将旋转后的图像移动回图像中心；H r为图像二维旋转公式。
步骤62，根据平移量对中间图像进行平移处理，得到处理后的第一图像。
这里的平移量可以理解为平移总量，该平移总量需要均匀地分布至warping区间，即在该warping区间内，用户输入倍率与平移量之间呈线性变化，当用户输入倍率到达warping区域中的倍率最大值时，即切换倍率，平移量达到最大，即上述平移总量。
基于上述，首先要根据平移量和用户输入倍率，确定平移矩阵；基于该平移矩阵对中间图像进行平 移处理，得到处理后的第一图像。其中，该平移矩阵
Figure PCTCN2022091747-appb-000026
其中，
Figure PCTCN2022091747-appb-000027
Figure PCTCN2022091747-appb-000028
switchLevel为切换倍率；userLevel为用户输入倍率；warpping长度为切换倍率与第二摄像头的最低倍率的差值；T为平移量；t x为t的x维度的数值；t y为t的y维度的数值。
上述t的公式中，
Figure PCTCN2022091747-appb-000029
中的userLevel此时等于wScale，即第一摄像头的图像裁剪系数；平移量T是在切换倍率switchLevel下的平移量，通过
Figure PCTCN2022091747-appb-000030
可以将平移量转换到wScale下。
综上，对第一图像进行图像旋转和平移的总的表达式为：H algo＝H shift*H rotate。第一图像处理完成后，为了便于后续的IPE模块对图像进行进一步处理，需要先确定处理后的第一图像的裁剪区域Rect＝[(w-w/c]/2,(h-h/c]/2,w/c,h/c]；其中，w为第一图像的宽度；
Figure PCTCN2022091747-appb-000031
userLevel为用户输入倍率；wScale为第一摄像头的图像裁剪系数；h为第一图像的高度。然后根据裁剪区域对第一图像进行裁剪处理；具体的，可以通过图像处理引擎IPE模块对第一图像进行裁剪处理，并显示裁剪处理后的所述第一图像。IPE模块除了对第一图像进行裁剪处理，通常还对第一图像进行降噪、颜色处理、细节增强等图像处理过程，从而提高第一图像的显示效果。
上述剪裁区域的计算公式，可以应用于图3或图4所示的双摄区间内的用户输入倍率，在双摄区间以外的倍率下，即单摄区间中，由于第一图像的wScale＝userLevel，此时无需对第一图像进行处理，第一图像的裁剪区域为第一图像的全图，即Rect＝[0,0,w,h]。
为了便于理解，图5示出了以双摄模组为例的变焦控制方法流程；图5中的图像数据流包括前述第一摄像头和第二摄像头采集的初始图像，IFE模块基于图像裁剪系数wScale和tScale对初始图像进行尺度调整，得到前述第一图像和第二图像；当用户输入倍率属于双摄区域时，对第一图像和第二图像进行尺度校正后，通过前述实施例的方式计算旋转角度、平移量和切换倍率；然后根据旋转角度和平移量处理图像，根据切换倍率、平移量和userLevel计算裁剪区域Rect。如果用户输入倍率不属于双摄区域，则根据用户输入倍率userLevel本身计算裁剪区域Rect；计算得到的裁剪区域输入值IPE模块，用于对第一图像进行进一步处理。
上述实施例中，提出了一种无需标定参数的变焦控制方案。基于图像匹配的方式计算出准确的旋转角度和切换倍率，在该切换倍率时，两个摄像头对应的图像的视场角，实现图像的平滑过渡，与基于标定参数的变焦控制方案具有同样的用户体验。
该方案的基本思路为：首先获取双摄模组的基本信息，包括焦距、像元尺寸、图像分辨率和模组组装位置参数(Tx，Ty，Tz)。然后根据模组的基本信息计算出初始切换倍率，该倍率由于是模组参数计算出来的，是存在一定误差的。然后采用大视场角的图像向小视场角的图像旋转平移的方案，在双摄共存区间，利用多线程方式计算当前模组准确切换倍率，图像旋转对齐角度和平移量。最后可以根据上述计 算数据实现图像平滑切换方案。上述变焦控制方式无需模组生产线标定参数，计算方式简单可靠，大大降低了模组的生产成本，且变焦控制的性能较好，能够适用于各种中高端手机等终端设备，以及双摄，三摄乃至多摄模组的光学平滑切换方案，推动了多摄变焦方案的推广。
需要说明的是，上述实施例以双摄模组为例描述变焦控制方法，由于多摄模组也是由双摄组合而成，因此多摄模组也可以应用前述实施例中的变焦控制方法。
实施例七：
对应于上述方法实施例，参见图6所示的一种变焦控制装置的结构示意图，该装置设置于配置有第一摄像头和第二摄像头的设备；该装置可以包括：
图像获取模块60，可以配置成用于获取第一摄像头对应的第一图像和第二摄像头对应的第二图像；
参数确定模块62，可以配置成用于当用户输入倍率处于切换倍率区间时，根据第一图像和第二图像的匹配关系，确定第一图像与第二图像之间的平移量，以及第一摄像头和第二摄像头的相对位置关系；
倍率确定模块64，可以配置成用于根据第一摄像头和第二摄像头的焦距，以及第一图像的尺度，确定切换倍率；
图像处理模块66，可以配置成用于根据平移量和相对位置关系处理第一图像，以使用户输入倍率到达切换倍率时，第一图像与第二图像具有相同的尺度和视场角。
上述变焦控制装置，首先获取第一摄像头对应的第一图像和第二摄像头对应的第二图像；当用户输入倍率处于切换倍率区间时，根据第一图像和第二图像的匹配关系，确定第一图像与第二图像之间的平移量，以及第一摄像头和第二摄像头的相对位置关系；根据第一摄像头和第二摄像头的焦距，以及第一图像的尺度，确定切换倍率；根据平移量和相对位置关系处理第一图像，以使用户输入倍率到达切换倍率时，第一图像与第二图像具有相同的尺度和视场角。
该方式通过第一图像和第二图像之间的匹配关系，确定图像之间的平移量和摄像头之间的相对位置关系等参数，基于这些参数处理第一图像；同时，通过摄像头的焦距和图像的尺度，确定切换倍率；从而使用户输入倍率到达切换倍率时，处理后的第一图像和第二图像具有相同的尺度和视场角，实现显示图像的平滑切换，提高了变焦效果；另外，由于该方式无需标定参数，在模组老化或碰撞的情况下，依然能够保证平滑稳定的变焦效果。
上述图像获取模块还可以配置成用于：获取第一摄像头和第二摄像头采集的初始图像；根据用户输入倍率，确定第一摄像头和第二摄像头的图像裁剪系数；根据第一摄像头的图像裁剪系数，对第一摄像头采集的初始图像进行裁剪，得到第一图像；根据第二摄像头的图像裁剪系数，对第一摄像头采集的初始图像进行裁剪，得到第二图像。
上述图像获取模块还可以配置成用于：将用户输入倍率确定为第一摄像头的图像裁剪系数；通过下述算式，计算第二摄像头的图像裁剪系数：
tScale＝fw/ft*wScale；
其中，tScale为第二摄像头的图像裁剪系数；fw为第一摄像头的焦距；ft为第二摄像头的焦距；wScale为第一摄像头的图像裁剪系数。
上述参数确定模块，还可以配置成用于：从第一图像中确定指定区域；其中，指定区域包括第一图像的焦点区域或中心区域；确定指定区域在第二图像中的匹配区域；根据指定区域在第一图像中的位置，以及匹配区域在第二图像中的位置，确定平移量。
上述参数确定模块还可以配置成用于：从第一图像和第二图像中获取多对相互匹配的特征点；基于多对相互匹配的特征点，确定第一摄像头和第二摄像头的相对位置关系。
上述装置还包括图像预处理模块，可以配置成用于将第一图像的尺度和第二图像的尺度压缩至预设尺度范围；调整压缩后的第一图像和第二图像的尺度，以使第一图像和第二图像的尺度相匹配。
上述参数确定模块还可以配置成用于：按照预设的网格参数，将第一图像划分为多个网格区域；从每个网格区域中提取一个特征点，从第二图像中获取与每个特征点相匹配的特征点，得到多对相互匹配的特征点。
上述参数确定模块还可以配置成用于：将多对相互匹配的特征点输入至对极约束公式中，计算得到本质矩阵；基于本质矩阵，确定第一摄像头与第二摄像头之间的相对位置关系；该相对位置关系包括旋转关系。
上述装置还包括参数优化模块，可以配置成用于：设置误差方程
Figure PCTCN2022091747-appb-000032
Figure PCTCN2022091747-appb-000033
将第一图像和第二图像行对齐作为约束条件，通过预设的优化算法，计算得到当误差方程CostF的值最小时，R wt的值；其中，R wt为优化后的旋转关系；i代表第i个特征点；n代表特征点总量；y ti代表第二图像中第i个特征点的y坐标值；
Figure PCTCN2022091747-appb-000034
代表第一图像中第i个特征点的匹配点与第二图像中第i个特征点对齐后的y坐标；K t代表第二摄像头的内参数；
Figure PCTCN2022091747-appb-000035
代表第一摄像头的内参数矩阵的逆矩阵；U w为第一图像。
上述倍率确定模块，还可以配置成用于：根据第一摄像头和第二摄像头的焦距，确定初始倍率；基于第一图像的尺度，调整初始倍率，得到切换倍率。
上述倍率确定模块，还可以配置成用于：将第二摄像头的焦距与第一摄像头的焦距的比值，确定为初始倍率。
上述倍率确定模块，还可以配置成用于：通过下述算式，计算得到切换倍率：
Figure PCTCN2022091747-appb-000036
其中，switchLevel为切换倍率；ft为第二摄像头的焦距；fw为第一摄像头的焦距；s为第一图像的尺度值；
Figure PCTCN2022091747-appb-000037
tScale为第二摄像头的图像裁剪系数；fw为第一摄像头的焦距；ft为第二摄像头的焦距；wScale为第一摄像头的图像裁剪系数。
上述相对位置关系包括：第一摄像头与第二摄像头之间的旋转关系；上述图像处理模块，还包括：根据旋转关系对第一图像进行旋转处理，得到中间图像；根据平移量对中间图像进行平移处理，得到处 理后的第一图像。
上述图像处理模块，还可以配置成用于：通过下述算式，计算得到中间图像：
Figure PCTCN2022091747-appb-000038
其中，I′ W为中间图像；
Figure PCTCN2022091747-appb-000039
(u 0，v 0)为第一图像的中心点坐标；
Figure PCTCN2022091747-appb-000040
switchLevel为切换倍率；userLevel为用户输入倍率；warpping长度为切换倍率与第二摄像头的最低倍率的差值；θ z为旋转关系中z纬度上的旋转角度；
Figure PCTCN2022091747-appb-000041
为H ot的逆矩阵；I W为第一图像。
上述图像处理模块，还可以配置成用于：根据平移量和用户输入倍率，确定平移矩阵；基于平移矩阵对中间图像进行平移处理，得到处理后的第一图像。
上述裁剪区域确定模块，可以配置成用于：确定处理后的第一图像的裁剪区域Rect＝[(w-w/c]/2,(h-h/c]/2,w/c,h/c]；其中，w为第一图像的宽度；
Figure PCTCN2022091747-appb-000042
userLevel为用户输入倍率；wScale为第一摄像头的图像裁剪系数；h为第一图像的高度；根据裁剪区域对第一图像进行裁剪处理。
上述当用户输入倍率逐渐放大至切换倍率时，切换倍率区间可以包括：第二摄像头的倍率范围的最小值至第一摄像头的倍率范围的最大值；当用户输入倍率逐渐缩小至切换倍率时，切换倍率区间可以包括：第二摄像头的倍率范围的最大值至第一摄像头的倍率范围的最小值。
本实施例还提供一种电子设备，该电子设备可以包括：处理设备和存储装置；存储装置上存储有计算机程序，计算机程序在被处理设备运行时执行如上述变焦控制方法。
本实施例还提供一种计算机可读存储介质，计算机可读存储介质上存储有计算机程序，计算机程序被处理设备运行时执行如上述变焦控制方法的步骤。
本公开实施例所提供的变焦控制方法、装置、电子设备和计算机可读存储介质的计算机程序产品，包括存储了程序代码的计算机可读存储介质，所述程序代码包括的指令可用于执行前面方法实施例中所述的方法，具体实现可参见方法实施例，在此不再赘述。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的系统和装置的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
另外，在本公开实施例的描述中，除非另有明确的规定和限定，术语“安装”、“相连”、“连接”应做广义理解，例如，可以是固定连接，也可以是可拆卸连接，或一体地连接；可以是机械连接，也可以是电连接；可以是直接相连，也可以通过中间媒介间接相连，可以是两个元件内部的连通。对于本领域技术人员而言，可以具体情况理解上述术语在本公开中的具体含义。
所述功能如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本公开的技术方案本质上或者说对相关技术做出贡献的部分或者 该技术方案的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本公开各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、磁碟或者光盘等各种可以存储程序代码的介质。
在本公开的描述中，需要说明的是，术语“中心”、“上”、“下”、“左”、“右”、“竖直”、“水平”、“内”、“外”等指示的方位或位置关系为基于附图所示的方位或位置关系，仅是为了便于描述本公开和简化描述，而不是指示或暗示所指的装置或元件必须具有特定的方位、以特定的方位构造和操作，因此不能理解为对本公开的限制。此外，术语“第一”、“第二”、“第三”仅用于描述目的，而不能理解为指示或暗示相对重要性。
最后应说明的是：以上实施例，仅为本公开的具体实施方式，用以说明本公开的技术方案，而非对其限制，本公开的保护范围并不局限于此，尽管参照前述实施例对本公开进行了详细的说明，本领域技术人员应当理解：任何熟悉本技术领域的技术人员在本公开揭露的技术范围内，其依然可以对前述实施例所记载的技术方案进行修改或可轻易想到变化，或者对其中部分技术特征进行等同替换；而这些修改、变化或者替换，并不使相应技术方案的本质脱离本公开实施例技术方案的精神和范围，都应涵盖在本公开的保护范围之内。因此，本公开的保护范围应以权利要求的保护范围为准。
工业实用性
本申请提供了本公开提供了一种变焦控制方法、装置、电子设备和计算机可读存储介质；其中，该方法包括：当用户输入倍率处于切换倍率区间时，通过第一图像和第二图像之间的匹配关系，确定图像之间的平移量和摄像头之间的相对位置关系等参数，基于这些参数处理第一图像；同时，通过摄像头的焦距和图像的尺度，确定切换倍率；从而使用户输入倍率到达切换倍率时，处理后的第一图像和第二图像具有相同的尺度和视场角，实现显示图像的平滑切换，提高了变焦效果；另外，由于该方式无需标定参数，在模组老化或碰撞的情况下，依然能够保证平滑稳定的变焦效果。
此外，可以理解的是，本申请的变焦控制方法、装置、电子设备和计算机可读存储介质是可以重现的，并且可以用在多种工业应用中。例如，本申请的变焦控制方法、装置、电子设备和计算机可读存储介质可以用于图像处理技术领域。

Claims (20)

  1. 一种变焦控制方法，其特征在于，所述方法应用于配置有第一摄像头和第二摄像头的设备；所述方法包括：
    获取所述第一摄像头对应的第一图像和所述第二摄像头对应的第二图像；
    当用户输入倍率处于切换倍率区间时，根据所述第一图像和所述第二图像的匹配关系，确定所述第一图像与所述第二图像之间的平移量，以及所述第一摄像头和所述第二摄像头的相对位置关系；
    根据所述第一摄像头和所述第二摄像头的焦距，以及所述第一图像的尺度，确定切换倍率；
    根据所述平移量和所述相对位置关系处理所述第一图像，以使所述用户输入倍率到达所述切换倍率时，所述第一图像与所述第二图像具有相同的尺度和视场角。
  2. 根据权利要求1所述的方法，其特征在于，获取所述第一摄像头对应的第一图像和所述第二摄像头对应的第二图像的步骤，包括：
    获取所述第一摄像头和所述第二摄像头采集的初始图像；根据所述用户输入倍率，确定所述第一摄像头和所述第二摄像头的图像裁剪系数；
    根据所述第一摄像头的图像裁剪系数，对所述第一摄像头采集的初始图像进行裁剪，得到所述第一图像；根据所述第二摄像头的图像裁剪系数，对所述第一摄像头采集的初始图像进行裁剪，得到所述第二图像。
  3. 根据权利要求2所述的方法，其特征在于，根据所述用户输入倍率，确定所述第一摄像头和所述第二摄像头的图像裁剪系数的步骤，包括：
    将所述用户输入倍率确定为所述第一摄像头的图像裁剪系数；
    通过下述算式，计算所述第二摄像头的图像裁剪系数：
    tScale＝fw/ft*wScale；
    其中，tScale为所述第二摄像头的图像裁剪系数；fw为所述第一摄像头的焦距；ft为所述第二摄像头的焦距；wScale为所述第一摄像头的图像裁剪系数。
  4. 根据权利要求1至3中任一项所述的方法，其特征在于，根据所述第一图像和所述第二图像的匹配关系，确定所述第一图像与所述第二图像之间的平移量的步骤，包括：
    从所述第一图像中确定指定区域；其中，所述指定区域包括所述第一图像的焦点区域或中心区域；
    确定所述指定区域在所述第二图像中的匹配区域；
    根据所述指定区域在所述第一图像中的位置，以及所述匹配区域在所述第二图像中的位置，确定所述平移量。
  5. 根据权利要求1至4中任一项所述的方法，其特征在于，根据所述第一图像和所述第二图像的匹配关系，确定所述第一摄像头和所述第二摄像头的相对位置关系的步骤，包括：
    从所述第一图像和所述第二图像中获取多对相互匹配的特征点；
    基于所述多对相互匹配的特征点，确定所述第一摄像头和所述第二摄像头的相对位置关系。
  6. 根据权利要求5所述的方法，其特征在于，从所述第一图像和所述第二图像中获取多对相互匹配的特征点的步骤之前，所述方法还包括：
    将所述第一图像的尺度和所述第二图像的尺度压缩至预设尺度范围；
    调整压缩后的所述第一图像和所述第二图像的尺度，以使所述第一图像和所述第二图像的尺度相匹配。
  7. 根据权利要求5或6所述的方法，其特征在于，从所述第一图像和所述第二图像中获取多对相互匹配的特征点的步骤，包括：
    按照预设的网格参数，将所述第一图像划分为多个网格区域；
    从每个所述网格区域中提取一个特征点，从所述第二图像中获取与每个所述特征点相匹配的特征点，得到多对相互匹配的特征点。
  8. 根据权利要求5至7中任一项所述的方法，其特征在于，基于所述多对相互匹配的特征点，确定所述第一摄像头和所述第二摄像头的相对位置关系的步骤，包括：
    将所述多对相互匹配的特征点输入至对极约束公式中，计算得到本质矩阵；基于所述本质矩阵，确定所述第一摄像头与所述第二摄像头之间的相对位置关系；所述相对位置关系包括旋转关系。
  9. 根据权利要求8所述的方法，其特征在于，基于所述本质矩阵，确定所述第一摄像头与所述第二摄像头之间的相对位置关系的步骤之后，所述方法还包括：
    设置误差方程
    Figure PCTCN2022091747-appb-100001
    将所述第一图像和所述第二图像行对齐作为约束条件，通过预设的优化算法，计算得到当所述误差方程CostF的值最小时，所述R wt的值；
    其中，所述R wt为优化后的所述旋转关系；i代表第i个特征点；n代表特征点总量；y ti代表所述第二图像中第i个特征点的y坐标值；
    Figure PCTCN2022091747-appb-100002
    代表所述第一图像中第i个特征点的匹配点与所述第二图像中第i个特征点对齐后的y坐标；K t代表所述第二摄像头的内参数；
    Figure PCTCN2022091747-appb-100003
    代表所述第一摄像头的内参数矩阵的逆矩阵；U w为所述第一图像。
  10. 根据权利要求1至9中任一项所述的方法，其特征在于，根据所述第一摄像头和所述第二摄像头的焦距，以及所述第一图像的尺度，确定切换倍率的步骤，包括：
    根据所述第一摄像头和所述第二摄像头的焦距，确定初始倍率；
    基于所述第一图像的尺度，调整所述初始倍率，得到所述切换倍率。
  11. 根据权利要求10所述的方法，其特征在于，根据所述第一摄像头和所述第二摄像头的焦距，确定初始倍率的步骤，包括：将所述第二摄像头的焦距与所述第一摄像头的焦距的比值，确定为所述初始倍率。
  12. 根据权利要求10或11所述的方法，其特征在于，基于所述第一图像的尺度，调整所述初始倍率，得到所述切换倍率的步骤，包括：
    通过下述算式，计算得到所述切换倍率：
    Figure PCTCN2022091747-appb-100004
    其中，switchLevel为所述切换倍率；ft为所述第二摄像头的焦距；fw为所述第一摄像头的焦距；s为所述第一图像的尺度值；
    Figure PCTCN2022091747-appb-100005
    tScale为所述第二摄像头的图像裁剪系数；fw为所述第一摄像头的焦距；ft为所述第二摄像头的焦距；wScale为所述第一摄像头的图像裁剪系数。
  13. 根据权利要求1至12中任一项所述的方法，其特征在于，所述相对位置关系包括：所述第一摄像头与所述第二摄像头之间的旋转关系；所述根据所述平移量和所述相对位置关系处理所述第一图像的步骤，包括：
    根据所述旋转关系对所述第一图像进行旋转处理，得到中间图像；
    根据所述平移量对所述中间图像进行平移处理，得到处理后的所述第一图像。
  14. 根据权利要求13所述的方法，其特征在于，根据所述旋转关系对所述第一图像进行旋转处理，得到中间图像的步骤，包括：
    通过下述算式，计算得到所述中间图像：
    Figure PCTCN2022091747-appb-100006
    其中，I′ W为所述中间图像；
    Figure PCTCN2022091747-appb-100007
    (u 0，v 0)为所述第一图像的中心点坐标；
    Figure PCTCN2022091747-appb-100008
    switchLevel为所述切换倍率；userLevel为所述用户输入倍率；warpping长度为所述切换倍率与所述第二摄像头的最低倍率的差值；θ z为所述旋转关系中z纬度上的旋转角度；
    Figure PCTCN2022091747-appb-100009
    为所述H ot的逆矩阵；I W为所述第一图像。
  15. 根据权利要求13或14所述的方法，其特征在于，根据所述平移量对所述中间图像进行平移处理，得到处理后的所述第一图像的步骤，包括：
    根据所述平移量和所述用户输入倍率，确定平移矩阵；
    基于所述平移矩阵对所述中间图像进行平移处理，得到处理后的所述第一图像。
  16. 根据权利要求1至15中任一项所述的方法，其特征在于，根据所述平移量和所述相对位置关系处理所述第一图像的步骤之后，所述方法还包括：
    确定处理后的所述第一图像的裁剪区域Rect＝[(w-w/c]/2,(h-h/c]/2,w/c,h/c]；其中，w为所述第一图像的宽度；
    Figure PCTCN2022091747-appb-100010
    userLevel为所述用户输入倍率；wScale为所述第一摄像头的图像裁剪系数；h为所述第一图像的高度；
    根据所述裁剪区域对所述第一图像进行裁剪处理。
  17. 根据权利要求1至16中任一项所述的方法，其特征在于，
    当所述用户输入倍率逐渐放大至所述切换倍率时，所述切换倍率区间包括：所述第二摄像头的倍率范围的最小值至所述第一摄像头的倍率范围的最大值；
    当所述用户输入倍率逐渐缩小至所述切换倍率时，所述切换倍率区间包括：所述第二摄像头的倍率范围的最大值至所述第一摄像头的倍率范围的最小值。
  18. 一种变焦控制装置，其特征在于，所述装置设置于配置有第一摄像头和第二摄像头的设备；所述装置包括：
    图像获取模块，配置成用于获取所述第一摄像头对应的第一图像和所述第二摄像头对应的第二图像；
    参数确定模块，配置成用于当用户输入倍率处于切换倍率区间时，根据所述第一图像和所述第二图像的匹配关系，确定所述第一图像与所述第二图像之间的平移量，以及所述第一摄像头和所述第二摄像头的相对位置关系；
    倍率确定模块，配置成用于根据所述第一摄像头和所述第二摄像头的焦距，以及所述第一图像的尺度，确定切换倍率；
    图像处理模块，配置成用于根据所述平移量和所述相对位置关系处理所述第一图像，以使所述用户输入倍率到达所述切换倍率时，所述第一图像与所述第二图像具有相同的尺度和视场角。
  19. 一种电子设备，其特征在于，所述电子设备包括：处理设备和存储装置；
    所述存储装置上存储有计算机程序，所述计算机程序在被所述处理设备运行时执行根据权利要求1至17中任一项所述的变焦控制方法。
  20. 一种计算机可读存储介质，所述计算机可读存储介质上存储有计算机程序，其特征在于，所述计算机程序被处理设备运行时执行根据权利要求1至17中任一项所述的变焦控制方法的步骤。
PCT/CN2022/091747 2021-06-18 2022-05-09 变焦控制方法、装置、电子设备和计算机可读存储介质 WO2022262474A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110682766.2A CN113630549B (zh) 2021-06-18 2021-06-18 变焦控制方法、装置、电子设备和计算机可读存储介质
CN202110682766.2 2021-06-18

Publications (1)

Publication Number Publication Date
WO2022262474A1 true WO2022262474A1 (zh) 2022-12-22

Family

ID=78378166

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/091747 WO2022262474A1 (zh) 2021-06-18 2022-05-09 变焦控制方法、装置、电子设备和计算机可读存储介质

Country Status (2)

Country Link
CN (1) CN113630549B (zh)
WO (1) WO2022262474A1 (zh)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113630549B (zh) * 2021-06-18 2023-07-14 北京旷视科技有限公司 变焦控制方法、装置、电子设备和计算机可读存储介质
CN113793259B (zh) * 2021-11-15 2022-02-15 深圳思谋信息科技有限公司 图像变焦方法、计算机设备和存储介质
CN113963072B (zh) * 2021-12-22 2022-03-25 深圳思谋信息科技有限公司 双目摄像头标定方法、装置、计算机设备和存储介质

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080055429A1 (en) * 2006-09-04 2008-03-06 Casio Computer Co., Ltd. Imaging apparatus and method for displaying zoom information
US20110096204A1 (en) * 2009-10-22 2011-04-28 Canon Kabushiki Kaisha Image pickup apparatus
CN111641775A (zh) * 2020-04-14 2020-09-08 北京迈格威科技有限公司 多摄变焦控制方法、装置及电子系统
CN111654631A (zh) * 2020-06-19 2020-09-11 厦门紫光展锐科技有限公司 变焦控制方法、系统、设备及介质
CN111917941A (zh) * 2019-05-08 2020-11-10 杭州海康威视数字技术股份有限公司 摄像机画面处理方法及摄像机
CN111935398A (zh) * 2020-07-07 2020-11-13 北京迈格威科技有限公司 图像的处理方法、装置、电子设备和计算机可读介质
CN111935397A (zh) * 2020-07-07 2020-11-13 北京迈格威科技有限公司 图像的处理方法、装置、电子设备和计算机可读介质
CN113630549A (zh) * 2021-06-18 2021-11-09 北京旷视科技有限公司 变焦控制方法、装置、电子设备和计算机可读存储介质

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109544615B (zh) * 2018-11-23 2021-08-24 深圳市腾讯信息技术有限公司 基于图像的重定位方法、装置、终端及存储介质
CN111432117B (zh) * 2020-03-23 2021-08-10 北京迈格威科技有限公司 图像矫正方法、装置和电子系统

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080055429A1 (en) * 2006-09-04 2008-03-06 Casio Computer Co., Ltd. Imaging apparatus and method for displaying zoom information
US20110096204A1 (en) * 2009-10-22 2011-04-28 Canon Kabushiki Kaisha Image pickup apparatus
CN111917941A (zh) * 2019-05-08 2020-11-10 杭州海康威视数字技术股份有限公司 摄像机画面处理方法及摄像机
CN111641775A (zh) * 2020-04-14 2020-09-08 北京迈格威科技有限公司 多摄变焦控制方法、装置及电子系统
CN111654631A (zh) * 2020-06-19 2020-09-11 厦门紫光展锐科技有限公司 变焦控制方法、系统、设备及介质
CN111935398A (zh) * 2020-07-07 2020-11-13 北京迈格威科技有限公司 图像的处理方法、装置、电子设备和计算机可读介质
CN111935397A (zh) * 2020-07-07 2020-11-13 北京迈格威科技有限公司 图像的处理方法、装置、电子设备和计算机可读介质
CN113630549A (zh) * 2021-06-18 2021-11-09 北京旷视科技有限公司 变焦控制方法、装置、电子设备和计算机可读存储介质

Also Published As

Publication number Publication date
CN113630549A (zh) 2021-11-09
CN113630549B (zh) 2023-07-14

Similar Documents

Publication Publication Date Title
WO2022262474A1 (zh) 变焦控制方法、装置、电子设备和计算机可读存储介质
WO2021208371A1 (zh) 多摄变焦控制方法、装置、电子系统及存储介质
EP2926552B1 (en) Window blanking for pan/tilt/zoom camera
WO2022156640A1 (zh) 一种图像的视线矫正方法、装置、电子设备、计算机可读存储介质及计算机程序产品
CN116018616A (zh) 保持帧中的目标对象的固定大小
US9813693B1 (en) Accounting for perspective effects in images
WO2020134238A1 (zh) 活体检测方法、装置以及存储介质
WO2022156626A1 (zh) 一种图像的视线矫正方法、装置、电子设备、计算机可读存储介质及计算机程序产品
WO2023024697A1 (zh) 图像拼接方法和电子设备
CN111507333B (zh) 一种图像矫正方法、装置、电子设备和存储介质
CN113096185B (zh) 视觉定位方法、视觉定位装置、存储介质与电子设备
WO2021086702A1 (en) Camera having vertically biased field of view
CN111723707B (zh) 一种基于视觉显著性的注视点估计方法及装置
WO2021189804A1 (zh) 图像矫正方法、装置和电子系统
CN112868224A (zh) 捕获和编辑动态深度图像的技术
CN110944201A (zh) 一种视频去重压缩的方法、装置、服务器及存储介质
Luo et al. Cloud Chaser: real time deep learning computer vision on low computing power devices
WO2022148248A1 (zh) 图像处理模型的训练方法、图像处理方法、装置、电子设备及计算机程序产品
CN114677422A (zh) 深度信息生成方法、图像虚化方法和视频虚化方法
WO2024021504A1 (zh) 人脸识别模型训练方法、识别方法、装置、设备及介质
CN117058343A (zh) 一种基于nerf的vr看展方法、系统、电子设备和存储介质
CN116912467A (zh) 图像拼接方法、装置、设备及存储介质
CN112565730B (zh) 路侧感知方法、装置、电子设备、存储介质及路侧设备
CN116137025A (zh) 视频图像矫正方法及装置、计算机可读介质和电子设备
Zhou et al. Improved YOLOv7 models based on modulated deformable convolution and swin transformer for object detection in fisheye images

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22823957

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE