WO2020102976A1 - 云台调整方法、云台调整设备、移动平台及介质 - Google Patents

云台调整方法、云台调整设备、移动平台及介质

Info

Publication number
WO2020102976A1
WO2020102976A1 PCT/CN2018/116439 CN2018116439W WO2020102976A1 WO 2020102976 A1 WO2020102976 A1 WO 2020102976A1 CN 2018116439 W CN2018116439 W CN 2018116439W WO 2020102976 A1 WO2020102976 A1 WO 2020102976A1
Authority
WO
WIPO (PCT)
Prior art keywords
image frame
current
image
shooting
trajectory
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2018/116439
Other languages
English (en)
French (fr)
Inventor
周游
刘洁
叶长春
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
SZ DJI Technology Co Ltd
Original Assignee
SZ DJI Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by SZ DJI Technology Co Ltd filed Critical SZ DJI Technology Co Ltd
Priority to CN201880038409.9A priority Critical patent/CN110800282B/zh
Priority to CN202110768639.4A priority patent/CN113473025A/zh
Priority to PCT/CN2018/116439 priority patent/WO2020102976A1/zh
Publication of WO2020102976A1 publication Critical patent/WO2020102976A1/zh
Priority to US17/302,922 priority patent/US11892812B1/en
Anticipated expiration legal-status Critical
Priority to US18/544,120 priority patent/US20240152108A1/en
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B19/00—Program-control systems
    • G05B19/02—Program-control systems electric
    • G05B19/18—Numerical control [NC], i.e. automatically operating machines, in particular machine tools, e.g. in a manufacturing environment, so as to execute positioning, movement or co-ordinated operations by means of program data in numerical form
    • G05B19/19—Numerical control [NC], i.e. automatically operating machines, in particular machine tools, e.g. in a manufacturing environment, so as to execute positioning, movement or co-ordinated operations by means of program data in numerical form characterised by positioning or contouring control systems, e.g. to control position from one programmed point to another or to control movement along a programmed continuous path
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/62—Control of parameters via user interfaces
    • F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
    • F16—ENGINEERING ELEMENTS AND UNITS; GENERAL MEASURES FOR PRODUCING AND MAINTAINING EFFECTIVE FUNCTIONING OF MACHINES OR INSTALLATIONS; THERMAL INSULATION IN GENERAL
    • F16M—FRAMES, CASINGS OR BEDS OF ENGINES, MACHINES OR APPARATUS, NOT SPECIFIC TO ENGINES, MACHINES OR APPARATUS PROVIDED FOR ELSEWHERE; STANDS; SUPPORTS
    • F16M11/00—Stands or trestles as supports for apparatus or articles placed thereon ; Stands for scientific apparatus such as gravitational force meters
    • F16M11/02—Heads
    • F16M11/04—Means for attachment of apparatus; Means allowing adjustment of the apparatus relatively to the stand
    • F16M11/06—Means for attachment of apparatus; Means allowing adjustment of the apparatus relatively to the stand allowing pivoting
    • F16M11/12—Means for attachment of apparatus; Means allowing adjustment of the apparatus relatively to the stand allowing pivoting in more than one direction
    • F16M11/121—Means for attachment of apparatus; Means allowing adjustment of the apparatus relatively to the stand allowing pivoting in more than one direction constituted of several dependent joints
    • F16M11/123—Means for attachment of apparatus; Means allowing adjustment of the apparatus relatively to the stand allowing pivoting in more than one direction constituted of several dependent joints the axis of rotation intersecting in a single point, e.g. by using gimbals
    • F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
    • F16—ENGINEERING ELEMENTS AND UNITS; GENERAL MEASURES FOR PRODUCING AND MAINTAINING EFFECTIVE FUNCTIONING OF MACHINES OR INSTALLATIONS; THERMAL INSULATION IN GENERAL
    • F16M—FRAMES, CASINGS OR BEDS OF ENGINES, MACHINES OR APPARATUS, NOT SPECIFIC TO ENGINES, MACHINES OR APPARATUS PROVIDED FOR ELSEWHERE; STANDS; SUPPORTS
    • F16M11/00—Stands or trestles as supports for apparatus or articles placed thereon ; Stands for scientific apparatus such as gravitational force meters
    • F16M11/02—Heads
    • F16M11/18—Heads with mechanism for moving the apparatus relatively to the stand
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D3/00—Control of position or direction
    • G05D3/12—Control of position or direction using feedback
    • G05D3/20—Control of position or direction using feedback using a digital comparing device
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/63—Control of cameras or camera modules by using electronic viewfinders
    • H04N23/631—Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/695—Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/80—Camera processing pipelines; Components thereof
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/0094—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target

Definitions

  • the present invention relates to the field of electronic technology, and in particular, to a pan-tilt adjustment method, a pan-tilt adjustment device, a mobile platform, and a storage medium.
  • the process of repeatedly shooting the same scene may be referred to as image retake.
  • the gimbal will perform an angle transformation according to the angle parameter corresponding to the shooting track of the sample, so that the camera fixed on the gimbal can shoot the scene.
  • the so-called gimbal is a kind of Support equipment for installing and fixing the camera.
  • Practice shows that during each retake, there are usually differences in shooting time, shooting trajectory, and the position of each subject in the scene. If the gimbal only performs angle transformation according to the angle parameter corresponding to the shooting trajectory of the sample, then Will result in poor camera shooting. Therefore, how to adjust the gimbal during shooting has become a research hotspot.
  • Embodiments of the present invention provide a pan-tilt adjustment method, a pan-tilt adjustment device, a mobile platform, and a storage medium, which can automatically and effectively adjust the pan-tilt.
  • an embodiment of the present invention provides a pan-tilt adjustment method.
  • the pan-tilt adjustment method is applied to a mobile platform configured with a pan-tilt and a camera assembly.
  • the pan-tilt adjustment method includes:
  • the reference image frame sequence Acquiring a reference image frame sequence and a reference shooting trajectory formed by the movement of the mobile platform when shooting the reference image frame sequence, the reference image frame sequence including at least two reference image frames;
  • the shooting angle is estimated according to the reference shooting trajectory, and the gimbal is adjusted according to the estimated shooting angle;
  • the adjustment control amount of the PTZ is determined according to the current image frame and the reference image frame sequence, and the PTZ is adjusted according to the adjustment control amount.
  • an embodiment of the present invention provides a pan-tilt adjustment device.
  • the pan-tilt adjustment device includes: a memory and a processor, wherein the memory is used to store program instructions; the processor is configured to call According to the program instructions, perform the following steps:
  • the reference image frame sequence Acquiring a reference image frame sequence and a reference shooting trajectory formed by the movement of the mobile platform when shooting the reference image frame sequence, the reference image frame sequence including at least two reference image frames;
  • the shooting angle is estimated according to the reference shooting trajectory, and the gimbal is adjusted according to the estimated shooting angle;
  • the adjustment control amount of the PTZ is determined according to the current image frame and the reference image frame sequence, and the PTZ is adjusted according to the adjustment control amount.
  • an embodiment of the present invention provides a mobile platform.
  • the mobile platform includes:
  • Camera component used to capture images
  • PTZ used to control the orientation of the camera assembly
  • a power system installed on the fuselage, for providing power to the mobile platform
  • an embodiment of the present invention provides a computer storage medium that stores computer program instructions, and when the computer program instructions are executed, is used to implement the above-mentioned pan / tilt adjustment method.
  • the embodiment of the present invention can acquire the reference image frame sequence and the reference shooting trajectory formed by the movement of the mobile platform when shooting the reference image frame sequence; during the image retake process, the shooting angle is estimated according to the reference shooting trajectory, and the estimated Adjust the gimbal by shooting angle; obtain the current image frame; determine the adjustment control amount of the gimbal according to the sequence of the current image frame and the reference image frame, and adjust the gimbal according to the adjustment control amount. It can be seen that during the above gimbal adjustment process, the gimbal can be coarsely adjusted according to the reference shooting trajectory.
  • the adjustment control amount can be determined according to the current image frame and the previously taken reference image frame sequence , And adjust the gimbal again according to the adjustment control amount on the basis of coarse adjustment, so that the entire retake process is smoother and faster, and the image frame sequence obtained by the image retake meets the needs of users, thereby reducing the number of retakes and improving the efficiency of retake .
  • FIG. 1a is an application scenario diagram of a pan / tilt adjustment method provided by an embodiment of the present invention
  • FIG. 1b is an application scenario diagram of another PTZ adjustment method provided by an embodiment of the present invention.
  • FIG. 2 is a schematic flowchart of a method for adjusting a pan / tilt provided by an embodiment of the present invention
  • FIG. 3 is a schematic flowchart of a method for adjusting a gimbal according to another embodiment of the present invention.
  • FIG. 4 is a schematic diagram of a shooting trajectory provided by an embodiment of the present invention.
  • FIG. 5 is a schematic diagram of a shooting trajectory provided by another embodiment of the present invention.
  • FIG. 6 is a schematic structural diagram of a pan-tilt adjustment device provided by an embodiment of the present invention.
  • a component when a component is said to be “fixed” to another component, it can be directly on another component or it can also exist in a centered component. When a component is considered to be “connected” to another component, it can be directly connected to another component or there can be centered components at the same time.
  • An embodiment of the present invention proposes a pan-tilt adjustment method.
  • the pan-tilt adjustment method can be applied to a mobile platform equipped with a pan-tilt and a camera assembly.
  • the so-called pan-tilt is a support for installing or fixing a camera assembly device. Therefore, the camera assembly configured on the mobile platform can be fixed on the gimbal, and the orientation of the gimbal is the same as the orientation of the camera assembly.
  • the mobile platforms here include but are not limited to: drones, pan-tilt cameras; the so-called pan-tilt camera refers to a camera with a pan-tilt, which can control the camera to shoot from multiple angles.
  • the pan / tilt adjustment method proposed in the embodiment of the present invention also relates to a device with a shooting preview function, for example: a camera display screen with a front camera, an intelligent device connected to a drone, and an image stabilization system, and so on.
  • the pan / tilt adjustment method proposed in the embodiment of the present invention can realize the adjustment of the pan / tilt during the image re-shooting based on the reference image frame sequence, so that the camera components fixed on the pan / tilt can better perform the scene Shoot to improve the shooting effect.
  • a sample Before the image is retaken, a sample needs to be taken once, and the sample refers to a sequence of image frames obtained by taking the target scene for the first time.
  • the main purpose of shooting samples is to determine the approximate target scene, camera position, and composition; where the target scene refers to the scene including one or more target objects to be photographed, and the camera position refers to the shooting position of the camera component (including the shooting angle) ), Composition (also called framing) refers to the photographer's organization of the image to be expressed within a certain spatial range, forming a specific structure and form between the image part and the whole, and between the image space.
  • the visual system can also record the shooting trajectory formed by the movement of the mobile platform when shooting the samples.
  • the visual system here can be: VIO (Visual Inertial Odometry, visual inertial mileage calculation method) ).
  • VIO Visual Inertial Odometry, visual inertial mileage calculation method
  • the shooting track and the sample can correspond to each other through the physical time axis. Any shooting position in the shooting track corresponds to an angle parameter of the gimbal.
  • the angle parameter can indicate the angle of the gimbal at the position. That is, the shooting angle of the camera assembly at this position.
  • the mobile platform can acquire a reference image frame sequence and a reference shooting trajectory formed by the movement of the mobile platform when the reference image frame sequence is captured, the reference image frame sequence includes at least two reference image frames, and The reference image frames are arranged in the order of shooting time; the reference image frame sequence may be a sample taken before the image retake, or the history before the current image retake (or called the current image retake) The sequence of image frames captured by the image retake, for example, the current image retake is the 5th retake, then the reference image frame sequence may be the image frame captured by the historical image retake of the second, third or fourth time, etc.
  • the mobile platform can move according to the reference shooting trajectory; since any shooting position in the reference shooting trajectory corresponds to an angle parameter of the gimbal, during the movement of the mobile platform, the shooting can also be estimated based on the reference shooting trajectory Angle, and adjust the gimbal according to the estimated shooting angle. For example, if the estimated shooting angle according to the reference shooting trajectory is 120 degrees, then the angle of the gimbal can be adjusted to 120 degrees, as shown in FIG. 1a.
  • the mobile platform can also obtain the current image frame for better image retake.
  • the current image frame and the reference image frame sequence determine the adjustment control amount of the gimbal, and adjust the gimbal again according to the adjustment control amount.
  • adjusting the control amount includes: the shooting angle is 125 degrees, then the gimbal can be adjusted again so that the angle of the gimbal is 125 degrees, as shown in FIG. 1b.
  • the gimbal can be roughly adjusted according to the reference shooting track.
  • the current image can be adjusted according to Frame and the previously taken reference image frame sequence determine the adjustment control amount, and adjust the gimbal again according to the adjustment control amount on the basis of coarse adjustment, so that the entire retake process is smoother and faster, and the image frame sequence obtained by the image retake is satisfied User needs, thereby reducing the number of retakes and improving the efficiency of retakes.
  • an embodiment of the present invention proposes a pan-tilt adjustment method, which can be applied to a mobile platform that is equipped with a pan-tilt and a camera assembly.
  • the gimbal adjustment method may include the following steps S201-S204:
  • the reference image frame sequence may include at least two reference image frames, and each reference image frame may be arranged in sequence according to the order of shooting time.
  • the sample taken before the image retake can be obtained, and the sample is used as the reference image frame sequence.
  • the image frame sequence captured by the historical image retake before the current image retake (or referred to as the current image retake) can be obtained, and the historical image The sequence of captured image frames is taken as the sequence of reference image frames.
  • any shooting position in the reference shooting trajectory corresponds to an angle parameter of the gimbal.
  • the angle parameter can indicate the cloud The angle (or orientation) of the stage in this position. It can be known from the foregoing that the camera assembly is fixed on the gimbal, so the angle (or orientation) of the gimbal is consistent with the shooting angle of the camera assembly, that is, the angle parameter can also indicate the shooting angle of the camera assembly. While the angle of the gimbal is changed, the shooting angle of the camera assembly will also change as the angle of the gimbal changes.
  • the shooting angle is estimated according to the reference shooting trajectory, and the gimbal is adjusted according to the estimated shooting angle.
  • the mobile platform moves according to the reference shooting trajectory, and the position of the pan-tilt head configured on the mobile platform also changes accordingly. Therefore, the location of the gimbal can be determined in real time or periodically, and the target angle parameter of the gimbal corresponding to the location of the gimbal in the reference shooting trajectory can be obtained. After acquiring the target angle parameter of the gimbal, the shooting angle can be estimated according to the target angle parameter.
  • the angle represented by the target angle parameter can be used as the estimated shooting angle, for example, the target angle parameter is “90, 60”, and the angle expressed by the target angle parameter is “90 degrees in the vertical direction and 60 degrees in the horizontal direction ", Then the estimated shooting angle is” 90 degrees in the vertical direction and 60 degrees in the horizontal direction ".
  • the shooting angle of the camera assembly can be adjusted by adjusting the angle of the gimbal. Therefore, after the estimated shooting angle is determined, the gimbal can be adjusted according to the estimated shooting angle, and the angle of the gimbal can be adjusted to the estimated shooting angle.
  • the current image frame here refers to an image frame sampled from the image collection taken by the camera component based on a preset image sampling rule, and the image sampling rule includes: the current image frame obtained by sampling and the image obtained by the previous sampling
  • the number of difference pixels between frames is within a preset interval.
  • the difference pixel refers to a different pixel between the current image frame and the image frame obtained by the last sampling, and the value of the preset interval can be set according to an empirical value or business requirements.
  • the specific implementation method of obtaining the current image frame may be: first obtain an image collection captured by the camera component, and the image collection includes at least one frame image; the camera component may perform image capture in real time, and add the captured image frame to In the image collection, the image collection is updated in real time. Then, the current image frame is sampled from the image collection according to a preset image sampling rule, and the current image frame obtained by sampling may be the image frame newly added to the image collection.
  • the image collection includes image frame a, image frame b, and image frame c.
  • the camera component captures image frame d and adds image frame d to the image collection. At this time, the latest image frame added to the image collection is Image frame d. Since the number of difference pixels between the image frame d and the image frame obtained by the last sampling is within a preset interval, the image frame d can be used as the current image frame.
  • S204 Determine the adjustment control amount of the gimbal according to the current image frame and the reference image frame sequence, and adjust the gimbal according to the adjustment control amount.
  • the quantity may include the first angle parameter and the second angle parameter of the gimbal.
  • the first angle parameter is the angle parameter of the gimbal in the horizontal direction
  • the second angle parameter is the angle parameter of the gimbal in the vertical direction.
  • the current shooting trajectory formed by the movement of the mobile platform during the shooting of the current image frame may also be recorded according to the recording of the VIO , And the measured values of the IMU (Inertial Measurement Unit) and the electronic compass to determine whether the difference between the current shooting trajectory and the reference shooting trajectory meets the preset conditions. If it is satisfied, the current image frame and reference image frame sequence are executed to determine the adjustment control amount of the gimbal; if it is not satisfied, then the orientation of the gimbal is corrected.
  • the direction of the gimbal here can be understood as the angle of the gimbal .
  • the embodiment of the present invention can acquire the reference image frame sequence and the reference shooting trajectory formed by the movement of the mobile platform when shooting the reference image frame sequence; during the image retake process, the shooting angle is estimated according to the reference shooting trajectory, and the estimated Adjust the gimbal by shooting angle; obtain the current image frame; determine the adjustment control amount of the gimbal according to the sequence of the current image frame and the reference image frame, and adjust the gimbal according to the adjustment control amount. It can be seen that during the above gimbal adjustment process, the gimbal can be coarsely adjusted according to the reference shooting trajectory.
  • the adjustment control amount can be determined according to the current image frame and the previously taken reference image frame sequence , And adjust the gimbal again according to the adjustment control amount on the basis of coarse adjustment, so that the entire retake process is smoother and faster, and the image frame sequence obtained by the image retake meets the user's needs, thereby reducing the number of retakes and improving the efficiency of retake .
  • an embodiment of the present invention proposes a pan-tilt adjustment method, which can be applied to a mobile platform that is equipped with a pan-tilt and a camera assembly.
  • the method for adjusting the gimbal may include the following steps S301-S307:
  • the reference image frame sequence includes at least two reference image frames.
  • the reference image frame sequence is obtained from the camera component.
  • the embodiments of the present invention may perform compression processing on the reference image frame sequence captured by the camera component to obtain an image compression package, and obtain the image compression package from the camera component, thereby improving the acquisition efficiency.
  • the image compression package here can include I-frames and P-frames.
  • I-frames can also be called internal frames, which refers to the first frame of the compression process.
  • I-frames can be understood to be based on the original image captured by the camera component.
  • Compressed frame; P frame refers to the frame obtained by compressing the data of this frame according to the difference between this frame and the adjacent previous frame (I frame or P frame).
  • P frame can be understood as the original shot based on the camera component The frame where the image difference between the image and the previous frame image is compressed.
  • the initial position here can be determined according to any shooting position in the reference shooting track, that is, the image can be retaken from any shooting position, which improves the flexibility and compatibility of the image retake.
  • the initial shooting position in the reference shooting track can be selected as the initial position, that is, the starting position of the current shooting track at this time is the same as or similar to the starting shooting position of the reference shooting track; the distance can also be selected from the reference shooting track
  • the starting position of the starting shooting position in the preset distance is taken as the starting position, that is, the distance between the starting position of the current shooting track and the starting shooting position of the reference shooting track at this time is equal to the preset distance;
  • the midpoint shooting position in the reference shooting track is selected as the starting position, that is, the starting position of the current shooting track at this time is the same as or similar to the midpoint shooting position of the reference shooting track, and so on.
  • the mobile platform When re-taking images, you can place the mobile platform in the initial position and adjust the camera orientation that is roughly the same as when taking the reference image frame sequence, that is, adjust the angle of the gimbal so that the angle of the adjusted gimbal is the same as before The angle of the gimbal when shooting the sequence of reference image frames is approximately the same. Then enter the re-recording mode, the photographer can click the re-recording function button in the user interface to start the re-recording mode.
  • the start instruction of the image retake may include: a user's click instruction, that is, when the photographer's click instruction is detected, it is determined that the start instruction of the image retake is detected.
  • the start instruction of the image retake may further include: a press instruction, a voice instruction, and so on.
  • the initial position of the image retake can be determined by a measuring device such as an IMU inertial measurement unit or an electronic compass.
  • the corresponding reference starting position may be determined from the reference shooting track according to the initial position, for example, the initial position is the same as or similar to the starting shooting position of the reference shooting track, then the reference starting position is the starting point of the reference shooting track The starting shooting position; for another example, the initial position is the same as or similar to the midpoint shooting position of the reference shooting track, then the reference starting position is the midpoint shooting position in the reference shooting track, and so on.
  • one or more image frames captured at or near the reference start position in the reference image frame sequence are used as reference image frames, that is, the number of reference image frames may be more than one frame; the reference image frame here may be I frame.
  • the reference image frame may also be an IDR frame.
  • the so-called IDR frame is the first first I frame.
  • the present invention can perform image registration on the reference image frame and the real-time initial image frame captured at the initial position by the camera assembly.
  • the gimbal in the process of image registration, can be swung slightly from side to side and / or up and down, and the camera component can acquire the initial image frame in real time with the slight swing of the gimbal, and image the initial image frame with the reference image frame Registration. If the image registration is successful, you can directly retake the image. Optionally, if the image registration is successful, a prompt box of “start” may be displayed on the user interface; if a click instruction of the prompt box clicked by the photographer is detected, the image retake is started. It should be noted that successful image registration means that the target reference image frame that best matches the initial image frame can be found in each reference image frame.
  • an image may include multiple feature points.
  • the feature points refer to points in the image that have distinctive characteristics and can effectively reflect the essential characteristics of the image and can identify the target object in the image.
  • a feature point can be composed of two parts : a key point (Keypoint) and a descriptor (Descriptor): the key point refers to the position of the feature point in the image, which can have information such as direction and scale; the descriptor is usually a vector used for Describe information about pixels around key points.
  • the feature point may be employed to describe the algorithm were extracted from the original image frames (represented by I R) and the reference image frame (of I T) and a feature point descriptors
  • feature points described herein Algorithms may include, but are not limited to: SIFT (Scale-invariant feature transform) algorithm, SURF (Speeded up robust features) algorithm, ORB (Oriented FAST and rotated BRIEF) algorithm, and so on.
  • the feature points are matched according to the descriptors in the initial image frame and the descriptors in the reference image frame, and a matching result is obtained, which includes the matching degree between each reference image frame and the initial image frame; then, according to the matching As a result, the target reference image frame that most closely matches the initial image frame is found, and the outlier points (outlier points) are eliminated.
  • the RANSAC Random sample consensus
  • Epipolar constraint Epipolar constraint
  • the reference image frame corresponding to the maximum matching degree may be used as the target reference image frame.
  • the unique ratio method can also be used to verify whether the target reference image frame is the reference image frame that best matches the initial image frame (or is called the first match); if so, then Make sure the image registration is successful.
  • d 1 is used to indicate the matching degree between the reference image frames (that is, the target reference image frame) that most closely match the initial image frame in each reference image frame, and the most matching reference image frame The reference image frame with the largest matching degree between each reference image frame and the initial image frame;
  • d 2 is used to denote the reference image frame in each reference image frame that matches the initial image frame twice (or called the second match)
  • the reference image frame for the second match is the reference image frame with the second highest matching degree among the reference image frames and the initial image frame.
  • T d the value of the preset ratio can be set and adjusted according to actual needs, is an engineering experience value; if d 1 and d 2 satisfy the formula 1.1, then determine the target reference image frame as the initial image frame The best matching reference image frame.
  • the shooting angle is estimated according to the reference shooting trajectory, and the gimbal is adjusted according to the estimated shooting angle.
  • Obtain a set of images captured by the camera component where the set of images includes at least one frame of image; a current image frame is obtained from the image set according to a preset image sampling rule, and the image sampling rule includes: the current image frame obtained by sampling and the The number of difference pixels between image frames obtained by one sampling is within a preset interval.
  • the current shooting trajectory formed by the movement of the mobile platform when shooting the current image frame can also be obtained; it is determined whether the difference between the reference shooting trajectory and the current shooting trajectory satisfies the preset condition; if the reference shooting trajectory and the current If the difference between the shooting trajectories meets the preset condition, the step of determining the adjustment control amount of the gimbal according to the sequence of the current image frame and the reference image frame is executed, that is, step S307 is executed. In another embodiment, if the difference between the reference shooting trajectory and the current shooting trajectory does not satisfy the preset condition, the target orientation of the gimbal is calculated according to the current sampling position and the historical reference position.
  • the offset angle of the gimbal can be calculated according to the current sampling position and the historical reference position, and the target orientation of the gimbal can be calculated according to the offset angle.
  • the target orientation of the gimbal can also be calculated according to the current sampling position and the current reference position.
  • the orientation of the gimbal can be corrected according to the target orientation. If the pan-tilt head after the correction can be aimed at the target object to a certain extent, the camera component at this time can accurately shoot the target object to a certain extent, and step S307 can be performed; The degree of alignment with the target object indicates that the calibration has failed.
  • the specific implementation manner for determining whether the difference between the reference shooting track and the current shooting track satisfies the preset condition may include the following steps s11-s14:
  • the sampling position corresponding to the current image frame is taken as the current sampling position, and the sampling position corresponding to the image frame obtained by the last sampling is taken as the historical sampling position.
  • s12 Determine the current reference position from the reference shooting track according to the current sampling position, and determine the historical reference position from the reference shooting track according to the historical sampling position.
  • the position information in the reference shooting trajectory and the position information in the current shooting trajectory have One-to-one corresponding position correspondence.
  • the current reference position corresponding to the current sampling position can be determined from the reference shooting trajectory according to the position correspondence; and the historical reference position corresponding to the historical sampling position can be determined from the reference shooting trajectory according to the position correspondence.
  • the first reference image may be determined from the reference image frame sequence according to the current image frame corresponding to the current sampling position, the image frame obtained from the last sampling corresponding to the historical sampling position, and preset image sampling rules Frame and second reference image frame; the shooting position corresponding to the first image frame is used as the current reference position, and the shooting position corresponding to the second image frame is used as the historical reference position.
  • the first reference image frame and the second reference image frame satisfy the following conditions: the first reference image frame matches the current image frame, and the second reference image frame matches the image frame obtained by the previous sampling.
  • the first reference image frame and the second reference image frame may also satisfy that: the number of difference pixels between the first reference image frame and the second reference image frame is within a preset interval.
  • s13 Determine the first movement direction according to the current sampling position and the historical sampling position, and determine the second movement direction according to the current reference position and the historical reference position.
  • the second included angle formed by the line and the preset horizontal line, and the second included angle is used as the second moving direction.
  • s14 Determine whether the difference between the reference shooting trajectory and the current shooting trajectory satisfies the preset condition according to the first movement direction and the second movement direction.
  • the difference between the first movement direction and the second movement direction is less than a preset threshold, it is determined that the difference between the reference shooting trajectory and the current shooting trajectory satisfies the preset condition; as shown in FIG. 4, The difference between the reference shooting trajectory and the current shooting trajectory satisfies the preset condition, and a step of determining the adjustment control amount of the gimbal according to the current image frame and the reference image frame sequence may be performed. If the difference between the first movement direction and the second movement direction is greater than or equal to the preset threshold, it is determined that the difference between the reference shooting trajectory and the current shooting trajectory does not satisfy the preset condition; as shown in FIG. 5, The difference between the reference shooting trajectory and the current shooting trajectory does not satisfy the preset condition, and the gimbal needs to be corrected for orientation.
  • the preset threshold can be set according to actual needs or empirical values.
  • S307 Determine the adjustment control amount of the gimbal according to the current image frame and the reference image frame sequence, and adjust the gimbal according to the adjustment control amount.
  • image registration can be performed according to the current image frame and the reference image frame sequence; the adjustment control amount of the gimbal is determined according to the image registration result.
  • the adjustment control amount here can include the first angle parameter of the gimbal (use Indicated) and the second angle parameter (denoted by ⁇ ).
  • the first angle parameter is the angle parameter of the gimbal in the horizontal direction
  • the second angle parameter is the angle parameter of the gimbal in the vertical direction.
  • the specific implementation manner of image registration according to the current image frame and the reference image frame sequence may include the following steps s21-s23:
  • s21 determine the starting position of the current shooting track, and calculate the path length between the current sampling position and the starting position.
  • the set of key image frames includes at least two key image frames.
  • the total path and total duration of the reference shooting trajectory can be determined.
  • the starting position (ie, the initial position) of the current shooting track can be determined according to any shooting position in the reference shooting track, therefore, the total path of the reference shooting track here refers to: the reference shooting track The path between the reference starting position in and the end shooting position in the reference shooting track.
  • the reference starting position refers to the shooting position in the reference shooting track that corresponds to the starting position of the current shooting track.
  • the starting position of the current shooting track is determined according to the starting shooting position in the reference shooting track, then the reference starting position is the starting shooting position in the reference shooting track; in another example, the starting position of the current shooting track is According to the midpoint shooting position in the reference shooting track, the reference starting position is the midpoint shooting position in the reference shooting track, and so on.
  • the total duration of the reference shooting track refers to the duration between the shooting time corresponding to the reference start position in the reference shooting track and the shooting time corresponding to the end shooting position in the reference shooting track.
  • the shooting time of the current image frame can be calculated according to the path length, the total path and the total duration of the reference shooting trajectory. Specifically, the path ratio between the path length and the total path of the reference shooting trajectory can be calculated; the product of the path ratio and the total duration of the reference shooting trajectory is taken as the shooting time of the current image frame.
  • the path length is I c
  • the total path of the reference shooting track is I d
  • the total duration of the reference shooting track is T
  • the time difference between the shooting time of each key image frame in the key image frame set and the shooting time of the current image frame is less than the preset difference value
  • the value of the preset difference can be set according to actual business needs or empirical values. Specifically, the shooting time of each reference image frame in the reference image frame sequence can be obtained, and the time difference between the shooting time of each reference image frame and the shooting time of the current image frame can be calculated, and the time difference is less than the preset difference value.
  • the reference image frame is used as a key image frame and added to the key image frame set.
  • s23 Perform image registration on each key image frame in the current image frame and the key image frame set.
  • the feature point description algorithm such as SIFT algorithm can be used to extract the feature points and descriptors in the current image frame and each key image frame; and the feature points are performed according to the descriptor in the current image frame and the descriptors in each key image frame Match to get the result of image registration.
  • the image registration result includes the matching degree of each key image frame in the set of key image frames and the current image frame.
  • the specific implementation of determining the adjustment control amount of the gimbal according to the image registration result may include the following steps s31-s32:
  • s31 Determine the target image frame from the key image frame set according to the image registration result, and the target image frame is the key image frame with the largest matching degree in the key image frame set.
  • the key image frame corresponding to the maximum matching degree may be selected as the target image frame from the set of key image frames according to the image registration result.
  • the key image in order to avoid mismatches caused by noise and improve the accuracy of the target image frame, when determining the target image frame from the set of key image frames according to the image registration result, the key image can also be selected from the key image according to the image registration result.
  • the first key image frame corresponding to the maximum matching degree and the second key image frame corresponding to the second largest matching degree in the frame set are determined.
  • the maximum matching here may also be referred to as the first largest matching degree, and the second largest matching degree It can be called the second largest matching degree.
  • the matching ratio between the first key image frame and the second key image frame if the matching degree ratio is greater than the preset ratio, the first key image frame is determined as the target image frame, the preset ratio can be set according to actual needs
  • the fixed adjustment can be an engineering experience value.
  • the difference in matching degree between the first key image frame and the second key image frame may also be calculated. If the difference in match degree is greater than a preset threshold, the first key image frame is determined as the target image frame.
  • the preset threshold can be set and adjusted according to actual needs, and can be an engineering experience value.
  • the Epipolar Constraint method and / or the RANSAC method can also be used to eliminate the abnormal points of the target image frame, so as to improve the accuracy of the subsequent calculation of the angle difference.
  • the target function when calculating the angle difference between the target image frame and the current image frame, the target function may be established according to the target image frame and the current image frame first, and the target function includes the image scale parameter (s) and the pixel offset parameter ( ⁇ p ).
  • the objective function f (x) can be shown in Equation 1.2:
  • the objective function can be optimized to determine the image scale value of the image scale parameter and the pixel offset value of the pixel offset parameter, as shown in Equation 1.3.
  • the minimum value of the objective function f (x) is obtained by optimizing the objective function.
  • the objective function can be optimized by adjusting the values of s and ⁇ p.
  • the value of the optimized objective function meets the minimization condition;
  • the image scale value is the value of the image scale parameter in the objective function when the minimization condition is met, that is, the value of s, and
  • the pixel offset value is The value of the pixel offset parameter in the objective function when the minimization condition is met, that is, the value of ⁇ p.
  • the angle difference includes the first angle parameter And the second angle parameter ⁇ .
  • Fov field of view
  • Fov is the diagonal Fov of the camera component sensor
  • hFov horizontal field of view
  • vFov vertical field of view
  • w is the frame width (pixels) in the resolution
  • h is the frame height (pixels) in the resolution
  • the ratio of w and h is generally 16: 9 or 4: 3, such as 1920 * 1080.
  • the PTZ is adjusted according to the adjustment instruction.
  • the photographer can adjust the composition and camera position according to the actual scene requirements, and input adjustment instructions to correct the orientation of the gimbal based on the reference image frame sequence.
  • the reference image frame sequence can be adjusted according to the adjustment instruction, and the adjusted reference image frame sequence can be used as a new reference image frame sequence to cover the reference image frame sequence before adjustment.
  • the gimbal will be adjusted according to the new reference image frame sequence. It can be seen that in the subsequent image retake, the reference image frame sequence is adjusted and iterated, which is not exactly the same as the previously taken reference image frame sequence.
  • the reference image frame sequence is updated in the continuous adjustment iteration.
  • the photographer can input an adjustment instruction according to the current orientation of the gimbal. After receiving the photographer's adjustment instruction, the gimbal can be directly adjusted according to the adjustment instruction.
  • the embodiment of the present invention can acquire the reference image frame sequence and the reference shooting trajectory formed by the movement of the mobile platform when shooting the reference image frame sequence; during the image retake process, the shooting angle is estimated according to the reference shooting trajectory Adjust the gimbal by shooting angle; obtain the current image frame; determine the adjustment control amount of the gimbal according to the sequence of the current image frame and the reference image frame, and adjust the gimbal according to the adjustment control amount. It can be seen that during the above gimbal adjustment process, the gimbal can be coarsely adjusted according to the reference shooting trajectory.
  • the adjustment control amount can be determined according to the current image frame and the previously taken reference image frame sequence , And adjust the gimbal again according to the adjustment control amount on the basis of coarse adjustment, so that the entire retake process is smoother and faster, and the image frame sequence obtained by the image retake meets the user's needs, thereby reducing the number of retakes and improving the efficiency of retake .
  • an embodiment of the present invention provides a schematic structural diagram of a pan-tilt adjustment device as shown in FIG. 6.
  • the pan-tilt adjustment device shown in FIG. 6 may include at least: : A memory 101 and a processor 102, wherein the memory 101 is used to store program instructions; the processor 102 is configured to call and execute the program instructions.
  • the processor 102 may be a central processing unit (Central Processing Unit, CPU), and the processor may also be another general-purpose processor, that is, a microprocessor or any conventional processor, for example: digital signal Processor (Digital Signal Processor, DSP), application specific integrated circuit (Application Specific Integrated Circuit, ASIC), ready-made programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic device, discrete gate or transistor logic device, Discrete hardware components, etc.
  • DSP Digital Signal Processor
  • ASIC Application Specific Integrated Circuit
  • FPGA ready-made programmable gate array
  • FPGA Field-Programmable Gate Array
  • the memory 101 may include a read-only memory and a random access memory, and provide instructions and data to the processor 102. Therefore, the processor 102 and the memory 101 are not limited here.
  • the memory 101 is used to store program instructions; the processor 102 is configured to call the program instructions and perform the following steps:
  • the reference image frame sequence Acquiring a reference image frame sequence and a reference shooting trajectory formed by the movement of the mobile platform when shooting the reference image frame sequence, the reference image frame sequence including at least two reference image frames;
  • the shooting angle is estimated according to the reference shooting trajectory, and the gimbal is adjusted according to the estimated shooting angle;
  • the adjustment control amount of the PTZ is determined according to the current image frame and the reference image frame sequence, and the PTZ is adjusted according to the adjustment control amount.
  • the processor 102 when acquiring the current image frame, is configured to call the program instruction and perform the following steps:
  • the current image frame is obtained by sampling from the image set according to a preset image sampling rule.
  • the image sampling rule includes: the number of difference pixels between the current image frame obtained by sampling and the image frame obtained by the previous sampling is within a preset Within the interval.
  • processor 102 is further configured to call the program instructions and perform the following steps:
  • a step of determining the adjustment control amount of the PTZ based on the current image frame and the reference image frame sequence is performed.
  • processor 102 is configured to call the program instructions and perform the following steps:
  • the processor 102 when determining whether the difference between the reference shooting trajectory and the current shooting trajectory satisfies a preset condition, the processor 102 is configured to call the program instruction and perform the following steps:
  • Whether the difference between the reference shooting trajectory and the current shooting trajectory meets a preset condition is determined according to the first movement direction and the second movement direction.
  • the processor 102 when determining whether the difference between the reference shooting trajectory and the current shooting trajectory satisfies a preset condition according to the first movement direction and the second movement direction, the processor 102 It is configured to call the program instructions and perform the following steps:
  • the difference between the first movement direction and the second movement direction is greater than or equal to the preset threshold, it is determined that the difference between the reference shooting trajectory and the current shooting trajectory does not satisfy the preset condition .
  • the processor 102 when determining the adjustment control amount of the gimbal according to the current image frame and the reference image frame sequence, is configured to call the program instruction and perform the following steps :
  • the adjustment control amount of the gimbal is determined according to the image registration result.
  • the processor 102 when image registration is performed according to the current image frame and the reference image frame sequence, the processor 102 is configured to call the program instruction and perform the following steps:
  • the processor 102 when determining a set of key image frames from the sequence of reference image frames according to the path length and the reference shooting trajectory, is configured to call the program instructions and execute The following steps:
  • the image registration result includes the matching degree of each key image frame in the key image frame set with the current image frame; the adjustment control of the PTZ is determined according to the image registration result
  • the processor 102 is configured to call the program instructions and perform the following steps:
  • a target image frame from the set of key image frames according to an image registration result, the target image frame being the key image frame with the greatest degree of matching in the set of key image frames;
  • the processor 102 when the target image frame is determined from the set of key image frames according to the image registration result, the processor 102 is configured to call the program instruction and perform the following steps:
  • the first key image frame is determined as the target image frame.
  • the processor 102 when calculating the angle difference between the target image frame and the current image frame, is configured to call the program instruction and perform the following steps:
  • the objective function including an image scale parameter and a pixel offset parameter
  • the value of the optimized objective function satisfies the minimization condition
  • the image scale value is the value of the image scale parameter in the objective function when the minimization condition is satisfied
  • the pixel offset value is the value of the pixel offset parameter in the objective function when the minimization condition is satisfied value.
  • processor 102 is further configured to call the program instructions and perform the following steps:
  • the initial position of the image retake is determined, and the initial position is used as the starting position of the current shooting trajectory;
  • image retake is started.
  • the initial position is determined according to any shooting position in the reference shooting trajectory.
  • processor 102 is further configured to call the program instructions and perform the following steps:
  • the PTZ is adjusted according to the adjustment instruction.
  • the embodiment of the present invention can acquire the reference image frame sequence and the reference shooting trajectory formed by the movement of the mobile platform when shooting the reference image frame sequence; during the image retake process, the shooting angle is estimated according to the reference shooting trajectory, and the estimated Adjust the gimbal by shooting angle; obtain the current image frame; determine the adjustment control amount of the gimbal according to the sequence of the current image frame and the reference image frame, and adjust the gimbal according to the adjustment control amount. It can be seen that during the above gimbal adjustment process, the gimbal can be coarsely adjusted according to the reference shooting trajectory.
  • the adjustment control amount can be determined according to the current image frame and the previously taken reference image frame sequence , And adjust the gimbal again according to the adjustment control amount on the basis of coarse adjustment, so that the entire retake process is smoother and faster, and the image frame sequence obtained by the image retake meets the needs of users, thereby reducing the number of retakes and improving the efficiency of retake .
  • an embodiment of the present invention also provides a mobile platform, the mobile platform may include at least: a camera assembly for capturing images; a gimbal for controlling the orientation of the camera assembly; a fuselage; a power system, installed in On the fuselage, it is used to provide power for the mobile platform; and the gimbal adjustment device of the above embodiment.
  • the mobile platform includes any one of the following: a drone or a gimbal camera.
  • the storage medium may be a magnetic disk, an optical disk, a read-only memory (Read-Only Memory, ROM) or a random access memory (Random Access Memory, RAM), etc. That is, an embodiment of the present invention further provides a computer storage medium that stores computer program instructions, which are suitable for being loaded by a processor and executing the pan / tilt adjustment method shown in FIG. 2 or FIG. 3.

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Automation & Control Theory (AREA)
  • Mechanical Engineering (AREA)
  • Manufacturing & Machinery (AREA)
  • Studio Devices (AREA)

Abstract

一种云台调整方法、云台调整设备、移动平台及存储介质。该云台调整方法应用于移动平台，该移动平台配置有云台和摄像组件，该云台调整方法包括：获取参考图像帧序列，以及在拍摄所述参考图像帧序列时所述移动平台移动所形成的参考拍摄轨迹，所述参考图像帧序列包括至少两帧参考图像帧；在图像复拍的过程中，根据所述参考拍摄轨迹预估拍摄角度，并根据预估的拍摄角度调整所述云台；获取当前图像帧；根据所述当前图像帧和所述参考图像帧序列确定所述云台的调整控制量，并根据所述调整控制量调整所述云台。

Description

云台调整方法、云台调整设备、移动平台及介质 技术领域
本发明涉及电子技术领域，尤其涉及一种云台调整方法、云台调整设备、移动平台及存储介质。
背景技术
在影像制作的拍摄过程中，为了拍摄同一个场景，通常需要重复拍摄多次以获取最好的拍摄图像，重复拍摄同一个场景的过程可以简称为图像复拍。在每次图像复拍的过程中，云台会根据样片的拍摄轨迹所对应的角度参数进行角度变换，以使得云台上所固定的摄像机可以对该场景进行拍摄，所谓的云台是一种安装、固定摄像机的支撑设备。实践表明，在每次复拍的过程中，通常存在拍摄时间、拍摄轨迹以及场景中的各拍摄对象的位置等差异，若云台只根据样片的拍摄轨迹所对应的角度参数进行角度变换，则会导致摄像机的拍摄效果较差。因此，如何在拍摄过程中对云台进行调整成为了研究热点。
发明内容
本发明实施例提供了一种云台调整方法、云台调整设备、移动平台以及存储介质，可自动对云台进行有效的调整。
一方面，本发明实施例提供了一种云台调整方法，该云台调整方法应用于移动平台，该移动平台配置有云台和摄像组件，该云台调整方法包括：
获取参考图像帧序列，以及在拍摄所述参考图像帧序列时所述移动平台移动所形成的参考拍摄轨迹，所述参考图像帧序列包括至少两帧参考图像帧；
在图像复拍的过程中，根据所述参考拍摄轨迹预估拍摄角度，并根据预估的拍摄角度调整所述云台；
获取当前图像帧；
根据所述当前图像帧和所述参考图像帧序列确定所述云台的调整控制量，并根据所述调整控制量调整所述云台。
另一方面，本发明实施例提供了一种云台调整设备，该云台调整设备包括：存储器和处理器，其中，所述存储器用于存储程序指令；所述处理器被配置用于调用所述程序指令，执行如下步骤：
获取参考图像帧序列，以及在拍摄所述参考图像帧序列时所述移动平台移动所形成的参考拍摄轨迹，所述参考图像帧序列包括至少两帧参考图像帧；
在图像复拍的过程中，根据所述参考拍摄轨迹预估拍摄角度，并根据预估的拍摄角度调整所述云台；
获取当前图像帧；
根据所述当前图像帧和所述参考图像帧序列确定所述云台的调整控制量，并根据所述调整控制量调整所述云台。
再一方面，本发明实施例提供了一种移动平台，该移动平台包括：
摄像组件，用于拍摄图像；
云台，用于控制所述摄像组件的朝向；
机身；
动力系统、安装在所述机身上，用于为所述移动平台提供动力；
以及如上述的云台调整设备。
再一方面，本发明实施例提供一种计算机存储介质，所述计算机存储介质存储有计算机程序指令，该计算机程序指令被执行时用于实现上述的云台调整方法。
本发明实施例可以获取参考图像帧序列，以及在拍摄参考图像帧序列时移动平台移动所形成的参考拍摄轨迹；在图像复拍过程中，根据参考拍摄轨迹预估拍摄角度，并根据预估的拍摄角度调整云台；获取当前图像帧；根据当前图像帧和参考图像帧序列确定云台的调整控制量，并根据调整控制量调整云台。由此可见，上述的云台调整过程中，可以先根据参考拍摄轨迹对云台进行粗调整，在拍摄轨迹发生微小变化时，可以根据当前图像帧和之前拍摄的参考图像帧序列确定调整控制量，并在粗调整的基础上再次根据调整控制量调整云台，从而使得整个复拍过程更加流畅快捷，使得图像复拍得到的图像帧序列满足用户需求，从而减少复拍次数，提高复拍效率。
附图说明
为了更清楚地说明本发明实施例技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1a是本发明实施例提供的一种云台调整方法的应用场景图；
图1b是本发明实施例提供的另一种云台调整方法的应用场景图；
图2是本发明实施例提供的一种云台调整方法的流程示意图；
图3是本发明另一实施例提供的一种云台调整方法的流程示意图；
图4是本发明实施例提供的一种拍摄轨迹的示意图；
图5是本发明另一实施例提供的一种拍摄轨迹的示意图；
图6是本发明实施例提供的一种云台调整设备的结构示意图。
具体实施方式
下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚地描述，显然，所描述的实施例仅仅是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。
需要说明的是，当组件被称为“固定于”另一个组件，它可以直接在另一个组件上或者也可以存在居中的组件。当一个组件被认为是“连接”另一个组件，它可以是直接连接到另一个组件或者可能同时存在居中组件。
除非另有定义，本文所使用的所有的技术和科学术语与属于本发明的技术领域的技术人员通常理解的含义相同。本文中在本发明的说明书中所使用的术语只是为了描述具体的实施例的目的，不是旨在于限制本发明。本文所使用的术语“及/或”包括一个或多个相关的所列项目的任意的和所有的组合。
下面结合附图，对本发明的一些实施方式作详细说明。在不冲突的情况下，下述的实施例及实施例中的特征可以相互组合。
本发明实施例提出了一种云台调整方法，该云台调整方法可以应用在移动平台中，该移动平台配置有云台和摄像组件，所谓的云台是一种安装或者固定 摄像组件的支撑设备。因此，移动平台所配置的摄像组件可以固定在云台上，云台的朝向与摄像组件的朝向相同。其中，此处的移动平台包括但不限于：无人机、云台摄像机；所谓的云台摄像机是指带有云台的摄像机，可控制摄像机从多个角度进行拍摄。本发明实施例所提出的云台调整方法还涉及带有拍摄预览功能的设备，例如：带有前置摄像机的相机显示屏、连接无人机、影像增稳系统的智能设备，等等。本发明实施例所提出的云台调整方法可以实现在基于参考图像帧序列进行图像复拍的过程中，对云台进行调整，以使得云台上所固定的摄像组件可以对场景进行更好的拍摄，提高拍摄效果。
在图像复拍之前，需要拍摄一次样片，所述样片是指对目标场景进行第一次拍摄所得到的图像帧序列。拍摄样片的主要目的为：确定大致的目标场景，机位、构图；其中，目标场景是指包括一个或多个待拍摄的目标对象的场景，机位是指摄像组件的拍摄位置(包括拍摄角度)，构图(又可称为取景)是指拍摄者在一定的空间范围内，对要表达的形象进行组织安排，形成形象的部分和整体之间，形象空间之间的特定的结构、形式。在拍摄样片的过程中，还可通过视觉系统记录拍摄样片时移动平台移动所形成的拍摄轨迹，此处的视觉系统可以为：下视视觉定位系统的VIO(Visual Inertial Odometry，视觉惯性里程计算法)。需要说明的是，拍摄轨迹和样片通过物理时间轴线可以一一对应，拍摄轨迹中的任一个拍摄位置均对应着云台的一个角度参数，该角度参数可表明云台在该位置时的角度，即摄像组件在该位置时的拍摄角度。
在拍摄好样片之后，可以进行一次或多次的图像复拍。在后续的图像复拍的过程中，移动平台可以获取参考图像帧序列以及在拍摄参考图像帧序列时移动平台移动所形成的参考拍摄轨迹，该参考图像帧序列包括至少两帧参考图像帧，且各参考图像帧按照拍摄时间的先后顺序依次排列；该参考图像帧序列可以是图像复拍前所拍摄的样片，也可以是在当前图像复拍(或称为本次图像复拍)之前的历史图像复拍所拍摄的图像帧序列，例如当前图像复拍是第5次复拍，那么参考图像帧序列可以是第2次、第3次或者第4次等历史图像复拍所拍摄的图像帧序列。移动平台可以根据该参考拍摄轨迹进行移动；由于参考拍摄轨迹中的任一个拍摄位置均对应着云台的一个角度参数，因此在移动平台的移动过程中，还可以根据该参考拍摄轨迹预估拍摄角度，并根据预估的拍摄角 度调整云台。例如，根据参考拍摄轨迹预估的拍摄角度为120度，那么可以将云台的角度调整至120度，如图1a所示。
由于每次图像复拍的过程中，通常存在拍摄时间、拍摄轨迹以及目标场景中的各目标对象的位置等差异，因此为了更好地进行图像复拍，移动平台还可获取当前图像帧，根据当前图像帧和参考图像帧序列确定云台的调整控制量，并根据调整控制量再次调整云台。例如，调整控制量包括：拍摄角度为125度，那么可以再次调整云台，以使得云台的角度为125度，如图1b所示。
由此可见，上述的云台调整过程中，可以先根据参考拍摄轨迹对云台进行粗调整，在拍摄轨迹、拍摄时间以及目标场景中的各目标对象的位置发生微小变化时，可以根据当前图像帧和之前拍摄的参考图像帧序列确定调整控制量，并在粗调整的基础上再次根据调整控制量调整云台，从而使得整个复拍过程更加流畅快捷，使得图像复拍得到的图像帧序列满足用户需求，从而减少复拍次数，提高复拍效率。
基于上述描述，本发明实施例提出了一种云台调整方法，该云台调整方法可应用于移动平台，该移动平台配置有云台和摄像组件。请参见图2，该云台调整方法可以包括如下步骤S201-S204：
S201，获取参考图像帧序列，以及在拍摄参考图像帧序列时移动平台移动所形成的参考拍摄轨迹。
参考图像帧序列可以包括至少两帧参考图像帧，各参考图像帧可以按照拍摄时间的先后顺序依次排列。在一种实施方式中，在获取参考图帧序列时，可以获取图像复拍前所拍摄的样片，并将该样片作为参考图像帧序列。再一种实施方式中，在获取参考图像帧序列时，可以获取在当前图像复拍(或称为本次图像复拍)之前的历史图像复拍所拍摄的图像帧序列，并将历史图像复拍所拍摄的图像帧序列作为参考图像帧序列。
由于在每次图像复拍的过程中，可通过VIO记录拍摄图像帧序列时移动平台移动所形成的拍摄轨迹。因此还可获取VIO记录的在拍摄参考图像帧序列时移动平台移动所形成的参考拍摄轨迹，该参考拍摄轨迹中的任一个拍摄位置均对应着云台的一个角度参数，该角度参数可表明云台在该位置时的角度 (或者朝向)。由前述可知，摄像组件是固定在云台上的，因此云台的角度(或朝向)与摄像组件的拍摄角度是一致的，即该角度参数也可表明摄像组件的拍摄角度。云台的角度变换的同时，摄像组件的拍摄角度也会随着云台的角度变换而发生变换。
S202，在图像复拍的过程中，根据参考拍摄轨迹预估拍摄角度，并根据预估的拍摄角度调整云台。
在图像复拍的过程中，移动平台根据参考拍摄轨迹进行移动，移动平台上所配置的云台的位置也随之发生变化。因此，可以实时性或周期性地确定云台的所在位置，并获取在参考拍摄轨迹中，该云台的所在位置所对应的云台的目标角度参数。在获取到云台的目标角度参数之后，可以根据该目标角度参数预估拍摄角度。具体的，可以将目标角度参数所表示的角度作为预估的拍摄角度，例如，目标角度参数为“90，60”，该目标角度参数所表示的角度为“垂直方向90度，水平方向60度”，那么预估的拍摄角度为“垂直方向90度，水平方向60度”。
由于摄像组件是固定在云台上的，因此可以通过调整云台的角度来调整摄像组件的拍摄角度。因此，在确定了预估的拍摄角度之后，可以根据预估的拍摄角度调整云台，将云台的角度调整至预估的拍摄角度。
S203，获取当前图像帧。
此处的当前图像帧是指基于预设的图像采样规则从摄像组件拍摄的图像集合中所采样得到的图像帧，所述图像采样规则包括：采样得到的当前图像帧与上一次采样得到的图像帧之间的差异像素的数量在预设区间内。其中，差异像素是指当前图像帧与上一次采样得到的图像帧之间的不同像素，预设区间的取值可以根据经验值或者业务需求设置。实践表明，若将与上一次采样得到的图像帧之间的差异像素的数量较少的图像帧作为当前图像帧，容易导致步骤S204的计算量较大；若将与上一次采样得到的图像帧之间的差异像素的数量较多的图像帧作为当前图像帧，容易导致步骤S204的准确性较低。因此，本发明实施例在获取当前图像帧时，是将与上一次采样得到的图像帧之间的差异像素的数量在预设区间内的图像帧作为当前图像帧，从而降低步骤S204的计算量且提高步骤S204的准确性。
相应的，获取当前图像帧的具体实施方式可以是：先获取摄像组件拍摄的图像集合，该图像集合中包括至少一帧图像；摄像组件可以实时进行图像拍摄，并将拍摄到的图像帧添加至图像集合中，即图像集合是实时更新的。然后，根据预设的图像采样规则从图像集合中采样得到当前图像帧，采样得到的当前图像帧可以是最新添加至图像集合中的图像帧。例如，图像集合中包括图像帧a、图像帧b以及图像帧c，摄像组件又拍摄到图像帧d，并将图像帧d添加至图像集合中，此时最新添加至图像集合中的图像帧为图像帧d。由于图像帧d与上一次采样得到的图像帧之间的差异像素的数量在预设区间内，因此可以将图像帧d作为当前图像帧。
S204，根据当前图像帧和参考图像帧序列确定云台的调整控制量，并根据调整控制量调整云台。
在根据当前图像帧和参考图像帧序列确定云台的调整控制量时，可以根据当前图像帧和参考图像帧序列进行图像配准，根据图像配准结果确定云台的调整控制量，该调整控制量可以包括云台的第一角度参数和第二角度参数。其中，第一角度参数为云台在水平方向上的角度参数，第二角度参数为云台在垂直方向上的角度参数。
在一种实施方式中，在根据当前图像帧和参考图像帧序列确定云台的调整控制量之前，还可以根据VIO所记录的在拍摄该当前图像帧时，移动平台移动所形成的当前拍摄轨迹，以及IMU(Inertial Measurement Unit，惯性测量单元)和电子罗盘的测量值判断当前拍摄轨迹和参考拍摄轨迹之间的差异是否满足预设条件。若满足，则执行当前图像帧和参考图像帧序列确定云台的调整控制量的步骤；若不满足，则对云台的朝向进行校正，此处的云台的朝向可以理解成云台的角度。
本发明实施例可以获取参考图像帧序列，以及在拍摄参考图像帧序列时移动平台移动所形成的参考拍摄轨迹；在图像复拍过程中，根据参考拍摄轨迹预估拍摄角度，并根据预估的拍摄角度调整云台；获取当前图像帧；根据当前图像帧和参考图像帧序列确定云台的调整控制量，并根据调整控制量调整云台。由此可见，上述的云台调整过程中，可以先根据参考拍摄轨迹对云台进行粗调整，在拍摄轨迹发生微小变化时，可以根据当前图像帧和之前拍摄的参考图像 帧序列确定调整控制量，并在粗调整的基础上再次根据调整控制量调整云台，从而使得整个复拍过程更加流畅快捷，使得图像复拍得到的图像帧序列满足用户需求，从而减少复拍次数，提高复拍效率。
基于上述描述，本发明实施例提出了一种云台调整方法，该云台调整方法可应用于移动平台，该移动平台配置有云台和摄像组件。请参见图3，该云台调整方法可以包括如下步骤S301-S307：
S301，获取参考图像帧序列，以及在拍摄参考图像帧序列时移动平台移动所形成的参考拍摄轨迹，该参考图像帧序列包括至少两帧参考图像帧。
具体的，从摄像组件处获取参考图像帧序列。为了减少传输的数据量，提高传输效率，本发明实施例可以对摄像组件拍摄到的参考图像帧序列进行压缩处理以得到图像压缩包，从摄像组件处获取该图像压缩包，从而提高获取效率。此处的图像压缩包中可以包括I帧和P帧，其中，I帧又可称为内部画面，是指压缩处理的第一个帧，I帧可以理解成基于摄像组件拍摄到的原始图像进行压缩的帧；P帧是指根据本帧和相邻的前一帧(I帧或者P帧)的不同点来压缩本帧数据所得到的帧，P帧可以理解成基于摄像组件拍摄到的原始图像和前一帧图像之间的图像差异进行压缩的帧。
S302，若检测到图像复拍的开始指令，则确定图像复拍的初始位置，该初始位置作为当前拍摄轨迹的起始位置。
此处的初始位置可以根据参考拍摄轨迹中的任意拍摄位置确定，即可以从任意拍摄位置进行图像复拍，提高了图像复拍的灵活性以及兼容性。例如：可以选取参考拍摄轨迹中的起始拍摄位置作为所述初始位置，即此时的当前拍摄轨迹的起始位置与参考拍摄轨迹的起始拍摄位置相同或相近；还可以选取距离参考拍摄轨迹中的起始拍摄位置预设距离的拍摄位置作为所述起始位置，即此时的当前拍摄轨迹的起始位置与参考拍摄轨迹的起始拍摄位置之间的距离等于预设距离；还可以选取参考拍摄轨迹中的中点拍摄位置作为所述起始位置，即此时的当前拍摄轨迹的起始位置与参考拍摄轨迹的中点拍摄位置相同或相近，等等。
在进行图像复拍时，可以将移动平台放置在初始位置，并调整好大致与拍 摄参考图像帧序列时相同的机位朝向，即调整云台的角度以使得调整后的云台的角度与之前拍摄参考图像帧序列时云台的角度大致相同。然后进入重录模式，拍摄者可以点击用户界面中的重录功能按钮以开启重录模式。相应的，图像复拍的开始指令可以包括：用户的点击指令，即当检测到拍摄者的点击指令时，则确定检测到图像复拍的开始指令。可选的，图像复拍的开始指令还可包括：按压指令、语音指令，等等。在检测到图像复拍的开始指令后，可以通过IMU惯性测量单元或者电子罗盘等测量装置确定图像复拍的初始位置。
S303，根据初始位置从参考图像帧序列中获取基准图像帧，该基准图像帧所对应的拍摄位置与初始位置相对应。
具体的，可以先根据初始位置从参考拍摄轨迹中确定相对应的参考起始位置，例如，初始位置与参考拍摄轨迹的起始拍摄位置相同或相近，那么参考起始位置为参考拍摄轨迹的起始拍摄位置；又如，初始位置与参考拍摄轨迹的中点拍摄位置相同或相近，那么参考起始位置为参考拍摄轨迹中的中点拍摄位置，等等。然后将参考图像帧序列中在参考起始位置处或者附近拍摄得到的一帧或多帧图像帧均作为基准图像帧，即基准图像帧的数量可以不止一帧；此处的基准图像帧可以是I帧。在一个实施例中，该基准图像帧还可以是IDR帧，所谓的IDR帧是第一个首个I帧。
S304，若基准图像帧和在初始位置拍摄到的初始图像帧进行图像配准成功，则开始图像复拍。
由于初始位置可能与基准图像帧所对应的拍摄位置存在偏差，和/或开始拍摄时的云台的角度也存在偏差，从而导致当前图像复拍所拍摄到的图像帧与参考图像帧序列有偏差。因此，为了避免该偏差，提高拍摄的准确性，本发明实施在获取到基准图像帧之后，可以将基准图像帧和摄像组件在初始位置拍摄到的实时的初始图像帧进行图像配准。具体的，在图像配准的过程中，云台可以左右和/或上下轻微摆动扫描，摄像组件随着云台的轻微摆动可以实时获取初始图像帧，并将初始图像帧与基准图像帧进行图像配准。若图像配准成功，则可以直接进行图像复拍。可选的，若图像配准成功，则可以在用户界面显示一个“开始”的提示框；若检测到拍摄者点击该“开始”的提示框的点击指令，则开始图像复拍。需要说明的是，图像配准成功是指：可以在各基准图像帧中 找到与初始图像帧最匹配的目标基准图像帧。
通常，图像可以包括多个特征点，该特征点是指图像中具有鲜明特性并能够有效反映图像本质特征且可标识图像中目标对象的点。特征点可由两部分构成 ：关键点(Keypoint)和描述子(Descriptor)：关键点指的是该特征点在图像中的位置，可具有方向、尺度等信息；描述子通常是一个向量，用于描述关键点周围像素的信息。因此，在进行图像匹配时，可以先采用特征点描述算法分别提取初始图像帧(用I R表示)和基准图像帧(用I T表示)中的特征点及描述子，此处的特征点描述算法可以包括但不限于：SIFT(Scale-invariant feature transform，尺度不变特征转换)算法、SURF(Speeded up robust features，加速鲁棒特征)算法，ORB(Oriented FAST and rotated BRIEF)算法，等等。其次，根据初始图像帧中的描述子和基准图像帧中的描述子进行特征点匹配，得到匹配结果，该匹配结果中包括各基准图像帧与初始图像帧之间的匹配度；然后，根据匹配结果查找与初始图像帧最匹配的目标基准图像帧，并剔除异常点(outlier点)。可选的，可以采用RANSAC(Random sample consensus)和/或Epipolar constraint(极线约束)方法剔除异常点。
其中，在根据匹配结果查找与初始图像帧最匹配的目标基准图像帧的过程中，可以将最大匹配度所对应的基准图像帧作为目标基准图像帧。在一个实施例中，为了避免误匹配的情况，还可以采用unique ratio的方法来验证目标基准图像帧是否为与初始图像帧最匹配(或称为第一匹配)的基准图像帧；若是，则确定图像配准成功。其中，unique ratio的方法的原理如下：用d 1表示各基准图像帧中与初始图像帧最匹配的基准图像帧(即目标基准图像帧)之间的匹配度，该最匹配的基准图像帧为各基准图像帧中与初始图像帧之间的匹配度最大的基准图像帧；用d 2表示各基准图像帧中与初始图像帧次匹配(或称为第二匹配)的基准图像帧之间的匹配度，次匹配的基准图像帧为各基准图像帧中与初始图像帧之间的匹配度第二大的基准图像帧。设置一个预设比值T d，该预设比值的取值可以根据实际需要设定调整，是一个工程经验值；若d 1和d 2满足式1.1，则确定目标基准图像帧为与初始图像帧最匹配的基准图像帧。
Figure PCTCN2018116439-appb-000001
S305，在图像复拍的过程中，根据参考拍摄轨迹预估拍摄角度，并根据预估的拍摄角度调整云台。
S306，获取当前图像帧。
获取摄像组件拍摄的图像集合，该图像集合中包括至少一帧图像；根据预设的图像采样规则从图像集合中采样得到当前图像帧，所述图像采样规则包括：采样得到的当前图像帧与上一次采样得到的图像帧之间的差异像素的数量在预设区间内。
在一个实施例中，还可以获取在拍摄当前图像帧时，移动平台移动所形成的当前拍摄轨迹；判断参考拍摄轨迹与当前拍摄轨迹之间的差异是否满足预设条件；若参考拍摄轨迹与当前拍摄轨迹之间的差异满足预设条件，则执行根据当前图像帧和参考图像帧序列确定云台的调整控制量的步骤，即执行步骤S307。再一个实施例中，若参考拍摄轨迹与当前拍摄轨迹之间的差异不满足预设条件，则根据当前采样位置和历史参考位置计算云台的目标朝向。具体的，可以根据当前采样位置和历史参考位置计算云台的偏移角度，并根据偏移角度计算云台的目标朝向。可选的，还可以根据当前采样位置和当前参考位置计算云台的目标朝向。在计算得到目标朝向之后，可以根据目标朝向对云台进行朝向校正。若朝向校正后的云台可以在一定程度上对准目标对象，则此时的摄像组件可以在一定程度上准确地拍摄目标对象，并可以执行步骤S307；若朝向校正后的云台无法在一定程度上对准目标对象，则表明校正失败。
其中，判断参考拍摄轨迹与当前拍摄轨迹之间的差异是否满足预设条件的具体实施方式可以包括如下步骤s11-s14：
s11，将当前图像帧所对应的采样位置作为当前采样位置，将上一次采样得到的图像帧所对应的采样位置作为历史采样位置。
s12，根据当前采样位置从参考拍摄轨迹中确定当前参考位置，以及根据历史采样位置从参考拍摄轨迹中确定历史参考位置。
在一种实施方式中，由于在当前图像复拍的过程中，移动平台是根据参考拍摄轨迹进行移动从而形成当前拍摄轨迹的，因此参考拍摄轨迹中的位置信息和当前拍摄轨迹中的位置信息具有一一对应的位置对应关系。可以根据该位置对应关系从参考拍摄轨迹中确定与当前采样位置相对应的当前参考位置；以及 根据该位置对应关系从参考拍摄轨迹中确定与历史采样位置相对应的历史参考位置。
再一种实施方式中，可以根据当前采样位置所对应的当前图像帧、历史采样位置所对应的上一次采样得到的图像帧以及预设的图像采样规则从参考图像帧序列中确定第一参考图像帧和第二参考图像帧；将第一图像帧所对应的拍摄位置作为当前参考位置，将第二图像帧所对应的拍摄位置作为历史参考位置。其中，第一参考图像帧和第二参考图像帧满足如下条件：第一参考图像帧与当前图像帧匹配，第二参考图像帧与所述上一次采样得到的图像帧匹配。可选的，第一参考图像帧和第二参考图像帧还可满足：第一参考图像帧与第二参考图像帧之间的差异像素的数量在预设区间内。
s13，根据当前采样位置和历史采样位置确定第一移动方向，以及根据当前参考位置和历史参考位置确定第二移动方向。
可以计算当前采样位置和历史采样位置之间的连线与预设水平线所形成的第一夹角，并将第一夹角作为第一移动方向；计算当前参考位置和历史参考位置之间的连线与预设水平线所形成的第二夹角，并将第二夹角作为第二移动方向。如图4所示，设历史采样位置为A’、当前采样位置为B’，计算A’与B’之间的连线(用实线表示)与预设水平线所形成的第一夹角为α，那么第一移动方向为α；设历史参考位置为A、当前参考位置为B，计算A与B之间的连线(用虚线表示)与预设水平线所形成的第一夹角为β，那么第二移动方向为β。
s14，根据第一移动方向和第二移动方向判断参考拍摄轨迹与当前拍摄轨迹之间的差异是否满足预设条件。
具体的，若第一移动方向和第二移动方向之间的差值小于预设阈值，则确定参考拍摄轨迹与当前拍摄轨迹之间的差异满足预设条件；如图4所示，此时的参考拍摄轨迹与当前拍摄轨迹之间的差异满足预设条件，可以执行根据当前图像帧和参考图像帧序列确定云台的调整控制量的步骤。若第一移动方向和第二移动方向之间的差值大于或等于预设阈值，则确定参考拍摄轨迹与所当前拍摄轨迹之间的差异不满足预设条件；如图5所示，此时的参考拍摄轨迹与当前拍摄轨迹之间的差异不满足预设条件，则需要对云台进行朝向校正。预设阈值 可以根据实际需求或者经验值设置。
S307，根据当前图像帧和参考图像帧序列确定云台的调整控制量，并根据调整控制量调整云台。
具体的，可以根据当前图像帧和参考图像帧序列进行图像配准；根据图像配准结果确定云台的调整控制量。此处的调整控制量可以包括云台的第一角度参数(用
Figure PCTCN2018116439-appb-000002
表示)和第二角度参数(用θ表示)。其中，第一角度参数为云台在水平方向上的角度参数，第二角度参数为云台在垂直方向上的角度参数。
其中，根据当前图像帧和参考图像帧序列进行图像配准的具体实施方式可以包括如下步骤s21-s23：
s21，确定当前拍摄轨迹的起始位置，并计算当前采样位置与起始位置之间的路径长度。
s22，根据路径长度和参考拍摄轨迹从参考图像帧序列中确定关键图像帧集合，该关键图像帧集合中包括至少两帧关键图像帧。
具体的，可以确定参考拍摄轨迹的总路径和总时长。需要说明的是，由前述可知，当前拍摄轨迹的起始位置(即初始位置)可以根据参考拍摄轨迹中的任意拍摄位置确定，因此，此处的参考拍摄轨迹的总路径是指：参考拍摄轨迹中的参考起始位置与参考拍摄轨迹中的终点拍摄位置之间的路径，参考起始位置是指参考拍摄轨迹中与当前拍摄轨迹的起始位置相对应的拍摄位置。例如，当前拍摄轨迹的起始位置是根据参考拍摄轨迹中的起始拍摄位置确定的，那么参考起始位置为参考拍摄轨迹中的起始拍摄位置；又如，当前拍摄轨迹的起始位置是根据参考拍摄轨迹中的中点拍摄位置确定的，那么参考起始位置为参考拍摄轨迹中的中点拍摄位置，等等。相应的，参考拍摄轨迹的总时长是指：参考拍摄轨迹中的参考起始位置所对应的拍摄时间与参考拍摄轨迹中的终点拍摄位置所对应的拍摄时间之间的时长。
在确定了参考拍摄轨迹的总路径和总时长之后，可以根据路径长度、参考拍摄轨迹的总路径和总时长，计算当前图像帧的拍摄时间。具体的，可以计算路径长度和参考拍摄轨迹的总路径之间的路径比值；将路径比值和参考拍摄轨迹的总时长的乘积作为当前图像帧的拍摄时间。设路径长度为I c，参考拍摄轨迹的总路径为I d，参考拍摄轨迹的总时长为T，那么当前图像帧的拍摄时间t c 为：t c＝(I c/I d)T。
然后根据当前图像帧的拍摄时间从参考图像帧序列中确定关键图像帧集合，该关键图像帧集合中的各关键图像帧的拍摄时间与当前图像帧的拍摄时间之间的时间差小于预设差值，该预设差值的取值可以根据实际的业务需求或者经验值设置。具体的，可以获取参考图像帧序列中的各参考图像帧的拍摄时间，并计算各参考图像帧的拍摄时间与当前图像帧的拍摄时间之间的时间差，将时间差小于预设差值所对应的参考图像帧作为关键图像帧并添加至关键图像帧集合中。
s23，将当前图像帧和关键图像帧集合中的各关键图像帧进行图像配准。
具体的，可以采用SIFT算法等特征点描述算法分别提取当前图像帧和各关键图像帧中的特征点及描述子；根据当前图像帧中的描述子和各关键图像帧中的描述子进行特征点匹配，得到图像配准结果。
其中，图像配准结果包括关键图像帧集合中的各关键图像帧与当前图像帧的匹配度。相应的，根据图像配准结果确定云台的调整控制量的具体实施方式可以包括如下步骤s31-s32：
s31，根据图像配准结果从关键图像帧集合中确定目标图像帧，目标图像帧为关键图像帧集合中匹配度最大的关键图像帧。
具体的，可以根据图像配准结果从关键图像帧集合中选取最大匹配度所对应的关键图像帧作为目标图像帧。在一个实施例中，为了避免噪声引起的误匹配，提高目标图像帧的准确性，在根据图像配准结果从关键图像帧集合中确定目标图像帧时，还可以根据图像配准结果从关键图像帧集合中确定最大匹配度所对应的第一关键图像帧，以及次大匹配度所对应的第二关键图像帧，此处的最大匹配又可称为第一大匹配度，次大匹配度又可称为第二大匹配度。计算第一关键图像帧与第二关键图像帧之间的匹配度比值；若匹配度比值大于预设比值，则将第一关键图像帧确定为目标图像帧，该预设比值可以根据实际需要设定调整，可以是一个工程经验值。可选的，还可以计算第一关键图像帧与第二关键图像帧之间的匹配度差值，若匹配度差值大于预设阈值，则将第一关键图像帧确定为目标图像帧，该预设阈值可以根据实际需要设定调整，可以是一个工程经验值。
可选的，在确定目标图像帧之后，还可以采用Epipolar constraint方法和/或RANSAC方法剔除目标图像帧的异常点，以便于提高后续计算角度差异的准确性。
s32，计算目标图像帧和当前图像帧之间的角度差异，并将角度差异作为云台的调整控制量。
具体的，在计算目标图像帧和当前图像帧之间的角度差异时，可以先根据目标图像帧和当前图像帧建立目标函数，该目标函数包括图像比例参数(s)和像素偏移参数(Δp)。目标函数f(x)可以如式1.2所示：
Figure PCTCN2018116439-appb-000003
Figure PCTCN2018116439-appb-000004
表示的是当前图像帧上的第i个点，
Figure PCTCN2018116439-appb-000005
表示的是目标图像帧上对应匹配的第i个点；
Figure PCTCN2018116439-appb-000006
表示的是：
Figure PCTCN2018116439-appb-000007
移动了Δp(Δu,Δv)(行、列像素)后，经过图像比例参数s的变换后，与
Figure PCTCN2018116439-appb-000008
的差值。
其次，可以按照减小目标函数的值的原则，优化目标函数以确定图像比例参数的图像比例值和像素偏移参数的像素偏移值，如式1.3所示。
Figure PCTCN2018116439-appb-000009
Figure PCTCN2018116439-appb-000010
表示的是通过优化目标函数的方法求取目标函数f(x)的最小值。在具体实施过程中，可以通过调整s和Δp的值以优化目标函数。在一个实施例中，优化后的目标函数的值满足最小化条件；图像比例值为满足最小化条件时的目标函数中的图像比例参数的取值，即s的取值，像素偏移值为满足最小化条件时的目标函数中的像素偏移参数的取值，即Δp的值。
然后，可以获取摄像组件的内参以及图像分辨率，并根据内参、图像分辨率和图像位移值计算目标图像帧和当前图像帧之间的角度差异，角度差异包括第一角度参数
Figure PCTCN2018116439-appb-000011
和第二角度参数θ。具体的计算过程如下：
i.根据出厂标定或是厂商提供的摄像组件的内参(Fov)以及图像分辨率，计算hFov和vFov:
Figure PCTCN2018116439-appb-000012
Figure PCTCN2018116439-appb-000013
其中，Fov(field of view)为摄像组件sensor对角线的Fov；
hFov(horizontal field of view)为水平方向上的Fov；
vFov(vertical field of view)为垂直方向上的Fov；
w为分辨率中的画幅宽(像素)，h为分辨率中的画幅高(像素)，w和h的比值一般是16:9或者4:3，比如1920*1080。
ii.根据图像位移值Δp(Δu,Δv)计算第一角度参数
Figure PCTCN2018116439-appb-000014
和第二角度参数θ：
Figure PCTCN2018116439-appb-000015
Figure PCTCN2018116439-appb-000016
再一个实施例中，若接收到调整指令，则根据调整指令对云台进行调整。在一种应用场景中，拍摄者可以根据实际的场景需求调整构图以及机位，并输入调整指令以在参考图像帧序列的基础上修正云台的朝向。在接收到拍摄者的调整指令后，可以根据调整指令对参考图像帧序列进行调整，并将调整后的参考图像帧序列作为新的参考图像帧序列以覆盖调整前的参考图像帧序列。在后续的云台调整过程中，则会根据新的参考图像帧序列对云台进行调整。由此可见，在后续的图像复拍时，参考图像帧序列是经过调整迭代的，与之前拍摄的参考图像帧序列不完全一样，参考图像帧序列是在不断调整迭代中更新的。
再一种应用场景中，拍摄者可以根据云台的当前朝向输入调整指令。在接收到拍摄者的调整指令后，可以根据该调整指令直接对云台进行调整。
本发明实施例可以获取参考图像帧序列，以及在拍摄参考图像帧序列时移动平台移动所形成的参考拍摄轨迹；在图像复拍过程中，根据参考拍摄轨迹预估拍摄角度，并根据预估的拍摄角度调整云台；获取当前图像帧；根据当前图像帧和参考图像帧序列确定云台的调整控制量，并根据调整控制量调整云台。由此可见，上述的云台调整过程中，可以先根据参考拍摄轨迹对云台进行粗调整，在拍摄轨迹发生微小变化时，可以根据当前图像帧和之前拍摄的参考图像帧序列确定调整控制量，并在粗调整的基础上再次根据调整控制量调整云台，从而使得整个复拍过程更加流畅快捷，使得图像复拍得到的图像帧序列满足用 户需求，从而减少复拍次数，提高复拍效率。
基于上述方法实施例的描述，在一种实施方式中，本发明实施例提供了一种如图6所示的云台调整设备的结构示意图，如图6所示的云台调整设备可以至少包括：存储器101和处理器102，其中，所述存储器101用于存储程序指令；所述处理器102被配置用于调用执行所述程序指令。
在一种实施方式中，该处理器102可以是中央处理单元(Central Processing Unit，CPU)，该处理器还可以是其他通用处理器，即微处理器或者任何常规的处理器，例如：数字信号处理器(Digital Signal Processor，DSP)、专用集成电路(Application Specific Integrated Circuit，ASIC)、现成可编程门阵列(Field-Programmable Gate Array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件，等等。
该存储器101可以包括只读存储器和随机存取存储器，并向处理器102提供指令和数据。因此，在此对于处理器102和存储器101不作限定。
在一个实施例中，所述存储器101用于存储程序指令；所述处理器102被配置用于调用所述程序指令，执行如下步骤：
获取参考图像帧序列，以及在拍摄所述参考图像帧序列时所述移动平台移动所形成的参考拍摄轨迹，所述参考图像帧序列包括至少两帧参考图像帧；
在图像复拍的过程中，根据所述参考拍摄轨迹预估拍摄角度，并根据预估的拍摄角度调整所述云台；
获取当前图像帧；
根据所述当前图像帧和所述参考图像帧序列确定所述云台的调整控制量，并根据所述调整控制量调整所述云台。
在一种实施方式中，在获取当前图像帧时，所述处理器102被配置用于调用所述程序指令，执行如下步骤：
获取所述摄像组件拍摄的图像集合，所述图像集合中包括至少一帧图像；
根据预设的图像采样规则从所述图像集合中采样得到当前图像帧，所述图像采样规则包括：采样得到的当前图像帧与上一次采样得到的图像帧之间的差异像素的数量在预设区间内。
再一种实施方式中，所述处理器102还被配置用于调用所述程序指令，执行如下步骤：
获取在拍摄所述当前图像帧时，所述移动平台移动所形成的当前拍摄轨迹；
判断所述参考拍摄轨迹与所述当前拍摄轨迹之间的差异是否满足预设条件；
若所述参考拍摄轨迹与所述当前拍摄轨迹之间的差异满足预设条件，则执行根据所述当前图像帧和所述参考图像帧序列确定所述云台的调整控制量的步骤。
再一种实施方式中，所述处理器102被配置用于调用所述程序指令，执行如下步骤：
若所述参考拍摄轨迹与所述当前拍摄轨迹之间的差异不满足预设条件，则根据所述当前采样位置和所述历史参考位置计算所述云台的目标朝向；
根据所述目标朝向对所述云台进行朝向校正。
再一种实施方式中，在判断所述参考拍摄轨迹与所述当前拍摄轨迹之间的差异是否满足预设条件时，所述处理器102被配置用于调用所述程序指令，执行如下步骤：
将所述当前图像帧所对应的采样位置作为当前采样位置，将上一次采样得到的图像帧所对应的采样位置作为历史采样位置；
根据所述当前采样位置从所述参考拍摄轨迹中确定当前参考位置，以及根据所述历史采样位置从所述参考拍摄轨迹中确定历史参考位置；
根据所述当前采样位置和所述历史采样位置确定第一移动方向，以及根据所述当前参考位置和所述历史参考位置确定第二移动方向；
根据所述第一移动方向和所述第二移动方向判断所述参考拍摄轨迹与所述当前拍摄轨迹之间的差异是否满足预设条件。
再一种实施方式中，在根据所述第一移动方向和所述第二移动方向判断所述参考拍摄轨迹与所述当前拍摄轨迹之间的差异是否满足预设条件时，所述处理器102被配置用于调用所述程序指令，执行如下步骤：
若所述第一移动方向和所述第二移动方向之间的差值小于预设阈值，则确定所述参考拍摄轨迹与所述当前拍摄轨迹之间的差异满足预设条件；
若所述第一移动方向和所述第二移动方向之间的差值大于或等于所述预设阈值，则确定所述参考拍摄轨迹与所述当前拍摄轨迹之间的差异不满足预设条件。
再一种实施方式中，在根据所述当前图像帧和所述参考图像帧序列确定所述云台的调整控制量时，所述处理器102被配置用于调用所述程序指令，执行如下步骤：
根据所述当前图像帧和所述参考图像帧序列进行图像配准；
根据图像配准结果确定所述云台的调整控制量。
再一种实施方式中，在根据所述当前图像帧和所述参考图像帧序列进行图像配准时，所述处理器102被配置用于调用所述程序指令，执行如下步骤：
确定所述当前拍摄轨迹的起始位置，并计算当前采样位置与所述起始位置之间的路径长度；
根据所述路径长度和所述参考拍摄轨迹从所述参考图像帧序列中确定关键图像帧集合，所述关键图像帧集合中包括至少两帧关键图像帧；
将所述当前图像帧和所述关键图像帧集合中的各关键图像帧进行图像配准。
再一种实施方式中，在根据所述路径长度和所述参考拍摄轨迹从所述参考图像帧序列中确定关键图像帧集合时，所述处理器102被配置用于调用所述程序指令，执行如下步骤：
确定所述参考拍摄轨迹的总路径和总时长；
根据所述路径长度、所述参考拍摄轨迹的总路径和所述总时长，计算所述当前图像帧的拍摄时间；
根据所述当前图像帧的拍摄时间从所述参考图像帧序列中确定关键图像帧集合，所述关键图像帧集合中的各关键图像帧的拍摄时间与所述当前图像帧的拍摄时间之间的时间差小于预设差值。
再一种实施方式中，所述图像配准结果包括所述关键图像帧集合中的各关键图像帧与所述当前图像帧的匹配度；在根据图像配准结果确定所述云台的调整控制量时，所述处理器102被配置用于调用所述程序指令，执行如下步骤：
根据图像配准结果从所述关键图像帧集合中确定目标图像帧，所述目标图 像帧为所述关键图像帧集合中匹配度最大的关键图像帧；
计算所述目标图像帧和所述当前图像帧之间的角度差异，并将所述角度差异作为所述云台的调整控制量。
再一种实施方式中，在根据图像配准结果从所述关键图像帧集合中确定目标图像帧时，所述处理器102被配置用于调用所述程序指令，执行如下步骤：
根据所述图像配准结果从所述关键图像帧集合中确定最大匹配度所对应的第一关键图像帧，以及次大匹配度所对应的第二关键图像帧；
计算所述第一关键图像帧与所述第二关键图像帧之间的匹配度比值；
若所述匹配度比值大于预设比值，则将所述第一关键图像帧确定为目标图像帧。
再一种实施方式中，在计算所述目标图像帧和所述当前图像帧之间的角度差异时，所述处理器102被配置用于调用所述程序指令，执行如下步骤：
根据所述目标图像帧和所述当前图像帧建立目标函数，所述目标函数包括图像比例参数和像素偏移参数；
按照减小所述目标函数的值的原则，优化所述目标函数以确定所述图像比例参数的图像比例值和所述像素偏移参数的像素偏移值；
获取所述摄像组件的内参以及图像分辨率，并根据所述内参、所述图像分辨率和所述图像位移值计算所述目标图像帧和所述当前图像帧之间的角度差异。
再一种实施方式中，优化后的目标函数的值满足最小化条件；
所述图像比例值为满足所述最小化条件时的目标函数中的图像比例参数的取值，所述像素偏移值为满足所述最小化条件时的目标函数中的像素偏移参数的取值。
再一种实施方式中，所述处理器102被配置还用于调用所述程序指令，执行如下步骤：
若检测到图像复拍的开始指令，则确定图像复拍的初始位置，所述初始位置作为所述当前拍摄轨迹的起始位置；
根据所述初始位置从所述参考图像帧序列中获取基准图像帧，所述基准图像帧所对应的拍摄位置与所述初始位置相对应；
若所述基准图像帧和在所述初始位置拍摄到的初始图像帧进行图像配准成功，则开始图像复拍。
再一种实施方式中，所述初始位置根据所述参考拍摄轨迹中的任意拍摄位置确定。
再一种实施方式中，所述处理器102被配置还用于调用所述程序指令，执行如下步骤：
若接收到调整指令，则根据所述调整指令对所述云台进行调整。
本发明实施例可以获取参考图像帧序列，以及在拍摄参考图像帧序列时移动平台移动所形成的参考拍摄轨迹；在图像复拍过程中，根据参考拍摄轨迹预估拍摄角度，并根据预估的拍摄角度调整云台；获取当前图像帧；根据当前图像帧和参考图像帧序列确定云台的调整控制量，并根据调整控制量调整云台。由此可见，上述的云台调整过程中，可以先根据参考拍摄轨迹对云台进行粗调整，在拍摄轨迹发生微小变化时，可以根据当前图像帧和之前拍摄的参考图像帧序列确定调整控制量，并在粗调整的基础上再次根据调整控制量调整云台，从而使得整个复拍过程更加流畅快捷，使得图像复拍得到的图像帧序列满足用户需求，从而减少复拍次数，提高复拍效率。
基于上述描述，本发明实施例还提供了一种移动平台，该移动平台可以至少包括：摄像组件，用于拍摄图像；云台，用于控制摄像组件的朝向；机身；动力系统，安装在机身上，用于为移动平台提供动力；以及上述实施例的云台调整设备。
在一个实施例中，所述移动平台包括以下任一种：无人机或者云台摄像机。
本实施例提供的移动平台其执行方式和有益效果与前述实施例提供的云台调整设备类似，在这里不再赘述。
需要说明的是，上述描述的移动平台和云台调整设备的具体工作过程，可以参考前述各个实施例中的相关描述，在此不再赘述。
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程，是可以通过计算机程序来指令相关的硬件来完成，所述的程序可存储于一计算机可读取存储介质中，该程序在执行时，可包括如上述各方法的实施例的流程。 其中，所述的存储介质可为磁碟、光盘、只读存储记忆体(Read-Only Memory，ROM)或随机存储记忆体(Random Access Memory，RAM)等。即本发明实施例还提供一种计算机存储介质，所述计算机存储介质存储有计算机程序指令，该计算机程序指令适于由处理器加载并执行如图2或图3所示的云台调整方法。
以上所揭露的仅为本发明的部分实施例而已，当然不能以此来限定本发明之权利范围，本领域普通技术人员可以理解实现上述实施例的全部或部分流程，并依本发明权利要求所作的等同变化，仍属于发明所涵盖的范围。

Claims (34)

  1. 一种云台调整方法，所述云台调整方法应用于移动平台，所述移动平台配置有云台和摄像组件，其特征在于，包括：
    获取参考图像帧序列，以及在拍摄所述参考图像帧序列时所述移动平台移动所形成的参考拍摄轨迹，所述参考图像帧序列包括至少两帧参考图像帧；
    在图像复拍的过程中，根据所述参考拍摄轨迹预估拍摄角度，并根据预估的拍摄角度调整所述云台；
    获取当前图像帧；
    根据所述当前图像帧和所述参考图像帧序列确定所述云台的调整控制量，并根据所述调整控制量调整所述云台。
  2. 如权利要求1所述的方法，其特征在于，所述获取当前图像帧，包括：
    获取所述摄像组件拍摄的图像集合，所述图像集合中包括至少一帧图像；
    根据预设的图像采样规则从所述图像集合中采样得到当前图像帧，所述图像采样规则包括：采样得到的当前图像帧与上一次采样得到的图像帧之间的差异像素的数量在预设区间内。
  3. 如权利要求2所述的方法，其特征在于，所述方法还包括：
    获取在拍摄所述当前图像帧时，所述移动平台移动所形成的当前拍摄轨迹；
    判断所述参考拍摄轨迹与所述当前拍摄轨迹之间的差异是否满足预设条件；
    若所述参考拍摄轨迹与所述当前拍摄轨迹之间的差异满足预设条件，则执行根据所述当前图像帧和所述参考图像帧序列确定所述云台的调整控制量的步骤。
  4. 如权利要求3所述的方法，其特征在于，所述方法还包括：
    若所述参考拍摄轨迹与所述当前拍摄轨迹之间的差异不满足预设条件，则根据所述当前采样位置和所述历史参考位置计算所述云台的目标朝向；
    根据所述目标朝向对所述云台进行朝向校正。
  5. 如权利要求3所述的方法，其特征在于，所述判断所述参考拍摄轨迹与所述当前拍摄轨迹之间的差异是否满足预设条件，所述方法还包括：
    将所述当前图像帧所对应的采样位置作为当前采样位置，将上一次采样得到的图像帧所对应的采样位置作为历史采样位置；
    根据所述当前采样位置从所述参考拍摄轨迹中确定当前参考位置，以及根据所述历史采样位置从所述参考拍摄轨迹中确定历史参考位置；
    根据所述当前采样位置和所述历史采样位置确定第一移动方向，以及根据所述当前参考位置和所述历史参考位置确定第二移动方向；
    根据所述第一移动方向和所述第二移动方向判断所述参考拍摄轨迹与所述当前拍摄轨迹之间的差异是否满足预设条件。
  6. 如权利要求5所述的方法，其特征在于，所述根据所述第一移动方向和所述第二移动方向判断所述参考拍摄轨迹与所述当前拍摄轨迹之间的差异是否满足预设条件，包括：
    若所述第一移动方向和所述第二移动方向之间的差值小于预设阈值，则确定所述参考拍摄轨迹与所述当前拍摄轨迹之间的差异满足预设条件；
    若所述第一移动方向和所述第二移动方向之间的差值大于或等于所述预设阈值，则确定所述参考拍摄轨迹与所述当前拍摄轨迹之间的差异不满足预设条件。
  7. 如权利要求3所述的方法，其特征在于，所述根据所述当前图像帧和所述参考图像帧序列确定所述云台的调整控制量，包括：
    根据所述当前图像帧和所述参考图像帧序列进行图像配准；
    根据图像配准结果确定所述云台的调整控制量。
  8. 如权利要求7所述的方法，其特征在于，所述根据所述当前图像帧和所述参考图像帧序列进行图像配准，包括：
    确定所述当前拍摄轨迹的起始位置，并计算当前采样位置与所述起始位置之间的路径长度；
    根据所述路径长度和所述参考拍摄轨迹从所述参考图像帧序列中确定关键图像帧集合，所述关键图像帧集合中包括至少两帧关键图像帧；
    将所述当前图像帧和所述关键图像帧集合中的各关键图像帧进行图像配准。
  9. 如权利要求8所述的方法，其特征在于，所述根据所述路径长度和所述参考拍摄轨迹从所述参考图像帧序列中确定关键图像帧集合，包括：
    确定所述参考拍摄轨迹的总路径和总时长；
    根据所述路径长度、所述参考拍摄轨迹的总路径和所述总时长，计算所述当前图像帧的拍摄时间；
    根据所述当前图像帧的拍摄时间从所述参考图像帧序列中确定关键图像帧集合，所述关键图像帧集合中的各关键图像帧的拍摄时间与所述当前图像帧的拍摄时间之间的时间差小于预设差值。
  10. 如权利要求8所述的方法，其特征在于，所述图像配准结果包括所述关键图像帧集合中的各关键图像帧与所述当前图像帧的匹配度；所述根据图像配准结果确定所述云台的调整控制量，包括：
    根据图像配准结果从所述关键图像帧集合中确定目标图像帧，所述目标图像帧为所述关键图像帧集合中匹配度最大的关键图像帧；
    计算所述目标图像帧和所述当前图像帧之间的角度差异，并将所述角度差异作为所述云台的调整控制量。
  11. 如权利要求10所述的方法，其特征在于，所述根据图像配准结果从所述关键图像帧集合中确定目标图像帧，包括：
    根据所述图像配准结果从所述关键图像帧集合中确定最大匹配度所对应的第一关键图像帧，以及次大匹配度所对应的第二关键图像帧；
    计算所述第一关键图像帧与所述第二关键图像帧之间的匹配度比值；
    若所述匹配度比值大于预设比值，则将所述第一关键图像帧确定为目标图像帧。
  12. 如权利要求10所述的方法，其特征在于，所述计算所述目标图像帧和所述当前图像帧之间的角度差异，包括：
    根据所述目标图像帧和所述当前图像帧建立目标函数，所述目标函数包括图像比例参数和像素偏移参数；
    按照减小所述目标函数的值的原则，优化所述目标函数以确定所述图像比例参数的图像比例值和所述像素偏移参数的像素偏移值；
    获取所述摄像组件的内参以及图像分辨率，并根据所述内参、所述图像分辨率和所述图像位移值计算所述目标图像帧和所述当前图像帧之间的角度差异。
  13. 如权利要求11所述的方法，其特征在于，优化后的目标函数的值满足最小化条件；
    所述图像比例值为满足所述最小化条件时的目标函数中的图像比例参数的取值，所述像素偏移值为满足所述最小化条件时的目标函数中的像素偏移参数的取值。
  14. 如权利要求3所述的方法，其特征在于，所述方法还包括：
    若检测到图像复拍的开始指令，则确定图像复拍的初始位置，所述初始位置作为所述当前拍摄轨迹的起始位置；
    根据所述初始位置从所述参考图像帧序列中获取基准图像帧，所述基准图像帧所对应的拍摄位置与所述初始位置相对应；
    若所述基准图像帧和在所述初始位置拍摄到的初始图像帧进行图像配准成功，则开始图像复拍。
  15. 如权利要求14所述的方法，其特征在于，所述初始位置根据所述参考拍摄轨迹中的任意拍摄位置确定。
  16. 如权利要求1-15任一项所述的方法，其特征在于，所述方法还包括：
    若接收到调整指令，则根据所述调整指令对所述云台进行调整。
  17. 一种云台调整设备，其特征在于，包括存储器和处理器，其中，所述存储器用于存储程序指令，所述处理器被配置用于调用所述程序指令，执行如下步骤：
    获取参考图像帧序列，以及在拍摄所述参考图像帧序列时所述移动平台移动所形成的参考拍摄轨迹，所述参考图像帧序列包括至少两帧参考图像帧；
    在图像复拍的过程中，根据所述参考拍摄轨迹预估拍摄角度，并根据预估的拍摄角度调整所述云台；
    获取当前图像帧；
    根据所述当前图像帧和所述参考图像帧序列确定所述云台的调整控制量，并根据所述调整控制量调整所述云台。
  18. 如权利要求17所述的云台调整设备，其特征在于，在获取当前图像帧时，所述处理器被配置用于调用所述程序指令，执行如下步骤：
    获取所述摄像组件拍摄的图像集合，所述图像集合中包括至少一帧图像；
    根据预设的图像采样规则从所述图像集合中采样得到当前图像帧，所述图像采样规则包括：采样得到的当前图像帧与上一次采样得到的图像帧之间的差异像素的数量在预设区间内。
  19. 如权利要求18所述的云台调整设备，其特征在于，所述处理器被配置用于调用所述程序指令，执行如下步骤：
    获取在拍摄所述当前图像帧时，所述移动平台移动所形成的当前拍摄轨迹；
    判断所述参考拍摄轨迹与所述当前拍摄轨迹之间的差异是否满足预设条件；
    若所述参考拍摄轨迹与所述当前拍摄轨迹之间的差异满足预设条件，则执行根据所述当前图像帧和所述参考图像帧序列确定所述云台的调整控制量的 步骤。
  20. 如权利要求19所述的云台调整设备，其特征在于，所述处理器被配置用于调用所述程序指令，执行如下步骤：
    若所述参考拍摄轨迹与所述当前拍摄轨迹之间的差异不满足预设条件，则根据所述当前采样位置和所述历史参考位置计算所述云台的目标朝向；
    根据所述目标朝向对所述云台进行朝向校正。
  21. 如权利要求19所述的云台调整设备，其特征在于，在判断所述参考拍摄轨迹与所述当前拍摄轨迹之间的差异是否满足预设条件时，所述处理器被配置用于调用所述程序指令，执行如下步骤：
    将所述当前图像帧所对应的采样位置作为当前采样位置，将上一次采样得到的图像帧所对应的采样位置作为历史采样位置；
    根据所述当前采样位置从所述参考拍摄轨迹中确定当前参考位置，以及根据所述历史采样位置从所述参考拍摄轨迹中确定历史参考位置；
    根据所述当前采样位置和所述历史采样位置确定第一移动方向，以及根据所述当前参考位置和所述历史参考位置确定第二移动方向；
    根据所述第一移动方向和所述第二移动方向判断所述参考拍摄轨迹与所述当前拍摄轨迹之间的差异是否满足预设条件。
  22. 如权利要求21所述的云台调整设备，其特征在于，在根据所述第一移动方向和所述第二移动方向判断所述参考拍摄轨迹与所述当前拍摄轨迹之间的差异是否满足预设条件时，所述处理器被配置用于调用所述程序指令，执行如下步骤：
    若所述第一移动方向和所述第二移动方向之间的差值小于预设阈值，则确定所述参考拍摄轨迹与所述当前拍摄轨迹之间的差异满足预设条件；
    若所述第一移动方向和所述第二移动方向之间的差值大于或等于所述预设阈值，则确定所述参考拍摄轨迹与所述当前拍摄轨迹之间的差异不满足预设条件。
  23. 如权利要求19所述的云台调整设备，其特征在于，在根据所述当前图像帧和所述参考图像帧序列确定所述云台的调整控制量时，所述处理器被配置用于调用所述程序指令，执行如下步骤：
    根据所述当前图像帧和所述参考图像帧序列进行图像配准；
    根据图像配准结果确定所述云台的调整控制量。
  24. 如权利要求23所述的云台调整设备，其特征在于，在根据所述当前图像帧和所述参考图像帧序列进行图像配准时，所述处理器被配置用于调用所述程序指令，执行如下步骤：
    确定所述当前拍摄轨迹的起始位置，并计算当前采样位置与所述起始位置之间的路径长度；
    根据所述路径长度和所述参考拍摄轨迹从所述参考图像帧序列中确定关键图像帧集合，所述关键图像帧集合中包括至少两帧关键图像帧；
    将所述当前图像帧和所述关键图像帧集合中的各关键图像帧进行图像配准。
  25. 如权利要求24所述的云台调整设备，其特征在于，在根据所述路径长度和所述参考拍摄轨迹从所述参考图像帧序列中确定关键图像帧集合时，所述处理器被配置用于调用所述程序指令，执行如下步骤：
    确定所述参考拍摄轨迹的总路径和总时长；
    根据所述路径长度、所述参考拍摄轨迹的总路径和所述总时长，计算所述当前图像帧的拍摄时间；
    根据所述当前图像帧的拍摄时间从所述参考图像帧序列中确定关键图像帧集合，所述关键图像帧集合中的各关键图像帧的拍摄时间与所述当前图像帧的拍摄时间之间的时间差小于预设差值。
  26. 如权利要求24所述的云台调整设备，其特征在于，所述图像配准结果包括所述关键图像帧集合中的各关键图像帧与所述当前图像帧的匹配度；在 根据图像配准结果确定所述云台的调整控制量时，所述处理器被配置用于调用所述程序指令，执行如下步骤：
    根据图像配准结果从所述关键图像帧集合中确定目标图像帧，所述目标图像帧为所述关键图像帧集合中匹配度最大的关键图像帧；
    计算所述目标图像帧和所述当前图像帧之间的角度差异，并将所述角度差异作为所述云台的调整控制量。
  27. 如权利要求26所述的云台调整设备，其特征在于，在根据图像配准结果从所述关键图像帧集合中确定目标图像帧时，所述处理器被配置用于调用所述程序指令，执行如下步骤：
    根据所述图像配准结果从所述关键图像帧集合中确定最大匹配度所对应的第一关键图像帧，以及次大匹配度所对应的第二关键图像帧；
    计算所述第一关键图像帧与所述第二关键图像帧之间的匹配度比值；
    若所述匹配度比值大于预设比值，则将所述第一关键图像帧确定为目标图像帧。
  28. 如权利要求26所述的云台调整设备，其特征在于，在计算所述目标图像帧和所述当前图像帧之间的角度差异时，所述处理器被配置用于调用所述程序指令，执行如下步骤：
    根据所述目标图像帧和所述当前图像帧建立目标函数，所述目标函数包括图像比例参数和像素偏移参数；
    按照减小所述目标函数的值的原则，优化所述目标函数以确定所述图像比例参数的图像比例值和所述像素偏移参数的像素偏移值；
    获取所述摄像组件的内参以及图像分辨率，并根据所述内参、所述图像分辨率和所述图像位移值计算所述目标图像帧和所述当前图像帧之间的角度差异。
  29. 如权利要求27所述的云台调整设备，其特征在于，优化后的目标函数的值满足最小化条件；
    所述图像比例值为满足所述最小化条件时的目标函数中的图像比例参数的取值，所述像素偏移值为满足所述最小化条件时的目标函数中的像素偏移参数的取值。
  30. 如权利要求19所述的云台调整设备，其特征在于，所述处理器被配置还用于调用所述程序指令，执行如下步骤：
    若检测到图像复拍的开始指令，则确定图像复拍的初始位置，所述初始位置作为所述当前拍摄轨迹的起始位置；
    根据所述初始位置从所述参考图像帧序列中获取基准图像帧，所述基准图像帧所对应的拍摄位置与所述初始位置相对应；
    若所述基准图像帧和在所述初始位置拍摄到的初始图像帧进行图像配准成功，则开始图像复拍。
  31. 如权利要30所述的云台调整设备，其特征在于，所述初始位置根据所述参考拍摄轨迹中的任意拍摄位置确定。
  32. 如权利要求17-31任一项所述的云台调整设备，其特征在于，所述处理器被配置还用于调用所述程序指令，执行如下步骤：
    若接收到调整指令，则根据所述调整指令对所述云台进行调整。
  33. 一种移动平台，其特征在于，包括：
    摄像组件，用于拍摄图像；
    云台，用于控制所述摄像组件的朝向；
    机身；
    动力系统、安装在所述机身上，用于为所述移动平台提供动力；
    以及如权利要求17-32任一项所述的云台调整设备。
  34. 如权利要求33所述的移动平台，其特征在于，所述移动平台包括以下任一种：无人机或者云台摄像机。
PCT/CN2018/116439 2018-11-20 2018-11-20 云台调整方法、云台调整设备、移动平台及介质 Ceased WO2020102976A1 (zh)

Priority Applications (5)

Application Number Priority Date Filing Date Title
CN201880038409.9A CN110800282B (zh) 2018-11-20 2018-11-20 云台调整方法、云台调整设备、移动平台及介质
CN202110768639.4A CN113473025A (zh) 2018-11-20 2018-11-20 云台调整方法、云台调整设备、移动平台及介质
PCT/CN2018/116439 WO2020102976A1 (zh) 2018-11-20 2018-11-20 云台调整方法、云台调整设备、移动平台及介质
US17/302,922 US11892812B1 (en) 2018-11-20 2021-05-14 Gimbal adjustment method, gimbal adjustment device, mobile platform and storage medium
US18/544,120 US20240152108A1 (en) 2018-11-20 2023-12-18 Gimbal adjustment method, gimbal adjustment device, mobile platform and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2018/116439 WO2020102976A1 (zh) 2018-11-20 2018-11-20 云台调整方法、云台调整设备、移动平台及介质

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/302,922 Continuation US11892812B1 (en) 2018-11-20 2021-05-14 Gimbal adjustment method, gimbal adjustment device, mobile platform and storage medium

Publications (1)

Publication Number Publication Date
WO2020102976A1 true WO2020102976A1 (zh) 2020-05-28

Family

ID=69425420

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/116439 Ceased WO2020102976A1 (zh) 2018-11-20 2018-11-20 云台调整方法、云台调整设备、移动平台及介质

Country Status (3)

Country Link
US (2) US11892812B1 (zh)
CN (2) CN110800282B (zh)
WO (1) WO2020102976A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113727022A (zh) * 2021-08-30 2021-11-30 杭州申昊科技股份有限公司 巡检图像的采集方法及装置、电子设备、存储介质
CN114491993A (zh) * 2022-01-11 2022-05-13 深圳快造科技有限公司 数据处理方法、装置、服务器及存储介质
CN117896621A (zh) * 2024-03-14 2024-04-16 深圳市浩瀚卓越科技有限公司 云台的拍摄轨迹记录方法、装置、设备及存储介质

Families Citing this family (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110800282B (zh) 2018-11-20 2021-07-27 深圳市大疆创新科技有限公司 云台调整方法、云台调整设备、移动平台及介质
CN112492215B (zh) * 2020-12-09 2022-04-12 维沃移动通信有限公司 拍摄控制方法、装置和电子设备
CN112866583B (zh) * 2020-12-30 2022-06-21 深圳追一科技有限公司 数据采集系统、方法、装置、电子设备及存储介质
CN117044218A (zh) * 2021-03-25 2023-11-10 深圳市大疆创新科技有限公司 可移动平台及其组件的控制方法、装置
CN113411547B (zh) * 2021-06-17 2023-06-16 浙江大华技术股份有限公司 一种云台的位置校正方法及装置
CN116724561A (zh) * 2021-06-30 2023-09-08 深圳市大疆创新科技有限公司 云台系统处理方法、装置及云台系统
CN117422761A (zh) * 2022-07-11 2024-01-19 浙江宇视科技有限公司 一种设备移动检测方法、装置及存储介质
CN115026840B (zh) * 2022-08-10 2022-12-16 艾斯特国际安全技术（深圳）有限公司 自动校准方法、机械手及计算机可读存储介质
CN116668814A (zh) * 2023-06-02 2023-08-29 重庆大学 一种用于环境试验的旋转校正拍照方法、装置及介质
CN118843004A (zh) * 2024-06-20 2024-10-25 北京积加科技有限公司 装置调整方法、装置、电子设备和计算机可读介质
CN119376433A (zh) * 2024-10-23 2025-01-28 普联技术有限公司 云台偏差确定方法、装置、程序产品及云台摄像设备
CN120812391B (zh) * 2025-08-01 2026-03-20 大连交通大学 基于几何算法的云台与拍摄物动态平行调整系统

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102122172A (zh) * 2010-12-31 2011-07-13 中国科学院计算技术研究所 机器运动控制的摄像系统及其控制方法
US20120162475A1 (en) * 2010-12-23 2012-06-28 Lin Chincheng Method and apparatus for raster output of rotated interpolated pixels optimized for digital image stabilization
CN105554394A (zh) * 2016-01-27 2016-05-04 努比亚技术有限公司 一种拍摄方法及装置
CN107370941A (zh) * 2017-06-29 2017-11-21 联想(北京)有限公司 一种信息处理方法及电子设备
CN108111752A (zh) * 2017-12-12 2018-06-01 北京达佳互联信息技术有限公司 视频拍摄方法、装置及移动终端

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101860732B (zh) * 2010-06-04 2014-08-27 天津市亚安科技股份有限公司 一种控制云台摄像机自动跟踪目标的方法
US8903568B1 (en) * 2013-07-31 2014-12-02 SZ DJI Technology Co., Ltd Remote control method and terminal
US9854168B2 (en) * 2014-03-07 2017-12-26 Futurewei Technologies, Inc. One-pass video stabilization
KR101515847B1 (ko) * 2014-08-06 2015-05-04 스타십벤딩머신 주식회사 Nui 장치에 의해 수집된 동작정보의 보정 방법 및 장치
US20160292865A1 (en) * 2015-04-02 2016-10-06 Sportvision, Inc. Automated framing and selective discard of parts of high resolution videos of large event space
WO2018032436A1 (zh) * 2016-08-17 2018-02-22 深圳市大疆灵眸科技有限公司 云台控制方法、装置、存储介质以及无人机
WO2018098704A1 (zh) * 2016-11-30 2018-06-07 深圳市大疆创新科技有限公司 控制方法、设备、系统、无人机和可移动平台
WO2018187916A1 (zh) * 2017-04-10 2018-10-18 深圳市大疆创新科技有限公司 云台随动控制方法及控制设备
CN107807659A (zh) * 2017-10-24 2018-03-16 北京臻迪科技股份有限公司 一种无人机飞行控制方法及装置
WO2019084719A1 (zh) * 2017-10-30 2019-05-09 深圳市大疆创新科技有限公司 图像处理方法及无人机
CN110800282B (zh) * 2018-11-20 2021-07-27 深圳市大疆创新科技有限公司 云台调整方法、云台调整设备、移动平台及介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120162475A1 (en) * 2010-12-23 2012-06-28 Lin Chincheng Method and apparatus for raster output of rotated interpolated pixels optimized for digital image stabilization
CN102122172A (zh) * 2010-12-31 2011-07-13 中国科学院计算技术研究所 机器运动控制的摄像系统及其控制方法
CN105554394A (zh) * 2016-01-27 2016-05-04 努比亚技术有限公司 一种拍摄方法及装置
CN107370941A (zh) * 2017-06-29 2017-11-21 联想(北京)有限公司 一种信息处理方法及电子设备
CN108111752A (zh) * 2017-12-12 2018-06-01 北京达佳互联信息技术有限公司 视频拍摄方法、装置及移动终端

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113727022A (zh) * 2021-08-30 2021-11-30 杭州申昊科技股份有限公司 巡检图像的采集方法及装置、电子设备、存储介质
CN114491993A (zh) * 2022-01-11 2022-05-13 深圳快造科技有限公司 数据处理方法、装置、服务器及存储介质
CN117896621A (zh) * 2024-03-14 2024-04-16 深圳市浩瀚卓越科技有限公司 云台的拍摄轨迹记录方法、装置、设备及存储介质
CN117896621B (zh) * 2024-03-14 2024-05-14 深圳市浩瀚卓越科技有限公司 云台的拍摄轨迹记录方法、装置、设备及存储介质

Also Published As

Publication number Publication date
US20240152108A1 (en) 2024-05-09
CN110800282A (zh) 2020-02-14
CN110800282B (zh) 2021-07-27
US11892812B1 (en) 2024-02-06
CN113473025A (zh) 2021-10-01

Similar Documents

Publication Publication Date Title
CN110800282B (zh) 云台调整方法、云台调整设备、移动平台及介质
US11012614B2 (en) Image processing device, image processing method, and program
CN110969097B (zh) 监控目标联动跟踪控制方法、设备及存储装置
US9648229B2 (en) Image processing device and associated methodology for determining a main subject in an image
CN101662587B (zh) 摄像装置及其跟踪方法
CN107223330B (zh) 一种深度信息获取方法、装置及图像采集设备
US9558543B2 (en) Image fusion method and image processing apparatus
CN111932636A (zh) 双目摄像头的标定及图像矫正方法、装置、存储介质、终端、智能设备
JP6694281B2 (ja) ステレオカメラおよび撮像システム
CN108307118B (zh) 一种基于惯导参数流形优化的低延时视频稳像方法
CN110611767B (zh) 图像处理方法、装置和电子设备
CN109922275B (zh) 曝光参数的自适应调整方法、装置及一种拍摄设备
CN104243796B (zh) 摄影装置、摄影方法、模板创建装置和模板创建方法
US10887525B2 (en) Delivery of notifications for feedback over visual quality of images
JP2015012482A (ja) 画像処理装置及び画像処理方法
WO2017128750A1 (zh) 一种图像采集方法和图像采集装置
CN114072837A (zh) 红外图像处理方法、装置、设备及存储介质
CN110337668B (zh) 图像增稳方法和装置
US20140022338A1 (en) Method for Producing a Panoramic Image on the Basis of a Video Sequence and Implementation Apparatus
CN113391644A (zh) 一种基于图像信息熵的无人机拍摄距离半自动寻优方法
CN110602377B (zh) 一种视频稳像方法及装置
WO2025195277A1 (zh) 图像处理方法及其装置
JP2010245658A (ja) 画像処理装置、それを備えたデジタルカメラ及び画像処理プログラム
CN111279352A (zh) 通过投球练习的三维信息获取系统及摄像头参数算出方法
JP2011135380A (ja) 撮像装置、画像共有方法および画像共有プログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18940958

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18940958

Country of ref document: EP

Kind code of ref document: A1