WO2020199198A1 - 图像采集控制方法，图像采集控制设备和可移动平台 - Google Patents

图像采集控制方法，图像采集控制设备和可移动平台 Download PDF

Info

Publication number
WO2020199198A1
WO2020199198A1 PCT/CN2019/081518 CN2019081518W WO2020199198A1 WO 2020199198 A1 WO2020199198 A1 WO 2020199198A1 CN 2019081518 W CN2019081518 W CN 2019081518W WO 2020199198 A1 WO2020199198 A1 WO 2020199198A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
reference image
salient
reference images
images
Prior art date
Application number
PCT/CN2019/081518
Other languages
English (en)
French (fr)
Inventor
邹文
胡攀
Original Assignee
深圳市大疆创新科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市大疆创新科技有限公司 filed Critical 深圳市大疆创新科技有限公司
Priority to CN201980008880.8A priority Critical patent/CN111656763B/zh
Priority to PCT/CN2019/081518 priority patent/WO2020199198A1/zh
Publication of WO2020199198A1 publication Critical patent/WO2020199198A1/zh
Priority to US17/317,887 priority patent/US20210266456A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/10—Image acquisition
    • G06V10/17—Image acquisition using hand-held instruments
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/61—Control of cameras or camera modules based on recognised objects
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/61—Control of cameras or camera modules based on recognised objects
    • H04N23/611—Control of cameras or camera modules based on recognised objects where the recognised objects include parts of the human body
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/64—Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/695—Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/80—Camera processing pipelines; Components thereof
    • H04N23/81—Camera processing pipelines; Components thereof for suppressing or minimising disturbance in the image signal generation

Definitions

  • the present disclosure relates to the field of image acquisition, and in particular to an image acquisition control method, an image acquisition control device and a movable platform.
  • the shooting process of most cameras requires users to manually complete.
  • Some cameras can provide assistance to users, but the assistance provided is limited to displaying horizontal lines and displaying very basic information such as face position frames.
  • users still need to manually operate according to their own aesthetics to determine the appropriate framing to complete the shooting.
  • the present disclosure provides an image acquisition control method, an image acquisition control device and a movable platform, which can realize the automatic shooting of the image acquisition device while ensuring that the images obtained by the automatic shooting meet the aesthetic needs of users.
  • an image acquisition control method includes:
  • the posture of the image collection device when the target image is collected is set.
  • an image acquisition control device including a memory and a processor
  • the memory is used to store program codes
  • the processor calls the program code, and when the program code is executed, is used to perform the following operations:
  • the posture of the image collection device when the target image is collected is set.
  • a movable platform including:
  • Image acquisition equipment for acquiring images
  • the image acquisition control device can automatically select the target image from multiple reference images, and then can automatically adjust the posture according to the posture when the target image is collected, so as to capture the user’s aesthetic needs.
  • Image While realizing the automatic shooting of the image acquisition device, it can also ensure that the automatically captured images meet the user's aesthetic requirements, in which there is no need for the user to manually adjust the posture, which is conducive to achieving a higher degree of automated shooting.
  • Fig. 1 is a schematic flowchart showing an image capture control method according to an embodiment of the present disclosure.
  • Fig. 2 is a schematic flowchart of performing saliency detection on each reference image to determine the salient area in each reference image according to an embodiment of the present disclosure.
  • Fig. 3 is a schematic flowchart of determining the evaluation parameter of each reference image according to an embodiment of the present disclosure.
  • Fig. 4 is a schematic flowchart of determining evaluation parameters of a salient area with respect to each composition rule according to an embodiment of the present disclosure.
  • Fig. 5 is a schematic flow chart showing another method of determining evaluation parameters of a salient area relative to each of the composition rules according to an embodiment of the present disclosure.
  • Fig. 6 is a schematic flow chart of another image capture control method according to an embodiment of the present disclosure.
  • Fig. 7 is a schematic flow chart showing a method for eliminating errors caused by lens distortion and jelly effect of the image acquisition device with respect to the reference image according to an embodiment of the present disclosure.
  • Fig. 8 is a schematic flow chart showing yet another image capture control method according to an embodiment of the present disclosure.
  • Fig. 9 is a schematic diagram showing an image capture control device according to an embodiment of the present disclosure.
  • Fig. 10 is a schematic structural diagram of a movable platform according to an embodiment of the present disclosure.
  • the embodiment of the present invention provides a movable platform, which includes a body, an image acquisition device, and an image acquisition control device.
  • image acquisition equipment can be used to acquire images.
  • the image acquisition control device can acquire multiple reference images collected by the image acquisition device in the process of changing the posture of the image acquisition device; perform a saliency detection on each reference image to determine the location of each reference image.
  • the salient area in the reference image; according to the salient area in each reference image and a preset composition rule, the evaluation parameter of each reference image is determined; the evaluation parameter is determined in the multiple reference images according to the evaluation parameter Target image; according to the posture of the image capture device when the target image is captured, the posture of the image capture device when the image is captured is set.
  • the image acquisition control device can automatically select the target image from multiple reference images, and then can automatically adjust the posture of the image acquisition device according to the posture when the target image is acquired, so as to collect an image that meets the user's aesthetic needs. While realizing the automatic shooting of the image acquisition device, it can also ensure that the automatically captured images meet the user's aesthetic requirements, in which there is no need for the user to manually adjust the posture, which is conducive to achieving a higher degree of automated shooting.
  • the movable platform further includes a communication device that can be used to provide a communication connection between the movable platform and an external device.
  • the communication connection can be a wired communication connection or a wireless communication connection
  • the external device can be a remote control or Mobile phones, tablets, wearable devices and other terminals.
  • the movable platform is one of unmanned aerial vehicles, unmanned vehicles, handheld devices, and mobile robots.
  • Fig. 1 is a schematic flowchart showing an image capture control method according to an embodiment of the present disclosure. As shown in Fig. 1, the image acquisition control method may include the following steps:
  • Step S0 in the process of changing the posture of the image acquisition device, acquire multiple reference images collected by the image acquisition device;
  • Step S1 performing saliency detection on each of the reference images to determine the salient areas in each of the reference images
  • Step S2 determining the evaluation parameters of each reference image according to the salient area in each reference image and a preset composition rule
  • Step S3 Determine a target image among the multiple reference images according to the evaluation parameter.
  • Step S4 Set the posture of the image acquisition device when the image is collected according to the posture of the image acquisition device when the target image is collected.
  • the image capture device may first be directed toward the target area, and the target area may be an area set by the user or an area automatically generated by the image capture control device. Then the posture of the image capture device can be adjusted. For example, one or more posture angles of the image capture device can be adjusted within the preset angle range (including roll angle, translation angle and pitch angle), and also within the preset distance range Adjust the position of the image capture device in one or more directions, so that the image capture device changes its posture.
  • the preset angle range including roll angle, translation angle and pitch angle
  • the reference image can be obtained, for example, each time the posture is changed, the reference image is obtained once, so that the image acquisition device can obtain multiple reference images, and then perform saliency detection on the reference image to determine the salientity in the reference image area.
  • the operation of changing the posture of the image acquisition device may be performed manually by the user, or may be performed automatically by the image acquisition device.
  • a reference image can be obtained, where the reference image refers to the image captured by the image capture device before the shutter is pressed, and the reference image and the image captured by the image capture device after the shutter is pressed exist in many ways The difference, for example, the processing fineness of the image acquisition device for the two is different, for example, the resolution of the two is different.
  • the reference image can be provided to the user for preview.
  • the saliency detection specifically refers to the visual saliency detection (Visual Saliency Detection).
  • the saliency detection can simulate the visual characteristics of people through intelligent algorithms, and extract the regions of human interest in the reference image as the salient regions. A significant area can be determined in a reference image, or multiple display areas can be determined, depending on the actual situation.
  • the evaluation parameter of the salient area in the reference image relative to the preset composition rule is determined. Based on the evaluation parameter, the reference image can be determined for humans. In terms of whether it meets the aesthetic needs of human beings.
  • the evaluation parameter may be a numerical value, and the numerical value may be displayed in association with the reference image, for example, displayed in the reference image for the user's reference, and specifically may be displayed in the reference image as a score.
  • the posture of the image capturing device when capturing images can be set.
  • the evaluation parameter can represent the aesthetic needs of humans, and then according to the evaluation parameters, an image that meets the aesthetic needs of humans can be determined from multiple reference images as the target image, and then the target image can be acquired according to the image acquisition device Set the posture when the image acquisition device collects the image, for example, set the posture when the image acquisition device collects the image to the posture when the image is collected, so as to ensure that the collected image can meet the aesthetic needs of human beings.
  • the target image may be one reference image or multiple reference images.
  • the evaluation parameter being a numerical value as an example, the reference image with the largest numerical value can be selected as the target image, or the reference image with the numerical value greater than the first preset value can be selected as the target image.
  • the posture of the image capture device when capturing images can also be adjusted to a posture that has a specific relationship (for example, symmetry, rotation) with the posture of the image capture device when capturing the target image, so that the captured image Meet specific needs.
  • the image capture device can automatically adjust the posture of the image capture device when capturing images according to the evaluation parameters of the salient area in each reference image relative to the preset composition rule, so as to capture images that meet the user's aesthetic needs. While realizing the automatic shooting of the image acquisition device, it can also ensure that the images obtained by the automatic shooting meet the user's aesthetic requirements, and there is no need for the user to manually adjust the posture, which is conducive to achieving a higher degree of automatic shooting.
  • Fig. 2 is a schematic flowchart of performing saliency detection on each reference image to determine the salient area in each reference image according to an embodiment of the present disclosure. As shown in FIG. 2, the performing saliency detection on each of the reference images to determine the salient areas in each of the reference images includes:
  • Step S11 Perform Fourier transform on the reference image
  • Step S12 acquiring the phase spectrum of the reference image according to the first result of the Fourier transform
  • Step S13 Gaussian filtering is performed on the second result of the inverse Fourier transform of the phase spectrum to determine a salient area in the reference image.
  • the image evaluation parameters of the pixels located at the coordinates (x, y) in the reference image can be determined, such as the pixel value, denoted as I(x, y), and then Fourier is performed for each pixel in the reference image.
  • the calculation formula is as follows:
  • the phase spectrum p(x,y) of the reference image can be obtained, and the calculation formula is as follows:
  • Gaussian filtering is performed on the second result of the inverse Fourier transform of the phase spectrum, where the exponential expression e i ⁇ p(x,y) of e can be constructed first with p(x,y) as a power , Gaussian filtering is performed on the inverse Fourier transform result of the exponential expression to obtain the saliency evaluation parameter sM(x,y) of each pixel in the reference image.
  • the calculation formula is as follows:
  • the saliency evaluation parameter of the pixel Based on the saliency evaluation parameter of the pixel, it can be determined whether the pixel belongs to the salient area. For example, if the saliency evaluation parameter is a saliency value, then the saliency value can be compared with the second preset value, and the saliency value is greater than the second preset value Pixels are classified into salient areas, so as to determine salient areas.
  • the steps in the embodiment shown in FIG. 2 are only an implementation method for determining the salient area.
  • the method for determining the display area in the present disclosure includes but is not limited to the steps in the embodiment shown in FIG.
  • the salient area can be determined according to the LC algorithm, or the salient area can be determined according to the HC algorithm, or the salient area can be determined by the AC algorithm, or the salient area can be determined according to the FT algorithm.
  • the saliency detection may include the detection of human faces or the detection of objects, and the specific method is selected according to needs.
  • Fig. 3 is a schematic flowchart of determining evaluation parameters of a salient area relative to a preset composition rule for each reference image according to an embodiment of the present disclosure.
  • the preset composition rule includes at least one composition rule
  • the determination of the evaluation parameter of the salient area relative to the preset composition rule for each reference image includes:
  • Step S21 determining the evaluation parameters of the salient regions in each reference image relative to each of the composition rules
  • Step S22 Perform a weighted summation of the evaluation parameters corresponding to each of the composition rules to determine the evaluation parameters of the salient regions relative to the preset composition rules.
  • the aesthetic angle to which each composition rule conforms is different.
  • This embodiment performs a weighted summation of the evaluation parameters corresponding to each composition rule to determine the evaluation parameters of the salient area relative to the preset composition rule.
  • the aesthetic angle of each composition rule can be comprehensively considered to obtain the evaluation parameters of the salient area relative to the preset composition rule, and then the target image can be determined according to the obtained evaluation parameters, so that the determined target image can satisfy multiple aesthetic angles It is required that even if different users have different aesthetic angles, the target image can meet the aesthetic needs of different users.
  • composition rule includes at least one of the following:
  • Fig. 4 is a schematic flowchart of determining evaluation parameters of a salient area with respect to each composition rule according to an embodiment of the present disclosure.
  • the composition rule includes a rule of thirds, and the evaluation parameters for determining the salient area relative to each composition rule include:
  • Step S211 Calculate the shortest distance among the distances from the coordinates of the center of the salient area to the intersection of the four bisectors in the reference image;
  • Step S212 Calculate the evaluation parameters of the salient area relative to the rule of thirds according to the coordinates of the centroid of the salient area and the shortest distance.
  • the rule of thirds divides the reference image into 9 blocks by two thirds along the length of the reference image and two thirds along the width of the reference image.
  • the bisectors intersect to form 4 intersections.
  • the composition of the salient area in the reference image conforms to the rule of thirds, and the salient area in the reference image conforms to the rule of thirds For example, the closer the salient area is to the intersection, the larger the evaluation parameter of the salient area relative to the rule of thirds.
  • the evaluation parameter S RT of the salient region relative to the rule of thirds can be calculated by the following formula:
  • G j represents the j-th intersection point
  • C (S i) represents the coordinates of the center of the reference image in the i-th significant region S i is, d M (C (S i ), G j) represents a significant area of the i-th reference image
  • D(S i ) is the shortest distance in d M (C(S i ),G j )
  • M(S i ) represents the i-th salient area S in the reference image
  • the coordinates of the centroid of i , ⁇ 1 is the variance control factor, which can be set as needed.
  • the relationship between all the display areas in the reference image and the intersection of the bisecting line as a whole is considered.
  • the evaluation parameter S RT of the salient area relative to the rule of thirds. The closer the intersection of all the display areas in the reference image with the bisecting line as a whole, the larger the S RT . Accordingly, all display areas in the reference image region farther whole the intersection of the bisector, the smaller S RT.
  • Fig. 5 is a schematic flow chart showing another method of determining evaluation parameters of a salient area relative to each of the composition rules according to an embodiment of the present disclosure.
  • the composition rule includes a subject visual balance method
  • the evaluation parameter for determining the salient area relative to each composition rule includes:
  • Step S213 Calculate the normalized Manhattan distance based on the coordinates of the center of the reference image and the coordinates of the center of the salient area and the coordinates of the center of mass;
  • Step S214 Calculate the evaluation parameter of the salient area relative to the subject visual balance method according to the normalized Manhattan distance.
  • the composition of the salient area in the reference image conforms to the subject's visual balance method, and the salient area in the reference image is more in line with the subject
  • the visual balance method for example, the more even the content in the salient area is distributed around the center point of the reference image, the larger the evaluation parameter of the salient area relative to the subject's visual balance method.
  • the evaluation parameter S VB of the salient area relative to the subject's visual balance method can be calculated by the following formula:
  • C represents the coordinates of the reference image center
  • C (S i) represents the coordinates of the center of the reference image in the i-th significant region S i is
  • M (S i) represents the coordinates of the centroid of the reference image in the i-th significant region S i
  • d M represents the calculation of the normalized Manhattan distance
  • ⁇ 2 is the variance control factor, which can be set as required.
  • the coordinates of the center of all salient areas as a whole can be in the reference image, and then according to the overall center of all salient areas and the reference
  • the relationship between the image center can determine the distribution of all salient areas relative to the center of the reference image, and then determine the salient area relative to the evaluation parameter S VB of the subject visual balance method.
  • the preset composition rule includes two composition rules: the rule of thirds and the subject visual balance method, then when the salient area is determined relative to the evaluation parameter S RT of the rule of thirds, and the salient area is determined relative after the evaluation method body visual balance parameter S VB, may be weighted and summed S RT S VB, significantly evaluation parameter S a region with respect to the default rules of composition;
  • ⁇ RT is the weight of S RT
  • ⁇ VB is the weight of S VB .
  • the user may set evaluation parameter S RT weights corresponding to the weight of the rule of thirds, and the body weight evaluation parameter S VB visual balance weight corresponding method, in order to meet the aesthetic needs of users.
  • Fig. 6 is a schematic flow chart of another image capture control method according to an embodiment of the present disclosure. As shown in FIG. 6, before performing saliency detection on each of the reference images, the method further includes:
  • Step S5 Eliminate errors caused by lens distortion and jelly effect of the image acquisition device for the reference image.
  • the lens of the image acquisition device (such as a fisheye lens) acquires a reference image
  • the salient area is mainly the area containing the object.
  • the shutter of the image capture device is a rolling shutter
  • the image capture device when the image capture device is acquiring a reference image, when the object in the reference image moves or vibrates rapidly relative to the image capture device, the reference image is
  • the content may have problems such as tilt, partial exposure, and afterimages. This problem is the jelly effect, which will also cause some objects in the reference image to be different from the objects in the actual scene (such as differences in shape), which is not conducive to accurate accuracy. Identify significant areas.
  • the error caused by the lens distortion and the jelly effect of the image acquisition device is eliminated for the reference images, so that the saliency area can be accurately determined subsequently.
  • Fig. 7 is a schematic flow chart showing a method for eliminating errors caused by lens distortion and jelly effect of the image acquisition device for the reference image according to an embodiment of the present disclosure. As shown in FIG. 7, the elimination of the error caused by the lens distortion and jelly effect of the image acquisition device with respect to the reference image includes:
  • Step S51 Perform line-to-line synchronization between the vertical synchronization signal count value of the reference image and the data of the reference image to determine the motion information of each line of data in the reference image during the exposure process;
  • Step S52 generating a grid on the reference image through backward mapping or forward mapping
  • Step S53 calculating the motion information by an iterative method to determine the offset of the coordinates at the intersection of the grid during the exposure process
  • Step S54 Dewarp the reference image according to the offset to eliminate the error.
  • the difference between the object in the reference image and the object in the actual scene caused by nonlinear distortion is mainly in the lens radial direction and the lens tangential direction.
  • the object in the reference image caused by the jelly effect is different from the actual scene.
  • the difference between the objects in the scene is mainly in the row direction of the photoelectric sensor array in the image acquisition device (the photoelectric sensor array adopts a progressive scan method for exposure).
  • the vertical synchronization signal count value of the reference image and the data of the reference image are synchronized between lines to determine the motion evaluation parameters of each line of data in the reference image during the exposure process, and then through backward mapping or Forward mapping generates a grid on the reference image, and calculates the motion information through an iterative method.
  • the offset of the coordinates at the intersection of the grid during the exposure process can be determined, and the grid In the reference image represented, the offset of the coordinates at the intersection of the grid during the exposure process.
  • This offset can represent the offset of the object at the corresponding position relative to the object in the actual scene during the exposure process.
  • the offset can be de-distorted to eliminate errors caused by lens distortion and jelly effect.
  • Fig. 8 is a schematic flow chart showing yet another image capture control method according to an embodiment of the present disclosure.
  • the setting the posture of the image acquisition device when acquiring the image according to the posture when the image acquisition device acquires the target image includes:
  • Step S41 According to the posture of the image acquisition device when the target image is acquired, the posture of the image acquisition device when the image is acquired is set through the pan/tilt.
  • the posture of the image capturing device when capturing images can be set through a pan-tilt.
  • the target image may be determined in the multiple reference images according to the evaluation parameters; according to the target posture when the image acquisition device acquires the target image, the image acquisition device is set to acquire the The posture of the image.
  • the PTZ includes at least one of the following:
  • the stabilization method of the PTZ includes at least one of the following:
  • the present disclosure also proposes an embodiment of the image capture control device.
  • the image acquisition control device proposed in the embodiment of the present disclosure includes a memory 901 and a processor 902;
  • the memory 901 is used to store program codes
  • the processor 902 calls the program code, and when the program code is executed, is configured to perform the following operations:
  • the posture of the image collection device when the target image is collected is set.
  • the processor 902 is configured to:
  • Gaussian filtering is performed on the second result of the inverse Fourier transform of the phase spectrum to determine a salient area in the reference image.
  • the preset composition rule includes at least one composition rule
  • the processor 902 is configured to:
  • a weighted summation is performed on the evaluation parameters corresponding to each of the composition rules to determine the evaluation parameters of the salient area relative to the preset composition rules.
  • composition rule includes at least one of the following:
  • composition rule includes a rule of thirds
  • processor 902 is configured to:
  • the evaluation parameter of the salient area relative to the rule of thirds is calculated according to the coordinates of the centroid of the salient area and the shortest distance.
  • the composition rule includes a subject visual balance method
  • the processor 902 is configured to:
  • the evaluation parameter of the salient area relative to the subject visual balance method is calculated according to the normalized Manhattan distance.
  • the processor 902 is configured to:
  • the processor 902 is configured to:
  • it further includes a pan-tilt, and the processor 902 is configured to:
  • the posture of the image acquisition device when collecting images is set through the pan/tilt.
  • the PTZ includes at least one of the following:
  • the stabilization method of the PTZ includes at least one of the following:
  • the embodiments of the present disclosure also provide a movable platform, which is characterized in that it includes:
  • Image acquisition equipment for acquiring images
  • Fig. 10 is a schematic structural diagram of a movable platform according to an embodiment of the present disclosure.
  • the movable platform is a handheld camera.
  • the handheld camera includes a lens 101, a three-axis pan/tilt and an inertial measurement unit (IMU) 102, where the three axes are respectively the pitch axis 103 , The roll axis 104 and the yaw axis 105, the three-axis pan/tilt is connected to the lens 101, where the pitch axis is used to adjust the pitch angle of the lens, the roll axis is used to adjust the roll angle of the lens, and the yaw axis is used to adjust the lens The yaw angle.
  • IMU inertial measurement unit
  • the inertial measurement unit 102 is set behind the lens 101, and the pins of the inertial measurement unit 102 can be connected to the vertical synchronization pin of the photoelectric sensor to sample the posture of the photoelectric sensor.
  • the sampling frequency can be set as required, for example, it can be set 8KHz, so that the sampling can record the posture and motion information of the lens 101 when acquiring the reference image, and it can also deduct the motion information of each row of pixels in the reference image according to the vertical synchronization signal, for example, according to step S51 in the embodiment shown in FIG. 7 Determine the motion information, thereby de-distorting the reference image.
  • the systems, devices, modules, or units illustrated in the above embodiments may be specifically implemented by computer chips or entities, or implemented by products with certain functions.
  • the functions are divided into various units and described separately.
  • the functions of each unit can be implemented in the same one or more software and/or hardware.
  • the embodiments of the present disclosure may be provided as methods, systems, or computer program products. Therefore, the present disclosure may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware.
  • the present disclosure may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Studio Devices (AREA)

Abstract

本公开提出一种图像采集控制方法，包括：在改变所述图像采集设备的姿态的过程中，获取所述图像采集设备采集到的多张参考图像；对每张所述参考图像分别进行显著性检测，以确定每张所述参考图像中的显著区域；根据每张所述参考图像中的显著区域和预设构图法则，确定每张所述参考图像的评价参数；根据所述评价参数在所述多张参考图像中确定目标图像；根据所述图像采集设备采集所述目标图像时的姿态，设置所述图像采集设备采集图像时的姿态。本发明实施例在实现图像采集设备自动拍摄的同时，还可以保证自动拍摄得到的图像满足用户的审美需求。

Description

图像采集控制方法，图像采集控制设备和可移动平台 技术领域
本公开涉及图像采集领域，尤其涉及图像采集控制方法，图像采集控制设备和可移动平台。
背景技术
目前大多数的相机的拍摄过程，需要用户人工完成。一些相机能够为用户提供辅助，但是提供的辅助仅限于显示水平线，显示人脸位置框等非常基础的信息，最终还是需要用户人工操作根据自己的审美来确定合适的取景来完成拍摄。
虽然有的相机能够进行自动拍摄，但是并不会考虑取景的美学效果，最终拍摄得到的照片往往无法满足用户对于审美的需求。
发明内容
本公开提供图像采集控制方法，图像采集控制设备和可移动平台，可在实现图像采集设备自动拍摄的同时，保证自动拍摄得到的图像满足用户的审美需求。
根据本公开实施例的第一方面，提出一种图像采集控制方法，所述方法包括：
在改变所述图像采集设备的姿态的过程中，获取所述图像采集设备采集到的多张参考图像；
对每张所述参考图像分别进行显著性检测，以确定每张所述参考图像中的显著区域；
根据每张所述参考图像中的显著区域和预设构图法则，确定每张所述参 考图像的评价参数；
根据所述评价参数在所述多张参考图像中确定目标图像；
根据所述图像采集设备采集所述目标图像时的姿态，设置所述图像采集设备采集图像时的姿态。
根据本公开实施例的第二方面，提出一种图像采集控制设备，包括存储器、处理器；
所述存储器用于存储程序代码；
所述处理器，调用所述程序代码，当程序代码被执行时，用于执行以下操作：
在改变所述图像采集设备的姿态的过程中，获取所述图像采集设备采集到的多张参考图像；
对每张所述参考图像分别进行显著性检测，以确定每张所述参考图像中的显著区域；
根据每张所述参考图像中的显著区域和预设构图法则，确定每张所述参考图像的评价参数；
根据所述评价参数在所述多张参考图像中确定目标图像；
根据所述图像采集设备采集所述目标图像时的姿态，设置所述图像采集设备采集图像时的姿态。
根据本公开实施例的第三方面，提出一种可移动平台，包括：
机身；
图像采集设备，用于采集图像；
以及上述实施例所述的图像采集控制设备。
由以上本公开实施例提供的技术方案可见，图像采集控制设备能够自动从多张参考图像中挑选出目标图像，进而可以根据采集目标图像时的姿态自动调整姿态，以便采集到满足用户对于审美需求的图像。在实现图像采集设备自动拍摄的同时，还可以保证自动拍摄得到的图像满足用户对于审美需求，其中无需用户进行手动调整姿态，有利于实现更高程度的自动化拍摄。
附图说明
为了更清楚地说明本公开实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本公开的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。
图1是根据本公开的实施例示出的一种图像采集控制方法的示意流程图。
图2是根据本公开的实施例示出的一种对每张所述参考图像分别进行显著性检测，以确定每张所述参考图像中的显著区域的示意流程图。
图3是根据本公开的实施例示出的一种确定每张所述参考图像的评价参数的示意流程图。
图4是根据本公开的实施例示出的一种确定显著区域相对于每种所述构图法则的评价参数的示意流程图。
图5是根据本公开的实施例示出的另一种确定显著区域相对于每种所述构图法则的评价参数的示意流程图。
图6是根据本公开的实施例示出的另一种图像采集控制方法的示意流程图。
图7是根据本公开的实施例示出的一种针对所述参考图像消除所述图像采集设备的镜头畸变和果冻效应引发的误差的示意流程图。
图8是根据本公开的实施例示出的又一种图像采集控制方法的示意流程图。
图9是根据本公开的实施例示出的一种图像采集控制设备的示意图。
图10是根据本公开的实施例示出的一种可移动平台的示意结构图。
具体实施方式
下面将结合本公开实施例中的附图，对本公开实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本公开一部分实施例，而 不是全部的实施例。基于本公开中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本公开保护的范围。另外，在不冲突的情况下，下述的实施例及实施例中的特征可以相互组合。
本发明实施例提供了一种可移动平台，所述可移动平台包括：机身、图像采集设备和图像采集控制设备。其中，图像采集设备可用于采集图像。图像采集控制设备可在改变所述图像采集设备的姿态的过程中，获取所述图像采集设备采集到的多张参考图像；对每张所述参考图像分别进行显著性检测，以确定每张所述参考图像中的显著区域；根据每张所述参考图像中的显著区域和预设构图法则，确定每张所述参考图像的评价参数；根据所述评价参数在所述多张参考图像中确定目标图像；根据所述图像采集设备采集所述目标图像时的姿态，设置所述图像采集设备采集图像时的姿态。
由此，图像采集控制设备能够自动从多张参考图像中挑选出目标图像，进而可以根据采集目标图像时的姿态自动调整图像采集设备的姿态，以便采集到满足用户对于审美需求的图像。在实现图像采集设备自动拍摄的同时，还可以保证自动拍摄得到的图像满足用户对于审美需求，其中无需用户进行手动调整姿态，有利于实现更高程度的自动化拍摄。
可选的，可移动平台还包括通信装置，所述通信装置可用于提供可移动平台与外部设备的通信连接，该通信连接可为有线通信连接或无线通信连接，该外部设备可为遥控器或手机，平板电脑，可穿戴设备等终端。
可选的，可移动平台为无人飞行器，无人车，手持设备，移动机器人中的一种。
图1是根据本公开的实施例示出的一种图像采集控制方法的示意流程图。如图1所示，所述图像采集控制方法可以包括以下步骤：
步骤S0，在改变所述图像采集设备的姿态的过程中，获取所述图像采集设备采集到的多张参考图像；
步骤S1，对每张所述参考图像分别进行显著性检测，以确定每张所述参考图像中的显著区域；
步骤S2，根据每张所述参考图像中的显著区域和预设构图法则，确定每张所述参考图像的评价参数；
步骤S3，根据所述评价参数在所述多张参考图像中确定目标图像。
步骤S4，根据所述图像采集设备采集所述目标图像时的姿态，设置所述图像采集设备采集图像时的姿态。
在一个实施例中，可以先将图像采集设备朝着目标区域，所述目标区域可以为用户设定的区域，或是图像采集控制设备自动生成的区域。然后可以调节图像采集设备的姿态，例如可以在预设角度范围内调节图像采集设备的一个或多个姿态角(可以包括横滚角，平移角和俯仰角)，还可以在预设距离范围内调节图像采集设备在一个或多个方向上的位置，从而使得图像采集设备改变姿态。
并且在改变姿态的过程中，可以获取参考图像，例如每改变一次姿态，获取一次参考图像，从而图像采集设备可以获取多张参考图像，然后针对参考图像进行显著性检测来确定参考图像中的显著区域。
其中，改变图像采集设备的姿态的操作可以是用户人工执行的，也可以是图像采集设备自动执行的。
在一个实施例中，可以获取参考图像，其中，参考图像是指在按下快门之前，图像采集设备所采集到的图像，参考图像与按下快门后图像采集设备采集到的图像在多方面存在区别，例如图像采集设备对于两者的处理精细程度不同，例如两者的分辨率不同等。可选的，参考图像可提供给用户进行预览。
显著性检测具体是指视觉显著性检测(Visual Saliency Detection)，显著性检测可以通过智能算法模拟人的视觉特点，提取参考图像中人类感兴趣的区域作为显著区域。在一张参考图像中可以确定出一块显著区域，也可以确定出多块显示区域，具体根据实际情况而定。
由于显著区域是人眼感兴趣的区域，而预设构图法则满足一定的美学标准，所以确定参考图像中显著区域相对于预设构图法则的评价参数，基于该 评价参数就可以确定参考图像对于人类来说是否满足人类的审美需求。其中，所述评价参数可以是数值，所述数值可以与参考图像关联显示，例如显示在参考图像中供用户参考，具体可以作为评分显示在参考图像中。
进而根据所述评价参数，可以设置图像采集设备采集图像时的姿态。
在一个实施例中，所述评价参数可以表征人类的审美需求，那么根据所述评价参数可以在多张参考图像中确定满足人类审美需求的图像作为目标图像，进而可以根据图像采集设备获取目标图像时的姿态，设置图像采集设备采集图像时的姿态，例如将图像采集设备采集图像时的姿态设置为采集图像时的姿态，从而确保采集到的图像能够满足人类的审美需求。
需要说明的是，目标图像可以是一张参考图像，也可以是多张参考图像。以所述评价参数是数值为例，可以选取数值最大的参考图像作为目标图像，也可以选取数值大于第一预设值的参考图像作为目标图像。
在本实施例中，也可以根据需要将图像采集设备采集图像时的姿态调整到为与图像采集设备获取目标图像时的姿态存在特定关系(例如对称，旋转)的姿态，以使采集到的图像满足特定需要。
根据上述实施例，图像采集设备能够根据每张参考图像中显著区域相对于预设构图法则的评价参数自动调整图像采集设备采集图像时的姿态，以便采集到满足用户对于审美需求的图像。在实现图像采集设备自动拍摄的同时，还可以保证自动拍摄得到的图像满足用户对于审美需求，其中无需户进行手动调整姿态，有利于实现更高程度的自动化拍摄。
图2是根据本公开的实施例示出的一种对每张所述参考图像分别进行显著性检测，以确定每张所述参考图像中的显著区域的示意流程图。如图2所示，所述对每张所述参考图像分别进行显著性检测，以确定每张所述参考图像中的显著区域包括：
步骤S11，针对所述参考图像进行傅里叶变换；
步骤S12，根据傅里叶变换的第一结果获取所述参考图像的相位谱；
步骤S13，对所述相位谱的反傅里叶变换的第二结果进行高斯滤波，以确 定所述参考图像中的显著区域。
在一个实施例中，针对参考图像中位于坐标(x,y)的像素可以确定其图像评价参数，例如像素值，记做I(x,y)，然后针对参考图像中每个像素进行傅里叶变换，计算式如下所示：
f(x,y)＝F(I(x,y))；
进而针对傅里叶变换的第一结果f(x,y)，可以获取参考图像的相位谱p(x,y)，计算式如下所示：
p(x,y)＝P(f(x,y))；
然后对对所述相位谱的反傅里叶变换的第二结果进行高斯滤波，其中，可以先以p(x,y)作为幂来构建e的指数表达式e i·p(x,y)，对于该指数表达式的反傅里叶变换结果进行高斯滤波，得到参考图像中每个像素的显著性评价参数sM(x,y)，计算式如下所示：
sM(x,y)＝g(x,y)*||F -1[e i·p(x,y)]|| 2；
基于像素的显著性评价参数，可以确定像素是否属于显著区域，例如显著性评价参数为显著性数值，那么可以比较显著性数值与第二预设值，将显著性数值大于第二预设值的像素归入显著区域，从而确定显著区域。
需要说明的是，图2所示实施例中的步骤，只是确定显著区域的一种实现方式，本公开用于确定显示区域的方式包括但不限于图2所示实施例中的步骤，例如还可以根据LC算法确定显著区域，或者根据HC算法确定显著区域，或者AC算法确定显著区域，或者根据FT算法确定显著区域。显著性检测可以包括对人脸的检测，或者对物体的检测，具体方式根据需要进行选择。
图3是根据本公开的实施例示出的一种针对每张所述参考图像确定显著区域相对于预设构图法则的评价参数的示意流程图。如图3所示，所述预设构图法则包括至少一种构图法则，所述针对每张所述参考图像确定所述显著区域相对于预设构图法则的评价参数包括：
步骤S21，确定每张所述参考图像中的显著区域相对于每种所述构图法则的评价参数；
步骤S22，针对每种所述构图法则对应的评价参数进行加权求和，以确定所述显著区域相对于预设构图法则的评价参数。
在一个实施例中，每种构图法则所符合的审美角度有所不同，本实施例针对每种构图法则对应的评价参数进行加权求和，以确定显著区域相对于预设构图法则的评价参数，可以综合考虑每种构图法则所符合的审美角度得到显著区域相对于预设构图法则的评价参数，进而根据得到的评价参数确定目标图像，使得所确定目标图像能够在多个审美角度上都能够满足要求，即使不同用户的审美角度不尽相同，目标图像也能够满足不同用户的审美需求。
可选地，所述构图法则包括以下至少之一：
三分法、主体视觉平衡法、黄金分割法、中心对称法。
以下以三分法和主体视觉平衡法为例，对本公开的实施例进行示例性说明。
图4是根据本公开的实施例示出的一种确定显著区域相对于每种所述构图法则的评价参数的示意流程图。如图4所示，所述构图法则包括三分法，所述确定所述显著区域相对于每种所述构图法则的评价参数包括：
步骤S211，计算所述显著区域的中心的坐标到所述参考图像中4个等分线的交点的距离中最短的距离；
步骤S212，根据所述显著区域的质心的坐标和所述最短的距离计算所述显著区域相对于所述三分法的评价参数。
在一个实施例中，三分法通过沿着参考图像长度方向的两条三等分线，和沿着参考图像宽度方向的两条三等分线将参考图像划分为9块，其中4条三等分线相交形成4个交点。
若参考图像中的显著区域位于某个交点附近，或者沿着某条等分线分布，那么可以判定显著区域在参考图像中的构图符合三分法，而显著区域在参考图像越符合三分法，例如显著区域距离交点越近，那么显著区域相对三分法 的评价参数就越大。
在一个实施例中，显著区域相对于三分法的评价参数S RT可以通过下式计算：
Figure PCTCN2019081518-appb-000001
其中，
Figure PCTCN2019081518-appb-000002
G j表示第j个交点，C(S i)表示参考图像中第i个显著区域S i的中心的坐标，d M(C(S i),G j)表示第i个参考图像中显著区域的中心的坐标与第j个交点的距离，D(S i)是d M(C(S i),G j)中最短的距离，M(S i)表示参考图像中第i个显著区域S i的质心的坐标，σ 1为方差控制因子，可以根据需要进行设置，参考图像中可以包括n个显著区域，i≤n，加和可以从i＝1加到i＝n。
根据本实施例的计算方式，依据显示区域的中心到等分线的交点的最短距离和显著区域的质心之间的关系，来考量参考图像中所有显示区域整体上与等分线的交点的关系，进而确定出显著区域相对于所述三分法的评价参数S RT，参考图像中所有显示区域整体上与等分线的交点越近，那么S RT越大，相应地，参考图像中所有显示区域整体上与等分线的交点越远，那么S RT越小。
图5是根据本公开的实施例示出的另一种确定显著区域相对于每种所述构图法则的评价参数的示意流程图。如图5所示，所述构图法则包括主体视觉平衡法，所述确定所述显著区域相对于每种所述构图法则的评价参数包括：
步骤S213，基于所述参考图像中心的坐标以及所述显著区域的中心的坐标和质心的坐标，计算归一化曼哈顿距离；
步骤S214，根据所述归一化曼哈顿距离计算所述显著区域相对于所述主体视觉平衡法的评价参数。
在一个实施例中，若参考图像中显著区域中内容均匀分布在参考图像的中心点周围，那么可以判定显著区域在参考图像中的构图符合主体视觉平衡法，而显著区域在参考图像越符合主体视觉平衡法，例如显著区域中内容在 参考图像的中心点周围分布越均匀，那么显著区域相对主体视觉平衡法的评价参数就越大。
在一个实施例中，显著区域相对于主体视觉平衡法的评价参数S VB可以通过下式计算：
Figure PCTCN2019081518-appb-000003
其中，
Figure PCTCN2019081518-appb-000004
C表示参考图像中心的坐标，C(S i)表示参考图像中第i个显著区域S i的中心的坐标，M(S i)表示参考图像中第i个显著区域S i的质心的坐标，d M表示计算归一化曼哈顿距离，σ 2为方差控制因子，可以根据需要进行设置，参考图像中可以包括n个显著区域，i≤n，加和可以从i＝1加到i＝n。
根据本实施例的计算方式，基于全部显著区域的中心的坐标和质心的坐标之间的关系，可以在全部显著区域整体上中心在参考图像中的坐标，进而根据全部显著区域整体上中心与参考图像中心的关系，可以确定全部显著区域相对参考图像中心的分布情况，进而确定出显著区域相对于主体视觉平衡法的评价参数S VB。
在一个实施例中，例如预设构图法则包括三分法和主体视觉平衡法这两种构图法则，那么在确定出显著区域相对于所述三分法的评价参数S RT，以及确定显著区域相对于主体视觉平衡法的评价参数S VB之后，可以对S RT和S VB进行加权求和，得到显著区域相对于预设构图法则的评价参数S A；
Figure PCTCN2019081518-appb-000005
其中，ω RT为S RT的权重，ω VB为S VB的权重。
在一种实施方式中，用户可以预先设置三分法的评价参数S RT对应的权重，以及主体视觉平衡法的评价参数S VB对应的权重，以满足用户的审美需求。
图6是根据本公开的实施例示出的另一种图像采集控制方法的示意流程图。如图6所示，在对每张所述参考图像分别进行显著性检测之前，所述方法还包括：
步骤S5，针对所述参考图像消除所述图像采集设备的镜头畸变和果冻效应引发的误差。
由于图像采集设备的镜头(例如鱼眼镜头)在获取参考图像时，可能会在参考图像的边缘存在非线性畸变效果，导致参考图像中的部分物体与实际场景中物体存在差异(例如形状存在差异)，而显著区域主要是包含物体的区域，在物体与实际场景中物体不同时，不利于准确地确定显著区域。
另外，若图像采集设备的快门为卷帘快门(Rolling shutter)，当图像采集设备在获取参考图像时，参考图像中的物体相对图像采集设备高速运动或者快速振动时，拍摄得到的参考图像中的内容可能会出现倾斜，部分曝光，有残影等问题，这种问题即果冻效应，这也会导致参考图像中的部分物体与实际场景中物体存在差异(例如形状存在差异)，不利于准确地确定显著区域。
本实施例对每张所述参考图像分别进行显著性检测之前，先针对所述参考图像消除所述图像采集设备的镜头畸变和果冻效应引发的误差，以便后续能够准确地确定显著区域。
图7是根据本公开的实施例示出的一种针对所述参考图像消除所述图像采集设备的镜头畸变和果冻效应引发的误差的示意流程图。如图7所示，所述针对所述参考图像消除所述图像采集设备的镜头畸变和果冻效应引发的误差包括：
步骤S51，将所述参考图像的垂直同步信号计数值和所述参考图像的数据进行行间同步，以确定所述参考图像中每一行数据在曝光过程中的运动信息；
步骤S52，通过后向映射或前向映射在所述参考图像上生成网格；
步骤S53，通过迭代法对所述运动信息进行计算，以确定所述网格的交点处的坐标在曝光过程中的偏移量；
步骤S54，根据所述偏移量对所述参考图像进行去畸变(dewarp)，以消 除所述误差。
在一个实施例中，非线性畸变引发的参考图像中的物体与实际场景中物体之间的差异，主要是在镜头径向和镜头切向存在的，果冻效应引发的参考图像中的物体与实际场景中物体之间的差异，主要是在图像采集设备中光电传感器阵列行方向上存在的(光电传感器阵列采用逐行扫描的方式进行曝光)。
而无论哪种差异，本质上都是参考图像中的物体相对于实际场景中物体的偏移，而这种偏移可以等效为物体在曝光过程中的运动，从而可以通过参考图像中的数据在曝光过程中的运动信息得出。
本实施例通过将所述参考图像的垂直同步信号计数值和所述参考图像的数据进行行间同步，以确定参考图像中每一行数据在曝光过程中的运动评价参数，进而通过后向映射或前向映射在所述参考图像上生成网格，通过迭代法对所述运动信息进行计算，可以确定网格的交点处的坐标在曝光过程中的偏移量，据此就得到了以网格表示的参考图像中，网格的交点处的坐标在曝光过程中的偏移量，这个偏移量可以表示对应位置的物体相对实际场景中的物体在曝光过程中的偏移量，从而根据该偏移量可以去畸变，来消除镜头畸变和果冻效应引发的误差。
图8是根据本公开的实施例示出的又一种图像采集控制方法的示意流程图。如图8所示，所述根据所述图像采集设备获取所述目标图像时的姿态，设置所述图像采集设备采集图像时的姿态包括：
步骤S41，根据所述图像采集设备获取所述目标图像时的姿态，通过云台设置所述图像采集设备采集图像时的姿态。
在一个实施例中，可以通过云台设置所述图像采集设备采集图像时的姿态。
在一个实施例中，可以根据所述评价参数在所述多张参考图像中确定目标图像；根据所述图像采集设备获取所述目标图像时的目标姿态，通过云台设置所述图像采集设备采集图像时的姿态。
可选地，所述云台包括以下至少之一：
单轴云台、双轴云台、三轴云台。
可选地，所述云台的增稳方式包括以下至少之一：
机械增稳、电子增稳、机械电子混合式增稳。
与上述图像采集控制方法的实施例相对应地，本公开还提出了图像采集控制设备的实施例。
如图9所示，本公开实施例提出的图像采集控制设备包括存储器901、处理器902；
所述存储器901用于存储程序代码；
所述处理器902，调用所述程序代码，当程序代码被执行时，用于执行以下操作：
在改变所述图像采集设备的姿态的过程中，获取所述图像采集设备采集到的多张参考图像；
对每张所述参考图像分别进行显著性检测，以确定每张所述参考图像中的显著区域；
根据每张所述参考图像中的显著区域和预设构图法则，确定每张所述参考图像的评价参数；
根据所述评价参数在所述多张参考图像中确定目标图像；
根据所述图像采集设备采集所述目标图像时的姿态，设置所述图像采集设备采集图像时的姿态。
在一个实施例中，所述处理器902用于：
针对所述参考图像进行傅里叶变换；
根据傅里叶变换的第一结果获取所述参考图像的相位谱；
对所述相位谱的反傅里叶变换的第二结果进行高斯滤波，以确定所述参考图像中的显著区域。
在一个实施例中，所述预设构图法则包括至少一种构图法则，所述处理 器902用于：
确定每张所述参考图像中的显著区域相对于每种所述构图法则的评价参数；
针对每种所述构图法则对应的评价参数进行加权求和，以确定所述显著区域相对于预设构图法则的评价参数。
在一个实施例中，所述构图法则包括以下至少之一：
三分法、主体视觉平衡法、黄金分割法、中心对称法。
在一个实施例中，所述构图法则包括三分法，所述处理器902用于：
计算所述显著区域的中心的坐标到所述参考图像中4个等分线的交点的距离中最短的距离；
根据所述显著区域的质心的坐标和所述最短的距离计算所述显著区域相对于所述三分法的评价参数。
在一个实施例中，所述构图法则包括主体视觉平衡法，所述处理器902用于：
基于所述参考图像中心的坐标以及所述显著区域的中心的坐标和质心的坐标，计算归一化曼哈顿距离；
根据所述归一化曼哈顿距离计算所述显著区域相对于所述主体视觉平衡法的评价参数。
在一个实施例中，所述处理器902用于：
在对每张所述参考图像分别进行显著性检测之前，针对所述参考图像消除所述图像采集设备的镜头畸变和果冻效应引发的误差。
在一个实施例中，所述处理器902用于：
将所述参考图像的垂直同步信号计数值和所述参考图像的数据进行行间同步，以确定所述参考图像中每一行数据在曝光过程中的运动信息；
通过后向映射或前向映射在所述参考图像上生成网格；
通过迭代法对所述运动信息进行计算，以确定所述网格的交点处的坐标在曝光过程中的偏移量；
根据所述偏移量对所述参考图像进行去畸变，以消除所述误差。
在一个实施例中，还包括云台，所述处理器902用于：
通过云台设置所述图像采集设备采集图像时的姿态。
在一个实施例中，所述云台包括以下至少之一：
单轴云台、双轴云台、三轴云台。
在一个实施例中，所述云台的增稳方式包括以下至少之一：
机械增稳、电子增稳、机械电子混合式增稳。
本公开的实施例还提出一种可移动平台，其特征在于，包括：
机身；
图像采集设备，用于采集图像；
以及上述任一实施例所述的图像采集控制设备。
图10是根据本公开的实施例示出的一种可移动平台的示意结构图。如图10所示，该可移动平台为手持式拍摄装置，手持式拍摄装置包括镜头101、三轴云台和惯性测量单元(Inertial measurement unit，IMU)102，其中三个轴分别为俯仰轴103，横滚轴104和偏航轴105，三轴云台连接于镜头101，其中俯仰轴用于调整镜头的俯仰角，横滚轴用于调整镜头的横滚角，偏航轴用于调整镜头的偏航角。
惯性测量单元102设置在镜头101下后方，惯性测量单元102的引脚可以与光电传感器的垂直同步引脚连接，从而对光电传感器的姿态进行采样，采样的频率可以根据需要进行设置，例如可以设置为8KHz，从而采样可以记录镜头101获取参考图像时的姿态以及运动信息，还可以根据垂直同步信号反推出参考图像中每一行像素的运动信息，例如按照图7所示实施例中的步骤S51来确定运动信息，从而对参考图像进行去畸变。
上述实施例阐明的系统、装置、模块或单元，具体可以由计算机芯片或实体实现，或者由具有某种功能的产品来实现。为了描述的方便，描述以上装置时以功能分为各种单元分别描述。当然，在实施本申请时可以把各单元的功能在同一个或多个软件和/或硬件中实现。本领域内的技术人员应明白， 本公开的实施例可提供为方法、系统、或计算机程序产品。因此，本公开可采用完全硬件实施例、完全软件实施例、或结合软件和硬件方面的实施例的形式。而且，本公开可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。
本说明书中的各个实施例均采用递进的方式描述，各个实施例之间相同相似的部分互相参见即可，每个实施例重点说明的都是与其他实施例的不同之处。尤其，对于系统实施例而言，由于其基本相似于方法实施例，所以描述的比较简单，相关之处参见方法实施例的部分说明即可。
需要说明的是，在本文中，诸如第一和第二等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来，而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者设备所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括所述要素的过程、方法、物品或者设备中还存在另外的相同要素。
以上所述仅为本申请的实施例而已，并不用于限制本申请。对于本领域技术人员来说，本申请可以有各种更改和变化。凡在本申请的精神和原理之内所作的任何修改、等同替换、改进等，均应包含在本申请的权利要求范围之内。

Claims (24)

  1. 一种图像采集控制方法，其特征在于，所述方法包括：
    在改变所述图像采集设备的姿态的过程中，获取所述图像采集设备采集到的多张参考图像；
    对每张所述参考图像分别进行显著性检测，以确定每张所述参考图像中的显著区域；
    根据每张所述参考图像中的显著区域和预设构图法则，确定每张所述参考图像的评价参数；
    根据所述评价参数在所述多张参考图像中确定目标图像；
    根据所述图像采集设备采集所述目标图像时的姿态，设置所述图像采集设备采集图像时的姿态。
  2. 根据权利要求1所述的方法，其特征在于，所述对每张所述参考图像分别进行显著性检测，以确定每张所述参考图像中的显著区域，包括：
    针对所述参考图像进行傅里叶变换；
    根据傅里叶变换的第一结果获取所述参考图像的相位谱；
    对所述相位谱的反傅里叶变换的第二结果进行高斯滤波，以确定所述参考图像中的显著区域。
  3. 根据权利要求1所述的方法，其特征在于，所述预设构图法则包括至少一种构图法则，所述根据每张所述参考图像中的显著区域和预设构图法则，确定每张所述参考图像的评价参数，包括：
    确定每张所述参考图像中的显著区域相对于每种所述构图法则的评价参数；
    针对每种所述构图法则对应的评价参数进行加权求和，以确定每张所述参考图像的评价参数。
  4. 根据权利要求3所述的方法，其特征在于，所述构图法则包括以下至少之一：
    三分法、主体视觉平衡法、黄金分割法、中心对称法。
  5. 根据权利要求4所述的方法，其特征在于，所述构图法则包括三分法，所述确定每张所述参考图像中的显著区域相对于每种所述构图法则的评价参数包括：
    计算所述显著区域的中心的坐标到所述参考图像中4个等分线的交点的距离中最短的距离；
    根据所述显著区域的质心的坐标和所述最短的距离计算所述显著区域相对于所述三分法的评价参数。
  6. 根据权利要求4所述的方法，其特征在于，所述构图法则包括主体视觉平衡法，所述确定每张所述参考图像中的显著区域相对于每种所述构图法则的评价参数包括：
    基于所述参考图像中心的坐标以及所述显著区域的中心的坐标和质心的坐标，计算归一化曼哈顿距离；
    根据所述归一化曼哈顿距离计算所述显著区域相对于所述主体视觉平衡法的评价参数。
  7. 根据权利要求1至6中任一项所述的方法，其特征在于，在对每张所述参考图像分别进行显著性检测之前，所述方法还包括：
    针对所述参考图像消除所述图像采集设备的镜头畸变和果冻效应引发的误差。
  8. 根据权利要求7所述的方法，其特征在于，所述针对所述参考图像消除所述图像采集设备的镜头畸变和果冻效应引发的误差包括：
    将所述参考图像的垂直同步信号计数值和所述参考图像的数据进行行间同步，以确定所述参考图像中每一行数据在曝光过程中的运动信息；
    通过后向映射或前向映射在所述参考图像上生成网格；
    通过迭代法对所述运动信息进行计算，以确定所述网格的交点处的坐标在曝光过程中的偏移量；
    根据所述偏移量对所述参考图像进行去畸变，以消除所述误差。
  9. 根据权利要求1至6中任一项所述的方法，其特征在于，所述设置所 述图像采集设备采集图像时的姿态包括：
    通过云台设置所述图像采集设备采集图像时的姿态。
  10. 根据权利要求9所述的方法，其特征在于，所述云台包括以下至少之一：
    单轴云台、双轴云台、三轴云台。
  11. 根据权利要求9所述的方法，其特征在于，所述云台的增稳方式包括以下至少之一：
    机械增稳、电子增稳、机械电子混合式增稳。
  12. 一种图像采集控制设备，其特征在于，包括存储器、处理器；
    所述存储器用于存储程序代码；
    所述处理器，调用所述程序代码，当程序代码被执行时，用于执行以下操作：
    在改变所述图像采集设备的姿态的过程中，获取所述图像采集设备采集到的多张参考图像；
    对每张所述参考图像分别进行显著性检测，以确定每张所述参考图像中的显著区域；
    根据每张所述参考图像中的显著区域和预设构图法则，确定每张所述参考图像的评价参数；
    根据所述评价参数在所述多张参考图像中确定目标图像；
    根据所述图像采集设备采集所述目标图像时的姿态，设置所述图像采集设备采集图像时的姿态。
  13. 根据权利要求12所述的图像采集控制设备，其特征在于，所述处理器用于：
    针对所述参考图像进行傅里叶变换；
    根据傅里叶变换的第一结果获取所述参考图像的相位谱；
    对所述相位谱的反傅里叶变换的第二结果进行高斯滤波，以确定所述参考图像中的显著区域。
  14. 根据权利要求12所述的图像采集控制设备，其特征在于，所述预设构图法则包括至少一种构图法则，所述处理器用于：
    确定每张所述参考图像中的显著区域相对于每种所述构图法则的评价参数；
    针对每种所述构图法则对应的评价参数进行加权求和，以确定所述显著区域相对于预设构图法则的评价参数。
  15. 根据权利要求14所述的图像采集控制设备，其特征在于，所述构图法则包括以下至少之一：
    三分法、主体视觉平衡法、黄金分割法、中心对称法。
  16. 根据权利要求15所述的图像采集控制设备，其特征在于，所述构图法则包括三分法，所述处理器用于：
    计算所述显著区域的中心的坐标到所述参考图像中4个等分线的交点的距离中最短的距离；
    根据所述显著区域的质心的坐标和所述最短的距离计算所述显著区域相对于所述三分法的评价参数。
  17. 根据权利要求15所述的图像采集控制设备，其特征在于，所述构图法则包括主体视觉平衡法，所述处理器用于：
    基于所述参考图像中心的坐标以及所述显著区域的中心的坐标和质心的坐标，计算归一化曼哈顿距离；
    根据所述归一化曼哈顿距离计算所述显著区域相对于所述主体视觉平衡法的评价参数。
  18. 根据权利要求12至17中任一项所述的图像采集控制设备，其特征在于，所述处理器用于：
    在对每张所述参考图像分别进行显著性检测之前，针对所述参考图像消除所述图像采集设备的镜头畸变和果冻效应引发的误差。
  19. 根据权利要求18所述的图像采集控制设备，其特征在于，所述处理器用于：
    将所述参考图像的垂直同步信号计数值和所述参考图像的数据进行行间同步，以确定所述参考图像中每一行数据在曝光过程中的运动信息；
    通过后向映射或前向映射在所述参考图像上生成网格；
    通过迭代法对所述运动信息进行计算，以确定所述网格的交点处的坐标在曝光过程中的偏移量；
    根据所述偏移量对所述参考图像进行去畸变，以消除所述误差。
  20. 根据权利要求12至17中任一项所述的图像采集控制设备，其特征在于，还包括云台，所述处理器用于：
    通过云台设置所述图像采集设备采集图像时的姿态。
  21. 根据权利要求20所述的图像采集控制设备，其特征在于，所述云台包括以下至少之一：
    单轴云台、双轴云台、三轴云台。
  22. 根据权利要求20所述的图像采集控制设备，其特征在于，所述云台的增稳方式包括以下至少之一：
    机械增稳、电子增稳、机械电子混合式增稳。
  23. 一种可移动平台，其特征在于，包括：
    机身；
    图像采集设备，用于采集图像；
    以及权利要求12至22中任一项所述的图像采集控制设备。
  24. 根据权利要求23所述的可移动平台，其特征在于，所述可移动平台为无人飞行器，无人车，手持设备，移动机器人中的一种。
PCT/CN2019/081518 2019-04-04 2019-04-04 图像采集控制方法，图像采集控制设备和可移动平台 WO2020199198A1 (zh)

Priority Applications (3)

Application Number Priority Date Filing Date Title
CN201980008880.8A CN111656763B (zh) 2019-04-04 2019-04-04 图像采集控制方法，图像采集控制设备和可移动平台
PCT/CN2019/081518 WO2020199198A1 (zh) 2019-04-04 2019-04-04 图像采集控制方法，图像采集控制设备和可移动平台
US17/317,887 US20210266456A1 (en) 2019-04-04 2021-05-11 Image capture control method, image capture control device, and mobile platform

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2019/081518 WO2020199198A1 (zh) 2019-04-04 2019-04-04 图像采集控制方法，图像采集控制设备和可移动平台

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/317,887 Continuation US20210266456A1 (en) 2019-04-04 2021-05-11 Image capture control method, image capture control device, and mobile platform

Publications (1)

Publication Number Publication Date
WO2020199198A1 true WO2020199198A1 (zh) 2020-10-08

Family

ID=72350338

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/081518 WO2020199198A1 (zh) 2019-04-04 2019-04-04 图像采集控制方法，图像采集控制设备和可移动平台

Country Status (3)

Country Link
US (1) US20210266456A1 (zh)
CN (1) CN111656763B (zh)
WO (1) WO2020199198A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112488966A (zh) * 2020-12-23 2021-03-12 深圳疆程技术有限公司 一种图像的反畸变方法、装置、电子设备及汽车

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120146171A1 (en) * 2006-07-20 2012-06-14 Akira Kosaka Image Sensor Unit and Image Sensor Apparatus
CN105120144A (zh) * 2015-07-31 2015-12-02 小米科技有限责任公司 图像拍摄方法和装置
CN106973221A (zh) * 2017-02-24 2017-07-21 北京大学 基于美学评价的无人机摄像方法和系统
CN109547689A (zh) * 2018-08-27 2019-03-29 幻想动力(上海)文化传播有限公司 自动拍摄控制方法、装置及计算机可读存储介质

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2010011441A (ja) * 2008-05-26 2010-01-14 Sanyo Electric Co Ltd 撮像装置及び画像再生装置
JP5565640B2 (ja) * 2012-02-09 2014-08-06 フリュー株式会社 写真シール作成装置および方法、並びにプログラム
US20190096041A1 (en) * 2017-09-25 2019-03-28 Texas Instruments Incorporated Methods and system for efficient processing of generic geometric correction engine
CN108322666B (zh) * 2018-02-12 2020-06-26 广州视源电子科技股份有限公司 摄像头快门的调控方法、装置、计算机设备及存储介质
CN108921130B (zh) * 2018-07-26 2022-03-01 聊城大学 基于显著性区域的视频关键帧提取方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120146171A1 (en) * 2006-07-20 2012-06-14 Akira Kosaka Image Sensor Unit and Image Sensor Apparatus
CN105120144A (zh) * 2015-07-31 2015-12-02 小米科技有限责任公司 图像拍摄方法和装置
CN106973221A (zh) * 2017-02-24 2017-07-21 北京大学 基于美学评价的无人机摄像方法和系统
CN109547689A (zh) * 2018-08-27 2019-03-29 幻想动力(上海)文化传播有限公司 自动拍摄控制方法、装置及计算机可读存储介质

Also Published As

Publication number Publication date
US20210266456A1 (en) 2021-08-26
CN111656763B (zh) 2022-02-25
CN111656763A (zh) 2020-09-11

Similar Documents

Publication Publication Date Title
US20200198149A1 (en) Robot vision image feature extraction method and apparatus and robot using the same
CN108765498B (zh) 单目视觉跟踪方法、装置及存储介质
CN111750820B (zh) 影像定位方法及其系统
JP6663040B2 (ja) 奥行き情報取得方法および装置、ならびに画像取得デバイス
CN106973228B (zh) 一种拍摄方法及电子设备
CN106529495A (zh) 一种飞行器的障碍物检测方法和装置
JPWO2018047687A1 (ja) 三次元モデル生成装置及び三次元モデル生成方法
JP2008506953A5 (zh)
CN108717704B (zh) 基于鱼眼图像的目标跟踪方法、计算机装置及计算机可读存储介质
US20210227145A1 (en) Imaging apparatus
CN112132908B (zh) 一种基于智能检测技术的相机外参数标定方法及设备
WO2018214778A1 (zh) 一种虚拟对象的展示方法及装置
CN112232279A (zh) 一种人员间距检测方法和装置
CN114022560A (zh) 标定方法及相关装置、设备
JP2015036841A (ja) 画像処理装置、距離計測装置、撮像装置、画像処理方法
CN114640833A (zh) 投影画面调整方法、装置、电子设备和存储介质
CN115457176A (zh) 一种图像生成方法、装置、电子设备及存储介质
CN116170689A (zh) 视频生成方法、装置、计算机设备和存储介质
CN105488780A (zh) 一种用于工业生产线的单目视觉测距追踪装置及其追踪方法
WO2020199198A1 (zh) 图像采集控制方法，图像采集控制设备和可移动平台
CN109785439A (zh) 人脸素描图像生成方法及相关产品
JP2019027882A (ja) 物体距離検出装置
WO2021138856A1 (zh) 相机控制方法、设备及计算机可读存储介质
JP6875646B2 (ja) 画像処理装置、および、画像処理プログラム
CN112330726B (zh) 一种图像处理方法及装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19923597

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19923597

Country of ref document: EP

Kind code of ref document: A1