WO2019237775A1 - Image information acquisition method and device - Google Patents

Image information acquisition method and device Download PDF

Info

Publication number
WO2019237775A1
WO2019237775A1 PCT/CN2019/077355 CN2019077355W WO2019237775A1 WO 2019237775 A1 WO2019237775 A1 WO 2019237775A1 CN 2019077355 W CN2019077355 W CN 2019077355W WO 2019237775 A1 WO2019237775 A1 WO 2019237775A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
target
pose
movable
information
Prior art date
Application number
PCT/CN2019/077355
Other languages
French (fr)
Chinese (zh)
Inventor
杨帆
何庭波
许占
陈海
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority to CN201810626611.5A priority Critical patent/CN110609562A/en
Priority to CN201810626611.5 priority
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2019237775A1 publication Critical patent/WO2019237775A1/en

Links

Images

Classifications

    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05DSYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
    • G05D1/08Control of attitude, i.e. control of roll, pitch, or yaw
    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05DSYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
    • G05D1/08Control of attitude, i.e. control of roll, pitch, or yaw
    • G05D1/0808Control of attitude, i.e. control of roll, pitch, or yaw specially adapted for aircraft
    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05DSYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
    • G05D1/10Simultaneous control of position or course in three dimensions
    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05DSYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
    • G05D1/10Simultaneous control of position or course in three dimensions
    • G05D1/101Simultaneous control of position or course in three dimensions specially adapted for aircraft
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/222Studio circuitry; Studio devices; Studio equipment ; Cameras comprising an electronic image sensor, e.g. digital cameras, video cameras, TV cameras, video cameras, camcorders, webcams, camera modules for embedding in other devices, e.g. mobile phones, computers or vehicles
    • H04N5/225Television cameras ; Cameras comprising an electronic image sensor, e.g. digital cameras, video cameras, camcorders, webcams, camera modules specially adapted for being embedded in other devices, e.g. mobile phones, computers or vehicles
    • H04N5/232Devices for controlling television cameras, e.g. remote control ; Control of cameras comprising an electronic image sensor
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/222Studio circuitry; Studio devices; Studio equipment ; Cameras comprising an electronic image sensor, e.g. digital cameras, video cameras, TV cameras, video cameras, camcorders, webcams, camera modules for embedding in other devices, e.g. mobile phones, computers or vehicles
    • H04N5/225Television cameras ; Cameras comprising an electronic image sensor, e.g. digital cameras, video cameras, camcorders, webcams, camera modules specially adapted for being embedded in other devices, e.g. mobile phones, computers or vehicles
    • H04N5/232Devices for controlling television cameras, e.g. remote control ; Control of cameras comprising an electronic image sensor
    • H04N5/23218Control of camera operation based on recognized objects
    • H04N5/23219Control of camera operation based on recognized objects where the recognized objects include parts of the human body, e.g. human faces, facial parts or facial expressions

Abstract

The embodiments of the present application provide an image information acquisition method and device, being used for a movable device, relating to the field of computer control techniques, and being able to enable the movable device to automatically adjust an image information acquisition angle and position, thereby acquiring more accurate image information. The method comprises: a movable device acquiring a first image using a first pose, so that the first image comprises a target object; determining whether the first image satisfies a preset condition; when the first image does not satisfy the preset condition, adjusting the first pose to a second pose; and acquiring a second image using the second pose. The embodiments of the present application are applied to a scenario where a movable device performs image information acquisition and image information processing.

Description

一种图像信息采集方法和装置Image information acquisition method and device

本申请要求于2018年06月15日提交中国专利局、申请号为201810626611.5、申请名称为“一种图像信息采集方法和装置”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。This application claims priority from a Chinese patent application filed on June 15, 2018 with the Chinese Patent Office, application number 201810626611.5, and application name "A Method and Device for Image Information Collection", the entire contents of which are incorporated herein by reference. in.

技术领域Technical field

本申请涉及计算机控制技术领域,尤其涉及一种图像信息采集方法和装置。The present application relates to the field of computer control technology, and in particular, to a method and a device for acquiring image information.

背景技术Background technique

目前,可移动装置(例如,无人机)正在迅速发展壮大,能够被应用到越来越多的行业和领域中。可移动装置主要的环境信息输入源是摄像头,为了使可移动装置获取用户所需的图像信息,需要用户根据可移动装置当前位置,人为操控可移动装置进行移动,以获得更好的图像信息获取角度和位置。Currently, mobile devices (for example, drones) are rapidly growing and can be applied to more and more industries and fields. The main input source of environmental information for a mobile device is a camera. In order for the mobile device to obtain the image information required by the user, the user needs to manually control the mobile device to move according to the current position of the mobile device to obtain better image information acquisition. Angle and position.

但是,随着人们对于可移动装置智能需求的提升,人们对可移动装置的操作也逐渐从由用户手动控制无人机来完成某项任务向直接向无人机发布某项任务,并由可移动装置自行完成进行转变。因此,亟需一种能够使可移动装置自动获取更准确的图像信息的方法。However, as people ’s intelligent needs for mobile devices increase, people ’s operations on mobile devices also gradually change from manual control of drones by users to complete certain tasks to publishing a task directly to drones. The mobile device performs the conversion itself. Therefore, there is an urgent need for a method that enables a mobile device to automatically obtain more accurate image information.

发明内容Summary of the Invention

本申请实施例提供一种图像信息采集方法和装置,使可移动装置能够自动调整图像信息获取角度和位置,从而获取更准确的图像信息。Embodiments of the present application provide an image information acquisition method and device, so that a mobile device can automatically adjust an image information acquisition angle and position, thereby acquiring more accurate image information.

第一方面,本申请实施例提供一种图像信息采集方法,用于可移动装置,该方法包括:采用第一位姿采集第一图像,以使该第一图像包括目标物体;确定该第一图像是否满足预设条件;当该第一图像不满足预设条件时,将该第一位姿调整为第二位姿;采用该第二位姿采集第二图像。According to a first aspect, an embodiment of the present application provides an image information acquisition method for a mobile device. The method includes: acquiring a first image using a first pose so that the first image includes a target object; and determining the first image. Whether the image satisfies a preset condition; when the first image does not satisfy the preset condition, adjust the first pose to a second pose; and use the second pose to acquire a second image.

也就是说,当可移动装置采用第一位姿采集到的第一图像不满足预设条件时,可移动装置可以将该第一位姿调整为第二位姿,即能够自动调整图像信息获取角度和位置,以便采用该第二位姿采集更准确的图像信息。That is, when the first image collected by the mobile device using the first pose does not meet the preset conditions, the mobile device can adjust the first pose to the second pose, that is, it can automatically adjust the image information acquisition Angle and position in order to acquire more accurate image information using this second pose.

在一种可能的实现方式中,该方法还包括:采用该第一位姿和/或该第二位姿采集该目标物体的状态信息;对应的,将该第一位姿调整为第二位姿,包括:根据该第一图像和该目标物体的状态信息将该第一位姿调整为第二位姿;其中,该目标物体的状态信息包括以下信息的至少一种:该目标物体相对于该可移动装置的距离信息、速度信息和移动方向信息。In a possible implementation manner, the method further includes: using the first pose and / or the second pose to collect status information of the target object; correspondingly, adjusting the first pose to the second position The pose includes: adjusting the first pose to the second pose according to the first image and the status information of the target object; wherein the status information of the target object includes at least one of the following information: the target object is relative to Distance information, speed information, and movement direction information of the movable device.

由此,可移动装置可以根据该第一图像和该目标物体的状态信息调整位姿,能够得到更好的针对目标物体的图像信息获取角度和位置。例如,当第一图像中的目标物体处于图像的边缘位置,且目标物体距离可移动装置较远时,可移动装置可以在靠近目标物体的同时调整自身姿态,以便使目标物体处于图像中心位置,从而可以获取到更准确的图像信息。Therefore, the movable device can adjust the pose according to the first image and the state information of the target object, and can obtain a better image information acquisition angle and position for the target object. For example, when the target object in the first image is at the edge of the image and the target object is far away from the movable device, the movable device may adjust its posture while approaching the target object so that the target object is at the center of the image. Thereby, more accurate image information can be obtained.

在一种可能的实现方式中,根据该第一图像和该目标物体的状态信息将该第一位姿调 整为第二位姿,包括:根据该第一图像和该目标物体的状态信息确定该可移动装置的移动路线和姿态角;根据该移动路线和该姿态角将该第一位姿调整为该第二位姿。In a possible implementation manner, adjusting the first pose to the second pose according to the first image and the state information of the target object includes: determining the according to the first image and the state information of the target object. The moving route and attitude angle of the movable device; the first pose is adjusted to the second pose according to the moving route and the attitude angle.

在一种可能的实现方式中。该目标物体存在至少两个外观不同的面。举例来说,假设目标物体为人,从人脸的前方观察人和从人脸的后方(后脑勺正对的方向)观察人,可以得到两个或两个以上外观不同的面。该至少两个外观不同的面包括第一面,例如,可以设定从人脸前方观察到的面是第一面。当该目标物体存在第一面时,该移动路线用于指示该可移动装置从当前位置移动到第一目标空间区域;该第一目标空间区域包括该第一面的法线方向与该可移动装置和目标物体的连线方向的夹角满足第一预设阈值,且该可移动装置与该目标物体之间的距离满足第二预设阈值的空间区域。可以理解的是,当可移动装置处于第一目标区域内时,可移动装置可以更好的获取该第一面的相关信息。In one possible implementation. The target object has at least two faces with different appearances. For example, assuming that the target object is a person, observing the person from the front of the face and observing the person from the back of the face (the direction facing the back of the head), two or more faces with different appearances can be obtained. The at least two faces having different appearances include a first face. For example, the face viewed from the front of a human face can be set to be the first face. When the target object has a first surface, the moving route is used to instruct the movable device to move from the current position to a first target space area; the first target space area includes a normal direction of the first surface and the movable The included angle between the device and the target object in the direction of the connection line satisfies a first preset threshold, and the distance between the movable device and the target object satisfies a second preset threshold spatial area. It can be understood that when the movable device is within the first target area, the movable device can better obtain the related information of the first face.

在一种可能的实现方式中,若该目标物体不存在外观不同的面,该移动路线用于指示该可移动装置从当前位置移动到第二目标空间区域,该第二目标空间区域包括该目标物体与该可移动装置之间的距离满足第三预设阈值的空间区域。In a possible implementation manner, if the target object does not have surfaces with different appearances, the moving route is used to instruct the movable device to move from the current position to a second target space area, where the second target space area includes the target A space area where the distance between the object and the movable device satisfies a third preset threshold.

这样一来,可移动装置可以通过改变自身与目标物体的相对位置的方式重新获取目标物体的图像信息,以便获取更为准确的图像信息。In this way, the movable device can re-acquire the image information of the target object by changing the relative position of itself and the target object, so as to obtain more accurate image information.

在一种可能的实现方式中,该可移动装置包括摄像头和机体,该姿态角包括该机体相对于该目标物体的角度和该摄像头相对于该目标物体的角度。In a possible implementation manner, the movable device includes a camera and a body, and the attitude angle includes an angle of the body with respect to the target object and an angle of the camera with respect to the target object.

这样一来,可移动装置可以分别调整其机体和摄像头相对于目标物体的角度,即调整其机体和摄像头相对于目标物体的姿态,以得到更准确的图像信息获取角度和位置,从而能够采集到更准确的图像信息。In this way, the movable device can adjust the angle of its body and camera with respect to the target object, that is, adjust the attitude of its body and camera with respect to the target object, so as to obtain a more accurate image information acquisition angle and position, so that it can collect More accurate image information.

在一种可能的实现方式中,确定该第一图像不满足预设条件包括:确定该目标物体在该第一图像中占据的像素数量或像素比例小于第四预设阈值;和/或,确定该第一图像的清晰度小于第五预设阈值;和/或,确定该第一图像的检测结果信任度小于第六预设阈值;和/或,确定该目标物体包括第一特征,该第一特征是预设的第二特征的父特征。In a possible implementation manner, determining that the first image does not satisfy a preset condition includes: determining that the number of pixels or pixel ratio occupied by the target object in the first image is less than a fourth preset threshold; and / or, determining The sharpness of the first image is less than a fifth preset threshold; and / or, it is determined that the confidence of the detection result of the first image is less than the sixth preset threshold; and / or, it is determined that the target object includes a first feature, and the first One feature is the parent feature of the preset second feature.

其中,预设的第二特征可以是用户希望从目标物体检测到的特征。第一特征是第二特征的父特征,即当目标物体具有第一特征时,目标物体可能也具有第二特征。例如,第一特征(例如,人)是第二特征(例如,儿童)的父特征。由于可移动装置对目标物体是否存在预设的第二特征的检测结果可能不太准确,因此可以同时确定目标物体是否包括第一特征。若目标物体包括第一特征,则第一图像不满足预设条件,从而可移动装置可以调整位姿重新采集图像信息进行判断,以便减少可移动装置对目标物体的特征误判的可能性。The preset second feature may be a feature that the user wishes to detect from the target object. The first feature is the parent feature of the second feature, that is, when the target object has the first feature, the target object may also have the second feature. For example, a first feature (eg, a person) is a parent feature of a second feature (eg, a child). Since the detection result of the movable device on whether the target object has the preset second feature may not be accurate, it may be determined whether the target object includes the first feature at the same time. If the target object includes the first feature, the first image does not satisfy the preset condition, so that the mobile device can adjust the posture and re-acquire the image information for judgment, so as to reduce the possibility of the mobile device misjudging the feature of the target object.

在一种可能的实现方式中,该方法还包括:确定该第二图像是否满足该预设条件;当该第二图像不满足该预设条件时,根据该第一图像和该第二图像,将该第二位姿调整为第三位姿;采用该第三位姿采集第三图像。In a possible implementation manner, the method further includes: determining whether the second image meets the preset condition; and when the second image does not satisfy the preset condition, according to the first image and the second image, Adjust the second pose to a third pose; use the third pose to acquire a third image.

也就是说,可移动装置可以根据多次采集到的图像信息调整位姿,可以更准确的调整图像信息获取角度和位置,从而采集到更准确的图像信息。That is, the movable device can adjust the pose according to the image information collected multiple times, and can more accurately adjust the image information acquisition angle and position, thereby acquiring more accurate image information.

在一种可能的实现方式中,该可移动装置为无人机,该无人机具备空间机动性和悬停的功能。In a possible implementation manner, the movable device is a drone, and the drone has functions of space mobility and hovering.

第二方面,本申请实施例提供一种可移动装置,包括:采集单元,用于采用第一位姿采集第一图像,以使该第一图像包括目标物体;确定单元,用于确定该第一图像是否满足 预设条件;调整单元,用于当该第一图像不满足预设条件时,将该第一位姿调整为第二位姿;该采集单元,还用于采用该第二位姿采集第二图像。In a second aspect, an embodiment of the present application provides a movable device, including: an acquisition unit configured to acquire a first image using a first pose so that the first image includes a target object; and a determination unit configured to determine the first Whether an image satisfies a preset condition; an adjustment unit configured to adjust the first posture to a second posture when the first image does not satisfy the preset condition; and the acquisition unit is further configured to use the second position The posture acquires a second image.

在一种可能的实现方式中,该采集单元还用于:采用该第一位姿和/或该第二位姿采集该目标物体的状态信息;对应的,该调整单元用于:根据该第一图像和该目标物体的状态信息将该第一位姿调整为第二位姿;其中,该目标物体的状态信息包括以下信息的至少一种:该目标物体相对于该可移动装置的距离信息、速度信息和移动方向信息。In a possible implementation manner, the collecting unit is further configured to: use the first pose and / or the second pose to collect status information of the target object; correspondingly, the adjusting unit is configured to: An image and state information of the target object adjust the first pose to a second pose; wherein the state information of the target object includes at least one of the following information: distance information of the target object relative to the movable device , Speed information and moving direction information.

在一种可能的实现方式中,该调整单元用于:根据该第一图像和该目标物体的状态信息确定该可移动装置的移动路线和姿态角;根据该移动路线和该姿态角将该第一位姿调整为该第二位姿。In a possible implementation manner, the adjusting unit is configured to determine a moving route and an attitude angle of the movable device according to the first image and state information of the target object; and determine the first route according to the moving route and the attitude angle. One posture is adjusted to the second posture.

在一种可能的实现方式中,若该目标物体存在至少两个外观不同的面,该至少两个外观不同的面包括第一面,该移动路线用于指示该可移动装置从当前位置移动到第一目标空间区域;该第一目标空间区域包括该第一面的法线方向与该可移动装置和目标物体的连线方向的夹角满足第一预设阈值,且该可移动装置与该目标物体之间的距离满足第二预设阈值的空间区域。In a possible implementation manner, if the target object has at least two faces with different appearances, the at least two faces with different appearances include a first face, and the movement route is used to instruct the movable device to move from the current position to A first target space region; the first target space region includes an included angle between a normal direction of the first surface and a connection direction of the movable device and the target object meeting a first preset threshold, and the movable device and the A space area where the distance between the target objects satisfies a second preset threshold.

在一种可能的实现方式中,若该目标物体不存在外观不同的面,该移动路线用于指示该可移动装置从当前位置移动到第二目标空间区域,该第二目标空间区域包括该目标物体与该可移动装置之间的距离满足第三预设阈值的空间区域。In a possible implementation manner, if the target object does not have surfaces with different appearances, the moving route is used to instruct the movable device to move from the current position to a second target space area, where the second target space area includes the target A space area where the distance between the object and the movable device satisfies a third preset threshold.

在一种可能的实现方式中,该可移动装置包括摄像头和机体,该姿态角包括该机体相对于该目标物体的角度和该摄像头相对于该目标物体的角度。In a possible implementation manner, the movable device includes a camera and a body, and the attitude angle includes an angle of the body with respect to the target object and an angle of the camera with respect to the target object.

在一种可能的实现方式中,确定单元确定该第一图像不满足预设条件包括:确定单元确定该目标物体在该第一图像中占据的像素数量或像素比例小于第四预设阈值;和/或,该确定单元确定该第一图像的清晰度小于第五预设阈值;和/或,该确定单元确定该第一图像的检测结果信任度小于第六预设阈值;和/或,该确定单元确定该目标物体包括第一特征,该第一特征是预设的第二特征的父特征。In a possible implementation manner, the determining unit determines that the first image does not satisfy a preset condition includes: the determining unit determines that the number of pixels or the pixel ratio occupied by the target object in the first image is less than a fourth preset threshold; and / Or, the determining unit determines that the sharpness of the first image is less than a fifth preset threshold; and / or, the determining unit determines that the degree of trust in the detection result of the first image is less than a sixth preset threshold; and / or, the The determining unit determines that the target object includes a first feature, and the first feature is a parent feature of a preset second feature.

在一种可能的实现方式中,该确定单元还用于:确定该第二图像是否满足该预设条件;该调整单元还用于,当该第二图像不满足该预设条件时,根据该第一图像和该第二图像,将该第二位姿调整为第三位姿;该采集单元,还用于采用该第三位姿采集第三图像。In a possible implementation manner, the determining unit is further configured to determine whether the second image meets the preset condition; the adjusting unit is further configured to, when the second image does not satisfy the preset condition, according to the The first image and the second image adjust the second pose to a third pose; the acquisition unit is further configured to use the third pose to acquire a third image.

第二方面及其各种可能的实现方式的技术效果可以参见第一方面及其各种可能的实现方式的技术效果,此处不再赘述。For the technical effects of the second aspect and its various possible implementation manners, refer to the technical effects of the first aspect and its various possible implementation manners, which are not repeated here.

第三方面,本申请实施例提供了一种装置,该装置以芯片的产品形态存在,该装置的结构中包括处理器和存储器,该存储器用于与处理器耦合,保存该装置必要的程序指令和数据,该处理器用于执行存储器中存储的程序指令,使得该装置执行上述方法中可移动装置的功能。In a third aspect, an embodiment of the present application provides a device. The device exists in the form of a chip product. The structure of the device includes a processor and a memory. The memory is used for coupling with the processor and stores program instructions necessary for the device. And data, the processor is configured to execute program instructions stored in the memory, so that the device performs the functions of the movable device in the above method.

第四方面,本申请实施例提供了一种可移动装置,该可移动装置可以实现上述方法实施例中可移动装置所执行的功能,功能可以通过硬件实现,也可以通过硬件执行相应的软件实现。硬件或软件包括一个或多个上述功能相应的模块。In a fourth aspect, an embodiment of the present application provides a movable device, which can implement functions performed by the movable device in the foregoing method embodiments, and the functions may be implemented by hardware or by software executing corresponding software. . The hardware or software includes one or more modules corresponding to the above functions.

在一种可能的设计中,该可移动装置的结构中包括处理器和通信接口,该处理器被配置为支持该可移动装置执行上述方法中相应的功能。该通信接口用于支持该可移动装置与其他网元之间的通信。该可移动装置还可以包括存储器,该存储器用于与处理器耦合,其 保存该可移动装置必要的程序指令和数据。In a possible design, the structure of the mobile device includes a processor and a communication interface, and the processor is configured to support the mobile device to perform a corresponding function in the above method. The communication interface is used to support communication between the mobile device and other network elements. The removable device may further include a memory for coupling with the processor, which stores program instructions and data necessary for the removable device.

第五方面,本申请实施例提供一种计算机可读存储介质,包括指令,当其在计算机上运行时,使得计算机执行第一方面提供的任意一种方法。In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium including instructions that, when run on a computer, cause the computer to execute any one of the methods provided in the first aspect.

第六方面,本申请实施例提供了一种包含指令的计算机程序产品,当其在计算机上运行时,使得计算机执行第一方面提供的任意一种方法。According to a sixth aspect, an embodiment of the present application provides a computer program product including instructions, which when executed on a computer, causes the computer to execute any one of the methods provided by the first aspect.

附图说明BRIEF DESCRIPTION OF THE DRAWINGS

图1为本申请实施例提供的一种可移动装置进行图像采集的场景示意图;FIG. 1 is a schematic diagram of a scene where a mobile device performs image collection according to an embodiment of the present application; FIG.

图2为本申请实施例提供的一种可移动装置的结构示意图一;FIG. 2 is a first schematic structural diagram of a mobile device according to an embodiment of the present application; FIG.

图3为本申请实施例提供的一种可移动装置的结构示意图二;FIG. 3 is a second schematic structural diagram of a mobile device according to an embodiment of the present application; FIG.

图4为本申请实施例提供的一种可移动装置的机体和摄像头分别调整姿态角的示意图;FIG. 4 is a schematic diagram of respectively adjusting a posture angle of a body and a camera of a movable device according to an embodiment of the present application; FIG.

图5为本申请实施例提供的一种本申请实施例提供的一种图像分类、物体检测、语义分割、实例分割算法的输出结果示意图;5 is a schematic diagram of an output result of an image classification, object detection, semantic segmentation, and instance segmentation algorithm provided by an embodiment of the present application;

图6为本申请实施例提供的一种2D包围盒和3D包围盒示意图;6 is a schematic diagram of a 2D bounding box and a 3D bounding box according to an embodiment of the present application;

图7为本申请实施例提供的一种图像信息采集方法的流程示意图;7 is a schematic flowchart of an image information collection method according to an embodiment of the present application;

图8为本申请实施例提供的一种第一目标空间区域的示意图;FIG. 8 is a schematic diagram of a first target space region according to an embodiment of the present application; FIG.

图9为本申请实施例提供的一种可移动装置的结构示意图三。FIG. 9 is a third schematic structural diagram of a mobile device according to an embodiment of the present application.

具体实施方式detailed description

本申请实施例提供一种图像信息采集方法和装置,可以应用于可移动装置进行图像信息采集和图像信息处理的场景中。如图1所示,在可移动装置进行图像信息采集的过程中,可移动装置可以自动调整可移动装置的位姿,以便采集到目标物体更准确的图像信息。进一步的,可移动装置可以根据采集到的图像信息进行图像处理任务。The embodiments of the present application provide an image information collection method and device, which can be applied to a scenario where a mobile device performs image information collection and image information processing. As shown in FIG. 1, during the process of collecting image information by the mobile device, the mobile device can automatically adjust the posture of the mobile device in order to acquire more accurate image information of the target object. Further, the mobile device may perform an image processing task according to the collected image information.

如图2所示,可移动装置可以包括摄像头和具有空间移动性的机体。例如,可移动装置可以为无人机,可以具备空间机动性和悬停的功能。其中,摄像头例如但不限于为各种类型的摄像传感器,例如红外线摄像头、双目摄像头、单目摄像头、光场摄像头、多目摄像头等。可选的,可移动装置还可以包括各种类型的传感器,例如超声波雷达、激光雷达、红外线摄像头、毫米波雷达、红外线测量仪、陀螺仪、加速度传感器等等。As shown in FIG. 2, the movable device may include a camera and a body with spatial mobility. For example, the mobile device can be a drone, which can have space mobility and hovering capabilities. The camera is, for example but not limited to, various types of camera sensors, such as an infrared camera, a binocular camera, a monocular camera, a light field camera, a multi-eye camera, and the like. Optionally, the movable device may further include various types of sensors, such as an ultrasonic radar, a lidar, an infrared camera, a millimeter wave radar, an infrared measuring instrument, a gyroscope, an acceleration sensor, and the like.

在一种可能的设计中,本申请实施例中的可移动装置可以通过图3中的装置来实现。图3所示为本申请实施例提供的装置的硬件结构示意图。该装置300包括至少一个处理器301,通信总线302,存储器303以及至少一个通信接口304。In a possible design, the movable device in the embodiment of the present application may be implemented by the device in FIG. 3. FIG. 3 is a schematic diagram of a hardware structure of a device according to an embodiment of the present application. The device 300 includes at least one processor 301, a communication bus 302, a memory 303, and at least one communication interface 304.

处理器301可以是一个通用中央处理器(central processing unit,CPU),微处理器,特定应用集成电路(application-specific integrated circuit,ASIC),或一个或多个用于控制本申请方案程序执行的集成电路。The processor 301 may be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more processors for controlling the execution of the program program of the present application. integrated circuit.

通信总线302可包括一通路,在上述组件之间传送信息。The communication bus 302 may include a path for transmitting information between the aforementioned components.

通信接口304,使用任何收发器一类的装置,用于与其他设备或通信网络通信,如以太网,无线接入网(radio access network,RAN),无线局域网(wireless local area networks,WLAN)等。The communication interface 304 uses any device such as a transceiver to communicate with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area networks (WLAN), etc. .

存储器303可以是只读存储器(read-only memory,ROM)或可存储静态信息和指 令的其他类型的静态存储设备,随机存取存储器(random access memory,RAM)或者可存储信息和指令的其他类型的动态存储设备,也可以是电可擦可编程只读存储器(electrically erasable programmable read-only memory,EEPROM)、只读光盘(compact disc read-only memory,CD-ROM)或其他光盘存储、光碟存储(包括压缩光碟、激光碟、光碟、数字通用光碟、蓝光光碟等)、磁盘存储介质或者其他磁存储设备、或者能够用于携带或存储具有指令或数据结构形式的期望的程序代码并能够由计算机存取的任何其他介质,但不限于此。存储器可以是独立存在,通过通信总线302与处理器相连接。存储器也可以和处理器集成在一起。The memory 303 may be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM) or other type that can store information and instructions Dynamic storage device, can also be electrically erasable programmable read-only memory (electrically erasable programmable read-only memory (EEPROM)), read-only compact disc (compact disc-read-only memory (CD-ROM) or other optical disc storage, optical disc storage (Including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), disk storage media or other magnetic storage devices, or can be used to carry or store desired program code in the form of instructions or data structures and can be used by a computer Any other media accessed, but not limited to this. The memory may exist independently, and is connected to the processor through the communication bus 302. The memory can also be integrated with the processor.

其中,存储器303用于存储执行本申请方案的计算机执行指令,并由处理器301来控制执行。处理器301用于执行存储器303中存储的计算机执行指令,从而实现本申请下述实施例提供的承载处理方法。The memory 303 is configured to store a computer execution instruction that executes the solution of the present application, and the processor 301 controls execution. The processor 301 is configured to execute computer execution instructions stored in the memory 303, so as to implement the bearer processing method provided in the following embodiments of the present application.

可选的,本申请实施例中的计算机执行指令也可以称之为应用程序代码,本申请实施例对此不作具体限定。Optionally, the computer-executable instructions in the embodiments of the present application may also be referred to as application program codes, which are not specifically limited in the embodiments of the present application.

在具体实现中,作为一种实施例,处理器301可以包括一个或多个CPU,例如图3中的CPU0和CPU1。In specific implementation, as an embodiment, the processor 301 may include one or more CPUs, such as CPU0 and CPU1 in FIG. 3.

在具体实现中,作为一种实施例,装置300可以包括多个处理器,例如图3中的处理器301和处理器307。这些处理器中的每一个可以是一个单核(single-CPU)处理器,也可以是一个多核(multi-CPU)处理器。这里的处理器可以指一个或多个设备、电路、和/或用于处理数据(例如计算机程序指令)的处理核。In a specific implementation, as an embodiment, the apparatus 300 may include multiple processors, such as the processor 301 and the processor 307 in FIG. 3. Each of these processors can be a single-CPU processor or a multi-CPU processor. A processor herein may refer to one or more devices, circuits, and / or processing cores for processing data (such as computer program instructions).

在具体实现中,作为一种实施例,装置300还可以包括输出设备305和输入设备306。输出设备305和处理器301通信,可以以多种方式来显示信息。例如,输出设备305可以是发光二级管(light emitting diode,LED)显示设备,阴极射线管(cathode ray tube,CRT)显示设备,或投影仪(projector)等。输入设备306和处理器301通信,可以以多种方式接收用户的输入。输入设备306可以是各种类型的传感设备等。例如,输入设备可以包括图像传感器、超声波传感器、超声波雷达、激光雷达、红外线摄像头、毫米波雷达、红外线测量仪、双目摄像头、单目摄像头、加速度传感器、全球定位系统(global positioning system,GPS)等等。输入设备306可以用于执行下述方法实施例中可移动装置执行的各种信息采集。In specific implementation, as an embodiment, the apparatus 300 may further include an output device 305 and an input device 306. The output device 305 communicates with the processor 301 and can display information in a variety of ways. For example, the output device 305 may be a light emitting diode (LED) display device, a cathode ray tube (CRT) display device, or a projector. The input device 306 communicates with the processor 301 and can receive user input in a variety of ways. The input device 306 may be various types of sensing devices and the like. For example, the input device may include an image sensor, an ultrasonic sensor, an ultrasonic radar, a lidar, an infrared camera, a millimeter wave radar, an infrared measuring instrument, a binocular camera, a monocular camera, an acceleration sensor, a global positioning system (GPS) and many more. The input device 306 may be used to perform various types of information collection performed by the mobile device in the method embodiments described below.

在具体实现中,作为一种实施例,装置300还可以包括动力系统308。动力系统308可以包括电动的动力系统或内燃机。其中,电动的动力系统可以包含电机(例如无刷电机)、电调(控制电机转速)、螺旋桨以及电池等。In a specific implementation, as an embodiment, the apparatus 300 may further include a power system 308. The power system 308 may include an electric power system or an internal combustion engine. Among them, the electric power system may include a motor (such as a brushless motor), an ESC (controlling the speed of the motor), a propeller, and a battery.

其中,处理器301用于控制动力系统308执行下述方法实施例中可移动装置的位姿调整等操作。The processor 301 is configured to control the power system 308 to perform operations such as posture adjustment of the mobile device in the following method embodiments.

在具体实现中,装置300可以是无人机、无人飞行器或有图3中类似结构的设备。本申请实施例不限定装置300的类型。In a specific implementation, the device 300 may be an unmanned aerial vehicle, an unmanned aerial vehicle, or a device having a similar structure in FIG. 3. The embodiment of the present application does not limit the type of the device 300.

为了下述各实施例的描述清楚简洁,首先给出相关概念或技术的简要介绍:In order to make the description of the following embodiments clear and concise, a brief introduction of related concepts or technologies is given first:

目标物体:可以包括预设的至少一个物体。物体可以是自然界客观存在的一切有形体的物质。目标物体可以具有相应的状态,例如静止或运动状态。可移动装置可以分别对至少一个物体中的每个物体的图像信息进行采集,也可以对符合相应的条件的 多个物体(例如距离较近的两个物体)的图像信息同时进行采集。Target object: may include at least one preset object. Objects can be all physical matter that exists objectively in nature. The target object can have a corresponding state, such as a stationary or moving state. The movable device may collect image information of each of the at least one object separately, or may simultaneously acquire image information of multiple objects (for example, two objects closer to each other) that meet the corresponding conditions.

目标物体的面:是以不同位置和角度对目标物体进行观察获得的“观察平面”。在一种可能的情况中,目标物体可以存在至少两个外观不同的面。至少两个外观不同的面包括第一面,可以将第一面定义为目标物体的朝向参考区域,可以将朝向参考区域的法线方向定义为目标物体的朝向,从而目标物体具有朝向性。其中,朝向参考区域的法线方向可以包括正方向和负方向,可以将由目标物体内部指向外部的方向定义为法线正方向,将目标物体外部指向内部的方向定义为法线负方向。例如,假设目标物体为人,可以将人的面部(人脸)定义为人的朝向参考区域,人脸的法线正方向(从人体内指向人体外,即人脸正对的方向)是人的朝向。The surface of the target object: the "observation plane" obtained by observing the target object at different positions and angles. In a possible case, the target object may have at least two faces with different appearances. At least two faces with different appearances include a first face. The first face may be defined as a reference area of the target object, and a normal direction facing the reference area may be defined as the direction of the target object, so that the target object is directional. The normal direction toward the reference area may include a positive direction and a negative direction. The direction from the inside of the target object to the outside may be defined as the normal direction of the normal, and the direction from the outside of the target object to the inside may be defined as the negative direction of the normal. For example, assuming that the target object is a person, the person's face (face) can be defined as the reference direction of the person's face, and the normal direction of the face's normal (pointing from the inside of the body to the outside of the person, that is, the direction in which the face is facing) is the direction of the person. .

位姿:可以包括可移动装置相对于目标物体的三维距离位置参数或者可移动装置在世界坐标系(如,经度和纬度)的位置参数,以及可移动装置的姿态角。Posture: may include the three-dimensional distance position parameter of the mobile device relative to the target object or the position parameter of the mobile device in the world coordinate system (eg, longitude and latitude), and the attitude angle of the mobile device.

其中,姿态角包括可移动装置的机体相对于目标物体的角度和摄像头相对于目标物体的角度。示例性的,如图4中的(a)所示,所示,假设目标物体为人,以人的面部中心为坐标原点建立三维坐标,X轴、Y轴和Z轴相互垂直,且Y轴与人的面部正前方的方向平行。通常情况下,当可移动装置悬停时,可移动装置的机体相对于Z轴的角度不变。如图4中的(b)所示,可移动装置的机体可以在X轴和Y轴组成的平面进行360度的旋转。如图4中的(c)所示,可移动装置的摄像头相对于Z轴的角度可以不断变化。并且摄像头也可以在以X轴和Y轴所在的平面进行旋转。The attitude angle includes an angle of the body of the movable device with respect to the target object and an angle of the camera with respect to the target object. Exemplarily, as shown in (a) of FIG. 4, it is assumed that the target object is a person, and the three-dimensional coordinates are established with the center of the face of the person as the origin of the coordinates, the X axis, Y axis, and Z axis are perpendicular to each other, and the Y axis The direction of the person's face is parallel. Generally, when the movable device is hovered, the angle of the body of the movable device with respect to the Z axis does not change. As shown in FIG. 4 (b), the body of the movable device can be rotated 360 degrees in a plane composed of the X axis and the Y axis. As shown in FIG. 4 (c), the angle of the camera of the movable device with respect to the Z axis can be continuously changed. And the camera can also rotate on the plane where the X axis and Y axis are located.

图像分类(image classification)算法(任务):也称为图像识别(image recognition)算法(任务),是常见的图像处理算法(任务)之一。其操作是将采集到的图像输入分类器,判断该图像是否包含目标物体。例如,图像分类的输出可以对应图5中的(a),目标物体包括瓶子、杯子和立方体。其中,分类器可以采用统计学分类器,例如但不限于支持向量机(Support Vector Machines,SVM)或隐马尔可夫模型(Hidden Markov Model,HMM)等。通过大量带有相应物体/事件标记的图像对分类器进行训练,从而使分类器具备对于标记内容的图像分类和识别功能。上述标记不限于是对图像中物体、图像自身属性、图像中的事件等内容的标记。举例来说,训练图像集可以包含一个较为抽象的标记,比如对图像风格属性的标记,标记内容为“风景照”。经过使用训练图像对分类器进行训练,分类器学习训练图像集中所有带有“风景照”标记的图像和没有带有“风景照”标记的图像在图像信息中的区别,最终具备针对“风景照”这一物体的识别能力。又比如,用1000张猫的图像(对应标记:猫,非狗)和1000张狗(对应标记:非猫,狗)以及1000张既不是猫也不是狗的图像(对应标记:非猫,非狗)来训练分类器,则分类器具备一定识别猫与狗的图像的能力。随着深度学习的发展以及终端芯片算力的增长,业界越来越多采用深度神经网络(Deep Neural Network,DNN)和卷积神经网络(Convolutional Neural Network,CNN)处理图像分类和识别任务。其本质仍然与统计学分类器相同,需要大量带有标记的图像对CNN模型进行训练。Image classification (image classification) algorithm (task): also known as image recognition (image recognition) algorithm (task), is one of the common image processing algorithms (tasks). Its operation is to input the collected image to a classifier and determine whether the image contains a target object. For example, the output of image classification may correspond to (a) in FIG. 5, and the target objects include bottles, cups, and cubes. Among them, the classifier may adopt a statistical classifier, such as, but not limited to, a Support Vector Machine (SVM) or a Hidden Markov Model (HMM). The classifier is trained through a large number of images with corresponding object / event markers, so that the classifier has the function of image classification and recognition for the marked content. The above mark is not limited to a mark on an object in an image, an attribute of the image itself, an event in the image, and the like. For example, the training image set may include a more abstract tag, such as a tag for the image style attribute, and the tag content is "landscape photo". After training the classifier using training images, the classifier learns the difference in image information between all images with a "landscape photo" tag and images without a "landscape photo" tag in the training image set. "The ability to recognize this object. For another example, use 1000 images of cats (corresponding tags: cat, non-dog) and 1000 dogs (corresponding tags: non-cat, dog) and 1000 images of neither cats or dogs (corresponding tags: non-cat, non-dog) Dog) to train a classifier, the classifier has the ability to recognize images of cats and dogs. With the development of deep learning and the increase in computing power of terminal chips, the industry increasingly uses deep neural networks (DNN) and convolutional neural networks (CNN) to process image classification and recognition tasks. Its essence is still the same as the statistical classifier, and a large number of labeled images are needed to train the CNN model.

物体检测(object detection)算法(任务):除了需要判断图像中是否包含目标物体,还需要输出目标物体在图像中的位置。例如,如图5中的(b)所示,为物体检测任务的一种输出结果,定位出瓶子、杯子和立方体所在的位置。物体检测结果的输出通常是以包围盒的(bounding box)的形式,包围盒可以是2D的或者3D的。如图6所 中的(a)所示,为2D包围盒输出形式;如图6所中的(b)所示,为3D包围盒输出形式。与物体定位(object localization)不同的是,在物体定位中,通常只有一个或固定数目的目标,而物体检测更一般化,其图像中出现的目标种类和数目都不定。Object detection algorithm (task): In addition to determining whether the image contains a target object, it is also necessary to output the position of the target object in the image. For example, as shown in (b) of FIG. 5, it is an output result of an object detection task, and positions of bottles, cups, and cubes are located. The output of the object detection result is usually in the form of a bounding box, which can be 2D or 3D. As shown in (a) of FIG. 6, it is a 2D bounding box output form; as shown in (b) of FIG. 6, it is a 3D bounding box output form. Different from object localization, in object localization, there is usually only one or a fixed number of objects, while object detection is more general, and the types and number of objects appearing in the image are uncertain.

语义分割(semantic segmentation)算法(任务):相比物体检测任务,语义分割任务需要进一步判断图像中哪些像素属于哪个物体。如图5中的(c)所示,为语义分割的一种可能的输出结果,可以标识出瓶子、杯子和立方体所占的像素。但是,语义分割不区分属于相同类别的不同实例,比如无法区分图5中(c)中的三个立方体。Semantic segmentation algorithm (task): Compared to the object detection task, the semantic segmentation task needs to further determine which pixels in the image belong to which object. As shown in (c) of FIG. 5, as a possible output of semantic segmentation, the pixels occupied by bottles, cups, and cubes can be identified. However, semantic segmentation does not distinguish between different instances that belong to the same category, such as the inability to distinguish the three cubes in (c) in FIG. 5.

实例分割(instance segmentation)算法(任务):如图5中的(d)所示,相比语义分割,实例分割可以进一步区分出哪些像素属于哪个立方体,即区分出立方体1、立方体2和立方体3分别占据的像素。Instance segmentation algorithm (task): As shown in (d) of Figure 5, compared with semantic segmentation, instance segmentation can further distinguish which pixels belong to which cube, that is, cube 1, cube 2, and cube 3. Pixels respectively occupied.

可以理解的是,图像分类、物体检测、语意分割、实例分割等算法或任务的难度是逐渐升高的,相应的计算开销也逐渐增大。目前,通常采用CNN进行各种类型的图像处理,现有技术中各种类型的图像算法对应的CNN模型很多,本申请不做限定。It can be understood that the difficulty of algorithms or tasks such as image classification, object detection, semantic segmentation, and instance segmentation is gradually increasing, and the corresponding computing overhead is gradually increasing. At present, CNNs are generally used for various types of image processing. There are many CNN models corresponding to various types of image algorithms in the prior art, which is not limited in this application.

下面将结合本申请实施例中的附图,对本申请实施例中的技术方案进行描述。其中,在本申请的描述中,除非另有说明,“多个”是指两个或多于两个。另外,为了便于清楚描述本申请实施例的技术方案,在本申请的实施例中,采用了“第一”、“第二”等字样对功能和作用基本相同的相同项或相似项进行区分。本领域技术人员可以理解“第一”、“第二”等字样并不对数量和执行次序进行限定,并且“第一”、“第二”等字样也并不限定一定不同。The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application. Wherein, in the description of the present application, unless stated otherwise, "plurality" means two or more. In addition, in order to facilitate a clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, words such as “first” and “second” are used to distinguish the same or similar items having substantially the same functions and functions. Those skilled in the art can understand that the words "first", "second" and the like do not limit the number and execution order, and the words "first" and "second" are not necessarily different.

本申请实施例提供一种图像信息采集方法,如图7所示,包括:An embodiment of the present application provides a method for collecting image information, as shown in FIG. 7, including:

701、可移动装置采用第一位姿采集第一图像,以使第一图像包括目标物体。701. The movable device acquires a first image by using the first pose, so that the first image includes a target object.

可以理解的是,可移动装置可以根据预设的算法检测目标物体。预设算法包括但不限于图像检测处理中的图像分类(识别)、物体检测、语义分割、实例分割等算法。It can be understood that the movable device can detect the target object according to a preset algorithm. The preset algorithms include, but are not limited to, image classification (recognition), object detection, semantic segmentation, and instance segmentation in image detection processing.

进一步的,可移动装置还可以采用第一位姿采集目标物体的状态信息,目标物体的状态信息包括以下信息的至少一种:目标物体相对于可移动装置的距离信息、速度信息、移动方向信息。Further, the movable device may also use the first pose to collect the state information of the target object. The state information of the target object includes at least one of the following information: distance information, speed information, and direction information of the target object relative to the movable device. .

目标物体的状态信息可以是可移动装置通过至少一个(或至少一种)传感器采集得到的。例如,可以通过超声波雷达、激光雷达、毫米波雷达、红外线测量仪、双目摄像头、单目摄像头等传感器采集目标物体相对于可移动装置的距离信息;可以通过加速度传感器、激光雷达、GPS等传感器获得可移动装置相对地面的速度信息,再根据不同时间点获取到的距离信息计算目标物体的速度信息;可以通过双目摄像头、单目摄像头进行图像分析,或通过激光雷达获得点云数据来分析目标物体位移等方法,获取目标物体的移动方向信息。The state information of the target object may be acquired by the mobile device through at least one (or at least one) sensor. For example, you can use sensors such as ultrasonic radar, lidar, millimeter-wave radar, infrared measuring instrument, binocular camera, and monocular camera to collect distance information of the target object relative to the mobile device; you can use sensors such as acceleration sensors, lidar, and GPS Obtain the speed information of the mobile device relative to the ground, and then calculate the speed information of the target object based on the distance information obtained at different time points; the image analysis can be performed by the binocular camera and the monocular camera, or the point cloud data can be obtained by laser radar for analysis Target object displacement and other methods to obtain the moving direction information of the target object.

702、可移动装置确定第一图像是否满足预设条件。702. The movable device determines whether the first image meets a preset condition.

其中,预设条件包括以下条件的至少一个:目标物体在第一图像中占据的像素数量或像素比例的条件;第一图像的清晰度的条件;目标物体的特征的条件;第一图像的检测结果信任度的条件。Wherein, the preset conditions include at least one of the following conditions: the number of pixels or the ratio of the pixels occupied by the target object in the first image; the conditions of the sharpness of the first image; the conditions of the characteristics of the target object; Condition of the result trust.

在一种可能的实现方式中,当目标物体在第一图像中占据的像素数量或像素比例小于预设阈值(第四预设阈值)时,确定第一图像不满足预设条件。其中,目标物体 在第一图像中占据的像素数量或像素比例与目标物体投影到摄像头上的覆盖面积正相关。例如,对于人脸识别任务,一般需要保证人脸图像大于28x28像素。In a possible implementation manner, when the number of pixels or the pixel ratio occupied by the target object in the first image is less than a preset threshold (a fourth preset threshold), it is determined that the first image does not meet the preset condition. The number of pixels or pixel ratio occupied by the target object in the first image is positively related to the coverage area of the target object projected on the camera. For example, for face recognition tasks, it is generally necessary to ensure that the face image is larger than 28x28 pixels.

在一种可能的实现方式中,当确定第一图像的清晰度小于第五预设阈值时,确定第一图像不满足预设条件。第一图像的清晰度与第一图像的锐度正相关,第一图像的锐度可通过调制传递函数(modulation transfer function,MTF)、光学传递函数(optical transfer function,OTF)或空间频率响应(spatial frequency response,SFR)曲线来衡量。第一图像的清晰度与第一图像的噪声程度逆相关。第一图像的噪声程度可通过感光度或信噪比(Signal-to-noise ratio,SNR)来衡量。In a possible implementation manner, when it is determined that the sharpness of the first image is less than a fifth preset threshold, it is determined that the first image does not satisfy a preset condition. The sharpness of the first image is positively related to the sharpness of the first image. The sharpness of the first image can be determined by a modulation transfer function (MTF), an optical transfer function (OTF), or a spatial frequency response ( spatial frequency response (SFR) curve. The sharpness of the first image is inversely related to the noise level of the first image. The noise level of the first image can be measured by sensitivity or signal-to-noise ratio (SNR).

在一种可能的实现方式中,当第一图像的检测结果信任度小于第六预设阈值时,确定第一图像不满足预设条件。其中,第一图像的检测结果可以包括:图像中出现目标物体的概率、目标物体的相应位置以及对目标物体状态信息判断结果(例如当目标物体为人时,确定其是否受伤)等,第一图像的检测结果信任度可以是对上述概率、相应位置和/或状态信息判断结果的信任值。In a possible implementation manner, when the trust degree of the detection result of the first image is less than the sixth preset threshold, it is determined that the first image does not meet the preset condition. The detection result of the first image may include: the probability of the target object appearing in the image, the corresponding position of the target object, and the result of determining the status information of the target object (for example, when the target object is a person, determining whether it is injured), etc. The degree of trust in the detection result may be the trust value of the judgment result of the probability, corresponding position, and / or state information.

在一种可能的实现方式中,当确定目标物体包括第一特征时,确定第一图像不满足预设条件。其中,第一特征是预设的第二特征的父特征,或者说,预设的第二特征是第一特征的子特征。即,当目标物体具有第一特征时,目标物体可能也具有第二特征。举例来说,第一特征(例如,人)是第二特征(例如,儿童或成人)的父特征。又例如,第二特征(例如,拉布拉多犬)为第一特征(例如,狗)的子特征。当然,也可以预设第一特征的父特征,或第二特征的子特征。例如,假设目标物体为人,预设的第二特征为受伤,则可以规定第一特征包括流血,呈躺卧状或表情痛苦等等。其中,预设的第二特征可以是用户希望从目标物体检测到的特征。由于可移动装置对目标物体是否存在预设的第二特征的检测结果可能不太准确,因此可以同时确定目标物体是否包括第一特征。若目标物体包括第一特征,则第一图像不满足预设条件,从而可移动装置可以调整位姿重新采集图像信息进行判断,以便减少可移动装置对目标物体的特征误判的可能性。In a possible implementation manner, when it is determined that the target object includes the first feature, it is determined that the first image does not satisfy a preset condition. The first feature is a parent feature of the preset second feature, or the preset second feature is a child feature of the first feature. That is, when the target object has the first feature, the target object may also have the second feature. For example, a first feature (eg, a person) is a parent feature of a second feature (eg, a child or an adult). As another example, the second feature (eg, a Labrador dog) is a sub-feature of the first feature (eg, a dog). Of course, the parent feature of the first feature or the child feature of the second feature can also be preset. For example, assuming that the target object is a person and the preset second feature is injury, the first feature may be specified to include bleeding, lying down or painful expression. The preset second feature may be a feature that the user wishes to detect from the target object. Since the detection result of the movable device on whether the target object has the preset second feature may not be accurate, it may be determined whether the target object includes the first feature at the same time. If the target object includes the first feature, the first image does not satisfy the preset condition, so that the mobile device can adjust the posture and re-acquire the image information for judgment, so as to reduce the possibility of the mobile device misjudging the feature of the target object.

可以理解的是,可以通过对可移动装置上的分类器进行相应的训练使分类器获得识别上述特征的能力。进一步的,分类器可以判断第一图像中的目标物体是否同时包括多个特征。比如当第一图像中包括的目标物体为一条“拉布拉多犬”时,该目标物体可以同时包括“狗”、“短毛犬”等多个特征。It can be understood that the classifier on the mobile device can be trained to enable the classifier to recognize the above features. Further, the classifier can determine whether the target object in the first image includes multiple features at the same time. For example, when the target object included in the first image is a "Labrador dog", the target object may include multiple features such as "dog" and "short-haired dog" at the same time.

在一种可能的设计中,预设的条件还可以包括目标物体的朝向的条件。具体的,对于具有朝向的目标物体,当确定第一图像不包括目标物体的朝向的特征时,确定第一图像不满足预设条件。例如,假设目标物体为人且人的朝向为其面部正前方,那么需要确定是否采集到人的面部特征,若未采集到面部特征,确定不满足目标物体的朝向的条件。In a possible design, the preset condition may further include a condition of the orientation of the target object. Specifically, for a target object with an orientation, when it is determined that the first image does not include a feature of the orientation of the target object, it is determined that the first image does not satisfy a preset condition. For example, assuming that the target object is a person and the orientation of the person is directly in front of their face, then it is necessary to determine whether the facial features of the person are collected, and if no facial features are collected, determine that the condition of the orientation of the target object is not satisfied.

需要说明的是,可以对目标物体是否具有朝向性进行预先定义。比如,可以定义人、椅子、电视机等有朝向性;定义水桶、桌子等没有朝向性。在检测出图像中包含上述预先定义过的物体后,再通过查找表的方式查看这些物体是否具备朝向性。对于具有朝向性的物体,可以定义相应的朝向参考区域。物体的状态中的至少一种取决于该物体在该朝向参考区域的状态。例如,可以将人脸定义为人的朝向参考区域,当判 断人是否处于受伤这一状态,可以通过识别人面部表情是否痛苦来进行判断。可以通过对目标物体进行3D建模并与预先设定的物体参考模型进行朝向参考区域匹配。也可以对目标物体本身朝向参考区域直接进行检测。比如,若目标物体为人,其朝向参考区域为人脸,则可以采用人的位置进行人脸识别来获得人脸的具体位置,再通过头部朝向检测、眼睛位置检测等技术确定人脸的朝向。It should be noted that whether the target object has a directivity can be defined in advance. For example, you can define that people, chairs, televisions, etc. are directional; define buckets, tables, etc. that are not directional. After detecting that the image contains the above-defined objects, a look-up table is used to check whether the objects have orientation. For directional objects, you can define the corresponding directional reference area. At least one of the states of the object depends on the state of the object in the orientation-oriented area. For example, a human face can be defined as a person's facing reference area. When determining whether a person is injured, it can be judged by identifying whether a person's facial expression is painful. 3D modeling of the target object and matching with a preset object reference model toward the reference region can be performed. It is also possible to directly detect the target object towards the reference area. For example, if the target object is a person and its reference area is a face, the position of the person may be used for face recognition to obtain the specific position of the face, and then the head orientation and eye position detection may be used to determine the face orientation.

进一步的,可移动装置可以确定目标物体的状态信息是否满足相应的条件。例如,目标物体相对于可移动装置的距离应该小于第七预设阈值。第七预设阈值的大小与摄像传感器的焦距以及摄像传感器的感光元件分辨率有关。Further, the movable device may determine whether the state information of the target object satisfies a corresponding condition. For example, the distance of the target object relative to the movable device should be less than the seventh preset threshold. The magnitude of the seventh preset threshold is related to the focal length of the imaging sensor and the resolution of the light receiving element of the imaging sensor.

703、当第一图像不满足预设条件时,可移动装置将第一位姿调整为第二位姿。703. When the first image does not satisfy the preset condition, the movable device adjusts the first pose to the second pose.

在一种可能的设计中,当可移动装置确定第一图像不满足预设条件时,可移动装置可以根据第一图像和目标物体的状态信息将第一位姿调整为第二位姿。具体的,可移动装置可以根据第一图像和目标物体的状态信息确定可移动装置的移动路线和姿态角。而后,根据移动路线和姿态角将第一位姿调整为第二位姿。In a possible design, when the movable device determines that the first image does not satisfy the preset condition, the movable device may adjust the first pose to the second pose according to the first image and the state information of the target object. Specifically, the movable device may determine a moving route and a posture angle of the movable device according to the first image and the state information of the target object. Then, the first pose is adjusted to the second pose according to the movement route and the attitude angle.

若目标物体存在至少两个外观不同的面,至少两个外观不同的面包括第一面,移动路线用于指示可移动装置从当前位置移动到第一目标空间区域;第一目标空间区域包括第一面的法线方向与可移动装置和目标物体的连线方向的夹角满足预设阈值(第一预设阈值),且可移动装置与目标物体之间的距离满足第二预设阈值的空间区域。即当目标物体具备朝向性时,第一目标空间区域包括目标物体的朝向方向与可移动装置和目标物体的连线方向的夹角满足第一预设阈值,且可移动装置与目标物体之间的距离满足第二预设阈值的空间区域。If the target object has at least two faces with different appearances, the at least two faces with different appearances include the first face, and the movement route is used to instruct the movable device to move from the current position to the first target space area; the first target space area includes the first The included angle between the normal direction of one side and the connection direction between the movable device and the target object satisfies a preset threshold (a first preset threshold), and the distance between the movable device and the target object satisfies a second preset threshold Space area. That is, when the target object is directional, the first target space region includes the included angle between the direction of the target object and the connection direction of the mobile device and the target object to meet the first preset threshold, and the distance between the mobile device and the target object A spatial region whose distance satisfies a second preset threshold.

举例来说,如图8所示,假设目标物体为人,可以以人的面部中心为坐标原点建立三维坐标,X轴、Y轴和Z轴相互垂直,且Y轴与人的面部正前方的方向平行。假设人的朝向为人的面部正前方的方向,即Y轴指示的方向。第一目标空间区域可以包括Y轴的方向与可移动装置和人的连线方向的夹角为a,且与坐标原点的距离在d1与d2之间的空间区域。其中,a、d1及d2可以是采用人的面部大小以及可移动装置的摄像传感器的参数(例如,焦距、分辨率、最小对焦距离等)得到的。For example, as shown in FIG. 8, assuming that the target object is a person, a three-dimensional coordinate can be established with the center of the face of the person as the origin of the coordinate, the X axis, the Y axis, and the Z axis are perpendicular to each other, and the Y axis is directly in front of the person's face parallel. It is assumed that the direction of the person is the direction directly in front of the person's face, that is, the direction indicated by the Y axis. The first target space region may include a space region where the angle between the direction of the Y axis and the connection direction of the movable device and the person is a, and the distance from the coordinate origin is between d1 and d2. Among them, a, d1, and d2 can be obtained by using the size of a person's face and parameters (for example, focal length, resolution, minimum focusing distance, etc.) of the imaging sensor of the mobile device.

若目标物体不存在外观不同的面,即目标物体不具备朝向性或未检测到目标物体的朝向时,可移动装置可以根据移动路线从当前位置移动到第二目标空间区域,第二目标空间区域包括目标物体与可移动装置之间的距离满足第三预设阈值的空间区域。If the target object does not have faces with different appearances, that is, when the target object does not have orientation or the orientation of the target object is not detected, the movable device can move from the current position to the second target space area and the second target space area according to the movement route A space region including a distance between a target object and a movable device that satisfies a third preset threshold.

其中,可移动装置向第一目标空间区域或第二目标空间区域运动的方式例如但不限于以下方式:沿着可移动装置朝向目标物体的方向或重力方向,进行水平和/或垂直移动,以缩短可移动装置与目标物体之间的距离。或者,以目标物体为圆心,维持无人机高度,保持与物体距离,围绕物体旋转。或者,降低或升高无人机高度,围绕物体旋转。在移动过程中,可移动装置可以通过相应的传感器避让遮挡物。例如,可移动装置可以通过图像传感器、超声波传感器、激光雷达、毫米波雷达、红外线摄像头等传感器感知移动途径上是否存在障碍物,从而进行避让。The manner in which the movable device moves toward the first target space region or the second target space region is, for example, but not limited to, the following manners: horizontal and / or vertical movement along the direction of the movable device toward the target object or the direction of gravity, to Reduce the distance between the mobile device and the target object. Or, take the target object as the center, maintain the height of the drone, keep a distance from the object, and rotate around the object. Alternatively, lower or raise the drone's height and rotate around the object. During the movement, the movable device can avoid the obstruction by corresponding sensors. For example, the mobile device may use an image sensor, an ultrasonic sensor, a lidar, a millimeter-wave radar, or an infrared camera to detect whether there is an obstacle on the moving path, thereby avoiding.

可以理解的是,可移动装置向第一目标空间区域或第二目标空间区域运动的过程中,或者到达第一目标空间区域或第二目标空间区域中的任一位置后,可移动装置的机体和摄像头可以分别改变其相对于目标物体的角度,以便获取更符合预设条件的图 像信息。另外,为了保证可移动装置在移动过程中的稳定性,其俯仰角可以自动调节,相关方法可以参考现有技术。It can be understood that during the movement of the movable device toward the first target space area or the second target space area, or after reaching any position in the first target space area or the second target space area, the body of the mobile device The camera and the camera can respectively change their angles with respect to the target object in order to obtain image information that better meets the preset conditions. In addition, in order to ensure the stability of the movable device during the movement, its pitch angle can be adjusted automatically. For related methods, refer to the prior art.

704、可移动装置采用第二位姿采集第二图像。704. The movable device acquires a second image by using the second pose.

在一种可能的设计中,可移动装置还可以采用第二位姿采集目标物体的状态信息,目标物体的状态信息包括以下信息的至少一种:目标物体相对于可移动装置的距离信息、速度信息和移动方向信息。In a possible design, the movable device may also use the second pose to collect the state information of the target object, and the state information of the target object includes at least one of the following information: distance information and speed of the target object relative to the movable device Information and moving direction information.

705、可移动装置确定第二图像是否满足预设条件。705. The movable device determines whether the second image meets a preset condition.

具体过程可以参考步骤702。The specific process may refer to step 702.

706、当第二图像不满足预设条件时,可移动装置根据第一图像和第二图像,将第二位姿调整为第三位姿。706. When the second image does not satisfy the preset condition, the movable device adjusts the second pose to the third pose according to the first image and the second image.

具体过程可以参考步骤703。For the specific process, refer to step 703.

在一种可能的设计中,若可移动装置采用第一位姿和采用第二位姿采集目标物体的状态信息,可移动装置可以根据第一图像、采用第一位姿采集到的目标物体的状态信息、第二图像以及采用第二位姿采集到的目标物体的状态信息,将可移动装置将第二位姿调整为第三位姿。也就是说,可移动装置可以根据多次(两次或两次以上)采集到的图像信息调整位姿,可以更准确的调整图像信息获取角度和位置,从而采集到更准确的图像信息。In a possible design, if the mobile device adopts the first pose and the second pose to collect the state information of the target object, the mobile device can use the first image and the The state information, the second image, and the state information of the target object collected by the second pose adjust the movable device to the second pose to the third pose. That is, the movable device can adjust the pose based on the image information collected multiple times (two or more times), and can more accurately adjust the image information acquisition angle and position, thereby acquiring more accurate image information.

707、可移动装置采用第三位姿采集第三图像。707. The movable device uses a third pose to acquire a third image.

可移动装置可以进一步确定第三图像是否满足预设条件,具体过程可以参考步骤702。The movable device may further determine whether the third image meets a preset condition. For a specific process, refer to step 702.

在一种可能的设计中,当采用第三位姿采集到的第三图像满足预设条件时,可移动装置可以不再调整位姿。若可移动装置多次改变位姿后采集到的图像仍无法满足预设条件,可以将最后一次采集到的图像信息作为最终的图像信息。或者,可以为预设条件包括的至少一个条件标记重要程度,将满足最重要的条件的图像信息作为最终的图像信息。In a possible design, when the third image acquired by using the third pose satisfies a preset condition, the movable device may no longer adjust the pose. If the image acquired after the mobile device changes the posture multiple times still fails to meet the preset conditions, the image information acquired last time may be used as the final image information. Alternatively, the degree of importance may be marked for at least one condition included in the preset condition, and image information satisfying the most important condition is used as final image information.

基于上述方法,当可移动装置确定该第一图像不满足预设条件时,该可移动装置可以将第一位姿调整为第二位姿;并采用该第二位姿采集第二图像。也就是说,当可移动装置采用第一位姿采集到的第一图像不满足预设条件时,可移动装置可以将第一位姿调整为第二位姿,即能够自动调整图像信息获取角度和位置,并采用该第二位姿采集更准确的图像信息。Based on the above method, when the movable device determines that the first image does not satisfy a preset condition, the movable device can adjust the first pose to a second pose; and use the second pose to acquire a second image. That is, when the first image collected by the movable device using the first pose does not satisfy the preset condition, the movable device can adjust the first pose to the second pose, that is, it can automatically adjust the image information acquisition angle And position, and use this second pose to collect more accurate image information.

进一步的,当第二图像不满足预设条件时,可移动装置可以将第二位姿调整为第三位姿。若可移动装置多次改变位姿后采集到的图像仍无法满足预设条件,可以将最后一次采集到的图像信息作为最终的图像信息。Further, when the second image does not satisfy the preset condition, the movable device may adjust the second pose to a third pose. If the image acquired after the mobile device changes the posture multiple times still fails to meet the preset conditions, the image information acquired last time may be used as the final image information.

上述主要从可移动装置的角度对本申请实施例提供的方案进行了介绍。可以理解的是,可移动装置为了实现上述功能,其包括了执行各个功能相应的硬件结构和/或软件模块。本领域技术人员应该很容易意识到,结合本文中所公开的实施例描述的算法步骤,本申请能够以硬件或硬件和软件的结合形式来实现。某个功能究竟以硬件还是软件驱动硬件的方式来执行,取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能,但是这种实现不应 认为超出本申请的范围。The above mainly introduces the solution provided by the embodiment of the present application from the perspective of a movable device. It can be understood that, in order to implement the above functions, the mobile device includes a hardware structure and / or a software module corresponding to each function. Those skilled in the art should easily realize that, in combination with the algorithm steps described in the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and software. Whether a certain function is performed by hardware or software-driven hardware depends on the specific application and design constraints of the technical solution. A professional technician may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

本申请实施例可以根据上述方法示例对可移动装置进行功能模块的划分,例如,可以对应各个功能划分各个功能模块,也可以将两个或两个以上的功能集成在一个处理模块中。上述集成的模块既可以采用硬件的形式实现,也可以采用软件功能模块的形式实现。需要说明的是,本申请实施例中对模块的划分是示意性的,仅仅为一种逻辑功能划分,实际实现时可以有另外的划分方式。In the embodiment of the present application, functional modules may be divided into mobile devices according to the foregoing method examples. For example, each functional module may be divided corresponding to each function, or two or more functions may be integrated into one processing module. The above integrated modules can be implemented in the form of hardware or software functional modules. It should be noted that the division of the modules in the embodiments of the present application is schematic, and is only a logical function division. In actual implementation, there may be another division manner.

在采用对应各个功能划分各个功能模块的情况下,图9示出了上述实施例中所涉及的可移动装置9的一种可能的结构示意图,可移动装置包括:采集单元901、确定单元902和调整单元903。采集单元901用于支持可移动装置执行图7中的过程701、704和707。确定单元902用于支持可移动装置执行图7中的过程702、703、705和706。调整单元903用于支持可移动装置执行图7中的过程703和706。其中,上述方法实施例涉及的各步骤的所有相关内容均可以援引到对应功能模块的功能描述,在此不再赘述。In a case where each functional module is divided according to each function, FIG. 9 shows a possible structural schematic diagram of the movable device 9 involved in the foregoing embodiment. The movable device includes: a collecting unit 901, a determining unit 902, and Adjusting unit 903. The collecting unit 901 is configured to support the mobile device to perform the processes 701, 704, and 707 in FIG. The determining unit 902 is configured to support the mobile device to perform the processes 702, 703, 705, and 706 in FIG. The adjusting unit 903 is configured to support the mobile device to perform the processes 703 and 706 in FIG. 7. Wherein, all relevant content of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module, which will not be repeated here.

结合本申请公开内容所描述的方法或者算法的步骤可以硬件的方式来实现,也可以是由处理器执行软件指令的方式来实现。软件指令可以由相应的软件模块组成,软件模块可以被存放于RAM、闪存、ROM、EPROM、EEPROM、寄存器、硬盘、移动硬盘、只读光盘或者本领域熟知的任何其它形式的存储介质中。一种示例性的存储介质耦合至处理器,从而使处理器能够从该存储介质读取信息,且可向该存储介质写入信息。当然,存储介质也可以是处理器的组成部分。处理器和存储介质可以位于ASIC中。另外,该ASIC可以位于核心网接口设备中。当然,处理器和存储介质也可以作为分立组件存在于核心网接口设备中。The steps of the method or algorithm described in combination with the disclosure of this application may be implemented in a hardware manner, or may be implemented in a manner that a processor executes software instructions. Software instructions may be composed of corresponding software modules, and the software modules may be stored in RAM, flash memory, ROM, EPROM, EEPROM, registers, hard disk, mobile hard disk, read-only optical disk, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor such that the processor can read information from, and write information to, the storage medium. Of course, the storage medium may also be an integral part of the processor. The processor and the storage medium may reside in an ASIC. In addition, the ASIC can be located in a core network interface device. Of course, the processor and the storage medium can also exist as discrete components in the core network interface device.

本领域技术人员应该可以意识到,在上述一个或多个示例中,本申请所描述的功能可以用硬件、软件、固件或它们的任意组合来实现。当使用软件实现时,可以将这些功能存储在可移动装置可读介质中或者作为可移动装置可读介质上的一个或多个指令或代码进行传输。可移动装置可读介质包括可移动装置存储介质和通信介质,其中通信介质包括便于从一个地方向另一个地方传送可移动装置程序的任何介质。存储介质可以是通用或专用可移动装置能够存取的任何可用介质。Those skilled in the art should be aware that, in one or more of the above examples, the functions described in this application may be implemented by hardware, software, firmware, or any combination thereof. When implemented in software, these functions may be stored in a removable device-readable medium or transmitted as one or more instructions or code on a removable device-readable medium. Removable device readable media include removable device storage media and communication media, where communication media includes any medium that facilitates transfer of a removable device program from one place to another. A storage medium may be any available medium that can be accessed by a general purpose or special purpose removable device.

以上所述的具体实施方式,对本申请的目的、技术方案和有益效果进行了进一步详细说明,所应理解的是,以上所述仅为本申请的具体实施方式而已,并不用于限定本申请的保护范围,凡在本申请的技术方案的基础之上,所做的任何修改、等同替换、改进等,均应包括在本申请的保护范围之内。The specific implementation manners described above further describe the purpose, technical solution, and beneficial effects of the present application. It should be understood that the foregoing descriptions are merely specific implementation manners of the present application, and are not intended to limit the present application. The scope of protection, any modification, equivalent replacement, or improvement made on the basis of the technical solution of this application shall be included in the scope of protection of this application.

Claims (16)

  1. 一种图像信息采集方法,用于可移动装置,其特征在于,包括:An image information collection method for a mobile device is characterized in that it includes:
    采用第一位姿采集第一图像,以使所述第一图像包括目标物体;Acquiring a first image using a first pose, so that the first image includes a target object;
    确定所述第一图像是否满足预设条件;Determining whether the first image meets a preset condition;
    当所述第一图像不满足预设条件时,将所述第一位姿调整为第二位姿;Adjusting the first pose to a second pose when the first image does not satisfy a preset condition;
    采用所述第二位姿采集第二图像。A second image is acquired using the second pose.
  2. 根据权利要求1所述的图像信息采集方法,其特征在于,还包括:The image information collection method according to claim 1, further comprising:
    采用所述第一位姿和/或所述第二位姿采集所述目标物体的状态信息;Adopting the first pose and / or the second pose to collect state information of the target object;
    对应的,所述将所述第一位姿调整为第二位姿,包括:Correspondingly, the adjusting the first posture to a second posture includes:
    根据所述第一图像和所述目标物体的状态信息将所述第一位姿调整为第二位姿;Adjusting the first pose to a second pose according to the first image and state information of the target object;
    其中,所述目标物体的状态信息包括以下信息的至少一种:所述目标物体相对于所述可移动装置的距离信息、速度信息和移动方向信息。The state information of the target object includes at least one of the following information: distance information, speed information, and moving direction information of the target object with respect to the movable device.
  3. 根据权利要求2所述的图像信息采集方法,其特征在于,所述根据所述第一图像和所述目标物体的状态信息将所述第一位姿调整为第二位姿,包括:The image information collection method according to claim 2, wherein the adjusting the first pose to a second pose according to the first image and state information of the target object comprises:
    根据所述第一图像和所述目标物体的状态信息确定所述可移动装置的移动路线和姿态角;Determining a movement route and an attitude angle of the movable device according to the first image and state information of the target object;
    根据所述移动路线和所述姿态角将所述第一位姿调整为所述第二位姿。Adjusting the first pose to the second pose according to the moving route and the attitude angle.
  4. 根据权利要求3所述的图像信息采集方法,其特征在于,The image information collection method according to claim 3, wherein:
    若所述目标物体存在至少两个外观不同的面,所述至少两个外观不同的面包括第一面,所述移动路线用于指示所述可移动装置从当前位置移动到第一目标空间区域;所述第一目标空间区域包括所述第一面的法线方向与所述可移动装置和所述目标物体的连线方向的夹角满足第一预设阈值,且所述可移动装置与所述目标物体之间的距离满足第二预设阈值的空间区域。If the target object has at least two faces with different appearances, the at least two faces with different appearances include a first face, and the movement route is used to instruct the movable device to move from a current position to a first target space area ; The first target space region includes an included angle between a normal direction of the first surface and a line connecting direction of the movable device and the target object, which meets a first preset threshold, and the movable device and A space area where the distance between the target objects satisfies a second preset threshold.
  5. 根据权利要求3所述的图像信息采集方法,其特征在于,The image information collection method according to claim 3, wherein:
    若所述目标物体不存在外观不同的面,所述移动路线用于指示所述可移动装置从当前位置移动到第二目标空间区域,所述第二目标空间区域包括所述目标物体与所述可移动装置之间的距离满足第三预设阈值的空间区域。If there are no faces with different appearances on the target object, the moving route is used to instruct the movable device to move from the current position to a second target space area, where the second target space area includes the target object and the A space region where the distance between the movable devices satisfies a third preset threshold.
  6. 根据权利要求3-5任一项所述的图像信息采集方法,其特征在于,所述可移动装置包括摄像头和机体,所述姿态角包括所述机体相对于所述目标物体的角度和所述摄像头相对于所述目标物体的角度。The image information collection method according to any one of claims 3-5, wherein the movable device includes a camera and a body, and the attitude angle includes an angle of the body with respect to the target object and the The angle of the camera relative to the target object.
  7. 根据权利要求1-6任一项所述的图像信息采集方法,其特征在于,所述确定所述第一图像不满足预设条件包括:The method for acquiring image information according to any one of claims 1-6, wherein the determining that the first image does not satisfy a preset condition includes:
    确定所述目标物体在所述第一图像中占据的像素数量或像素比例小于第四预设阈值;和/或Determine that the number of pixels or the ratio of pixels occupied by the target object in the first image is less than a fourth preset threshold; and / or
    确定所述第一图像的清晰度小于第五预设阈值;和/或Determine that the sharpness of the first image is less than a fifth preset threshold; and / or
    确定所述第一图像的检测结果信任度小于第六预设阈值;和/或Determining that the trust degree of the detection result of the first image is less than a sixth preset threshold; and / or
    确定所述目标物体包括第一特征,所述第一特征是预设的第二特征的父特征。It is determined that the target object includes a first feature, and the first feature is a parent feature of a preset second feature.
  8. 根据权利要求1-7任一项所述的图像信息采集方法,其特征在于,所述方法还包括:The method for collecting image information according to any one of claims 1-7, wherein the method further comprises:
    确定所述第二图像是否满足所述预设条件;Determining whether the second image satisfies the preset condition;
    当所述第二图像不满足所述预设条件时,根据所述第一图像和所述第二图像,将所述第二位姿调整为第三位姿;When the second image does not satisfy the preset condition, adjusting the second pose to a third pose according to the first image and the second image;
    采用所述第三位姿采集第三图像。A third image is acquired using the third pose.
  9. 一种可移动装置,其特征在于,包括:A movable device, comprising:
    采集单元,用于采用第一位姿采集第一图像,以使所述第一图像包括目标物体;An acquisition unit, configured to acquire a first image using a first pose, so that the first image includes a target object;
    确定单元,用于确定所述第一图像是否满足预设条件;A determining unit, configured to determine whether the first image meets a preset condition;
    调整单元,用于当所述第一图像不满足预设条件时,将所述第一位姿调整为第二位姿;An adjustment unit, configured to adjust the first pose to a second pose when the first image does not satisfy a preset condition;
    所述采集单元,还用于采用所述第二位姿采集第二图像。The acquisition unit is further configured to acquire a second image by using the second pose.
  10. 根据权利要求9所述的可移动装置,其特征在于,所述采集单元还用于:The movable device according to claim 9, wherein the collecting unit is further configured to:
    采用所述第一位姿和/或所述第二位姿采集所述目标物体的状态信息;Adopting the first pose and / or the second pose to collect state information of the target object;
    对应的,所述调整单元用于:Correspondingly, the adjustment unit is configured to:
    根据所述第一图像和所述目标物体的状态信息将所述第一位姿调整为第二位姿;Adjusting the first pose to a second pose according to the first image and state information of the target object;
    其中,所述目标物体的状态信息包括以下信息的至少一种:所述目标物体相对于所述可移动装置的距离信息、速度信息和移动方向信息。The state information of the target object includes at least one of the following information: distance information, speed information, and moving direction information of the target object with respect to the movable device.
  11. 根据权利要求10所述的可移动装置,其特征在于,所述调整单元用于:The movable device according to claim 10, wherein the adjustment unit is configured to:
    根据所述第一图像和所述目标物体的状态信息确定所述可移动装置的移动路线和姿态角;Determining a movement route and an attitude angle of the movable device according to the first image and state information of the target object;
    根据所述移动路线和所述姿态角将所述第一位姿调整为所述第二位姿。Adjusting the first pose to the second pose according to the moving route and the attitude angle.
  12. 根据权利要求11所述的可移动装置,其特征在于,The movable device according to claim 11, wherein:
    若所述目标物体存在至少两个外观不同的面,所述至少两个外观不同的面包括第一面,所述移动路线用于指示所述可移动装置从当前位置移动到第一目标空间区域;所述第一目标空间区域包括所述第一面的法线方向与所述可移动装置和所述目标物体的连线方向的夹角满足第一预设阈值,且所述可移动装置与所述目标物体之间的距离满足第二预设阈值的空间区域。If the target object has at least two faces with different appearances, the at least two faces with different appearances include a first face, and the movement route is used to instruct the movable device to move from a current position to a first target space area ; The first target space region includes an included angle between a normal direction of the first surface and a line connecting direction of the movable device and the target object, which meets a first preset threshold, and the movable device and A space area where the distance between the target objects satisfies a second preset threshold.
  13. 根据权利要求11所述的可移动装置,其特征在于,The movable device according to claim 11, wherein:
    若所述目标物体不存在外观不同的面,所述移动路线用于指示所述可移动装置从当前位置移动到第二目标空间区域,所述第二目标空间区域包括所述目标物体与所述可移动装置之间的距离满足第三预设阈值的空间区域。If there are no faces with different appearances on the target object, the moving route is used to instruct the movable device to move from the current position to a second target space area, where the second target space area includes the target object and the A space region where the distance between the movable devices satisfies a third preset threshold.
  14. 根据权利要求11-13任一项所述的可移动装置,其特征在于,所述可移动装置包括摄像头和机体,所述姿态角包括所述机体相对于所述目标物体的角度和所述摄像头相对于所述目标物体的角度。The movable device according to any one of claims 11-13, wherein the movable device includes a camera and a body, and the attitude angle includes an angle of the body with respect to the target object and the camera The angle relative to the target object.
  15. 根据权利要求9-14任一项所述的可移动装置,其特征在于,所述确定单元确定所述第一图像不满足预设条件包括:The movable device according to any one of claims 9 to 14, wherein the determining unit determines that the first image does not satisfy a preset condition, including:
    所述确定单元确定所述目标物体在所述第一图像中占据的像素数量或像素比例小于第四预设阈值;和/或The determining unit determines that the number of pixels or the pixel ratio occupied by the target object in the first image is less than a fourth preset threshold; and / or
    所述确定单元确定所述第一图像的清晰度小于第五预设阈值;和/或The determining unit determines that the sharpness of the first image is less than a fifth preset threshold; and / or
    所述确定单元确定所述第一图像的检测结果信任度小于第六预设阈值;和/或The determining unit determines that the trust degree of the detection result of the first image is less than a sixth preset threshold; and / or
    所述确定单元确定所述目标物体包括第一特征,所述第一特征是预设的第二特征的父特征。The determining unit determines that the target object includes a first feature, and the first feature is a parent feature of a preset second feature.
  16. 根据权利要求9-15任一项所述的可移动装置,其特征在于,所述确定单元还用于:The movable device according to any one of claims 9 to 15, wherein the determining unit is further configured to:
    确定所述第二图像是否满足所述预设条件;Determining whether the second image satisfies the preset condition;
    所述调整单元还用于,当所述第二图像不满足所述预设条件时,根据所述第一图像和所述第二图像,将所述第二位姿调整为第三位姿;The adjusting unit is further configured to adjust the second pose to a third pose according to the first image and the second image when the second image does not satisfy the preset condition;
    所述采集单元,还用于采用所述第三位姿采集第三图像。The acquisition unit is further configured to acquire a third image by using the third pose.
PCT/CN2019/077355 2018-06-15 2019-03-07 Image information acquisition method and device WO2019237775A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201810626611.5A CN110609562A (en) 2018-06-15 2018-06-15 Image information acquisition method and device
CN201810626611.5 2018-06-15

Publications (1)

Publication Number Publication Date
WO2019237775A1 true WO2019237775A1 (en) 2019-12-19

Family

ID=68842750

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/077355 WO2019237775A1 (en) 2018-06-15 2019-03-07 Image information acquisition method and device

Country Status (2)

Country Link
CN (1) CN110609562A (en)
WO (1) WO2019237775A1 (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105512643A (en) * 2016-01-06 2016-04-20 北京二郎神科技有限公司 Image acquisition method and device
CN106092054A (en) * 2016-05-30 2016-11-09 广东能飞航空科技发展有限公司 A kind of power circuit identification precise positioning air navigation aid
CN106973221A (en) * 2017-02-24 2017-07-21 北京大学 Unmanned plane image capture method and system based on aesthetic evaluation
CN107664996A (en) * 2016-07-28 2018-02-06 深圳曼塔智能科技有限公司 UAV Intelligent image pickup method and its UAS

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102542299B (en) * 2011-12-07 2015-03-25 惠州Tcl移动通信有限公司 Face recognition method, device and mobile terminal capable of recognizing face
CN102831392B (en) * 2012-07-09 2015-06-24 哈尔滨工业大学 Device for remote iris tracking and acquisition, and method thereof
CN107703963B (en) * 2014-07-30 2020-12-01 深圳市大疆创新科技有限公司 Target tracking system and method
CN105844202A (en) * 2015-01-12 2016-08-10 芋头科技(杭州)有限公司 Image recognition system and method
CN205015741U (en) * 2015-10-15 2016-02-03 杨珊珊 Companion flies unmanned aerial vehicle system of taking photo by plane and control cabinet and unmanned aerial vehicle thereof
CN205353774U (en) * 2015-10-15 2016-06-29 杨珊珊 Accompany unmanned aerial vehicle system of taking photo by plane of shooing aircraft
CN105446351B (en) * 2015-11-16 2018-03-16 杭州码全信息科技有限公司 It is a kind of can lock onto target Qu Yu lookout the unmanned airship system based on independent navigation
CN105867362A (en) * 2016-04-20 2016-08-17 北京博瑞爱飞科技发展有限公司 Terminal equipment and control system of unmanned aerial vehicle
CN105929850B (en) * 2016-05-18 2018-10-19 中国计量大学 A kind of UAV system and method with lasting locking and tracking target capability
CN107426490A (en) * 2017-05-16 2017-12-01 深圳市金立通信设备有限公司 A kind of photographic method and terminal

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105512643A (en) * 2016-01-06 2016-04-20 北京二郎神科技有限公司 Image acquisition method and device
CN106092054A (en) * 2016-05-30 2016-11-09 广东能飞航空科技发展有限公司 A kind of power circuit identification precise positioning air navigation aid
CN107664996A (en) * 2016-07-28 2018-02-06 深圳曼塔智能科技有限公司 UAV Intelligent image pickup method and its UAS
CN106973221A (en) * 2017-02-24 2017-07-21 北京大学 Unmanned plane image capture method and system based on aesthetic evaluation

Also Published As

Publication number Publication date
CN110609562A (en) 2019-12-24

Similar Documents

Publication Publication Date Title
Xu et al. Pointfusion: Deep sensor fusion for 3d bounding box estimation
US10712747B2 (en) Data processing method, apparatus and terminal
JP6288221B2 (en) Enhanced layer-based object detection by deep convolutional neural networks
US9778662B2 (en) Camera configuration on movable objects
Lin et al. Autonomous aerial navigation using monocular visual‐inertial fusion
US10102429B2 (en) Systems and methods for capturing images and annotating the captured images with information
US9435911B2 (en) Visual-based obstacle detection method and apparatus for mobile robot
JP2019527832A (en) System and method for accurate localization and mapping
US9177481B2 (en) Semantics based safe landing area detection for an unmanned vehicle
Kumar et al. A LiDAR and IMU integrated indoor navigation system for UAVs and its application in real-time pipeline classification
US10475209B2 (en) Camera calibration
JP6571274B2 (en) System and method for laser depth map sampling
Liang et al. Image based localization in indoor environments
Rodríguez-Canosa et al. A real-time method to detect and track moving objects (DATMO) from unmanned aerial vehicles (UAVs) using a single camera
Fraundorfer et al. Vision-based autonomous mapping and exploration using a quadrotor MAV
Bills et al. Autonomous MAV flight in indoor environments using single image perspective cues
WO2017045251A1 (en) Systems and methods for uav interactive instructions and control
US8818043B2 (en) Traffic signal mapping and detection
Krajník et al. A practical multirobot localization system
Rangesh et al. No blind spots: Full-surround multi-object tracking for autonomous vehicles using cameras and lidars
Yang et al. An onboard monocular vision system for autonomous takeoff, hovering and landing of a micro aerial vehicle
US20170075356A1 (en) Classifying objects detected by 3d sensors for autonomous vehicle operation
Levinson et al. Traffic light mapping, localization, and state detection for autonomous vehicles
Steder et al. Place recognition in 3D scans using a combination of bag of words and point feature based relative pose estimation
US9996731B2 (en) Human head detection in depth images

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19818898

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE