WO2020037553A1 - 图像处理方法及装置、移动设备 - Google Patents

图像处理方法及装置、移动设备 Download PDF

Info

Publication number
WO2020037553A1
WO2020037553A1 PCT/CN2018/101745 CN2018101745W WO2020037553A1 WO 2020037553 A1 WO2020037553 A1 WO 2020037553A1 CN 2018101745 W CN2018101745 W CN 2018101745W WO 2020037553 A1 WO2020037553 A1 WO 2020037553A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
target
map
image processing
tracked
Prior art date
Application number
PCT/CN2018/101745
Other languages
English (en)
French (fr)
Inventor
吴博
刘昂
张立天
Original Assignee
深圳市大疆创新科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市大疆创新科技有限公司 filed Critical 深圳市大疆创新科技有限公司
Priority to CN201880040265.0A priority Critical patent/CN110892449A/zh
Priority to PCT/CN2018/101745 priority patent/WO2020037553A1/zh
Publication of WO2020037553A1 publication Critical patent/WO2020037553A1/zh
Priority to US17/166,977 priority patent/US20210156697A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/38—Electronic maps specially adapted for navigation; Updating thereof
    • G01C21/3804—Creation or updating of map data
    • G01C21/3833—Creation or updating of map data characterised by the source of data
    • G01C21/3837—Data obtained from a single source
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/26—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network
    • G01C21/28—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network with correlation of data from several navigational instruments
    • G01C21/30—Map- or contour-matching
    • G01C21/32—Structuring or formatting of map data
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/38—Electronic maps specially adapted for navigation; Updating thereof
    • G01C21/3804—Creation or updating of map data
    • G01C21/3833—Creation or updating of map data characterised by the source of data
    • G01C21/3848—Data obtained from both position sensors and additional sensors
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0212—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory
    • G05D1/0219—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory ensuring the processing of the whole working surface
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0231—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
    • G05D1/0246—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0268—Control of position or course in two dimensions specially adapted to land vehicles using internal positioning means
    • G05D1/0274—Control of position or course in two dimensions specially adapted to land vehicles using internal positioning means using mapping information stored in a memory device
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/23—Clustering techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G06T7/507—Depth or shape recovery from shading
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/004—Artificial life, i.e. computing arrangements simulating life
    • G06N3/008—Artificial life, i.e. computing arrangements simulating life based on physical entities controlled by simulated intelligence so as to replicate intelligent life forms, e.g. based on robots replicating pets or humans in their appearance or behaviour
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]

Definitions

  • the present application relates to the field of image processing technologies, and in particular, to an image processing method and device, and a mobile device.
  • the robot needs to rely on the map to obtain the feasible area during the navigation process.
  • Depth maps are usually used to build maps. In the process of building maps, specific objects are usually not distinguished, and all data is treated equally. Therefore, in the tracking task, the map contains the tracked target and other environmental information, and the robot needs to avoid obstacles while tracking the tracked target. However, when the tracked target is closer to the robot, the tracked target will be regarded as an obstacle, so that the planned path of the robot avoids the tracked target.
  • Embodiments of the present application provide an image processing method and device, and a mobile device.
  • the image processing method according to the embodiment of the present application is applied to a mobile device, and the image processing method includes steps:
  • the image of the tracked target is eliminated from the map constructed according to the environment image.
  • an image of a tracked object is eliminated from a map so that the tracked object is not included in the map, thereby preventing a situation in which the mobile device avoids the tracked object in the process of tracking the tracked object.
  • the image processing device is used for a mobile device, and the image processing device includes:
  • An image acquisition module for acquiring an environment image
  • a processing module configured to process the environment image to obtain an image of the tracked target
  • a culling module is configured to cull the image of the tracked target in a map constructed according to the environment image.
  • the image processing device removes the tracked target image from the map so that the tracked target is not included in the map, thereby preventing the mobile device from evading the tracked target during the tracking of the tracked target.
  • the image processing apparatus is used in a mobile device.
  • the image processing apparatus includes a memory and a processor.
  • the memory stores executable instructions
  • the processor is configured to execute the instructions to implement the above-mentioned embodiment. Steps of image processing method.
  • the image processing device removes the tracked target image from the map so that the tracked target is not included in the map, thereby preventing the mobile device from evading the tracked target during the tracking of the tracked target.
  • a mobile device includes the image processing apparatus according to the foregoing embodiment.
  • the mobile device in the embodiment of the present application removes the tracked target image from the map so that the tracked target is not included in the map, thereby preventing the mobile device from evading the tracked target during the tracking of the tracked target.
  • FIG. 1 is a schematic flowchart of an image processing method according to an embodiment of the present application
  • FIG. 2 is another schematic flowchart of an image processing method according to an embodiment of the present application.
  • FIG. 3 is another schematic flowchart of an image processing method according to an embodiment of the present application.
  • FIG. 4 is a schematic diagram of an image of a tracked target not excluded from a map according to an embodiment of the present application
  • FIG. 5 is a schematic diagram of an image from which a tracked target has been eliminated in a map according to an embodiment of the present application
  • FIG. 6 is a schematic block diagram of an image processing apparatus according to an embodiment of the present application.
  • FIG. 7 is a schematic diagram of another module of an image processing apparatus according to an embodiment of the present application.
  • FIG. 8 is another schematic block diagram of an image processing apparatus according to an embodiment of the present application.
  • FIG. 9 is another schematic block diagram of an image processing apparatus according to an embodiment of the present application.
  • FIG. 10 is a schematic block diagram of a mobile device according to an embodiment of the present application.
  • Image processing device 100 image acquisition module 10, processing module 20, detection module 22, clustering module 24, elimination module 30, building module 40, filling module 50, memory 80, processor 90, mobile device 1000, target area TA, Unknown area UA.
  • first and second are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as “first” and “second” may explicitly or implicitly include one or more of the features. In the description of the present application, the meaning of "a plurality" is two or more, unless specifically defined otherwise.
  • connection should be understood in a broad sense unless otherwise specified and limited. For example, they may be fixed connections or removable. Connection, or integral connection; can be mechanical, electrical, or can communicate with each other; can be directly connected, or indirectly connected through an intermediate medium, can be the internal communication of two elements or the interaction of two elements relationship.
  • connection or integral connection; can be mechanical, electrical, or can communicate with each other; can be directly connected, or indirectly connected through an intermediate medium, can be the internal communication of two elements or the interaction of two elements relationship.
  • the image processing method according to the embodiment of the present application may be implemented by the image processing apparatus 100 according to the embodiment of the present application and applied to the mobile device 1000 according to the embodiment of the present application.
  • the image processing method includes steps:
  • S20 process the environment image to obtain an image of the tracked target
  • the image of the tracked target is eliminated from the map so that the tracked target is not included in the map, thereby preventing the mobile device 1000 from evading the tracked target in the process of tracking the tracked target.
  • the mobile device 1000 needs to rely on the map to obtain the feasible area during the navigation process.
  • the map includes the tracked target and other environmental information.
  • the mobile device 1000 needs to avoid obstacles while tracking the tracked target. When the tracked target is closer to the mobile device 1000, the mobile device 1000 treats the tracked target as an obstacle. In this way, the path planned by the mobile device 1000 will avoid the tracked target, thereby affecting tracking. For example, when the track of the tracked target is a straight line, because the planned path of the mobile device 1000 avoids the tracked target, the track of the mobile device 1000 will not be consistent with the track of the tracked target, and the track of the mobile device 1000 may be It turns into a curve and does not meet expectations.
  • the image processing method it is necessary to use the image processing method according to the embodiment of the present application to remove the image of the tracked object from the map so that the tracked object is not included in the map.
  • the mobile device 1000 will not regard the tracked object as an obstacle. That is to say, the path planned by the mobile device 1000 will not avoid the tracked target.
  • the data of the mobile device 1000 tracking the tracked target and the obstacle avoidance data may be processed separately.
  • step S20 includes: using a first deep neural network algorithm to process the environment image to obtain an image of the tracked target.
  • the environment image can be input into a first deep neural network (such as a convolutional neural network), and the image characteristics of the tracked target output by the first deep neural network can be obtained to obtain the image of the tracked target. . That is, the image features of the tracked object can be obtained through deep learning to obtain the image of the tracked object.
  • a first deep neural network such as a convolutional neural network
  • the image features of the tracked object can be obtained through deep learning to obtain the image of the tracked object.
  • an environment image is acquired and input to the first deep neural network that has been trained.
  • the trained first deep neural network can recognize the image characteristics of a specific type of object. If the type of the tracked target is consistent with the specific type, the first deep neural network model can identify the tracked target in the environmental image. Image features to obtain the image of the tracked target.
  • step S20 includes:
  • Step S22 Detect a tracked target using an environmental image to obtain a target area in the environmental image
  • Step S24 Cluster the target regions to obtain images of the tracked targets.
  • the environment image may include a depth map.
  • the image processing method includes: constructing a map according to a depth map.
  • Step S22 includes detecting a tracked target using a depth map to obtain a target area TA in the depth map.
  • the image processing method includes: constructing a map according to a depth map.
  • the depth map contains depth data, and the data of each pixel in the depth map contains the actual distance between the camera and the object.
  • Depth maps can express three-dimensional scene information. Therefore, depth maps are usually used to build maps, and the technology for building maps based on depth maps is mature. Depth maps can be obtained with TOF (Time of Flight) cameras, binocular cameras, or structured light cameras.
  • TOF Time of Flight
  • step S22 includes: detecting the tracked target using the color map to obtain the target area TA in the color map; and according to the position correspondence between the depth map and the color map Obtain the target area TA in the depth map.
  • the environment image includes a depth map and a gray map
  • step S22 includes: detecting the tracked target using the gray map to obtain the target area TA in the gray map; and according to the position correspondence between the depth map and the gray map, The depth map obtains the target area TA.
  • the depth map, color map, and gray map can be obtained by the same camera configured on the body of the mobile device 1000. Therefore, the pixel coordinates of the depth map, color map, and gray map have a one-to-one correspondence. That is, the position of each pixel in the depth map on the gray map or the color map is the same as the position of each pixel in the depth map on the depth map.
  • the depth map, color map, and gray map can also be obtained through different cameras configured on the body of the mobile device 1000. At this time, the pixel coordinates of the depth map, color map, and gray map do not correspond one-to-one.
  • the pixel coordinates of the depth map, color map, and gray map can be obtained through mutual transformation of the coordinate transformation relationship.
  • a tracked target may be detected in the depth map to obtain a target area TA.
  • the environment image includes a depth map and a color map
  • the tracked target can be detected in the color map to obtain a target area TA, and the corresponding target area TA can be obtained from the depth map through the pixel point coordinate correspondence between the color map and the depth map.
  • the environment image includes a depth map and a gray map
  • the tracked target can be detected in the gray map to obtain the target area TA, and the corresponding target can be obtained from the depth map through the correspondence between the pixel coordinates of the gray map and the depth map.
  • Area TA there are various options for obtaining the target area TA in the environment image.
  • step S22 includes: using a second deep neural network algorithm to detect the tracked target in the environmental image to obtain the target area TA in the environmental image.
  • the environment image can be input into the second deep neural network, and the target area TA output by the second deep neural network can be obtained.
  • an environment image is acquired and input to the trained second deep neural network.
  • the trained second deep neural network can recognize specific types of objects. If the type of the tracked target is consistent with the specific type, the second deep neural network model can identify the tracked target in the environment image and output it. The target area TA containing the tracked target.
  • the mobile device 1000 is configured with corresponding application software (APP).
  • APP application software
  • the user can frame the tracked target on the APP's human-machine interface, so that the target area TA can be obtained according to the characteristics of the tracked target in the previous frame of the environment image.
  • the human-machine interface may be displayed on a display screen of the mobile device 1000 or a display screen of a remote control device (including but not limited to a remote control, a mobile phone, a tablet computer, a wearable smart device, etc.) that communicates with the mobile device 1000.
  • the target area TA includes the background of the image of the tracked target and the image of the environment.
  • Step S24 includes: clustering the target area TA to remove the background of the environmental image and acquiring an image of the tracked target.
  • step S24 includes: using a breadth-first search clustering algorithm to cluster the target area TA to obtain an image of the tracked target.
  • a breadth-first search clustering algorithm is used to obtain multiple connected areas in the target area TA and determine the largest connected area among the multiple connected areas as the image of the tracked target.
  • the breadth-first search clustering algorithm is used to analyze the connected area of the target area TA. That is, connect the pixels with similar chroma or similar pixel values in the target area TA to obtain multiple connected areas. .
  • the largest connected area among the multiple connected areas is the image of the tracked target. In this way, the image of the tracked target can be eliminated in the target area TA, and the background of the environmental image in the target area TA can be retained to avoid losing environmental information.
  • a clustering operation may be performed on a pixel in the center of the target area TA in the environment image (depth map) as a starting point.
  • the clustering algorithm can determine the same type of pixels, that is, the clustering algorithm can distinguish the image of the tracked target in the target area from the background of the environmental image, and then obtain the depth image area that belongs only to the tracked target, that is, in the The image of the tracked target is acquired in the depth map.
  • the map includes a blank area corresponding to the image position of the tracked object.
  • the image processing method includes step S40: filling a blank area with a preset image, and determining an area where the preset image is located as an unknown area UA.
  • the image position of the tracked target becomes a blank area.
  • the blank area is filled with a preset image to make it an unknown area UA.
  • the preset image may be composed of pixels defined as invalid values. It can be understood that in other embodiments, the blank area may also be determined as the unknown area UA.
  • FIG. 4 is a map of the image of the tracked target is not removed
  • FIG. 5 is a map of the image of the tracked target is removed.
  • the area surrounded by the rectangular frame is the target area TA.
  • the area surrounded by the rectangular frame includes an unknown area UA.
  • an image processing apparatus 100 is used in a mobile device 1000.
  • the image processing apparatus 100 includes an image acquisition module 10, a processing module 20, and a rejection module 30.
  • the image acquisition module 10 is configured to acquire an environment image.
  • the processing module 20 is configured to process an environment image to obtain an image of a tracked target.
  • the culling module 30 is used for culling the image of the tracked target in the map constructed according to the environment image.
  • step S10 of the image processing method according to the embodiment of the present application may be implemented by the image acquisition module 10
  • step S20 may be implemented by the processing module 20
  • step S30 may be implemented by the rejection module 30.
  • the image processing apparatus 100 removes the tracked target image from the map so that the tracked target is not included in the map, thereby preventing the mobile device 1000 from evading the tracked target during the tracking of the tracked target.
  • the processing module 20 is configured to process the environment image using a first deep neural network algorithm to obtain an image of the tracked target.
  • the processing module 20 includes a detection module 22 and a clustering module 24.
  • the detection module 22 is configured to detect a tracked target by using an environment image to obtain a target area in the environment image.
  • the clustering module 24 is configured to cluster the target area to obtain an image of the tracked target.
  • the environment image includes a depth map.
  • the detection module 22 is configured to detect a tracked target using a depth map to obtain a target area TA in the depth map.
  • the image processing apparatus 100 includes a building module 40.
  • the building module 40 is configured to build a map according to the depth map.
  • the environment image includes a depth map and a color map.
  • the detection module 22 is configured to detect a tracked target using a color map to obtain a target area TA in the color map; and obtain a target area TA in the depth map according to a position correspondence between the depth map and the color map.
  • the image processing apparatus 100 includes a building module 40.
  • the building module 40 is configured to build a map according to the depth map.
  • the environment image includes a depth map and a gray map.
  • the detection module 22 is configured to detect the tracked target using the gray map to obtain the target area TA in the gray map; and obtain the target area TA in the depth map according to the position correspondence between the depth map and the gray map.
  • the image processing apparatus 100 includes a building module 40.
  • the building module 40 is configured to build a map according to the depth map.
  • the image acquisition module 10 includes a TOF camera or a binocular camera or a structured light camera, and the depth map is obtained by shooting with a TOF camera or a binocular camera or a structured light camera.
  • the detection module 22 is configured to detect a tracked target in an environmental image using a second deep neural network algorithm to obtain a target area TA in the environmental image.
  • the target area TA includes the background of the image of the tracked target and the image of the environment.
  • the clustering module 24 is configured to cluster the target area TA to remove the background of the environmental image and obtain an image of the tracked target.
  • the clustering module 24 is configured to use a breadth-first search clustering algorithm to cluster the target area TA to obtain an image of the tracked target.
  • the clustering module 24 is configured to use a breadth-first search clustering algorithm to obtain multiple connected regions in the target region TA and determine the largest connected region among the multiple connected regions as the image of the tracked target.
  • the map includes a blank area corresponding to the image position of the tracked object.
  • the image processing apparatus 100 includes an area processing module 50.
  • the area processing module 50 is configured to fill a blank area with a preset image and determine the area where the preset image is located as the unknown area UA; or directly determine the blank area as the unknown area UA.
  • an image processing apparatus 100 is used in a mobile device 1000.
  • the image processing apparatus 100 includes a memory 80 and a processor 90.
  • the memory 80 stores executable instructions.
  • the processor 90 is configured to execute instructions to implement the steps of the image processing method in any one of the foregoing embodiments.
  • the image processing apparatus 100 removes the tracked target image from the map so that the tracked target is not included in the map, thereby preventing the mobile device 1000 from avoiding the tracked target during the tracking of the tracked target.
  • a mobile device 1000 includes the image processing apparatus 100 according to any one of the foregoing embodiments.
  • the mobile device 1000 removes the tracked target image from the map so that the tracked target is not included in the map, thereby preventing the mobile device 1000 from evading the tracked target during the tracking of the tracked target.
  • the illustrated image processing apparatus 100 includes a memory 80 (for example, a non-volatile storage medium) and a processor 90.
  • the memory 80 stores executable instructions.
  • the processor 90 may execute instructions to implement the steps of the image processing method in any one of the foregoing embodiments.
  • the mobile device 1000 may be a mobile cart, a mobile robot, a drone, or the like.
  • the mobile device 1000 shown in FIG. 10 is a mobile robot.
  • Any process or method description in a flowchart or otherwise described herein can be understood as representing a module, fragment, or portion of code that includes one or more executable instructions for performing a particular logical function or step of a process
  • the scope of the preferred embodiments of the present application includes additional executions, which may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order according to the functions involved, which should It is understood by those skilled in the art to which the embodiments of the present application pertain.
  • Logic and / or steps represented in a flowchart or otherwise described herein, for example, a ordered list of executable instructions that may be considered to perform a logical function may be embodied in any computer-readable medium, For use by, or in combination with, an instruction execution system, device, or device (such as a computer-based system, a system that includes a processor, or another system that can fetch and execute instructions from an instruction execution system, device, or device) Or equipment.
  • a "computer-readable medium” may be any device that can contain, store, communicate, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device.
  • computer-readable media include the following: electrical connections (electronic devices) with one or more wirings, portable computer disk cartridges (magnetic devices), random access memory (RAM), Read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disk read-only memory (CDROM).
  • the computer-readable medium may even be paper or other suitable medium on which the program can be printed, because, for example, by optically scanning the paper or other medium, followed by editing, interpretation, or other suitable Processing to obtain the program electronically and then store it in computer memory.
  • each part of the present application may be executed by hardware, software, firmware, or a combination thereof.
  • multiple steps or methods may be performed by software or firmware stored in a memory and executed by a suitable instruction execution system.
  • a suitable instruction execution system For example, if executed by hardware, as in another embodiment, it may be executed by any one or a combination of the following techniques known in the art: Discrete logic circuits, application-specific integrated circuits with suitable combinational logic gate circuits, programmable gate arrays (PGA), field programmable gate arrays (FPGA), etc.
  • a person of ordinary skill in the art can understand that performing all or part of the steps carried by the foregoing implementation method can be completed by a program instructing related hardware.
  • the program can be stored in a computer-readable storage medium, and the program is executing , Including one or a combination of steps of a method embodiment.
  • each functional unit in each embodiment of the present application may be integrated in one processing module, or each unit may exist separately physically, or two or more units may be integrated in one module.
  • the above integrated modules can be executed in the form of hardware or software functional modules. When the integrated module is executed in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.
  • the aforementioned storage medium may be a read-only memory, a magnetic disk, or an optical disk.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Automation & Control Theory (AREA)
  • Evolutionary Computation (AREA)
  • Data Mining & Analysis (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Medical Informatics (AREA)
  • Databases & Information Systems (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Electromagnetism (AREA)
  • Molecular Biology (AREA)
  • Mathematical Physics (AREA)
  • Computational Linguistics (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Image Analysis (AREA)

Abstract

一种图像处理方法用于移动设备。图像处理方法包括步骤：获取环境图像；处理环境图像以获取被跟踪目标的图像；以及在根据环境图像所构建的地图中将被跟踪目标的图像剔除。此外，本申请还公开了一种图像处理装置(100)和移动设备(1000)。

Description

图像处理方法及装置、移动设备 技术领域
本申请涉及图像处理技术领域，特别涉及一种图像处理方法及装置、移动设备。
背景技术
机器人在导航过程中需要依赖地图获得可行区域。通常采用深度图来构建地图，在构建地图的过程中一般不进行特定物体的区分，将所有的数据一视同仁来构建地图。因此，在跟踪任务中，地图中包含被跟踪目标和其它环境信息，机器人需要一边跟踪被跟踪目标，一边规避障碍物。但是当被跟踪目标与机器人距离较近时，被跟踪目标会被当作障碍物，从而出现机器人所规划的路径躲避被跟踪目标的情况。
发明内容
本申请的实施方式提供一种图像处理方法及装置、移动设备。
本申请实施方式的图像处理方法用于移动设备，所述图像处理方法包括步骤：
获取环境图像；
处理所述环境图像以获取所述被跟踪目标的图像；
在根据所述环境图像所构建的地图中将所述被跟踪目标的图像剔除。
本申请实施方式的图像处理方法，在地图中将被跟踪目标的图像剔除使地图中不包含被跟踪目标，从而防止出现移动设备在跟踪被跟踪目标的过程中躲避被跟踪目标的情况。
本申请实施方式的图像处理装置用于移动装置，所述图像处理装置包括：
图像获取模块，用于获取环境图像；
处理模块，用于处理所述环境图像以获取所述被跟踪目标的图像；
剔除模块，用于在根据所述环境图像所构建的地图中将所述被跟踪目标的图像剔除。
本申请实施方式的图像处理装置，在地图中将被跟踪目标的图像剔除使地图中不包含被跟踪目标，从而防止出现移动设备在跟踪被跟踪目标的过程中躲避被跟踪目标的情况。
本申请实施方式的图像处理装置用于移动设备，所述图像处理装置包括存储器和处理器，所述存储器存储有可执行指令，所述处理器用于执行所述指令以实现上述实施方式所述的图像处理方法的步骤。
本申请实施方式的图像处理装置，在地图中将被跟踪目标的图像剔除使地图中不包含被跟踪目标，从而防止出现移动设备在跟踪被跟踪目标的过程中躲避被跟踪目标的情况。
本申请实施方式的移动设备，包括上述实施方式所述的图像处理装置。
本申请实施方式的移动设备，在地图中将被跟踪目标的图像剔除使地图中不包含被跟踪目标，从而防止出现移动设备在跟踪被跟踪目标的过程中躲避被跟踪目标的情况。
本申请的实施方式的附加方面和优点将在下面的描述中部分给出，部分将从下面的描述中变得明显，或通过本申请的实施方式的实践了解到。
附图说明
本申请的上述和/或附加的方面和优点从结合下面附图对实施方式的描述中将变得明显和容易理解，其中：
图1是本申请实施方式的图像处理方法的流程示意图；
图2是本申请实施方式的图像处理方法的另一流程示意图；
图3是本申请实施方式的图像处理方法的又一流程示意图；
图4是本申请实施方式的地图未剔除被跟踪目标的图像的示意图；
图5是本申请实施方式的地图已剔除被跟踪目标的图像的示意图；
图6是本申请实施方式的图像处理装置的模块示意图；
图7是本申请实施方式的图像处理装置的另一模块示意图；
图8是本申请实施方式的图像处理装置的又一模块示意图；
图9是本申请实施方式的图像处理装置的再一模块示意图；
图10是本申请实施方式的移动设备的模块示意图。
主要元件符号说明：
图像处理装置100、图像获取模块10、处理模块20、检测模块22、聚类模块24、剔除模块30、构建模块40、填充模块50、存储器80、处理器90、移动设备1000、目标区域TA、未知区域UA。
具体实施方式
下面详细描述本申请的实施方式，所述实施方式的示例在附图中示出，其中自始至终相同或类似的标号表示相同或类似的元件或具有相同或类似功能的元件。下面通过参考附图描述的实施方式是示例性的，仅用于解释本申请，而不能理解为对本申请的限制。
在本申请的描述中，需要理解的是，术语“第一”、“第二”仅用于描述目的，而不能理解为指示或暗示相对重要性或者隐含指明所指示的技术特征的数量。由此，限定有“第一”、“第二”的特征可以明示或者隐含地包括一个或者更多个所述特征。在本申请的描述中，“多个”的含义是两个或两个以上，除非另有明确具体的限定。
在本申请的描述中，需要说明的是，除非另有明确的规定和限定，术语“安装”、“相连”、 “连接”应做广义理解，例如，可以是固定连接，也可以是可拆卸连接，或一体地连接；可以是机械连接，也可以是电连接或可以相互通信；可以是直接相连，也可以通过中间媒介间接相连，可以是两个元件内部的连通或两个元件的相互作用关系。对于本领域的普通技术人员而言，可以根据具体情况理解上述术语在本申请中的具体含义。
下文的公开提供了许多不同的实施方式或例子用来实现本申请的不同结构。为了简化本申请的公开，下文中对特定例子的部件和设置进行描述。当然，它们仅仅为示例，并且目的不在于限制本申请。此外，本申请可以在不同例子中重复参考数字和/或参考字母，这种重复是为了简化和清楚的目的，其本身不指示所讨论各种实施方式和/或设置之间的关系。此外，本申请提供了的各种特定的工艺和材料的例子，但是本领域普通技术人员可以意识到其他工艺的应用和/或其他材料的使用。
下面详细描述本申请的实施方式，所述实施方式的示例在附图中示出，其中自始至终相同或类似的标号表示相同或类似的元件或具有相同或类似功能的元件。下面通过参考附图描述的实施方式是示例性的，仅用于解释本申请，而不能理解为对本申请的限制。
请参阅图1、图4、图6和图10，本申请实施方式的图像处理方法可由本申请实施方式的图像处理装置100实现，并应用于本申请实施方式的移动设备1000。图像处理方法包括步骤：
S10：获取环境图像；
S20：处理环境图像以获取被跟踪目标的图像；
S30：在根据环境图像所构建的地图中将被跟踪目标的图像剔除。
本申请实施方式的图像处理方法，在地图中将被跟踪目标的图像剔除使地图中不包含被跟踪目标，从而防止出现移动设备1000在跟踪被跟踪目标的过程中躲避被跟踪目标的情况。
可以理解，移动设备1000在导航过程中需要依赖地图获得可行区域。在跟踪任务中，地图中包含被跟踪目标和其它环境信息，移动设备1000需要一边跟踪被跟踪目标，一边规避障碍物。当被跟踪目标与移动设备1000距离较近时，移动设备1000会把被跟踪目标当作障碍物。如此，移动设备1000所规划的路径会躲避被跟踪目标，从而影响跟踪。例如，当被跟踪目标的移动轨迹为直线时，由于移动设备1000所规划的路径躲避被跟踪目标，移动设备1000的移动轨迹不会与被跟踪目标的移动轨迹一致，移动设备1000的移动轨迹可能变成曲线，不符合预期。因此，需要利用本申请实施方式的图像处理方法在地图中将被跟踪目标的图像剔除使地图中不包含被跟踪目标。这样，将被跟踪目标的图像从地图中剔除后，即使被跟踪目标与移动设备1000距离较近，移动设备1000也不会把被跟踪目标当作障碍物。也即是说，移动设备1000所规划的路径不会躲避被跟踪目标。
需要说明的是，在本申请中，移动设备1000跟踪被跟踪目标的数据和避障的数据可以是分开处理的。
在某些实施方式中，步骤S20包括：采用第一深度神经网络算法处理环境图像以获取被跟踪目标的图像。
可以理解，获取环境图像后，可将环境图像输入到第一深度神经网络(例如卷积神经网络)中，并获取第一深度神经网络输出的被跟踪目标的图像特征从而获取被跟踪目标的图像。也即是说，可以通过深度学习得到被跟踪目标的图像特征来获取被跟踪目标的图像。具体地，获取环境图像并将环境图像输入到已经训练好的第一深度神经网络。其中，已经训练好的第一深度神经网络可以对特定类型的对象的图像特征进行识别，若被跟踪目标的类型与特定类型一致，则第一深度神经网络模型可以识别环境图像中的被跟踪目标的图像特征，从而获取被跟踪目标的图像。
在某些实施方式中，请参阅图2，步骤S20包括：
步骤S22：利用环境图像检测被跟踪目标以在环境图像中获得目标区域；
步骤S24：对目标区域进行聚类以获取被跟踪目标的图像。
在某些实施方式中，环境图像可包括深度图。图像处理方法包括：根据深度图构建地图。步骤S22包括：利用深度图检测被跟踪目标以在深度图中获得目标区域TA。图像处理方法包括：根据深度图构建地图。
可以理解，深度图包含深度数据，深度图中每个像素点的数据包含相机与物体的实际距离。深度图能够表达三维场景信息，因此，通常采用深度图来构建地图，而且根据深度图构建地图的技术成熟。深度图可以由TOF(Time of Flight)相机或双目相机或结构光相机拍摄获得。
在某些实施方式中，环境图像可包括深度图和彩色图时，步骤S22包括：利用彩色图检测被跟踪目标以在彩色图中获得目标区域TA；以及根据深度图和彩色图的位置对应关系，在深度图获得目标区域TA。
在某些实施方式中，环境图像包括深度图和灰色图，步骤S22包括：利用灰色图检测被跟踪目标以在灰色图中获得目标区域TA；以及根据深度图和灰色图的位置对应关系，在深度图获得目标区域TA。
可以理解，可通过配置在移动设备1000的机身上的同一个相机来获取深度图、彩色图和灰度图，因此深度图、彩色图和灰度图的像素点坐标一一对应的关系，即深度图中的每一个像素点在灰度图上的位置或在彩色图上的位置与深度图的每一个像素点在深度图上的位置相同。当然，也可通过配置在移动设备1000的机身上的不同的相机来获取深度图、彩色图和灰度图，此时深度图、彩色图和灰度图的像素点坐标不是一一对应，深度图、彩色 图和灰度图的像素点坐标均可通过坐标转换关系相互转换得到。
当环境图像为深度图时，可以在深度图中检测被跟踪目标以获得目标区域TA。当环境图像包括深度图和彩色图时，可以在彩色图中检测被跟踪目标以获得目标区域TA，通过彩色图和深度图的像素点坐标对应关系，从深度图中获得对应的目标区域TA。当环境图像包括深度图和灰度图时，可以在灰度图中检测被跟踪目标以获得目标区域TA，通过灰度图和深度图的像素点坐标对应关系，从深度图中获得对应的目标区域TA。如此，在环境图像中获得目标区域TA的方式有多种选择。
进一步地，步骤S22包括：采用第二深度神经网络算法在环境图像检测被跟踪目标以在环境图像中获得目标区域TA。
可以理解，获取环境图像后，可将环境图像输入到第二深度神经网络中，并获取该第二深度神经网络输出的目标区域TA。具体地，获取环境图像并将环境图像输入到已经训练好的第二深度神经网络。其中，已经训练好的第二深度神经网络可以对特定类型的对象进行识别，若被跟踪目标的类型与特定类型一致，则第二深度神经网络模型可以识别环境图像中的被跟踪目标，并输出包含被跟踪目标的目标区域TA。
移动设备1000配置有对应的应用软件(APP)。在其他的实施方式中，在获得初始的环境图像之后，用户可以在APP的人机界面上框选被跟踪目标，从而可以根据上一帧环境图像中被跟踪目标的特征来获得目标区域TA。人机界面可显示在移动设备1000的显示屏，或显示在与移动设备1000通信的遥控设备(包括但不限于遥控器、手机、平板电脑、可穿戴智能设备等)的显示屏。
在某些实施方式中，目标区域TA包括被跟踪目标的图像和环境图像的背景。步骤S24包括：对目标区域TA进行聚类以去除环境图像的背景并获取被跟踪目标的图像。
进一步地，步骤S24包括：采用广度优先搜索的聚类算法对目标区域TA进行聚类以获取被跟踪目标的图像。具体地，采用广度优先搜索的聚类算法在目标区域TA获取多个连通区域并将多个连通区域中最大的连通区域确定为被跟踪目标的图像。
可以理解，色度相近或像素值相近的像素点可以连接起来而得到连通区域。在环境图像中获得目标区域TA后，采用广度优先搜索的聚类算法对目标区域TA进行连通区域分析，即将目标区域TA中色度相近或像素值相近的像素点连起来以获取多个连通区域。多个连通区域中最大的连通区域则为被跟踪目标的图像。如此，可以在目标区域TA中剔除被跟踪目标的图像，保留目标区域TA中环境图像的背景，避免丢失环境信息。
当然，在其他实施方式中，可对环境图像(深度图)中的目标区域TA内中心的像素点作为起点进行聚类运算。聚类算法可以确定出同一类的像素点，即聚类算法可以将目标区域内的被跟踪目标的图像与环境图像的背景区分开来，进而得到只属于被跟踪目标的深 度图像区域，即在深度图中获取了被跟踪目标的图像。
在某些实施方式中，在被跟踪目标的图像被剔除后，地图包括与被跟踪目标的图像位置对应的空白区域。请参阅图3和图5，图像处理方法包括步骤S40：采用预设图像填充空白区域，并将预设图像所在的区域确定为未知区域UA。
可以理解，当地图中被跟踪目标的图像被剔除后，被跟踪目标的图像位置成为空白区域。此时，采用预设图像来填充空白区域使其成为未知区域UA。这样，移动设备1000不会把被跟踪目标当作障碍物，其所规划避障路径不会躲避被跟踪目标。预设图像可以由定义为无效值的像素点组成。可以理解，在其它实施方式中，也可将空白区域确定为未知区域UA。
请参阅图4和图5，图4为未剔除被跟踪目标的图像的地图，图5为已剔除被跟踪目标的图像的地图。在图4中，矩形框所包围的区域为目标区域TA。在图5中，矩形框所包围的区域包括未知区域UA。
请参阅图6，本申请实施方式的图像处理装置100用于移动设备1000。图像处理装置100包括图像获取模块10、处理模块20和剔除模块30。图像获取模块10用于获取环境图像。处理模块20用于处理环境图像以获取被跟踪目标的图像。剔除模块30用于在根据环境图像所构建的地图中将被跟踪目标的图像剔除。
也即是说，本申请实施方式的图像处理方法的步骤S10可以由图像获取模块10实现，步骤S20可以由处理模块20实现，步骤S30可以由剔除模块30实现。
本申请实施方式的图像处理装置100，在地图中将被跟踪目标的图像剔除使地图中不包含被跟踪目标，从而防止出现移动设备1000在跟踪被跟踪目标的过程中躲避被跟踪目标的情况。
需要说明的是，上述对图像处理方法的实施方式和有益效果的解释说明，也适用于本实施方式的图像处理装置100，为避免冗余，在此不再详细展开。
在某些实施方式中，处理模块20用于采用第一深度神经网络算法处理环境图像以获取被跟踪目标的图像。
在某些实施方式中，请参阅图7，处理模块20包括检测模块22和聚类模块24。检测模块22用于利用环境图像检测被跟踪目标以在环境图像中获得目标区域。聚类模块24用于对目标区域进行聚类以获取被跟踪目标的图像。
在某些实施方式中，环境图像包括深度图。检测模块22用于利用深度图检测被跟踪目标以在深度图中获得目标区域TA。请参阅图8，图像处理装置100包括构建模块40。构建模块40用于根据深度图构建地图。
在某些实施方式中，环境图像包括深度图和彩色图。检测模块22用于利用彩色图检测 被跟踪目标以在彩色图中获得目标区域TA；以及根据深度图和彩色图的位置对应关系，在深度图获得目标区域TA。请参阅图8，图像处理装置100包括构建模块40。构建模块40用于根据深度图构建地图。
在某些实施方式中，环境图像包括深度图和灰色图。检测模块22用于利用灰色图检测被跟踪目标以在灰色图中获得目标区域TA；以及根据深度图和灰色图的位置对应关系，在深度图获得目标区域TA。请参阅图8，图像处理装置100包括构建模块40。构建模块40用于根据深度图构建地图。
在某些实施方式中，图像获取模块10包括TOF相机或双目相机或结构光相机，深度图由TOF相机或双目相机或结构光相机拍摄获得。
在某些实施方式中，检测模块22用于采用第二深度神经网络算法在环境图像检测被跟踪目标以在环境图像中获得目标区域TA。
在某些实施方式中，目标区域TA包括被跟踪目标的图像和环境图像的背景。聚类模块24用于对目标区域TA进行聚类以去除环境图像的背景并获取被跟踪目标的图像。
在某些实施方式中，聚类模块24用于采用广度优先搜索的聚类算法对目标区域TA进行聚类以获取被跟踪目标的图像。
在某些实施方式中，聚类模块24用于采用广度优先搜索的聚类算法在目标区域TA获取多个连通区域并将多个连通区域中最大的连通区域确定为被跟踪目标的图像。
在某些实施方式中，在被跟踪目标的图像被剔除后，地图包括与被跟踪目标的图像位置对应的空白区域。请参阅图9，图像处理装置100包括区域处理模块50。区域处理模块50用于采用预设图像填充空白区域，并将预设图像所在的区域确定为未知区域UA；或直接将空白区域确定为未知区域UA。
请参阅图10，本申请另一实施方式的图像处理装置100用于移动设备1000。图像处理装置100包括存储器80和处理器90。存储器80存储有可执行指令。处理器90用于执行指令以实现上述任一实施方式的图像处理方法的步骤。
本申请实施方式的图像处理装置100，在地图中将被跟踪目标的图像剔除使地图中不包含被跟踪目标，从而防止出现移动设备1000在跟踪被跟踪目标的过程中躲避被跟踪目标的情况。
请参阅图10，本申请实施方式的移动设备1000包括上述任一实施方式的图像处理装置100。
本申请实施方式的移动设备1000，在地图中将被跟踪目标的图像剔除使地图中不包含被跟踪目标，从而防止出现移动设备1000在跟踪被跟踪目标的过程中躲避被跟踪目标的情况。
图示的图像处理装置100包括存储器80(例如为非易失性存储介质)和处理器90。存储器80存储有可执行指令。处理器90可执行指令以实现上述任一实施方式的图像处理方法的步骤。移动设备1000可以是移动小车、移动机器人、无人机等。图10所示的移动设备1000为移动机器人。
需要说明的是，上述对图像处理方法及图像处理装置100的实施方式和有益效果的解释说明，也适用于本实施方式的移动设备1000，为避免冗余，在此不再详细展开。
在本说明书的描述中，参考术语“一个实施方式”、“一些实施方式”、“示意性实施方式”、“示例”、“具体示例”、或“一些示例”等的描述意指结合所述实施方式或示例描述的具体特征、结构、材料或者特点包含于本申请的至少一个实施方式或示例中。在本说明书中，对上述术语的示意性表述不一定指的是相同的实施方式或示例。而且，描述的具体特征、结构、材料或者特点可以在任何的一个或多个实施方式或示例中以合适的方式结合。
流程图中或在此以其他方式描述的任何过程或方法描述可以被理解为，表示包括一个或更多个用于执行特定逻辑功能或过程的步骤的可执行指令的代码的模块、片段或部分，并且本申请的优选实施方式的范围包括另外的执行，其中可以不按所示出或讨论的顺序，包括根据所涉及的功能按基本同时的方式或按相反的顺序，来执行功能，这应被本申请的实施方式所属技术领域的技术人员所理解。
在流程图中表示或在此以其他方式描述的逻辑和/或步骤，例如，可以被认为是用于执行逻辑功能的可执行指令的定序列表，可以具体执行在任何计算机可读介质中，以供指令执行系统、装置或设备(如基于计算机的系统、包括处理器的系统或其他可以从指令执行系统、装置或设备取指令并执行指令的系统)使用，或结合这些指令执行系统、装置或设备而使用。就本说明书而言，"计算机可读介质"可以是任何可以包含、存储、通信、传播或传输程序以供指令执行系统、装置或设备或结合这些指令执行系统、装置或设备而使用的装置。计算机可读介质的更具体的示例(非穷尽性列表)包括以下：具有一个或多个布线的电连接部(电子装置)，便携式计算机盘盒(磁装置)，随机存取存储器(RAM)，只读存储器(ROM)，可擦除可编辑只读存储器(EPROM或闪速存储器)，光纤装置，以及便携式光盘只读存储器(CDROM)。另外，计算机可读介质甚至可以是可在其上打印所述程序的纸或其他合适的介质，因为可以例如通过对纸或其他介质进行光学扫描，接着进行编辑、解译或必要时以其他合适方式进行处理来以电子方式获得所述程序，然后将其存储在计算机存储器中。
应当理解，本申请的各部分可以用硬件、软件、固件或它们的组合来执行。在上述实施方式中，多个步骤或方法可以用存储在存储器中且由合适的指令执行系统执行的软件或固件来执行。例如，如果用硬件来执行，和在另一实施方式中一样，可用本领域公知的下列技术中的任一项或他们的组合来执行：具有用于对数据信号执行逻辑功能的逻辑门电路 的离散逻辑电路，具有合适的组合逻辑门电路的专用集成电路，可编程门阵列(PGA)，现场可编程门阵列(FPGA)等。
本技术领域的普通技术人员可以理解执行上述实施方法携带的全部或部分步骤是可以通过程序来指令相关的硬件完成，所述的程序可以存储于一种计算机可读存储介质中，该程序在执行时，包括方法实施方式的步骤之一或其组合。
此外，在本申请各个实施方式中的各功能单元可以集成在一个处理模块中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个模块中。上述集成的模块既可以采用硬件的形式执行，也可以采用软件功能模块的形式执行。所述集成的模块如果以软件功能模块的形式执行并作为独立的产品销售或使用时，也可以存储在一个计算机可读取存储介质中。
上述提到的存储介质可以是只读存储器，磁盘或光盘等。尽管上面已经示出和描述了本申请的实施方式，可以理解的是，上述实施方式是示例性的，不能理解为对本申请的限制，本领域的普通技术人员在本申请的范围内可以对上述实施方式进行变化、修改、替换和变型。

Claims (26)

  1. 一种图像处理方法，用于移动设备，其特征在于，所述图像处理方法包括步骤：
    获取环境图像；
    处理所述环境图像以获取所述被跟踪目标的图像；
    在根据所述环境图像所构建的地图中将所述被跟踪目标的图像剔除。
  2. 如权利要求1所述的图像处理方法，其特征在于，处理所述环境图像以获取所述被跟踪目标的图像，包括：
    采用深度神经网络算法处理所述环境图像以获取所述被跟踪目标的图像。
  3. 如权利要求1所述的图像处理方法，其特征在于，处理所述环境图像以获取所述被跟踪目标的图像，包括：
    利用所述环境图像检测被跟踪目标以在所述环境图像中获得目标区域；
    对所述目标区域进行聚类以获取所述被跟踪目标的图像。
  4. 如权利要求3所述的图像处理方法，其特征在于，所述环境图像包括深度图，利用所述环境图像检测被跟踪目标以在所述环境图像中获得目标区域，包括：利用所述深度图检测所述被跟踪目标以在所述深度图中获得所述目标区域；
    所述图像处理方法包括：
    根据所述深度图构建所述地图。
  5. 如权利要求3所述的图像处理方法，其特征在于，所述环境图像包括深度图和彩色图，利用所述环境图像检测被跟踪目标以在所述环境图像中获得目标区域，包括：
    利用所述彩色图检测所述被跟踪目标以在所述彩色图中获得所述目标区域；
    根据所述深度图和所述彩色图的位置对应关系，在所述深度图获得所述目标区域；
    所述图像处理方法包括：
    根据所述深度图构建所述地图。
  6. 如权利要求3所述的图像处理方法，其特征在于，所述环境图像包括深度图和灰色图，利用所述环境图像检测被跟踪目标以在所述环境图像中获得目标区域，包括：
    利用所述灰色图检测所述被跟踪目标以在所述灰色图中获得所述目标区域；
    根据所述深度图和所述灰色图的位置对应关系，在所述深度图获得所述目标区域；
    所述图像处理方法包括：
    根据所述深度图构建所述地图。
  7. 如权利要求4-6任一项所述的图像处理方法，其特征在于，所述深度图由TOF相机或双目相机或结构光相机拍摄获得。
  8. 如权利要求3所述的图像处理方法，其特征在于，利用所述环境图像检测被跟踪目标以在所述环境图像中获得目标区域，包括：
    采用深度神经网络算法在所述环境图像检测所述被跟踪目标以在所述环境图像中获得所述目标区域。
  9. 如权利要求3所述的图像处理方法，其特征在于，所述目标区域包括所述被跟踪目标的图像和所述环境图像的背景，对所述目标区域进行聚类以获取所述被跟踪目标的图像，包括：
    对所述目标区域进行聚类以去除所述环境图像的背景并获取所述被跟踪目标的图像。
  10. 如权利要求3所述的图像处理方法，其特征在于，对所述目标区域进行聚类以获取所述被跟踪目标的图像，包括：
    采用广度优先搜索的聚类算法对所述目标区域进行聚类以获取所述被跟踪目标的图像。
  11. 如权利要求10所述的图像处理方法，其特征在于，采用广度优先搜索的聚类算法对所述目标区域进行聚类以获取所述被跟踪目标的图像，包括：
    采用所述广度优先搜索的聚类算法在所述目标区域获取多个连通区域并将所述多个连通区域中最大的连通区域确定为所述被跟踪目标的图像。
  12. 如权利要求1所述的图像处理方法，其特征在于，在所述被跟踪目标的图像被剔除后，所述地图包括与所述被跟踪目标的图像位置对应的空白区域，所述图像处理方法包括：
    将所述空白区域确定为未知区域；或
    采用预设图像填充所述空白区域，并将所述预设图像所在的区域确定为未知区域。
  13. 一种图像处理装置，用于移动设备，其特征在于，所述图像处理装置包括：
    图像获取模块，用于获取环境图像；
    处理模块，用于处理所述环境图像以获取所述被跟踪目标的图像；
    剔除模块，用于在根据所述环境图像所构建的地图中将所述被跟踪目标的图像剔除。
  14. 如权利要求13所述的图像处理装置，其特征在于，所述处理模块用于：
    采用深度神经网络算法处理所述环境图像以获取所述被跟踪目标的图像。
  15. 如权利要求13所述的图像处理装置，其特征在于，所述处理模块包括检测模块和聚类模块，所述检测模块用于：
    利用所述环境图像检测被跟踪目标以在所述环境图像中获得目标区域；
    所述聚类模块用于：
    对所述目标区域进行聚类以获取所述被跟踪目标的图像。
  16. 如权利要求15所述的图像处理装置，其特征在于，所述环境图像包括深度图，所述检测模块用于：
    利用所述深度图检测所述被跟踪目标以在所述深度图中获得所述目标区域；
    所述图像处理装置包括构建模块，所述构建模块用于：
    根据所述深度图构建所述地图。
  17. 如权利要求15所述的图像处理装置，其特征在于，所述环境图像包括深度图和彩色图，所述检测模块用于：
    利用所述彩色图检测所述被跟踪目标以在所述彩色图中获得所述目标区域；
    根据所述深度图和所述彩色图的位置对应关系，在所述深度图获得所述目标区域；
    所述图像处理装置包括构建模块，所述构建模块用于：
    根据所述深度图构建所述地图。
  18. 如权利要求15所述的图像处理装置，其特征在于，所述环境图像包括深度图和灰色图，所述检测模块用于：
    利用所述灰色图检测所述被跟踪目标以在所述灰色图中获得所述目标区域；
    根据所述深度图和所述灰色图的位置对应关系，在所述深度图获得所述目标区域；
    所述图像处理装置包括构建模块，所述构建模块用于：
    根据所述深度图构建所述地图。
  19. 如权利要求16-18任一项所述的图像处理装置，其特征在于，所述图像获取模块包括TOF相机或双目相机或结构光相机，所述深度图由所述TOF相机或所述双目相机或所述结构光相机拍摄获得。
  20. 如权利要求15所述的图像处理装置，其特征在于，所述检测模块用于：
    采用深度神经网络算法在所述环境图像检测所述被跟踪目标以在所述环境图像中获得所述目标区域。
  21. 如权利要求15所述的图像处理装置，其特征在于，所述目标区域包括所述被跟踪目标的图像和所述环境图像的背景，所述聚类模块用于：
    对所述目标区域进行聚类以去除所述环境图像的背景并获取所述被跟踪目标的图像。
  22. 如权利要求15所述的图像处理装置，其特征在于，所述聚类模块用于：
    采用广度优先搜索的聚类算法对所述目标区域进行聚类以获取所述被跟踪目标的图像。
  23. 如权利要求22所述的图像处理装置，其特征在于，所述聚类模块用于：
    采用所述广度优先搜索的聚类算法在所述目标区域获取多个连通区域并将所述多个连通区域中最大的连通区域确定为所述被跟踪目标的图像。
  24. 如权利要求13所述的图像处理装置，其特征在于，在所述被跟踪目标的图像被剔除后，所述地图包括与所述被跟踪目标的图像位置对应的空白区域，所述图像处理装置包括区域处理模块，所述区域处理模块用于：
    将所述空白区域确定为未知区域；或
    采用预设图像填充所述空白区域，并将所述预设图像所在的区域确定为未知区域。
  25. 一种图像处理装置，用于移动设备，其特征在于，所述图像处理装置包括存储器和处理器，所述存储器存储有可执行指令，所述处理器用于执行所述指令以实现权利要求1-12任一项所述的图像处理方法的步骤。
  26. 一种移动设备，其特征在于，包括权利要求13-25任一项所述的图像处理装置。
PCT/CN2018/101745 2018-08-22 2018-08-22 图像处理方法及装置、移动设备 WO2020037553A1 (zh)

Priority Applications (3)

Application Number Priority Date Filing Date Title
CN201880040265.0A CN110892449A (zh) 2018-08-22 2018-08-22 图像处理方法及装置、移动设备
PCT/CN2018/101745 WO2020037553A1 (zh) 2018-08-22 2018-08-22 图像处理方法及装置、移动设备
US17/166,977 US20210156697A1 (en) 2018-08-22 2021-02-03 Method and device for image processing and mobile apparatus

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2018/101745 WO2020037553A1 (zh) 2018-08-22 2018-08-22 图像处理方法及装置、移动设备

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/166,977 Continuation US20210156697A1 (en) 2018-08-22 2021-02-03 Method and device for image processing and mobile apparatus

Publications (1)

Publication Number Publication Date
WO2020037553A1 true WO2020037553A1 (zh) 2020-02-27

Family

ID=69592110

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/101745 WO2020037553A1 (zh) 2018-08-22 2018-08-22 图像处理方法及装置、移动设备

Country Status (3)

Country Link
US (1) US20210156697A1 (zh)
CN (1) CN110892449A (zh)
WO (1) WO2020037553A1 (zh)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11409306B2 (en) * 2018-08-14 2022-08-09 Chiba Institute Of Technology Movement robot
KR20200143960A (ko) * 2019-06-17 2020-12-28 현대자동차주식회사 영상을 이용한 객체 인식 장치 및 그 방법

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105354563A (zh) * 2015-12-14 2016-02-24 南京理工大学 结合深度和彩色图像的遮挡人脸检测预警装置及实现方法
CN105760846A (zh) * 2016-03-01 2016-07-13 北京正安维视科技股份有限公司 基于深度数据的目标检测与定位方法及系统
CN106501968A (zh) * 2017-01-09 2017-03-15 深圳市金立通信设备有限公司 一种屏蔽生物体的方法及眼镜
CN107273852A (zh) * 2017-06-16 2017-10-20 华南理工大学 基于机器视觉的手扶电梯楼层板物件及乘客行为检测算法
CN107301377A (zh) * 2017-05-26 2017-10-27 浙江大学 一种基于深度相机的人脸与行人感知系统
CN107741234A (zh) * 2017-10-11 2018-02-27 深圳勇艺达机器人有限公司 一种基于视觉的离线地图构建及定位方法

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9933264B2 (en) * 2015-04-06 2018-04-03 Hrl Laboratories, Llc System and method for achieving fast and reliable time-to-contact estimation using vision and range sensor data for autonomous navigation

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105354563A (zh) * 2015-12-14 2016-02-24 南京理工大学 结合深度和彩色图像的遮挡人脸检测预警装置及实现方法
CN105760846A (zh) * 2016-03-01 2016-07-13 北京正安维视科技股份有限公司 基于深度数据的目标检测与定位方法及系统
CN106501968A (zh) * 2017-01-09 2017-03-15 深圳市金立通信设备有限公司 一种屏蔽生物体的方法及眼镜
CN107301377A (zh) * 2017-05-26 2017-10-27 浙江大学 一种基于深度相机的人脸与行人感知系统
CN107273852A (zh) * 2017-06-16 2017-10-20 华南理工大学 基于机器视觉的手扶电梯楼层板物件及乘客行为检测算法
CN107741234A (zh) * 2017-10-11 2018-02-27 深圳勇艺达机器人有限公司 一种基于视觉的离线地图构建及定位方法

Also Published As

Publication number Publication date
US20210156697A1 (en) 2021-05-27
CN110892449A (zh) 2020-03-17

Similar Documents

Publication Publication Date Title
US10970864B2 (en) Method and apparatus for recovering point cloud data
EP3627180B1 (en) Sensor calibration method and device, computer device, medium, and vehicle
US11915502B2 (en) Systems and methods for depth map sampling
US10026017B2 (en) Scene labeling of RGB-D data with interactive option
US20210190513A1 (en) Navigation map updating method and apparatus and robot using the same
CN110793544B (zh) 路侧感知传感器参数标定方法、装置、设备及存储介质
EP3568334A1 (en) System, method and non-transitory computer readable storage medium for parking vehicle
US11788845B2 (en) Systems and methods for robust self-relocalization in a visual map
CN111060101A (zh) 视觉辅助的距离slam方法及装置、机器人
CN111209978B (zh) 三维视觉重定位方法、装置及计算设备、存储介质
JP2018530825A (ja) 非障害物エリア検出のためのシステムおよび方法
JP6850399B2 (ja) 単眼画像の深度回復方法及び装置、コンピュータ機器
US11120707B2 (en) Cognitive snapshots for visually-impaired users
US10776943B2 (en) System and method for 3D association of detected objects
JP7438320B2 (ja) クロスモーダルセンサデータの位置合わせ
US11669977B2 (en) Processing images to localize novel objects
CN111079079B (zh) 数据修正方法、装置、电子设备及计算机可读存储介质
JP2021101365A (ja) 測位方法、測位装置及び電子機器
KR102595787B1 (ko) 전자 장치 및 그 제어 방법
CN110648363A (zh) 相机姿态确定方法、装置、存储介质及电子设备
JP2020524355A5 (zh)
CN110853085B (zh) 基于语义slam的建图方法和装置及电子设备
US20190325600A1 (en) Determining a pose of a handheld object
US20210004978A1 (en) Method for acquiring depth information of target object and movable platform
US11100670B2 (en) Positioning method, positioning device and nonvolatile computer-readable storage medium

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18931265

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18931265

Country of ref document: EP

Kind code of ref document: A1