WO2021003587A1 - 语义地图的构建方法、系统、可移动平台和存储介质 - Google Patents

语义地图的构建方法、系统、可移动平台和存储介质 Download PDF

Info

Publication number
WO2021003587A1
WO2021003587A1 PCT/CN2019/094799 CN2019094799W WO2021003587A1 WO 2021003587 A1 WO2021003587 A1 WO 2021003587A1 CN 2019094799 W CN2019094799 W CN 2019094799W WO 2021003587 A1 WO2021003587 A1 WO 2021003587A1
Authority
WO
WIPO (PCT)
Prior art keywords
movable platform
semantic
semantic map
landing point
multiple images
Prior art date
Application number
PCT/CN2019/094799
Other languages
English (en)
French (fr)
Inventor
王涛
李思晋
李鑫超
Original Assignee
深圳市大疆创新科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市大疆创新科技有限公司 filed Critical 深圳市大疆创新科技有限公司
Priority to CN201980007922.6A priority Critical patent/CN111670417A/zh
Priority to PCT/CN2019/094799 priority patent/WO2021003587A1/zh
Publication of WO2021003587A1 publication Critical patent/WO2021003587A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
    • G05D1/08—Control of attitude, i.e. control of roll, pitch, or yaw
    • G05D1/0808—Control of attitude, i.e. control of roll, pitch, or yaw specially adapted for aircraft
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
    • G05D1/10—Simultaneous control of position or course in three dimensions
    • G05D1/101—Simultaneous control of position or course in three dimensions specially adapted for aircraft
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformation in the plane of the image
    • G06T3/40—Scaling the whole image or part thereof
    • G06T3/4038—Scaling the whole image or part thereof for image mosaicing, i.e. plane images composed of plane sub-images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2200/00—Indexing scheme for image data processing or generation, in general
    • G06T2200/32—Indexing scheme for image data processing or generation, in general involving image mosaicing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30181—Earth observation

Definitions

  • This application relates to the field of intelligent recognition technology, and specifically, to a method for constructing a semantic map, a system for constructing a semantic map, a movable platform, a computer-readable storage medium, and a movable platform. A way to search for landing points.
  • the embodiments of the present application provide a method, system, removable platform, and storage medium for constructing a semantic map, which can construct a more complete semantic map.
  • the first aspect of this application is to propose a method for constructing a semantic map.
  • the second aspect of this application is to propose a semantic map construction system.
  • the third aspect of this application is to propose a movable platform.
  • the fourth aspect of this application is to provide a computer-readable storage medium.
  • the fifth aspect of this application is to propose a movable platform.
  • the sixth aspect of this application is to propose a method for searching for a landing point.
  • a method for constructing a semantic map includes: acquiring semantic segmentation information of multiple images; performing a splicing operation on the multiple images to generate a spliced image. Semantic segmentation information to obtain a semantic map of stitched images.
  • the method for constructing a semantic map obtains the semantic segmentation information of multiple images, and the multiple images can be images of scenes with different perspectives and different background information, which is conducive to obtaining complete semantic segmentation information from multiple images. , Accurate the information of multiple entities in the real scene.
  • the spliced image is generated by splicing multiple images, which helps to ensure the completeness and authenticity of the scene.
  • the semantics of the spliced image can be obtained.
  • the map makes the semantic map better tend to the real scene, and fully and accurately reflects the multiple entity content of the real scene, so that the obtained semantic map has a higher degree of confidence and improves the accuracy of scene understanding. Semantic maps can accurately obtain location information.
  • a semantic map construction system which includes: a memory for storing a computer program; a processor for executing the computer program to achieve: acquiring semantic segmentation information of multiple images; Multiple images are spliced to generate a spliced image, and a semantic map of the spliced image is obtained according to the semantic segmentation information of the multiple images.
  • the semantic map construction system allows the processor to obtain the semantic segmentation information of multiple images, and the multiple images can be images of scenes with different perspectives and different background information, which facilitates the semantic segmentation information of multiple images Obtain complete and accurate information about multiple entities of the real scene, and use the processor to splice multiple images to generate a spliced image, which helps to ensure the integrity and authenticity of the scene.
  • the semantic segmentation information of multiple images Obtaining the semantic map of the spliced image, making the semantic map better tend to the real scene, and fully and accurately reflect the multiple entity content of the real scene, so that the obtained semantic map has a higher degree of confidence and improves the scene understanding
  • the accuracy of the location information can be accurately obtained through the semantic map.
  • a mobile platform including a semantic map construction system of any of the above technical solutions; and a collection device, which is connected to the construction system, and the collection device is used to collect images and send the images To the processor. Since the mobile platform includes the semantic map construction system of any of the above technical solutions, it has all the beneficial effects of the semantic map construction system of any of the above technical solutions, and will not be repeated here.
  • the fourth aspect of the present application proposes a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, a method for constructing a semantic map of any of the above technical solutions is realized. Therefore, it has the beneficial effects of the semantic map construction method of any of the above technical solutions, which will not be repeated here.
  • a movable platform which includes a body, a power supply battery, a power system, a collection device, and a controller provided on the body, and the power supply battery for powering the power system, and the power system for
  • the movable platform provides flight power; the acquisition device is used to acquire multiple images during the flight of the movable platform; the controller is used to acquire the semantic segmentation information of multiple images; the multiple images are spliced to generate spliced images, According to the semantic segmentation information of multiple images, a semantic map of the stitched image is obtained.
  • the sixth aspect of this application provides a method for searching for a landing point, which is suitable for a mobile platform, and includes the steps:
  • control the movable platform According to the landing point of the movable platform, control the movable platform to land.
  • the mobile platform provided by this application includes: an airframe, a power supply battery arranged on the airframe, a power system, an acquisition device, and a controller, wherein the power supply battery is used to power the power system, and the power system is used to provide flight power for the movable platform
  • the collection device is used to obtain multiple images during the flight of the movable platform, and obtain the semantic segmentation information of multiple images through the controller, and the multiple images can be images of scenes with different perspectives and different background information, which is beneficial to control
  • the device obtains complete and accurate information of multiple entity contents of the real scene, and generates a spliced image by splicing multiple images through the controller, which helps to ensure the integrity and authenticity of the scene.
  • the semantic map of the spliced image is obtained, so that the semantic map better tends to the real scene, and fully and accurately reflects the multiple entity content of the real scene, so that the acquired semantic map has a better
  • the high confidence level improves the accuracy of scene understanding and enables the controller to accurately obtain location information through the semantic map.
  • Fig. 1 shows a schematic flowchart of a method for constructing a semantic map according to an embodiment of the present application
  • Fig. 2 shows an image obtained by an embodiment of the present application
  • Figure 3 shows a semantic recognition diagram of an embodiment of the present application
  • FIG. 4 shows a rendering diagram of occlusion according to an embodiment of the present application
  • Fig. 5 shows a schematic flowchart of a method for constructing a semantic map according to another embodiment of the present application
  • FIG. 6 shows a schematic flowchart of a method for constructing a semantic map according to still another embodiment of the present application
  • FIG. 7 shows a schematic flowchart of a method for constructing a semantic map according to another embodiment of the present application.
  • FIG. 8 shows a schematic flowchart of a method for constructing a semantic map according to another embodiment of the present application.
  • FIG. 9 shows an image obtained by another embodiment of the present application.
  • FIG. 10 shows a semantic recognition diagram of another embodiment of the present application.
  • FIG. 11 shows a rendering diagram of occlusion according to another embodiment of the present application.
  • Fig. 12 shows a schematic diagram of a landing point of an embodiment of the present application
  • FIG. 13 shows a schematic diagram of a landing point of another embodiment of the present application.
  • FIG. 14 shows a schematic flowchart of a method for constructing a semantic map according to another embodiment of the present application.
  • FIG. 15 shows a schematic flowchart of a method for constructing a semantic map according to another embodiment of the present application.
  • FIG. 16 shows a schematic flowchart of a method for constructing a semantic map according to another embodiment of the present application.
  • FIG. 17 shows a schematic flow chart of a method for constructing a semantic map according to another embodiment of the present application.
  • FIG. 18 shows a schematic flowchart of a method for constructing a semantic map according to another embodiment of the present application.
  • FIG. 19 shows a schematic flowchart of a method for constructing a semantic map according to another embodiment of the present application.
  • FIG. 20 shows a schematic flowchart of a method for constructing a semantic map according to another embodiment of the present application
  • Fig. 21 shows a schematic block diagram of a semantic map construction system according to an embodiment of the present application.
  • FIG. 22 shows a schematic structural diagram of a movable platform according to an embodiment of the present application.
  • FIG. 23 shows a schematic structural diagram of a movable platform according to another embodiment of the present application.
  • FIG. 24 shows a schematic diagram of a semantic recognition process of an image according to an embodiment of the present application.
  • FIG. 25 shows a schematic diagram of a process of acquiring position information of a landing point according to an embodiment of the present application.
  • the following describes a method for constructing a semantic map, a system for constructing a semantic map, a removable platform, a computer-readable storage medium and a removable platform, and a method for searching for landing points according to some embodiments of the present application with reference to FIGS. 1-25.
  • FIG. 1 shows a schematic flowchart of a method for constructing a semantic map according to an embodiment of the present application.
  • the method for constructing the semantic map includes:
  • S102 Acquire semantic segmentation information of multiple images
  • the process of obtaining the occluded rendering image is shown in Figures 2 to 4, where the image shown in Figure 2 is an image obtained at any time, and the image shown in Figure 2 is semantically segmented to generate
  • the semantic recognition map is occluded on the original image to obtain the occluded rendering map shown in Figure 4, where different background colors in Figure 3 refer to different semantics.
  • the recognition result for example, can be exemplified by color in specific embodiments, setting sky blue to represent the sky, blue purple to represent the ground, sapphire blue to represent trees, lake blue to represent water, yellow to represent buildings, and white to represent other entities. It is understandable that other different colors can also be used to represent different semantic recognition results.
  • S104 Perform a splicing operation on multiple images to generate a spliced image, and obtain a semantic map of the spliced image according to the semantic segmentation information of the multiple images.
  • the semantic recognition map of the single frame image is overlaid on the spliced image to obtain the semantic map of the spliced image.
  • the method for constructing a semantic map obtains the semantic segmentation information of multiple images, and the multiple images can be images of scenes with different perspectives and different background information, which is conducive to obtaining complete semantic segmentation information from multiple images. , Accurate the information of multiple entities in the real scene.
  • the spliced image is generated by splicing multiple images, which helps to ensure the completeness and authenticity of the scene.
  • the semantics of the spliced image can be obtained.
  • the map makes the semantic map better tend to the real scene, and fully and accurately reflects the multiple entity content of the real scene, so that the obtained semantic map has a higher degree of confidence and improves the accuracy of scene understanding. Semantic maps can accurately obtain location information.
  • multiple images are collected by an image acquisition device, and the acquisition time of the multiple images is continuous.
  • the image acquisition device includes but is not limited to: a vision sensor, a radar, and a multispectral sensor.
  • acquiring semantic segmentation information of multiple images specifically includes: performing semantic segmentation on multiple images through a preset convolutional neural network module to obtain semantic segmentation information of multiple images.
  • multiple images are semantically segmented through a preset convolutional neural network module to obtain semantic segmentation information of multiple images, which can completely and accurately obtain the information of the entity content in the image, which is beneficial to
  • the semantic map obtained by the semantic segmentation information of multiple images completely and accurately reflects the multiple entity content of the real scene, has a high degree of confidence, and improves the accuracy of scene understanding.
  • CNN Convolutional Neural Networks
  • CNN Convolutional Neural Networks
  • a plurality of images are continuously collected by an image acquisition device, and the plurality of images are input into a preset convolutional neural network module for semantic segmentation to obtain semantic separation information, which is generated by splicing the semantic separation information and multiple images.
  • Mosaic images build a semantic map. By continuously acquiring images, a more complete and detailed semantic map can be obtained; further by obtaining semantic segmentation information of multiple images through a convolutional neural network, the entity scene in the image can be obtained more accurately, and then a more accurate semantic map can be obtained.
  • Fig. 5 shows a schematic flowchart of a method for constructing a semantic map according to another embodiment of the present application.
  • the method for constructing the semantic map includes:
  • S202 Acquire semantic segmentation information of multiple images, where the semantic segmentation information corresponding to any one of the multiple images includes semantic recognition results of several pixels;
  • S206 Perform a splicing operation on multiple images to generate a spliced image, and obtain a semantic map of the spliced image according to the semantic segmentation information of the multiple images.
  • the semantic segmentation information corresponding to any one of the multiple images includes the semantic recognition results of several pixels, before the step of splicing multiple images to generate a spliced image, by acquiring each pixel
  • the confidence level of the semantic recognition result is deleted, and the semantic recognition results whose confidence level is lower than the preset threshold are deleted, so that only the semantic recognition results with higher confidence are included in the semantic segmentation information, that is, the semantic segmentation information can truly and completely reflect the image correspondence
  • the information of the entity content which in turn enables the semantic map obtained based on the semantic segmentation information of multiple images to have a high degree of confidence, which can completely and accurately reflect the entity content of the real scene, improve the accuracy of scene understanding, and make the semantic map Can accurately obtain location information.
  • the semantic recognition result may be the entity content in the real scene corresponding to the image. It is understandable that there may be multiple semantic recognition results, corresponding to multiple entity contents in the real scene.
  • the semantic recognition result may correspond to the sky, ground, trees, buildings, etc. in the physical scene, and the entities in the image are semantically recognized according to different pixels corresponding to different entities.
  • the semantic segmentation information includes the semantic recognition results of several pixels and the confidence corresponding to the semantic recognition results.
  • the semantic recognition results with the confidence lower than the preset threshold are deleted. Therefore, the semantic segmentation information only includes the semantic recognition results with high confidence, that is, the semantic segmentation information can truly and completely reflect the information of the entity content corresponding to the image, so that the location information can be accurately obtained through the semantic map.
  • FIG. 6 shows a schematic flowchart of a method for constructing a semantic map according to another embodiment of the present application. As shown in Figure 6, the method for constructing the semantic map includes:
  • S302 Acquire semantic segmentation information of multiple images, where the semantic segmentation information corresponding to any one of the multiple images includes semantic recognition results of several pixels;
  • S304 Obtain the confidence of the semantic recognition result of each pixel, and delete the semantic recognition result whose confidence is lower than a preset threshold;
  • S308 Obtain a semantic map of the stitched image according to the semantic segmentation information of the multiple images.
  • the method for constructing a semantic map is suitable for a movable platform, which can be an airplane, an unmanned aerial vehicle, or other movable platforms that meet the requirements.
  • a movable platform which can be an airplane, an unmanned aerial vehicle, or other movable platforms that meet the requirements.
  • the multiple images are spliced to generate a spliced image, which helps to ensure that the generated spliced image has high definition and reduction degree, and the height of each entity is real Reflected in the stitched image, it is beneficial to obtain the semantic map of the stitched image with higher confidence and improve the accuracy of scene understanding.
  • the height information of the entity is measured by a monocular camera, a binocular camera or a laser on the movable platform.
  • the two-dimensional semantic map is spliced according to the recognition results of the height information and semantic segmentation information of the entities corresponding to the multiple images by the movable platform, so that the semantic map of the spliced images can accurately obtain the target in the scene.
  • Semantic information and distance information make the location information obtained through the semantic map more accurate, which in turn facilitates the accurate movement of the movable platform according to the target semantic information and distance information, improves the safety and accuracy of the movement of the movable platform, and helps improve the product Reliability.
  • the target semantic information may be information of a target entity in multiple entity contents corresponding to the image
  • the distance information may be the distance between the movable platform and the target entity.
  • the movable platform is a drone
  • the target semantic information is the semantic information corresponding to the ground in the image
  • the distance information is the distance between the drone and the ground. The distance from the ground can accurately obtain the location information of the ground, so that the drone can land on the ground safely and accurately.
  • single frame recognition can be performed on any image to obtain the semantic recognition result of each pixel, and multiple images can be continuously collected and combined with the height information of the movable platform relative to the entity corresponding to the multiple images to perform image stitching , To achieve multi-frame construction of real-time semantic map. It is understandable that the semantic recognition result of each pixel can also be obtained in other ways. Specifically, when the movable platform is a drone, the height is the distance between the entity structure corresponding to the image and the drone. Specifically, during image stitching, the overlapping parts of multiple images can be merged. For example, the confidence of the recognition result of each pixel in the overlapping part of multiple images is compared, and the confidence is deleted by retaining the image with higher confidence.
  • the images with a low degree of accuracy are merged with the images of the overlapping parts, that is, the beneficial information in each image is extracted to the greatest extent, so that the merged spliced image can ensure the integrity and authenticity of the scene, thereby making the semantic map have a higher Confidence level.
  • the semantic recognition result includes at least one of the following: buildings, sky, trees, water surface, and ground.
  • the semantic recognition result includes one or more of buildings, sky, trees, water surface, and ground
  • the multiple types of semantic recognition results include multiple entity contents in the real scene corresponding to the picture.
  • the semantic results can truly and completely reflect the entity content corresponding to the image, which is beneficial to improve the accuracy of scene understanding.
  • semantic recognition result may also include other content that meets the requirements.
  • Fig. 7 shows a schematic flow chart of a method for constructing a semantic map according to another embodiment of the present application.
  • the method for constructing the semantic map includes:
  • S402 Collect multiple images according to a preset frequency
  • S404 Acquire semantic segmentation information of multiple images, where the semantic segmentation information corresponding to any one of the multiple images includes semantic recognition results of several pixels;
  • S406 Obtain the confidence of the semantic recognition result of each pixel, and delete the semantic recognition result whose confidence is lower than a preset threshold;
  • S410 Obtain a semantic map of the stitched image according to the semantic segmentation information of the multiple images.
  • the entity content of the background information is conducive to obtaining complete and accurate multiple entity content in the real scene through the semantic segmentation information of multiple images, thereby ensuring the reliability and accuracy of the semantic map.
  • Fig. 8 shows a schematic flowchart of a method for constructing a semantic map according to another embodiment of the present application. As shown in Figure 8, the method for constructing the semantic map includes:
  • any image obtained at any time is semantically segmented to generate a single-frame image semantic recognition map as shown in Figure 10.
  • the image semantic recognition The picture includes sky, ground, trees, water, and buildings.
  • the process of obtaining the occluded rendering image is shown in FIG. 9 to FIG. 11.
  • the image shown in FIG. 9 is an image obtained at any time.
  • the single-frame image semantics shown in FIG. 10 is generated.
  • Recognition map occlude the semantic recognition map on the original image to obtain the occluded rendering map shown in Figure 11, where the different background colors shown in Figure 10 refer to different semantic recognition results, for example, specific implementation
  • color can be used as an example, such as setting sky blue to represent the sky, blue purple to represent the ground, sapphire blue to represent trees, lake blue to represent water, yellow to represent buildings, and white to represent other entities. It is understandable that other different colors can also be used to represent different semantic recognition results.
  • S504 Acquire semantic segmentation information of multiple images, where the semantic segmentation information corresponding to any one of the multiple images includes semantic recognition results of several pixels;
  • S506 Obtain the confidence of the semantic recognition result of each pixel, and delete the semantic recognition result whose confidence is lower than a preset threshold;
  • S510 Obtain a semantic map of the stitched image according to the semantic segmentation information of the multiple images
  • S512 Determine the landing point of the movable platform according to the semantic map.
  • the semantic recognition map of a single frame image is overlaid on the spliced image to obtain the semantic map of the single frame spliced image as shown in Figure 12, where A in Figure 12 represents the dropable area displayed in the single frame of the spliced image It is understandable that A in Fig. 12 can also be a specific point, which represents the landing point.
  • the semantic recognition map of the multi-frame image is overlaid on the spliced image to obtain the semantic map of the multi-frame spliced image as shown in Figure 13.
  • B in Figure 13 represents the dropable area displayed in the multi-frame spliced image It can be understood that B in Fig. 13 can also be a specific point, which represents the landing point. It can be seen from the comparison of the landing areas in Figures 12 and 13 that the embodiment shown in Figure 13 can completely and accurately reflect the entity content in the real scene through multiple images, thereby constructing a more complete, accurate and detailed semantic map, and then By using a relatively complete and detailed semantic map to guide the flight of the movable platform, the controllability of the flight of the movable platform is improved.
  • S514 Control the movable platform to land according to the landing point of the movable platform.
  • the position information can be accurately obtained according to the semantic map with high confidence and high accuracy of scene understanding, and then the landing point of the movable platform is determined , And the landing point is safe and reliable, and the movable platform is controlled to land according to the landing point of the movable platform, so that the movable platform can safely, reliably and accurately land to the landing point determined by the semantic map, avoiding the movable in related technologies
  • the problem of damaging or damaging the movable platform when the platform falls in the water, on trees, buildings, etc. greatly prolongs the service life of the movable platform, improves the safety of the use of the movable platform, and improves the reliability of the product.
  • Fig. 14 shows a schematic flowchart of a method for constructing a semantic map according to another embodiment of the present application. As shown in Figure 14, the method for constructing the semantic map includes:
  • S602 Collect multiple images according to a preset frequency
  • S604 Acquire semantic segmentation information of multiple images, where the semantic segmentation information corresponding to any one of the multiple images includes semantic recognition results of several pixels;
  • S606 Obtain the confidence of the semantic recognition result of each pixel, and delete the semantic recognition result whose confidence is lower than a preset threshold;
  • S610 Obtain a semantic map of the stitched image according to the semantic segmentation information of the multiple images
  • S612 Determine the landing area of the movable platform according to the semantic map
  • S614 According to the state information of the movable platform, select a landing point in the landing area;
  • S616 Control the movable platform to land according to the landing point of the movable platform.
  • the landable area can be a safe and reliable area that allows the movable platform to land according to the semantic map, that is, does not include There are dangerous or destructive areas in the landing of the platform, such as water, trees, buildings and other areas, thereby avoiding damage or damage to the movable platform during landing, which is conducive to extending the service life of the movable platform; according to the status information of the movable platform , Selecting a landing point in the landing area is conducive to combining the status information of the movable platform, so that the selected landing point can ensure the safe and reliable landing of the movable platform, and avoid the failure of the movable platform to reach smoothly due to its own state
  • the landing point may not be able to successfully land at the landing point, which further ensures that the movable platform can safely, smoothly, reliably and accurately land at the landing point, and improves the reliability of the movable platform.
  • Fig. 15 shows a schematic flowchart of a method for constructing a semantic map according to another embodiment of the present application.
  • the method for constructing the semantic map includes:
  • S702 Collect multiple images according to a preset frequency
  • S704 Acquire semantic segmentation information of multiple images, where the semantic segmentation information corresponding to any one of the multiple images includes semantic recognition results of several pixels;
  • S706 Obtain the confidence of the semantic recognition result of each pixel, and delete the semantic recognition result whose confidence is lower than a preset threshold;
  • S710 Obtain a semantic map of the stitched image according to the semantic segmentation information of the multiple images
  • S712 Determine the landing area of the movable platform according to the semantic map
  • S716 According to the remaining power and the semantic map, select a landing point in the landing area;
  • the step of selecting a landing point in the landable area according to the state information of the movable platform is specifically defined.
  • the landing point is selected in the landing area, so that the selected landing point can ensure that the mobile platform uses the remaining power to land on the landing point smoothly, avoiding The remaining power of the battery cannot make the movable platform reach the landing point smoothly and damage or destroy the movable equipment, so that the selected landing point has high accuracy, thereby ensuring that the movable platform can reliably and safely complete the landing.
  • Fig. 16 shows a schematic flowchart of a method for constructing a semantic map according to another embodiment of the present application.
  • the method for constructing the semantic map includes:
  • S802 Collect multiple images according to a preset frequency
  • S804 Acquire semantic segmentation information of multiple images, where the semantic segmentation information corresponding to any one of the multiple images includes semantic recognition results of several pixels;
  • S806 Obtain the confidence of the semantic recognition result of each pixel, and delete the semantic recognition result whose confidence is lower than a preset threshold;
  • S810 Obtain a semantic map of the stitched image according to the semantic segmentation information of the multiple images
  • the landing point is selected according to the flight trajectory and the remaining power, so that the selected landing point is adapted to the flight trajectory, which is beneficial to
  • the movable platform realizes the return home according to the flight trajectory, improves the accuracy of the return home of the movable platform, and can ensure that the movable platform uses the remaining power to land at the landing point smoothly, thereby improving the reliability and safety of the landing of the movable platform.
  • Fig. 17 shows a schematic flowchart of a method for constructing a semantic map according to another embodiment of the present application. As shown in Figure 17, the method for constructing the semantic map includes:
  • S902 Collect multiple images according to a preset frequency
  • S904 Acquire semantic segmentation information of multiple images, where the semantic segmentation information corresponding to any one of the multiple images includes semantic recognition results of several pixels;
  • S906 Obtain the confidence of the semantic recognition result of each pixel, and delete the semantic recognition result whose confidence is lower than a preset threshold;
  • S910 Obtain a semantic map of the stitched image according to the semantic segmentation information of the multiple images
  • S912 Determine the landing area of the movable platform according to the semantic map
  • S916 Determine the remaining cruising range of the battery according to the remaining power of the battery
  • the remaining power of the battery of the movable platform and the flight trajectory of the movable platform are obtained according to the semantic map, and the remaining range of the battery is determined according to the remaining power of the battery, and the remaining power of the battery is specifically quantified as a battery According to the remaining cruising range and flight trajectory, the landing point is selected, so that the landing point can be accurately and reasonably selected according to the quantified remaining cruising range and flight trajectory, which is conducive to improving the accuracy of the landing position information. It can ensure the safe and reliable landing of the movable platform at the landing point, and the remaining cruising range determined based on the remaining power to complete the return according to the flight trajectory to the maximum, thereby improving the accuracy of the return of the movable platform.
  • Fig. 18 shows a schematic flowchart of a method for constructing a semantic map according to another embodiment of the present application. As shown in Figure 18, the method for constructing the semantic map includes:
  • S1004 Acquire semantic segmentation information of multiple images, where the semantic segmentation information corresponding to any one of the multiple images includes semantic recognition results of several pixels;
  • S1006 Obtain the confidence level of the semantic recognition result of each pixel, and delete the semantic recognition result whose confidence is lower than a preset threshold;
  • S1010 Obtain a semantic map of the stitched image according to the semantic segmentation information of the multiple images
  • S1012 Determine the landing area of the movable platform according to the semantic map
  • S1016 Determine the remaining cruising range of the battery according to the remaining power of the battery
  • S1020 Determine the estimated return mileage of the movable platform according to the flight trajectory and semantic map
  • the step of selecting the landing point according to the remaining cruising range and flight trajectory is specifically defined.
  • the mileage returned to the departure point of the flight trajectory is based on two situations where the estimated return mileage is less than or equal to the remaining cruising mileage and the estimated return mileage is greater than the remaining cruising mileage.
  • the mobile platform can return to the take-off point of the flight trajectory by using the remaining battery power, and then use the take-off point of the flight trajectory as the landing point to further improve the accuracy of the landing point, so that the mobile platform can be safely, reliably and accurately Landing at the take-off point improves the accuracy of the return of the movable platform.
  • the mobile platform cannot return to the take-off point of the flight trajectory by using the remaining battery power.
  • the mobile platform is guaranteed Able to successfully complete the landing, and achieve a safe and reliable landing, avoiding the estimated return mileage to be greater than the remaining cruising mileage, and setting the landing point as the take-off point of the flight trajectory makes the movable platform unable to complete the landing smoothly and there is a problem of damage or damage , Further improve the reliability of the movable platform and extend the service life of the movable platform.
  • the take-off point may be the starting point of the flight trajectory, or a designated home point, or a point in the designated flight plan, such as other points set close to the home point.
  • Fig. 19 shows a schematic flowchart of a method for constructing a semantic map according to another embodiment of the present application.
  • the method for constructing the semantic map includes:
  • S1104 Acquire semantic segmentation information of multiple images, where the semantic segmentation information corresponding to any one of the multiple images includes semantic recognition results of several pixels;
  • S1106 Obtain the confidence of the semantic recognition result of each pixel, and delete the semantic recognition result whose confidence is lower than a preset threshold;
  • S1110 Obtain a semantic map of the stitched image according to the semantic segmentation information of the multiple images
  • S1112 Determine the landing area of the movable platform according to the semantic map
  • S1116 Determine the remaining cruising range of the battery according to the remaining power of the battery
  • S1120 Determine the estimated return mileage of the movable platform according to the flight trajectory and semantic map
  • S1126 Control the movable platform to perform obstacle avoidance flight according to the semantic map; among them, obstacle avoidance flight includes detour flight or climb flight.
  • the landing area of the movable platform is determined according to the semantic map and the movable platform is controlled to fly against obstacles.
  • the semantic map has high confidence. It can completely and accurately obtain the position information of obstacles in the real scene, and control the movable platform to fly obstacles to avoid obstacles, which is beneficial to improve the reliability of the flight of the movable platform, thereby extending the service life of the movable platform and increasing Product reliability.
  • obstacle avoidance flight includes detour flight or climb flight.
  • Detour flight means flying around obstacles, and climbing flight means flying upwards over obstacles. Understandably, it can also include other flight modes, such as detour flight and crawling flight at the same time. .
  • the obstacle avoidance flight can be used for obstacle avoidance flight during the return home process, or the mobile platform can perform obstacle avoidance flight based on the semantic map to further improve the reliability of flight.
  • the movable platform includes a collection device
  • the construction method further includes: controlling the collection device to collect multiple images.
  • the method for collecting multiple images in the construction method of the semantic map is specifically limited, and the multiple images are collected by controlling the collecting device of the movable platform, which is simple to operate and easy to implement.
  • multiple collection devices there may be multiple collection devices, and multiple collection devices can collect images of scenes with different viewing angles and different background information, thereby helping to improve the confidence of the semantic map. It can be understood that multiple collection devices are arranged at different positions of the movable platform, so as to collect images of different flight attitudes, different viewing angles, and different background information of the movable platform.
  • the method further includes: according to the flying posture of the movable platform, controlling the collecting device on the side of the movable platform toward the ground to collect multiple images.
  • the landing point of the movable platform is generally set on the ground, that is, the landing point of the movable platform is finally landing on the ground.
  • the acquisition device collects multiple images to obtain a semantic map on the side of the ground, which is conducive to the safe, reliable and accurate landing of the movable platform on the landing point on the ground. It is highly operable, easy to implement, and suitable for popularization and application.
  • the acquisition device on the side close to the ideal landing point can also collect multiple images according to the orientation of the ideal landing point, so that the movable platform can land on the ideal landing point safely, reliably and accurately. , To further expand the scope of use of products.
  • the acquisition device includes a radar, a vision sensor or a multispectral sensor.
  • the collection device may be a radar, a vision sensor or a multispectral sensor.
  • the various types of collection devices can meet the requirements of different installation positions of the collection device, collection of images from different perspectives, and collection of images with different background information.
  • the different cost requirements of mobile platforms are conducive to expanding the scope of use of products.
  • the collection device may also be other devices that meet the requirements.
  • FIG. 20 shows a schematic flowchart of a method for constructing a semantic map according to another embodiment of the present application. As shown in Figure 20, the method for constructing the semantic map includes:
  • S1202 Receive a take-off instruction and control the start of the collection device to collect multiple images
  • S1204 Acquire semantic segmentation information of multiple images, where the semantic segmentation information corresponding to any one of the multiple images includes semantic recognition results of several pixels;
  • S1206 Obtain the confidence of the semantic recognition result of each pixel, and delete the semantic recognition result whose confidence is lower than a preset threshold;
  • S1210 Obtain a semantic map of the stitched image according to the semantic segmentation information of the multiple images
  • S1214 Determine the landing area of the movable platform according to the semantic map
  • S1218 Determine the remaining cruising range of the battery according to the remaining power of the battery
  • S1222 Determine the estimated return mileage of the movable platform according to the flight trajectory and semantic map
  • S1228 Control the movable platform to perform obstacle avoidance flight according to the semantic map; wherein, obstacle avoidance flight includes detour flight or climb flight.
  • the acquisition device is controlled to start to collect multiple images, that is, when the movable platform takes off, it starts to collect multiple images, and constructs a semantic map in real time.
  • the control acquisition device is turned off, that is, when the movable platform needs to return home, the control acquisition device is turned off, stops collecting images, and accurately obtains position information according to the construction of a semantic map, and then determines the landing point of the movable platform, that is, landing
  • the location information enables the movable platform to safely, reliably and accurately land at the landing point and complete the return flight, avoiding the problem of damage or damage to the movable platform in related technologies when the movable platform is landed in the water, on a tree, or on a building.
  • the service life of the movable platform is greatly extended, the safety of the use of the movable platform is improved, and the reliability of the product is improved.
  • the return instruction may be a return instruction triggered by the return key selected by the user; on the other hand, the return instruction is a return instruction sent by the controller of the movable platform when the movable platform flies to the home point of the flight trajectory.
  • the different ways of returning instructions can meet the needs of different working conditions of the movable platform, thereby expanding the scope of use of the product. At the same time, it is conducive to flexible control of the movable platform to return to home safely, and further improves the reliability of the movable platform.
  • an embodiment of the second aspect of the present application proposes a semantic map construction system 10, including: a memory 12 for storing computer programs; a processor 14 for executing the computer programs to realize: Acquire semantic segmentation information of multiple images; perform a stitching operation on multiple images to generate a stitched image, and obtain a semantic map of the stitched image according to the semantic segmentation information of the multiple images.
  • the semantic map construction system 10 includes a memory 12 and a processor 14.
  • the memory 12 is used to store a computer program.
  • the processor 14 obtains semantic segmentation information of multiple images, and the multiple images can be different Images of scenes with different perspectives and background information are conducive to obtaining complete and accurate information about multiple entities of the real scene through the semantic segmentation information of multiple images, and the processor 14 performs a splicing operation on multiple images to generate a spliced image It helps to ensure the completeness and authenticity of the scene.
  • the semantic map of the spliced image is obtained, so that the semantic map is better to the real scene, and it fully and accurately reflects the many real scenes.
  • Each entity content makes the obtained semantic map have a high degree of confidence, improves the accuracy of scene understanding, and accurately obtains location information through the semantic map.
  • multiple images are collected by an image acquisition device, and the acquisition time of the multiple images is continuous.
  • the image acquisition device includes but is not limited to: vision sensors, radars, and multi-spectral sensors.
  • the processor 14 configured to perform the acquisition of semantic segmentation information of multiple images is specifically: performing semantic segmentation on multiple images through a preset convolutional neural network module to obtain multiple images Semantic segmentation information.
  • the processor 14 is configured to perform the acquisition of semantic segmentation information of multiple images. Specifically, the processor 14 performs semantic segmentation on multiple images through a preset convolutional neural network module to obtain semantic segmentation of multiple images. Information, can obtain the information of the entity content in the image completely and accurately, which is conducive to the semantic map obtained based on the semantic segmentation information of multiple images, which fully and accurately reflects the multiple entity content of the real scene, with high confidence Increase the accuracy of scene understanding.
  • the processor 14 may also obtain the semantic segmentation information of multiple images through other methods.
  • Convolutional Neural Networks CNN, Convolutional Neural Networks
  • CNN is suitable for various scene tasks, especially the acquisition of semantic information and location information of scene targets. Therefore, CNN can be used to identify the semantic information and semantic information of various targets in aerial scenes. location information.
  • the processor 14 continuously collects multiple images through an image acquisition device, inputs the multiple images into a preset convolutional neural network module for semantic segmentation to obtain semantic separation information, and performs processing based on the semantic separation information and multiple images.
  • the stitched images generated by stitching construct a semantic map.
  • the processor 14 can obtain a more complete and detailed semantic map by continuously collecting images; further, the processor 14 can obtain the semantic segmentation information of multiple images through a convolutional neural network, and can obtain more accurate physical scenes in the images, and then obtain more Accurate semantic map.
  • the semantic segmentation information corresponding to any one of the multiple images includes the semantic recognition results of a plurality of pixels
  • the processor 14 is configured to perform stitching on the multiple images to generate stitching. Before the image step, it is also used to: obtain the confidence of the semantic recognition result of each pixel, and delete the semantic recognition result whose confidence is lower than the preset threshold.
  • the processor 14 obtains each image before the step of stitching the multiple images to generate a stitched image.
  • the confidence level of the semantic recognition result of a pixel is deleted.
  • the semantic recognition result whose confidence is lower than the preset threshold is deleted, so that only the semantic recognition result with higher confidence is included in the semantic segmentation information, that is, the semantic segmentation information can be truly and completely Reflects the information of the entity content corresponding to the image, which in turn makes the semantic map obtained from the semantic segmentation information of multiple images have a high degree of confidence, can completely and accurately reflect the entity content of the real scene, and improve the accuracy of scene understanding. Location information can be accurately obtained through semantic maps.
  • the semantic recognition result may be the entity content in the real scene corresponding to the image. It is understandable that there may be multiple semantic recognition results, corresponding to multiple entity contents in the real scene.
  • the semantic segmentation information includes the semantic recognition results of several pixels and the confidence corresponding to the semantic recognition results.
  • the semantic recognition results with the confidence lower than the preset threshold are deleted. Therefore, the semantic segmentation information only includes the semantic recognition results with high confidence, that is, the semantic segmentation information can truly and completely reflect the information of the entity content corresponding to the image, so that the location information can be accurately obtained through the semantic map.
  • the processor 14 is used to implement: according to the height information of the movable platform relative to the entity corresponding to the multiple images, the multiple images are spliced Generate stitched images.
  • the semantic map construction system 10 is suitable for a movable platform, and the movable platform may be an airplane, an unmanned aerial vehicle, or other movable platforms that meet the requirements.
  • the processor 14 splices multiple images to generate a spliced image according to the height information of the entity corresponding to the multiple images of the movable platform, which is beneficial to ensure that the generated spliced image has a high degree of clarity and reduction.
  • the high degree of reality is embodied in the stitched image, which in turn facilitates the acquisition of the semantic map of the stitched image with a higher degree of confidence, and improves the accuracy of scene understanding.
  • the height information of the entity is measured by binocular cameras on the movable platform.
  • the processor 14 performs the stitching of the two-dimensional semantic map according to the recognition result of the height information of the entity corresponding to the multiple images and the semantic segmentation information of the movable platform, so that the semantic map of the stitched image can accurately obtain the scene.
  • the target semantic information and distance information in the semantic map make the position information obtained through the semantic map more accurate, which in turn facilitates the accurate movement of the movable platform according to the target semantic information and distance information, and improves the safety and accuracy of the movement of the movable platform. Conducive to improving product reliability.
  • the target semantic information may be information of a target entity in multiple entity contents corresponding to the image
  • the distance information may be the distance between the movable platform and the target entity.
  • the movable platform is a drone
  • the target semantic information is the semantic information corresponding to the ground in the image
  • the distance information is the distance between the drone and the ground. The distance from the ground can accurately obtain the location information of the ground, so that the drone can land on the ground safely and accurately.
  • the processor 14 may perform single-frame recognition on any image to obtain the semantic recognition result of each pixel, and continuously collect multiple images and combine the height information of the movable platform relative to the entity corresponding to the multiple images. Carry out image stitching to realize multi-frame construction of real-time semantic map. It is understandable that the processor 14 may also obtain the semantic recognition result of each pixel in other ways. Specifically, when the movable platform is a drone, the height is the distance between the entity structure corresponding to the image and the drone. Specifically, during image stitching, the overlapping parts of multiple images can be merged. For example, the confidence of the recognition result of each pixel in the overlapping part of multiple images is compared, and the confidence is deleted by retaining the image with higher confidence.
  • the images with a low degree of accuracy are merged with the images of the overlapping parts, that is, the beneficial information in each image is extracted to the greatest extent, so that the merged spliced image can ensure the integrity and authenticity of the scene, thereby making the semantic map have a higher Confidence level.
  • the semantic recognition result includes at least one of the following: buildings, sky, trees, water surface, and ground.
  • the semantic recognition result includes one or more of buildings, sky, trees, water surface, and ground, and the multiple types of semantic recognition results include multiple entity contents in the real scene corresponding to the picture.
  • the semantic results can truly and completely reflect the entity content corresponding to the image, which is beneficial to improve the accuracy of scene understanding.
  • the semantic recognition result may also include other content that meets the requirements.
  • the processor 14 is further configured to implement: collecting multiple images according to a preset frequency.
  • the processor 14 can obtain images of scenes with different viewing angles and different background information by collecting multiple images at a preset frequency, so that multiple images can completely and accurately reflect the different perspectives and different background information in the real scene.
  • the entity content of the location and different background information is conducive to obtaining a complete and accurate multiple entity content in the real scene through the semantic segmentation information of multiple images, thereby ensuring the reliability and accuracy of the semantic map.
  • the processor 14 is further configured to: determine the landing point of the movable platform according to the semantic map; and control the movable platform to land according to the landing point of the movable platform.
  • the processor 14 determines the landing point of the movable platform according to the semantic map, so that the semantic map with high confidence and high accuracy of scene understanding can accurately obtain position information, and then determine the movable platform The landing point is safe and reliable.
  • the processor 14 controls the movable platform to land according to the landing point of the movable platform, so that the movable platform can safely, reliably and accurately land to the landing point determined by the semantic map, avoiding It solves the problem of damage or damage to the movable platform when the movable platform falls in the water, on trees, buildings, etc. in the related technology, greatly prolongs the service life of the movable platform, improves the safety of the use of the movable platform, and improves the product reliability.
  • the processor 14 is configured to determine the landing point of the movable platform according to the semantic map: determining the landing area of the movable platform according to the semantic map; Status information, select the landing point in the landing area.
  • the processor 14 determines the landable area of the movable platform according to the semantic map.
  • the landable area may be a safe and reliable area that allows the movable platform to land according to the semantic map, that is, does not include
  • the movable platform can be landed in dangerous or destructive areas, such as water, trees, buildings, etc., so as to avoid damage or damage to the movable platform during landing, which is beneficial to prolong the service life of the movable platform; the processor 14 passes according to The status information of the movable platform and the selection of the landing point in the landable area is beneficial to combine the status information of the movable platform, so that the selected landing point can ensure the safe and reliable landing of the movable platform, and avoid the movable platform itself
  • the state cannot meet the requirements of reaching the landing point smoothly or failing to complete the landing at the landing point, which further ensures that the movable platform can safely, smoothly, reliably and accurately land at the landing point, and improves the reliability of the movable platform.
  • the processor 14 is configured to implement the step of selecting a landing point in the landable area according to the status information of the movable platform, specifically: obtaining the remaining battery of the movable platform Power; according to the remaining power and semantic map, select the landing point in the landing area.
  • the steps for the processor 14 to select a landing point in the landing area according to the state information of the movable platform are specifically defined.
  • the processor 14 obtains the remaining power of the battery of the mobile platform, and selects a landing point in the landing area according to the remaining power and the semantic map, so that the selected landing point can ensure that the mobile platform uses the remaining power to land at the landing point smoothly , To avoid the remaining battery power can not make the movable platform reach the landing point smoothly and damage or damage the movable equipment, so that the selected landing point has a high accuracy, thereby ensuring that the movable platform can reliably and safely complete the landing , Extend the service life of the movable platform.
  • the processor 14 is further configured to obtain the flight trajectory of the movable platform, and select the landing point according to the flight trajectory and the remaining power.
  • the processor 14 obtains the flight trajectory of the movable platform, and selects the landing point according to the flight trajectory and the remaining power, so that the selected landing point is adapted to the flight trajectory, which is beneficial for the movable platform to follow the flight trajectory Realize the return to home, improve the accuracy of the return of the movable platform, and at the same time ensure that the movable platform uses the remaining power to land at the landing point, thereby improving the reliability and safety of the landing of the movable platform.
  • the processor 14 is further configured to determine the remaining cruising range of the battery according to the remaining power of the battery, and selecting the landing point according to the remaining cruising range and flight trajectory.
  • the processor 14 determines the remaining cruising range of the battery according to the remaining power of the battery, and quantifies the remaining power of the battery as the remaining cruising range of the battery, so that the quantified remaining cruising range and flight trajectory are accurate and reasonable. Choosing a landing point on the ground is conducive to improving the accuracy of the landing position information. It can ensure the safe and reliable landing of the movable platform at the landing point, and the remaining cruising range determined based on the remaining power to complete the return flight according to the flight trajectory to the maximum. Improve the accuracy of the return of the movable platform.
  • the processor 14 is configured to implement the step of selecting a landing point according to the remaining cruising range and flight trajectory, specifically: determining the estimation of the movable platform according to the flight trajectory and the semantic map Return mileage: If the estimated return mileage is less than or equal to the remaining cruising mileage, the take-off point of the flight trajectory is taken as the landing point.
  • the steps for the processor 14 to select the landing point according to the remaining cruising range and flight trajectory are specifically defined.
  • the processor 14 determines the estimated return mileage of the movable platform based on the flight trajectory and semantic map, that is, the estimated return mileage is the mileage of the movable platform back to the departure point of the flight trajectory, and the processor 14 is based on the estimated return mileage less than or When it is equal to the remaining cruising range, it means that the mobile platform can return to the take-off point of the flight trajectory by using the remaining battery power, and then use the take-off point of the flight trajectory as the landing point to further improve the accuracy of the landing point, so that the mobile platform can Landing at the take-off point safely, reliably and accurately improves the accuracy of the return of the movable platform.
  • the take-off point may be the starting point of the flight trajectory, or a designated home point, or a point in the designated flight plan, such as other points set close to the home point.
  • the processor 14 is also used to realize: according to the remaining cruising mileage and the take-off point, select a landing point in the landable area .
  • the processor 14 is based on the condition that the estimated return mileage is greater than the remaining cruising range, indicating that the mobile platform cannot return to the take-off point of the flight trajectory by using the remaining battery power.
  • the processor 14 is further configured to: control the movable platform to perform obstacle avoidance flight according to the semantic map; wherein, obstacle avoidance flight includes detour flight or climb flight.
  • the processor 14 controls the movable platform to perform obstacle avoidance flight based on the semantic map. Since the semantic map has a high degree of confidence, it can completely and accurately obtain the location information of obstacles in the real scene. 14 Control the movable platform to fly obstacles to avoid obstacles, which is helpful to improve the reliability of the movable platform, thereby extending the service life of the movable platform and improving the reliability of the product.
  • obstacle avoidance flight includes detour flight or climb flight.
  • Detour flight means flying around obstacles, and climbing flight means flying upwards over obstacles. Understandably, it can also include other flight modes, such as detour flight and crawling flight at the same time. .
  • the obstacle avoidance flight can be used for obstacle avoidance flight during the return home process, or the mobile platform can perform obstacle avoidance flight based on the semantic map to further improve the reliability of flight.
  • the movable platform includes an acquisition device, and the processor 14 is further configured to control the acquisition device to acquire multiple images.
  • the method for collecting multiple images in the construction method of the semantic map is specifically limited, and the multiple images are collected by controlling the collecting device of the movable platform, which is simple to operate and easy to implement.
  • multiple collection devices there may be multiple collection devices, and multiple collection devices can collect images of scenes with different viewing angles and different background information, thereby helping to improve the confidence of the semantic map. It can be understood that multiple collection devices are arranged at different positions of the movable platform, so as to collect images of different flight attitudes, different viewing angles, and different background information of the movable platform.
  • the processor 14 is further configured to realize: according to the flying posture of the movable platform, control the collecting device on the ground side of the movable platform to collect multiple images.
  • the landing point of the movable platform is generally set on the ground, that is, the mobile platform is finally the landing point on the ground.
  • the processor 14 controls the direction of the movable platform according to the flight attitude of the movable platform.
  • the collection device on the ground side collects multiple images, and then obtains the semantic map on the ground side, which is conducive to the safe, reliable and accurate landing of the movable platform on the landing point on the ground. It is highly operable, easy to implement, and suitable for promotion application.
  • the processor 14 can also make the acquisition device on the side close to the ideal landing point to collect multiple images according to the orientation of the ideal landing point, so that the movable platform can land safely, reliably and accurately. Ideal landing point to further expand the scope of use of the product.
  • the processor 14 is further configured to: receive a take-off instruction, control the start of the acquisition device to acquire multiple images; and receive a return instruction or detect a failure of the movable platform, and control the acquisition The device is turned off.
  • the processor 14 controls the start of the collection device by receiving the take-off instruction to collect multiple images, that is, when the movable platform takes off, it starts collecting multiple images and constructs a semantic map in real time.
  • the processor 14 receives Return to home instruction or detection of failure of the movable platform, control the acquisition device to close, that is, when the movable platform needs to return to home, control the acquisition device to turn off, stop collecting images, and accurately obtain position information according to the construction of semantic map, and then determine the movable platform
  • the landing point that is, the landing position information, enables the movable platform to safely, reliably and accurately land at the landing point and complete the return flight, avoiding damage or damage to the movable platform in related technologies such as landing on the water, trees, and buildings
  • the problem of the movable platform greatly extends the service life of the movable platform, improves the safety of the movable platform, and improves the reliability of the product.
  • the return instruction may be a return instruction triggered by the return key selected by the user; on the other hand, the return instruction is a return instruction sent by the controller of the movable platform when the movable platform flies to the home point of the flight trajectory.
  • the different ways of returning instructions can meet the needs of different working conditions of the movable platform, thereby expanding the scope of use of the product. At the same time, it is conducive to flexible control of the movable platform to return to home safely, and further improves the reliability of the movable platform.
  • the embodiment of the third aspect of the present application proposes a movable platform 20, including the semantic map construction system 10 of any of the above embodiments; and a collection device 22, which is similar to the construction system. Connected, the collection device 22 is used to collect images and send them to the processor. Since the movable platform 20 includes the semantic map construction system 10 of any of the above embodiments, it has all the beneficial effects of the semantic map construction system 10 of any of the above embodiments, and will not be repeated here.
  • the collection device 22 includes a radar, a vision sensor, or a multispectral sensor.
  • the collection device 22 can be a radar, a vision sensor, or a multispectral sensor.
  • the multiple types of the collection device 22 can meet the requirements of different installation positions of the collection device 22, collection of images from different perspectives, and collection of images with different background information. It can meet the different cost requirements of the movable platform 20, which is beneficial to expand the use range of the product. It can be understood that the collection device 22 may also be other devices that meet the requirements.
  • the embodiment of the fourth aspect of the present application proposes a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the method for constructing a semantic map of any of the foregoing embodiments is implemented. Therefore, it has the beneficial effects of the semantic map construction method of any of the above technical solutions, which will not be repeated here.
  • a computer-readable storage medium may include any medium capable of storing or transmitting information.
  • Examples of computer-readable storage media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, and so on.
  • the code segment can be downloaded via a computer network such as the Internet, an intranet, etc.
  • the embodiment of the fifth aspect of the present application proposes a movable platform 20, which includes a body 24, a power supply battery 26 provided on the body 24, a power system 28, a collection device 22, and a controller 21,
  • the power supply battery 26 is used to supply power to the power system 28, and the power system 28 is used to provide flight power to the movable platform 20;
  • the acquisition device 22 is used to obtain multiple images during the flight of the movable platform 20;
  • the movable platform 20 provided by the embodiment of the present application includes: a body 24, a power supply battery 26 provided on the body 24, a power system 28, a collection device 22 and a controller 21, wherein the power supply battery 26 is used to supply power to the power system 28
  • the power system 28 is used to provide flight power for the movable platform 20; the acquisition device 22 is used to acquire multiple images during the flight of the movable platform 20, and obtain the semantic segmentation information of the multiple images through the controller 21.
  • the image can be an image of a scene with different perspectives and different background information, which is conducive to obtaining complete and accurate information of multiple entity contents of the real scene through the semantic segmentation information of multiple images, and the multiple images are spliced by the controller 21
  • the operation to generate spliced images helps to ensure the completeness and authenticity of the scene.
  • the semantic map of the spliced image is obtained, so that the semantic map is better to the real scene, and fully and accurately reflects
  • the multiple entity contents of the real scene in turn enable the obtained semantic map to have a higher degree of confidence, improve the accuracy of scene understanding, and accurately obtain location information through the semantic map.
  • multiple images are acquired by the image acquisition device 22, and the acquisition time of the multiple images is continuous.
  • the image acquisition device 22 includes but is not limited to: a vision sensor, a radar, and a multispectral sensor.
  • the controller 21 is specifically configured to perform semantic segmentation on multiple images through a preset convolutional neural network module to obtain semantic segmentation information of the multiple images.
  • the controller 21 performs semantic segmentation on multiple images through a preset convolutional neural network module to obtain semantic segmentation information of multiple images, which can completely and accurately obtain the information of the entity content in the image, and then It is conducive to the semantic map obtained based on the semantic segmentation information of multiple images to completely and accurately reflect the multiple entity content of the real scene, with higher confidence, and improving the accuracy of scene understanding.
  • the controller 21 may obtain the semantic segmentation information of multiple images through other methods.
  • Convolutional Neural Networks CNN, Convolutional Neural Networks
  • CNN is suitable for various scene tasks, especially the acquisition of semantic information and location information of scene targets. Therefore, CNN can be used to identify the semantic information and semantic information of various targets in aerial scenes. location information.
  • a plurality of images are continuously collected by the image acquisition device 22, and the controller 21 inputs the plurality of images into a preset convolutional neural network module for semantic segmentation to obtain semantic separation information, and the multiple images are separated according to the semantics.
  • the stitched images generated by stitching construct a semantic map.
  • the controller 21 continuously collects images through the image acquisition device 22 to obtain a more complete and detailed semantic map; further, the controller 21 obtains the semantic segmentation information of multiple images through a convolutional neural network, which can more accurately obtain the physical scene in the image , And then get a more accurate semantic map.
  • the semantic segmentation information corresponding to any one of the multiple images includes the semantic recognition results of several pixels
  • the controller 21 is in the step of splicing the multiple images to generate a spliced image.
  • the controller 21 was also used to obtain the confidence of the semantic recognition result of each pixel, and delete the semantic recognition results whose confidence is lower than the preset threshold.
  • the controller 21 obtains each image before the step of stitching the multiple images to generate a stitched image.
  • the confidence level of the semantic recognition result of a pixel is deleted.
  • the semantic recognition result whose confidence is lower than the preset threshold is deleted, so that only the semantic recognition result with higher confidence is included in the semantic segmentation information, that is, the semantic segmentation information can be truly and completely Reflects the information of the entity content corresponding to the image, which in turn makes the semantic map obtained from the semantic segmentation information of multiple images have a high degree of confidence, can completely and accurately reflect the entity content of the real scene, and improve the accuracy of scene understanding. Location information can be accurately obtained through semantic maps.
  • the semantic recognition result may be the entity content in the real scene corresponding to the image. It is understandable that there may be multiple semantic recognition results, corresponding to multiple entity contents in the real scene.
  • the semantic segmentation information includes the semantic recognition results of several pixels and the confidence corresponding to the semantic recognition results.
  • the semantic recognition results with the confidence lower than the preset threshold are deleted. Therefore, the semantic segmentation information only includes the semantic recognition results with high confidence, that is, the semantic segmentation information can truly and completely reflect the information of the entity content corresponding to the image, so that the location information can be accurately obtained through the semantic map.
  • the controller 21 is specifically configured to splice multiple images to generate a spliced image according to the entity and height information of the movable platform 20 relative to the multiple images.
  • the controller 21 splices multiple images to generate a spliced image according to the height information of the movable platform 20 relative to the entity corresponding to the multiple images, which helps to ensure that the generated spliced image has high clarity.
  • the height of each entity is truly reflected in the spliced image, which is beneficial for obtaining a higher confidence in the semantic map of the spliced image and improves the accuracy of scene understanding.
  • the height information of the entity is measured by binocular cameras on the movable platform.
  • the controller 21 performs the stitching of the two-dimensional semantic map according to the recognition result of the height information and the semantic segmentation information of the entities corresponding to the multiple images of the movable platform 20, so that the semantic map of the stitched images can be accurately obtained.
  • the target semantic information and distance information in the scene make the position information obtained through the semantic map more accurate, which in turn facilitates the accurate movement of the movable platform 20 according to the target semantic information and distance information, and improves the safety and accuracy of the movement of the movable platform 20 It is helpful to improve the reliability of the product.
  • the target semantic information may be information of a target entity in multiple entity contents corresponding to the image, and the distance information may be the distance between the movable platform 20 and the target entity.
  • the movable platform is a drone
  • the target semantic information is the semantic information corresponding to the ground in the image
  • the distance information is the distance between the drone and the ground. The distance from the ground can accurately obtain the location information of the ground, so that the drone can land on the ground safely and accurately.
  • the controller 21 can perform single-frame recognition on any image to obtain the semantic recognition result of each pixel, and control the collection device 22 to continuously collect multiple images, combined with the movable platform 20 corresponding to the multiple images.
  • the height information of the entity is used for image splicing to realize multi-frame construction of real-time semantic map. It is understandable that the controller 21 may also obtain the semantic recognition result of each pixel in other ways. Specifically, when the movable platform 20 is a drone, the height is the distance between the physical structure corresponding to the image and the drone. Specifically, during image stitching, the overlapping parts of multiple images can be merged.
  • the confidence of the recognition result of each pixel in the overlapping part of multiple images is compared, and the confidence is deleted by retaining the image with higher confidence.
  • the images with a low degree of accuracy are merged with the images of the overlapping parts, that is, the beneficial information in each image is extracted to the greatest extent, so that the merged spliced image can ensure the integrity and authenticity of the scene, thereby making the semantic map have a higher Confidence level.
  • the semantic recognition result includes at least one of the following: buildings, sky, trees, water surface, and ground.
  • the semantic recognition result includes one or more of buildings, sky, trees, water surface, and ground, and the multiple types of semantic recognition results include multiple entity contents in the real scene corresponding to the picture.
  • the semantic results can truly and completely reflect the entity content corresponding to the image, which is beneficial to improve the accuracy of scene understanding.
  • the semantic recognition result may also include other content that meets the requirements.
  • the collection device 22 is specifically configured to collect multiple images according to a preset frequency.
  • the collection device 22 collects multiple images at a preset frequency to obtain images of scenes with different viewing angles and different background information, so that the controller 21 can completely and accurately reflect different real scenes through multiple images.
  • the entity content of the perspective, different locations, and different background information helps to obtain a complete and accurate multiple entity content in the real scene through the semantic segmentation information of multiple images, thereby ensuring the reliability and accuracy of the semantic map.
  • the controller 21 is specifically configured to: determine the landing point of the movable platform 20 according to the semantic map; according to the landing point of the movable platform 20, control the movable platform 20 to land.
  • the controller 21 determines the landing point of the movable platform 20 according to the semantic map, so that the position information can be accurately obtained according to the semantic map with high confidence and high accuracy of scene understanding, and then determine the movable The landing point of the platform 20, and the landing point is safe and reliable.
  • the controller 21 controls the power system 28 to work according to the landing point of the movable platform 20 to make the movable platform 20 land, so that the movable platform 20 can land safely, reliably and accurately
  • To the landing point determined by the semantic map the problem of damage or damage to the movable platform 20 when the movable platform 20 falls in water, trees, buildings, etc. in related technologies is avoided, and the service life of the movable platform 20 is greatly extended.
  • the safety of the use of the movable platform 20 is improved, and the reliability of the product is improved.
  • the controller 21 determines the landing point of the movable platform 20 according to the semantic map, specifically: determining the landing area of the movable platform 20 according to the semantic map; Status information, select the landing point in the landing area.
  • the controller 21 determines the landable area of the movable platform 20 according to the semantic map.
  • the landable area may be a safe and reliable area that allows the movable platform 20 to land according to the semantic map, that is, no Including dangerous or destructive areas such as water, trees, buildings, etc., which can make the movable platform 20 land, thereby avoiding damage or damage to the movable platform 20 during landing, and is beneficial to prolong the service life of the movable platform 20;
  • the controller 21 selects a landing point in the landable area according to the status information of the movable platform 20, which is beneficial to combine the status information of the movable platform 20, so that the selected landing point can ensure the safe and reliable landing of the movable platform 20 , Avoiding that the state of the movable platform 20 cannot meet the requirements of reaching the landing point smoothly or failing to complete the landing at the landing point, and further ensuring that the movable platform 20 can safely, smoothly, reliably and accurately land at the landing point, improving the mobility The reliability of the platform 20.
  • the controller 21 selects a landing point in the landable area according to the state information of the movable platform 20, specifically: obtaining the remaining power of the power supply battery 26; And semantic map, select the landing point in the landing area.
  • the steps for the controller 21 to select a landing point in the landing area according to the state information of the movable platform 20 are specifically defined.
  • the controller 21 obtains the remaining power of the power supply battery 26 of the movable platform 20, and selects a landing point in the landing area according to the remaining power and the semantic map, so that the selected landing point can ensure that the movable platform 20 uses the power supply battery 26
  • the remaining power is smoothly landed at the landing point, avoiding that the remaining power of the power supply battery 26 cannot make the movable platform 20 reach the landing point smoothly and damaging or damaging the movable equipment, so that the selected landing point has a higher accuracy, thereby ensuring
  • the movable platform 20 can complete the landing reliably and safely, and extend the service life of the movable platform 20.
  • the controller 21 is specifically used to obtain the flight trajectory of the movable platform 20, and select the landing point according to the flight trajectory and the remaining power.
  • the controller 21 obtains the flight trajectory of the movable platform 20, and selects the landing point according to the flight trajectory and the remaining power of the power supply battery 26, so that the selected landing point is adapted to the flight trajectory, which is beneficial to the mobility
  • the platform 20 realizes the return home according to the flight trajectory, improves the accuracy of the return home of the movable platform 20, and can ensure that the movable platform 20 uses the remaining power of the power supply battery 26 to land at the landing point smoothly, thereby improving the reliability and safety of the landing of the movable platform 20 .
  • the controller 21 is specifically used to determine the remaining cruising range of the power supply battery 26 according to the remaining power of the power supply battery 26, and to select the landing point according to the remaining cruising range and flight trajectory.
  • the controller 21 determines the remaining cruising range of the battery according to the remaining power of the power supply battery 26, and quantifies the remaining power of the power supply battery 26 as the remaining cruising range of the battery, so that the quantified remaining cruising range and flight trajectory , Accurate and reasonable selection of the landing point is conducive to improving the accuracy of the landing position information. It can not only ensure that the movable platform 20 is safely and reliably landed at the landing point, and the remaining cruising range determined based on the remaining power will be based on the maximum flight distance. The trajectory completes the return home, thereby improving the accuracy of the movable platform 20 returning home.
  • the controller 21 selects the landing point according to the remaining cruising range and flight trajectory, specifically: determining the estimated return mileage of the movable platform 20 according to the flight trajectory and the semantic map ; When the estimated return mileage is less than or equal to the remaining cruising mileage, the take-off point of the flight trajectory is taken as the landing point.
  • the controller 21 determines the estimated return mileage of the movable platform 20 based on the flight trajectory and the semantic map, that is, the estimated return mileage is the mileage of the movable platform 20 returning to the takeoff point of the flight trajectory, based on the prediction
  • the estimated return mileage is less than or equal to the remaining cruising mileage, it means that the mobile platform 20 can return to the take-off point of the flight trajectory by using the remaining battery power, and the controller 21 uses the take-off point of the flight trajectory as the landing point to further increase the landing point.
  • the accuracy of this enables the movable platform 20 to land at the take-off point safely, reliably and accurately, and improves the accuracy of the movable platform 20 returning home.
  • the take-off point may be the starting point of the flight trajectory, or a designated home point, or a point in the designated flight plan, such as other points set close to the home point.
  • the controller 21 is specifically further configured to: select a landing point in the landable area according to the remaining cruising mileage and the take-off point .
  • the mobile platform 20 cannot return to the take-off point of the flight trajectory by using the remaining battery power.
  • the controller 21 selects the landing area in the landing area. Point to ensure that the movable platform 20 can successfully complete the landing, and can achieve safe and reliable landing, avoiding the estimated return mileage to be greater than the remaining cruising range, and setting the landing point as the take-off point of the flight trajectory makes the movable platform 20 unable to complete successfully
  • the problem of damage or damage due to landing further improves the reliability of the movable platform 20 and prolongs the service life of the movable platform 20.
  • the controller 21 is specifically used to control the movable platform 20 to perform obstacle avoidance flight according to the semantic map; wherein, the obstacle avoidance flight includes a detour flight or a climb flight.
  • the controller 21 controls the movable platform 20 to perform obstacle avoidance flight according to the semantic map. Since the semantic map has a high degree of confidence, it can completely and accurately obtain the location information of obstacles in the real scene, and control The device 21 controls the movable platform 20 to perform obstacle flight to avoid obstacles, which is beneficial to improve the flight reliability of the movable platform 20, thereby extending the service life of the movable platform 20 and improving the reliability of the product.
  • obstacle avoidance flight includes detour flight or climb flight.
  • Detour flight means flying around obstacles, and climbing flight means flying upwards over obstacles. Understandably, it can also include other flight modes, such as detour flight and crawling flight at the same time. .
  • the obstacle avoidance flight can be used for obstacle avoidance flight during the return home process, or the mobile platform can perform obstacle avoidance flight based on the semantic map to further improve the reliability of flight.
  • the controller 21 is further configured to: according to the flying posture of the movable platform 20, control the collection device 22 on the ground side of the movable platform 20 to collect multiple images.
  • the landing point of the movable platform 20 is generally set on the ground, that is, the mobile platform 20 is finally the landing point landing on the ground.
  • the controller 21 controls the movable platform 20 according to the flight attitude of the movable platform 20.
  • the collection device 22 on the side of the mobile platform 20 facing the ground collects multiple images to obtain a semantic map on the side of the ground, which is conducive to the safe, reliable and accurate landing of the movable platform 20 on the landing point on the ground, which is highly operable. Easy to implement and suitable for popularization and application.
  • controller 21 can also enable the collection device 22 on the side close to the ideal landing point to collect multiple images according to the orientation of the ideal landing point, so that the movable platform 20 can be safely, reliably and accurately Landing at the ideal landing point further expands the use range of the product.
  • the collection device 22 includes a radar, a vision sensor or a multispectral sensor.
  • the collection device 22 can be a radar, a vision sensor, or a multispectral sensor.
  • the multiple types of the collection device 22 can meet the requirements of different installation positions of the collection device 22, collection of images from different perspectives, and collection of images with different background information. It can meet the different cost requirements of the movable platform 20, which is beneficial to expand the use range of the product. It can be understood that the collection device 22 may also be other devices that meet the requirements.
  • the controller 21 is further used to: receive a take-off instruction, control the power system 28 and the acquisition device 22 to start, to control the flight of the movable platform 20 and the acquisition device 22 to collect multiple images; and Receiving a return instruction or detecting a failure of the movable platform 20, the control acquisition device 22 is turned off.
  • the controller 21 receives a take-off instruction, controls the power system 28 to start, the movable platform takes off, and controls the collection device 22 to start to collect multiple images, that is, the collection device 22 starts when the movable platform 20 takes off. Collect multiple images, and the controller 21 constructs a semantic map in real time.
  • the controller 21 controls the collection device 22 to turn off by receiving a return instruction or detecting a failure of the movable platform 20, that is, when the movable platform 20 needs to return, it controls the collection device 22 Close, stop collecting images, and stop building semantic maps, and accurately obtain location information according to the semantic map construction, and then determine the landing point of the movable platform 20, so that the movable platform 20 can land on the landing point safely, reliably and accurately , To complete the return flight, avoid the problem of damage or damage to the movable platform 20 when the movable platform 20 is landed in the water, on a tree, or on a building in the related technology, greatly extend the service life of the movable platform 20, and improve the movable platform 20 Use safety and improve product reliability.
  • the return instruction may be a return instruction triggered by the return key selected by the user; on the other hand, the return instruction may be a return instruction sent by the controller 21 of the movable platform 20 when the movable platform 20 flies to the return point of the flight path. instruction.
  • Different ways of returning instructions can meet the requirements of different working conditions of the movable platform 20, thereby expanding the use range of the product, and at the same time, it is beneficial to flexibly control the movable platform 20 to return home safely, and further improve the reliability of the movable platform 20.
  • the movable platform 20 of the present application is an unmanned aerial vehicle, while the unmanned aerial vehicle in the related technology, the unmanned aerial vehicle scene usually cannot find a safe and reliable landing environment when returning home due to the complex environment and falls. Conditions such as water, trees, and buildings have caused damage to the drone.
  • the drone of this application includes a body 24, a power supply battery 26 arranged on the body 24, a power system 28, a collection device 22, and a controller 21.
  • the controller 21 receives the take-off instruction to control the power system 28 to start, the drone takes off, and controls the acquisition device 22 to start, that is, the acquisition device 22 (such as radar, vision sensor, and multispectral sensor) collects data in real time during takeoff. Multiple images, and stitch multiple images in real time.
  • the acquisition device 22 such as radar, vision sensor, and multispectral sensor
  • the multiple images can be images of scenes with different perspectives and different background information that are continuously collected.
  • the images can be collected when the drone is blocking the scene.
  • the controller 21 performs the multiple images through the preset convolutional neural network module.
  • Semantic segmentation is used to obtain semantic segmentation information of multiple images, wherein the semantic segmentation information corresponding to any image includes semantic recognition results of several pixels.
  • the semantic recognition process of a specific image can be: the preprocessed image data is sent to the network model as RGB (Red Green Blue, color mode) three-channel data, and the network is obtained after the forward propagation in turn, that is, after the iteration of the network model Output the result.
  • the specific process is shown in Figure 24.
  • the image data input format is N ⁇ 4 ⁇ H ⁇ W.
  • the network After the input data is processed by a convolutional neural network composed of multiple "Conv+bn+Relu" operation layers, the network is obtained The output result is an N ⁇ K ⁇ H ⁇ W tensor. After processing it, the recognition result and recognition confidence are obtained.
  • the semantic segmentation information By deleting the semantic recognition results with confidence lower than the preset threshold, the semantic recognition results only includes Semantic recognition results with high confidence, the semantic recognition results can include: buildings, sky, trees, water, ground, etc., that is, the semantic segmentation information can truly and completely reflect the entity content corresponding to the image.
  • the controller 21 performs the stitching of two-dimensional semantic maps according to the recognition results of the height information and semantic segmentation information of the entities corresponding to the multiple images by the drone.
  • multiple frames of images are used in combination with the drone.
  • the stitching is performed at the height of the entities corresponding to multiple images, and the overlapping parts of the multiple images are merged to obtain the stitched image.
  • the semantic map of the stitched image is obtained, so that the semantic map is better. It is based on the real scene, and fully and accurately reflects the information of multiple entity contents of the real scene, thereby making the obtained semantic map have a higher degree of confidence, improving the accuracy of scene understanding, and accurately obtaining the location through the semantic map information.
  • the drone controller 21 When the drone controller 21 receives a return instruction or detects that the drone is malfunctioning, it controls the collection device 22 to stop collecting images. According to the semantic map constructed at the current moment, the semantic map construction method realizes the analysis of the drone scene Accurate acquisition of semantic information and landing point information. Specifically, by obtaining the flight trajectory of the drone and the remaining power of the power supply battery 26 of the drone, the landing point is selected according to the flight trajectory and the remaining power, and accurate landing position information is obtained, thereby avoiding the landing of the drone in related technologies.
  • the problem of damaging or destroying UAVs in water, trees, buildings, etc. greatly prolongs the service life of UAVs, improves the safety of UAVs, and improves the reliability of products.
  • the selected landing point is adapted to the flight trajectory, which is conducive to the return of the drone according to the landing position information, improving the accuracy of the return of the drone, and ensuring that the drone can use the remaining power to land on the landing point. In turn, the reliability and safety of drone landing are improved.
  • the controller 21 controls the UAV to fly to avoid obstacles when returning home, which is beneficial to improve the unmanned The reliability of aircraft flight, thereby extending the service life of the UAV and improving the reliability of the product.
  • the embodiment of the sixth aspect of the present application provides a method for searching for a landing point, which is applicable to a mobile platform, and includes the steps: obtaining a semantic map according to the semantic map construction method provided in any of the above embodiments; , Determine the landing point of the movable platform; According to the landing point of the movable platform, control the movable platform to land.
  • the mobile platform provided by this application includes: an airframe, a power supply battery arranged on the airframe, a power system, an acquisition device, and a controller, wherein the power supply battery is used to power the power system, and the power system is used to provide flight power for the movable platform
  • the collection device is used to obtain multiple images during the flight of the movable platform, and obtain the semantic segmentation information of multiple images through the controller, and the multiple images can be images of scenes with different perspectives and different background information, which is beneficial to control
  • the device obtains complete and accurate information of multiple entity contents of the real scene, and generates a spliced image by splicing multiple images through the controller, which helps to ensure the integrity and authenticity of the scene.
  • the semantic map of the spliced image is obtained, so that the semantic map better tends to the real scene, and fully and accurately reflects the multiple entity content of the real scene, so that the acquired semantic map has a better
  • the high confidence level improves the accuracy of scene understanding and enables the controller to accurately obtain location information through the semantic map.
  • the position information can be accurately obtained according to the semantic map with high confidence and high accuracy of scene understanding, and then the landing point of the movable platform is determined, and the landing The point is safe and reliable, and the movable platform is controlled to land according to the landing point of the movable platform, so that the movable platform can safely, reliably and accurately land to the landing point determined by the semantic map, avoiding the movable platform in the related technology from landing on
  • the problem of damage or damage to the movable platform in the water, on trees, buildings, etc. greatly extends the service life of the movable platform, improves the safety of the use of the movable platform, and improves the reliability of the product.
  • determining the landing point of the movable platform according to the semantic map specifically includes: determining the landing area of the movable platform according to the semantic map; selecting the landing area according to the state information of the movable platform Landing point.
  • the landable area can be a safe and reliable area that allows the movable platform to land according to the semantic map, that is, does not include There are dangerous or destructive areas in the landing of the platform, such as water, trees, buildings and other areas, thereby avoiding damage or damage to the movable platform during landing, which is conducive to extending the service life of the movable platform; according to the status information of the movable platform , Selecting a landing point in the landing area is conducive to combining the status information of the movable platform, so that the selected landing point can ensure the safe and reliable landing of the movable platform, and avoid the failure of the movable platform to reach smoothly due to its own state
  • the landing point may not be able to successfully land at the landing point, which further ensures that the movable platform can safely, smoothly, reliably and accurately land at the landing point, and improves the reliability of the movable platform.
  • the step of selecting a landing point in the landable area according to the status information of the movable platform is specifically: obtaining the remaining power of the battery of the movable platform; according to the remaining power and the semantic map, Select the landing point in the landing area.
  • the step of selecting a landing point in the landable area according to the state information of the movable platform is specifically defined.
  • the landing point is selected in the landing area, so that the selected landing point can ensure that the mobile platform uses the remaining power to land on the landing point smoothly, avoiding The remaining power of the battery cannot make the movable platform reach the landing point smoothly and damage or destroy the movable equipment, so that the selected landing point has high accuracy, thereby ensuring that the movable platform can reliably and safely complete the landing.
  • the method of searching for a landing point further includes: obtaining the flight trajectory of the movable platform, and selecting the landing point according to the flight trajectory and the remaining power.
  • the landing point is selected according to the flight trajectory and the remaining power, so that the selected landing point is adapted to the flight trajectory, which is beneficial to
  • the movable platform realizes the return home according to the flight trajectory, improves the accuracy of the return home of the movable platform, and can ensure that the movable platform uses the remaining power to land at the landing point smoothly, thereby improving the reliability and safety of the landing of the movable platform.
  • the method for searching for a landing point further includes: determining the remaining cruising range of the battery according to the remaining power of the battery, and selecting the landing point according to the remaining cruising range and flight trajectory.
  • the remaining power of the battery of the movable platform and the flight trajectory of the movable platform are obtained according to the semantic map, and the remaining range of the battery is determined according to the remaining power of the battery, and the remaining power of the battery is specifically quantified as a battery According to the remaining cruising range and flight trajectory, the landing point is selected, so that the landing point can be accurately and reasonably selected according to the quantified remaining cruising range and flight trajectory, which is conducive to improving the accuracy of the landing position information. It can ensure the safe and reliable landing of the movable platform at the landing point, and the remaining cruising range determined based on the remaining power to complete the return according to the flight trajectory to the maximum, thereby improving the accuracy of the return of the movable platform.
  • the steps of selecting a landing point based on the remaining cruising range and flight trajectory are specifically: determining the estimated return mileage of the movable platform based on the flight trajectory and semantic map; based on the estimated return mileage being less than In the case of or equal to the remaining cruising range, the takeoff point of the flight trajectory is used as the landing point; based on the estimated return mileage greater than the remaining cruising range, based on the remaining cruising range and take-off point, the landing point is selected in the landable area.
  • the step of selecting the landing point according to the remaining cruising range and flight trajectory is specifically defined.
  • the mileage returned to the departure point of the flight trajectory is based on two situations where the estimated return mileage is less than or equal to the remaining cruising mileage and the estimated return mileage is greater than the remaining cruising mileage.
  • the mobile platform can return to the take-off point of the flight trajectory by using the remaining battery power, and then use the take-off point of the flight trajectory as the landing point to further improve the accuracy of the landing point, so that the mobile platform can be safely, reliably and accurately Landing at the take-off point improves the accuracy of the return of the movable platform.
  • the mobile platform cannot return to the take-off point of the flight trajectory by using the remaining battery power.
  • the mobile platform is guaranteed Able to successfully complete the landing, and achieve a safe and reliable landing, avoiding the estimated return mileage to be greater than the remaining cruising mileage, and setting the landing point as the take-off point of the flight trajectory makes the movable platform unable to complete the landing smoothly and there is a problem of damage or damage , Further improve the reliability of the movable platform and extend the service life of the movable platform.
  • the take-off point may be the starting point of the flight trajectory, or a designated home point, or a point in the designated flight plan, such as other points set close to the home point.
  • the specific process for the movable platform 20 to obtain the position information of the landing point is shown in FIG. 25.
  • the controller 21 of the movable platform 20 controls the acquisition device 22 to input multiple images acquired in real time to the convolutional neural network ( CNN) module and perform semantic segmentation on multiple images to obtain semantic segmentation information.
  • the output semantic segmentation information includes the semantic recognition results and semantic recognition confidence of several pixels of any image in the multiple images. Further, the semantic recognition results whose confidence is lower than the preset threshold are deleted according to the semantic confidence, so that only semantic recognition results with higher confidence are included in the semantic segmentation information.
  • Judgment and intelligent search (such as the remaining battery power of the movable platform 20, flight trajectory, etc.) can obtain accurate location information of the landing point, which is beneficial to the safe and reliable landing of the movable platform 20 and extends the movable platform 20 Life.
  • the term “plurality” refers to two or more than two, unless specifically defined otherwise.
  • the terms “installed”, “connected”, “connected”, “fixed” and other terms should be understood in a broad sense.
  • “connected” can be a fixed connection, a detachable connection, or an integral connection;
  • “connected” can be It is directly connected or indirectly connected through an intermediary.
  • the specific meaning of the above-mentioned terms in this application can be understood according to specific circumstances.

Abstract

一种语义地图的构建方法、一种语义地图的构建系统、一种可移动平台、一种计算机可读存储介质和一种可移动平台、一种搜索降落点的方法。其中，语义地图的构建方法，包括：获取多个图像的语义分割信息（S102）；对多个图像进行拼接操作生成拼接图像，根据多个图像的语义分割信息，获取拼接图像的语义地图（S104）。所述语义地图的构建方法，通过获取多个图像的语义分割信息得到完整的、精确的真实场景的多个实体内容的信息，通过对多个图像进行拼接操作生成拼接图像，有利于保证场景的完整度和真实性，使得根据多个图像的语义分割信息获取的语义地图具有较高的置信度，提高了场景理解的精确度，通过语义地图能够精准的获取位置信息。

Description

语义地图的构建方法、系统、可移动平台和存储介质 技术领域
本申请涉及智能识别技术领域，具体而言，涉及一种语义地图的构建方法、一种语义地图的构建系统、一种可移动平台、一种计算机可读存储介质和一种可移动平台、一种搜索降落点的方法。
背景技术
目前的飞机、无人机等设备的场景由于背景信息复杂、视角多变，很难做到精确的场景理解，进而无法更有效地指导飞行器等设备进行飞行。因此，亟需构建一个语义地图来指导飞行器等设备进行后续运动。
申请内容
本申请实施例提供了语义地图的构建方法、系统、可移动平台和存储介质，能够构建更为完整的语义地图。
为此，本申请的第一个方面在于，提出一种语义地图的构建方法。
本申请的第二个方面在于，提出一种语义地图的构建系统。
本申请的第三个方面在于，提出一种可移动平台。
本申请的第四个方面在于，提出一种计算机可读存储介质。
本申请的第五个方面在于，提出一种可移动平台。
本申请的第六个方面在于，提出一种搜索降落点的方法。
有鉴于此，根据本申请的第一方面，提供了一种语义地图的构建方法，方法包括：获取多个图像的语义分割信息；对多个图像进行拼接操作生成拼接图像，根据多个图像的语义分割信息，获取拼接图像的语义地图。
本申请提供的语义地图的构建方法，通过获取多个图像的语义分割信息，而多个图像可以为不同视角、不同背景信息的场景的图像，有利于通过多个图像的语义分割信息得到完整的、精确的真实场景的多个实体内容 的信息，通过对多个图像进行拼接操作生成拼接图像，有利于保证场景的完整度和真实性，根据多个图像的语义分割信息，获取拼接图像的语义地图，使得语义地图较好的趋于真实场景，并完整、精确地体现了真实场景的多个实体内容，进而使得获取的语义地图具有较高的置信度，提高了场景理解的精确度，通过语义地图能够精准的获取位置信息。
本申请的第二方面，提出了一种语义地图的构建系统，其中，包括：存储器，用于存储计算机程序；处理器，用于执行计算机程序以实现：获取多个图像的语义分割信息；对多个图像进行拼接操作生成拼接图像，根据多个图像的语义分割信息，获取拼接图像的语义地图。
本申请提供的语义地图的构建系统，通过令处理器获取多个图像的语义分割信息，而多个图像可以为不同视角、不同背景信息的场景的图像，有利于通过多个图像的语义分割信息得到完整的、精确的真实场景的多个实体内容的信息，通过处理器对多个图像进行拼接操作生成拼接图像，有利于保证场景的完整度和真实性，根据多个图像的语义分割信息，获取拼接图像的语义地图，使得语义地图较好的趋于真实场景，并完整、精确地体现了真实场景的多个实体内容，进而使得获取的语义地图具有较高的置信度，提高了场景理解的精确度，通过语义地图能够精准的获取位置信息。
本申请的第三方面，提出了一种可移动平台，包括上述任一技术方案的语义地图的构建系统；以及采集装置，采集装置与构建系统相连接，采集装置用于采集图像并将图像发送至处理器。由于可移动平台包括上述任一技术方案的语义地图的构建系统，因此具有上述任一技术方案的语义地图的构建系统的全部有益效果，在此不再赘述。
本申请的第四方面，提出了一种计算机可读存储介质，其上存储有计算机程序，计算机程序被处理器执行时实现上述任一技术方案的语义地图的构建方法。因此具有上述任一技术方案的语义地图的构建方法的有益效果，在此不再赘述。
本申请的第五方面，提出了一种可移动平台，包括机体、设于机体上的供电电池、动力系统、采集装置和控制器，供电电池，用于为动力系统供电，动力系统，用于可移动平台提供飞行动力；采集装置，用于在可移 动平台飞行过程中，获取多个图像；控制器，用于获取多个图像的语义分割信息；对多个图像进行拼接操作生成拼接图像，根据多个图像的语义分割信息，获取拼接图像的语义地图。
本申请的第六方面，提供了一种搜索降落点的方法，适用于一可移动平台，包括步骤：
根据以上任一项所述的语义地图的构建方法获取语义地图；
根据语义地图，确定可移动平台的降落点；
根据可移动平台的降落点，控制可移动平台进行降落。
本申请提供的可移动平台包括：机体、设于机体上的供电电池、动力系统、采集装置和控制器，其中，供电电池用于为动力系统供电，动力系统用于为可移动平台提供飞行动力；采集装置用于在可移动平台飞行过程中，获取多个图像，通过控制器获取多个图像的语义分割信息，而多个图像可以为不同视角、不同背景信息的场景的图像，有利于控制器通过多个图像的语义分割信息得到完整的、精确的真实场景的多个实体内容的信息，通过控制器对多个图像进行拼接操作生成拼接图像，有利于保证场景的完整度和真实性，根据多个图像的语义分割信息，获取拼接图像的语义地图，使得语义地图较好的趋于真实场景，并完整、精确地体现了真实场景的多个实体内容，进而使得获取的语义地图具有较高的置信度，提高了场景理解的精确度，使得控制器通过语义地图能够精准的获取位置信息。
本申请的附加方面和优点将在下面的描述部分中变得明显，或通过本申请的实践了解到。
附图说明
本申请的上述和/或附加的方面和优点从结合下面附图对实施例的描述中将变得明显和容易理解，其中：
图1示出了本申请的一个实施例的语义地图的构建方法的示意流程图；
图2示出了本申请的一个实施例的获取的一个图像；
图3示出了本申请的一个实施例的语义识别图；
图4示出了本申请的一个实施例的遮挡的渲染图；
图5示出了本申请的另一个实施例的语义地图的构建方法的示意流程图；
图6示出了本申请的再一个实施例的语义地图的构建方法的示意流程图；
图7示出了本申请的又一个实施例的语义地图的构建方法的示意流程图；
图8示出了本申请的又一个实施例的语义地图的构建方法的示意流程图；
图9示出了本申请的另一个实施例的获取的一个图像；
图10示出了本申请的另一个实施例的语义识别图；
图11示出了本申请的另一个实施例的遮挡的渲染图；
图12示出了本申请的一个实施例的降落点的示意图；
图13示出了本申请的另一个实施例的降落点的示意图；
图14示出了本申请的又一个实施例的语义地图的构建方法的示意流程图；
图15示出了本申请的又一个实施例的语义地图的构建方法的示意流程图；
图16示出了本申请的又一个实施例的语义地图的构建方法的示意流程图；
图17示出了本申请的又一个实施例的语义地图的构建方法的示意流程图；
图18示出了本申请的又一个实施例的语义地图的构建方法的示意流程图；
图19示出了本申请的又一个实施例的语义地图的构建方法的示意流程图；
图20示出了本申请的又一个实施例的语义地图的构建方法的示意流程图；
图21示出了本申请的一个实施例的语义地图的构建系统的示意框图；
图22示出了本申请的一个实施例的可移动平台的结构示意图；
图23示出了本申请的又一个实施例的可移动平台的结构示意图；
图24示出了本申请的一个实施例的图像的语义识别过程示意图；
图25示出了本申请的一个实施例的获取降落点位置信息的过程示意图。
其中，图21至图23中附图标记与部件名称之间的对应关系为：
12存储器，14处理器，22采集装置，24机体，26供电电池，28动力系统，21控制器，20可移动平台，10语义地图的构建系统。
具体实施方式
为了能够更清楚地理解本申请的上述目的、特征和优点，下面结合附图和具体实施方式对本申请进行进一步的详细描述。需要说明的是，在不冲突的情况下，本申请的实施例及实施例中的特征可以相互组合。
在下面的描述中阐述了很多具体细节以便于充分理解本申请，但是，本申请还可以采用其他不同于在此描述的其他方式来实施，因此，本申请的保护范围并不受下面公开的具体实施例的限制。
下面参照图1至图25描述根据本申请一些实施例的语义地图的构建方法、语义地图的构建系统、可移动平台、计算机可读存储介质和可移动平台、搜索降落点的方法。
根据本申请的第一方面的实施例，提供了一种语义地图的构建方法，图1示出了根据本申请一个实施例的语义地图的构建方法的示意流程图。如图1所示，该语义地图的构建方法包括：
S102，获取多个图像的语义分割信息；
例如，以视觉传感器为例，如图2所示，为任意时刻获取得到的任一图像，对图2进行语义分割后，生成如图3所示的单帧图像语义识别图，该图像语义识别图包括天空、地面和树。将语义识别图遮挡在原始图像上，即可获得如图4所示的遮挡的渲染图。
具体实施例中，获取遮挡的渲染图的过程如图2至图4所示，其中，图2所示的图像为任意时刻获取得到的图像，对图2所示的图像进行语义 分割后生成如图3所示的单帧图像语义识别图，将语义识别图遮挡在原始图像上，即可获得如图4所示的遮挡的渲染图，其中，图3中不同的背景色指代不同的语义识别结果，例如，具体实施例中可以通过彩色进行示例，设置天蓝色代表天空，蓝紫色代表地面，宝石蓝代表树，湖蓝代表水，黄色代表建筑物，白色代表其他实体。可以理解的是，也可以用其他不同的颜色来表示不同的语义识别结果。
S104，对多个图像进行拼接操作生成拼接图像，根据多个图像的语义分割信息，获取拼接图像的语义地图。
优选地，将单帧图像的语义识别图覆盖在拼接图像上，即可获得拼接图像的语义地图。
本申请提供的语义地图的构建方法，通过获取多个图像的语义分割信息，而多个图像可以为不同视角、不同背景信息的场景的图像，有利于通过多个图像的语义分割信息得到完整的、精确的真实场景的多个实体内容的信息，通过对多个图像进行拼接操作生成拼接图像，有利于保证场景的完整度和真实性，根据多个图像的语义分割信息，获取拼接图像的语义地图，使得语义地图较好的趋于真实场景，并完整、精确地体现了真实场景的多个实体内容，进而使得获取的语义地图具有较高的置信度，提高了场景理解的精确度，通过语义地图能够精准的获取位置信息。
具体实施例中，通过图像采集装置采集多个图像，多个图像的采集时间为连续的，图像采集装置包括并不限于：视觉传感器、雷达、多光谱传感器。
在本申请的一个实施例中，优选地，获取多个图像的语义分割信息具体为：通过预设的卷积神经网络模块对多个图像进行语义分割以得到多个图像的语义分割信息。
在该实施例中，通过预设的卷积神经网络模块对多个图像进行语义分割以得到多个图像的语义分割信息，能够完整、精确地获得图像中的实体内容的信息，进而有利于根据多个图像的语义分割信息获取的语义地图完整、精确地体现了真实场景的多个实体内容，具有较高的置信度，提高场景理解的精确度。
可以理解的是，也可以通过其他方法获得多个图像的语义分割信息。具体地，卷积神经网络(CNN，Convolutional Neural Networks)适用于各种场景任务，特别是场景目标语义信息、位置信息的获取，因此CNN可以被用来识别航拍场景的各种目标的语义信息和位置信息。
具体实施例中，通过图像采集装置进行连续采集多个图像，将多个图像输入预设的卷积神经网络模块进行语义分割以得到语义分隔信息，根据语义分隔信息和多个图像进行拼接生成的拼接图像构建出语义地图。通过连续采集图像，可以获得更完整和详细的语义地图；进一步通过卷积神经网络获取多个图像的语义分割信息，可以更准确的获取图像中的实体场景，进而得到更精确的语义地图。
图5示出了根据本申请另一个实施例的语义地图的构建方法的示意流程图。如图5所示，该语义地图的构建方法包括：
S202，获取多个图像的语义分割信息，其中，多个图像中的任一图像对应的语义分割信息包括若干个像素点的语义识别结果；
S204，获取每一个像素点的语义识别结果的置信度，将置信度低于预设阈值的语义识别结果删除；
S206，对多个图像进行拼接操作生成拼接图像，根据多个图像的语义分割信息，获取拼接图像的语义地图。
在该实施例中，由于多个图像中的任一图像对应的语义分割信息包括若干个像素点的语义识别结果，在对多个图像进行拼接生成拼接图像的步骤之前，通过获取每一个像素点的语义识别结果的置信度，将置信度低于预设阈值的语义识别结果删除，使得语义分割信息中仅包括置信度较高的语义识别结果，即语义分割信息能够真实、完整地体现图像对应的实体内容的信息，进而使得根据多个图像的语义分割信息获取的语义地图具有较高的置信度，能够完整、精确地体现真实场景的实体内容，提高场景理解的精确度，使得通过语义地图能够精准的获取位置信息。
进一步地，语义识别结果可以为与图像对应的真实场景中的实体内容，可以理解的是，语义识别结果可以为多个，对应于真实场景中的多个实体内容。具体实施例中，语义识别结果可以对应于实体场景中的天空、地面、 树及建筑物等，通过根据不同实体对应不同的像素点对图像中的实体进行语义识别。
可以理解的是，语义分割信息包括若干个像素点的语义识别结果和语义识别结果对应的置信度，通过置信度与预设阈值的比较，将置信度低于预设阈值的语义识别结果删除，使得语义分割信息中仅包括置信度较高的语义识别结果，即语义分割信息能够真实、完整地体现图像对应的实体内容的信息，进而使得通过语义地图能够精准的获取位置信息。
在本申请的实施例中，语义地图的构建方法适用于可移动平台，图6示出了根据本申请再一个实施例的语义地图的构建方法的示意流程图。如图6所示，该语义地图的构建方法包括：
S302，获取多个图像的语义分割信息，其中，多个图像中的任一图像对应的语义分割信息包括若干个像素点的语义识别结果；
S304，获取每一个像素点的语义识别结果的置信度，将置信度低于预设阈值的语义识别结果删除；
S306，根据可移动平台相对于多个图像所对应的实体的高度信息，对多个图像进行拼接生成拼接图像；
S308，根据多个图像的语义分割信息，获取拼接图像的语义地图。
在该实施例中，语义地图的构建方法适用于可移动平台，可移动平台可以为飞机、无人机，也可以为满足要求的其他可移动平台。通过根据可移动平台相对于多个图像所对应的实体的高度信息，对多个图像进行拼接生成拼接图像，有利于保证生成的拼接图像具有较高的清晰度和还原度，各实体的高度真实体现在拼接图像中，进而有利于获取的拼接图像的语义地图具有较高的置信度，并提高了场景理解的精确度。
优选地，实体的高度信息由可移动平台上的单目摄像头、双目摄像头或激光器等测量得到。
进一步地，通过根据可移动平台相对于多个图像所对应的实体的高度信息和语义分割信息的识别结果进行二维语义地图的拼接，使得通过拼接图像的语义地图能够精确地获取场景中的目标语义信息和距离信息，使得通过语义地图获取的位置信息更加准确，进而有利于可移动平台根据目标 语义信息和距离信息准确移动，提高了可移动平台移动的安全性和准确性，有利于提高产品的可靠性。其中，目标语义信息可以为与图像相对应的多个实体内容中的目标实体的信息，距离信息可以为可移动平台与目标实体的距离。具体实施例中，可移动平台为无人机，目标语义信息为图像中的地面所对应的语义信息，距离信息为无人机与地面之间的距离，通过根据地面的语义信息及无人机与地面之间的距离，能够精准地获取地面的位置信息，进而使无人机能够安全、准确地降落在地面上。
具体地，可以对任一图像进行单帧识别来获取每一个像素点的语义识别结果，进行连续采集多个图像并结合可移动平台相对于多个图像所对应的实体的高度信息，进行图像拼接，实现多帧构建实时的语义地图。可以理解的是，也可以通过其他方式获取每一个像素点的语义识别结果。具体地，当可移动平台为无人机时，高度为图像对应的实体结构与无人机的距离。具体地，图像拼接时，多个图像重叠的部分可以进行融合，例如，对多个图像重叠部分的每一个像素点的识别结果的置信度进行比较，通过保留置信度较高的图像、删除置信度较低的图像对重叠部分的图像进行融合，即最大限度的提取每个图像中的有利信息，能够使融合后的拼接图像保证了场景的完整度和真实性，进而使语义地图具有较高的置信度。在本申请的一个实施例中，优选地，语义识别结果包括以下至少一种：建筑物、天空、树、水面、地面。
在该实施例中，语义识别结果包括建筑物、天空、树、水面、地面中的一种或多种，语义识别结果的多种类型包括了与图片对应的真实场景中的多种实体内容，进而使得语义结果能够真实、完整地体现图像对应的实体内容，有利于提高场景理解的精确度。
进一步地，语义识别结果也可以包括满足要求的其他内容。
图7示出了根据本申请又一个实施例的语义地图的构建方法的示意流程图。如图7所示，该语义地图的构建方法包括：
S402，按照预设频率采集多个图像；
S404，获取多个图像的语义分割信息，其中，多个图像中的任一图像对应的语义分割信息包括若干个像素点的语义识别结果；
S406，获取每一个像素点的语义识别结果的置信度，将置信度低于预设阈值的语义识别结果删除；
S408，根据可移动平台相对于多个图像所对应的实体的高度信息，对多个图像进行拼接生成拼接图像；
S410，根据多个图像的语义分割信息，获取拼接图像的语义地图。
在该实施例中，通过按照预设频率采集多个图像，可以得到不同视角、不同背景信息的场景的图像，使得通过多个图像能够完整、精确地反映真实场景中不同视角、不同位置、不同背景信息的实体内容，有利于通过多个图像的语义分割信息得到完整的、精确的真实场景中的多个实体内容，进而保证语义地图的可靠性和准确性。
图8示出了根据本申请又一个实施例的语义地图的构建方法的示意流程图。如图8所示，该语义地图的构建方法包括：
S502，按照预设频率采集多个图像；
例如，以视觉传感器为例，如图9所示，为任意时刻获取得到的任一图像，对图9进行语义分割后，生成如图10所示的单帧图像语义识别图，该图像语义识别图包括天空、地面、树、水、建筑物。将语义识别图遮挡在原始图像上，即可获得如图11所示的遮挡的渲染图。
具体实施例中，获取遮挡的渲染图的过程如图9至图11所示，图9所示的图像为任意时刻获取得到的图像，进行语义分割后生成如图10所示的单帧图像语义识别图，将语义识别图遮挡在原始图像上，即可获得如图11所示的遮挡的渲染图，其中，图10所示的不同的背景色指代不同的语义识别结果，例如，具体实施例中可以通过彩色进行示例，如设置天蓝色代表天空，蓝紫色代表地面，宝石蓝代表树，湖蓝代表水，黄色代表建筑物，白色代表其他实体。可以理解的是，也可以用其他不同的颜色来表示不同的语义识别结果。
S504，获取多个图像的语义分割信息，其中，多个图像中的任一图像对应的语义分割信息包括若干个像素点的语义识别结果；
S506，获取每一个像素点的语义识别结果的置信度，将置信度低于预设阈值的语义识别结果删除；
S508，根据可移动平台相对于多个图像所对应的实体的高度信息，对多个图像进行拼接生成拼接图像；
S510，根据多个图像的语义分割信息，获取拼接图像的语义地图；
S512，根据语义地图，确定可移动平台的降落点。
优选地，将单帧图像的语义识别图覆盖在拼接图像上，即可获得如图12所示的单帧拼接图像的语义地图，图12中的A代表单帧拼接图像中显示的可降落区域，可以理解的是，图12中的A也可以为具体的一点，该点代表降落点。
优选地，将多帧图像的语义识别图覆盖在拼接图像上，即可获得如图13所示的多帧拼接图像的语义地图，图13中的B代表多帧拼接图像中显示的可降落区域，可以理解的是，图13中的B也可以为具体的一点，该点代表降落点。通过图12和图13的降落区域进行对比可见，图13所示实施例通过多个图像能够完整、精确地反映真实场景中的实体内容，进而构建出较为完整、精确及详细的语义地图，进而通过利用较为完整、详细的语义地图对可移动平台进行指导飞行，以提升可移动平台飞行的可控性。
S514，根据可移动平台的降落点，控制可移动平台进行降落。
在该实施例中，通过根据语义地图，确定可移动平台的降落点，使得根据置信度较高、场景理解精确度较高的语义地图能够精准地获取位置信息，进而确定可移动平台的降落点，且降落点安全可靠，并根据可移动平台的降落点控制可移动平台进行降落，使得可移动平台能够安全、可靠、精准地降落至通过语义地图确定的降落点，避免了相关技术中可移动平台降落在水中、树上、建筑物上等损坏或损毁可移动平台的问题，大大延长了可移动平台的使用寿命，提高可移动平台使用的安全性，并提高产品的可靠性。
图14示出了根据本申请又一个实施例的语义地图的构建方法的示意流程图。如图14所示，该语义地图的构建方法包括：
S602，按照预设频率采集多个图像；
S604，获取多个图像的语义分割信息，其中，多个图像中的任一图像对应的语义分割信息包括若干个像素点的语义识别结果；
S606，获取每一个像素点的语义识别结果的置信度，将置信度低于预 设阈值的语义识别结果删除；
S608，根据可移动平台相对于多个图像所对应的实体的高度信息，对多个图像进行拼接生成拼接图像；
S610，根据多个图像的语义分割信息，获取拼接图像的语义地图；
S612，根据语义地图，确定可移动平台的可降落区域；
S614，根据可移动平台的状态信息，在可降落区域中选定降落点；
S616，根据可移动平台的降落点，控制可移动平台进行降落。
在该实施例中，通过根据语义地图，确定可移动平台的可降落区域，可降落区域可以为根据语义地图得到的安全、可靠的允许可移动平台降落的区域，即，不包括能够使可移动平台降落存在危险或破坏性的区域，如水、树、建筑物等区域，进而避免了可移动平台在降落时损坏或损毁，有利于延长可移动平台的使用寿命；通过根据可移动平台的状态信息，在可降落区域中选定降落点，有利于结合可移动平台的状态信息，使选择的降落点能够保证可移动平台安全、可靠地降落，避免由于可移动平台的自身的状态无法满足顺利抵达降落点或无法在降落点顺利完成降落，进一步保证了可移动平台能够安全、顺利、可靠、精准地降落在降落点，提高可移动平台的可靠性。
图15示出了根据本申请又一个实施例的语义地图的构建方法的示意流程图。如图15所示，该语义地图的构建方法包括：
S702，按照预设频率采集多个图像；
S704，获取多个图像的语义分割信息，其中，多个图像中的任一图像对应的语义分割信息包括若干个像素点的语义识别结果；
S706，获取每一个像素点的语义识别结果的置信度，将置信度低于预设阈值的语义识别结果删除；
S708，根据可移动平台相对于多个图像所对应的实体的高度信息，对多个图像进行拼接生成拼接图像；
S710，根据多个图像的语义分割信息，获取拼接图像的语义地图；
S712，根据语义地图，确定可移动平台的可降落区域；
S714，获取可移动平台的电池的剩余电量；
S716，根据剩余电量和语义地图，在可降落区域中选定降落点；
S718，根据可移动平台的降落点，控制可移动平台进行降落。
在该实施例中，具体限定了根据可移动平台的状态信息，在可降落区域中选定降落点的步骤。通过获取可移动平台的电池的剩余电量，根据剩余电量和语义地图，在可降落区域中选定降落点，使得选定的降落点能够保证可移动平台利用剩余电量顺利降落在降落点，避免了电池的剩余电量无法使可移动平台顺利抵达降落点而损坏或损毁可移动设备，使得选定的降落点具有较高的准确性，进而保证了可移动平台能够可靠、安全地完成降落，延长可移动平台的使用寿命。
图16示出了根据本申请又一个实施例的语义地图的构建方法的示意流程图。如图16所示，该语义地图的构建方法包括：
S802，按照预设频率采集多个图像；
S804，获取多个图像的语义分割信息，其中，多个图像中的任一图像对应的语义分割信息包括若干个像素点的语义识别结果；
S806，获取每一个像素点的语义识别结果的置信度，将置信度低于预设阈值的语义识别结果删除；
S808，根据可移动平台相对于多个图像所对应的实体的高度信息，对多个图像进行拼接生成拼接图像；
S810，根据多个图像的语义分割信息，获取拼接图像的语义地图；
S812，根据语义地图，确定可移动平台的可降落区域；
S814，获取可移动平台的电池的剩余电量；
S816，获取可移动平台的飞行轨迹，根据飞行轨迹和剩余电量，选定降落点；
S818，根据可移动平台的降落点，控制可移动平台进行降落。
在该实施例中，通过获取可移动平台的飞行轨迹和可移动平台的电池的剩余电量，根据飞行轨迹和剩余电量选定降落点，使得选定的降落点与飞行轨迹相适配，有利于可移动平台依照飞行轨迹实现返航，提高可移动平台返航的准确性，同时能够保证可移动平台利用剩余电量顺利降落在降落点，进而提高可移动平台降落的可靠性和安全性。
图17示出了根据本申请又一个实施例的语义地图的构建方法的示意流程图。如图17所示，该语义地图的构建方法包括：
S902，按照预设频率采集多个图像；
S904，获取多个图像的语义分割信息，其中，多个图像中的任一图像对应的语义分割信息包括若干个像素点的语义识别结果；
S906，获取每一个像素点的语义识别结果的置信度，将置信度低于预设阈值的语义识别结果删除；
S908，根据可移动平台相对于多个图像所对应的实体的高度信息，对多个图像进行拼接生成拼接图像；
S910，根据多个图像的语义分割信息，获取拼接图像的语义地图；
S912，根据语义地图，确定可移动平台的可降落区域；
S914，获取可移动平台的电池的剩余电量；
S916，根据电池的剩余电量确定电池的剩余续航里程；
S918，获取可移动平台的飞行轨迹；
S920，根据剩余续航里程和飞行轨迹，选定降落点；
S922，根据可移动平台的降落点，控制可移动平台进行降落。
在该实施例中，根据语义地图分别获取可移动平台的电池的剩余电量和获取可移动平台的飞行轨迹，通过根据电池的剩余电量确定电池的剩余续航里程，将电池的剩余电量具体量化为电池的剩余续航里程，并根据剩余续航里程和飞行轨迹，选定降落点，使得根据量化的剩余续航里程和飞行轨迹，准确、合理地选定降落点，有利于提高降落位置信息的精确性，既能够保证可移动平台安全、可靠地降落在降落点，并基于剩余电量确定的剩余续航里程最大限度的依照飞行轨迹完成返航，进而提高可移动平台返航的准确性。
图18示出了根据本申请又一个实施例的语义地图的构建方法的示意流程图。如图18所示，该语义地图的构建方法包括：
S1002，按照预设频率采集多个图像；
S1004，获取多个图像的语义分割信息，其中，多个图像中的任一图像对应的语义分割信息包括若干个像素点的语义识别结果；
S1006，获取每一个像素点的语义识别结果的置信度，将置信度低于预设阈值的语义识别结果删除；
S1008，根据可移动平台相对于多个图像所对应的实体的高度信息，对多个图像进行拼接生成拼接图像；
S1010，根据多个图像的语义分割信息，获取拼接图像的语义地图；
S1012，根据语义地图，确定可移动平台的可降落区域；
S1014，获取可移动平台的电池的剩余电量；
S1016，根据电池的剩余电量确定电池的剩余续航里程；
S1018，获取可移动平台的飞行轨迹；
S1020，根据飞行轨迹和语义地图，确定可移动平台的预估返航里程；
S1022，基于预估返航里程小于或等于剩余续航里程的情况下，将飞行轨迹的起飞点作为降落点；
基于预估返航里程大于剩余续航里程的情况下，根据剩余续航里程和起飞点，在可降落区域中选定降落点；
S1024，根据可移动平台的降落点，控制可移动平台进行降落。
在该实施例中，具体限定了根据剩余续航里程和飞行轨迹，选定降落点的步骤。根据语义地图分别获取可移动平台的电池的剩余电量和获取可移动平台的飞行轨迹，并通过根据飞行轨迹和语义地图，确定可移动平台的预估返航里程，即预估返航里程为可移动平台返回至飞行轨迹的起飞点的里程，基于预估返航里程小于或等于剩余续航里程和预估返航里程大于剩余续航里程两种情况，一方面，基于预估返航里程小于或等于剩余续航里程的情况下，说明利用电池的剩余电量可移动平台能够返回至飞行轨迹的起飞点，进而将飞行轨迹的起飞点作为降落点，进一步提高降落点的精确性，使得可移动平台能够安全、可靠、精准地降落在起飞点，提高了可移动平台返航的精准度。
另一方面，基于预估返航里程大于剩余续航里程的情况下，说明利用电池的剩余电量可移动平台无法返回至飞行轨迹的起飞点，通过在可降落区域中选定降落点，保证可移动平台能够顺利完成降落，并能够实现安全、可靠降落，避免预估返航里程大于剩余续航里程，而将降落点设为飞行轨 迹的起飞点而使可移动平台无法顺利完成降落而存在损坏或损毁的问题，进一步提高了可移动平台的可靠性，延长了可移动平台的使用寿命。
可以理解的是，起飞点可以为飞行轨迹的起始点，也可以为指定的home点，也可以为指定飞行计划中的点，如靠近home点设定的其他点。
图19示出了根据本申请又一个实施例的语义地图的构建方法的示意流程图。如图19所示，该语义地图的构建方法包括：
S1102，按照预设频率采集多个图像；
S1104，获取多个图像的语义分割信息，其中，多个图像中的任一图像对应的语义分割信息包括若干个像素点的语义识别结果；
S1106，获取每一个像素点的语义识别结果的置信度，将置信度低于预设阈值的语义识别结果删除；
S1108，根据可移动平台相对于多个图像所对应的实体的高度信息，对多个图像进行拼接生成拼接图像；
S1110，根据多个图像的语义分割信息，获取拼接图像的语义地图；
S1112，根据语义地图，确定可移动平台的可降落区域；
S1114，获取可移动平台的电池的剩余电量；
S1116，根据电池的剩余电量确定电池的剩余续航里程；
S1118，获取可移动平台的飞行轨迹；
S1120，根据飞行轨迹和语义地图，确定可移动平台的预估返航里程；
S1122，基于预估返航里程小于或等于剩余续航里程的情况下，将飞行轨迹的起飞点作为降落点；基于预估返航里程大于剩余续航里程的情况下，根据剩余续航里程和起飞点，在可降落区域中选定降落点；
S1124，根据可移动平台的降落点，控制可移动平台进行降落；
S1126，根据语义地图，控制可移动平台进行避障飞行；其中，避障飞行包括绕道飞行或爬升飞行。
在该实施例中，根据语义地图分别确定可移动平台的可降落区域和控制可移动平台进行壁障飞行，通过根据语义地图，控制可移动平台进行避障飞行，由于语义地图具有较高的置信度，能够完整、精确地获取真实场景的障碍物的位置信息，控制可移动平台进行障碍飞行避开障碍物，有利于提 高可移动平台飞行的可靠性，进而延长可移动平台的使用寿命，提高产品的可靠性。
其中，避障飞行包括绕道飞行或爬升飞行，绕道飞行即绕过障碍物飞行，爬升飞行即向上飞行越过障碍物，可以理解的是，也可以包括其他飞行方式，如绕道飞行和爬行飞行同时进行。
进一步地，避障飞行可以在返航过程中进行避障飞行，也可以是可移动平台根据语义地图进行避障飞行，进一步提高飞行的可靠性。
在本申请的一个实施例中，优选地，可移动平台包括采集装置，构建方法还包括：控制采集装置采集多个图像。
在该实施例中，具体限定了语义地图的构建方法中采集多个图像的方式，通过控制可移动平台的采集装置采集多个图像，操纵简单，易于实现。
可以理解的是，采集装置可以为多个，多个采集装置能够采集不同视角、不同背景信息的场景的图像，进而有利于提高语义地图的置信度。可以理解的是，多个采集装置设置在可移动平台的不同位置，以便于采集可移动平台不同飞行姿态、不同视角、不同背景信息的图像。
在本申请的一个实施例中，优选地，还包括：根据可移动平台的飞行姿态，控制可移动平台朝向地面一侧的采集装置采集多个图像。
在该实施例中，由于可移动平台的降落点一般设置在地面上，即可移动平台最终是降落在地面上的降落点，通过根据可移动平台的飞行姿态，控制可移动平台朝向地面一侧的采集装置采集多个图像，进而获得地面一侧的语义地图，有利于使可移动平台安全、可靠、精准地降落在地面上的降落点，可操作强，易于实现，适于推广应用。
可以理解的是，也可以根据理想的降落点的方位，使靠近理想降落点的方位的一侧的采集装置采集多个图像，进而使可移动平台能够安全、可靠、精准地降落在理想降落点，进一步扩大产品的使用范围。
在本申请的一个实施例中，优选地，采集装置包括：雷达、视觉传感器或多光谱传感器。
在该实施例中，采集装置可以为雷达、视觉传感器或多光谱传感器，采集装置的多种类型能够满足采集装置不同安装位置、采集不同视角图像、 采集不同背景信息图像的需求，同时能够满足可移动平台不同成本的需求，有利于扩大产品的使用范围。
可以理解的是，采集装置也可以为满足要求的其他装置。
图20示出了根据本申请又一个实施例的语义地图的构建方法的示意流程图。如图20所示，该语义地图的构建方法包括：
S1202，接收起飞指令，控制采集装置启动，以采集多个图像；
S1204，获取多个图像的语义分割信息，其中，多个图像中的任一图像对应的语义分割信息包括若干个像素点的语义识别结果；
S1206，获取每一个像素点的语义识别结果的置信度，将置信度低于预设阈值的语义识别结果删除；
S1208，根据可移动平台相对于多个图像所对应的实体的高度信息，对多个图像进行拼接生成拼接图像；
S1210，根据多个图像的语义分割信息，获取拼接图像的语义地图；
S1212，接收返航指令或检测到可移动平台发生故障，控制采集装置关闭；
S1214，根据语义地图，确定可移动平台的可降落区域；
S1216，获取可移动平台的电池的剩余电量；
S1218，根据电池的剩余电量确定电池的剩余续航里程；
S1220，获取可移动平台的飞行轨迹；
S1222，根据飞行轨迹和语义地图，确定可移动平台的预估返航里程；
S1224，基于预估返航里程小于或等于剩余续航里程的情况下，将飞行轨迹的起飞点作为降落点；
基于预估返航里程大于剩余续航里程的情况下，根据剩余续航里程和起飞点，在可降落区域中选定降落点；
S1226，根据可移动平台的降落点，控制可移动平台进行降落；
S1228，根据语义地图，控制可移动平台进行避障飞行；其中，避障飞行包括绕道飞行或爬升飞行。
在该实施例中，通过接收起飞指令，控制采集装置启动，以采集多个图像，即当可移动平台起飞时就开始采集多个图像，并实时构建语义地图， 通过接收返航指令或检测到可移动平台发生故障，控制采集装置关闭，即当可移动平台需要返航时，控制采集装置关闭，停止采集图像，并根据构建语义地图精准地获取位置信息，进而确定可移动平台的降落点，即降落位置信息，使可移动平台能够安全、可靠、精准地降落在降落点，完成返航，避免了相关技术中可移动平台降落在水中、树上、建筑物上等损坏或损毁可移动平台的问题，大大延长了可移动平台的使用寿命，提高可移动平台使用的安全性，并提高产品的可靠性。
进一步地，一方面，返航指令可以为用户选择的返航键触发的返航指令，另一方面，返航指令为可移动平台飞行至飞行轨迹的返航点时可移动平台的控制器发送的返航指令。返航指令的不同方式能够满足可移动平台不同工况的需求，进而扩大产品的使用范围，同时，有利于灵活控制可移动平台安全返航，进一步提高可移动平台的可靠性。
如图21所示，本申请的第二方面的实施例，提出了一种语义地图的构建系统10，包括：存储器12，用于存储计算机程序；处理器14，用于执行计算机程序以实现：获取多个图像的语义分割信息；对多个图像进行拼接操作生成拼接图像，根据多个图像的语义分割信息，获取拼接图像的语义地图。
本申请实施例提供的语义地图的构建系统10，包括存储器12和处理器14，存储器12用于存储计算机程序，通过令处理器14获取多个图像的语义分割信息，而多个图像可以为不同视角、不同背景信息的场景的图像，有利于通过多个图像的语义分割信息得到完整的、精确的真实场景的多个实体内容的信息，通过处理器14对多个图像进行拼接操作生成拼接图像，有利于保证场景的完整度和真实性，根据多个图像的语义分割信息，获取拼接图像的语义地图，使得语义地图较好的趋于真实场景，并完整、精确地体现了真实场景的多个实体内容，进而使得获取的语义地图具有较高的置信度，提高了场景理解的精确度，通过语义地图能够精准的获取位置信息。
具体实施例中，通过图像采集装置采集多个图像，多个图像的采集时间为连续的，图像采集装置包括并不限于：视觉传感器、雷达、多光谱传 感器。
在本申请的一个实施例中，优选地，处理器14用于执行获取多个图像的语义分割信息具体为：通过预设的卷积神经网络模块对多个图像进行语义分割以得到多个图像的语义分割信息。
在该实施例中，处理器14用于执行获取多个图像的语义分割信息具体为：处理器14通过预设的卷积神经网络模块对多个图像进行语义分割以得到多个图像的语义分割信息，能够完整、精确地获得图像中的实体内容的信息，进而有利于根据多个图像的语义分割信息获取的语义地图完整、精确地体现了真实场景的多个实体内容，具有较高的置信度，提高场景理解的精确度。
可以理解的是，处理器14也可以通过其他方法获得多个图像的语义分割信息。具体地，卷积神经网络(CNN，Convolutional Neural Networks)适用于各种场景任务，特别是场景目标语义信息、位置信息的获取，因此CNN可以被用来识别航拍场景的各种目标的语义信息和位置信息。
具体实施例中，处理器14通过图像采集装置进行连续采集多个图像，将多个图像输入预设的卷积神经网络模块进行语义分割以得到语义分隔信息，根据语义分隔信息和多个图像进行拼接生成的拼接图像构建出语义地图。处理器14通过连续采集图像，可以获得更完整和详细的语义地图；进一步处理器14通过卷积神经网络获取多个图像的语义分割信息，可以更准确的获取图像中的实体场景，进而得到更精确的语义地图。
在本申请的一个实施例中，优选地，多个图像中的任一图像对应的语义分割信息包括若干个像素点的语义识别结果，处理器14用于执行在对多个图像进行拼接生成拼接图像的步骤之前，还用于：获取每一个像素点的语义识别结果的置信度，将置信度低于预设阈值的语义识别结果删除。
在该实施例中，由于多个图像中的任一图像对应的语义分割信息包括若干个像素点的语义识别结果，处理器14在对多个图像进行拼接生成拼接图像的步骤之前，通过获取每一个像素点的语义识别结果的置信度，将置信度低于预设阈值的语义识别结果删除，使得语义分割信息中仅包括置信度较高的语义识别结果，即语义分割信息能够真实、完整地体现图像对应的 实体内容的信息，进而使得根据多个图像的语义分割信息获取的语义地图具有较高的置信度，能够完整、精确地体现真实场景的实体内容，提高场景理解的精确度，使得通过语义地图能够精准的获取位置信息。
进一步地，语义识别结果可以为与图像对应的真实场景中的实体内容，可以理解的是，语义识别结果可以为多个，对应于真实场景中的多个实体内容。
可以理解的是，语义分割信息包括若干个像素点的语义识别结果和语义识别结果对应的置信度，通过置信度与预设阈值的比较，将置信度低于预设阈值的语义识别结果删除，使得语义分割信息中仅包括置信度较高的语义识别结果，即语义分割信息能够真实、完整地体现图像对应的实体内容的信息，进而使得通过语义地图能够精准的获取位置信息。
在本申请的一个实施例中，优选地，适用于可移动平台，其中，处理器14用于实现：根据可移动平台相对于多个图像所对应的实体的高度信息，对多个图像进行拼接生成拼接图像。
在该实施例中，语义地图的构建系统10适用于可移动平台，可移动平台可以为飞机、无人机，也可以为满足要求的其他可移动平台。处理器14通过根据可移动平台相对于多个图像所对应的实体的高度信息，对多个图像进行拼接生成拼接图像，有利于保证生成的拼接图像具有较高的清晰度和还原度，各实体的高度真实体现在拼接图像中，进而有利于获取的拼接图像的语义地图具有较高的置信度，并提高了场景理解的精确度。
优选地，实体的高度信息由可移动平台上的双目摄像头测量得到。
进一步地，处理器14通过根据可移动平台相对于多个图像所对应的实体的高度信息和语义分割信息的识别结果进行二维语义地图的拼接，使得通过拼接图像的语义地图能够精确地获取场景中的目标语义信息和距离信息，使得通过语义地图获取的位置信息更加准确，进而有利于可移动平台根据目标语义信息和距离信息准确移动，提高了可移动平台移动的安全性和准确性，有利于提高产品的可靠性。其中，目标语义信息可以为与图像相对应的多个实体内容中的目标实体的信息，距离信息可以为可移动平台与目标实体的距离。具体实施例中，可移动平台为无人机，目标语义信息 为图像中的地面所对应的语义信息，距离信息为无人机与地面之间的距离，通过根据地面的语义信息及无人机与地面之间的距离，能够精准地获取地面的位置信息，进而使无人机能够安全、准确地降落在地面上。
具体地，处理器14可以对任一图像进行单帧识别来获取每一个像素点的语义识别结果，进行连续采集多个图像并结合可移动平台相对于多个图像所对应的实体的高度信息，进行图像拼接，实现多帧构建实时的语义地图。可以理解的是，处理器14也可以通过其他方式获取每一个像素点的语义识别结果。具体地，当可移动平台为无人机时，高度为图像对应的实体结构与无人机的距离。具体地，图像拼接时，多个图像重叠的部分可以进行融合，例如，对多个图像重叠部分的每一个像素点的识别结果的置信度进行比较，通过保留置信度较高的图像、删除置信度较低的图像对重叠部分的图像进行融合，即最大限度的提取每个图像中的有利信息，能够使融合后的拼接图像保证了场景的完整度和真实性，进而使语义地图具有较高的置信度。
在本申请的一个实施例中，优选地，语义识别结果包括以下至少一种：建筑物、天空、树、水面、地面。
在该实施例中，语义识别结果包括建筑物、天空、树、水面、地面中的一种或多种，语义识别结果的多种类型包括了与图片对应的真实场景中的多种实体内容，进而使得语义结果能够真实、完整地体现图像对应的实体内容，有利于提高场景理解的精确度。进一步地，语义识别结果也可以包括满足要求的其他内容。
在本申请的一个实施例中，优选地，处理器14还用于实现：按照预设频率采集多个图像。
在该实施例中，处理器14通过按照预设频率采集多个图像，可以得到不同视角、不同背景信息的场景的图像，使得通过多个图像能够完整、精确地反映真实场景中不同视角、不同位置、不同背景信息的实体内容，有利于通过多个图像的语义分割信息得到完整的、精确的真实场景中的多个实体内容，进而保证语义地图的可靠性和准确性。
在本申请的一个实施例中，优选地，处理器14还用于实现：根据语义 地图，确定可移动平台的降落点；根据可移动平台的降落点，控制可移动平台进行降落。
在该实施例中，处理器14通过根据语义地图，确定可移动平台的降落点，使得根据置信度较高、场景理解精确度较高的语义地图能够精准地获取位置信息，进而确定可移动平台的降落点，且降落点安全可靠，处理器14并根据可移动平台的降落点控制可移动平台进行降落，使得可移动平台能够安全、可靠、精准地降落至通过语义地图确定的降落点，避免了相关技术中可移动平台降落在水中、树上、建筑物上等损坏或损毁可移动平台的问题，大大延长了可移动平台的使用寿命，提高可移动平台使用的安全性，并提高产品的可靠性。
在本申请的一个实施例中，优选地，处理器14用于实现根据语义地图，确定可移动平台的降落点具体为：根据语义地图，确定可移动平台的可降落区域；根据可移动平台的状态信息，在可降落区域中选定降落点。
在该实施例中，处理器14通过根据语义地图，确定可移动平台的可降落区域，可降落区域可以为根据语义地图得到的安全、可靠的允许可移动平台降落的区域，即，不包括能够使可移动平台降落存在危险或破坏性的区域，如水、树、建筑物等区域，进而避免了可移动平台在降落时损坏或损毁，有利于延长可移动平台的使用寿命；处理器14通过根据可移动平台的状态信息，在可降落区域中选定降落点，有利于结合可移动平台的状态信息，使选择的降落点能够保证可移动平台安全、可靠地降落，避免由于可移动平台的自身的状态无法满足顺利抵达降落点或无法在降落点顺利完成降落，进一步保证了可移动平台能够安全、顺利、可靠、精准地降落在降落点，提高可移动平台的可靠性。
在本申请的一个实施例中，优选地，处理器14用于实现：根据可移动平台的状态信息，在可降落区域中选定降落点的步骤，具体为：获取可移动平台的电池的剩余电量；根据剩余电量和语义地图，在可降落区域中选定降落点。
在该实施例中，具体限定了处理器14根据可移动平台的状态信息，在可降落区域中选定降落点的步骤。处理器14通过获取可移动平台的电池的 剩余电量，根据剩余电量和语义地图，在可降落区域中选定降落点，使得选定的降落点能够保证可移动平台利用剩余电量顺利降落在降落点，避免了电池的剩余电量无法使可移动平台顺利抵达降落点而损坏或损毁可移动设备，使得选定的降落点具有较高的准确性，进而保证了可移动平台能够可靠、安全地完成降落，延长可移动平台的使用寿命。
在本申请的一个实施例中，优选地，处理器14还用于实现：获取可移动平台的飞行轨迹，根据飞行轨迹和剩余电量，选定降落点。
在该实施例中，处理器14通过获取可移动平台的飞行轨迹，根据飞行轨迹和剩余电量选定降落点，使得选定的降落点与飞行轨迹相适配，有利于可移动平台依照飞行轨迹实现返航，提高可移动平台返航的准确性，同时能够保证可移动平台利用剩余电量顺利降落在降落点，进而提高可移动平台降落的可靠性和安全性。
在本申请的一个实施例中，优选地，处理器14还用于实现：根据电池的剩余电量确定电池的剩余续航里程，根据剩余续航里程和飞行轨迹，选定降落点。
在该实施例中，处理器14通过根据电池的剩余电量确定电池的剩余续航里程，将电池的剩余电量具体量化为电池的剩余续航里程，使得根据量化的剩余续航里程和飞行轨迹，准确、合理地选定降落点，有利于提高降落位置信息的精确性，既能够保证可移动平台安全、可靠地降落在降落点，并基于剩余电量确定的剩余续航里程最大限度的依照飞行轨迹完成返航，进而提高可移动平台返航的准确性。
在本申请的一个实施例中，优选地，处理器14用于实现根据剩余续航里程和飞行轨迹，选定降落点的步骤，具体为：根据飞行轨迹和语义地图，确定可移动平台的预估返航里程；基于预估返航里程小于或等于剩余续航里程的情况下，将飞行轨迹的起飞点作为降落点。
在该实施例中，具体限定了处理器14根据剩余续航里程和飞行轨迹，选定降落点的步骤。处理器14通过根据飞行轨迹和语义地图，确定可移动平台的预估返航里程，即预估返航里程为可移动平台返回至飞行轨迹的起飞点的里程，处理器14基于预估返航里程小于或等于剩余续航里程的情况 下，说明利用电池的剩余电量可移动平台能够返回至飞行轨迹的起飞点，进而将飞行轨迹的起飞点作为降落点，进一步提高降落点的精确性，使得可移动平台能够安全、可靠、精准地降落在起飞点，提高了可移动平台返航的精准度。
可以理解的是，起飞点可以为飞行轨迹的起始点，也可以为指定的home点，也可以为指定飞行计划中的点，如靠近home点设定的其他点。
在本申请的一个实施例中，优选地，基于预估返航里程大于剩余续航里程的情况下，处理器14还用于实现：根据剩余续航里程和起飞点，在可降落区域中选定降落点。
在该实施例中，处理器14基于预估返航里程大于剩余续航里程的情况下，说明利用电池的剩余电量可移动平台无法返回至飞行轨迹的起飞点，通过在可降落区域中选定降落点，保证可移动平台能够顺利完成降落，并能够实现安全、可靠降落，避免预估返航里程大于剩余续航里程，而将降落点设为飞行轨迹的起飞点而使可移动平台无法顺利完成降落而存在损坏或损毁的问题，进一步提高了可移动平台的可靠性，延长了可移动平台的使用寿命。
在本申请的一个实施例中，优选地，处理器14还用于实现：根据语义地图，控制可移动平台进行避障飞行；其中，避障飞行包括绕道飞行或爬升飞行。
在该实施例中，处理器14通过根据语义地图，控制可移动平台进行避障飞行，由于语义地图具有较高的置信度，能够完整、精确地获取真实场景的障碍物的位置信息，处理器14控制可移动平台进行障碍飞行避开障碍物，有利于提高可移动平台飞行的可靠性，进而延长可移动平台的使用寿命，提高产品的可靠性。
其中，避障飞行包括绕道飞行或爬升飞行，绕道飞行即绕过障碍物飞行，爬升飞行即向上飞行越过障碍物，可以理解的是，也可以包括其他飞行方式，如绕道飞行和爬行飞行同时进行。
进一步地，避障飞行可以在返航过程中进行避障飞行，也可以是可移动平台根据语义地图进行避障飞行，进一步提高飞行的可靠性。
在本申请的一个实施例中，优选地，可移动平台包括采集装置，处理器14还用于实现：控制采集装置采集多个图像。
在该实施例中，具体限定了语义地图的构建方法中采集多个图像的方式，通过控制可移动平台的采集装置采集多个图像，操纵简单，易于实现。
可以理解的是，采集装置可以为多个，多个采集装置能够采集不同视角、不同背景信息的场景的图像，进而有利于提高语义地图的置信度。可以理解的是，多个采集装置设置在可移动平台的不同位置，以便于采集可移动平台不同飞行姿态、不同视角、不同背景信息的图像。
在本申请的一个实施例中，优选地，处理器14还用于实现：根据可移动平台的飞行姿态，控制可移动平台朝向地面一侧的采集装置采集多个图像。
在该实施例中，由于可移动平台的降落点一般设置在地面上，即可移动平台最终是降落在地面上的降落点，处理器14通过根据可移动平台的飞行姿态，控制可移动平台朝向地面一侧的采集装置采集多个图像，进而获得地面一侧的语义地图，有利于使可移动平台安全、可靠、精准地降落在地面上的降落点，可操作强，易于实现，适于推广应用。
可以理解的是，处理器14也可以根据理想的降落点的方位，使靠近理想降落点的方位的一侧的采集装置采集多个图像，进而使可移动平台能够安全、可靠、精准地降落在理想降落点，进一步扩大产品的使用范围。
在本申请的一个实施例中，优选地，处理器14还用于实现：接收起飞指令，控制采集装置启动，以采集多个图像；以及接收返航指令或检测到可移动平台发生故障，控制采集装置关闭。
在该实施例中，处理器14通过接收起飞指令，控制采集装置启动，以采集多个图像，即当可移动平台起飞时就开始采集多个图像，并实时构建语义地图，处理器14通过接收返航指令或检测到可移动平台发生故障，控制采集装置关闭，即当可移动平台需要返航时，控制采集装置关闭，停止采集图像，并根据构建语义地图精准地获取位置信息，进而确定可移动平台的降落点，即降落位置信息，使可移动平台能够安全、可靠、精准地降落在降落点，完成返航，避免了相关技术中可移动平台降落在水中、树上、 建筑物上等损坏或损毁可移动平台的问题，大大延长了可移动平台的使用寿命，提高可移动平台使用的安全性，并提高产品的可靠性。
进一步地，一方面，返航指令可以为用户选择的返航键触发的返航指令，另一方面，返航指令为可移动平台飞行至飞行轨迹的返航点时可移动平台的控制器发送的返航指令。返航指令的不同方式能够满足可移动平台不同工况的需求，进而扩大产品的使用范围，同时，有利于灵活控制可移动平台安全返航，进一步提高可移动平台的可靠性。
如图22所示，本申请的第三方面的实施例提出了一种可移动平台20，包括上述任一实施例的语义地图的构建系统10；以及采集装置22，采集装置22与构建系统相连接，采集装置22用于采集图像并发送至处理器。由于可移动平台20包括上述任一实施例的语义地图的构建系统10，因此具有上述任一实施例的语义地图的构建系统10的全部有益效果，在此不再赘述。
在本申请的一个实施例中，采集装置22包括：雷达、视觉传感器或多光谱传感器。
在该实施例中，采集装置22可以为雷达、视觉传感器或多光谱传感器，采集装置22的多种类型能够满足采集装置22不同安装位置、采集不同视角图像、采集不同背景信息图像的需求，同时能够满足可移动平台20不同成本的需求，有利于扩大产品的使用范围。可以理解的是，采集装置22也可以为满足要求的其他装置。
本申请的第四方面的实施例提出了一种计算机可读存储介质，其上存储有计算机程序，计算机程序被处理器执行时实现上述任一实施例的语义地图的构建方法。因而具备上述任一技术方案的语义地图的构建方法的有益效果，在此不再赘述。
具体地，计算机可读存储介质可以包括能够存储或传输信息的任何介质。计算机可读存储介质的例子包括电子电路、半导体存储器设备、ROM、闪存、可擦除ROM(EROM)、软盘、CD-ROM、光盘、硬盘、光纤介质、射频(RF)链路，等等。代码段可以经由诸如因特网、内联网等的计算机网络被下载。
如图23所示，本申请的第五方面的实施例提出了一种可移动平台20， 包括机体24、设于机体24上的供电电池26、动力系统28、采集装置22和控制器21，供电电池26，用于为动力系统28供电，动力系统28，用于可移动平台20提供飞行动力；采集装置22，用于在可移动平台20飞行过程中，获取多个图像；控制器21，用于获取多个图像的语义分割信息；对多个图像进行拼接操作生成拼接图像，根据多个图像的语义分割信息，获取拼接图像的语义地图。
本申请的实施例提供的可移动平台20包括：机体24、设于机体24上的供电电池26、动力系统28、采集装置22和控制器21，其中，供电电池26用于为动力系统28供电，动力系统28用于为可移动平台20提供飞行动力；采集装置22用于在可移动平台20飞行过程中，获取多个图像，通过控制器21获取多个图像的语义分割信息，而多个图像可以为不同视角、不同背景信息的场景的图像，有利于通过多个图像的语义分割信息得到完整的、精确的真实场景的多个实体内容的信息，通过控制器21对多个图像进行拼接操作生成拼接图像，有利于保证场景的完整度和真实性，根据多个图像的语义分割信息，获取拼接图像的语义地图，使得语义地图较好的趋于真实场景，并完整、精确地体现了真实场景的多个实体内容，进而使得获取的语义地图具有较高的置信度，提高了场景理解的精确度，通过语义地图能够精准的获取位置信息。
具体实施例中，通过图像采集装置22采集多个图像，多个图像的采集时间为连续的，图像采集装置22包括并不限于：视觉传感器、雷达、多光谱传感器。
在本申请的一个实施例中，优选地，控制器21具体用于：通过预设的卷积神经网络模块对多个图像进行语义分割以得到多个图像的语义分割信息。
在该实施例中，控制器21通过预设的卷积神经网络模块对多个图像进行语义分割以得到多个图像的语义分割信息，能够完整、精确地获得图像中的实体内容的信息，进而有利于根据多个图像的语义分割信息获取的语义地图完整、精确地体现了真实场景的多个实体内容，具有较高的置信度，提高场景理解的精确度。
可以理解的是，控制器21可以通过其他方法获得多个图像的语义分割信息。具体地，卷积神经网络(CNN，Convolutional Neural Networks)适用于各种场景任务，特别是场景目标语义信息、位置信息的获取，因此CNN可以被用来识别航拍场景的各种目标的语义信息和位置信息。
具体实施例中，通过图像采集装置22进行连续采集多个图像，控制器21将多个图像输入预设的卷积神经网络模块进行语义分割以得到语义分隔信息，根据语义分隔信息和多个图像进行拼接生成的拼接图像构建出语义地图。控制器21通过图像采集装置22连续采集图像，可以获得更完整和详细的语义地图；进一步控制器21通过卷积神经网络获取多个图像的语义分割信息，可以更准确的获取图像中的实体场景，进而得到更精确的语义地图。
在本申请的一个实施例中，优选地，多个图像中的任一图像对应的语义分割信息包括若干个像素点的语义识别结果，控制器21在对多个图像进行拼接生成拼接图像的步骤之前，控制器21还用于：获取每一个像素点的语义识别结果的置信度，将置信度低于预设阈值的语义识别结果删除。
在该实施例中，由于多个图像中的任一图像对应的语义分割信息包括若干个像素点的语义识别结果，控制器21在对多个图像进行拼接生成拼接图像的步骤之前，通过获取每一个像素点的语义识别结果的置信度，将置信度低于预设阈值的语义识别结果删除，使得语义分割信息中仅包括置信度较高的语义识别结果，即语义分割信息能够真实、完整地体现图像对应的实体内容的信息，进而使得根据多个图像的语义分割信息获取的语义地图具有较高的置信度，能够完整、精确地体现真实场景的实体内容，提高场景理解的精确度，使得通过语义地图能够精准的获取位置信息。
进一步地，语义识别结果可以为与图像对应的真实场景中的实体内容，可以理解的是，语义识别结果可以为多个，对应于真实场景中的多个实体内容。
可以理解的是，语义分割信息包括若干个像素点的语义识别结果和语义识别结果对应的置信度，通过置信度与预设阈值的比较，将置信度低于预设阈值的语义识别结果删除，使得语义分割信息中仅包括置信度较高的 语义识别结果，即语义分割信息能够真实、完整地体现图像对应的实体内容的信息，进而使得通过语义地图能够精准的获取位置信息。
在本申请的一个实施例中，优选地，控制器21具体用于：根据可移动平台20相对于多个图像所对应的实体的与高度信息，对多个图像进行拼接生成拼接图像。
在该实施例中，控制器21通过根据可移动平台20相对于多个图像所对应的实体的高度信息，对多个图像进行拼接生成拼接图像，有利于保证生成的拼接图像具有较高的清晰度和还原度，各实体的高度真实体现在拼接图像中，进而有利于获取的拼接图像的语义地图具有较高的置信度，并提高了场景理解的精确度。
优选地，实体的高度信息由可移动平台上的双目摄像头测量得到。
进一步地，控制器21通过根据可移动平台20相对于多个图像所对应的实体的高度信息和语义分割信息的识别结果进行二维语义地图的拼接，使得通过拼接图像的语义地图能够精确地获取场景中的目标语义信息和距离信息，使得通过语义地图获取的位置信息更加准确，进而有利于可移动平台20根据目标语义信息和距离信息准确移动，提高了可移动平台20移动的安全性和准确性，有利于提高产品的可靠性。其中，目标语义信息可以为与图像相对应的多个实体内容中的目标实体的信息，距离信息可以为可移动平台20与目标实体的距离。具体实施例中，可移动平台为无人机，目标语义信息为图像中的地面所对应的语义信息，距离信息为无人机与地面之间的距离，通过根据地面的语义信息及无人机与地面之间的距离，能够精准地获取地面的位置信息，进而使无人机能够安全、准确地降落在地面上。
具体地，控制器21可以对任一图像进行单帧识别来获取每一个像素点的语义识别结果，控制采集装置22进行连续采集多个图像，并结合可移动平台20相对于多个图像所对应的实体的高度信息，进行图像拼接，实现多帧构建实时的语义地图。可以理解的是，控制器21也可以通过其他方式获取每一个像素点的语义识别结果。具体地，当可移动平台20为无人机时，高度为图像对应的实体结构与无人机的距离。具体地，图像拼接时，多个 图像重叠的部分可以进行融合，例如，对多个图像重叠部分的每一个像素点的识别结果的置信度进行比较，通过保留置信度较高的图像、删除置信度较低的图像对重叠部分的图像进行融合，即最大限度的提取每个图像中的有利信息，能够使融合后的拼接图像保证了场景的完整度和真实性，进而使语义地图具有较高的置信度。
在本申请的一个实施例中，优选地，语义识别结果包括以下至少一种：建筑物、天空、树、水面、地面。
在该实施例中，语义识别结果包括建筑物、天空、树、水面、地面中的一种或多种，语义识别结果的多种类型包括了与图片对应的真实场景中的多种实体内容，进而使得语义结果能够真实、完整地体现图像对应的实体内容，有利于提高场景理解的精确度。进一步地，语义识别结果也可以包括满足要求的其他内容。
在本申请的一个实施例中，优选地，采集装置22具体用于按照预设频率采集多个图像。
在该实施例中，通过采集装置22按照预设频率采集多个图像，可以得到不同视角、不同背景信息的场景的图像，使得控制器21通过多个图像能够完整、精确地反映真实场景中不同视角、不同位置、不同背景信息的实体内容，有利于通过多个图像的语义分割信息得到完整的、精确的真实场景中的多个实体内容，进而保证语义地图的可靠性和准确性。
在本申请的一个实施例中，优选地，控制器21具体用于：根据语义地图，确定可移动平台20的降落点；根据可移动平台20的降落点，控制可移动平台20进行降落。
在该实施例中，控制器21通过根据语义地图，确定可移动平台20的降落点，使得根据置信度较高、场景理解精确度较高的语义地图能够精准地获取位置信息，进而确定可移动平台20的降落点，且降落点安全可靠，控制器21并根据可移动平台20的降落点控制动力系统28工作使可移动平台20进行降落，使得可移动平台20能够安全、可靠、精准地降落至通过语义地图确定的降落点，避免了相关技术中可移动平台20降落在水中、树上、建筑物上等损坏或损毁可移动平台20的问题，大大延长了可移动平台20 的使用寿命，提高可移动平台20使用的安全性，并提高产品的可靠性。
在本申请的一个实施例中，优选地，控制器21根据语义地图，确定可移动平台20的降落点具体为：根据语义地图，确定可移动平台20的可降落区域；根据可移动平台20的状态信息，在可降落区域中选定降落点。
在该实施例中，控制器21通过根据语义地图，确定可移动平台20的可降落区域，可降落区域可以为根据语义地图得到的安全、可靠的允许可移动平台20降落的区域，即，不包括能够使可移动平台20降落存在危险或破坏性的区域，如水、树、建筑物等区域，进而避免了可移动平台20在降落时损坏或损毁，有利于延长可移动平台20的使用寿命；控制器21通过根据可移动平台20的状态信息，在可降落区域中选定降落点，有利于结合可移动平台20的状态信息，使选择的降落点能够保证可移动平台20安全、可靠地降落，避免由于可移动平台20的自身的状态无法满足顺利抵达降落点或无法在降落点顺利完成降落，进一步保证了可移动平台20能够安全、顺利、可靠、精准地降落在降落点，提高可移动平台20的可靠性。
在本申请的一个实施例中，优选地，控制器21根据可移动平台20的状态信息，在可降落区域中选定降落点的步骤，具体为：获取供电电池26的剩余电量；根据剩余电量和语义地图，在可降落区域中选定降落点。
在该实施例中，具体限定了控制器21根据可移动平台20的状态信息，在可降落区域中选定降落点的步骤。控制器21通过获取可移动平台20的供电电池26的剩余电量，根据剩余电量和语义地图，在可降落区域中选定降落点，使得选定的降落点能够保证可移动平台20利用供电电池26剩余电量顺利降落在降落点，避免了供电电池26的剩余电量无法使可移动平台20顺利抵达降落点而损坏或损毁可移动设备，使得选定的降落点具有较高的准确性，进而保证了可移动平台20能够可靠、安全地完成降落，延长可移动平台20的使用寿命。
在本申请的一个实施例中，优选地，控制器21具体还用于：获取可移动平台20的飞行轨迹，根据飞行轨迹和剩余电量，选定降落点。
在该实施例中，控制器21通过获取可移动平台20的飞行轨迹，根据飞行轨迹和供电电池26剩余电量选定降落点，使得选定的降落点与飞行轨迹 相适配，有利于可移动平台20依照飞行轨迹实现返航，提高可移动平台20返航的准确性，同时能够保证可移动平台20利用供电电池26剩余电量顺利降落在降落点，进而提高可移动平台20降落的可靠性和安全性。
在本申请的一个实施例中，优选地，控制器21具体还用于：根据供电电池26的剩余电量确定供电电池26的剩余续航里程，根据剩余续航里程和飞行轨迹，选定降落点。
在该实施例中，控制器21通过根据供电电池26的剩余电量确定电池的剩余续航里程，将供电电池26的剩余电量具体量化为电池的剩余续航里程，使得根据量化的剩余续航里程和飞行轨迹，准确、合理地选定降落点，有利于提高降落位置信息的精确性，既能够保证可移动平台20安全、可靠地降落在降落点，并基于剩余电量确定的剩余续航里程最大限度的依照飞行轨迹完成返航，进而提高可移动平台20返航的准确性。
在本申请的一个实施例中，优选地，控制器21根据剩余续航里程和飞行轨迹，选定降落点的步骤，具体为：根据飞行轨迹和语义地图，确定可移动平台20的预估返航里程；基于预估返航里程小于或等于剩余续航里程的情况下，将飞行轨迹的起飞点作为降落点。
在该实施例中，控制器21通过根据飞行轨迹和语义地图，确定可移动平台20的预估返航里程，即预估返航里程为可移动平台20返回至飞行轨迹的起飞点的里程，基于预估返航里程小于或等于剩余续航里程的情况下，说明利用电池的剩余电量可移动平台20能够返回至飞行轨迹的起飞点，进而控制器21将飞行轨迹的起飞点作为降落点，进一步提高降落点的精确性，使得可移动平台20能够安全、可靠、精准地降落在起飞点，提高了可移动平台20返航的精准度。
可以理解的是，起飞点可以为飞行轨迹的起始点，也可以为指定的home点，也可以为指定飞行计划中的点，如靠近home点设定的其他点。
在本申请的一个实施例中，优选地，基于预估返航里程大于剩余续航里程的情况下，控制器21具体还用于：根据剩余续航里程和起飞点，在可降落区域中选定降落点。
在该实施例中，基于预估返航里程大于剩余续航里程的情况下，说明利 用电池的剩余电量可移动平台20无法返回至飞行轨迹的起飞点，控制器21通过在可降落区域中选定降落点，保证可移动平台20能够顺利完成降落，并能够实现安全、可靠降落，避免预估返航里程大于剩余续航里程，而将降落点设为飞行轨迹的起飞点而使可移动平台20无法顺利完成降落而存在损坏或损毁的问题，进一步提高了可移动平台20的可靠性，延长了可移动平台20的使用寿命。
在本申请的一个实施例中，优选地，控制器21具体还用于：根据语义地图，控制可移动平台20进行避障飞行；其中，避障飞行包括绕道飞行或爬升飞行。
在该实施例中，控制器21通过根据语义地图，控制可移动平台20进行避障飞行，由于语义地图具有较高的置信度，能够完整、精确地获取真实场景的障碍物的位置信息，控制器21控制可移动平台20进行障碍飞行避开障碍物，有利于提高可移动平台20飞行的可靠性，进而延长可移动平台20的使用寿命，提高产品的可靠性。
其中，避障飞行包括绕道飞行或爬升飞行，绕道飞行即绕过障碍物飞行，爬升飞行即向上飞行越过障碍物，可以理解的是，也可以包括其他飞行方式，如绕道飞行和爬行飞行同时进行。
进一步地，避障飞行可以在返航过程中进行避障飞行，也可以是可移动平台根据语义地图进行避障飞行，进一步提高飞行的可靠性。
在本申请的一个实施例中，优选地，控制器21还用于：根据可移动平台20的飞行姿态，控制可移动平台20朝向地面一侧的采集装置22采集多个图像。
在该实施例中，由于可移动平台20的降落点一般设置在地面上，即可移动平台20最终是降落在地面上的降落点，控制器21通过根据可移动平台20的飞行姿态，控制可移动平台20朝向地面一侧的采集装置22采集多个图像，进而获得地面一侧的语义地图，有利于使可移动平台20安全、可靠、精准地降落在地面上的降落点，可操作强，易于实现，适于推广应用。
可以理解的是，控制器21也可以根据理想的降落点的方位，使靠近理想降落点的方位的一侧的采集装置22采集多个图像，进而使可移动平台20能 够安全、可靠、精准地降落在理想降落点，进一步扩大产品的使用范围。
在本申请的一个实施例中，优选地，采集装置22包括：雷达、视觉传感器或多光谱传感器。
在该实施例中，采集装置22可以为雷达、视觉传感器或多光谱传感器，采集装置22的多种类型能够满足采集装置22不同安装位置、采集不同视角图像、采集不同背景信息图像的需求，同时能够满足可移动平台20不同成本的需求，有利于扩大产品的使用范围。可以理解的是，采集装置22也可以为满足要求的其他装置。
在本申请的一个实施例中，优选地，控制器21还用于：接收起飞指令，控制动力系统28和采集装置22启动，以控制可移动平台20飞行及采集装置22采集多个图像；以及接收返航指令或检测到可移动平台20发生故障，控制采集装置22关闭。
在该实施例中，控制器21通过接收起飞指令，控制动力系统28启动，可移动平台起飞，并控制采集装置22启动以采集多个图像，即当可移动平台20起飞时采集装置22就开始采集多个图像，控制器21并实时构建语义地图，控制器21通过接收返航指令或检测到可移动平台20发生故障，控制采集装置22关闭，即当可移动平台20需要返航时，控制采集装置22关闭，停止采集图像，并停止构建语义地图，并根据构建语义地图精准地获取位置信息，进而确定可移动平台20的降落点，使可移动平台20能够安全、可靠、精准地降落在降落点，完成返航，避免了相关技术中可移动平台20降落在水中、树上、建筑物上等损坏或损毁可移动平台20的问题，大大延长了可移动平台20的使用寿命，提高可移动平台20使用的安全性，并提高产品的可靠性。
进一步地，一方面，返航指令可以为用户选择的返航键触发的返航指令，另一方面，返航指令为可移动平台20飞行至飞行轨迹的返航点时可移动平台20的控制器21发送的返航指令。返航指令的不同方式能够满足可移动平台20不同工况的需求，进而扩大产品的使用范围，同时，有利于灵活控制可移动平台20安全返航，进一步提高可移动平台20的可靠性。
在具体实施例中，本申请的可移动平台20为无人机，而相关技术中的 无人机，无人机场景由于复杂的环境，返航时通常不能找到安全可靠的降落环境而出现掉落水中、树上、建筑物之上等情况使得无人机出现损毁，而本申请的无人机，包括机体24、设于机体24上的供电电池26、动力系统28、采集装置22和控制器21，控制器21通过接收起飞指令，控制动力系统28启动，无人机起飞，并控制采集装置22启动，即在起飞时通过采集装置22(如雷达、视觉传感器、多光谱传感器)实时采集多个图像，并实时将多个图像进行拼接操作。而多个图像可以为连续采集的不同视角、不同背景信息的场景的图像，如可以为无人机遮挡场景时采集的图像，控制器21通过预设的卷积神经网络模块对多个图像进行语义分割以得到多个图像的语义分割信息，其中，任一图像对应的语义分割信息包括若干个像素点的语义识别结果。具体图像的语义识别过程可以为，将预处理之后的图像数据以RGB(Red Green Blue，色彩模式)三通道数据送入网络模型中，依次由前向传播后即经过网络模型的迭代后得到网络输出结果。具体过程如图24所示，图像数据输入的格式为N×4×H×W，将输入数据经过由多个“Conv+bn+Relu”运算层构成的卷积神经网络的处理后，得到网络输出结果为一个N×K×H×W的张量，对其进行处理后得到识别结果和识别置信度，通过将置信度低于预设阈值的语义识别结果删除，使得语义分割信息中仅包括置信度较高的语义识别结果，语义识别结果可以包括：建筑、天空、树、水面、地面等，即语义分割信息能够真实、完整地体现图像对应的实体内容。然后，控制器21通过根据无人机相对于多个图像所对应的实体的高度信息和语义分割信息的识别结果进行二维语义地图的拼接，具体地，采用多帧图像并结合无人机相对于多个图像所对应的实体的高度进行拼接，而对于多个图像重叠的部分进行融合得到拼接图像，根据多个图像的语义分割信息，获取拼接图像的语义地图，使得语义地图较好的趋于真实场景，并完整、精确地体现了真实场景的多个实体内容的信息，进而使得获取的语义地图具有较高的置信度，提高了场景理解的精确度，通过语义地图能够精准的获取位置信息。
当无人机控制器21接收返航指令或检测到无人机发生故障时，控制采集装置22停止采集图像，根据当前时刻构成的语义地图，通过语义地图的 构建方法实现了对无人机场景内语义信息和降落点信息的精确获取。具体地，通过获取无人机的飞行轨迹和无人机的供电电池26的剩余电量，根据飞行轨迹和剩余电量选定降落点，获取精准的降落位置信息，避免了相关技术中无人机降落在水中、树上、建筑物上等损坏或损毁无人机的问题，大大延长了无人机的使用寿命，提高无人机使用的安全性，并提高产品的可靠性。其中，选定的降落点与飞行轨迹相适配，有利于无人机依照降落位置信息实现返航，提高无人机返航的准确性，同时能够保证无人机利用剩余电量顺利降落在降落点，进而提高无人机降落的可靠性和安全性。
由于语义地图具有较高的置信度，能够完整、精确地获取真实场景的障碍物的位置信息，通过控制器21控制无人机在返航时进行避障飞行避开障碍物，有利于提高无人机飞行的可靠性，进而延长无人机的使用寿命，提高产品的可靠性。
本申请的第六方面的实施例提供了一种搜索降落点的方法，适用于一可移动平台，包括步骤：根据以上任一项实施例提供的语义地图的构建方法获取语义地图；根据语义地图，确定可移动平台的降落点；根据可移动平台的降落点，控制可移动平台进行降落。
本申请提供的可移动平台包括：机体、设于机体上的供电电池、动力系统、采集装置和控制器，其中，供电电池用于为动力系统供电，动力系统用于为可移动平台提供飞行动力；采集装置用于在可移动平台飞行过程中，获取多个图像，通过控制器获取多个图像的语义分割信息，而多个图像可以为不同视角、不同背景信息的场景的图像，有利于控制器通过多个图像的语义分割信息得到完整的、精确的真实场景的多个实体内容的信息，通过控制器对多个图像进行拼接操作生成拼接图像，有利于保证场景的完整度和真实性，根据多个图像的语义分割信息，获取拼接图像的语义地图，使得语义地图较好的趋于真实场景，并完整、精确地体现了真实场景的多个实体内容，进而使得获取的语义地图具有较高的置信度，提高了场景理解的精确度，使得控制器通过语义地图能够精准的获取位置信息。
进一步地，通过根据语义地图，确定可移动平台的降落点，使得根据置信度较高、场景理解精确度较高的语义地图能够精准地获取位置信息，进 而确定可移动平台的降落点，且降落点安全可靠，并根据可移动平台的降落点控制可移动平台进行降落，使得可移动平台能够安全、可靠、精准地降落至通过语义地图确定的降落点，避免了相关技术中可移动平台降落在水中、树上、建筑物上等损坏或损毁可移动平台的问题，大大延长了可移动平台的使用寿命，提高可移动平台使用的安全性，并提高产品的可靠性。
本申请的一个实施例中，根据语义地图，确定可移动平台的降落点具体为：根据语义地图，确定可移动平台的可降落区域；根据可移动平台的状态信息，在可降落区域中选定降落点。
在该实施例中，通过根据语义地图，确定可移动平台的可降落区域，可降落区域可以为根据语义地图得到的安全、可靠的允许可移动平台降落的区域，即，不包括能够使可移动平台降落存在危险或破坏性的区域，如水、树、建筑物等区域，进而避免了可移动平台在降落时损坏或损毁，有利于延长可移动平台的使用寿命；通过根据可移动平台的状态信息，在可降落区域中选定降落点，有利于结合可移动平台的状态信息，使选择的降落点能够保证可移动平台安全、可靠地降落，避免由于可移动平台的自身的状态无法满足顺利抵达降落点或无法在降落点顺利完成降落，进一步保证了可移动平台能够安全、顺利、可靠、精准地降落在降落点，提高可移动平台的可靠性。
本申请的一个实施例中，根据可移动平台的状态信息，在可降落区域中选定降落点的步骤，具体为：获取可移动平台的电池的剩余电量；根据剩余电量和语义地图，在可降落区域中选定降落点。
在该实施例中，具体限定了根据可移动平台的状态信息，在可降落区域中选定降落点的步骤。通过获取可移动平台的电池的剩余电量，根据剩余电量和语义地图，在可降落区域中选定降落点，使得选定的降落点能够保证可移动平台利用剩余电量顺利降落在降落点，避免了电池的剩余电量无法使可移动平台顺利抵达降落点而损坏或损毁可移动设备，使得选定的降落点具有较高的准确性，进而保证了可移动平台能够可靠、安全地完成降落，延长可移动平台的使用寿命。
在本申请的一个实施例中，搜索降落点的方法还包括：获取可移动平台 的飞行轨迹，根据飞行轨迹和剩余电量，选定降落点。
在该实施例中，通过获取可移动平台的飞行轨迹和可移动平台的电池的剩余电量，根据飞行轨迹和剩余电量选定降落点，使得选定的降落点与飞行轨迹相适配，有利于可移动平台依照飞行轨迹实现返航，提高可移动平台返航的准确性，同时能够保证可移动平台利用剩余电量顺利降落在降落点，进而提高可移动平台降落的可靠性和安全性。
在本申请的一个实施例中，搜索降落点的方法还包括：根据电池的剩余电量确定电池的剩余续航里程，根据剩余续航里程和飞行轨迹，选定降落点。
在该实施例中，根据语义地图分别获取可移动平台的电池的剩余电量和获取可移动平台的飞行轨迹，通过根据电池的剩余电量确定电池的剩余续航里程，将电池的剩余电量具体量化为电池的剩余续航里程，并根据剩余续航里程和飞行轨迹，选定降落点，使得根据量化的剩余续航里程和飞行轨迹，准确、合理地选定降落点，有利于提高降落位置信息的精确性，既能够保证可移动平台安全、可靠地降落在降落点，并基于剩余电量确定的剩余续航里程最大限度的依照飞行轨迹完成返航，进而提高可移动平台返航的准确性。
在本申请的一个实施例中，根据剩余续航里程和飞行轨迹，选定降落点的步骤，具体为：根据飞行轨迹和语义地图，确定可移动平台的预估返航里程；基于预估返航里程小于或等于剩余续航里程的情况下，将飞行轨迹的起飞点作为降落点；基于预估返航里程大于剩余续航里程的情况下，根据剩余续航里程和起飞点，在可降落区域中选定降落点。
在该实施例中，具体限定了根据剩余续航里程和飞行轨迹，选定降落点的步骤。根据语义地图分别获取可移动平台的电池的剩余电量和获取可移动平台的飞行轨迹，并通过根据飞行轨迹和语义地图，确定可移动平台的预估返航里程，即预估返航里程为可移动平台返回至飞行轨迹的起飞点的里程，基于预估返航里程小于或等于剩余续航里程和预估返航里程大于剩余续航里程两种情况，一方面，基于预估返航里程小于或等于剩余续航里程的情况下，说明利用电池的剩余电量可移动平台能够返回至飞行轨迹的 起飞点，进而将飞行轨迹的起飞点作为降落点，进一步提高降落点的精确性，使得可移动平台能够安全、可靠、精准地降落在起飞点，提高了可移动平台返航的精准度。
另一方面，基于预估返航里程大于剩余续航里程的情况下，说明利用电池的剩余电量可移动平台无法返回至飞行轨迹的起飞点，通过在可降落区域中选定降落点，保证可移动平台能够顺利完成降落，并能够实现安全、可靠降落，避免预估返航里程大于剩余续航里程，而将降落点设为飞行轨迹的起飞点而使可移动平台无法顺利完成降落而存在损坏或损毁的问题，进一步提高了可移动平台的可靠性，延长了可移动平台的使用寿命。
可以理解的是，起飞点可以为飞行轨迹的起始点，也可以为指定的home点，也可以为指定飞行计划中的点，如靠近home点设定的其他点。
在具体实施例中，可移动平台20获取降落点的位置信息的具体过程如图25所示，可移动平台20的控制器21控制采集装置22实时获取的多个图像输入至卷积神经网络(CNN)模块，并对多个图像进行语义分割得到语义分割信息，输出的语义分割信息包括多个图像中的任一图像的若干个像素点的语义识别结果和语义识别置信度。进一步地，根据语义置信度将置信度低于预设阈值的语义识别结果删除，使得语义分割信息中仅包括置信度较高的语义识别结果。通过对多个图像进行拼接生成拼接图像，对拼接图像中的重叠部分进行融合，再将体现有语义识别结果的多帧图像的语义识别图覆盖在拼接图像上，即可获得多帧拼接图像的实时语义地图。根据构建的语义地图，一方面，通过语义地图的构建方法对目标降落点进行语义判断，即可获得精准的降落点的位置信息，另一方面，通过语义地图的构建方法对目标降落点进行语义判断和智能搜索(如可移动平台20的电池的剩余电量、飞行轨迹等)，即可获得精准的降落点的位置信息，进而有利于可移动平台20安全、可靠地降落，延长可移动平台20的使用寿命。
在本申请中，术语“多个”则指两个或两个以上，除非另有明确的限定。术语“安装”、“相连”、“连接”、“固定”等术语均应做广义理解，例如，“连接”可以是固定连接，也可以是可拆卸连接，或一体地连接；“相连”可以是直接相连，也可以通过中间媒介间接相连。对于本领域的普通技术人员而 言，可以根据具体情况理解上述术语在本申请中的具体含义。
在本说明书的描述中，术语“一个实施例”、“一些实施例”、“具体实施例”等的描述意指结合该实施例或示例描述的具体特征、结构、材料或特点包含于本申请的至少一个实施例或示例中。在本说明书中，对上述术语的示意性表述不一定指的是相同的实施例或实例。而且，描述的具体特征、结构、材料或特点可以在任何的一个或多个实施例或示例中以合适的方式结合。
以上仅为本申请的优选实施例而已，并不用于限制本申请，对于本领域的技术人员来说，本申请可以有各种更改和变化。凡在本申请的精神和原则之内，所作的任何修改、等同替换、改进等，均应包含在本申请的保护范围之内。

Claims (62)

  1. 一种语义地图的构建方法，所述方法包括：
    获取多个图像的语义分割信息；
    对所述多个图像进行拼接操作生成拼接图像，根据所述多个图像的语义分割信息，获取所述拼接图像的语义地图。
  2. 根据权利要求1所述的语义地图的构建方法，其中，所述获取多个图像的语义分割信息具体为：
    通过预设的卷积神经网络模块对所述多个图像进行语义分割以得到所述多个图像的语义分割信息。
  3. 根据权利要求2所述的语义地图的构建方法，其中，所述多个图像中的任一图像对应的语义分割信息包括若干个像素点的语义识别结果，在对所述多个图像进行拼接生成拼接图像的步骤之前，还包括：
    获取每一个像素点的语义识别结果的置信度，将所述置信度低于预设阈值的语义识别结果删除。
  4. 根据权利要求3所述的语义地图的构建方法，适用于可移动平台，所述方法还包括：
    根据所述可移动平台相对于所述多个图像所对应的实体的高度信息，对所述多个图像进行拼接生成所述拼接图像。
  5. 根据权利要求3所述的语义地图的构建方法，其中，所述语义识别结果包括以下至少一种：建筑物、天空、树、水面、地面。
  6. 根据权利要求1所述的语义地图的构建方法，所述方法还包括：
    按照预设频率采集所述多个图像。
  7. 根据权利要求4所述的语义地图的构建方法，所述方法还包括：
    根据所述语义地图，确定所述可移动平台的降落点；
    根据所述可移动平台的降落点，控制所述可移动平台进行降落。
  8. 根据权利要求7所述的语义地图的构建方法，其中，根据所述语义地图，确定所述可移动平台的降落点具体为：
    根据所述语义地图，确定所述可移动平台的可降落区域；
    根据所述可移动平台的状态信息，在所述可降落区域中选定降落点。
  9. 根据权利要求8所述的语义地图的构建方法，其中，所述根据所述可移动平台的状态信息，在所述可降落区域中选定降落点的步骤，具体为：
    获取所述可移动平台的电池的剩余电量；
    根据所述剩余电量和所述语义地图，在所述可降落区域中选定所述降落点。
  10. 根据权利要求9所述的语义地图的构建方法，所述方法还包括：
    获取所述可移动平台的飞行轨迹，根据所述飞行轨迹和所述剩余电量，选定所述降落点。
  11. 根据权利要求10所述的语义地图的构建方法，所述方法还包括：
    根据所述电池的剩余电量确定所述电池的剩余续航里程，根据所述剩余续航里程和所述飞行轨迹，选定所述降落点。
  12. 根据权利要求11所述的语义地图的构建方法，其中，所述根据所述剩余续航里程和所述飞行轨迹，选定所述降落点的步骤，具体为：
    根据所述飞行轨迹和所述语义地图，确定所述可移动平台的预估返航里程；
    基于所述预估返航里程小于或等于所述剩余续航里程的情况下，将所述飞行轨迹的起飞点作为所述降落点。
  13. 根据权利要求12所述的语义地图的构建方法，其中，基于所述预估返航里程大于所述剩余续航里程的情况下，还包括：
    根据所述剩余续航里程和所述起飞点，在所述可降落区域中选定所述降落点。
  14. 根据权利要求7至13中任一项所述的语义地图的构建方法，其中，所述方法还包括：
    根据所述语义地图，控制所述可移动平台进行避障飞行；
    其中，所述避障飞行包括绕道飞行或爬升飞行。
  15. 根据权利要求7至13中任一项所述的语义地图的构建方法，其中，所述可移动平台包括采集装置，所述构建方法还包括：
    控制所述采集装置采集所述多个图像。
  16. 根据权利要求15所述的语义地图的构建方法，其中，还包括：根据所述可移动平台的飞行姿态，控制所述可移动平台朝向地面一侧的所述采集装置采集所述多个图像。
  17. 根据权利要求15所述的语义地图的构建方法，其中，
    所述采集装置包括：雷达、视觉传感器或多光谱传感器。
  18. 根据权利要求15所述的语义地图的构建方法，其中，还包括：接收起飞指令，控制所述采集装置启动，以采集所述多个图像；以及
    接收返航指令或检测到所述可移动平台发生故障，控制所述采集装置关闭。
  19. 一种语义地图的构建系统，包括：
    存储器，用于存储计算机程序；
    处理器，用于执行所述计算机程序以实现：
    获取多个图像的语义分割信息；
    对所述多个图像进行拼接操作生成拼接图像，根据所述多个图像的语义分割信息，获取所述拼接图像的语义地图。
  20. 根据权利要求19所述的语义地图的构建系统，其中，所述处理器用于执行获取多个图像的语义分割信息具体为：
    通过预设的卷积神经网络模块对所述多个图像进行语义分割以得到所述多个图像的语义分割信息。
  21. 根据权利要求20所述的语义地图的构建系统，其中，所述多个图像中的任一图像对应的语义分割信息包括若干个像素点的语义识别结果，所述处理器用于执行在对所述多个图像进行拼接生成拼接图像的步骤之前，还用于：
    获取每一个像素点的语义识别结果的置信度，将所述置信度低于预设阈值的语义识别结果删除。
  22. 根据权利要求21所述的语义地图的构建系统，适用于可移动平台，其中，所述处理器用于实现：
    根据所述可移动平台相对于所述多个图像所对应的实体的高度信息，对所述多个图像进行拼接生成所述拼接图像。
  23. 根据权利要求21所述的语义地图的构建系统，其中，所述语义识别结果包括以下至少一种：建筑物、天空、树、水面、地面。
  24. 根据权利要求19所述的语义地图的构建系统，其中，所述处理器还用于实现：
    按照预设频率采集所述多个图像。
  25. 根据权利要求22所述的语义地图的构建系统，其中，所述处理器还用于实现：
    根据所述语义地图，确定所述可移动平台的降落点；
    根据所述可移动平台的降落点，控制所述可移动平台进行降落。
  26. 根据权利要求25所述的语义地图的构建系统，其中，所述处理器用于实现：
    根据所述语义地图，确定所述可移动平台的降落点具体为：
    根据所述语义地图，确定所述可移动平台的可降落区域；
    根据所述可移动平台的状态信息，在所述可降落区域中选定降落点。
  27. 根据权利要求26所述的语义地图的构建系统，其中，所述处理器用于实现：
    所述根据所述可移动平台的状态信息，在所述可降落区域中选定降落点的步骤，具体为：
    获取所述可移动平台的电池的剩余电量；
    根据所述剩余电量和所述语义地图，在所述可降落区域中选定所述降落点。
  28. 根据权利要求27所述的语义地图的构建系统，其中，所述处理器还用于实现：
    获取所述可移动平台的飞行轨迹，根据所述飞行轨迹和所述剩余电量，选定所述降落点。
  29. 根据权利要求28所述的语义地图的构建系统，其中，所述处理器还用于实现：
    根据所述电池的剩余电量确定所述电池的剩余续航里程，根据所述剩余续航里程和所述飞行轨迹，选定所述降落点。
  30. 根据权利要求29所述的语义地图的构建系统，其中，所述处理器用于实现所述根据所述剩余续航里程和所述飞行轨迹，选定所述降落点的步骤，具体为：
    根据所述飞行轨迹和所述语义地图，确定所述可移动平台的预估返航里程；
    基于所述预估返航里程小于或等于所述剩余续航里程的情况下，将所述飞行轨迹的起飞点作为所述降落点。
  31. 根据权利要求30所述的语义地图的构建系统，其中，基于所述预估返航里程大于所述剩余续航里程的情况下，所述处理器还用于实现：
    根据所述剩余续航里程和所述起飞点，在所述可降落区域中选定所述降落点。
  32. 根据权利要求25至31中任一项所述的语义地图的构建系统，其中，所述处理器还用于实现：根据所述语义地图，控制所述可移动平台进行避障飞行；
    其中，所述避障飞行包括绕道飞行或爬升飞行。
  33. 根据权利要求25至31中任一项所述的语义地图的构建系统，其中，所述可移动平台包括采集装置，所述处理器还用于实现：控制所述采集装置采集所述多个图像。
  34. 根据权利要求33所述的语义地图的构建系统，其中，所述处理器还用于实现：根据所述可移动平台的飞行姿态，控制所述可移动平台朝向地面一侧的所述采集装置采集所述多个图像。
  35. 根据权利要求33所述的语义地图的构建系统，其中，所述处理器还用于实现：
    接收起飞指令，控制所述采集装置启动，以采集所述多个图像；以及
    接收返航指令或检测到所述可移动平台发生故障，控制所述采集装置关闭。
  36. 一种可移动平台，其中，包括如权利要求19至35中任一项所述的语义地图的构建系统；以及
    采集装置，所述采集装置与所述构建系统相连接，所述采集装置用于采集 所述图像并发送至所述处理器。
  37. 根据权利要求36所述的可移动平台，其中，
    所述采集装置包括：雷达、视觉传感器或多光谱传感器。
  38. 一种计算机可读存储介质，其上存储有计算机程序，其中，所述计算机程序被处理器执行时实现如权利要求1至18中任一项所述的语义地图的构建方法。
  39. 一种可移动平台，其中，包括机体、设于所述机体上的供电电池、动力系统、采集装置和控制器，其中，
    所述供电电池，用于为所述动力系统供电；
    所述动力系统，用于所述可移动平台提供飞行动力；
    所述采集装置，用于在所述可移动平台飞行过程中，获取多个图像；
    所述控制器，用于获取所述多个图像的语义分割信息；对所述多个图像进行拼接操作生成拼接图像，根据所述多个图像的语义分割信息，获取所述拼接图像的语义地图。
  40. 根据权利要求39所述的可移动平台，其中，所述控制器具体用于通过预设的卷积神经网络模块对所述多个图像进行语义分割以得到所述多个图像的语义分割信息。
  41. 根据权利要求40所述的可移动平台，其中，所述多个图像中的任一图像对应的语义分割信息包括若干个像素点的语义识别结果，所述控制器在对所述多个图像进行拼接生成拼接图像的步骤之前，所述控制器还用于：
    获取每一个像素点的语义识别结果的置信度，将所述置信度低于预设阈值的语义识别结果删除。
  42. 根据权利要求41所述的可移动平台，其中，所述控制器具体用于：
    根据所述可移动平台相对于所述多个图像所对应的实体的与高度信息，对所述多个图像进行拼接生成所述拼接图像。
  43. 根据权利要求41所述的可移动平台，其中，所述语义识别结果包括以下至少一种：建筑物、天空、树、水面、地面。
  44. 根据权利要求39所述的可移动平台，其中，
    所述采集装置具体用于按照预设频率采集所述多个图像。
  45. 根据权利要求42所述的可移动平台，所述控制器具体用于：
    根据所述语义地图，确定所述可移动平台的降落点；
    根据所述可移动平台的降落点，控制所述可移动平台进行降落。
  46. 根据权利要求45所述的可移动平台，其中，所述控制器根据所述语义地图，确定所述可移动平台的降落点具体为：
    根据所述语义地图，确定所述可移动平台的可降落区域；
    根据所述可移动平台的状态信息，在所述可降落区域中选定降落点。
  47. 根据权利要求46所述的可移动平台，其中，所述控制器根据所述可移动平台的状态信息，在所述可降落区域中选定降落点的步骤，具体为：
    获取所述供电电池的剩余电量；
    根据所述剩余电量和所述语义地图，在所述可降落区域中选定所述降落点。
  48. 根据权利要求47所述的可移动平台，其中，所述控制器具体还用于：
    获取所述可移动平台的飞行轨迹，根据所述飞行轨迹和所述剩余电量，选定所述降落点。
  49. 根据权利要求48所述的可移动平台，所述控制器具体还用于：
    根据所述供电电池的剩余电量确定所述供电电池的剩余续航里程，根据所述剩余续航里程和所述飞行轨迹，选定所述降落点。
  50. 根据权利要求49所述的可移动平台，其中，所述控制器根据所述剩余续航里程和所述飞行轨迹，选定所述降落点的步骤，具体为：
    根据所述飞行轨迹和所述语义地图，确定所述可移动平台的预估返航里程；
    基于所述预估返航里程小于或等于所述剩余续航里程的情况下，将所述飞行轨迹的起飞点作为所述降落点。
  51. 根据权利要求50所述的可移动平台，其中，基于所述预估返航里程大于所述剩余续航里程的情况下，所述控制器具体还用于：
    根据所述剩余续航里程和所述起飞点，在所述可降落区域中选定所述 降落点。
  52. 根据权利要求39至51中任一项所述的可移动平台，其中，所述控制器具体还用于：
    根据所述语义地图，控制所述可移动平台进行避障飞行；
    其中，所述避障飞行包括绕道飞行或爬升飞行。
  53. 根据权利要求39至51中任一项所述的可移动平台，其中，所述控制器还用于：根据所述可移动平台的飞行姿态，控制所述可移动平台朝向地面一侧的所述采集装置采集所述多个图像。
  54. 根据权利要求39至51中任一项所述的可移动平台，其中，
    所述采集装置包括：雷达、视觉传感器或多光谱传感器。
  55. 根据权利要求39至51中任一项所述的可移动平台，其中，所述控制器还用于：接收起飞指令，控制所述动力系统和所述采集装置启动，以控制所述可移动平台飞行及所述采集装置采集所述多个图像；以及
    接收返航指令或检测到所述可移动平台发生故障，控制所述采集装置关闭。
  56. 一种搜索降落点的方法，适用于一可移动平台，其中，包括步骤：
    根据如权利要求1至6中任一项所述的语义地图的构建方法获取所述语义地图；
    根据所述语义地图，确定所述可移动平台的降落点；
    根据所述可移动平台的降落点，控制所述可移动平台进行降落。
  57. 根据权利要求56所述的搜索降落点的方法，其中，根据所述语义地图，确定所述可移动平台的降落点具体为：
    根据所述语义地图，确定所述可移动平台的可降落区域；
    根据所述可移动平台的状态信息，在所述可降落区域中选定降落点。
  58. 根据权利要求57所述的搜索降落点的方法，其中，所述根据所述可移动平台的状态信息，在所述可降落区域中选定降落点的步骤，具体为：
    获取所述可移动平台的电池的剩余电量；
    根据所述剩余电量和所述语义地图，在所述可降落区域中选定所述降落点。
  59. 根据权利要求58所述的搜索降落点的方法，所述方法还包括：
    获取所述可移动平台的飞行轨迹，根据所述飞行轨迹和所述剩余电量，选定所述降落点。
  60. 根据权利要求59所述的搜索降落点的方法，所述方法还包括：
    根据所述电池的剩余电量确定所述电池的剩余续航里程，根据所述剩余续航里程和所述飞行轨迹，选定所述降落点。
  61. 根据权利要求60所述的搜索降落点的方法，其中，所述根据所述剩余续航里程和所述飞行轨迹，选定所述降落点的步骤，具体为：
    根据所述飞行轨迹和所述语义地图，确定所述可移动平台的预估返航里程；
    基于所述预估返航里程小于或等于所述剩余续航里程的情况下，将所述飞行轨迹的起飞点作为所述降落点。
  62. 根据权利要求61所述的搜索降落点的方法，其中，基于所述预估返航里程大于所述剩余续航里程的情况下，还包括：
    根据所述剩余续航里程和所述起飞点，在所述可降落区域中选定所述降落点。
PCT/CN2019/094799 2019-07-05 2019-07-05 语义地图的构建方法、系统、可移动平台和存储介质 WO2021003587A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201980007922.6A CN111670417A (zh) 2019-07-05 2019-07-05 语义地图的构建方法、系统、可移动平台和存储介质
PCT/CN2019/094799 WO2021003587A1 (zh) 2019-07-05 2019-07-05 语义地图的构建方法、系统、可移动平台和存储介质

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2019/094799 WO2021003587A1 (zh) 2019-07-05 2019-07-05 语义地图的构建方法、系统、可移动平台和存储介质

Publications (1)

Publication Number Publication Date
WO2021003587A1 true WO2021003587A1 (zh) 2021-01-14

Family

ID=72381601

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/094799 WO2021003587A1 (zh) 2019-07-05 2019-07-05 语义地图的构建方法、系统、可移动平台和存储介质

Country Status (2)

Country Link
CN (1) CN111670417A (zh)
WO (1) WO2021003587A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112907574A (zh) * 2021-03-25 2021-06-04 成都纵横自动化技术股份有限公司 一种飞行器的降落点寻找方法、装置、系统及存储介质
CN113359810A (zh) * 2021-07-29 2021-09-07 东北大学 一种基于多传感器的无人机着陆区域识别方法

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113050664A (zh) * 2021-03-24 2021-06-29 北京三快在线科技有限公司 一种无人机降落方法及装置
CN115496930B (zh) * 2022-11-08 2023-03-21 之江实验室 一种图像处理方法、装置、存储介质及电子设备

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030113017A1 (en) * 2001-06-07 2003-06-19 Corinne Thomas Process for the automatic creation of a database of images accessible by semantic features
CN109117718A (zh) * 2018-07-02 2019-01-01 东南大学 一种面向道路场景的三维语义地图构建和存储方法
US20190050648A1 (en) * 2017-08-09 2019-02-14 Ydrive, Inc. Object localization within a semantic domain
CN109559320A (zh) * 2018-09-18 2019-04-02 华东理工大学 基于空洞卷积深度神经网络实现视觉slam语义建图功能的方法及系统

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR102314539B1 (ko) * 2017-06-09 2021-10-18 엘지전자 주식회사 인공지능 이동 로봇의 제어 방법
CN107444665B (zh) * 2017-07-24 2020-06-09 长春草莓科技有限公司 一种无人机自主降落方法
CN108596974B (zh) * 2018-04-04 2020-08-04 清华大学 动态场景机器人定位建图系统及方法
CN109737974B (zh) * 2018-12-14 2020-11-27 中国科学院深圳先进技术研究院 一种3d导航语义地图更新方法、装置及设备

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030113017A1 (en) * 2001-06-07 2003-06-19 Corinne Thomas Process for the automatic creation of a database of images accessible by semantic features
US20190050648A1 (en) * 2017-08-09 2019-02-14 Ydrive, Inc. Object localization within a semantic domain
CN109117718A (zh) * 2018-07-02 2019-01-01 东南大学 一种面向道路场景的三维语义地图构建和存储方法
CN109559320A (zh) * 2018-09-18 2019-04-02 华东理工大学 基于空洞卷积深度神经网络实现视觉slam语义建图功能的方法及系统

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112907574A (zh) * 2021-03-25 2021-06-04 成都纵横自动化技术股份有限公司 一种飞行器的降落点寻找方法、装置、系统及存储介质
CN112907574B (zh) * 2021-03-25 2023-10-17 成都纵横自动化技术股份有限公司 一种飞行器的降落点寻找方法、装置、系统及存储介质
CN113359810A (zh) * 2021-07-29 2021-09-07 东北大学 一种基于多传感器的无人机着陆区域识别方法
CN113359810B (zh) * 2021-07-29 2024-03-15 东北大学 一种基于多传感器的无人机着陆区域识别方法

Also Published As

Publication number Publication date
CN111670417A (zh) 2020-09-15

Similar Documents

Publication Publication Date Title
WO2021003587A1 (zh) 语义地图的构建方法、系统、可移动平台和存储介质
US11760484B2 (en) Detecting optical discrepancies in captured images
JP7274674B1 (ja) 無人航空機による３次元再構成の実行
US11715072B2 (en) System, devices and methods for tele-operated robotics
Al-Kaff et al. Intelligent vehicle for search, rescue and transportation purposes
CN110751336B (zh) 无人载具的避障方法、避障装置及无人载具
US20240062661A1 (en) Unmanned aerial vehicle contingency landing system
WO2022095067A1 (zh) 路径规划方法、路径规划装置、路径规划系统和介质
KR20210007830A (ko) 우발 사고 시나리오에서의 무인 항공기 착륙
Okada et al. Huecode: A meta-marker exposing relative pose and additional information in different colored layers
KR102289752B1 (ko) Gps 음영 지역에서 경로 비행을 수행하는 드론 및 그 방법
Dubey et al. Droan-disparity-space representation for obstacle avoidance: Enabling wire mapping & avoidance
WO2022085368A1 (ja) 情報処理装置、情報処理システム、および方法、並びにプログラム
Chen et al. Robust autonomous landing of UAVs in non-cooperative environments based on comprehensive terrain understanding
Ngo et al. UAV Platforms for Autonomous Navigation in GPS-Denied Environments for Search and Rescue Missions
WO2022004333A1 (ja) 情報処理装置、情報処理システム、および情報処理方法、並びにプログラム
WO2022014322A1 (ja) 情報処理システム、及び情報処理装置
CN115686062A (zh) 基于单目深度估计和反应式策略的飞行器自主导航方法
Chourey Cooperative human-robot search in a partially-known environment using multiple UAVs
KR102631315B1 (ko) Slam 기술 구현을 위한 비전 데이터 및 라이다 데이터 간의 실시간 분석 및 대조를 통해 위치 오차 보정이 가능한 시스템
EP4024155B1 (en) Method, system and computer program product of control of unmanned aerial vehicles
Lusk Vision-based emergency landing of small unmanned aircraft systems
Dong Visual Guidance for Unmanned Aerial Vehicles with Deep Learning
JP2024012827A (ja) 無人航空機対処システム及び映像追尾装置
Schaub et al. PRIMUS: autonomous navigation in open terrain with a tracked vehicle

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19936557

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 19936557

Country of ref document: EP

Kind code of ref document: A1