WO2022193193A1 - Procédé et dispositif de traitement de données - Google Patents

Procédé et dispositif de traitement de données Download PDF

Info

Publication number
WO2022193193A1
WO2022193193A1 PCT/CN2021/081387 CN2021081387W WO2022193193A1 WO 2022193193 A1 WO2022193193 A1 WO 2022193193A1 CN 2021081387 W CN2021081387 W CN 2021081387W WO 2022193193 A1 WO2022193193 A1 WO 2022193193A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
map
marked
movable platform
target data
Prior art date
Application number
PCT/CN2021/081387
Other languages
English (en)
Chinese (zh)
Inventor
江灿森
陈琦
衡量
沈劭劼
Original Assignee
深圳市大疆创新科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市大疆创新科技有限公司 filed Critical 深圳市大疆创新科技有限公司
Priority to PCT/CN2021/081387 priority Critical patent/WO2022193193A1/fr
Priority to CN202180079742.6A priority patent/CN116762094A/zh
Publication of WO2022193193A1 publication Critical patent/WO2022193193A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis

Definitions

  • the present application relates to the technical field of automatic driving, and in particular, to a data processing method and device.
  • High-precision map data is an important basis for vehicle autonomous driving and is of great significance to the development of intelligent vehicles.
  • HD map data is usually provided by map providers. Map suppliers usually only provide high-precision map data with relatively large usage, but do not provide high-precision map data with relatively small usage.
  • the embodiments of the present application provide a data processing method and device, aiming to provide a solution that can adapt to the personalized map requirements of different mobile platforms.
  • the present application provides a data processing method, comprising:
  • map metadata includes any one or a combination of three-dimensional feature points, texture data and semantic information
  • map data is generated according to the map metadata; wherein the map data is used to control the movable platform to move within the spatial scene.
  • control device comprising: a memory for storing instructions and a processor for executing the instructions stored in the memory, where the processor is used to specifically execute:
  • map metadata includes any one or a combination of three-dimensional feature points, texture data and semantic information
  • map data is generated according to the map metadata; wherein the map data is used to control the movable platform to move within the spatial scene.
  • the present application provides a movable platform including an image sensor and the data processing method involved in the second aspect.
  • the movable platform collects image data of a spatial scene when moving, generates map data based on the image data, and then uses the collected map data to control its movement, which can meet the requirements of the mobile platform.
  • Personalized map data requirements for mobile platforms To generate map data based on image data, the existing image sensors on the mobile platform can be used, and there is no need to configure high-cost sensors such as lidar to collect point clouds, reducing the cost of map construction.
  • the map data is generated according to the map metadata, so as to ensure the accuracy of the map data generated by the mobile platform.
  • map data is generated based on image data, and the storage of map data is lighter, which is very convenient for real-time update and maintenance of maps.
  • FIG. 1 is a schematic structural diagram of a movable platform according to an embodiment of the present application
  • FIG. 2 is a schematic flowchart of a data processing method provided by another embodiment of the present application.
  • FIG. 3 is a schematic flowchart of a data processing method provided by another embodiment of the present application.
  • FIG. 4 is a schematic flowchart of a data processing method provided by another embodiment of the present application.
  • FIG. 5 is a schematic flowchart of a data processing method provided by another embodiment of the present application.
  • FIG. 6 is a schematic structural diagram of a control device provided by another embodiment of the present application.
  • a component when referred to as being "fixed to" another component, it can be directly on the other component or there may also be a centered component.
  • a component When a component is considered to be “connected” to another component, it may be directly connected to the other component or there may be a co-existence of an intervening component.
  • the present application provides a data processing method and device.
  • the technical idea of the present application is: the image data of the space scene is collected by the movable platform while walking in the space scene, and the collected data is used to generate the map data, which can adapt to the personalized needs of the movable platform for map data.
  • map data Before generating map data, it is judged whether the generated map metadata meets the requirements of mapping quality, so as to ensure that the map data generated by the mobile platform can accurately reflect the spatial scene.
  • the map data is generated based on the image data collected by the image sensor, no high-cost image sensor is needed, and the cost of map construction is reduced.
  • an embodiment of the present application provides a movable platform 100 .
  • the movable platform 100 includes an image sensor 101 , a travel sensor (not shown) and a control device (not shown).
  • the image sensor 101 is used to collect the image data of the scene around the movable platform 100
  • the driving sensor is used to collect the driving data of the movable platform
  • the control device is used to execute the data processing method described below. Repeat.
  • This application can be used to solve the automatic parking problem in the automatic driving function, and can be used for map construction in the process of short-distance automatic parking, for example, within 300 meters.
  • the map is mainly used to record various landmarks in the parking lot, including parking spaces, traffic signs, road lane lines, landmark buildings, etc. After the map is constructed, it can assist in the realization of functions such as parking lot location recognition in the process of automatic parking, automatic search for parking spaces in the map area, and locating vehicles at any location in the map area.
  • the application provides a data processing method
  • the execution subject of the method is a control device
  • the method specifically includes the following steps:
  • control device controls the image sensor located on the movable platform to collect multiple frames of image data of the space scene.
  • the image sensor on the movable platform is controlled to work, the image sensor collects image data of the space scene, and the image sensor transmits the image data to the control device.
  • the driving sensor collects the position information of the movable platform, transmits the collected position information to the control device, and the control device determines whether to enter a certain space scene according to the position information, and controls the movable platform when it is determined to enter the designated space scene.
  • the upper image sensor works, and the image sensor transmits the collected multi-frame image data to the control device.
  • control device processes the multi-frame image data to obtain map metadata.
  • the map metadata is used to generate map data, and the map metadata includes any one or a combination of three-dimensional feature points, texture data, and semantic information.
  • the three-dimensional feature points are used to reflect the position and shape of objects in the scene space
  • the texture data are used to reflect the surface information of the objects in the scene space
  • the semantic information is used to reflect the categories of objects represented by the texture data and the three-dimensional feature points.
  • map metadata is obtained by extracting two-dimensional feature data from image data, matching two-dimensional feature data in multi-frame image data, and semantic recognition.
  • step S203 enumerates a specific implementation:
  • the control device If the map metadata meets the quality requirements for mapping, the control device generates map data according to the map metadata.
  • the mapping quality requirement is used to determine whether the above-mentioned map metadata is rich enough, that is, whether the quantity of map metadata is sufficient, and whether the data types of the map metadata are sufficient.
  • the quality of the map data constructed using the map metadata will be higher, that is, the map data will more accurately describe the spatial scene. If the data volume of the map metadata is small and the types are single, the quality of the map data constructed by using the map metadata is low, that is, the map data cannot accurately describe the spatial scene.
  • the map metadata is processed to obtain map data, for example, each layer is obtained by image processing of the map metadata.
  • the map data is used to control the movement of the movable platform within the spatial scene.
  • the control device can control the movable platform to move within the space scene at the current moment according to the map data generated at the previous moment.
  • the control device can also control the movable platform to move in the space scene according to the generated map data when entering the space scene again next time.
  • the movable platform collects image data during the walking process, generates map data based on the image data, and then uses the collected map data to control its walking, which can meet the personalized map data requirements of the movable platform, and can directly use
  • the existing image sensors of the mobile platform collect data, and there is no need to configure high-cost sensors, such as lidar.
  • map data is generated according to the map metadata to ensure the accuracy of the generated map data.
  • FIG. 3 another embodiment of the present application provides a data processing method, the execution subject of the method is a control device, and the method specifically includes the following steps:
  • control device controls the image sensor located on the movable platform to collect multiple frames of image data of the space scene.
  • a plurality of image sensors are arranged on the movable platform, and the plurality of image sensors are located around the movable platform and are used to collect image data in the space scene where the movable platform is located.
  • the image sensor for the image data.
  • control device processes the multi-frame image data to obtain map metadata.
  • control device uses the identification of the image sensor to mark the source of the map metadata, and obtain the marked map metadata
  • the obtained map metadata is marked with an identifier of an image sensor that collects the image data. That is to mark the data source of the map metadata.
  • the control device If the marked map metadata meets the quality requirements for mapping, the control device generates marked map data according to the marked map metadata.
  • the mapping quality requirement is used to judge whether the marked map data is rich enough.
  • the map metadata includes any one or a combination of three-dimensional feature points, texture data, and semantic information.
  • the mapping quality requirements include at least one of the following: the total number of three-dimensional feature points reaches the first threshold; there are at least two three-dimensional feature points with different components on the three coordinate axes; the total number of texture data reaches the second threshold. the number threshold; the number of types of texture data reaches a third number threshold; the total number of semantic information reaches a fourth number threshold; and the number of types of semantic information reaches a fifth number threshold.
  • the three-dimensional feature points are sufficiently abundant in number.
  • it is determined whether there are at least two three-dimensional feature points with different components on the three coordinate axes it is determined whether the three-dimensional feature points are sufficiently rich in type. If all 3D feature points are located on the same plane, that is, all 3D feature points have the same component on one of the coordinate axes, for example: all 3D feature points have the same component in the z-axis direction, that is, 3D feature points can only represent A plane cannot represent a rich three-dimensional space scene.
  • the second quantity threshold By judging whether the total quantity of texture data reaches the second quantity threshold, it is determined whether the quantity of texture data is sufficiently abundant. By judging whether the number of types of texture data reaches a third quantity threshold, it is determined whether the texture data is rich enough in type.
  • map metadata it is determined whether the map metadata meets the quality requirements of map construction in combination with its richness in quantity and type.
  • the map data generated based on the map metadata can accurately reflect the spatial scene.
  • map data is generated according to the marked map metadata.
  • the process of generating map data specifically includes at least one of the following:
  • a marked feature data layer is generated according to the marked three-dimensional feature points and marked texture data; and a marked semantic information layer is generated according to the marked semantic information.
  • image processing is performed on the marked three-dimensional features and the marked texture data to obtain a marked feature data layer.
  • the marked semantic information is imaged to obtain the marked semantic information layer.
  • the parking space layer can be generated according to the marked semantic information, which specifically includes: extracting semantic information representing the parking space from the marked semantic information, and performing image processing to generate the parking space according to the semantic information of the represented parking space.
  • the marked parking space layer can be generated according to the marked semantic information, which specifically includes: extracting semantic information representing the parking space from the marked semantic information, and performing image processing to generate the parking space according to the semantic information of the represented parking space.
  • the marked map data includes marking information used to indicate the source of the data.
  • the map data can be filtered according to the marker information and moving direction of the map data, and then the filtered map data can be used to control the movement of the movable platform, so as to reduce the amount of data processing in the process of using the map data, so that the movable platform can Generate control instructions from map data more quickly.
  • map metadata it is judged whether the map metadata meets the requirements of mapping quality according to the richness of each map metadata in quantity and type, and map data that accurately reflects the spatial scene can be obtained according to the map metadata.
  • map data that accurately reflects the spatial scene can be obtained according to the map metadata.
  • the source tagging process is performed on the map metadata, so that the obtained map data can also reflect the data source.
  • the data can be filtered according to the data source to reduce the data processing amount, and then the control instructions can be quickly generated according to the map data. To control the precise movement of the movable platform.
  • FIG. 4 another embodiment of the present application provides a data processing method, the execution subject of the method is a control device, and the method specifically includes the following steps:
  • control device controls the image sensor located on the movable platform to collect multiple frames of image data of the space scene.
  • control device processes the multi-frame image data to obtain map metadata.
  • control device uses the identification of the image sensor to mark the source of the map metadata, and obtains the marked map metadata
  • the control device If the marked map metadata meets the quality requirements for mapping, the control device generates marked map data according to the marked map metadata.
  • control device acquires the moving direction of the movable platform and the real-time data collected by the image sensor.
  • control device can control the movable platform to move in the space scene at the current moment according to the map data generated at the previous moment. It is also possible to control the movable platform to move in the space scene according to the generated map data when entering the space scene again next time.
  • the moving direction of the movable platform is collected by the driving sensor, and the real-time data of the space scene is collected by the image sensor.
  • the control device controls the movable platform to move in the space scene in real time based on the moving direction, real-time data and map data of the movable platform.
  • control device acquires target data matching the moving direction from the marked map data according to the marking information.
  • the marker information of the map data is used to reflect the data source of the map data, that is, the image sensor that collects the image data corresponding to the map data can also be determined.
  • the installation position of the image sensor on the movable platform is fixed, and then the position information of the image sensor that collects the image data corresponding to the map data can be determined according to the marker information.
  • the target data is selected from the map data in combination with the moving direction of the movable platform and the above-mentioned position information.
  • the map data derived from the image sensor installed in front of the movable platform is obtained from the map data as the target data. If the moving direction of the movable vehicle is to walk backward, the map data derived from the image sensor installed behind the movable platform is obtained from the map data as the target data.
  • control device generates a control instruction according to the real-time data and the target data.
  • the real-time data collected by the image sensor is also image data
  • the control device performs feature extraction on the image data, and determines the location information of the movable platform according to the processed real-time data and map data, and then generates a mobile platform based on the location information and map data.
  • the processed real-time data and the map data are matched to obtain a matching result, and the location information of the movable platform is determined according to the successfully matched map data.
  • the reliability value of the target data is set according to the matching result.
  • the reliability value of the target data is set to the first reliability value
  • the reliability value of the target data is set to the second reliability value.
  • the first reliability value is greater than the second reliability value.
  • the control device After obtaining the reliable value of the target data, the control device counts the reliable value of the target data to obtain the reliability statistical result. When the reliability statistical result meets the low reliability condition, the target data is deleted to realize the optimization of the map data.
  • the reliability statistics result is the average value of the reliability value
  • the low reliability condition is that the average value of the reliability value is smaller than the preset average value.
  • the control device after the control device generates the map data with the source mark, selects target data from the map data according to the mark information, and controls the movable platform to move according to the target data and the real-time data collected by the image sensor, and obtains by screening Target data, reducing the amount of data processing during the use of map data, the control device can generate control instructions more quickly, so that the movable device can move in the space scene reliably.
  • the control device matches the real-time data with the target data, the target data is marked with the matching result, so as to realize the optimization of the map data.
  • the data processing method provided by the present application is described below by taking the movable platform as an intelligent car as an example.
  • the execution subject of the method is a control device in the intelligent car, such as a trip computer, and the method specifically includes the following steps:
  • control device controls the image sensor located on the movable platform to collect multiple frames of image data of the space scene.
  • the smart car is equipped with a monocular camera, such as a driving recorder, and a fisheye camera installed around the smart car.
  • the above cameras are used to collect image data in a certain spatial scene, such as an underground parking lot.
  • Smart cars are also equipped with driving sensors, such as low-precision inertial navigation units, odometers, GPS, etc.
  • the data processing method provided by the present application does not require the smart car to add new sensors, and the above sensors can be used to generate map data and control the driving of the smart vehicle.
  • control device processes the multi-frame image data to obtain map metadata.
  • the control device processes the multi-frame image data to obtain map metadata. Specifically include the following steps:
  • the VIO and VO algorithms can be used to process the image data and the data collected by the driving sensor to estimate the frame-to-frame pose between two frames of image data.
  • the frame-to-frame pose serves as the basis for image data processing. It is also possible to estimate the frame-to-frame pose using driving sensors, such as integrating the data collected by the odometer and the inertial measurement unit to obtain the frame-to-frame pose.
  • feature extraction is performed on the image data collected by the monocular camera and the image data collected by the fisheye camera.
  • geometric features are extracted from the image data, such as: object edges, corners, planes, salient points, special textures, and the like. It also extracts texture data, gradient data, pixel color and other data in the image data.
  • These characteristic information have the characteristics of time stability, angle stability, scale stability, etc., and can be observed stably and consistently at different angles, distances, and time periods.
  • the extracted texture data, gradient data, pixel color, etc. are also used to encode the two-dimensional feature data, so as to perform feature matching and build a map dictionary.
  • the image data collected by the fisheye camera for example: convert the image data under the fisheye camera model to the image data under the pinhole camera model, and convert the two-dimensional feature data in the pinhole image data to the image data under the pinhole camera model.
  • the two-dimensional feature data in the resulting fisheye image data can be fused, which is conducive to feature matching processing.
  • Timing correlation methods include algorithms such as inter-frame correlation, window correlation, and loopback correlation.
  • the inter-frame correlation is mainly for two adjacent images, for example: an image with an acquisition time interval of 50 milliseconds, or an image with a displayed position interval of 20 cm. Usually, there will be more feature correlations in the inter-frame matching.
  • Window correlation mainly refers to correlating all features within a period of time or distance. Through the quantitative statistics of feature correlation, performance indicators such as feature stability and consistency can be measured.
  • the two-dimensional feature data when a two-dimensional feature data can be associated with a large number of images within a window, such as 30 frames of images, the two-dimensional feature data is high-quality two-dimensional feature data, and has better robustness to temporal and spatial changes sex.
  • the calculation of three-dimensional feature points is based on the three-point coplanarity assumption, using two images at different positions to observe the same object, and calculate the three-dimensional coordinates of the object.
  • the image feature data is triangulated by using the inter-frame poses and matching results of two frames of image data to obtain three-dimensional feature points.
  • semantic information is mainly to extract the information of objects with clear categories in the spatial scene, such as: ground lane lines, parking spaces, indicating arrows, etc., air collision bars, hanging signs, large walls, pillars, etc.
  • Semantic information is usually a relatively stable element. Usually, only when the environment changes, such as parking lot maintenance and reconstruction, will the semantic information fail, which can accurately reflect the spatial scene.
  • image data will be collected by using cameras located in different directions, for example, using monocular cameras and fisheye cameras to collect image data.
  • marking is performed according to the characteristics of image data collected by different cameras.
  • map data filter different map metadata for matching according to the driving direction of the intelligent vehicle.
  • the control device If the marked map metadata meets the quality requirements for mapping, the control device generates marked map data according to the marked map metadata.
  • map metadata meets the requirements of mapping quality through information such as the richness of spatial three-dimensional feature points, the richness of semantic information, and the richness of texture data. If it does not meet the requirements, the control device will send out warning information to warn you to choose a more suitable space scene and build a map in a more suitable time period.
  • the marked map data is generated. More specifically, image processing is performed on the marked three-dimensional features and the marked texture data to obtain a marked feature data layer.
  • the marked semantic information is imaged to obtain the marked semantic information layer.
  • the corresponding layers can be generated based on individual requirements for the autonomous driving of smart cars. For example: building a dictionary of keyframes.
  • the process of generating map data is described below by taking the generation of map data of the parking lot as an example: when the intelligent vehicle drives along the path in the figure, the fisheye camera and the front-view camera collect the image data in the parking lot, and stitch the image data collected by the fisheye camera. A look-around top view is formed, and then the images collected by the look-up top view and the monocular camera are spliced into a ground image, and then the deep learning method is used to realize the identification and extraction of parking spaces, mainly including lane lines, parking spaces, ground indicating arrows and other information, as semantics important part of the map.
  • the identification and storage of valid parking spaces, invalid parking spaces, exclusive parking spaces, parking space numbers and other information in the map can be realized, which can be used for interactive selection of customers during automatic parking.
  • the map data can be optimized. Specifically, it includes: the position of the 3D point, the pose of the camera, and the quality of the semantic map.
  • control device acquires the moving direction of the movable platform and the real-time data collected by the image sensor.
  • control device obtains the driving direction of the smart car, and obtains the image data collected by the fisheye camera and the monocular camera.
  • the monocular camera when the intelligent vehicle is driving forward, the monocular camera will be used to collect the map data corresponding to the image data for positioning, and when the intelligent vehicle is driving backward, the map data corresponding to the image data collected by the fisheye camera will be used for positioning.
  • the map data corresponding to the image data collected by the fisheye camera In order to make intelligent vehicles have faster and more powerful positioning capabilities.
  • the control device generates a control instruction according to the real-time data and the target data.
  • the control device matches the real-time data with the map data to obtain a matching result, determines the location information of the intelligent vehicle according to the successfully matched map data, and then controls the intelligent vehicle to drive according to the location information of the intelligent vehicle.
  • the control device after the control device generates the map data with the source mark, the target data is selected from the map data according to the mark information, and the driving of the vehicle is controlled according to the target data and the real-time data collected by the image sensor, so as to reduce the process of using the map data.
  • the driving of the smart car In order to realize the rapid positioning of the smart car, and then control the driving of the smart car more reliably.
  • FIG. 5 another embodiment of the present application provides a data processing method.
  • the execution body of the method is a control device in a smart car, such as a trip computer.
  • the method specifically includes the following steps:
  • the movable platform is a smart car
  • a driving recorder is installed in front of the smart car
  • a pinhole camera in the driving recorder is used as an image sensor.
  • a fisheye camera is installed around the smart car, which also acts as an image sensor. Pinhole cameras and fisheye cameras are used to collect image data of spatial scenes.
  • the smart car is also equipped with an odometer, an Inertial Measurement Unit (IMU) and GPS.
  • the inertial measurement unit, GPS and odometer are used to collect the driving data of the smart car, such as acceleration, speed, mileage, data such as driving location.
  • the inter-frame pose is estimated based on the multi-frame image data, the odometer and the driving data collected by the IMU.
  • VINS monocular visual inertial system
  • VIO visual inertial odometer
  • VO visual odometer
  • this step is mainly to perform feature extraction on the image data collected by the pinhole camera and the image data collected by the fisheye camera to obtain feature data.
  • the feature data is two-dimensional feature data.
  • Two-dimensional feature data includes geometric feature data, such as: object edges, corners, planes, salient points, special textures and other features. These characteristic information have the characteristics of time stability, angle stability, scale stability, etc., and are relatively stable at different angles, distances, and time periods, and maintain consistent observability.
  • the two-dimensional feature data is effectively expressed, using texture data, gradient data, pixel color data, etc. to encode the two-dimensional feature data, and the encoded two-dimensional feature data is used for feature matching and dictionary data.
  • the image data collected by the fisheye camera is corrected, so that the image data under the fisheye camera model is converted to the pinhole camera model.
  • the two-dimensional features of the image data collected by the pinhole camera The 2D feature data of the data and the image data collected by the fisheye camera can be fused to achieve feature matching with higher stability and consistency.
  • the semantic information processing part is mainly to process the objects with clear meaning in the image data of the spatial scene, such as the ground objects such as lane lines, parking spaces, indicating arrows, anti-collision bars, hanging signs, large walls, pillars and other spaces in the space.
  • Class recognition of objects is performed to obtain semantic information.
  • Semantic information is usually a relatively stable element, and it is usually only in the case of large-scale environmental changes, such as parking lot maintenance and reconstruction, that semantic information is unavailable.
  • the semantic information extraction of the look-up top view mainly includes the extraction of semantic information such as lane lines, parking spaces, ground indicating arrows, etc., which can be used as an important part of the semantic information layer.
  • the identification and storage of valid parking spaces, invalid parking spaces, exclusive parking spaces, parking space numbers and other information in the map can be realized, which can be used for interactive selection of customers during automatic parking.
  • the matching of two-dimensional and feature data is mainly to perform time series association on the extracted two-dimensional feature data.
  • Algorithms such as inter-frame association, window association, and loopback association can be used for matching.
  • the inter-frame correlation is mainly to correlate two adjacent images, for example, the interval is 50 milliseconds, or the interval is 20 centimeters. Usually, there will be more feature correlations in the inter-frame matching.
  • Window correlation is mainly to correlate all features within a fixed or non-fixed time or within a fixed or non-fixed distance range, and obtain performance indicators such as the stability and consistency of the feature by counting the number of feature correlations. For example, when a two-dimensional feature data can be associated with a large number of images within a window, such as 30 frames of images, it indicates that the two-dimensional feature data is of very high quality and has good robustness to temporal and spatial changes.
  • Loopback matching means that data may be collected multiple times in the same spatial scene.
  • loopback matching means that data may be collected multiple times in the same spatial scene.
  • it can not only identify whether a map has been constructed at the current location, but also effectively fuse and update the image data of the spatial scene observed multiple times.
  • S606 construct a dictionary of key frames in the map data based on the two-dimensional feature data extracted in S602.
  • constructing the key frame dictionary refers to clustering the features of the image data of the spatial scene, and expressing the current scene by using a combination of multiple two-dimensional feature data.
  • the construction of the key frame dictionary can use not only two-dimensional feature data, but also semantic information, deep learning descriptors, and so on.
  • the functions of the expression of the dictionary include: First, the expression of the richness of the scene. If the keyframe dictionary in the map data of a spatial scene is rich, the map data representing the spatial scene has rich texture data, geometric features, semantic information, etc. When the richness of the key frame dictionary in the map data of the spatial scene is low, it means that the quality of the map data of the spatial scene is poor, and the user cannot use the map to control the intelligent vehicle, such as automatic parking, which promotes the user's expected management. Second, the keyframe dictionary can be used for position recognition during parking relocation. During relocation initialization, the vehicle needs to find its current location on the map. Through the matching of the key frame dictionary, the approximate current position of the vehicle in the map can be quickly found, and then accurate position estimation can be achieved through the matching of semantic information and two-dimensional feature data.
  • the splicing of the image data collected by looking around the fisheye and the image data collected by the pinhole camera can reflect the complete ground image at a long distance.
  • the deep learning method is used to realize the identification and extraction of parking spaces, and the deep learning method will not be repeated here.
  • obtaining three-dimensional feature points according to the feature matching result is to realize triangulation of feature data, that is, to convert two-dimensional feature data into three-dimensional feature data.
  • the triangulation step is mainly based on the three-point coplanarity assumption, and the three-dimensional coordinates of the object are calculated by using the images obtained by observing the same object at two different positions.
  • the traditional SFM technology needs to estimate the pose between the cameras and the three-dimensional point cloud at the same time.
  • the two-dimensional feature data is matched based on the inter-frame pose, and then the feature data is triangulated quickly based on the matching result.
  • the map metadata refers to the three-dimensional feature points, semantic information and texture data obtained in the above steps. Through the number of 3D feature points, the richness of 3D feature points in space, the richness of semantic features, the richness of scene texture, the environmental illumination and other information, comprehensively judge the construction quality of map data constructed based on the currently obtained map metadata. Whether it meets the quality requirements of the drawing.
  • the specific mapping quality requirements have been described in detail in the foregoing embodiments, and will not be repeated this time.
  • control device will issue a warning message to warn the user to change the spatial scene, better weather, and better time to build the map.
  • the map data constructed based on the map metadata can meet the needs of automatic parking when it is used for automatic parking control, and the quality of the map data will be optimized uniformly.
  • the optimization contents include: the position of 3D feature points, the pose of the camera, and the quality of the semantic layer. For example: optimization of the position, angle, size, category, etc. of semantic elements.
  • the map data is stored in different layers, such as: feature data layer, semantic information layer, navigation data layer, key frame dictionary layer, etc. Different layers can be used to provide data for the automatic parking location function.
  • the map metadata of the image data collected by different cameras will be encoded, and the map metadata of the image data collected by different cameras will be used for matching in different positioning stages. For example, when the vehicle is moving forward, the map metadata of the image data collected by the front-view camera is used for matching and positioning, and when the vehicle is moving backward, the map metadata of the image data collected by the rear-view fisheye camera is used for matching and positioning to achieve more Fast and powerful map building capabilities.
  • the control device When using map data for automatic parking, the control device will increase the weight of the matched map metadata, while the unmatched map metadata will reduce the weight, and add a certain amount of high-quality map metadata to the map to realize the map.
  • the timely update of map metadata ensures the long-term timeliness and high quality of the map.
  • pure visual lightweight map data is constructed, and complex and manual intervention of high-precision map construction is avoided.
  • the constructed map data has extremely low hardware cost requirements, and can quickly realize high-quality map data construction. .
  • the constructed map data can be effectively used for positioning problems in the process of automatic parking. Through a lot of learning, it can realize functions such as exclusive parking space selection, selecting any parking space for parking, parking lot cruise parking and other functions, which can help people very well. It also provides map navigation and real-time path planning for functions such as valet parking and memory parking.
  • control device 700 specifically includes: a memory 701 for storing instructions and a processor 702 for executing the instructions stored in the memory, and the processor 702 is used for specific execution :
  • map metadata includes any one or a combination of three-dimensional feature points, texture data and semantic information
  • map data is generated according to the map metadata; wherein the map data is used to control the movable platform to move within the spatial scene.
  • mapping quality requirements include at least one of the following:
  • the total number of three-dimensional feature points reaches the first number threshold
  • the total amount of texture data reaches the second amount threshold
  • the number of types of texture data reaches the third number threshold
  • the total quantity of semantic information reaches the fourth quantity threshold
  • the number of kinds of semantic information reaches the fifth number threshold.
  • processor 702 is configured to specifically execute:
  • map metadata including:
  • the marked map data includes marking information used to indicate the source of the data.
  • the processor 702 is configured to specifically execute at least one of the following:
  • the marked feature data layer is generated
  • processor 702 is configured to specifically execute:
  • control instructions Generate control instructions according to real-time data and target data; wherein, the control instructions are used to control the movement of the movable platform.
  • processor 702 is configured to specifically execute:
  • the map data from the image sensor installed in front of the movable platform is obtained from the map data as the target data;
  • the map data derived from the image sensor installed behind the movable platform is obtained from the map data as the target data.
  • processor 702 is configured to specifically execute:
  • Control commands are generated according to the position information of the movable platform.
  • processor 702 is configured to specifically execute:
  • processor 702 is configured to specifically execute:
  • the first reliability value is greater than the second reliability value.
  • processor 702 is configured to specifically execute:
  • the image sensor includes a monocular camera and/or a fisheye camera.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

L'invention concerne un procédé de traitement de données et un dispositif. Le procédé consiste : lorsqu'une plateforme mobile se déplace dans une scène d'espace, à commander un capteur d'image, qui est situé sur la plateforme mobile, pour capturer des données d'image multi-trame de la scène d'espace ; à traiter les données d'image multi-trame pour obtenir des métadonnées de carte, les métadonnées de carte comprenant un point caractéristique tridimensionnel et/ou des données de texture et/ou des informations sémantiques ; à déterminer si les métadonnées de carte satisfont ou non une exigence de qualité de cartographie ; et si les métadonnées de carte satisfont l'exigence de qualité de cartographie, à générer des données de carte selon les métadonnées de carte, les données de carte étant utilisées pour commander la plateforme mobile pour se déplacer dans la scène d'espace. Au moyen de cette solution, une exigence de personnalisation d'une plateforme mobile pour des données de carte peut être satisfaite, et la précision des données de carte générées peut également être assurée.
PCT/CN2021/081387 2021-03-17 2021-03-17 Procédé et dispositif de traitement de données WO2022193193A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2021/081387 WO2022193193A1 (fr) 2021-03-17 2021-03-17 Procédé et dispositif de traitement de données
CN202180079742.6A CN116762094A (zh) 2021-03-17 2021-03-17 数据处理方法和设备

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2021/081387 WO2022193193A1 (fr) 2021-03-17 2021-03-17 Procédé et dispositif de traitement de données

Publications (1)

Publication Number Publication Date
WO2022193193A1 true WO2022193193A1 (fr) 2022-09-22

Family

ID=83321621

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/081387 WO2022193193A1 (fr) 2021-03-17 2021-03-17 Procédé et dispositif de traitement de données

Country Status (2)

Country Link
CN (1) CN116762094A (fr)
WO (1) WO2022193193A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117194704A (zh) * 2023-11-07 2023-12-08 航天宏图信息技术股份有限公司 针对部件级实景三维模型属性查询的方法、装置和设备

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111325843A (zh) * 2020-03-09 2020-06-23 北京航空航天大学 一种基于语义逆深度滤波的实时语义地图构建方法
CN111368759A (zh) * 2020-03-09 2020-07-03 河海大学常州校区 基于单目视觉的移动机器人语义地图构建系统
US10794710B1 (en) * 2017-09-08 2020-10-06 Perceptin Shenzhen Limited High-precision multi-layer visual and semantic map by autonomous units
WO2021018690A1 (fr) * 2019-07-31 2021-02-04 Continental Automotive Gmbh Procédé de détermination d'un modèle environnemental d'une scène
CN112348921A (zh) * 2020-11-05 2021-02-09 上海汽车集团股份有限公司 一种基于视觉语义点云的建图方法及系统

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10794710B1 (en) * 2017-09-08 2020-10-06 Perceptin Shenzhen Limited High-precision multi-layer visual and semantic map by autonomous units
WO2021018690A1 (fr) * 2019-07-31 2021-02-04 Continental Automotive Gmbh Procédé de détermination d'un modèle environnemental d'une scène
CN111325843A (zh) * 2020-03-09 2020-06-23 北京航空航天大学 一种基于语义逆深度滤波的实时语义地图构建方法
CN111368759A (zh) * 2020-03-09 2020-07-03 河海大学常州校区 基于单目视觉的移动机器人语义地图构建系统
CN112348921A (zh) * 2020-11-05 2021-02-09 上海汽车集团股份有限公司 一种基于视觉语义点云的建图方法及系统

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117194704A (zh) * 2023-11-07 2023-12-08 航天宏图信息技术股份有限公司 针对部件级实景三维模型属性查询的方法、装置和设备
CN117194704B (zh) * 2023-11-07 2024-02-06 航天宏图信息技术股份有限公司 针对部件级实景三维模型属性查询的方法、装置和设备

Also Published As

Publication number Publication date
CN116762094A (zh) 2023-09-15

Similar Documents

Publication Publication Date Title
KR102125958B1 (ko) 포인트 클라우드 데이터를 융합하기 위한 방법 및 장치
KR102434580B1 (ko) 가상 경로를 디스플레이하는 방법 및 장치
CN110617821B (zh) 定位方法、装置及存储介质
CN109029444B (zh) 一种基于图像匹配和空间定位的室内导航系统及导航方法
CN109887053A (zh) 一种slam地图拼接方法及系统
CN110497901A (zh) 一种基于机器人vslam技术的泊车位自动搜索方法和系统
CN113903011B (zh) 一种适用于室内停车场的语义地图构建及定位方法
CN103424112B (zh) 一种基于激光平面辅助的运动载体视觉导航方法
CN110361027A (zh) 基于单线激光雷达与双目相机数据融合的机器人路径规划方法
KR20180079428A (ko) 자동 로컬리제이션을 위한 장치 및 방법
CN105973236A (zh) 室内定位或导航方法、装置以及地图数据库生成方法
CN111242994B (zh) 一种语义地图构建方法、装置、机器人及存储介质
CN111986506A (zh) 基于多视觉系统的机械式停车位泊车方法
WO2022193508A1 (fr) Procédé et appareil d'optimisation de posture, dispositif électronique, support de stockage lisible par ordinateur, programme d'ordinateur et produit-programme
JP2015148601A (ja) マッピング、位置特定、及び姿勢補正のためのシステム及び方法
CN103941746A (zh) 无人机巡检图像处理系统及方法
CN106959691A (zh) 可移动电子设备和即时定位与地图构建方法
CN208323361U (zh) 一种基于深度视觉的定位装置及机器人
WO2022062480A1 (fr) Procédé de positionnement et appareil de positionnement de dispositif mobile
WO2021017211A1 (fr) Procédé et dispositif de positionnement de véhicule utilisant la détection visuelle, et terminal monté sur un véhicule
WO2024114119A1 (fr) Procédé de fusion de capteurs basé sur un guidage par caméra binoculaire
WO2022193193A1 (fr) Procédé et dispositif de traitement de données
Luo et al. Indoor mapping using low-cost MLS point clouds and architectural skeleton constraints
CN115409910A (zh) 一种语义地图构建方法、视觉定位方法及相关设备
CN112651991B (zh) 视觉定位方法、装置及计算机系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21930786

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 202180079742.6

Country of ref document: CN

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21930786

Country of ref document: EP

Kind code of ref document: A1