CN115346041A

CN115346041A - Point position marking method, device and equipment based on deep learning and storage medium

Info

Publication number: CN115346041A
Application number: CN202211080626.9A
Authority: CN
Inventors: 郑威; 支涛
Original assignee: Beijing Yunji Technology Co Ltd
Current assignee: Beijing Yunji Technology Co Ltd
Priority date: 2022-09-05
Filing date: 2022-09-05
Publication date: 2022-11-15

Abstract

The application provides a point location marking method, device and equipment based on deep learning and a storage medium. The method comprises the following steps: acquiring a sample data set consisting of point location information corresponding to a point cloud map and a point cloud map, and taking the point location information as a training label of a deep learning model; respectively cleaning the point cloud map and the training labels in the sample data set by using a preset data cleaning rule to obtain a cleaned sample data set; training a pre-configured deep learning model by using the cleaned sample data set to obtain a trained deep learning model, wherein the deep learning model adopts a target detection model; and inputting the point cloud map to be marked into the trained deep learning model, outputting point location information corresponding to the point cloud map to be marked by using the deep learning model, and carrying out point location marking based on the point location information corresponding to the point cloud map to be marked. The method and the device can realize automatic point location marking, and improve the efficiency and the precision of the point location marking.

Description

Point position marking method, device, equipment and storage medium based on deep learning

Technical Field

The present application relates to the field of computer technologies, and in particular, to a point location labeling method, apparatus, device, and storage medium based on deep learning.

Background

In the field of robot positioning navigation, such as indoor robot positioning navigation, a point location is a virtual object existing depending on a map. By carrying out point location marking on the map, the robot can implement strategies such as pose adjustment, environment prejudgment, map switching and the like through the point location. The strategies enable the robot to reach the target point position from the starting point position automatically, and the functions of automatic positioning, navigation, path planning and the like of the robot are achieved.

Currently, a known point location marking mode needs to manually mark a point location at a position corresponding to a map in advance, for example, in map scenes of hotels, hospitals and the like, when a robot arrives at different rooms, the point location needs to be manually marked at the corresponding position in advance, so that the point location inside and outside an elevator and destination point locations such as the rooms need to be manually marked in advance. However, the point marking efficiency is reduced by the artificial point marking mode, the number of point markings is large, the operation is complex, a large amount of manpower and material resources are consumed, the accuracy of the artificial point marking is poor, and the requirements of high accuracy, large scale and rapidness for point marking cannot be met.

Disclosure of Invention

In view of this, embodiments of the present application provide a point location labeling method, apparatus, device and storage medium based on deep learning, so as to solve the problems in the prior art that the point location labeling efficiency is low, the precision is poor, and the point location labeling requirements of high precision, large scale and rapidity cannot be met.

In a first aspect of the embodiments of the present application, a point location labeling method based on deep learning is provided, including: acquiring a sample data set consisting of point location information corresponding to a point cloud map and a point cloud map, and taking the point location information as a training label of a deep learning model; respectively cleaning the point cloud map and the training labels in the sample data set by using a preset data cleaning rule to obtain a cleaned sample data set; training a pre-configured deep learning model by using the cleaned sample data set to obtain a trained deep learning model, wherein the deep learning model adopts a target detection model; and inputting the point cloud map to be marked into the trained deep learning model, outputting point location information corresponding to the point cloud map to be marked by using the deep learning model, and carrying out point location marking based on the point location information corresponding to the point cloud map to be marked.

In a second aspect of the embodiments of the present application, a point location labeling device based on deep learning is provided, including: the acquisition module is configured to acquire a sample data set consisting of point cloud maps and point location information corresponding to the point cloud maps, and the point location information is used as a training label of the deep learning model; the cleaning module is configured to respectively clean the point cloud map and the training labels in the sample data set by using a preset data cleaning rule to obtain a cleaned sample data set; the training module is configured to train a pre-configured deep learning model by using the cleaned sample data set to obtain a trained deep learning model, wherein the deep learning model adopts a target detection model; and the marking module is configured to input the point cloud map to be marked into the trained deep learning model, output point location information corresponding to the point cloud map to be marked by using the deep learning model, and mark point locations based on the point location information corresponding to the point cloud map to be marked.

In a third aspect of the embodiments of the present application, there is provided an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor executes the computer program to implement the steps of the method.

In a fourth aspect of the embodiments of the present application, a computer-readable storage medium is provided, in which a computer program is stored, and the computer program realizes the steps of the above method when being executed by a processor.

The embodiment of the application adopts at least one technical scheme which can achieve the following beneficial effects:

acquiring a sample data set consisting of a point cloud map and point location information corresponding to the point cloud map, and taking the point location information as a training label of a deep learning model; respectively cleaning the point cloud map and the training labels in the sample data set by using a preset data cleaning rule to obtain a cleaned sample data set; training a pre-configured deep learning model by using the cleaned sample data set to obtain a trained deep learning model, wherein the deep learning model adopts a target detection model; and inputting the point cloud map to be marked into the trained deep learning model, outputting point location information corresponding to the point cloud map to be marked by using the deep learning model, and carrying out point location marking based on the point location information corresponding to the point cloud map to be marked. The method and the device have the advantages that the point position is automatically marked in a mode of training the deep learning model, the efficiency and the precision of point position marking are improved, the operation is simple and fast, and the requirements of high precision, large scale and rapidness for point position marking can be met.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed for the embodiments or the prior art descriptions will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.

Fig. 1 is a schematic flowchart of a point labeling method based on deep learning according to an embodiment of the present application;

fig. 2 is a schematic diagram of point location labeling in a point cloud map in an actual application scenario according to an embodiment of the present application;

fig. 3 is a schematic structural diagram of a point location labeling apparatus based on deep learning according to an embodiment of the present application;

fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

In the following description, for purposes of explanation and not limitation, specific details are set forth such as particular system structures, techniques, etc. in order to provide a thorough understanding of the embodiments of the present application. It will be apparent, however, to one skilled in the art that the present application may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.

As described above, for the indoor robot positioning navigation as an example, the point location is a virtual object existing attached to a map. By carrying out point location marking on the map, the robot can implement strategies such as pose adjustment, environment prejudgment, map switching and the like through the point location, and can also realize technical means such as path planning, mobile navigation, obstacle stopping and avoiding and the like among the point locations. The strategies and the technical means enable the robot to reach the target point position from the starting point position automatically, and the functions of automatic positioning, navigation, path planning and the like of the robot are achieved.

In most indoor scenes, the point cloud map generally has a plurality of point locations, the types of the point locations are not completely the same, and different positions and different types of point locations may have different functions and bear different responsibilities. For example, in a multi-floor scene, an elevator is an essential tool for cross-floor transportation, different strategies need to be adopted by the robot in and out of the elevator, and judgment and implementation are reasonable selection by combining point positions with environmental factors. In addition, in a scene such as a hotel and a hospital, when the robot arrives at different rooms, the robot needs to manually mark the corresponding positions in advance, so that the robot needs to manually mark the positions of the points inside and outside the elevator, and the destination points such as the rooms in advance. Moreover, no matter the point location is marked while scanning the map in the point cloud map scanning and constructing process, or the point location marking is uniformly carried out after the map scanning is completed, extra manpower and material resources are required to be consumed.

In view of this, the embodiment of the present application provides an improved point location labeling method based on deep learning, where a sample data set is composed of a large number of point cloud maps and point location information, the point cloud maps and training labels in the sample data set are cleaned, and a cleaned sample data set is generated; the method comprises the steps of training a pre-designed deep learning model by utilizing a cleaned sample data set to obtain a trained deep learning model, taking a point cloud map to be labeled as the input of the trained deep learning model, predicting point location information corresponding to the point cloud map to be labeled by utilizing the trained deep learning model, and labeling point locations based on the predicted point location information. The point location marking method and device have the advantages that automatic point location marking is achieved, point location marking efficiency and precision are improved, operation is simple and fast, and the point location marking requirements of high precision, large scale and rapidness can be met. The technical solution of the present application is described in detail with reference to specific embodiments.

Fig. 1 is a schematic flowchart of a point labeling method based on deep learning according to an embodiment of the present application. The point labeling method based on deep learning of fig. 1 may be performed by a robot or a server. As shown in fig. 1, the point labeling method based on deep learning may specifically include:

s101, acquiring a sample data set consisting of point location information corresponding to a point cloud map and a point cloud map, and taking the point location information as a training label of a deep learning model;

s102, respectively cleaning a point cloud map and a training label in the sample data set by using a preset data cleaning rule to obtain a cleaned sample data set;

s103, training a pre-configured deep learning model by using the cleaned sample data set to obtain a trained deep learning model, wherein the deep learning model adopts a target detection model;

and S104, inputting the point cloud map to be marked into the trained deep learning model, outputting point location information corresponding to the point cloud map to be marked by using the deep learning model, and carrying out point location marking based on the point location information corresponding to the point cloud map to be marked.

Specifically, the point location in the embodiment of the present application refers to a point location marked based on a point cloud map, for example, a corresponding point location is marked at a position where a robot needs to reach, such as a room door, an elevator, and an elevator. Based on the marked point locations, the robot can realize the functions of positioning, navigation, path planning and the like, and the functions are executed by depending on the point locations marked in the point cloud map, for example, the robot can smoothly reach a target position by placing one point location at the door of a room of a hotel.

In some embodiments, obtaining a sample data set composed of point cloud maps and point location information corresponding to the point cloud maps, and using the point location information as a training label of a deep learning model includes: acquiring a point cloud map obtained by scanning an external environment by using a laser radar in a historical working process of the robot, acquiring point location information corresponding to each point location in the point cloud map by using a manual marking mode, and generating a sample data set according to the point cloud map and the point location information; the point cloud map in the sample data set is used as a training sample of the deep learning model, and the point location information is used as a training label of the deep learning model.

Specifically, before the deep learning model is trained, a large number of point cloud maps and point location information are collected, the point cloud maps are used as samples, and the point location information is used as a label; the point cloud map is obtained by scanning an external environment by using a laser radar in the historical working process of the robot; the point location information is obtained by marking the collected point cloud map in a manual marking mode, and the point location information corresponding to each point location in the point cloud map is obtained.

Further, the point location information includes, but is not limited to, the following information: position information of the point location, posture information of the point location, and category information of the point location; in practical application, the position information of the point location refers to a coordinate in a two-dimensional rectangular coordinate system of the point cloud map corresponding to the position of the point location; the posture information of the point location refers to the direction of the point location, namely the direction of the point location in the point cloud map is represented, and the direction can be represented by an angle; the category information of the point location refers to a type to which the point location itself belongs, and the category of the point location is related to a location where the point location is located.

Further, after a large number of point cloud maps and point location information corresponding to the point cloud maps are obtained, a sample data set is generated by using the point cloud maps and the point location information, and each sample (namely, the point cloud map) in the sample data set corresponds to at least one label (namely, the point location information).

In some embodiments, the method for cleaning the point cloud map and the training labels in the sample data set by using a preset data cleaning rule to obtain a cleaned sample data set includes: cleaning the point cloud map with the concentrated sample data by using a preset sample cleaning rule, wherein the sample cleaning rule comprises the step of cleaning the map according to the consistency, the smoothness and the definition of black lines in the point cloud map; cleaning point location information in the sample data set by using a preset label cleaning rule, wherein the label cleaning rule comprises the step of cleaning a label according to the position accuracy and the angle pointing correctness of the point location; and regenerating a new sample data set by using the cleaned point cloud map and the point location information to obtain the cleaned sample data set.

Specifically, after a sample data set is generated, data cleaning needs to be performed on a point cloud map and a training label (i.e., point location information) in the sample data set, and in actual application, a cleaning rule corresponding to the point cloud map is not completely the same as a cleaning rule corresponding to the point location information. The following describes specific contents of the two cleaning rules with reference to specific embodiments, which may specifically include the following contents:

the sample cleaning rule is used for cleaning data of the point cloud map, and the sample cleaning rule comprises the steps of cleaning the map according to the consistency, the smoothness and the definition of black lines in the point cloud map; it should be noted that the point cloud map constructed by using the laser radar includes three pixels with different colors, and the contents represented by the pixels with different colors are also different, for example: the black pixel points are used for representing the boundary of an obstacle, the gray pixel points are used for representing an unknown area (namely, an area which cannot be detected by laser radar), and the white pixel points are used for representing a passage (namely, an area which can be passed by the robot). Therefore, the black lines in the point cloud map are lines formed by black pixel points, and the consistency, smoothness and definition of the black lines are used as screening standards to clean the data of the point cloud map.

The label cleaning rule is used for cleaning data of point location information, and the label cleaning rule comprises the steps of cleaning labels according to the position accuracy and the angle pointing accuracy of the point locations; it should be noted that the position accuracy of a point location refers to whether the point location is marked at an accurate position, that is, whether the position of the point location deviates from the position to be marked; the correctness of the angular direction refers to whether the angular direction of the point location in the point cloud map is normal or not.

Further, after the point cloud map and the point location information are respectively subjected to data cleaning, a new sample data set is regenerated according to the point cloud map and the point location information obtained after the data cleaning. According to the embodiment of the application, through the cleaning of the sample data set, the data which is good in map scanning quality and complete in point location information without errors and leaks is screened out from the original sample data set, and the quality of a new sample data set is improved.

In some embodiments, training the pre-configured deep learning model with the cleaned sample data set comprises: taking the position and the direction of a point location as parameters, establishing a loss function corresponding to the deep learning model according to the parameters, taking the cleaned sample data set as the input of the deep learning model, and training the deep learning model by using training samples and training labels in the cleaned sample data set; wherein, the deep learning model adopts a YOLOv5 target detection model.

Specifically, the deep learning model in the embodiment of the present application uses the YOLOv5 target detection model, and when designing the YOLOv5 target detection model, the position and the direction of the point location are used as parameters, the position of the point location refers to x and y coordinates of the point location in a two-dimensional rectangular coordinate system, and the direction of the point location refers to angle information of the point location, and the two parameters are used to establish a loss function of the YOLOv5 target detection model. In practical applications, besides designing the loss function of the YOLOv5 target detection model, it is also necessary to establish a structure, an optimizer, and various "hyper-parameters" of the YOLOv5 target detection model, which are not described herein again.

It should be noted that "YOLO" is an object detection algorithm, and YOLO redefines object detection as a regression problem. It applies a single Convolutional Neural Network (CNN) to the entire image, divides the image into meshes, and predicts class probabilities and bounding boxes for each mesh. The YOLOv5 target detection model is more complex in network structure compared with other YOLO series models, and meanwhile, a plurality of techniques are used for improving the detection precision and speed of the model in a data enhancement and training strategy.

Further, the YOLOv5 target detection model can be divided into three parts, wherein the first part is a Backbone network (Backbone) and is responsible for feature extraction of targets, and the first part consists of a Focus module, a Bottlen CSP module and an SPP module. The second part is a Neck (hack) network, which is mainly used for enhancing the features extracted by the main network, and the adopted module is a PANet path aggregation structure. The third part is a Head (Head) network, a Head detection network which is the same as a YOLOv3 network model is adopted, the three detection heads respectively carry out 8-time, 16-time and 32-time down-sampling on the original image, and finally, three feature vectors with different sizes are respectively generated and used for detecting targets with different sizes.

In some embodiments, inputting a point cloud map to be labeled into a trained deep learning model, and outputting point location information corresponding to the point cloud map to be labeled by using the deep learning model, including: the method comprises the steps of obtaining a point cloud map to be marked, inputting the point cloud map to be marked into a trained deep learning model, predicting point location information corresponding to the point cloud map to be marked by using the trained deep learning model, and outputting the point location information corresponding to each point location in the point cloud map to be marked.

Specifically, after the deep learning model is built and trained, the obtained point cloud map to be labeled is used as the input of the trained deep learning model, the trained deep learning model automatically predicts the point location information of the point cloud map to be labeled, and outputs the point location information corresponding to each point location in the point cloud map to be labeled.

In practical application, the point location information includes position information, posture information and category information, wherein the position information is used for representing coordinates of the point location in the point cloud map, the posture information is used for representing directions of the point location in the point cloud map, and the category information is used for representing grouping types corresponding to the point location. Therefore, the point location information output by the deep learning model after training in the embodiment of the present application at least includes position information, angle information (i.e., posture information) and category information of different point locations (e.g., an inside point location of an elevator, an outside point location of an elevator, a door point location, etc.).

In some embodiments, performing point location labeling based on point location information corresponding to a point cloud map to be labeled includes: marking each point location in the point cloud map to be marked according to the position information and the attitude information in the point location information corresponding to the point cloud map to be marked so as to display the position and the direction of the point location in the point cloud map to be marked.

Specifically, after point location information corresponding to a point cloud map to be marked is automatically predicted and output by using the trained deep learning model, point location marking can be performed on the map by using position information and posture information corresponding to each point location in the point location information. Next, a point cloud map in an actual scene is taken as an example to describe a process of labeling the point cloud map by using point location information, and fig. 2 is a schematic diagram of point location labeling in the point cloud map in an actual application scene according to an embodiment of the present application. As shown in fig. 2, the point location labeling process of the point cloud map may specifically include:

according to the point location information output by the trained deep learning model, automatically labeling each point location in the point cloud map, for example, in an embodiment, performing point location labeling on the map by using position information and posture information in the point location information, at this time, automatically labeling the position (which can be represented by a dot, a triangle, or the like) and the direction (which can be represented by an arrow) of the point location at each point location in the map; in another embodiment, in addition to point labeling using position information and posture information, point classification information may be labeled, that is, classification information corresponding to each point is added to the point, for example, classification information of the point (inside the elevator, outside the elevator, or the like) may be added to the front in the direction indicated by the arrow.

According to the technical scheme provided by the embodiment of the application, a large amount of point cloud maps and point location information are collected to generate a sample data set, data cleaning is respectively carried out on samples and labels in the sample data set, and a new sample data set is regenerated based on the samples and the labels after the data cleaning; training a pre-designed YOLOv5 target detection model by using a new sample data set, so that the trained YOLOv5 target detection model has the capability of automatically predicting each point location and point location information; inputting a point cloud map needing automatic point location marking into a trained YOLOv5 target detection model, and enabling the YOLOv5 target detection model to automatically output point location information corresponding to each point location; and finally, automatically marking the point cloud map based on the point location information, so that the marked point cloud map not only has the position information of the point location, but also contains the attitude information and the category information of the point location. The embodiment of the application can automatically detect most point positions and realize the automatic marking of the point positions, the efficiency and the precision of point position marking are improved, the operation is simple and fast, and the point position marking requirements of high precision, large scale and rapidness can be met.

The following are embodiments of the apparatus of the present application that may be used to perform embodiments of the method of the present application. For details which are not disclosed in the embodiments of the apparatus of the present application, reference is made to the embodiments of the method of the present application.

Fig. 3 is a schematic structural diagram of a point marking device based on deep learning according to an embodiment of the present application.

As shown in fig. 3, the point labeling device based on deep learning includes:

an obtaining module 301, configured to obtain a sample data set composed of a point cloud map and point location information corresponding to the point cloud map, and use the point location information as a training label of a deep learning model;

a cleaning module 302 configured to respectively clean the point cloud map and the training labels in the sample data set by using a preset data cleaning rule to obtain a cleaned sample data set;

a training module 303, configured to train a pre-configured deep learning model by using the cleaned sample data set to obtain a trained deep learning model, where the deep learning model is a target detection model;

the marking module 304 is configured to input the point cloud map to be marked into the trained deep learning model, output point location information corresponding to the point cloud map to be marked by using the deep learning model, and perform point location marking based on the point location information corresponding to the point cloud map to be marked.

In some embodiments, the obtaining module 301 in fig. 3 obtains a point cloud map obtained by scanning an external environment with a laser radar in a historical working process of a robot, obtains point location information corresponding to each point location in the point cloud map by using a manual marking method, and generates a sample data set according to the point cloud map and the point location information; and taking the point cloud map in the sample data set as a training sample of the deep learning model, and taking the point location information as a training label of the deep learning model.

In some embodiments, the cleaning module 302 of fig. 3 cleans the point cloud map in the sample data set by using a preset sample cleaning rule, wherein the sample cleaning rule includes map cleaning according to a consistency, a smoothness and a clarity of a black line in the point cloud map; cleaning the point location information in the sample data set by using a preset label cleaning rule, wherein the label cleaning rule comprises the step of cleaning a label according to the position accuracy of the point location and the correctness of the angle pointing; and regenerating a new sample data set by using the cleaned point cloud map and the point location information to obtain the cleaned sample data set.

In some embodiments, the training module 303 of fig. 3 uses the position and the direction of the point location as parameters, establishes a loss function corresponding to the deep learning model according to the parameters, uses the cleaned sample data set as an input of the deep learning model, and trains the deep learning model by using the training samples and the training labels in the cleaned sample data set; wherein the deep learning model adopts a YOLOv5 target detection model.

In some embodiments, the labeling module 304 of fig. 3 obtains a point cloud map to be labeled, inputs the point cloud map to be labeled into the trained deep learning model, predicts point location information corresponding to the point cloud map to be labeled by using the trained deep learning model, and outputs the point location information corresponding to each point location in the point cloud map to be labeled.

In some embodiments, the labeling module 304 of fig. 3 labels each point location in the point cloud map to be labeled respectively according to the position information and the posture information in the point location information corresponding to the point cloud map to be labeled, so as to display the position and the direction of the point location in the point cloud map to be labeled.

It should be understood that, the sequence numbers of the steps in the foregoing embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

Fig. 4 is a schematic structural diagram of an electronic device 4 provided in an embodiment of the present application. As shown in fig. 4, the electronic apparatus 4 of this embodiment includes: a processor 401, a memory 402 and a computer program 403 stored in the memory 402 and executable on the processor 401. The steps in the various method embodiments described above are implemented when the processor 401 executes the computer program 403. Alternatively, the processor 401 implements the functions of the respective modules/units in the above-described respective apparatus embodiments when executing the computer program 403.

Illustratively, the computer program 403 may be partitioned into one or more modules/units, which are stored in the memory 402 and executed by the processor 401 to accomplish the present application. One or more modules/units may be a series of computer program instruction segments capable of performing certain functions, which are used to describe the execution of the computer program 403 in the electronic device 4.

The electronic device 4 may be a desktop computer, a notebook, a palm computer, a cloud server, or other electronic devices. The electronic device 4 may include, but is not limited to, a processor 401 and a memory 402. Those skilled in the art will appreciate that fig. 4 is merely an example of the electronic device 4, and does not constitute a limitation of the electronic device 4, and may include more or fewer components than shown, or some of the components may be combined, or different components, e.g., the electronic device may also include an input-output device, a network access device, a bus, etc.

The Processor 401 may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, discrete hardware component, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The storage 402 may be an internal storage unit of the electronic device 4, for example, a hard disk or a memory of the electronic device 4. The memory 402 may also be an external storage device of the electronic device 4, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like provided on the electronic device 4. Further, the memory 402 may also include both internal storage units of the electronic device 4 and external storage devices. The memory 402 is used for storing computer programs and other programs and data required by the electronic device. The memory 402 may also be used to temporarily store data that has been output or is to be output.

It should be clear to those skilled in the art that, for convenience and simplicity of description, the foregoing division of the functional units and modules is only used for illustration, and in practical applications, the above function distribution may be performed by different functional units and modules as needed, that is, the internal structure of the device is divided into different functional units or modules, so as to perform all or part of the above described functions. Each functional unit and module in the embodiments may be integrated in one processing unit, or each unit may exist alone physically, or two or more units are integrated in one unit, and the integrated unit may be implemented in a form of hardware, or in a form of software functional unit. In addition, specific names of the functional units and modules are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working processes of the units and modules in the system may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment.

Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

In the embodiments provided in the present application, it should be understood that the disclosed apparatus/computer device and method may be implemented in other ways. For example, the above-described apparatus/computer device embodiments are merely illustrative, and for example, a division of modules or units, a division of logical functions only, an additional division may be made in actual implementation, multiple units or components may be combined or integrated with another system, or some features may be omitted, or not implemented. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

Units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The integrated modules/units, if implemented in the form of software functional units and sold or used as separate products, may be stored in a computer readable storage medium. Based on such understanding, all or part of the flow in the method of the foregoing embodiments may be implemented by a computer program, which may be stored in a computer readable storage medium and instructs related hardware to implement the steps of the foregoing method embodiments when executed by a processor. The computer program may comprise computer program code which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer readable medium may include: any entity or device capable of carrying computer program code, recording medium, U.S. disk, removable hard disk, magnetic diskette, optical disk, computer Memory, read-Only Memory (ROM), random Access Memory (RAM), electrical carrier wave signal, telecommunications signal, software distribution medium, etc. It should be noted that the computer readable medium may contain suitable additions or additions that may be required in accordance with legislative and patent practices within the jurisdiction, for example, in some jurisdictions, computer readable media may not include electrical carrier signals or telecommunications signals in accordance with legislative and patent practices.

The above embodiments are only used to illustrate the technical solutions of the present application, and not to limit the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not depart from the spirit and scope of the embodiments of the present application, and they should be construed as being included in the present application.

Claims

1. A point location labeling method based on deep learning is characterized by comprising the following steps:

acquiring a sample data set consisting of a point cloud map and point location information corresponding to the point cloud map, and using the point location information as a training label of a deep learning model;

respectively cleaning the point cloud map and the training labels in the sample data set by using a preset data cleaning rule to obtain a cleaned sample data set;

training a pre-configured deep learning model by using the cleaned sample data set to obtain a trained deep learning model, wherein the deep learning model adopts a target detection model;

and inputting the point cloud map to be marked into the trained deep learning model, outputting point location information corresponding to the point cloud map to be marked by using the deep learning model, and carrying out point location marking based on the point location information corresponding to the point cloud map to be marked.

2. The method of claim 1, wherein the obtaining a sample data set composed of a point cloud map and point location information corresponding to the point cloud map, and using the point location information as a training label of a deep learning model comprises:

acquiring a point cloud map obtained by scanning an external environment by using a laser radar in a historical working process of a robot, acquiring point location information corresponding to each point location in the point cloud map by using a manual marking mode, and generating a sample data set according to the point cloud map and the point location information;

and taking the point cloud map in the sample data set as a training sample of the deep learning model, and taking the point location information as a training label of the deep learning model.

3. The method of claim 2, wherein the step of cleaning the point cloud map and the training labels in the sample data set by using a preset data cleaning rule to obtain a cleaned sample data set comprises:

cleaning the point cloud map in the sample data set by using a preset sample cleaning rule, wherein the sample cleaning rule comprises the step of cleaning the map according to the consistency, smoothness and definition of black lines in the point cloud map;

cleaning the point location information in the sample data set by using a preset label cleaning rule, wherein the label cleaning rule comprises the step of cleaning a label according to the position accuracy and the angle pointing correctness of the point location;

and regenerating a new sample data set by using the cleaned point cloud map and the point location information to obtain the cleaned sample data set.

4. The method of claim 2, wherein training a pre-configured deep learning model using the cleaned sample data set comprises:

taking the position and the direction of a point location as parameters, establishing a loss function corresponding to the deep learning model according to the parameters, taking the cleaned sample data set as the input of the deep learning model, and training the deep learning model by using the training samples and the training labels in the cleaned sample data set; wherein the deep learning model adopts a YOLOv5 target detection model.

5. The method of claim 1, wherein the inputting the point cloud map to be labeled into the trained deep learning model, and outputting point location information corresponding to the point cloud map to be labeled by using the deep learning model, comprises:

acquiring a point cloud map to be marked, inputting the point cloud map to be marked into the trained deep learning model, predicting point location information corresponding to the point cloud map to be marked by using the trained deep learning model, and outputting the point location information corresponding to each point location in the point cloud map to be marked.

6. The method of claim 5, wherein the point location marking based on the point location information corresponding to the point cloud map to be marked comprises:

marking each point location in the point cloud map to be marked according to the position information and the posture information in the point location information corresponding to the point cloud map to be marked so as to display the position and the direction of the point location in the point cloud map to be marked.

7. The method according to any one of claims 1 to 6, wherein the point location information includes position information, posture information and category information, wherein the position information is used for representing coordinates of the point location in the point cloud map, the posture information is used for representing directions of the point location in the point cloud map, and the category information is used for representing grouping types corresponding to the point location.

8. The utility model provides a position marking device based on degree of deep learning which characterized in that includes:

the acquisition module is configured to acquire a sample data set consisting of a point cloud map and point location information corresponding to the point cloud map, and the point location information is used as a training label of the deep learning model;

the cleaning module is configured to respectively clean the point cloud map and the training labels in the sample data set by using a preset data cleaning rule to obtain a cleaned sample data set;

the training module is configured to train a pre-configured deep learning model by using the cleaned sample data set to obtain a trained deep learning model, wherein the deep learning model adopts a target detection model;

and the marking module is configured to input the point cloud map to be marked into the trained deep learning model, output point location information corresponding to the point cloud map to be marked by using the deep learning model, and mark points based on the point location information corresponding to the point cloud map to be marked.

9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the method of any one of claims 1 to 7 when the program is executed by the processor.

10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1 to 7.