CN109116374B - Method, device and equipment for determining distance of obstacle and storage medium - Google Patents
Method, device and equipment for determining distance of obstacle and storage medium Download PDFInfo
- Publication number
- CN109116374B CN109116374B CN201710488088.XA CN201710488088A CN109116374B CN 109116374 B CN109116374 B CN 109116374B CN 201710488088 A CN201710488088 A CN 201710488088A CN 109116374 B CN109116374 B CN 109116374B
- Authority
- CN
- China
- Prior art keywords
- obstacle
- training
- image
- semantic information
- key points
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/88—Lidar systems specially adapted for specific applications
- G01S17/93—Lidar systems specially adapted for specific applications for anti-collision purposes
- G01S17/931—Lidar systems specially adapted for specific applications for anti-collision purposes of land vehicles
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/86—Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
Abstract
The invention discloses a method, a device, equipment and a storage medium for determining the distance of an obstacle, wherein the method comprises the following steps: obtaining an obstacle detection model based on a deep learning mode; acquiring an image acquired by a visual sensor, and inputting the image to an obstacle detection model to obtain three-dimensional semantic information of an obstacle in the output image; and determining the distance between the obstacle and the unmanned vehicle according to the three-dimensional semantic information of the obstacle. By applying the scheme of the invention, the reliability, the accuracy and the like of the result can be improved.
Description
[ technical field ] A method for producing a semiconductor device
The present invention relates to computer application technologies, and in particular, to a method, an apparatus, a device, and a storage medium for determining a distance to an obstacle.
[ background of the invention ]
The unmanned vehicle senses the surroundings of the vehicle through various sensors, and controls the steering and speed of the vehicle according to the sensed road, vehicle position, obstacle information, and the like, so that the vehicle can safely and reliably travel on the road.
In the driving process of the unmanned vehicle, obstacle detection, distance measurement and the like need to be continuously carried out so as to take corresponding measures such as obstacle avoidance and the like.
Ranging refers to determining the distance between an obstacle and an unmanned vehicle, and in the prior art, the distance between obstacles is generally determined in the following two ways.
1) In a first mode
The distance of the obstacle is determined by adopting a multi-sensor fusion mode, the multi-sensor can comprise a distance sensor and a vision sensor, wherein the distance sensor can comprise a millimeter wave radar, a laser radar, an ultrasonic radar and the like, and the vision sensor can comprise a camera and the like.
However, this method involves calibration problems among multiple sensors, and is complex to implement and low in reliability.
2) Mode two
The distance of the obstacle is determined based on an algorithm for two-dimensional (2D) image object detection.
However, the method lacks three-dimensional space information, the precision is low, and the error is larger when the distance of the obstacle is farther.
[ summary of the invention ]
In view of the above, the present invention provides a method, an apparatus, a device and a storage medium for determining an obstacle distance, which can improve the reliability and accuracy of the result.
The specific technical scheme is as follows:
a method of determining an obstacle distance, comprising:
obtaining an obstacle detection model based on a deep learning mode;
acquiring an image acquired by a visual sensor, inputting the image to the obstacle detection model, and obtaining three-dimensional semantic information of an obstacle in the output image;
and determining the distance between the obstacle and the unmanned vehicle according to the three-dimensional semantic information of the obstacle.
According to a preferred embodiment of the present invention, the obtaining the obstacle detection model based on the deep learning method includes:
obtaining training samples, each training sample comprising: training images and three-dimensional semantic information of obstacles in the training images;
and training according to the training sample to obtain the obstacle detection model.
According to a preferred embodiment of the present invention, the three-dimensional semantic information of the obstacle includes:
n key points, wherein N is a positive integer greater than one;
the physical size of the obstacle;
the orientation angle of the obstacle.
According to a preferred embodiment of the present invention, N takes the value of 8;
the N key points are respectively 8 vertexes of a detection frame for framing the obstacle.
According to a preferred embodiment of the present invention, the obtaining of the training samples includes:
acquiring each group of training data acquired by an acquisition vehicle provided with a vision sensor and a laser radar, wherein each group of training data comprises: the image which is collected by the vision sensor and contains an obstacle and the corresponding point cloud data collected by the laser radar;
aiming at each group of training data, the following processing is respectively carried out:
taking the image in the training data as a training image;
determining the physical size and the orientation angle of the obstacle according to the point cloud data in the training data;
acquiring N key points of an obstacle manually marked on the basis of the point cloud data, and projecting the N key points to the training image;
and taking the training image, the N key points projected on the training image, the physical size of the obstacle and the orientation angle information of the obstacle as a training sample.
According to a preferred embodiment of the present invention, the determining the distance between the obstacle and the unmanned vehicle according to the three-dimensional semantic information of the obstacle includes:
and converting the obstacle from a two-dimensional space to a three-dimensional space according to the three-dimensional semantic information of the obstacle, and determining the distance between the obstacle and the unmanned vehicle according to the conversion result.
An apparatus for determining an obstacle distance, comprising: a preprocessing unit and an estimation unit;
the preprocessing unit is used for obtaining an obstacle detection model based on a deep learning mode;
the estimation unit is used for acquiring an image acquired by the vision sensor, inputting the image to the obstacle detection model, obtaining the three-dimensional semantic information of the obstacle in the output image, and determining the distance between the obstacle and the unmanned vehicle according to the three-dimensional semantic information of the obstacle.
According to a preferred embodiment of the present invention, the preprocessing unit comprises: a sample acquisition subunit and a model training subunit;
the sample obtaining subunit is configured to obtain training samples, where each training sample includes: training images and three-dimensional semantic information of obstacles in the training images;
and the model training subunit is used for obtaining the obstacle detection model according to the training of the training sample.
According to a preferred embodiment of the present invention, the three-dimensional semantic information of the obstacle includes:
n key points, wherein N is a positive integer greater than one;
the physical size of the obstacle;
the orientation angle of the obstacle.
According to a preferred embodiment of the present invention, N takes the value of 8;
the N key points are respectively 8 vertexes of a detection frame for framing the obstacle.
According to a preferred embodiment of the present invention, the sample acquiring subunit acquires each set of training data acquired by an acquisition vehicle on which a vision sensor and a laser radar are installed, and each set of training data includes: the image which is collected by the vision sensor and contains an obstacle and the corresponding point cloud data collected by the laser radar;
aiming at each group of training data, the following processing is respectively carried out:
taking the image in the training data as a training image;
determining the physical size and the orientation angle of the obstacle according to the point cloud data in the training data;
acquiring N key points of an obstacle manually marked on the basis of the point cloud data, and projecting the N key points to the training image;
and taking the training image, the N key points projected on the training image, the physical size of the obstacle and the orientation angle information of the obstacle as a training sample.
According to a preferred embodiment of the present invention, the estimation unit converts the obstacle from a two-dimensional space to a three-dimensional space according to the three-dimensional semantic information of the obstacle, and determines the distance between the obstacle and the unmanned vehicle according to the conversion result.
A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the method as described above when executing the program.
A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the method as set forth above.
Based on the introduction, the obstacle detection model can be obtained in advance based on a deep learning mode by adopting the scheme of the invention, so that the image acquired by the visual sensor can be input to the obstacle detection model after being acquired, the three-dimensional semantic information of the obstacle in the image output by the obstacle detection model can be obtained, and the distance between the obstacle and the unmanned vehicle can be further determined according to the three-dimensional semantic information of the obstacle.
[ description of the drawings ]
Fig. 1 is a flowchart of a first embodiment of a method for determining an obstacle distance according to the present invention.
Fig. 2 is a flowchart of a method for determining an obstacle distance according to a second embodiment of the present invention.
Fig. 3 is a schematic structural diagram of the device for determining the distance between obstacles according to the embodiment of the present invention.
FIG. 4 illustrates a block diagram of an exemplary computer system/server 12 suitable for use in implementing embodiments of the present invention.
[ detailed description ] embodiments
In order to make the technical solution of the present invention clearer and more obvious, the solution of the present invention is further described below by referring to the drawings and examples.
It is to be understood that the embodiments described are only a few embodiments of the present invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Fig. 1 is a flowchart of a first embodiment of a method for determining an obstacle distance according to the present invention, as shown in fig. 1, including the following specific implementation manners.
In 101, an obstacle detection model is obtained based on a deep learning method.
To obtain the obstacle detection model, training samples are first obtained, and each training sample may include: and training the training images and three-dimensional (3D) semantic information of the obstacles in the training images to obtain an obstacle detection model according to the training samples.
The three-dimensional semantic information of the obstacle may include: n key points, the physical size of the obstacle, the orientation angle of the obstacle, and the like.
N is a positive integer greater than one, and the specific value may be determined according to actual needs, and preferably, N may be 8, and accordingly, the N key points may be respectively 8 vertices of the detection frame framing the obstacle, that is, the detection frame framing the obstacle has 8 vertices, and the 8 vertices are the key points. The detection frame, namely the frame covering/wrapping the obstacle.
The physical size of the obstacle may refer to the length, width, height, etc. of the obstacle.
In practical applications, the collection vehicle may be utilized to collect training data, thereby generating training samples from the training data.
For example, the collection vehicle can be provided with a vision sensor, a laser radar and the like, the vision sensor and the laser radar can synchronously acquire data, and can acquire multiple groups of training data, wherein the specific number can be determined according to actual needs.
Each set of training data may include: the image which is collected by the vision sensor and contains the obstacle and the corresponding point cloud data collected by the laser radar.
For each set of training data, the following processes can be performed:
A. taking the images in the set of training data as training images;
B. determining the physical size and the orientation angle of the obstacle according to the point cloud data in the set of training data, and concretely realizing the prior art;
C. acquiring N key points of the barrier marked manually based on the point cloud data, and projecting the key points to a training image, namely marking the N key points of the barrier manually according to the point cloud data, and projecting the marked key points to a two-dimensional training image after marking is finished;
D. and taking the training image, the N key points projected on the training image, the physical size of the obstacle and the orientation angle information of the obstacle as a training sample.
According to the method, a plurality of training samples can be obtained, and after a sufficient number of training samples are obtained, the obstacle detection model can be obtained through training according to the training samples.
The obstacle detection model may be a neural network model or the like.
At 102, an image acquired by a vision sensor is acquired and input to an obstacle detection model, and three-dimensional semantic information of an obstacle in the output image is obtained.
After the obstacle detection model is trained, the distance to the obstacle may be determined based on the obstacle detection model.
For example, in the driving process of the unmanned vehicle, the images acquired by the vision sensor each time can be respectively input to the obstacle detection model, so that the three-dimensional semantic information of the obstacle in the image output by the obstacle detection model can be obtained.
As described above, the output three-dimensional semantic information of the obstacle may include N key points, a physical size of the obstacle, an orientation angle of the obstacle, and the like.
In 103, the distance between the obstacle and the unmanned vehicle is determined according to the three-dimensional semantic information of the obstacle.
After the three-dimensional semantic information of the obstacle is acquired, the obstacle can be converted from a two-dimensional space to a three-dimensional space according to the three-dimensional semantic information of the obstacle, the prior art is concretely implemented, and then the distance between the obstacle and the unmanned vehicle can be determined according to the conversion result.
For example, with the center point of the unmanned vehicle as the origin of coordinates in the three-dimensional space, the distance between the obstacle and the unmanned vehicle can be easily determined after the obstacle is converted into the three-dimensional space.
Based on the above description, fig. 2 is a flowchart of a second embodiment of the method for determining an obstacle distance according to the present invention, as shown in fig. 2, including the following specific implementation manners.
In 201, training data collected by a collection vehicle is acquired, and a training sample is generated according to the training data.
Each training sample may include: training images and three-dimensional semantic information of obstacles in the training images.
The three-dimensional semantic information of the obstacle may include: n key points, the physical size of the obstacle, the orientation angle of the obstacle, and the like.
In 202, an obstacle detection model is trained from the training samples.
After a sufficient number of training samples are obtained, an obstacle detection model, such as a neural network model, can be obtained by training according to the training samples.
At 203, the image collected by the vision sensor is acquired and input to the obstacle detection model, and the three-dimensional semantic information of the obstacle in the output image is obtained.
For example, in the driving process of the unmanned vehicle, the images acquired by the vision sensor each time can be respectively input to the obstacle detection model, so that the three-dimensional semantic information of the obstacle in the image output by the obstacle detection model can be obtained.
At 204, the obstacle is converted from a two-dimensional space to a three-dimensional space according to the three-dimensional semantic information of the obstacle, and the distance between the obstacle and the unmanned vehicle is determined according to the conversion result.
It should be noted that, for simplicity of description, the above-mentioned method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present application is not limited by the order of acts described, as some steps may occur in other orders or concurrently depending on the application. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required in this application.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
In a word, by adopting the solutions of the embodiments, the obstacle detection model can be obtained in advance based on a deep learning manner, so that after the image acquired by the visual sensor is obtained, the image can be input to the obstacle detection model, thereby obtaining the three-dimensional semantic information of the obstacle in the image output by the obstacle detection model, and further determining the distance between the obstacle and the unmanned vehicle according to the three-dimensional semantic information of the obstacle.
The above is a description of method embodiments, and the embodiments of the present invention are further described below by way of apparatus embodiments.
Fig. 3 is a schematic structural diagram of an embodiment of the apparatus for determining an obstacle distance according to the present invention, as shown in fig. 3, including: a preprocessing unit 301 and an estimation unit 302.
The preprocessing unit 301 is configured to obtain an obstacle detection model based on a deep learning manner.
The estimating unit 302 is configured to acquire an image acquired by the vision sensor, input the image to the obstacle detection model, obtain three-dimensional semantic information of an obstacle in the output image, and determine a distance between the obstacle and the unmanned vehicle according to the three-dimensional semantic information of the obstacle.
To obtain the obstacle detection model, training samples are first obtained, and each training sample may include: and training the training images and the three-dimensional semantic information of the obstacles in the training images to obtain an obstacle detection model according to the training samples.
Correspondingly, as shown in fig. 3, the preprocessing unit 301 may specifically include: a sample acquisition subunit 3011 and a model training subunit 3012.
A sample obtaining subunit 3011, configured to obtain training samples, where each training sample includes: training images and three-dimensional semantic information of obstacles in the training images.
And the model training subunit 3012 is configured to obtain an obstacle detection model according to training of the training samples.
The three-dimensional semantic information of the obstacle may include: n key points, the physical size of the obstacle, the orientation angle of the obstacle, and the like.
N is a positive integer greater than one, and the specific value may be determined according to actual needs, and preferably, N may be 8, and accordingly, the N key points may be 8 vertices of the detection frame framing the obstacle, respectively.
The physical size of the obstacle may refer to the length, width, height, etc. of the obstacle.
In practical applications, the collection vehicle may be utilized to collect training data, thereby generating training samples from the training data.
Like this, each group's training data that collection car that sample acquireed subunit 3011 can acquire to install vision sensor and lidar was gathered, includes in every group's training data: the image which is collected by the vision sensor and contains the obstacle and the corresponding point cloud data collected by the laser radar.
Then, for each set of training data, the sample obtaining subunit 3011 may perform the following processing:
taking the images in the training data as training images;
determining the physical size and the orientation angle of the obstacle according to the point cloud data in the training data;
acquiring N key points of an obstacle marked manually based on point cloud data, and projecting the key points to a training image;
and taking the training image, the N key points projected on the training image, the physical size of the obstacle and the orientation angle information of the obstacle as a training sample.
In this way, a plurality of training samples may be obtained, and after a sufficient number of training samples are obtained, the model training subunit 3012 may train to obtain an obstacle detection model, such as a neural network model, according to the training samples.
After the obstacle detection model is trained, the estimation unit 302 may input the images acquired by the vision sensor each time to the obstacle detection model, so as to obtain three-dimensional semantic information of the obstacle in the image output by the obstacle detection model.
Further, the estimation unit 302 may convert the obstacle from a two-dimensional space to a three-dimensional space according to the three-dimensional semantic information of the obstacle, and then determine the distance between the obstacle and the unmanned vehicle according to the conversion result.
For a specific work flow of the apparatus embodiment shown in fig. 3, please refer to the corresponding description in the foregoing method embodiment, which is not repeated.
It can be seen that, according to the scheme of the embodiment, the obstacle detection model can be obtained in advance based on a deep learning mode, and therefore after the image acquired by the visual sensor is obtained, the image can be input to the obstacle detection model, the three-dimensional semantic information of the obstacle in the image output by the obstacle detection model is obtained, and further the distance between the obstacle and the unmanned vehicle is determined according to the three-dimensional semantic information of the obstacle.
FIG. 4 illustrates a block diagram of an exemplary computer system/server 12 suitable for use in implementing embodiments of the present invention. The computer system/server 12 shown in FIG. 4 is only one example and should not be taken to limit the scope of use or functionality of embodiments of the present invention.
As shown in FIG. 4, computer system/server 12 is in the form of a general purpose computing device. The components of computer system/server 12 may include, but are not limited to: one or more processors (processing units) 16, a memory 28, and a bus 18 that connects the various system components, including the memory 28 and the processors 16.
Computer system/server 12 typically includes a variety of computer system readable media. Such media may be any available media that is accessible by computer system/server 12 and includes both volatile and nonvolatile media, removable and non-removable media.
The memory 28 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM)30 and/or cache memory 32. The computer system/server 12 may further include other removable/non-removable, volatile/nonvolatile computer system storage media. By way of example only, storage system 34 may be used to read from and write to non-removable, nonvolatile magnetic media (not shown in FIG. 4, and commonly referred to as a "hard drive"). Although not shown in FIG. 4, a magnetic disk drive for reading from and writing to a removable, nonvolatile magnetic disk (e.g., a "floppy disk") and an optical disk drive for reading from or writing to a removable, nonvolatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 by one or more data media interfaces. Memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of embodiments of the invention.
A program/utility 40 having a set (at least one) of program modules 42 may be stored, for example, in memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which examples or some combination thereof may comprise an implementation of a network environment. Program modules 42 generally carry out the functions and/or methodologies of the described embodiments of the invention.
The computer system/server 12 may also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), with one or more devices that enable a user to interact with the computer system/server 12, and/or with any devices (e.g., network card, modem, etc.) that enable the computer system/server 12 to communicate with one or more other computing devices. Such communication may be through an input/output (I/O) interface 22. Also, the computer system/server 12 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN) and/or a public network, such as the Internet) via the network adapter 20. As shown in FIG. 4, network adapter 20 communicates with the other modules of computer system/server 12 via bus 18. It should be appreciated that although not shown in the figures, other hardware and/or software modules may be used in conjunction with the computer system/server 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.
The processor 16 executes programs stored in the memory 28 to perform various functional applications and data processing, for example, to implement the method in the embodiment shown in fig. 1, that is, obtaining an obstacle detection model based on deep learning, acquiring an image acquired by the vision sensor, inputting the image to the obstacle detection model, obtaining three-dimensional semantic information of an obstacle in the output image, and determining a distance between the obstacle and the unmanned vehicle according to the three-dimensional semantic information of the obstacle.
For specific implementation, please refer to the related descriptions in the foregoing embodiments, and further description is omitted.
The invention also discloses a computer-readable storage medium, on which a computer program is stored, which program, when being executed by a processor, will carry out the method as in the embodiment shown in fig. 1.
Any combination of one or more computer-readable media may be employed. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus and method, etc., can be implemented in other ways. For example, the above-described device embodiments are merely illustrative, and for example, the division of the units is only one logical functional division, and other divisions may be realized in practice.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional unit.
The integrated unit implemented in the form of a software functional unit may be stored in a computer readable storage medium. The software functional unit is stored in a storage medium and includes several instructions to enable a computer device (which may be a personal computer, a server, or a network device) or a processor (processor) to execute some steps of the methods according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (12)
1. A method of determining an obstacle distance, comprising:
obtaining an obstacle detection model based on a deep learning mode;
acquiring an image acquired by a visual sensor, inputting the image to the obstacle detection model, and obtaining three-dimensional semantic information of an obstacle in the output image;
determining the distance between the obstacle and the unmanned vehicle according to the three-dimensional semantic information of the obstacle, wherein the method comprises the following steps: and converting the obstacle from a two-dimensional space to a three-dimensional space according to the three-dimensional semantic information of the obstacle, taking the central point of the unmanned vehicle as a coordinate origin in the three-dimensional space, and determining the distance between the obstacle and the unmanned vehicle according to the conversion result.
2. The method of claim 1,
the obtaining of the obstacle detection model based on the deep learning mode includes:
obtaining training samples, each training sample comprising: training images and three-dimensional semantic information of obstacles in the training images;
and training according to the training sample to obtain the obstacle detection model.
3. The method of claim 2,
the three-dimensional semantic information of the obstacle includes:
n key points, wherein N is a positive integer greater than one;
the physical size of the obstacle;
the orientation angle of the obstacle.
4. The method of claim 3,
the value of N is 8;
the N key points are respectively 8 vertexes of a detection frame for framing the obstacle.
5. The method of claim 3,
the obtaining of the training sample comprises:
acquiring each group of training data acquired by an acquisition vehicle provided with a vision sensor and a laser radar, wherein each group of training data comprises: the image which is collected by the vision sensor and contains an obstacle and the corresponding point cloud data collected by the laser radar;
aiming at each group of training data, the following processing is respectively carried out:
taking the image in the training data as a training image;
determining the physical size and the orientation angle of the obstacle according to the point cloud data in the training data;
acquiring N key points of an obstacle manually marked on the basis of the point cloud data, and projecting the N key points to the training image;
and taking the training image, the N key points projected on the training image, the physical size of the obstacle and the orientation angle information of the obstacle as a training sample.
6. An apparatus for determining an obstacle distance, comprising: a preprocessing unit and an estimation unit;
the preprocessing unit is used for obtaining an obstacle detection model based on a deep learning mode;
the estimation unit is configured to acquire an image acquired by a visual sensor, input the image to the obstacle detection model, obtain three-dimensional semantic information of an obstacle in the output image, and determine a distance between the obstacle and the unmanned vehicle according to the three-dimensional semantic information of the obstacle, and includes: and converting the obstacle from a two-dimensional space to a three-dimensional space according to the three-dimensional semantic information of the obstacle, taking the central point of the unmanned vehicle as a coordinate origin in the three-dimensional space, and determining the distance between the obstacle and the unmanned vehicle according to the conversion result.
7. The apparatus of claim 6,
the preprocessing unit comprises: a sample acquisition subunit and a model training subunit;
the sample obtaining subunit is configured to obtain training samples, where each training sample includes: training images and three-dimensional semantic information of obstacles in the training images;
and the model training subunit is used for obtaining the obstacle detection model according to the training of the training sample.
8. The apparatus of claim 7,
the three-dimensional semantic information of the obstacle includes:
n key points, wherein N is a positive integer greater than one;
the physical size of the obstacle;
the orientation angle of the obstacle.
9. The apparatus of claim 8,
the value of N is 8;
the N key points are respectively 8 vertexes of a detection frame for framing the obstacle.
10. The apparatus of claim 8,
the sample acquires the subunit and acquires each group training data that the collection car of installing vision sensor and laser radar gathered, and every group training data includes: the image which is collected by the vision sensor and contains an obstacle and the corresponding point cloud data collected by the laser radar;
aiming at each group of training data, the following processing is respectively carried out:
taking the image in the training data as a training image;
determining the physical size and the orientation angle of the obstacle according to the point cloud data in the training data;
acquiring N key points of an obstacle manually marked on the basis of the point cloud data, and projecting the N key points to the training image;
and taking the training image, the N key points projected on the training image, the physical size of the obstacle and the orientation angle information of the obstacle as a training sample.
11. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor when executing the program implements the method of any one of claims 1 to 5.
12. A computer-readable storage medium, on which a computer program is stored, which program, when being executed by a processor, carries out the method according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710488088.XA CN109116374B (en) | 2017-06-23 | 2017-06-23 | Method, device and equipment for determining distance of obstacle and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710488088.XA CN109116374B (en) | 2017-06-23 | 2017-06-23 | Method, device and equipment for determining distance of obstacle and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109116374A CN109116374A (en) | 2019-01-01 |
| CN109116374B true CN109116374B (en) | 2021-08-17 |
Family
ID=64732151
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710488088.XA Active CN109116374B (en) | 2017-06-23 | 2017-06-23 | Method, device and equipment for determining distance of obstacle and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109116374B (en) |
Families Citing this family (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109870698B (en) * | 2019-01-15 | 2021-12-24 | 阿波罗智能技术(北京)有限公司 | Ultrasonic array obstacle detection result processing method and system |
| US10956755B2 (en) | 2019-02-19 | 2021-03-23 | Tesla, Inc. | Estimating object properties using visual image data |
| EP3712802A1 (en) * | 2019-03-19 | 2020-09-23 | Robert Bosch GmbH | Method for representing an environment of a mobile platform |
| CN110008941B (en) * | 2019-06-05 | 2020-01-17 | 长沙智能驾驶研究院有限公司 | Method and device for detecting travelable area, computer equipment and storage medium |
| CN110488319B (en) * | 2019-08-22 | 2023-04-07 | 重庆长安汽车股份有限公司 | Ultrasonic wave and camera fusion-based collision distance calculation method and system |
| CN110502019A (en) * | 2019-09-06 | 2019-11-26 | 北京云迹科技有限公司 | A kind of barrier-avoiding method and device of Indoor Robot |
| CN111179300A (en) * | 2019-12-16 | 2020-05-19 | 新奇点企业管理集团有限公司 | Method, apparatus, system, device and storage medium for obstacle detection |
| CN111160172B (en) * | 2019-12-19 | 2024-04-16 | 武汉佑驾创新科技有限公司 | Parking space detection method, device, computer equipment and storage medium |
| CN111324115B (en) * | 2020-01-23 | 2023-09-19 | 北京百度网讯科技有限公司 | Obstacle position detection fusion method, obstacle position detection fusion device, electronic equipment and storage medium |
| CN111353453B (en) * | 2020-03-06 | 2023-08-25 | 北京百度网讯科技有限公司 | Obstacle detection method and device for vehicle |
| CN112613424A (en) * | 2020-12-27 | 2021-04-06 | 盛视达(天津)科技有限公司 | Rail obstacle detection method, rail obstacle detection device, electronic apparatus, and storage medium |
| CN113110425A (en) * | 2021-03-29 | 2021-07-13 | 重庆智行者信息科技有限公司 | Target car system based on automatic driving |
| CN113486837B (en) * | 2021-07-19 | 2023-07-18 | 安徽江淮汽车集团股份有限公司 | Automatic driving control method for low-pass obstacle |
| CN113777644B (en) * | 2021-08-31 | 2023-06-02 | 盐城中科高通量计算研究院有限公司 | Unmanned positioning method based on weak signal scene |
| CN113848931B (en) * | 2021-10-09 | 2022-09-27 | 上海联适导航技术股份有限公司 | Agricultural machinery automatic driving obstacle recognition method, system, equipment and storage medium |
| CN115607052A (en) * | 2022-12-19 | 2023-01-17 | 科大讯飞股份有限公司 | Cleaning method, device and equipment of robot and cleaning robot |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101975951B (en) * | 2010-06-09 | 2013-03-20 | 北京理工大学 | Field environment barrier detection method fusing distance and image information |
| CN104700414B (en) * | 2015-03-23 | 2017-10-03 | 华中科技大学 | A kind of road ahead pedestrian's fast ranging method based on vehicle-mounted binocular camera |
| CN105425809B (en) * | 2015-12-02 | 2018-01-23 | 深圳市易飞行科技有限公司 | A kind of unmanned plane barrier-avoiding method and system |
| CN105946853B (en) * | 2016-04-28 | 2018-05-29 | 中山大学 | The system and method for long range automatic parking based on Multi-sensor Fusion |
| CN106707293B (en) * | 2016-12-01 | 2019-10-29 | 百度在线网络技术（北京）有限公司 | Obstacle recognition method and device for vehicle |
| CN106502267B (en) * | 2016-12-06 | 2019-03-29 | 上海师范大学 | A kind of unmanned plane avoidance system |
| CN106873566B (en) * | 2017-03-14 | 2019-01-22 | 东北大学 | A kind of unmanned logistic car based on deep learning |
-
2017
- 2017-06-23 CN CN201710488088.XA patent/CN109116374B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN109116374A (en) | 2019-01-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109116374B (en) | Method, device and equipment for determining distance of obstacle and storage medium | |
| US11042762B2 (en) | Sensor calibration method and device, computer device, medium, and vehicle | |
| US11002840B2 (en) | Multi-sensor calibration method, multi-sensor calibration device, computer device, medium and vehicle | |
| US11379699B2 (en) | Object detection method and apparatus for object detection | |
| CN109271944B (en) | Obstacle detection method, obstacle detection device, electronic apparatus, vehicle, and storage medium | |
| EP3361278B1 (en) | Autonomous vehicle localization based on walsh kernel projection technique | |
| CN109931944B (en) | AR navigation method, AR navigation device, vehicle-side equipment, server side and medium | |
| CN109633688B (en) | Laser radar obstacle identification method and device | |
| CN110163930B (en) | Lane line generation method, device, equipment, system and readable storage medium | |
| CN109214980B (en) | Three-dimensional attitude estimation method, three-dimensional attitude estimation device, three-dimensional attitude estimation equipment and computer storage medium | |
| CN108629231B (en) | Obstacle detection method, apparatus, device and storage medium | |
| US11017244B2 (en) | Obstacle type recognizing method and apparatus, device and storage medium | |
| CN108734058B (en) | Obstacle type identification method, device, equipment and storage medium | |
| CN109931945B (en) | AR navigation method, device, equipment and storage medium | |
| CN110095752B (en) | Positioning method, apparatus, device and medium | |
| CN109059902A (en) | Relative pose determines method, apparatus, equipment and medium | |
| CN109870698B (en) | Ultrasonic array obstacle detection result processing method and system | |
| CN109146938B (en) | Method, device and equipment for calibrating position of dynamic obstacle and storage medium | |
| CN109118532B (en) | Visual field depth estimation method, device, equipment and storage medium | |
| CN108229364B (en) | Building contour generation method and device, computer equipment and storage medium | |
| US11227395B2 (en) | Method and apparatus for determining motion vector field, device, storage medium and vehicle | |
| CN109509236B (en) | Vehicle bounding box generation method and device in unmanned scene and storage medium | |
| CN112650300A (en) | Unmanned aerial vehicle obstacle avoidance method and device | |
| CN113705390B (en) | Positioning method, positioning device, electronic equipment and storage medium | |
| CN109300322B (en) | Guideline drawing method, apparatus, device, and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |