CN109116374A - Determine the method, apparatus, equipment and storage medium of obstacle distance - Google Patents

Determine the method, apparatus, equipment and storage medium of obstacle distance Download PDF

Info

Publication number
CN109116374A
CN109116374A CN201710488088.XA CN201710488088A CN109116374A CN 109116374 A CN109116374 A CN 109116374A CN 201710488088 A CN201710488088 A CN 201710488088A CN 109116374 A CN109116374 A CN 109116374A
Authority
CN
China
Prior art keywords
barrier
training
image
semantic information
detection
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201710488088.XA
Other languages
Chinese (zh)
Other versions
CN109116374B (en
Inventor
王睿
孙讯
夏添
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Baidu Online Network Technology Beijing Co Ltd
Beijing Baidu Netcom Science and Technology Co Ltd
Original Assignee
Beijing Baidu Netcom Science and Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Baidu Netcom Science and Technology Co Ltd filed Critical Beijing Baidu Netcom Science and Technology Co Ltd
Priority to CN201710488088.XA priority Critical patent/CN109116374B/en
Publication of CN109116374A publication Critical patent/CN109116374A/en
Application granted granted Critical
Publication of CN109116374B publication Critical patent/CN109116374B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G01MEASURING; TESTING
    • G01SRADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/88Lidar systems specially adapted for specific applications
    • G01S17/93Lidar systems specially adapted for specific applications for anti-collision purposes
    • G01S17/931Lidar systems specially adapted for specific applications for anti-collision purposes of land vehicles
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01SRADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/86Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Electromagnetism (AREA)
  • General Physics & Mathematics (AREA)
  • Traffic Control Systems (AREA)

Abstract

The invention discloses the method, apparatus, equipment and the storage mediums that determine obstacle distance, and wherein method includes: the mode based on deep learning, obtains detection of obstacles model;Visual sensor acquired image is obtained, inputs to detection of obstacles model, the three-dimensional semantic information of the barrier in image exported;Determine barrier at a distance from automatic driving vehicle according to the three-dimensional semantic information of barrier.Using scheme of the present invention, reliability and accuracy of result etc. can be improved.

Description

Determine the method, apparatus, equipment and storage medium of obstacle distance
[technical field]
The present invention relates to Computer Applied Technologies, in particular to determine method, apparatus, equipment and the storage of obstacle distance Medium.
[background technique]
Automatic driving vehicle refers to through various sensors and perceives vehicle-periphery, and obtained according to perceiving Road, vehicle location and obstacle information etc., control the steering and speed of vehicle, to enable the vehicle to reliably and securely exist It is travelled on road.
In the driving process of automatic driving vehicle, need constantly to carry out detection of obstacles and ranging etc., to take The measures such as corresponding avoidance.
Ranging, which refers to, determines the distance between barrier and automatic driving vehicle, in the prior art, main usually following two Kind of mode determines the distance of barrier.
1) mode one
Determine the distance of barrier by the way of Multi-sensor Fusion, the multisensor may include range sensor with And visual sensor, wherein range sensor may include millimetre-wave radar, laser radar, ultrasonic radar etc., visual sensor It may include camera etc..
But this mode can be related to the problem of calibrating between multisensor, realize that complexity, reliability are lower.
2) mode two
Based on the algorithm of two-dimentional (2D) image object detection, the distance of barrier is determined.
But this mode lacks three-dimensional spatial information, and precision is lower, and obstacle distance is remoter, and error is bigger.
[summary of the invention]
In view of this, the present invention provides the method, apparatus, equipment and the storage medium that determine obstacle distance, Neng Gouti The reliability and accuracy of high result.
Specific technical solution is as follows:
A kind of method of determining obstacle distance, comprising:
Mode based on deep learning obtains detection of obstacles model;
Visual sensor acquired image is obtained, inputs to the detection of obstacles model, the figure exported The three-dimensional semantic information of barrier as in;
Determine barrier at a distance from automatic driving vehicle according to the three-dimensional semantic information of the barrier.
According to one preferred embodiment of the present invention, the mode based on deep learning, obtaining detection of obstacles model includes:
Training sample is obtained, includes: the barrier in training image and the training image in each training sample Three-dimensional semantic information;
The detection of obstacles model is obtained according to training sample training.
According to one preferred embodiment of the present invention, the three-dimensional semantic information of the barrier includes:
N number of key point, N are the positive integer greater than one;
The physics size of barrier;
Barrier towards angle.
According to one preferred embodiment of the present invention, the value of N is 8;
N number of key point is respectively to outline 8 vertex of the detection block of barrier.
According to one preferred embodiment of the present invention, the acquisition training sample includes:
Obtain the collecting vehicle each group training data collected for being equipped with visual sensor and laser radar, every group of training It include: that the visual sensor image collected comprising barrier and the corresponding laser radar are acquired in data Point cloud data;
For every group of training data, carry out the following processing respectively:
Using the image in the training data as training image;
The physics size of barrier is determined according to the point cloud data in the training data and towards angle;
N number of key point of the barrier manually marked based on the point cloud data is obtained, and projects to the training figure As upper;
By the training image and project to N number of key point on the training image, the physics size of barrier and Barrier towards angle information as a training sample.
According to one preferred embodiment of the present invention, described that the obstacle is determined according to the three-dimensional semantic information of the barrier Object includes: at a distance from automatic driving vehicle
According to the three-dimensional semantic information of the barrier, barrier is transformed into three-dimensional space from two-dimensional space, according to turn Changing result determines barrier at a distance from automatic driving vehicle.
A kind of device of determining obstacle distance, comprising: pretreatment unit and estimation unit;
The pretreatment unit obtains detection of obstacles model for the mode based on deep learning;
The estimation unit inputs to the detection of obstacles model for obtaining visual sensor acquired image, The three-dimensional semantic information of the barrier in described image exported, and determined according to the three-dimensional semantic information of the barrier Barrier is at a distance from automatic driving vehicle out.
It according to one preferred embodiment of the present invention, include: sample acquisition subelement and model instruction in the pretreatment unit Practice subelement;
The sample acquisition subelement includes: training image and institute for obtaining training sample, in each training sample State the three-dimensional semantic information of the barrier in training image;
The model training subelement, for obtaining the detection of obstacles model according to training sample training.
According to one preferred embodiment of the present invention, the three-dimensional semantic information of the barrier includes:
N number of key point, N are the positive integer greater than one;
The physics size of barrier;
Barrier towards angle.
According to one preferred embodiment of the present invention, the value of N is 8;
N number of key point is respectively to outline 8 vertex of the detection block of barrier.
According to one preferred embodiment of the present invention, the sample acquisition subelement acquisition is equipped with visual sensor and laser The collecting vehicle of radar each group training data collected includes: visual sensor packet collected in every group of training data Image and corresponding laser radar point cloud data collected containing barrier;
For every group of training data, carry out the following processing respectively:
Using the image in the training data as training image;
The physics size of barrier is determined according to the point cloud data in the training data and towards angle;
N number of key point of the barrier manually marked based on the point cloud data is obtained, and projects to the training figure As upper;
By the training image and project to N number of key point on the training image, the physics size of barrier and Barrier towards angle information as a training sample.
According to one preferred embodiment of the present invention, the estimation unit will hinder according to the three-dimensional semantic information of the barrier Hinder object to be transformed into three-dimensional space from two-dimensional space, determines barrier at a distance from automatic driving vehicle according to transformation result.
A kind of computer equipment, including memory, processor and be stored on the memory and can be in the processor The computer program of upper operation, the processor realize method as described above when executing described program.
A kind of computer readable storage medium is stored thereon with computer program, real when described program is executed by processor Now method as described above.
It can be seen that based on above-mentioned introduction using scheme of the present invention, the mode of deep learning can be in advance based on, obtained Detection of obstacles model, in this way, detection of obstacles mould can be inputted to after the image for getting visual sensor acquisition Type, to obtain the three-dimensional semantic information of the barrier in the image of detection of obstacles model output, and then according to barrier Three-dimensional semantic information determines barrier at a distance from automatic driving vehicle, compared to existing way one, due to not being related to pass more Sensor fusion, therefore reliability has obtained very big promotion, and also reduces cost of implementation, in addition, compared to existing way two, The three-dimensional semantic information of barrier is provided, to improve the accuracy of result.
[Detailed description of the invention]
Fig. 1 is the flow chart of the method first embodiment of determining obstacle distance of the present invention.
Fig. 2 is the flow chart of the method second embodiment of determining obstacle distance of the present invention.
Fig. 3 is the composed structure schematic diagram of the Installation practice of determining obstacle distance of the present invention.
Fig. 4 shows the block diagram for being suitable for the exemplary computer system/server 12 for being used to realize embodiment of the present invention.
[specific embodiment]
In order to be clearer and more clear technical solution of the present invention, hereinafter, referring to the drawings and the embodiments, to institute of the present invention The scheme of stating is further described.
Obviously, described embodiments are some of the embodiments of the present invention, instead of all the embodiments.Based on the present invention In embodiment, those skilled in the art's all other embodiment obtained without creative efforts, all Belong to the scope of protection of the invention.
Fig. 1 is the flow chart of the method first embodiment of determining obstacle distance of the present invention, as shown in Figure 1, including Implementation in detail below.
In 101, the mode based on deep learning obtains detection of obstacles model.
To obtain detection of obstacles model, need to obtain training sample first, in each training sample can include: training figure Three-dimensional (3D) semantic information of picture and the barrier in training image can obtain obstacle quality testing according to training sample training later Survey model.
The three-dimensional semantic information of barrier can include: N number of key point, the physics size of barrier and the direction of barrier Angle etc..
Wherein, N is positive integer greater than one, and specific value can be determined according to actual needs, preferably, N can value be 8, Correspondingly, N number of key point may respectively be 8 vertex of the detection block for outlining barrier, that is, the detection block for outlining barrier has altogether Standby 8 vertex, this 8 vertex are key point.Detection block refers to the frame for covering/encasing barrier.
The physics size of barrier can refer to the length etc. of barrier.
In practical applications, it can use collecting vehicle to acquire training data, to generate training sample according to training data This.
For example, mountable in collecting vehicle have visual sensor and laser radar etc., the two is synchronous to carry out data acquisition, adopts Collect multiple groups training data, particular number can be determined according to actual needs.
In every group of training data can include: the visual sensor image collected comprising barrier and corresponding laser Radar point cloud data collected.
For every group of training data, can carry out the following processing respectively:
A, using the image in this group of training data as training image;
B, the physics size of barrier determined according to the point cloud data in this group of training data and towards angle, specifically It is embodied as the prior art;
C, N number of key point of the barrier manually marked based on point cloud data is obtained, and is projected on training image, i.e., It can be by being manually labeled according to N number of key point of the point cloud data to barrier, and it can be after the completion of mark, by the key of mark Point projects on two-dimensional training image;
D, by training image and N number of key point, the physics size of barrier and barrier on training image are projected to Towards angle information as a training sample.
In the manner described above, multiple training samples can be obtained, after getting sufficient amount of training sample, Ji Kegen It trains to obtain detection of obstacles model according to these training samples.
Detection of obstacles model can be neural network model etc..
In 102, visual sensor acquired image is obtained, inputs to detection of obstacles model, the figure exported The three-dimensional semantic information of barrier as in.
Training obtain detection of obstacles model after, can be determined based on detection of obstacles model barrier away from From.
For example, in the driving process of automatic driving vehicle each acquired image of visual sensor can be directed to, respectively It is inputted to detection of obstacles model, so that the three-dimensional for obtaining the barrier in the image of detection of obstacles model output is semantic Information.
As previously mentioned, output barrier three-dimensional semantic information may include N number of key point, the physics size of barrier with And barrier towards angle etc..
In 103, determine barrier at a distance from automatic driving vehicle according to the three-dimensional semantic information of barrier.
It, can be according to the three-dimensional semantic information of barrier, by barrier after getting the three-dimensional semantic information of barrier Be transformed into three-dimensional space from two-dimensional space, be implemented as the prior art, later, can be determined according to transformation result barrier with The distance between automatic driving vehicle.
For example, using the central point of automatic driving vehicle as the coordinate origin in three-dimensional space, then being converted by barrier It, can be it can be easily ascertained that the distance between barrier and automatic driving vehicle after into three-dimensional space.
Based on above-mentioned introduction, Fig. 2 is the flow chart of the method second embodiment of determining obstacle distance of the present invention, such as Shown in Fig. 2, including implementation in detail below.
In 201, the training data of collecting vehicle acquisition is obtained, training sample is generated according to training data.
In each training sample can include: the three-dimensional semantic information of the barrier in training image and training image.
The three-dimensional semantic information of barrier can include: N number of key point, the physics size of barrier and the direction of barrier Angle etc..
In 202, detection of obstacles model is obtained according to training sample training.
After getting sufficient amount of training sample, detection of obstacles can be obtained according to the training of these training samples Model, such as neural network model.
In 203, visual sensor acquired image is obtained, inputs to detection of obstacles model, the figure exported The three-dimensional semantic information of barrier as in.
For example, in the driving process of automatic driving vehicle each acquired image of visual sensor can be directed to, respectively It is inputted to detection of obstacles model, so that the three-dimensional for obtaining the barrier in the image of detection of obstacles model output is semantic Information.
In 204, according to the three-dimensional semantic information of barrier, barrier is transformed into three-dimensional space, root from two-dimensional space Determine barrier at a distance from automatic driving vehicle according to transformation result.
It should be noted that for the various method embodiments described above, for simple description, therefore, it is stated as a series of Combination of actions, but those skilled in the art should understand that, the application is not limited by the described action sequence because According to the application, certain steps can use other sequences or carry out simultaneously.Secondly, those skilled in the art should also know It knows, the embodiments described in the specification are all preferred embodiments, related actions and modules not necessarily the application It is necessary.
In the above-described embodiments, it all emphasizes particularly on different fields to the description of each embodiment, there is no the portion being described in detail in some embodiment Point, it may refer to the associated description of other embodiments.
In short, can be in advance based on the mode of deep learning using scheme described in the various embodiments described above, obtain detection of obstacles Model, in this way, it can be inputted to detection of obstacles model after the image for getting visual sensor acquisition, thus The three-dimensional semantic information of the barrier in the image of detection of obstacles model output is obtained, and then semantic according to the three-dimensional of barrier Information determines barrier at a distance from automatic driving vehicle, compared to existing way one, due to not being related to Multi-sensor Fusion, Therefore reliability has obtained very big promotion, and also reduces cost of implementation, in addition, providing barrier compared to existing way two Hinder the three-dimensional semantic information of object, to improve the accuracy of result.
The introduction about embodiment of the method above, below by way of Installation practice, to scheme of the present invention carry out into One step explanation.
Fig. 3 is the composed structure schematic diagram of the Installation practice of determining obstacle distance of the present invention, as shown in figure 3, It include: pretreatment unit 301 and estimation unit 302.
Pretreatment unit 301 obtains detection of obstacles model for the mode based on deep learning.
Estimation unit 302 inputs to detection of obstacles model, obtains for obtaining visual sensor acquired image The three-dimensional semantic information of barrier in the image of output, and barrier and nothing are determined according to the three-dimensional semantic information of barrier The distance of people's driving vehicle.
To obtain detection of obstacles model, need to obtain training sample first, in each training sample can include: training figure The three-dimensional semantic information of picture and the barrier in training image can obtain detection of obstacles mould according to training sample training later Type.
Correspondingly, as shown in figure 3, may particularly include in pretreatment unit 301: sample acquisition subelement 3011 and model Training subelement 3012.
Sample acquisition subelement 3011 includes: training image and instruction in each training sample for obtaining training sample Practice the three-dimensional semantic information of the barrier in image.
Model training subelement 3012, for obtaining detection of obstacles model according to training sample training.
The three-dimensional semantic information of barrier can include: N number of key point, the physics size of barrier and the direction of barrier Angle etc..
Wherein, N is positive integer greater than one, and specific value can be determined according to actual needs, preferably, N can value be 8, Correspondingly, N number of key point may respectively be 8 vertex of the detection block for outlining barrier.
The physics size of barrier can refer to the length etc. of barrier.
In practical applications, it can use collecting vehicle to acquire training data, to generate training sample according to training data This.
In this way, sample acquisition subelement 3011 can obtain and be equipped with the collecting vehicle of visual sensor and laser radar and be adopted The each group training data of collection includes: the visual sensor image collected comprising barrier in every group of training data and right The laser radar answered point cloud data collected.
Later, for every group of training data, sample acquisition subelement 3011 can carry out the following processing respectively:
Using the image in training data as training image;
The physics size of barrier is determined according to the point cloud data in training data and towards angle;
N number of key point of the barrier manually marked based on point cloud data is obtained, and is projected on training image;
By training image and project to N number of key point on training image, the physics size of barrier and barrier Towards angle information as a training sample.
In the manner described above, multiple training samples can be obtained, after getting sufficient amount of training sample, Ji Keyou Model training subelement 3012 obtains detection of obstacles model, such as neural network model to train according to these training samples.
After training detection of obstacles model, estimation unit 302 can be directed to visual sensor collected figure every time Picture is inputted to detection of obstacles model respectively, to obtain the barrier in the image of detection of obstacles model output Three-dimensional semantic information.
Further, estimation unit 302 can convert barrier from two-dimensional space according to the three-dimensional semantic information of barrier The distance between barrier and automatic driving vehicle are determined to three-dimensional space, and then according to transformation result.
The specific workflow of Fig. 3 shown device embodiment please refers to the respective description in preceding method embodiment, no longer It repeats.
As can be seen that can be in advance based on the mode of deep learning using scheme described in above-described embodiment, obtain obstacle quality testing Model is surveyed, in this way, it can be inputted to detection of obstacles model after the image for getting visual sensor acquisition, from And the three-dimensional semantic information of the barrier in the image of detection of obstacles model output is obtained, and then according to the three-dimensional language of barrier Adopted information determines that barrier at a distance from automatic driving vehicle, compared to existing way one, melts due to not being related to multisensor It closes, therefore reliability has obtained very big promotion, and also reduces cost of implementation, in addition, being provided compared to existing way two The three-dimensional semantic information of barrier, to improve the accuracy of result.
Fig. 4 shows the block diagram for being suitable for the exemplary computer system/server 12 for being used to realize embodiment of the present invention. The computer system/server 12 that Fig. 4 is shown is only an example, should not function and use scope to the embodiment of the present invention Bring any restrictions.
As shown in figure 4, computer system/server 12 is showed in the form of universal computing device.Computer system/service The component of device 12 can include but is not limited to: one or more processor (processing unit) 16, memory 28, connect not homology The bus 18 of system component (including memory 28 and processor 16).
Bus 18 indicates one of a few class bus structures or a variety of, including memory bus or Memory Controller, Peripheral bus, graphics acceleration port, processor or the local bus using any bus structures in a variety of bus structures.It lifts For example, these architectures include but is not limited to industry standard architecture (ISA) bus, microchannel architecture (MAC) Bus, enhanced isa bus, Video Electronics Standards Association (VESA) local bus and peripheral component interconnection (PCI) bus.
Computer system/server 12 typically comprises a variety of computer system readable media.These media, which can be, appoints What usable medium that can be accessed by computer system/server 12, including volatile and non-volatile media, it is moveable and Immovable medium.
Memory 28 may include the computer system readable media of form of volatile memory, such as random access memory Device (RAM) 30 and/or cache memory 32.Computer system/server 12 may further include it is other it is removable/no Movably, volatile/non-volatile computer system storage medium.Only as an example, storage system 34 can be used for reading and writing Immovable, non-volatile magnetic media (Fig. 4 do not show, commonly referred to as " hard disk drive ").Although not shown in fig 4, may be used To provide the disc driver for reading and writing to removable non-volatile magnetic disk (such as " floppy disk "), and it is non-volatile to moving Property CD (such as CD-ROM, DVD-ROM or other optical mediums) read and write CD drive.In these cases, each drive Dynamic device can be connected by one or more data media interfaces with bus 18.Memory 28 may include at least one program Product, the program product have one group of (for example, at least one) program module, these program modules are configured to perform the present invention The function of each embodiment.
Program/utility 40 with one group of (at least one) program module 42 can store in such as memory 28 In, such program module 42 includes --- but being not limited to --- operating system, one or more application program, other programs It may include the realization of network environment in module and program data, each of these examples or certain combination.Program mould Block 42 usually executes function and/or method in embodiment described in the invention.
Computer system/server 12 can also be (such as keyboard, sensing equipment, aobvious with one or more external equipments 14 Show device 24 etc.) communication, it is logical that the equipment interacted with the computer system/server 12 can be also enabled a user to one or more Letter, and/or with the computer system/server 12 any is set with what one or more of the other calculating equipment was communicated Standby (such as network interface card, modem etc.) communicates.This communication can be carried out by input/output (I/O) interface 22.And And computer system/server 12 can also pass through network adapter 20 and one or more network (such as local area network (LAN), wide area network (WAN) and/or public network, such as internet) communication.As shown in figure 4, network adapter 20 passes through bus 18 communicate with other modules of computer system/server 12.It should be understood that although not shown in the drawings, computer can be combined Systems/servers 12 use other hardware and/or software module, including but not limited to: microcode, device driver, at redundancy Manage unit, external disk drive array, RAID system, tape drive and data backup storage system etc..
The program that processor 16 is stored in memory 28 by operation, at various function application and data Reason, such as realize that the method in embodiment illustrated in fig. 1 obtains detection of obstacles model that is, based on the mode of deep learning, it obtains Visual sensor acquired image inputs to detection of obstacles model, the three-dimensional language of the barrier in image exported Adopted information determines barrier at a distance from automatic driving vehicle according to the three-dimensional semantic information of barrier.
Specific implementation please refers to the related description in foregoing embodiments, repeats no more.
The present invention discloses a kind of computer readable storage mediums, are stored thereon with computer program, the program quilt Processor will realize the method in embodiment as shown in Figure 1 when executing.
It can be using any combination of one or more computer-readable media.Computer-readable medium can be calculating Machine readable signal medium or computer readable storage medium.Computer readable storage medium for example can be --- but it is unlimited In system, device or the device of --- electricity, magnetic, optical, electromagnetic, infrared ray or semiconductor, or any above combination.It calculates The more specific example (non exhaustive list) of machine readable storage medium storing program for executing includes: electrical connection with one or more conducting wires, just Taking formula computer disk, hard disk, random access memory (RAM), read-only memory (ROM), erasable type may be programmed read-only storage Device (EPROM or flash memory), optical fiber, portable compact disc read-only memory (CD-ROM), light storage device, magnetic memory device, Or above-mentioned any appropriate combination.In this document, computer readable storage medium can be it is any include or storage journey The tangible medium of sequence, the program can be commanded execution system, device or device use or in connection.
Computer-readable signal media may include in a base band or as carrier wave a part propagate data-signal, Wherein carry computer-readable program code.The data-signal of this propagation can take various forms, including --- but It is not limited to --- electromagnetic signal, optical signal or above-mentioned any appropriate combination.Computer-readable signal media can also be Any computer-readable medium other than computer readable storage medium, which can send, propagate or Transmission is for by the use of instruction execution system, device or device or program in connection.
The program code for including on computer-readable medium can transmit with any suitable medium, including --- but it is unlimited In --- wireless, electric wire, optical cable, RF etc. or above-mentioned any appropriate combination.
The computer for executing operation of the present invention can be write with one or more programming languages or combinations thereof Program code, described program design language include object oriented program language-such as Java, Smalltalk, C++, Further include conventional procedural programming language-such as " C " language or similar programming language.Program code can be with It fully executes, partly execute on the user computer on the user computer, being executed as an independent software package, portion Divide and partially executes or executed on a remote computer or server completely on the remote computer on the user computer.? Be related in the situation of remote computer, remote computer can pass through the network of any kind --- including local area network (LAN) or Wide area network (WAN)-be connected to subscriber computer, or, it may be connected to outer computer (such as mentioned using Internet service It is connected for quotient by internet).
In several embodiments provided by the present invention, it should be understood that disclosed device and method etc. can pass through Other modes are realized.For example, the apparatus embodiments described above are merely exemplary, for example, the division of the unit, Only a kind of logical function partition, there may be another division manner in actual implementation.
The unit as illustrated by the separation member may or may not be physically separated, aobvious as unit The component shown may or may not be physical unit, it can and it is in one place, or may be distributed over multiple In network unit.It can select some or all of unit therein according to the actual needs to realize the mesh of this embodiment scheme 's.
It, can also be in addition, the functional units in various embodiments of the present invention may be integrated into one processing unit It is that each unit physically exists alone, can also be integrated in one unit with two or more units.Above-mentioned integrated list Member both can take the form of hardware realization, can also realize in the form of hardware adds SFU software functional unit.
The above-mentioned integrated unit being realized in the form of SFU software functional unit can store and computer-readable deposit at one In storage media.Above-mentioned SFU software functional unit is stored in a storage medium, including some instructions are used so that a computer It is each that equipment (can be personal computer, server or the network equipment etc.) or processor (processor) execute the present invention The part steps of embodiment the method.And storage medium above-mentioned include: USB flash disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic or disk etc. it is various It can store the medium of program code.
The foregoing is merely illustrative of the preferred embodiments of the present invention, is not intended to limit the invention, all in essence of the invention Within mind and principle, any modification, equivalent substitution, improvement and etc. done be should be included within the scope of the present invention.

Claims (14)

1. a kind of method of determining obstacle distance characterized by comprising
Mode based on deep learning obtains detection of obstacles model;
Visual sensor acquired image is obtained, inputs to the detection of obstacles model, in the described image exported Barrier three-dimensional semantic information;
Determine barrier at a distance from automatic driving vehicle according to the three-dimensional semantic information of the barrier.
2. the method according to claim 1, wherein
The mode based on deep learning, obtaining detection of obstacles model includes:
Training sample is obtained, includes: the three-dimensional of training image and the barrier in the training image in each training sample Semantic information;
The detection of obstacles model is obtained according to training sample training.
3. according to the method described in claim 2, it is characterized in that,
The three-dimensional semantic information of the barrier includes:
N number of key point, N are the positive integer greater than one;
The physics size of barrier;
Barrier towards angle.
4. according to the method described in claim 3, it is characterized in that,
The value of N is 8;
N number of key point is respectively to outline 8 vertex of the detection block of barrier.
5. according to the method described in claim 3, it is characterized in that,
The acquisition training sample includes:
Obtain the collecting vehicle each group training data collected for being equipped with visual sensor and laser radar, every group of training data In include: the visual sensor image collected comprising barrier and corresponding laser radar point collected Cloud data;
For every group of training data, carry out the following processing respectively:
Using the image in the training data as training image;
The physics size of barrier is determined according to the point cloud data in the training data and towards angle;
N number of key point of the barrier manually marked based on the point cloud data is obtained, and is projected on the training image;
By the training image and project to N number of key point, the physics size of barrier and obstacle on the training image Object towards angle information as a training sample.
6. the method according to claim 1, wherein
The three-dimensional semantic information according to the barrier determines that barrier includes: at a distance from automatic driving vehicle
According to the three-dimensional semantic information of the barrier, barrier is transformed into three-dimensional space from two-dimensional space, according to Change-over knot Fruit determines barrier at a distance from automatic driving vehicle.
7. a kind of device of determining obstacle distance characterized by comprising pretreatment unit and estimation unit;
The pretreatment unit obtains detection of obstacles model for the mode based on deep learning;
The estimation unit inputs to the detection of obstacles model, obtains for obtaining visual sensor acquired image The three-dimensional semantic information of barrier in the described image of output, and determine to hinder according to the three-dimensional semantic information of the barrier Hinder object at a distance from automatic driving vehicle.
8. device according to claim 7, which is characterized in that
It include: sample acquisition subelement and model training subelement in the pretreatment unit;
The sample acquisition subelement includes: training image and the instruction for obtaining training sample, in each training sample Practice the three-dimensional semantic information of the barrier in image;
The model training subelement, for obtaining the detection of obstacles model according to training sample training.
9. device according to claim 8, which is characterized in that
The three-dimensional semantic information of the barrier includes:
N number of key point, N are the positive integer greater than one;
The physics size of barrier;
Barrier towards angle.
10. device according to claim 9, which is characterized in that
The value of N is 8;
N number of key point is respectively to outline 8 vertex of the detection block of barrier.
11. device according to claim 9, which is characterized in that
The sample acquisition subelement obtains the collecting vehicle each group instruction collected for being equipped with visual sensor and laser radar Practice data, includes: the visual sensor image collected comprising barrier and corresponding institute in every group of training data State laser radar point cloud data collected;
For every group of training data, carry out the following processing respectively:
Using the image in the training data as training image;
The physics size of barrier is determined according to the point cloud data in the training data and towards angle;
N number of key point of the barrier manually marked based on the point cloud data is obtained, and is projected on the training image;
By the training image and project to N number of key point, the physics size of barrier and obstacle on the training image Object towards angle information as a training sample.
12. device according to claim 7, which is characterized in that
Barrier is transformed into three-dimensional space from two-dimensional space according to the three-dimensional semantic information of the barrier by the estimation unit Between, determine barrier at a distance from automatic driving vehicle according to transformation result.
13. a kind of computer equipment, including memory, processor and it is stored on the memory and can be on the processor The computer program of operation, which is characterized in that the processor is realized when executing described program as any in claim 1~6 Method described in.
14. a kind of computer readable storage medium, is stored thereon with computer program, which is characterized in that described program is processed Such as method according to any one of claims 1 to 6 is realized when device executes.
CN201710488088.XA 2017-06-23 2017-06-23 Method, device and equipment for determining distance of obstacle and storage medium Active CN109116374B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710488088.XA CN109116374B (en) 2017-06-23 2017-06-23 Method, device and equipment for determining distance of obstacle and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710488088.XA CN109116374B (en) 2017-06-23 2017-06-23 Method, device and equipment for determining distance of obstacle and storage medium

Publications (2)

Publication Number Publication Date
CN109116374A true CN109116374A (en) 2019-01-01
CN109116374B CN109116374B (en) 2021-08-17

Family

ID=64732151

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710488088.XA Active CN109116374B (en) 2017-06-23 2017-06-23 Method, device and equipment for determining distance of obstacle and storage medium

Country Status (1)

Country Link
CN (1) CN109116374B (en)

Cited By (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110008941A (en) * 2019-06-05 2019-07-12 长沙智能驾驶研究院有限公司 Drivable region detection method, device, computer equipment and storage medium
CN110488319A (en) * 2019-08-22 2019-11-22 重庆长安汽车股份有限公司 A kind of collision distance calculation method and system merged based on ultrasonic wave and camera
CN110502019A (en) * 2019-09-06 2019-11-26 北京云迹科技有限公司 A kind of barrier-avoiding method and device of Indoor Robot
CN111160172A (en) * 2019-12-19 2020-05-15 深圳佑驾创新科技有限公司 Parking space detection method and device, computer equipment and storage medium
CN111179300A (en) * 2019-12-16 2020-05-19 新奇点企业管理集团有限公司 Method, apparatus, system, device and storage medium for obstacle detection
CN111324115A (en) * 2020-01-23 2020-06-23 北京百度网讯科技有限公司 Obstacle position detection fusion method and device, electronic equipment and storage medium
CN111353453A (en) * 2020-03-06 2020-06-30 北京百度网讯科技有限公司 Obstacle detection method and apparatus for vehicle
WO2020147500A1 (en) * 2019-01-15 2020-07-23 北京百度网讯科技有限公司 Ultrasonic array-based obstacle detection result processing method and system
CN111950501A (en) * 2020-08-21 2020-11-17 东软睿驰汽车技术(沈阳)有限公司 Obstacle detection method and device and electronic equipment
CN112613424A (en) * 2020-12-27 2021-04-06 盛视达(天津)科技有限公司 Rail obstacle detection method, rail obstacle detection device, electronic apparatus, and storage medium
CN113110425A (en) * 2021-03-29 2021-07-13 重庆智行者信息科技有限公司 Target car system based on automatic driving
CN113228043A (en) * 2019-01-22 2021-08-06 深圳市大疆创新科技有限公司 System and method for obstacle detection and association of mobile platform based on neural network
CN113486837A (en) * 2021-07-19 2021-10-08 安徽江淮汽车集团股份有限公司 Automatic driving control method for low-pass obstacle
CN113557524A (en) * 2019-03-19 2021-10-26 罗伯特·博世有限公司 Method for representing a mobile platform environment
CN113692587A (en) * 2019-02-19 2021-11-23 特斯拉公司 Estimating object properties using visual images
CN113777644A (en) * 2021-08-31 2021-12-10 盐城中科高通量计算研究院有限公司 Unmanned positioning method based on weak signal scene
CN113848931A (en) * 2021-10-09 2021-12-28 上海联适导航技术股份有限公司 Agricultural machinery automatic driving obstacle recognition method, system, equipment and storage medium
CN114556449A (en) * 2020-12-17 2022-05-27 深圳市大疆创新科技有限公司 Obstacle detection and re-identification method and device, movable platform and storage medium
CN115607052A (en) * 2022-12-19 2023-01-17 科大讯飞股份有限公司 Cleaning method, device and equipment of robot and cleaning robot
WO2024087456A1 (en) * 2022-10-26 2024-05-02 北京三快在线科技有限公司 Determination of orientation information and autonomous vehicle

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101975951A (en) * 2010-06-09 2011-02-16 北京理工大学 Field environment barrier detection method fusing distance and image information
CN104700414A (en) * 2015-03-23 2015-06-10 华中科技大学 Rapid distance-measuring method for pedestrian on road ahead on the basis of on-board binocular camera
CN105425809A (en) * 2015-12-02 2016-03-23 深圳市易飞行科技有限公司 Obstacle avoiding method and system for unmanned plane
CN105946853A (en) * 2016-04-28 2016-09-21 中山大学 Long-distance automatic parking system and method based on multi-sensor fusion
CN106502267A (en) * 2016-12-06 2017-03-15 上海师范大学 A kind of unmanned plane avoidance system
CN106707293A (en) * 2016-12-01 2017-05-24 百度在线网络技术(北京)有限公司 Obstacle recognition method and device for vehicles
CN106873566A (en) * 2017-03-14 2017-06-20 东北大学 A kind of unmanned logistic car based on deep learning

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101975951A (en) * 2010-06-09 2011-02-16 北京理工大学 Field environment barrier detection method fusing distance and image information
CN104700414A (en) * 2015-03-23 2015-06-10 华中科技大学 Rapid distance-measuring method for pedestrian on road ahead on the basis of on-board binocular camera
CN105425809A (en) * 2015-12-02 2016-03-23 深圳市易飞行科技有限公司 Obstacle avoiding method and system for unmanned plane
CN105946853A (en) * 2016-04-28 2016-09-21 中山大学 Long-distance automatic parking system and method based on multi-sensor fusion
CN106707293A (en) * 2016-12-01 2017-05-24 百度在线网络技术(北京)有限公司 Obstacle recognition method and device for vehicles
CN106502267A (en) * 2016-12-06 2017-03-15 上海师范大学 A kind of unmanned plane avoidance system
CN106873566A (en) * 2017-03-14 2017-06-20 东北大学 A kind of unmanned logistic car based on deep learning

Cited By (29)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020147500A1 (en) * 2019-01-15 2020-07-23 北京百度网讯科技有限公司 Ultrasonic array-based obstacle detection result processing method and system
US11933921B2 (en) 2019-01-15 2024-03-19 Beijing Baidu Netcom Science And Technology Co., Ltd. Method and system for processing obstacle detection result of ultrasonic sensor array
CN113228043A (en) * 2019-01-22 2021-08-06 深圳市大疆创新科技有限公司 System and method for obstacle detection and association of mobile platform based on neural network
CN113692587A (en) * 2019-02-19 2021-11-23 特斯拉公司 Estimating object properties using visual images
US11790664B2 (en) 2019-02-19 2023-10-17 Tesla, Inc. Estimating object properties using visual image data
CN113557524A (en) * 2019-03-19 2021-10-26 罗伯特·博世有限公司 Method for representing a mobile platform environment
CN110008941A (en) * 2019-06-05 2019-07-12 长沙智能驾驶研究院有限公司 Drivable region detection method, device, computer equipment and storage medium
CN110488319A (en) * 2019-08-22 2019-11-22 重庆长安汽车股份有限公司 A kind of collision distance calculation method and system merged based on ultrasonic wave and camera
CN110488319B (en) * 2019-08-22 2023-04-07 重庆长安汽车股份有限公司 Ultrasonic wave and camera fusion-based collision distance calculation method and system
CN110502019A (en) * 2019-09-06 2019-11-26 北京云迹科技有限公司 A kind of barrier-avoiding method and device of Indoor Robot
CN111179300A (en) * 2019-12-16 2020-05-19 新奇点企业管理集团有限公司 Method, apparatus, system, device and storage medium for obstacle detection
CN111160172A (en) * 2019-12-19 2020-05-15 深圳佑驾创新科技有限公司 Parking space detection method and device, computer equipment and storage medium
CN111160172B (en) * 2019-12-19 2024-04-16 武汉佑驾创新科技有限公司 Parking space detection method, device, computer equipment and storage medium
CN111324115B (en) * 2020-01-23 2023-09-19 北京百度网讯科技有限公司 Obstacle position detection fusion method, obstacle position detection fusion device, electronic equipment and storage medium
CN111324115A (en) * 2020-01-23 2020-06-23 北京百度网讯科技有限公司 Obstacle position detection fusion method and device, electronic equipment and storage medium
CN111353453B (en) * 2020-03-06 2023-08-25 北京百度网讯科技有限公司 Obstacle detection method and device for vehicle
CN111353453A (en) * 2020-03-06 2020-06-30 北京百度网讯科技有限公司 Obstacle detection method and apparatus for vehicle
CN111950501A (en) * 2020-08-21 2020-11-17 东软睿驰汽车技术(沈阳)有限公司 Obstacle detection method and device and electronic equipment
CN111950501B (en) * 2020-08-21 2024-05-03 东软睿驰汽车技术(沈阳)有限公司 Obstacle detection method and device and electronic equipment
CN114556449A (en) * 2020-12-17 2022-05-27 深圳市大疆创新科技有限公司 Obstacle detection and re-identification method and device, movable platform and storage medium
CN112613424A (en) * 2020-12-27 2021-04-06 盛视达(天津)科技有限公司 Rail obstacle detection method, rail obstacle detection device, electronic apparatus, and storage medium
CN113110425A (en) * 2021-03-29 2021-07-13 重庆智行者信息科技有限公司 Target car system based on automatic driving
CN113486837A (en) * 2021-07-19 2021-10-08 安徽江淮汽车集团股份有限公司 Automatic driving control method for low-pass obstacle
CN113777644B (en) * 2021-08-31 2023-06-02 盐城中科高通量计算研究院有限公司 Unmanned positioning method based on weak signal scene
CN113777644A (en) * 2021-08-31 2021-12-10 盐城中科高通量计算研究院有限公司 Unmanned positioning method based on weak signal scene
WO2023056789A1 (en) * 2021-10-09 2023-04-13 上海联适导航技术股份有限公司 Obstacle identification method and system for automatic driving of agricultural machine, device, and storage medium
CN113848931A (en) * 2021-10-09 2021-12-28 上海联适导航技术股份有限公司 Agricultural machinery automatic driving obstacle recognition method, system, equipment and storage medium
WO2024087456A1 (en) * 2022-10-26 2024-05-02 北京三快在线科技有限公司 Determination of orientation information and autonomous vehicle
CN115607052A (en) * 2022-12-19 2023-01-17 科大讯飞股份有限公司 Cleaning method, device and equipment of robot and cleaning robot

Also Published As

Publication number Publication date
CN109116374B (en) 2021-08-17

Similar Documents

Publication Publication Date Title
CN109116374A (en) Determine the method, apparatus, equipment and storage medium of obstacle distance
CN109271944B (en) Obstacle detection method, obstacle detection device, electronic apparatus, vehicle, and storage medium
CN109084746B (en) Monocular mode for autonomous platform guidance system with auxiliary sensor
EP3627180B1 (en) Sensor calibration method and device, computer device, medium, and vehicle
US11379699B2 (en) Object detection method and apparatus for object detection
KR102126724B1 (en) Method and apparatus for restoring point cloud data
CN111079619B (en) Method and apparatus for detecting target object in image
CN109059902A (en) Relative pose determines method, apparatus, equipment and medium
CN109101861A (en) Obstacle identity recognition methods, device, equipment and storage medium
CN109298629B (en) System and method for guiding mobile platform in non-mapped region
CN109313810A (en) System and method for being surveyed and drawn to environment
CN108629231A (en) Obstacle detection method, device, equipment and storage medium
CN108734058B (en) Obstacle type identification method, device, equipment and storage medium
CN109214348A (en) A kind of obstacle detection method, device, equipment and storage medium
CN109145680A (en) A kind of method, apparatus, equipment and computer storage medium obtaining obstacle information
CN109117691A (en) Drivable region detection method, device, equipment and storage medium
CN109145677A (en) Obstacle detection method, device, equipment and storage medium
CN108352056A (en) System and method for correcting wrong depth information
CN109344804A (en) A kind of recognition methods of laser point cloud data, device, equipment and medium
CN109633688A (en) A kind of laser radar obstacle recognition method and device
CN109118532B (en) Visual field depth estimation method, device, equipment and storage medium
CN110378966A (en) Camera extrinsic scaling method, device, computer equipment and storage medium
US12073575B2 (en) Object-centric three-dimensional auto labeling of point cloud data
CN110349212B (en) Optimization method and device for instant positioning and map construction, medium and electronic equipment
CN111047634B (en) Scene depth determination method, device, equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant