CN108734058B - Obstacle type identification method, device, equipment and storage medium - Google Patents
Obstacle type identification method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN108734058B CN108734058B CN201710253581.3A CN201710253581A CN108734058B CN 108734058 B CN108734058 B CN 108734058B CN 201710253581 A CN201710253581 A CN 201710253581A CN 108734058 B CN108734058 B CN 108734058B
- Authority
- CN
- China
- Prior art keywords
- dimensional
- point cloud
- cloud data
- obstacle
- view angle
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a method, a device, equipment and a storage medium for identifying types of obstacles, wherein the method comprises the following steps: acquiring three-dimensional point cloud data corresponding to an obstacle to be identified; mapping the three-dimensional point cloud data to a two-dimensional image; and identifying the type of the obstacle through a deep learning algorithm based on the two-dimensional image. By applying the scheme of the invention, the accuracy of the identification result can be improved.
Description
[ technical field ] A method for producing a semiconductor device
The present invention relates to computer application technologies, and in particular, to a method, an apparatus, a device, and a storage medium for obstacle type identification.
[ background of the invention ]
An unmanned vehicle, also called an autonomous vehicle, senses the surroundings of the vehicle through various sensors, and controls the steering and speed of the vehicle according to the sensed road, vehicle position, obstacle information, and the like, so that the vehicle can safely and reliably travel on the road.
The laser radar is an important sensor for sensing a three-dimensional environment of an unmanned vehicle, scans a circle of scene, and returns point cloud data of a three-dimensional space of the scene, namely three-dimensional (3D) point cloud data.
Based on the scanned three-dimensional point cloud data, the detection of the obstacle, the identification of the type of the obstacle and the like can be carried out, so that the unmanned vehicle can carry out obstacle avoidance operation and the like.
In the prior art, the obstacle type is generally identified in a three-dimensional space, and a mature implementation mode is not available, so that the accuracy of an identification result is low.
[ summary of the invention ]
In view of this, the present invention provides a method, an apparatus, a device and a storage medium for identifying an obstacle type, which can improve the accuracy of an identification result.
The specific technical scheme is as follows:
an obstacle type identification method, comprising:
acquiring three-dimensional point cloud data corresponding to an obstacle to be identified;
mapping the three-dimensional point cloud data to a two-dimensional image;
and identifying the type of the obstacle through a deep learning algorithm based on the two-dimensional image.
In accordance with a preferred embodiment of the present invention,
the method further comprises the following steps:
acquiring each obstacle detected from three-dimensional point cloud data obtained by scanning;
respectively taking each detected obstacle as the obstacle to be identified;
the three-dimensional point cloud data is obtained by scanning the surrounding environment of the unmanned vehicle.
In accordance with a preferred embodiment of the present invention,
the two-dimensional image includes: a two-dimensional RGB image.
In accordance with a preferred embodiment of the present invention,
mapping the three-dimensional point cloud data to a two-dimensional RGB image comprises:
mapping the three-dimensional point cloud data to an R channel of a two-dimensional image from a first perspective;
mapping the three-dimensional point cloud data to a G channel of a two-dimensional image from a second perspective;
mapping the three-dimensional point cloud data to a B channel of a two-dimensional image from a third perspective;
and generating the two-dimensional RGB image according to each mapping result.
In accordance with a preferred embodiment of the present invention,
the first viewing angle is one of: a top view angle, a headstock forward view angle and a left side view angle;
the second viewing angle is one of: a top view angle, a headstock forward view angle and a left side view angle;
the third viewing angle is one of: a top view angle, a headstock forward view angle and a left side view angle;
the first, second, and third perspectives are different perspectives.
An obstacle type identification apparatus comprising: the device comprises an acquisition unit, a mapping unit and a classification unit;
the acquisition unit is used for acquiring three-dimensional point cloud data corresponding to the obstacle to be identified and sending the three-dimensional point cloud data to the mapping unit;
the mapping unit is used for mapping the three-dimensional point cloud data to a two-dimensional image and sending the two-dimensional image to the classification unit;
and the classification unit is used for identifying the type of the obstacle through a deep learning algorithm based on the two-dimensional image.
In accordance with a preferred embodiment of the present invention,
the obtaining unit is further configured to obtain,
acquiring each obstacle detected from three-dimensional point cloud data obtained by scanning;
respectively taking each detected obstacle as the obstacle to be identified;
the three-dimensional point cloud data is obtained by scanning the surrounding environment of the unmanned vehicle.
In accordance with a preferred embodiment of the present invention,
the two-dimensional image includes: a two-dimensional RGB image.
In accordance with a preferred embodiment of the present invention,
the mapping unit maps the three-dimensional point cloud data to an R channel of a two-dimensional image from a first visual angle, maps the three-dimensional point cloud data to a G channel of the two-dimensional image from a second visual angle, maps the three-dimensional point cloud data to a B channel of the two-dimensional image from a third visual angle, and generates the two-dimensional RGB image according to each mapping result.
In accordance with a preferred embodiment of the present invention,
the first viewing angle is one of: a top view angle, a headstock forward view angle and a left side view angle;
the second viewing angle is one of: a top view angle, a headstock forward view angle and a left side view angle;
the third viewing angle is one of: a top view angle, a headstock forward view angle and a left side view angle;
the first, second, and third perspectives are different perspectives.
A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the method as described above when executing the program.
A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the method as set forth above.
Based on the introduction, the scheme of the invention is adopted, firstly three-dimensional point cloud data corresponding to the obstacle to be identified is mapped to the two-dimensional image, then based on the obtained two-dimensional image, the type of the obstacle is identified through a deep learning algorithm, namely, the obstacle to be identified is converted into the two-dimensional space from the three-dimensional space, and dimension reduction processing is carried out, and in the field of two-dimensional image identification, the deep learning algorithm is a mature algorithm, so that the accuracy of the identification result is ensured, namely, compared with the prior art, the accuracy of the identification result is improved.
[ description of the drawings ]
Fig. 1 is a flowchart of an embodiment of an obstacle type identification method according to the present invention.
Fig. 2 is a schematic diagram of the correspondence between different viewing angles and different channels according to the present invention.
FIG. 3 is a schematic diagram of a three-dimensional point cloud according to the present invention.
Fig. 4 is a flowchart of a method for identifying obstacle types according to a preferred embodiment of the present invention.
Fig. 5 is a schematic structural diagram of the obstacle type identification apparatus according to an embodiment of the present invention.
FIG. 6 illustrates a block diagram of an exemplary computer system/server 12 suitable for use in implementing embodiments of the present invention.
[ detailed description ] A
In order to make the technical solution of the present invention clearer and more obvious, the solution of the present invention is further described in detail below by referring to the drawings and examples.
Fig. 1 is a flowchart of an embodiment of an obstacle type identification method according to the present invention, as shown in fig. 1, including the following specific implementation manners.
In 101, three-dimensional point cloud data corresponding to an obstacle to be recognized is acquired.
Before that, each obstacle detected from the scanned three-dimensional point cloud data may be first acquired, and each detected obstacle may be respectively used as an obstacle to be identified, that is, for each detected obstacle, the type of each detected obstacle may be respectively identified according to the method of the present invention.
The three-dimensional point cloud data obtained through scanning can be obtained through scanning the surrounding environment of the unmanned vehicle.
How to detect the obstacle from the scanned three-dimensional point cloud data can be determined according to actual needs, for example, a clustering algorithm can be adopted.
Clustering refers to dividing a data set into different classes or clusters according to some specific criteria such that the similarity between data within a class or cluster is as large as possible.
Common clustering algorithms can be classified into the following categories: partitioning methods, hierarchical methods, density-based methods, network-based methods, model-based methods, and the like.
For the three-dimensional point cloud data obtained by scanning, zero obstacles may be detected, one obstacle may be detected, and a plurality of obstacles may be detected.
For each obstacle, the corresponding three-dimensional point cloud data can be respectively determined according to the prior art, and for one obstacle, the corresponding three-dimensional point cloud data is one part of the three-dimensional point cloud data obtained by scanning.
At 102, the acquired three-dimensional point cloud data is mapped to a two-dimensional image.
Preferably, the two-dimensional image obtained by mapping may be a two-dimensional Red, Green, Blue (RGB, Red, Green, Blue) image.
The specific mapping method may be:
mapping the three-dimensional point cloud data to an R channel of the two-dimensional image from a first perspective;
mapping the three-dimensional point cloud data to a G channel of the two-dimensional image from a second view angle;
mapping the three-dimensional point cloud data to a channel B of the two-dimensional image from a third view angle;
and generating a two-dimensional RGB image according to each mapping result.
Wherein, the first visual angle can be one of the following: a top view angle, a headstock forward view angle and a left side view angle;
the second viewing angle may be one of: a top view angle, a headstock forward view angle and a left side view angle;
the third viewing angle may be one of: a top view angle, a headstock forward view angle and a left side view angle;
the first viewing angle, the second viewing angle and the third viewing angle are different viewing angles.
For example, the first viewing angle may be a top viewing angle, the second viewing angle may be a front viewing angle, and the third viewing angle may be a left viewing angle.
Accordingly, the three-dimensional point cloud data can be mapped to an R channel of the two-dimensional image from a top view angle, the three-dimensional point cloud data can be mapped to a G channel of the two-dimensional image from a head normal view angle, and the three-dimensional point cloud data can be mapped to a B channel of the two-dimensional image from a left side view angle.
Thus, the corresponding relationship between the viewing angle and the channel as shown in fig. 2 can be obtained, fig. 2 is a schematic diagram of the corresponding relationship between different viewing angles and different channels according to the present invention, as shown in fig. 2, the top viewing angle corresponds to the R channel, the front viewing angle corresponds to the G channel, and the left viewing angle corresponds to the B channel.
Of course, the above correspondence relationship is only an example, and the specific correspondence relationship may be determined according to actual needs.
How to perform the mapping can also be determined according to actual needs, for example, taking a top view as an example, the following mapping manner can be adopted.
For a point in three-dimensional space, its coordinate position is assumed to be (10, 20, 30), where 10 is the x-direction coordinate, 20 is the y-direction coordinate, and 30 is the z-direction coordinate.
When the mapping is performed from the top view angle, the z-direction coordinate can be set to be 0, and then the x-direction coordinate and the y-direction coordinate can be used to calibrate a coordinate position (10, 20) of a two-dimensional space, and the coordinate position corresponds to a two-dimensional image, that is, a pixel point with the coordinate position (10, 20) in the two-dimensional image can be designated, and the value of the pixel point on the R channel can be set to be 255, which represents the brightest color.
It should be noted that, in a three-dimensional space, the x-direction coordinate and the y-direction coordinate may be negative values, in this case, when mapping is performed, translation operation and the like are also required, which is specifically implemented as the prior art.
Fig. 3 is a schematic diagram of a three-dimensional point cloud according to the present invention, as shown in fig. 3, the point cloud is a group of discrete points, some points exist, and some points do not exist, so for each pixel point on the two-dimensional image, if there is a corresponding point in the three-dimensional space, the value on the R channel thereof may be set to 255, and if there is no corresponding point in the three-dimensional space, the value on the R channel thereof may be set to 0.
According to the method, the value of each pixel point on the two-dimensional image on the R channel can be obtained respectively.
According to a similar mode to the above, the value of each pixel point on the G channel and the value of each pixel point on the B channel on the two-dimensional image can be obtained respectively.
Because the values of each pixel point on the R channel, the G channel and the B channel are respectively obtained, a two-dimensional RGB image can be obtained.
In 103, based on the obtained two-dimensional image, the type of the obstacle is identified by a deep learning algorithm.
After the two-dimensional image is obtained, the type of the obstacle can be identified based on the two-dimensional image.
Preferably, a deep learning algorithm may be employed to identify the type of obstacle.
The specific deep learning algorithm may be determined according to actual needs, for example, a widely-applied Convolutional Neural Network (CNN) algorithm may be used.
Convolutional neural networks are a multi-layer neural network that is good at dealing with the relevant machine learning problem of images, especially large images.
The convolutional neural network successfully reduces the dimension of the image recognition problem with huge data through a series of methods, and finally enables the image recognition problem to be trained.
A typical convolutional neural network can be composed of convolutional layers, pooling layers and full-connection layers, wherein the convolutional layers and the pooling layers are matched to form a plurality of convolutional groups, characteristics are extracted layer by layer, and classification is finally completed through the full-connection layers.
The operations performed by the convolutional layer can be considered to be inspired by the concept of local receptive field, and the pooling layer is mainly to reduce the data dimension.
In general, the convolutional neural network simulates feature differentiation through convolution, reduces the magnitude of network parameters through weight sharing and pooling of the convolution, and finally completes tasks such as classification through the traditional neural network.
Specifically, the acquired two-dimensional RGB image is three-channel, and the characteristics of each visual angle can be fully learned through deep learning algorithms such as a convolutional neural network and the like, so that the accuracy of the recognition result is ensured.
By recognizing the two-dimensional RGB image, the type of the obstacle, such as a person, a bicycle, a motor vehicle and the like, can be determined.
Based on the above description, fig. 4 is a flowchart of a preferred embodiment of the obstacle type identification method according to the present invention, as shown in fig. 4, including the following specific implementation manners.
In 401, each obstacle detected from the scanned three-dimensional point cloud data is acquired, and for each detected obstacle, the obstacle is respectively used as an obstacle to be recognized and processed as shown in 402-404.
For the unmanned vehicle, the laser radar is an important sensor for the unmanned vehicle to sense the three-dimensional environment, and the laser radar scans a circle of scene and can return point cloud data of a three-dimensional space of the scene, namely three-dimensional point cloud data.
After the three-dimensional point cloud data is obtained, obstacle detection can be performed according to the three-dimensional point cloud data, namely, obstacles existing in scenes around the unmanned vehicle are detected, and the obstacles can be marked in a preset mode.
Then, the type of each detected obstacle can be further recognized, and accordingly, each detected obstacle can be respectively used as an obstacle to be recognized and processed in the manner shown in 402-404.
At 402, three-dimensional point cloud data corresponding to an obstacle is acquired.
Three-dimensional point cloud data of the obstacles can be obtained.
In 403, the three-dimensional point cloud data corresponding to the obstacle is mapped to an R channel, a G channel, and a B channel of the two-dimensional RGB image through three different viewing angles, respectively, so as to obtain the two-dimensional RGB image.
The three different viewing angles may be a top view viewing angle, a nose forward viewing angle, and a left side viewing angle, respectively.
The corresponding relationship between the viewing angle and the channel may be determined according to actual needs, for example, as shown in fig. 2, the top viewing angle may correspond to the R channel, the head front viewing angle may correspond to the G channel, and the left viewing angle may correspond to the B channel.
At 404, the type of the obstacle is identified by a deep learning algorithm based on the resulting two-dimensional RGB image.
The deep learning algorithm may be a convolutional neural network or the like.
By recognizing the two-dimensional RGB image, the type of the obstacle, such as a person, a bicycle, a motor vehicle, etc., can be determined.
Based on the above description, it can be seen that, by adopting the manner described in the above embodiment, the obstacle to be recognized is converted from the three-dimensional space to the two-dimensional space, that is, the dimension reduction processing is performed, the two-dimensional RGB image is obtained, and the type of the obstacle is recognized by the deep learning algorithm based on the two-dimensional RGB image, whereas in the field of two-dimensional image recognition, the deep learning algorithm is a very mature algorithm, thereby ensuring the accuracy of the recognition result.
The above is a description of method embodiments, and the embodiments of the present invention are further described below by way of apparatus embodiments.
Fig. 5 is a schematic diagram of a composition structure of an embodiment of the obstacle type identification apparatus according to the present invention, as shown in fig. 5, including: an acquisition unit 501, a mapping unit 502 and a classification unit 503.
The acquiring unit 501 is configured to acquire three-dimensional point cloud data corresponding to an obstacle to be identified, and send the three-dimensional point cloud data to the mapping unit 502.
The mapping unit 502 is configured to map the three-dimensional point cloud data to a two-dimensional image, and send the two-dimensional image to the classifying unit 503.
The classification unit 503 is configured to identify the type of the obstacle through a deep learning algorithm based on the two-dimensional image.
The acquisition unit 501 may acquire each obstacle detected from the three-dimensional point cloud data obtained by scanning, and may respectively take each detected obstacle as an obstacle to be identified.
The three-dimensional point cloud data can be obtained by scanning the surrounding environment of the unmanned vehicle.
For scanning to obtain three-dimensional point cloud data, zero obstacles may be detected, one obstacle may also be detected, and a plurality of obstacles may also be detected.
For each obstacle to be identified, the obtaining unit 501 may further obtain three-dimensional point cloud data corresponding to the obstacle, and send the three-dimensional point cloud data to the mapping unit 502.
Accordingly, the mapping unit 502 may map the three-dimensional point cloud data to a two-dimensional image, i.e., perform a dimension reduction process, converting from a three-dimensional space to a two-dimensional space.
Preferably, the two-dimensional image obtained by mapping is a two-dimensional RGB image.
Specifically, the mapping unit 502 may adopt the following mapping manner:
mapping the three-dimensional point cloud data to an R channel of the two-dimensional image from a first perspective;
mapping the three-dimensional point cloud data to a G channel of the two-dimensional image from a second view angle;
mapping the three-dimensional point cloud data to a channel B of the two-dimensional image from a third view angle;
and generating a two-dimensional RGB image according to each mapping result.
Wherein, the first visual angle can be one of the following: a top view angle, a headstock forward view angle and a left side view angle;
the second viewing angle may be one of: a top view angle, a headstock forward view angle and a left side view angle;
the third viewing angle may be one of: a top view angle, a headstock forward view angle and a left side view angle;
the first viewing angle, the second viewing angle and the third viewing angle are different viewing angles.
For example, the first viewing angle may be a top viewing angle, the second viewing angle may be a front viewing angle, and the third viewing angle may be a left viewing angle.
Accordingly, the mapping unit 502 may map the three-dimensional point cloud data from a top view perspective to an R channel of the two-dimensional image, map the three-dimensional point cloud data from a head elevation perspective to a G channel of the two-dimensional image, and map the three-dimensional point cloud data from a left side view perspective to a B channel of the two-dimensional image.
After obtaining the two-dimensional RGB image, the classification unit 503 may identify the type of the obstacle through a deep learning algorithm based on the two-dimensional RGB image.
Preferably, the deep learning algorithm may be a convolutional neural network algorithm or the like.
The acquired two-dimensional RGB image is three-channel, and the characteristics of each visual angle can be fully learned through deep learning algorithms such as a convolutional neural network, so that the accuracy of the recognition result is ensured.
By recognizing the two-dimensional RGB image, the type of the obstacle, such as a person, a bicycle, a motor vehicle and the like, can be determined.
For a specific work flow of the apparatus embodiment shown in fig. 5, reference is made to the related description in the foregoing method embodiment, and details are not repeated.
Based on the above description, it can be seen that, by adopting the manner described in the above embodiment, the obstacle to be recognized is converted from the three-dimensional space to the two-dimensional space, that is, the dimension reduction processing is performed, the two-dimensional RGB image is obtained, and the type of the obstacle is recognized by the deep learning algorithm based on the two-dimensional RGB image, whereas in the field of two-dimensional image recognition, the deep learning algorithm is a very mature algorithm, thereby ensuring the accuracy of the recognition result.
FIG. 6 illustrates a block diagram of an exemplary computer system/server 12 suitable for use in implementing embodiments of the present invention. The computer system/server 12 shown in FIG. 6 is only one example and should not be taken to limit the scope of use or functionality of embodiments of the present invention.
As shown in fig. 6, computer system/server 12 is in the form of a general purpose computing device. The components of computer system/server 12 may include, but are not limited to: one or more processors (processing units) 16, a memory 28, and a bus 18 that connects the various system components, including the memory 28 and the processors 16.
Computer system/server 12 typically includes a variety of computer system readable media. Such media may be any available media that is accessible by computer system/server 12 and includes both volatile and nonvolatile media, removable and non-removable media.
The memory 28 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM)30 and/or cache memory 32. The computer system/server 12 may further include other removable/non-removable, volatile/nonvolatile computer system storage media. By way of example only, storage system 34 may be used to read from and write to non-removable, nonvolatile magnetic media (not shown in FIG. 6, and commonly referred to as a "hard drive"). Although not shown in FIG. 6, a magnetic disk drive for reading from and writing to a removable, nonvolatile magnetic disk (e.g., a "floppy disk") and an optical disk drive for reading from or writing to a removable, nonvolatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 by one or more data media interfaces. Memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of embodiments of the invention.
A program/utility 40 having a set (at least one) of program modules 42 may be stored, for example, in memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which examples or some combination thereof may comprise an implementation of a network environment. Program modules 42 generally carry out the functions and/or methodologies of the described embodiments of the invention.
The computer system/server 12 may also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), with one or more devices that enable a user to interact with the computer system/server 12, and/or with any devices (e.g., network card, modem, etc.) that enable the computer system/server 12 to communicate with one or more other computing devices. Such communication may be through an input/output (I/O) interface 22. Also, the computer system/server 12 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN) and/or a public network, such as the Internet) via the network adapter 20. As shown in FIG. 6, network adapter 20 communicates with the other modules of computer system/server 12 via bus 18. It should be appreciated that although not shown in the figures, other hardware and/or software modules may be used in conjunction with the computer system/server 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.
The processor 16 executes various functional applications and data processing by executing programs stored in the memory 28, for example, implementing the methods in the embodiments shown in fig. 1 and 4, namely: the method comprises the steps of obtaining three-dimensional point cloud data corresponding to an obstacle to be identified, mapping the three-dimensional point cloud data to a two-dimensional image, and identifying the type and the like of the obstacle through a deep learning algorithm based on the two-dimensional image.
Wherein the two-dimensional image may be a two-dimensional RGB image.
Accordingly, the three-dimensional point cloud data can be mapped to an R channel of the two-dimensional image from a first view angle, the three-dimensional point cloud data can be mapped to a G channel of the two-dimensional image from a second view angle, the three-dimensional point cloud data can be mapped to a B channel of the two-dimensional image from a third view angle, and finally the two-dimensional RGB image is generated according to mapping results.
Wherein, the first visual angle can be one of the following: a top view angle, a headstock forward view angle and a left side view angle;
the second viewing angle may be one of: a top view angle, a headstock forward view angle and a left side view angle;
the third viewing angle may be one of: a top view angle, a headstock forward view angle and a left side view angle;
the first viewing angle, the second viewing angle and the third viewing angle are different viewing angles.
For concrete implementation, please refer to the corresponding description in the foregoing method embodiments, which is not repeated.
The invention also discloses a computer-readable storage medium, on which a computer program is stored, which program, when being executed by a processor, will carry out the method as in the embodiments of fig. 1 and 4.
Any combination of one or more computer-readable media may be employed. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus and method, etc., can be implemented in other ways. For example, the above-described device embodiments are merely illustrative, and for example, the division of the units is only one logical functional division, and other divisions may be realized in practice.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional unit.
The integrated unit implemented in the form of a software functional unit may be stored in a computer readable storage medium. The software functional unit is stored in a storage medium and includes several instructions to enable a computer device (which may be a personal computer, a server, or a network device) or a processor (processor) to execute some steps of the methods according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (10)
1. An obstacle type identification method, comprising:
acquiring three-dimensional point cloud data corresponding to an obstacle to be identified;
mapping the three-dimensional point cloud data to a two-dimensional image, comprising: respectively mapping the three-dimensional point cloud data to an R channel, a G channel and a B channel of a two-dimensional image through three different visual angles to obtain a two-dimensional RGB image;
and identifying the type of the obstacle through a deep learning algorithm based on the two-dimensional image.
2. The method of claim 1,
the method further comprises the following steps:
acquiring each obstacle detected from three-dimensional point cloud data obtained by scanning;
respectively taking each detected obstacle as the obstacle to be identified;
the three-dimensional point cloud data is obtained by scanning the surrounding environment of the unmanned vehicle.
3. The method of claim 1,
the obtaining of the two-dimensional RGB image comprises:
mapping the three-dimensional point cloud data to an R channel of a two-dimensional image from a first perspective;
mapping the three-dimensional point cloud data to a G channel of a two-dimensional image from a second perspective;
mapping the three-dimensional point cloud data to a B channel of a two-dimensional image from a third perspective;
and generating the two-dimensional RGB image according to each mapping result.
4. The method of claim 3,
the first viewing angle is one of: a top view angle, a headstock forward view angle and a left side view angle;
the second viewing angle is one of: a top view angle, a headstock forward view angle and a left side view angle;
the third viewing angle is one of: a top view angle, a headstock forward view angle and a left side view angle;
the first, second, and third perspectives are different perspectives.
5. An obstacle type identification device, comprising: the device comprises an acquisition unit, a mapping unit and a classification unit;
the acquisition unit is used for acquiring three-dimensional point cloud data corresponding to the obstacle to be identified and sending the three-dimensional point cloud data to the mapping unit;
the mapping unit is used for mapping the three-dimensional point cloud data to a two-dimensional image, and comprises: respectively mapping the three-dimensional point cloud data to an R channel, a G channel and a B channel of a two-dimensional image through three different visual angles to obtain a two-dimensional RGB image, and sending the two-dimensional image to the classification unit;
and the classification unit is used for identifying the type of the obstacle through a deep learning algorithm based on the two-dimensional image.
6. The apparatus of claim 5,
the obtaining unit is further configured to obtain,
acquiring each obstacle detected from three-dimensional point cloud data obtained by scanning;
respectively taking each detected obstacle as the obstacle to be identified;
the three-dimensional point cloud data is obtained by scanning the surrounding environment of the unmanned vehicle.
7. The apparatus of claim 5,
the mapping unit maps the three-dimensional point cloud data to an R channel of a two-dimensional image from a first visual angle, maps the three-dimensional point cloud data to a G channel of the two-dimensional image from a second visual angle, maps the three-dimensional point cloud data to a B channel of the two-dimensional image from a third visual angle, and generates the two-dimensional RGB image according to each mapping result.
8. The apparatus of claim 7,
the first viewing angle is one of: a top view angle, a headstock forward view angle and a left side view angle;
the second viewing angle is one of: a top view angle, a headstock forward view angle and a left side view angle;
the third viewing angle is one of: a top view angle, a headstock forward view angle and a left side view angle;
the first, second, and third perspectives are different perspectives.
9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor when executing the program implements the method of any one of claims 1 to 4.
10. A computer-readable storage medium, on which a computer program is stored, which program, when being executed by a processor, carries out the method according to any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710253581.3A CN108734058B (en) | 2017-04-18 | 2017-04-18 | Obstacle type identification method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710253581.3A CN108734058B (en) | 2017-04-18 | 2017-04-18 | Obstacle type identification method, device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108734058A CN108734058A (en) | 2018-11-02 |
| CN108734058B true CN108734058B (en) | 2022-05-27 |
Family
ID=63924745
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710253581.3A Active CN108734058B (en) | 2017-04-18 | 2017-04-18 | Obstacle type identification method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108734058B (en) |
Families Citing this family (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111160360B (en) * | 2018-11-07 | 2023-08-01 | 北京四维图新科技股份有限公司 | Image recognition method, device and system |
| CN110163064B (en) * | 2018-11-30 | 2022-04-05 | 腾讯科技（深圳）有限公司 | Method and device for identifying road marker and storage medium |
| CN111325049A (en) * | 2018-12-13 | 2020-06-23 | 北京京东尚科信息技术有限公司 | Commodity identification method and device, electronic equipment and readable medium |
| CN109919145B (en) * | 2019-01-21 | 2020-10-27 | 江苏徐工工程机械研究院有限公司 | Mine card detection method and system based on 3D point cloud deep learning |
| CN109948448B (en) * | 2019-02-20 | 2021-03-12 | 苏州风图智能科技有限公司 | Method, device and system for detecting 3D obstacle and computer storage medium |
| CN110046569B (en) * | 2019-04-12 | 2022-04-12 | 北京百度网讯科技有限公司 | Unmanned driving data processing method and device and electronic equipment |
| CN111310811B (en) * | 2020-02-06 | 2021-01-15 | 东华理工大学 | Large-scene three-dimensional point cloud classification method based on multi-dimensional feature optimal combination |
| CN112037521B (en) * | 2020-07-24 | 2021-10-22 | 长沙理工大学 | Vehicle type identification method and hazardous chemical substance vehicle identification method |
| CN112016638B (en) * | 2020-10-26 | 2021-04-06 | 广东博智林机器人有限公司 | Method, device and equipment for identifying steel bar cluster and storage medium |
| CN114721404B (en) * | 2022-06-08 | 2022-09-13 | 超节点创新科技(深圳)有限公司 | Obstacle avoidance method, robot and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101408931A (en) * | 2007-10-11 | 2009-04-15 | Mv科技软件有限责任公司 | System and method for 3D object recognition |
| CN103093191A (en) * | 2012-12-28 | 2013-05-08 | 中电科信息产业有限公司 | Object recognition method with three-dimensional point cloud data and digital image data combined |
| CN104636725A (en) * | 2015-02-04 | 2015-05-20 | 华中科技大学 | Gesture recognition method based on depth image and gesture recognition system based on depth images |
-
2017
- 2017-04-18 CN CN201710253581.3A patent/CN108734058B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101408931A (en) * | 2007-10-11 | 2009-04-15 | Mv科技软件有限责任公司 | System and method for 3D object recognition |
| CN103093191A (en) * | 2012-12-28 | 2013-05-08 | 中电科信息产业有限公司 | Object recognition method with three-dimensional point cloud data and digital image data combined |
| CN104636725A (en) * | 2015-02-04 | 2015-05-20 | 华中科技大学 | Gesture recognition method based on depth image and gesture recognition system based on depth images |
Non-Patent Citations (1)
| Title |
|---|
| Three-bit representation of three-dimensional range data;Nikolaus Karpinsky,etl.;《APPLIED OPTICS》;20130410;第52卷(第11期);正文第1页左栏第2段-第7页右栏第1段，附图1-5e * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108734058A (en) | 2018-11-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108734058B (en) | Obstacle type identification method, device, equipment and storage medium | |
| US11017244B2 (en) | Obstacle type recognizing method and apparatus, device and storage medium | |
| JP6842520B2 (en) | Object detection methods, devices, equipment, storage media and vehicles | |
| US11042762B2 (en) | Sensor calibration method and device, computer device, medium, and vehicle | |
| CN111328396B (en) | Pose estimation and model retrieval for objects in images | |
| EP3961485A1 (en) | Image processing method, apparatus and device, and storage medium | |
| CN110163930B (en) | Lane line generation method, device, equipment, system and readable storage medium | |
| CN109116374B (en) | Method, device and equipment for determining distance of obstacle and storage medium | |
| CN109271944B (en) | Obstacle detection method, obstacle detection device, electronic apparatus, vehicle, and storage medium | |
| CN108629231B (en) | Obstacle detection method, apparatus, device and storage medium | |
| WO2020094033A1 (en) | Method and system for converting point cloud data for use with 2d convolutional neural networks | |
| CN112967283B (en) | Target identification method, system, equipment and storage medium based on binocular camera | |
| WO2020108510A1 (en) | Method and system for semantic label generation using sparse 3d data | |
| CN111311675B (en) | Vehicle positioning method, device, equipment and storage medium | |
| CN109344804A (en) | A kind of recognition methods of laser point cloud data, device, equipment and medium | |
| CN108428248B (en) | Vehicle window positioning method, system, equipment and storage medium | |
| US11842440B2 (en) | Landmark location reconstruction in autonomous machine applications | |
| CN115147328A (en) | Three-dimensional target detection method and device | |
| CN113298044B (en) | Obstacle detection method, system, device and storage medium based on positioning compensation | |
| WO2022247628A1 (en) | Data annotation method and related product | |
| CN111765892A (en) | Positioning method, positioning device, electronic equipment and computer readable storage medium | |
| CN118097351A (en) | Fusion perception method, fusion perception device, computer equipment and readable storage medium | |
| CN116778262A (en) | Three-dimensional target detection method and system based on virtual point cloud | |
| CN118115792A (en) | Three-dimensional target detection and tracking method and system based on improvement YOLOv3 | |
| CN114821528A (en) | Lane line detection method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |