CN113901903A - Road identification method and device - Google Patents
Road identification method and device Download PDFInfo
- Publication number
- CN113901903A CN113901903A CN202111156969.4A CN202111156969A CN113901903A CN 113901903 A CN113901903 A CN 113901903A CN 202111156969 A CN202111156969 A CN 202111156969A CN 113901903 A CN113901903 A CN 113901903A
- Authority
- CN
- China
- Prior art keywords
- point cloud
- road
- plane
- point
- cloud data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
- G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
Abstract
The disclosure provides a road identification method and a road identification device, relates to the technical field of maps, and can be applied to the fields of automatic driving and intelligent transportation. The specific implementation scheme is as follows: acquiring a panoramic image set and point cloud data which are acquired at preset time sequence intervals within preset time; respectively carrying out image semantic segmentation on panoramic images in the panoramic image set, and identifying the category of pixel points in each panoramic image; segmenting the point cloud data according to scanning time, and extracting target point cloud data corresponding to a middle time period within the preset time; respectively mapping the target point cloud data to each panoramic image to obtain a category set of each point in the target point cloud data in different panoramic images; and carrying out statistical analysis on the category set of each point to determine the point of which the category is the road.
Description
Technical Field
The present disclosure relates to the field of map technology, and may be applied in the fields of automatic driving and intelligent transportation.
Background
With the development of robots and automatic driving, the three-dimensional sensing technology based on the laser radar point cloud plays an increasingly important role. Laser radar is widely used particularly in the field of automatic driving where safety is very important because of its excellent distance measuring ability. The radar point cloud-based three-dimensional target detection is a core technology of automatic driving perception capability and is also a precondition for subsequent tracking, path planning and the like.
In the field of mobile mapping or automatic driving, point cloud data acquired on a vehicle is a direct data source for manufacturing a true three-dimensional high-precision map, constructing a pavement information model and the like, wherein the identification of a point cloud road is the key for high-quality mapping and modeling.
Disclosure of Invention
The present disclosure provides a road identification method, apparatus, device, storage medium and computer program product.
According to a first aspect of the present disclosure, there is provided a road identification method, comprising: acquiring a panoramic image set and point cloud data which are acquired at preset time sequence intervals within preset time; respectively carrying out image semantic segmentation on panoramic images in the panoramic image set, and identifying the category of pixel points in each panoramic image; segmenting the point cloud data according to scanning time, and extracting target point cloud data corresponding to a middle time period within preset time; respectively mapping the target point cloud data to each panoramic image to obtain a category set of each point in the target point cloud data in different panoramic images; and carrying out statistical analysis on the category set of each point to determine the point of which the category is the road.
According to a second aspect of the present disclosure, there is provided a road recognition apparatus comprising: an acquisition unit configured to acquire a panoramic image set and point cloud data acquired at predetermined time intervals within a predetermined time; the segmentation unit is configured to perform image semantic segmentation on the panoramic images in the panoramic image set respectively and identify the category of pixel points in each panoramic image; the segmentation unit is configured to segment the point cloud data according to the scanning time and extract target point cloud data corresponding to a middle time period within a preset time; the mapping unit is configured to map the target point cloud data into each panoramic image respectively to obtain a category set of each point in the target point cloud data in different panoramic images; and the analysis unit is configured to perform statistical analysis on the category set of each point and determine the point of which the category is the road.
According to a third aspect of the present disclosure, there is provided an electronic device comprising: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of the first aspect.
According to a fourth aspect of the present disclosure, there is provided a non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of the first aspect.
According to a fifth aspect of the present disclosure, there is provided a computer program product comprising a computer program which, when executed by a processor, implements the method of the first aspect.
The road identification method and the road identification device provided by the embodiment of the application are based on a multi-time sequence fused panoramic image, point cloud segmentation and image semantic segmentation are combined, abundant texture information of an image and spatial information of point cloud data are fully exerted, the road identification accuracy of the point cloud data is greatly improved, road point clouds are accurately extracted, and an important role is played for high-precision maps or road surface modeling.
It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.
Drawings
The drawings are included to provide a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:
FIG. 1 is an exemplary system architecture diagram in which the present application may be applied;
FIG. 2 is a flow diagram of one embodiment of a road identification method according to the present application;
FIGS. 3a-3c are diagrams of effects of an embodiment of a road identification method according to the application;
FIG. 4 is a flow chart of yet another embodiment of a road identification method according to the present application;
FIGS. 5a-5e are diagrams of effects of still another embodiment of a road identification method according to the application;
FIG. 6 is a schematic diagram of an application scenario of a road identification method according to the application;
FIG. 7 is a schematic diagram illustrating the structure of one embodiment of an apparatus for detecting objects according to the present application;
FIG. 8 is a block diagram of a computer system suitable for use in implementing the electronic device of an embodiment of the present application.
Detailed Description
Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, in which various details of the embodiments of the disclosure are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
Fig. 1 illustrates an exemplary system architecture 100 to which the road identification method and apparatus of the embodiments of the present application may be applied.
As shown in fig. 1, system architecture 100 may include unmanned vehicles (also known as autonomous vehicles) 101, 102, a network 103, a database server 104, and a server 105. Network 103 is the medium used to provide communication links between the unmanned vehicles 101, 102, database server 104, and server 105. Network 103 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.
The unmanned vehicles 101 and 102 are provided therein with driving control equipment and equipment for collecting point cloud data, such as a laser radar and a millimeter wave radar. And a camera is also installed for collecting panoramic images. The driving control equipment (also called vehicle-mounted brain) is responsible for intelligent control of the unmanned vehicle. The driving control device may be a Controller separately arranged, such as a Programmable Logic Controller (PLC), a single chip microcomputer, an industrial Controller, and the like; or the equipment consists of other electronic devices which have input/output ports and have the operation control function; but also a computer device installed with a vehicle driving control type application.
It should be noted that, in practice, the unmanned vehicle may also be equipped with at least one sensor, such as, for example, a gravity sensor, a wheel speed sensor, etc. In some cases, the unmanned vehicle may further include GNSS (Global Navigation Satellite System) equipment, SINS (Strap-down Inertial Navigation System), and the like.
The server 105 may also be a server that provides various services, such as a background server that provides support for various applications displayed on the unmanned vehicles 101, 102. The background server may train the initial model using samples in the sample set collected by the unmanned vehicles 101 and 102, and may send a training result (e.g., a generated image semantic segmentation model) to the unmanned vehicles 101 and 102. Therefore, the user can use the generated image semantic segmentation model to segment the panoramic image and then combine the segmentation result with the point cloud data, thereby identifying the road.
Here, the database server 104 and the server 105 may be hardware or software. When they are hardware, they can be implemented as a distributed server cluster composed of a plurality of servers, or as a single server. When they are software, they may be implemented as multiple pieces of software or software modules (e.g., to provide distributed services), or as a single piece of software or software module. And is not particularly limited herein. Database server 104 and server 105 may also be servers of a distributed system or servers that incorporate a blockchain. Database server 104 and server 105 may also be cloud servers, or smart cloud computing servers or smart cloud hosts with artificial intelligence technology.
It should be noted that the road identification method provided in the embodiment of the present application is generally executed by the server 105. Accordingly, the road recognition device is also generally provided in the server. Under the condition that the unmanned vehicle has strong processing capacity, the road identification method provided by the embodiment of the application can be executed by the unmanned vehicle. Accordingly, the road recognition device is also generally provided in the unmanned vehicle.
It is noted that database server 104 may not be provided in system architecture 100, as server 105 may perform the relevant functions of database server 104.
It should be understood that the number of unmanned vehicles, networks, database servers, and servers in fig. 1 are merely illustrative. There may be any number of unmanned vehicles, networks, database servers, and servers, as desired for implementation.
With continued reference to FIG. 2, a flow 200 of one embodiment of a road identification method according to the present application is shown. The road identification method may include the steps of:
In the present embodiment, an execution subject of the road recognition method (for example, a server or an unmanned vehicle shown in fig. 1) may acquire a panoramic image and point cloud data from the unmanned vehicle. The camera of the unmanned vehicle collects panoramic images according to a preset time sequence interval (for example, 1 piece/second), and the laser radar also collects point cloud data according to a preset scanning time. The road recognition is carried out according to the panoramic image collected in the preset time each time, and the detection precision can be improved. For example, 3 panoramic images in succession for 2 seconds and point cloud data scanned for 2 seconds are acquired. The unmanned vehicle can acquire full-time panoramic images and point cloud data in real time in the driving process. Step 202 and 205 can be executed for the panoramic image and the point cloud data in a predetermined time in a segmented manner, and then the recognition results are summarized to obtain complete road information. For example, the road is identified according to the data of 0-2 seconds, then the road is identified according to the data of 2-4 seconds, and so on, the road information can be identified in real time, and finally the complete road identification result is obtained.
Preferably, the number of the panoramic images is odd, so that the categories of the point clouds can be determined according to a minority majority-compliant principle when the categories of the point clouds mapped to the panoramic images are different.
In this embodiment, the panoramic images in the panoramic image set may be subjected to image semantic segmentation respectively through an image semantic segmentation model (e.g., EfficientPS). As shown in fig. 3a, the left side is the original panoramic image and the right side is the segmented panoramic image. The different colored pixels represent different categories. So that a road region can be extracted from the panoramic image.
And step 203, segmenting the point cloud data according to the scanning time, and extracting target point cloud data corresponding to the middle time period within the preset time.
In this embodiment, the point cloud is segmented according to gpsm according to the acquisition time interval (for example, the acquisition interval of each frame of image is 1s) and the acquisition time of the panoramic image, corresponding to the scanning time (gpsm) attribute of the point cloud itself, and the point cloud in the same time sequence is divided into one segment. For example: and respectively shooting three frames of panoramic images in t-1, t and t +1 seconds, wherein the t time sequence section corresponding to the t moment is the second time section from t-0.5 to t +0.5 seconds, and the point cloud scanned in the second time section is divided into one section. And taking the extracted point cloud data as target point cloud data.
And 204, respectively mapping the target point cloud data to each panoramic image to obtain a category set of each point in the target point cloud data in different panoramic images.
In this embodiment, the point cloud data may be down-converted from the world coordinate system to the image coordinate system. The conventional coordinate system conversion method can be adopted, the point cloud data is firstly converted from a world coordinate system to a camera coordinate system, and then is converted from the camera coordinate system to an image coordinate system. The required prior knowledge and conditions: camera calibration, combined calibration of a laser radar and a camera, camera internal reference, and camera and radar external reference. Therefore, each point in the target point cloud data is mapped into the panoramic image, and the category of the pixel point with the same coordinate in the panoramic image can be taken as the category of the point cloud. The categories obtained by mapping the same laser point to different panoramic images may be different, and further statistical analysis is needed to determine the categories. And as shown in fig. 3b, calculating the effect on the panoramic image at the time t by the point cloud coordinates of the time sequence segments of t-1, t and t +1 from top to bottom.
In this embodiment, each laser spot acquires category information from the image at different times. And each laser point obtains a plurality of classification values, and the mode value of the classification values is taken as the classification value of the final point cloud. For example, one laser point is mapped into 5 frames of panoramic images, and if the category of 3 frames of panoramic images is a road, the category of the laser point is the road.
According to the road identification method provided by the embodiment of the application, the advantage that the panoramic image has higher semantic segmentation accuracy is utilized, the accuracy of point cloud classification is improved, and a new thought and direction are provided for point cloud processing. The method has very high practical value in high-precision map making, automatic driving and smart city data processing.
In some optional implementations of this embodiment, the panoramic image set includes three panoramic images; and carrying out statistical analysis on the category set of each point to determine the point of which the category is the road, wherein the statistical analysis comprises the following steps: for each point in the target point cloud data, if the type of the point in the three panoramic images is different, the type corresponding to the panoramic image at the middle time is taken as the type of the point, otherwise, the mode values of the three types are taken as the type of the point; and determining the point with the category as the road according to the category of each point in the target point cloud data.
In the acquisition process, the scanning of the point cloud is continuous in time, and the shooting of images is intermittent.
In order to make the class value of the laser point correspond to enough image data references, three continuously shot frames of images are used as matching references of a section of specific point cloud data.
The current time is t moment, the point cloud category of the t time sequence section is mapped and classified by the segmentation image corresponding to the 3 frames of images shot at the 3 moments of t-1, t, t +1 (mapping and classifying: point cloud single point coordinates are calculated into the segmentation image, and the classification category represented by the image pixel is the category of the point cloud single point). Thus, each laser point respectively obtains the category information from three frames of images at different time. And obtaining three classification values by each laser point, taking a mode value of the three classification values as a classification value of the final point cloud, and if the three values are different, taking a classification value of the t-time segmentation image mapping assignment.
The continuous 3 segmentation images are fused in a point cloud at a corresponding time, and due to the fact that moving objects such as automobiles exist in the images, laser points with parts being blocked and with conflicting categories can be determined according to the categories corresponding to 2 images, and the probability of wrong segmentation is reduced. As shown in fig. 3c, the sequence from top to bottom is: t time sequence segment point clouds, classification values obtained from t-1, t, t +1 images.
In some optional implementations of the present embodiment, mapping the target point cloud data into each panoramic image respectively includes: converting world coordinates of the target point cloud data into local horizontal coordinates; converting the local horizontal coordinates to inertial sensor coordinates; converting the coordinates of the inertial sensor into the coordinates of a panoramic ball; converting the panoramic ball coordinate into a two-dimensional coordinate of a segmentation image; and respectively mapping the two-dimensional coordinates of the segmentation image into each panoramic image.
The method for converting point cloud data coordinate into image coordinate can also utilize collinear relation to convert point cloud coordinate system.
1) The world coordinate system is converted into a local horizontal coordinate system by the calculation formula
In the formula：[X,Y,Z]nativeThe local horizontal coordinates of the point cloud are obtained; [ X, Y, Z ]]worldPoint cloud geodetic coordinates; [ X, Y, Z ]]imgCorresponding position coordinates when the image is shot; eworldConverting a matrix for world coordinates; b, L and H are corresponding geodetic coordinates of the image.
2) Converting the local horizontal coordinate system into the coordinates of the inertial sensor by the formula
In the formula: [ X, Y, Z ]]imuAre inertial sensor coordinates; enativeConverting a matrix for the local horizontal coordinates; alpha is a roll angle; beta pitch angle; and gamma is a yaw angle.
3) The coordinates of the inertial sensor are converted into the coordinates of a panoramic ball by the calculation formula
In the formula: [ X, Y, Z ]]gFor panoramic coordinates, also written as [ X ]g,Yg,Zg](ii) a Delta X, delta Y and delta Z are three translation amounts of a coordinate system; r is the distance from the point cloud to the sphere center; l is the radius of the panoramic ball; ekIs a transformation matrix.
4) The two-dimensional coordinates of the segmentation image corresponding to the panoramic ball coordinates are calculated according to the formula
In the formula: (X)s,Ys) Which are two-dimensional coordinates of pixels in the image.
The coordinate conversion mode does not need to calibrate camera parameters.
Referring to fig. 4, a flow 400 of an embodiment of a road identification method provided by the present application is shown. The road identification method may include the steps of:
And 402, performing image semantic segmentation on the panoramic images in the panoramic image set respectively, and identifying the category of pixel points in each panoramic image.
And 403, segmenting the point cloud data according to scanning time, and extracting target point cloud data corresponding to a middle time period within preset time.
And step 404, respectively mapping the target point cloud data to each panoramic image to obtain a category set of each point in the target point cloud data in different panoramic images.
Step 401-.
And 406, performing plane fitting segmentation on the target point cloud data to obtain at least one point cloud plane, and determining the point cloud plane including the points of the road in the at least one point cloud plane as a pre-classification road plane.
In this embodiment, a common three-dimensional hough transform may be used to perform plane fitting segmentation on the point cloud to obtain each segmented plane point cloud. Each plane contains a plurality of point clouds therein. As shown in fig. 5a (original point cloud) and fig. 5b (planar segmented point cloud). Using the coarse classification result in step 405 (as shown in fig. 5 c), each plane point cloud is subjected to fine classification of road class. And judging whether the point cloud in each plane has the road point category in the rough classification result. If the plane contains roughly classified road points, the segmentation plane is a pre-classification road plane.
Optionally, the point cloud identification results of the full time sequence can be merged and then subjected to plane fitting segmentation.
In this embodiment, the pre-classified road plane is analyzed to eliminate potential error planes:
removing conditions are as follows: 1. the proportion p, 2 of the road point classes in the plane in the whole plane point, and the included angle t between the normal vector of the plane and the vertical direction.
Analyzing the pre-classified road plane according to the following principle:
if t > a first angle (e.g., 45 degrees), then the plane is set as a non-road plane;
if t > a second angle (e.g., 25 degrees) and p < a predetermined ratio threshold (e.g., 0.75,) then the plane is set as a non-road plane.
Wherein the first angle is greater than the second angle. This allows filtering out the side areas of objects such as road teeth, fences, etc.
And step 408, all points in the road plane screened from the pre-classified road plane set are set as road points.
In the present embodiment, all the points in the pre-classified road plane remaining in step 407 are set as road points, and the fine classification of point cloud road identification is completed (as shown in fig. 5 d). The final road point cloud extraction result after texture mapping is shown in fig. 5e, which has a high-precision road surface edge and does not divide vehicle points into road surface points and road edge points by mistake.
The road identification method of the embodiment combines the point cloud plane segmentation method at the front edge with the semantic segmentation method of the panoramic image, combines the advantages of the panoramic image with higher semantic segmentation accuracy and the point cloud plane segmentation with higher local information consistency, further improves the accuracy of point cloud classification, and provides a new idea and direction for point cloud processing. The method has very high practical value in high-precision map making, automatic driving and smart city data processing.
With further reference to fig. 6, fig. 6 is a schematic diagram of an application scenario of the road identification method according to the present embodiment. In the application scenario of fig. 6, the unmanned vehicle acquires panoramic images and point cloud data in real time during the driving process. The unmanned vehicle identifies the panoramic images through the image semantic segmentation model, and extracts a road area from each panoramic image. The unmanned vehicle also segments the original point cloud data. For point cloud data of a time sequence t, road area auxiliary identification of a panoramic image at three moments t-1, t and t +1 is needed. And respectively mapping the point cloud data of the time sequence t to the 3 panoramic images to obtain the categories of the same laser point in different panoramic images, wherein if the categories in at least 2 panoramic images indicate that the laser point is a road, the laser point is the road. This time results in a coarse classification. And combining the rough classification results of the point clouds of each time sequence and performing fine classification. And performing plane fitting segmentation on the point cloud based on the three-dimensional Hough change of the road points. And analyzing and eliminating potential error planes, and locally optimizing the point cloud fine classification result. And finally, all the plane points which are classified into the roads in the subdivision mode are set as road points, and a final road map is obtained.
With continued reference to FIG. 7, the present application provides one embodiment of a road identification device as an implementation of the methods illustrated in the above figures. The embodiment of the device corresponds to the embodiment of the method shown in fig. 2, and the device can be applied to various electronic devices.
As shown in fig. 7, the road recognition device 700 of the present embodiment may include: an acquisition unit 701, a segmentation unit 702, a segmentation unit 703, a mapping unit 704 and an analysis unit 705. The acquiring unit 701 is configured to acquire a panoramic image set and point cloud data acquired at a predetermined time sequence interval within a predetermined time; a segmentation unit 702 configured to perform image semantic segmentation on the panoramic images in the panoramic image set, and identify a category of a pixel point in each panoramic image; a segmentation unit 703 configured to segment the point cloud data by scanning time and extract target point cloud data corresponding to an intermediate time period within a predetermined time; a mapping unit 704 configured to map the target point cloud data into each panoramic image, respectively, to obtain a category set of each point in the target point cloud data in different panoramic images; the analysis unit 705 is configured to perform statistical analysis on the category set of each point, and determine a point of which the category is a road.
In some optional implementations of this embodiment, the apparatus 700 further comprises a fitting unit (not shown in the drawings) configured to: performing plane fitting segmentation on the target point cloud data to obtain at least one point cloud plane; and determining a point cloud plane including points of the road in at least one point cloud plane as a pre-classification road plane.
In some optional implementations of this embodiment, the analyzing unit 705 is further configured to: and determining the pre-classified road plane with the included angle between the plane normal vector and the vertical direction larger than a first angle as a non-road plane.
In some optional implementations of this embodiment, the analyzing unit 705 is further configured to: and determining the pre-classified road plane with the plane normal vector included angle with the vertical direction larger than a second angle and the road point in the plane occupying ratio in the whole plane point smaller than a preset ratio threshold value as a non-road plane.
In some optional implementations of this embodiment, the analyzing unit 705 is further configured to: all points in the road plane screened from the pre-classified road plane set are set as road points.
In some optional implementations of this embodiment, the panoramic image set includes three panoramic images; and the analyzing unit 705 is further configured to: for each point in the target point cloud data, if the type of the point in the three panoramic images is different, the type corresponding to the panoramic image at the middle time is taken as the type of the point, otherwise, the mode values of the three types are taken as the type of the point; and determining the point with the category as the road according to the category of each point in the target point cloud data.
In some optional implementations of this embodiment, the mapping unit 704 is further configured to: converting world coordinates of the target point cloud data into local horizontal coordinates; converting the local horizontal coordinates to inertial sensor coordinates; converting the coordinates of the inertial sensor into the coordinates of a panoramic ball; converting the panoramic ball coordinate into a two-dimensional coordinate of a segmentation image; and respectively mapping the two-dimensional coordinates of the segmentation image into each panoramic image.
In the technical scheme of the disclosure, the collection, storage, use, processing, transmission, provision, disclosure and other processing of the personal information of the related user are all in accordance with the regulations of related laws and regulations and do not violate the good customs of the public order.
The present disclosure also provides an electronic device, a readable storage medium, and a computer program product according to embodiments of the present disclosure.
An electronic device, comprising: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of flows 200 or 400.
A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of flow 200 or 400.
A computer program product comprising a computer program which, when executed by a processor, implements the method of flow 200 or 400.
FIG. 8 illustrates a schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the disclosure described and/or claimed herein.
As shown in fig. 8, the apparatus 800 includes a computing unit 801 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM)802 or a computer program loaded from a storage unit 808 into a Random Access Memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The calculation unit 801, the ROM 802, and the RAM 803 are connected to each other by a bus 804. An input/output (I/O) interface 805 is also connected to bus 804.
A number of components in the device 800 are connected to the I/O interface 805, including: an input unit 806, such as a keyboard, a mouse, or the like; an output unit 807 such as various types of displays, speakers, and the like; a storage unit 808, such as a magnetic disk, optical disk, or the like; and a communication unit 809 such as a network card, modem, wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information/data with other devices via a computer network such as the internet and/or various telecommunication networks.
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.
The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server may be a cloud server, a server of a distributed system, or a server with a combined blockchain.
It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present disclosure may be executed in parallel, sequentially, or in different orders, as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved, and the present disclosure is not limited herein.
The above detailed description should not be construed as limiting the scope of the disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present disclosure should be included in the scope of protection of the present disclosure.
Claims (17)
1. A road identification method, comprising:
acquiring a panoramic image set and point cloud data which are acquired at preset time sequence intervals within preset time;
respectively carrying out image semantic segmentation on panoramic images in the panoramic image set, and identifying the category of pixel points in each panoramic image;
segmenting the point cloud data according to scanning time, and extracting target point cloud data corresponding to a middle time period within the preset time;
respectively mapping the target point cloud data to each panoramic image to obtain a category set of each point in the target point cloud data in different panoramic images;
and carrying out statistical analysis on the category set of each point to determine the point of which the category is the road.
2. The method of claim 1, wherein the method further comprises:
performing plane fitting segmentation on the target point cloud data to obtain at least one point cloud plane;
and determining the point cloud plane comprising the points of the road in the at least one point cloud plane as a pre-classification road plane.
3. The method of claim 2, wherein the method further comprises:
and determining the pre-classified road plane with the included angle between the plane normal vector and the vertical direction larger than a first angle as a non-road plane.
4. The method of claim 2, wherein the method further comprises:
and determining the pre-classified road plane with the plane normal vector included angle with the vertical direction larger than a second angle and the road point in the plane occupying ratio in the whole plane point smaller than a preset ratio threshold value as a non-road plane.
5. The method of claim 3 or 4, wherein the method further comprises:
all points in the road plane screened from the pre-classified road plane set are set as road points.
6. The method of claim 1, wherein the set of panoramic images includes three panoramic images; and
the statistical analysis is performed on the category set of each point, and the point of which the category is the road is determined, including:
for each point in the target point cloud data, if the type of the point in the three panoramic images is different, the type corresponding to the panoramic image at the middle time is taken as the type of the point, otherwise, the mode values of the three types are taken as the type of the point;
and determining the point with the category as the road according to the category of each point in the target point cloud data.
7. The method of claim 1, wherein the mapping the target point cloud data into each panoramic image separately comprises:
converting world coordinates of the target point cloud data into local horizontal coordinates;
converting the local horizontal coordinates to inertial sensor coordinates;
converting the coordinates of the inertial sensor into the coordinates of a panoramic ball;
converting the panoramic ball coordinate into a two-dimensional coordinate of a segmentation image;
and respectively mapping the two-dimensional coordinates of the segmentation image into each panoramic image.
8. A road identifying device comprising:
an acquisition unit configured to acquire a panoramic image set and point cloud data acquired at predetermined time intervals within a predetermined time;
the segmentation unit is configured to perform image semantic segmentation on the panoramic images in the panoramic image set respectively, and identify the category of pixel points in each panoramic image;
a segmentation unit configured to segment the point cloud data according to scanning time and extract target point cloud data corresponding to an intermediate time period within the predetermined time;
the mapping unit is configured to map the target point cloud data into each panoramic image respectively to obtain a category set of each point in the target point cloud data in different panoramic images;
and the analysis unit is configured to perform statistical analysis on the category set of each point and determine the point of which the category is the road.
9. The apparatus of claim 8, wherein the apparatus further comprises a fitting unit configured to:
performing plane fitting segmentation on the target point cloud data to obtain at least one point cloud plane;
and determining the point cloud plane comprising the points of the road in the at least one point cloud plane as a pre-classification road plane.
10. The apparatus of claim 9, wherein the analysis unit is further configured to:
and determining the pre-classified road plane with the included angle between the plane normal vector and the vertical direction larger than a first angle as a non-road plane.
11. The apparatus of claim 9, wherein the analysis unit is further configured to:
and determining the pre-classified road plane with the plane normal vector included angle with the vertical direction larger than a second angle and the road point in the plane occupying ratio in the whole plane point smaller than a preset ratio threshold value as a non-road plane.
12. The apparatus of claim 10 or 11, wherein the analysis unit is further configured to:
all points in the road plane screened from the pre-classified road plane set are set as road points.
13. The apparatus of claim 8, wherein the set of panoramic images comprises three panoramic images; and
the analysis unit is further configured to:
for each point in the target point cloud data, if the type of the point in the three panoramic images is different, the type corresponding to the panoramic image at the middle time is taken as the type of the point, otherwise, the mode values of the three types are taken as the type of the point;
and determining the point with the category as the road according to the category of each point in the target point cloud data.
14. The apparatus of claim 8, wherein the mapping unit is further configured to:
converting world coordinates of the target point cloud data into local horizontal coordinates;
converting the local horizontal coordinates to inertial sensor coordinates;
converting the coordinates of the inertial sensor into the coordinates of a panoramic ball;
converting the panoramic ball coordinate into a two-dimensional coordinate of a segmentation image;
and respectively mapping the two-dimensional coordinates of the segmentation image into each panoramic image.
15. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.
16. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-7.
17. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111156969.4A CN113901903A (en) | 2021-09-30 | 2021-09-30 | Road identification method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111156969.4A CN113901903A (en) | 2021-09-30 | 2021-09-30 | Road identification method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113901903A true CN113901903A (en) | 2022-01-07 |
Family
ID=79189523
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111156969.4A Pending CN113901903A (en) | 2021-09-30 | 2021-09-30 | Road identification method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113901903A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023207360A1 (en) * | 2022-04-29 | 2023-11-02 | 北京字跳网络技术有限公司 | Image segmentation method and apparatus, electronic device, and storage medium |
-
2021
- 2021-09-30 CN CN202111156969.4A patent/CN113901903A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023207360A1 (en) * | 2022-04-29 | 2023-11-02 | 北京字跳网络技术有限公司 | Image segmentation method and apparatus, electronic device, and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113902897B (en) | Training of target detection model, target detection method, device, equipment and medium | |
| CN111080693A (en) | Robot autonomous classification grabbing method based on YOLOv3 | |
| CN112257605B (en) | Three-dimensional target detection method, system and device based on self-labeling training sample | |
| CN113377888B (en) | Method for training object detection model and detection object | |
| CN113378760A (en) | Training target detection model and method and device for detecting target | |
| CN112825192B (en) | Object identification system and method based on machine learning | |
| CN113378693B (en) | Method and device for generating target detection system and detecting target | |
| EP4105600A2 (en) | Method for automatically producing map data, related apparatus and computer program product | |
| CN113255444A (en) | Training method of image recognition model, image recognition method and device | |
| CN113569911A (en) | Vehicle identification method and device, electronic equipment and storage medium | |
| CN114186007A (en) | High-precision map generation method and device, electronic equipment and storage medium | |
| CN113901903A (en) | Road identification method and device | |
| CN113378694A (en) | Method and device for generating target detection and positioning system and target detection and positioning | |
| CN116844129A (en) | Road side target detection method, system and device for multi-mode feature alignment fusion | |
| CN115147809B (en) | Obstacle detection method, device, equipment and storage medium | |
| CN113706705B (en) | Image processing method, device, equipment and storage medium for high-precision map | |
| CN116129422A (en) | Monocular 3D target detection method, monocular 3D target detection device, electronic equipment and storage medium | |
| CN115761698A (en) | Target detection method, device, equipment and storage medium | |
| CN116052097A (en) | Map element detection method and device, electronic equipment and storage medium | |
| CN116246033A (en) | Rapid semantic map construction method for unstructured road | |
| CN115719436A (en) | Model training method, target detection method, device, equipment and storage medium | |
| CN114495049A (en) | Method and device for identifying lane line | |
| CN113569912A (en) | Vehicle identification method and device, electronic equipment and storage medium | |
| CN113591569A (en) | Obstacle detection method, obstacle detection device, electronic apparatus, and storage medium | |
| CN115410173B (en) | Multi-mode fused high-precision map element identification method, device, equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |