CN113030881A - Point cloud rationality diagnosis method for laser radar, and vehicle including the same - Google Patents
Point cloud rationality diagnosis method for laser radar, and vehicle including the same Download PDFInfo
- Publication number
- CN113030881A CN113030881A CN201911250924.6A CN201911250924A CN113030881A CN 113030881 A CN113030881 A CN 113030881A CN 201911250924 A CN201911250924 A CN 201911250924A CN 113030881 A CN113030881 A CN 113030881A
- Authority
- CN
- China
- Prior art keywords
- point cloud
- laser radar
- fault
- cloud data
- lidar
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003745 diagnosis Methods 0.000 title claims abstract description 28
- 238000000034 method Methods 0.000 title claims abstract description 21
- 238000013528 artificial neural network Methods 0.000 claims abstract description 64
- 230000002159 abnormal effect Effects 0.000 claims abstract description 33
- 238000012545 processing Methods 0.000 claims description 14
- 238000002405 diagnostic procedure Methods 0.000 claims description 9
- 230000003287 optical effect Effects 0.000 claims description 7
- 238000012549 training Methods 0.000 claims description 5
- 239000000523 sample Substances 0.000 claims description 4
- 230000002547 anomalous effect Effects 0.000 claims description 2
- 238000001514 detection method Methods 0.000 description 11
- 230000005856 abnormality Effects 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 230000007257 malfunction Effects 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000001960 triggered effect Effects 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
- G01S7/02—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S13/00
- G01S7/40—Means for monitoring or calibrating
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Biophysics (AREA)
- Evolutionary Computation (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Radar, Positioning & Navigation (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Remote Sensing (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Computer Networks & Wireless Communication (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention provides a point cloud rationality diagnosis method for a laser radar, which comprises the following steps: receiving point cloud data of the laser radar and corresponding working parameters of the laser radar when the point cloud data are generated; inputting the point cloud data and the working parameters of the laser radar into a neural network, wherein the neural network is configured to output whether the point cloud data is reasonable and whether the laser radar has faults/abnormal work at least according to the point cloud data and the working parameters of the laser radar; and judging whether the point cloud data is reasonable or not and whether the laser radar has faults/abnormal work or not according to the output of the neural network.
Description
Technical Field
The invention relates to the field of photoelectric technology, in particular to a point cloud rationality diagnosis method for a laser radar, the laser radar and a vehicle comprising the laser radar.
Background
LiDAR is a general name of laser active detection sensor equipment, and the working principle of the LiDAR is roughly as follows: laser radar's transmitter launches a bundle of laser, and after laser beam met the object, through diffuse reflection, returned to laser receiver, radar module multiplies the velocity of light according to the time interval of sending and received signal, divides by 2 again, can calculate the distance of transmitter and object. Depending on the number of laser beams, there are generally, for example, a single line laser radar, a 4-line laser radar, an 8/16/32/64-line laser radar, and the like. One or more laser beams are emitted along different angles in the vertical direction and scanned in the horizontal direction to realize the detection of the three-dimensional profile of the target area. The multiple measurement channels (lines) correspond to the scan planes at multiple tilt angles, so that the more laser beams in the vertical field, the higher the angular resolution in the vertical direction, and the greater the density of the laser point cloud.
The whole laser radar product comprises optical, mechanical and electronic components and a software algorithm part. These may be faulty parts. When the laser radar has a fault, the reason of the fault is usually difficult to judge, and there may be a fault of the device itself (for example, some electrical component is burned out under high pressure) and a mismatch between the devices (for example, at high temperature, the component a and the component b deform and thus cannot be locked again). In the prior art, as an unmanned eye, if whether a laser radar has a fault or is not normally operated and cannot be timely found and confirmed, a vehicle cannot be controlled to perform corresponding running operation to cope with possible faults or abnormalities, and thus, many potential safety hazards exist. In addition, after the fault of the laser radar is found, the laser radar needs to be dismantled or detected to check possible fault reasons one by one. The detection process is complicated, and time and labor are wasted.
The statements in the background section are merely prior art as they are known to the inventors and do not, of course, represent prior art in the field.
Disclosure of Invention
In view of at least one of the drawbacks of the prior art, the present invention provides a point cloud rationality diagnosis method usable with a lidar, comprising:
receiving point cloud data of the laser radar and corresponding working parameters of the laser radar when the point cloud data are generated;
inputting the point cloud data and the working parameters of the laser radar into a neural network, wherein the neural network is configured to output whether the point cloud data is reasonable or not at least according to the point cloud data and the working parameters of the laser radar;
and judging whether the point cloud data is reasonable or not according to the output of the neural network.
According to an aspect of the invention, the point cloud rationality diagnosis method further comprises: and judging whether the laser radar has a fault or works abnormally according to the output of the neural network.
According to an aspect of the invention, the point cloud rationality diagnosis method further comprises: training the neural network to identify anomalous point cloud data, comprising:
and inputting abnormal point cloud data and parameters of a laser radar corresponding to the abnormal point cloud data generated in the generation process into the neural network so as to train the neural network to identify the abnormal point cloud data.
According to an aspect of the invention, the point cloud rationality diagnosis method further comprises: and inputting the fault corresponding to the abnormal point cloud data into the neural network so as to train the neural network to identify the corresponding fault.
According to an aspect of the invention, the point cloud rationality diagnosis method further comprises: and when the laser radar is judged to have a fault or work abnormally, sending the information of the fault to an electronic control unit of a vehicle provided with the laser radar.
According to one aspect of the invention, the fault comprises one or more of an optical component fault, a mechanical structure fault, a circuit fault.
According to one aspect of the invention, the output of the neural network includes whether the point cloud is abnormal, the possible fault name and probability of the lidar.
The invention also relates to a lidar comprising:
a transmitting unit configured to transmit a probe beam to the outside of the laser radar;
a receiving unit configured to receive a reflected light beam from outside the laser radar and convert the reflected light beam into an electric signal;
a signal processing unit coupled with the receiving unit and configured to generate point cloud data of the lidar according to the electrical signal; and
a point cloud rationality diagnosis unit configured to perform the point cloud rationality diagnosis method as described above, and configured to receive the point cloud data and output result information whether the point cloud data is rational.
According to an aspect of the invention, the point cloud rationality diagnosis unit is further configured to output fault information of the lidar.
According to one aspect of the invention, the signal processing unit and the point cloud rationality diagnosis unit are integrated.
According to an aspect of the invention, the fault information comprises at least one of: whether the point cloud is abnormal, a possible fault name and probability of the lidar.
The invention also relates to a vehicle comprising a lidar as described above.
According to one aspect of the invention, the vehicle further comprises an electronic control unit coupled to the lidar and operable to receive fault information output by a point cloud rationality diagnostic unit of the lidar.
According to one aspect of the invention, the vehicle further comprises a reminder unit coupled to the electronic control unit, the electronic control unit being configured to trigger the reminder unit upon receiving the fault information output by the point cloud rationality diagnosis unit.
According to an aspect of the invention, the electronic control unit is further adapted to control the vehicle to perform a corresponding running operation based on the failure information.
In the embodiment of the invention, the neural network is utilized to further reversely deduce the possible faults of the laser radar through the analysis of the point cloud output by the laser radar. Through the technical scheme of the embodiment of the invention, the technical personnel of the laser radar can be assisted to quickly locate the root cause of the failure of the laser radar; in addition, can be integrated to laser radar to the neural network module, after the customer buys laser radar, be connected laser radar with the ECU on the vehicle, when using laser radar, the inside neural network module of laser radar can detect the cloud of the point of laser radar output at any time, appears unusually when discovering the cloud of point, reminds the customer, and then can control the vehicle and carry out corresponding operation of traveling in order to deal with possible trouble or unusual to laser radar's security performance can be improved.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate an embodiment of the invention and, together with the description, serve to explain the invention and not to limit the invention. In the drawings:
FIG. 1 illustrates a point cloud rationality diagnostic method that may be used with a lidar in accordance with one embodiment of the invention; and
fig. 2 shows a lidar in accordance with one embodiment of the invention.
Detailed Description
In the following, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments may be modified in various different ways, all without departing from the spirit or scope of the present invention. Accordingly, the drawings and description are to be regarded as illustrative in nature, and not as restrictive.
In the description of the present invention, it is to be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", and the like, indicate orientations and positional relationships based on those shown in the drawings, and are used only for convenience of description and simplicity of description, and do not indicate or imply that the device or element being referred to must have a particular orientation, be constructed and operated in a particular orientation, and thus, should not be considered as limiting the present invention. Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, features defined as "first", "second", may explicitly or implicitly include one or more of the described features. In the description of the present invention, "a plurality" means two or more unless specifically defined otherwise.
In the description of the present invention, it should be noted that unless otherwise explicitly stated or limited, the terms "mounted," "connected," and "connected" are to be construed broadly, e.g., as meaning either a fixed connection, a removable connection, or an integral connection, either mechanically, electrically, or in communication with each other; either directly or indirectly through intervening media, either internally or in any other relationship. The specific meanings of the above terms in the present invention can be understood by those skilled in the art according to specific situations.
In the present invention, unless otherwise expressly stated or limited, "above" or "below" a first feature means that the first and second features are in direct contact, or that the first and second features are not in direct contact but are in contact with each other via another feature therebetween. Also, the first feature being "on," "above" and "over" the second feature includes the first feature being directly on and obliquely above the second feature, or merely indicating that the first feature is at a higher level than the second feature. A first feature being "under," "below," and "beneath" a second feature includes the first feature being directly above and obliquely above the second feature, or simply meaning that the first feature is at a lesser level than the second feature.
The following disclosure provides many different embodiments or examples for implementing different features of the invention. To simplify the disclosure of the present invention, the components and arrangements of specific examples are described below. Of course, they are merely examples and are not intended to limit the present invention. Furthermore, the present invention may repeat reference numerals and/or letters in the various examples, such repetition is for the purpose of simplicity and clarity and does not in itself dictate a relationship between the various embodiments and/or configurations discussed. In addition, the present invention provides examples of various specific processes and materials, but one of ordinary skill in the art may recognize applications of other processes and/or uses of other materials.
The preferred embodiments of the present invention will be described in conjunction with the accompanying drawings, and it will be understood that they are described herein for the purpose of illustration and explanation and not limitation.
FIG. 1 illustrates a point cloud rationality diagnostic method 100 that can be used with lidar in accordance with one embodiment of the present invention, and is described in detail below with reference to the accompanying figures.
In step S101, point cloud data of a laser radar and working parameters of the laser radar corresponding to the point cloud data are received.
The lidar is typically rotatable about a vertical axis to acquire 360 degrees of point cloud data in a horizontal plane. Taking 16-line lidar as an example for illustration, it may transmit 16 lines of laser beams (each line of laser beam corresponds to one channel of lidar, and there are 16 channels) in the vertical direction, which is L1, L2, …, L15, and L16, for detecting the surrounding environment.
In the detection process, the laser radar can rotate along the vertical axis of the laser radar, in the rotation process, each channel of the laser radar sequentially emits laser beams according to a certain time interval (for example, 1 microsecond) and detects the laser beams, so that line scanning on one vertical view field is completed, then line scanning of the next vertical view field is performed at a certain angle (for example, 0.1 degree or 0.2 degree) in the horizontal view field direction, and therefore point cloud is formed by detecting for multiple times in the rotation process, and the condition of the surrounding environment can be sensed.
During operation of the lidar, various operating parameters may be adjusted, such as the power or pulse strength of the transmitter may be relatively reduced (relative to the requirement of an expected detection range of up to 200 m) if only an obstacle at a relatively short distance (e.g., 100m) is desired to be detected. For another example, if the detection resolution of the lidar is desired to be high, the lidar may be controlled to rotate 360 degrees at 20HZ, while if the detection resolution is not so high, the laser may be controlled to scan over the horizontal field of view at 10 HZ. For example, most of the existing lidar is multi-line lidar (so-called multi-line, that is, a plurality of emitters are arranged on a vertical view field or a device capable of dividing a single outgoing light beam into a plurality of beams), if the lidar itself has 64 lines, that is, a point cloud of 64 lines on the vertical view field can be realized at most, but according to the detection requirement, the lidar can be controlled to scan only 32 lines. In addition, the lidar can realize 360-degree all-directional scanning of surrounding obstacles through 360-degree rotation, but in some application scenarios, for example, when the lidar is used as a forward-looking radar, it may be desirable for a user that the lidar only provides forward scanning (in the driving direction of the vehicle) within ± 70 degrees, and the lidar can be controlled to detect obstacles only within ± 70 degrees.
In step S102, the point cloud data and the working parameters are input to a neural network, and the neural network is configured to output a judgment result indicating whether the point cloud data is reasonable or not at least according to the point cloud data and the working parameters of the laser radar.
It should be noted that the unreasonable point cloud or the abnormal point cloud indicates that the point cloud data generated after the radar detection does not correspond to the current working parameters, and indicates that the point cloud data has a certain unreasonable state, and at this time, the laser radar may not work normally or have a fault. Therefore, the neural network can further judge whether the laser radar has faults or works abnormally according to the judgment result whether the point cloud data is reasonable or not.
The neural network is, for example, a neural network or a deep learning module which is trained in advance, and the input end of the neural network receives the point cloud data and the working parameters of the laser radar and can at least identify whether the point cloud data is reasonable or normal.
Preferably, after the point cloud data are identified to be abnormal, specific abnormal conditions of the point cloud data and faults of the corresponding laser radar can be judged. Wherein the neural network comprises one or more of a BP network, a multi-layer neural network, a fuzzy neural network, a wavelet neural network, the invention is not limited to a particular type of neural network.
In another embodiment of the present invention, the point cloud data obtained by the lidar detection may be subjected to a certain pre-processing, and then input into the neural network for subsequent identification processing.
In step S103, it is determined whether the point cloud data is reasonable according to the output of the neural network.
After training, the neural network or the deep learning module can output an indication whether the point cloud data is reasonable or not according to the point cloud data and the working parameters of the laser radar. For example, the laser radar is a 64-line radar, but the working parameter is 40 lines, and after the neural network receives 40 lines of point cloud data obtained by the laser radar, the point cloud data is judged to be reasonable; however, if the point cloud data received by the neural network at this time is 38 lines, it can be determined that the point cloud data is unreasonable or abnormal, and at least some unreasonable situations exist.
For another example, the laser radar can realize 360-degree all-around scanning of surrounding obstacles, but within a certain time period, the controlled working parameters of the laser radar only need to provide scanning within a forward +/-50-degree range, and at this time, if the laser radar is detected, no matter whether the laser radar is input into the neural network, the point cloud data obtained by scanning within a forward +/-90 or forward +/-30 or backward (reverse direction of the vehicle driving direction) ± 50-degree range, the point cloud representing the laser radar at a uniform fixed range is not reasonable, and the whole laser radar may have faults or abnormal working.
According to the output of the neural network, whether the point cloud data is reasonable or not can be judged, and preferably, after the point cloud data is judged to be unreasonable, the fault type and the specific position of the fault of the laser radar can be judged. The fault includes one or more of an optical component fault, a mechanical structure fault, and a circuit fault. After judging whether the laser radar has a fault, and optionally judging the fault type and the specific position of the fault of the laser radar, an alarm or a prompt can be sent to a user of the laser radar.
According to a preferred embodiment of the present invention, the point cloud rationality diagnosis method 100 further includes training the neural network to identify abnormal point cloud data, including, for example: and inputting abnormal point cloud data and working parameters of the laser radar corresponding to the abnormal point cloud data generated into the neural network so as to train the neural network to identify the abnormal point cloud data.
For example, according to one embodiment of the invention, the parameter of the lidar comprises, for example, the number of active lines of the lidar at a certain time. A 64-line laser radar will be described as an example. Under normal working conditions, 64 lines need to work simultaneously to detect obstacles, so that in the generated point cloud, if the number of lines is less than 64 lines, the point cloud is abnormal or some fault occurs in the laser radar.
Under some working conditions, a far obstacle does not need to be detected very finely, and only half of the number of lines (32 lines) needs to be used for detection, so that the generated point cloud data only comprises 32 lines, in this case, the point cloud data of 32 lines is normal, and the point cloud data of more or less than 32 lines is abnormal or unreasonable.
According to a further preferred embodiment of the invention, the neural network provides a certain margin for the plausibility determination of the point cloud data. For example, when a 64-line lidar is operating at 64 lines, but one of the lasers fails, resulting in only 63 lines of data in the point cloud. Although the laser radar fault also belongs to the laser radar fault, the deviation between the current state and the normal state is small, so that the point cloud of the laser radar is still credible, and the point cloud can be used as an abnormity and also can be used as a reliable sensor for unmanned driving.
In addition, the neural network is configured to determine whether the subsequent one or more frames of point cloud data are reasonable according to the prior one or more frames of point cloud data. For example, if the point cloud of frame 20 shows an object somewhere, the point cloud of frame 21 also shows the object, and the moving speed and direction of the object can be deduced according to the time interval of frames 20-21, so that it can be predicted that the object should be somewhere in frame 22 or 23, but the point cloud detected in frame 22 or 23 is very different from the prediction, which indicates that the point cloud is abnormal.
When training the neural network, the fault corresponding to the abnormal point cloud data may be input into the neural network to train the neural network to identify the corresponding fault. For example, various faults of the laser radar and abnormal states of the corresponding point clouds of the laser radar can be obtained through statistics in advance, and when abnormal point cloud data and parameters of the corresponding laser radar when the abnormal point cloud data are generated are input into the neural network, the corresponding faults are input into the neural network, so that the neural network can learn and judge fault conditions of the laser radar. The fault may include, for example, one or more of an optical component fault, a mechanical structure fault, and a circuit fault.
According to one embodiment of the invention, the output of the neural network comprises one or more of whether the point cloud is abnormal, a possible fault name and a probability of the lidar. Additionally in accordance with a preferred embodiment of the present invention the neural network is configured to output a plurality of faults and corresponding probabilities. For example, a model of a neural network is trained first, so that it can recognize point cloud image forms corresponding to different faults 1, 2 …, etc. (a mapping relation graph between an abnormal point cloud image and a fault cause is established), and then the neural network is reused in an actual scene, and a fault type, a fault component or a fault cause which may occur to a lidar is further reversely deduced through analysis of a point cloud output by the lidar, for example, in current point cloud data, the probability of fault 1 is 90%, the probability of fault 2 is 40%, the probability of fault 3 is 10%, and all faults with output probabilities higher than a preset value are provided for a user to refer.
According to a preferred embodiment of the present invention, the lidar is mounted on a vehicle, and a control unit of the lidar is coupled to an electronic control unit ECU of the vehicle, so that when it is determined that there is a malfunction or an operational abnormality in the lidar, the control unit of the lidar may transmit information of the malfunction to the electronic control unit of the vehicle on which the lidar is mounted. The fault information may include an indication that the lidar is faulty, and/or a specific fault type and fault location. After receiving the failure information, the electronic control unit may make a decision according to the failure information, for example, send an audible and visual prompt to a vehicle driver, or stop an automatic driving state of the vehicle to prompt the vehicle driver to take over a driving operation of the vehicle. It should be noted that the above-mentioned fault information may be whether the point cloud is abnormal, whether the radar works abnormally, whether the radar is faulty, the possible fault types, and the approximate probability.
According to one embodiment of the invention, the electronic control unit, upon receiving the fault information, may decide whether to continue to trust the lidar and continue the autonomous driving state based on the severity of the fault. For example, as previously described, when a 64-line lidar is operating at 64 lines, but one of the lasers fails, resulting in only 63 lines of data in the point cloud. Although the laser radar fault also belongs to the laser radar fault, the point cloud of the laser radar is credible due to the small deviation between the current state and the normal state, and the point cloud can also be used as a reliable sensor for unmanned driving, and the electronic control unit can continue the automatic driving state but can prompt the current state to the operator.
Fig. 2 shows a lidar 200 according to an embodiment of the invention. Described in detail below with reference to fig. 2.
As shown in fig. 2, the laser radar 200 includes a transmitting unit 201, a receiving unit 202, a signal processing unit 203, and a point cloud rationality diagnosis unit 204. The transmitting unit 201 generally includes a plurality of lasers and a transmitting lens, wherein the lasers are configured to emit laser beams, the laser beams are incident on the transmitting lens, and are shaped to form probe beams, and the probe beams are emitted into a three-dimensional space around the laser radar. The receiving unit 202 generally includes a receiving lens that receives a reflected beam (or radar echo) from outside the lidar and focuses it onto a detector, which may include, for example, an APD or SiPM, that converts an optical signal incident thereon into an electrical signal. A signal processing unit 203 is coupled to the receiving unit 202 and configured to generate point cloud data of the lidar from the electrical signal. The signal processing unit 203 may generally include signal processing circuits of various stages, including but not limited to an amplifying circuit (e.g., a transimpedance amplifier), a filtering circuit, an analog-to-digital conversion circuit, and the like, and may calculate parameters such as a distance and an orientation of an obstacle according to the optical signal and other related information, and generate point cloud data.
The point cloud rationality diagnosis unit 204 is coupled to the signal processing unit 203, may receive the point cloud data, and is configured to perform the point cloud rationality diagnosis method 100 as described above, and output result information whether the point cloud data is rational.
According to a preferred embodiment of the invention, the point cloud rationality diagnosis unit 24 is further configured to output fault information of the lidar. On the basis of judging that the point cloud data is unreasonable, the point cloud rationality diagnosis unit 24 may also judge specific fault information of the laser radar according to the point cloud data.
According to a preferred embodiment of the present invention, the signal processing unit 203 and the point cloud rationality diagnosis unit 204 may be integrated together, for example, a diagnosis module is integrated inside an FPGA or an ASIC of the signal processing unit 203, so that it is possible to determine whether hardware of the laser radar is in a problem or not, in addition to signal processing. If there is a problem, error information is output. If no problem exists, the point cloud is input to the neural network through the neural network, and whether the fault exists is output. According to one embodiment, the signal processing unit 203 and the point cloud rationality diagnosis unit 204 are both integrated on the lower deck of the lidar.
The invention also relates to a vehicle on which a lidar as described above is mounted.
An Electronic Control Unit (ECU) of the vehicle can be coupled with the laser radar and can receive fault information output by a point cloud rationality diagnosis unit of the laser radar. And a reminding unit, such as a sound reminding unit or a light reminding unit, can be installed on the vehicle, and the reminding unit is coupled with the electronic control unit ECU and can be triggered by the electronic control unit ECU. And when the electronic control unit is configured to receive fault information output by the point cloud rationality diagnosis unit, the reminding unit is triggered to send an alarm to a driver. .
In the embodiment of the invention, the neural network is utilized to further reversely deduce the possible faults of the laser radar through the analysis of the point cloud output by the laser radar. Through the technical scheme of the embodiment of the invention, the technical personnel of the laser radar can be assisted to quickly locate the root cause of the failure of the laser radar; in addition, can be integrated to the laser radar to the neural network module, the user is after purchasing the laser radar, is connected the ECU on laser radar and the vehicle, and when using laser radar, the inside neural network module of laser radar can detect the some cloud of laser radar output at any time, appears unusually when discovering some clouds, reminds the customer.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.
Finally, it should be noted that: although the present invention has been described in detail with reference to the foregoing embodiments, it will be apparent to those skilled in the art that changes may be made in the embodiments and/or equivalents thereof without departing from the spirit and scope of the invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (15)
1. A point cloud rationality diagnostic method usable with lidar, comprising:
receiving point cloud data of the laser radar and corresponding working parameters of the laser radar when the point cloud data are generated;
inputting the point cloud data and the working parameters of the laser radar into a neural network, wherein the neural network is configured to output whether the point cloud data is reasonable or not at least according to the point cloud data and the working parameters of the laser radar;
and judging whether the point cloud data is reasonable or not according to the output of the neural network.
2. The point cloud rationality diagnostic method of claim 1, further comprising: and judging whether the laser radar has a fault or works abnormally according to the output of the neural network.
3. The point cloud rationality diagnostic method of claim 1, further comprising: training the neural network to identify anomalous point cloud data, comprising:
and inputting abnormal point cloud data and parameters of a laser radar corresponding to the abnormal point cloud data generated in the generation process into the neural network so as to train the neural network to identify the abnormal point cloud data.
4. The point cloud rationality diagnostic method of claim 3, further comprising: and inputting the fault corresponding to the abnormal point cloud data into the neural network so as to train the neural network to identify the corresponding fault.
5. The point cloud rationality diagnostic method of claim 3 or 4, further comprising: and when the laser radar is judged to have a fault or work abnormally, sending the information of the fault to an electronic control unit of a vehicle provided with the laser radar.
6. The point cloud rationality diagnostic method of claim 1 or 2, wherein said fault comprises one or more of an optical component fault, a mechanical structure fault, a circuit fault.
7. The point cloud rationality diagnostic method according to claim 1 or 2, wherein the output of the neural network includes whether the point cloud is abnormal, a possible fault name and probability of the lidar.
8. A lidar comprising:
a transmitting unit configured to transmit a probe beam to the outside of the laser radar;
a receiving unit configured to receive a reflected light beam from outside the laser radar and convert the reflected light beam into an electric signal;
a signal processing unit coupled with the receiving unit and configured to generate point cloud data of the lidar according to the electrical signal; and
a point cloud rationality diagnosis unit configured to perform the point cloud rationality diagnosis method according to any one of claims 1 to 7, and configured to receive the point cloud data and output result information on whether the point cloud data is rational.
9. The lidar of claim 8, the point cloud rationality diagnostic unit further configured to output fault information of the lidar.
10. The lidar according to claim 9, wherein the signal processing unit and the point cloud rationality diagnostic unit are integrated.
11. The lidar of claim 9, wherein the fault information comprises at least one of: whether the point cloud is abnormal, a possible fault name and probability of the lidar.
12. A vehicle comprising a lidar according to any of claims 8 to 11.
13. The vehicle of claim 12, further comprising an electronic control unit coupled to the lidar and operable to receive fault information output by a point cloud rationality diagnostic unit of the lidar.
14. A vehicle according to claim 12 or 13, further comprising a reminder unit coupled to the electronic control unit, the electronic control unit being configured to trigger the reminder unit when fault information output by the point cloud rationality diagnosis unit is received.
15. The vehicle according to claim 14, said electronic control unit being further adapted to control the vehicle to perform a corresponding running operation according to said failure information.
Priority Applications (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911250924.6A CN113030881A (en) | 2019-12-09 | 2019-12-09 | Point cloud rationality diagnosis method for laser radar, and vehicle including the same |
| PCT/CN2020/084031 WO2021088313A1 (en) | 2019-11-07 | 2020-04-09 | State detection device for laser radar, laser radar, and state detection method |
| US17/738,236 US20220268904A1 (en) | 2019-11-07 | 2022-05-06 | State detection device for lidar, lidar, and state detection method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911250924.6A CN113030881A (en) | 2019-12-09 | 2019-12-09 | Point cloud rationality diagnosis method for laser radar, and vehicle including the same |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113030881A true CN113030881A (en) | 2021-06-25 |
Family
ID=76451070
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911250924.6A Pending CN113030881A (en) | 2019-11-07 | 2019-12-09 | Point cloud rationality diagnosis method for laser radar, and vehicle including the same |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113030881A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TWI771098B (en) * | 2021-07-08 | 2022-07-11 | 國立陽明交通大學 | Fault diagnosis system and method for state of radar system of roadside units |
| CN114755642A (en) * | 2022-03-23 | 2022-07-15 | 广州文远知行科技有限公司 | Radar line detection method and device, vehicle and storage medium |
| CN115032618A (en) * | 2022-08-12 | 2022-09-09 | 深圳市欢创科技有限公司 | Blind area repairing method and device applied to laser radar and laser radar |
| WO2023098040A1 (en) * | 2021-11-30 | 2023-06-08 | 驭势科技（北京）有限公司 | Diagnostic method and apparatus, control method and apparatus, medium, controller, and vehicle |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2006258484A (en) * | 2005-03-15 | 2006-09-28 | Mitsubishi Electric Corp | Laser radar system |
| CN106779066A (en) * | 2016-12-02 | 2017-05-31 | 上海无线电设备研究所 | A kind of radar circuit plate method for diagnosing faults |
| CN108802761A (en) * | 2017-05-03 | 2018-11-13 | 通用汽车环球科技运作有限责任公司 | Method and system for laser radar point cloud exception |
| CN109597088A (en) * | 2018-11-15 | 2019-04-09 | 深圳市速腾聚创科技有限公司 | Laser radar emerging system |
| CN110007315A (en) * | 2019-04-09 | 2019-07-12 | 深圳市速腾聚创科技有限公司 | Laser radar detection device, detection method and control system |
-
2019
- 2019-12-09 CN CN201911250924.6A patent/CN113030881A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2006258484A (en) * | 2005-03-15 | 2006-09-28 | Mitsubishi Electric Corp | Laser radar system |
| CN106779066A (en) * | 2016-12-02 | 2017-05-31 | 上海无线电设备研究所 | A kind of radar circuit plate method for diagnosing faults |
| CN108802761A (en) * | 2017-05-03 | 2018-11-13 | 通用汽车环球科技运作有限责任公司 | Method and system for laser radar point cloud exception |
| CN109597088A (en) * | 2018-11-15 | 2019-04-09 | 深圳市速腾聚创科技有限公司 | Laser radar emerging system |
| CN110007315A (en) * | 2019-04-09 | 2019-07-12 | 深圳市速腾聚创科技有限公司 | Laser radar detection device, detection method and control system |
Non-Patent Citations (2)
| Title |
|---|
| 刘福刚: "光电雷达部件故障诊断的模式识别研究", 《激光杂志》, vol. 39, no. 4, pages 58 - 61 * |
| 杨亚波 等: "基于CANopen协议的雷达故障诊断系统设计", 《雷达与对抗》, vol. 34, no. 2, 30 June 2014 (2014-06-30), pages 66 - 70 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TWI771098B (en) * | 2021-07-08 | 2022-07-11 | 國立陽明交通大學 | Fault diagnosis system and method for state of radar system of roadside units |
| WO2023098040A1 (en) * | 2021-11-30 | 2023-06-08 | 驭势科技（北京）有限公司 | Diagnostic method and apparatus, control method and apparatus, medium, controller, and vehicle |
| CN114755642A (en) * | 2022-03-23 | 2022-07-15 | 广州文远知行科技有限公司 | Radar line detection method and device, vehicle and storage medium |
| CN114755642B (en) * | 2022-03-23 | 2023-03-10 | 广州文远知行科技有限公司 | Radar line detection method and device, vehicle and storage medium |
| CN115032618A (en) * | 2022-08-12 | 2022-09-09 | 深圳市欢创科技有限公司 | Blind area repairing method and device applied to laser radar and laser radar |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113030881A (en) | Point cloud rationality diagnosis method for laser radar, and vehicle including the same | |
| US20220268904A1 (en) | State detection device for lidar, lidar, and state detection method | |
| US10295659B2 (en) | Angle calibration in light detection and ranging system | |
| CN113567961A (en) | Laser radar state detection device, laser radar, and state detection method | |
| US10891751B2 (en) | Range image generating method, range image generating apparatus, and on-board device having range image generating function | |
| CN110794383A (en) | Code disc, photoelectric encoder and laser radar | |
| JP2010249613A (en) | Obstacle recognition device and vehicle control unit | |
| JP2007122507A (en) | Intrusion detection apparatus | |
| US20160377701A1 (en) | System and method for detecting mounting angle of blind spot detection sensor | |
| CN102414578A (en) | Position monitoring system for a mobile machine | |
| KR102324511B1 (en) | Method for detecting defects in the 3d lidar sensor using point cloud data | |
| JP4815190B2 (en) | Intrusion detection device | |
| CN114488095A (en) | Diagnosis method for laser radar, laser radar and computer storage medium | |
| JP2024509876A (en) | LiDAR system with active fault monitoring | |
| US20220155450A1 (en) | Methods and systems for detecting degraded lidar range measurement accuracy | |
| KR102262384B1 (en) | DRONE OPERATIN SYSTEM and METHOD for INSPECTING POWER FACILITIES | |
| US20230258800A1 (en) | Checking the positioning of an ultrasonic sensor on a vehicle | |
| US20240034605A1 (en) | Safety device for self-propelled industrial vehicles | |
| CN110936937B (en) | Automatic braking method and system | |
| JPH1031064A (en) | Scan type laser radar device | |
| CN112782672A (en) | Laser radar receiving end assembly, fault diagnosis method thereof and laser radar | |
| WO2023123890A1 (en) | Lidar position and orientation diagnostic method, lidar and autonomous vehicle | |
| JP2017215642A (en) | Monitoring system | |
| KR102439487B1 (en) | Providing method for a drone-based transmission and distribution facility management system | |
| AU2022344976A1 (en) | Conveyor belt thickness measurement systems and methods for detecting changes in conveyor belt thicknesses |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: No.2 building, no.468 xinlai Road, Jiading District, Shanghai, 201821 Applicant after: Shanghai Hesai Technology Co.,Ltd. Address before: No.2 building, no.468 xinlai Road, Jiading District, Shanghai, 201821 Applicant before: Shanghai Hesai Technology Co., Ltd |
|
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |