CN111994066B - Intelligent automobile sensing system based on intelligent tire touch sensing - Google Patents
Intelligent automobile sensing system based on intelligent tire touch sensing Download PDFInfo
- Publication number
- CN111994066B CN111994066B CN202011177626.1A CN202011177626A CN111994066B CN 111994066 B CN111994066 B CN 111994066B CN 202011177626 A CN202011177626 A CN 202011177626A CN 111994066 B CN111994066 B CN 111994066B
- Authority
- CN
- China
- Prior art keywords
- vehicle
- tire
- intelligent
- road surface
- unit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000004927 fusion Effects 0.000 claims abstract description 50
- 230000008447 perception Effects 0.000 claims abstract description 45
- 230000004438 eyesight Effects 0.000 claims abstract description 35
- 230000010365 information processing Effects 0.000 claims abstract description 16
- 230000005540 biological transmission Effects 0.000 claims abstract description 13
- 238000012544 monitoring process Methods 0.000 claims abstract description 11
- 238000005259 measurement Methods 0.000 claims abstract description 9
- 239000011159 matrix material Substances 0.000 claims description 40
- 238000000034 method Methods 0.000 claims description 21
- 238000013135 deep learning Methods 0.000 claims description 19
- 238000012545 processing Methods 0.000 claims description 13
- 238000012360 testing method Methods 0.000 claims description 12
- 230000007613 environmental effect Effects 0.000 claims description 11
- 230000000007 visual effect Effects 0.000 claims description 11
- 238000012549 training Methods 0.000 claims description 9
- 230000016776 visual perception Effects 0.000 claims description 9
- 230000008859 change Effects 0.000 claims description 7
- 238000013527 convolutional neural network Methods 0.000 claims description 6
- 230000003287 optical effect Effects 0.000 claims description 6
- 238000007781 pre-processing Methods 0.000 claims description 6
- 238000000513 principal component analysis Methods 0.000 claims description 6
- 239000013598 vector Substances 0.000 claims description 6
- 238000011217 control strategy Methods 0.000 claims description 5
- 238000001514 detection method Methods 0.000 claims description 5
- 230000009466 transformation Effects 0.000 claims description 5
- 230000008569 process Effects 0.000 claims description 4
- 238000001914 filtration Methods 0.000 claims description 3
- 238000005457 optimization Methods 0.000 claims description 3
- 238000001228 spectrum Methods 0.000 claims description 3
- 230000035807 sensation Effects 0.000 claims 1
- 238000013433 optimization analysis Methods 0.000 abstract 1
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 239000004568 cement Substances 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 210000000056 organ Anatomy 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 230000008054 signal transmission Effects 0.000 description 1
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W30/00—Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units, or advanced driver assistance systems for ensuring comfort, stability and safety or drive control systems for propelling or retarding the vehicle
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W2420/00—Indexing codes relating to the type of sensors based on the principle of their operation
- B60W2420/40—Photo or light sensitive means, e.g. infrared sensors
- B60W2420/403—Image sensing, e.g. optical camera
-
- B60W2420/408—
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W2552/00—Input parameters relating to infrastructure
Landscapes
- Engineering & Computer Science (AREA)
- Automation & Control Theory (AREA)
- Transportation (AREA)
- Mechanical Engineering (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention provides an intelligent automobile perception system based on intelligent tire touch perception, which comprises an intelligent tire (1), a vision perception system (2), radar equipment (3), a computer vision unit (4), a tire information processing unit (5), a sensor fusion unit (6), an electronic circuit (7), a wireless data transmission device (8) and an entire automobile control unit (9), wherein when a vehicle runs on a road surface, in the aspect of vision, object identification and auxiliary positioning are carried out by combining a camera with a computer vision technology; in the aspect of hearing, positioning, speed measurement and obstacle monitoring are carried out through a radar and a laser radar; in the aspect of 'touch', the intelligent tire senses road surface information, the information is fused by a sensor fusion technology to generate a vehicle environment state which is comprehensively sensed and fed back in time, the vehicle environment state comprises surrounding traffic conditions, vehicle positioning and road information, and the vehicle dynamics optimization analysis and control of a vehicle control unit are facilitated.
Description
Technical Field
The invention belongs to the technical field of unmanned driving, particularly relates to the technical field of intelligent automobile sensing systems, and particularly relates to an intelligent automobile sensing system based on intelligent tire touch sensing.
Background
In recent years, the field of intelligent automobiles in China is rapidly developed, and the intelligent automobile is expected to be widely applied in the future. Among them, the sensing system has been rapidly developed in recent years as an important information source for intelligent driving control. At present, an intelligent automobile sensing system in the market mainly senses and judges external environments such as traffic signals, vehicles and pedestrians, plans and controls vehicle running, and is lack of sensing information such as road surface parameters, friction factors and wet and slippery degrees. The road surface is the only source for providing driving force and braking force for the vehicle, and the wet skid degree, friction factor, roughness and the like of the road have non-negligible influence on the driving power, safety and the like of the vehicle. If the information of the road surface can be sensed and fed back, on one hand, a whole vehicle control system can be optimized, the dynamic property and the sensitivity of a vehicle are improved, on the other hand, the abnormal state of the road surface can be responded in time, and the driving safety is improved.
At present, a sensing system of an intelligent automobile mainly senses the surrounding environment of the automobile by combining a camera with a computer vision technology, a radar, a laser radar, a V2X technology and the like, and is relatively lack of sensing of road surface information. At present, researchers have realized the monitoring of rain and snow road surfaces through cameras and by using a computer vision technology. However, such monitoring has many defects, such as poor monitoring reliability and large influence of light. In addition, other information about the road surface, such as roughness, friction factor, etc., cannot be achieved by computer vision techniques. Research has shown that by installing a sensor in a tire and analyzing and processing signals, the state of a road surface can be sensed, an ice surface and a cement road surface can be distinguished, and the slip rate of the tire and the like can be obtained. The tire is used as the only component of the automobile contacting with the road surface, and the road surface information can be obtained by arranging a sensor in the tire, so that the tire is intelligentized and the contact condition of the tire and the road surface is sensed. The traditional tire mainly focuses on tire pressure and temperature detection, and usually only a temperature or pressure sensor is arranged on the inner wall of the tire, so that the pressure and the temperature of the tire are monitored, real-time alarm is given, the contact condition of the tire and the road surface cannot be sensed, and the touch sensing of automobile driving is increased.
Disclosure of Invention
Aiming at the problems in the prior art, the intelligent automobile sensing system based on intelligent tire touch sensing is provided, an intelligent tire can be added into the automobile sensing system to serve as a touch sensing element of an automobile, and senses the surrounding environment together with a camera, a radar and the like to feed back environment information. When the vehicle runs on the road surface, in the aspect of 'vision', object identification and auxiliary positioning are carried out by combining a camera with a computer vision technology; in the aspect of hearing, positioning, speed measurement and obstacle monitoring are carried out through a radar and a laser radar; in the aspect of 'touch', the intelligent tire senses road information such as roughness, friction factor and the like; the information is fused by utilizing a sensor fusion technology to generate a vehicle environment state which is comprehensively sensed and fed back in time, contains surrounding traffic conditions, vehicle positioning and road information, and is beneficial to the optimal analysis and control of vehicle dynamics by a whole vehicle control unit.
The invention aims to provide an intelligent automobile sensing system based on intelligent tire touch sensing, which comprises an intelligent tire 1, a visual sensing system 2, radar equipment 3, a computer vision unit 4, a tire information processing unit 5, a sensor fusion unit 6, an electronic circuit 7, a wireless data transmission device 8 and a whole automobile control unit 9, wherein the intelligent tire 1 is mechanically connected with two ends of an axle, a plurality of strain sheets are arranged in the intelligent tire 1 along the circumferential direction to form a strain sensor matrix for sensing contact deformation of the intelligent tire 1 and the ground, and generated electric signals are communicated and transmitted to the tire information processing unit 5 through the wireless data transmission device 8; the visual perception system is arranged on the top of the vehicle and is electrically connected with the computer vision unit 4, and the visual perception system 2 is used for observing the road condition around the vehicle and transmitting the image to the computer vision unit 4 for analysis and processing; the radar equipment 3 is arranged on the top of the vehicle and used for speed measurement, positioning and obstacle monitoring, the radar equipment 3 is connected with the sensor fusion unit 6 through an electronic circuit, and monitored information is fed back to the sensor fusion unit 6; the computer vision unit 4 is positioned in an electronic controller of the vehicle and is electrically connected with the sensor fusion unit 6, and the computer vision unit 4 analyzes and processes the image acquired by the vehicle-mounted camera, identifies the surrounding environment and sends the analyzed and processed result to the sensor fusion unit 6; the tire information processing unit 5 is arranged in an electronic controller in the vehicle, is connected with the intelligent tire 1 through a wireless data transmission device 8, and is used for analyzing and processing signals acquired by the intelligent tire to obtain the road surface condition; the tire information processing unit 5 is also electrically connected with the sensor fusion unit 6, and feeds back sensed road surface information to the sensor fusion unit 6 for fusion; the sensor fusion unit 6 is mechanically installed in an electronic controller in the vehicle and is electrically connected with the radar equipment 3, the computer vision unit 4 and the tire information processing unit 5, and the sensor fusion unit 6 carries out deep fusion on the received vehicle environment information to form a comprehensive environment information sensing network and transmits the comprehensive environment information sensing network to the vehicle control unit 9 through an electronic circuit; the whole vehicle control unit 9 is installed in an electronic controller of a vehicle, and adjusts and optimizes a control strategy after receiving the environment sensing information of the sensor fusion unit 6, so that the driving power and safety of the vehicle are ensured.
Preferably, the strain sensor matrix is a piezoelectric strain sensor matrix.
Preferably, the intelligent tire 1 is used for tactile perception of a road surface and providing real-time three-dimensional road information perception for a vehicle, and comprises:
a plurality of strain gauges at different positions in a strain sensor matrix acquire road surface signals;
preprocessing the road surface signal;
carrying out sensor position marking on the preprocessed signals to generate input quantity required by a deep learning network;
carrying out supervised training on the deep network learning implemented by the deep learning network to obtain current road surface parameters, wherein the road surface comprises road surface types, friction factors and/or road surface gradients;
the method comprises the steps that the current motion state of a vehicle is obtained by comparing the time domain difference values of peak signals of strain sensors at different positions and combining the spatial distribution of each sensor in a strain sensor matrix through a deep learning algorithm, wherein the motion state of the vehicle comprises the rotating speed, the vehicle speed and/or the slip ratio of the vehicle;
the method comprises the steps of sensing the current tire pressure state of a tire by comparing the peak value change of a plurality of strain gauge acquisition signals in a strain sensor matrix in a time domain and combining a deep learning algorithm;
and (4) alarming: if the average peak value of the peak value signal of the strain sensor matrix increases along with the increase of time, the tire is considered to be in a state of increased tire pressure currently, and when the average peak value of the peak value signal of the strain sensor matrix is larger than a threshold value, an alarm of overlarge tire pressure is sent to a whole vehicle control unit; if the average peak value of the peak value signal of the strain sensor matrix is reduced along with the increase of time, the strain sensor matrix is considered to be in a state of reducing the tire pressure at present, and when the average peak value of the peak value signal of the strain sensor matrix is smaller than a threshold value, an alarm that the tire pressure is too small is sent to a whole vehicle control unit;
and (3) controlling and adjusting: responding to the situation that the intelligent tire 1 senses that the current road surface is an ice surface, and enabling a whole vehicle control unit to timely respond, wherein the response comprises the reduction of the vehicle speed and/or the adjustment of the oil pressure of a braking system; responding to the situation that the intelligent tire 1 obtains the current road surface friction factor, feeding the current road surface friction factor back to a whole vehicle control unit, optimizing and adjusting the dynamic control, monitoring the road surface information in real time by the intelligent tire 1, and ensuring reliable signals and no influence of the environment.
Preferably, the preprocessing the road surface signal includes:
performing principal component analysis on the road surface signal to eliminate noise and drift of the signal caused by tire vibration;
and carrying out filtering and wavelet transformation processing on the road surface signals obtained after the principal component analysis, and converting the road surface signals to a time domain and a frequency domain to carry out multi-scale detailed analysis.
Preferably, the supervised training of the deep network learning performed by the deep learning network to obtain the current road surface parameters includes: and carrying out a tire bench test and a real vehicle test, carrying out a stress test on the tire, acquiring strain data of the inner wall of the tire under different stress states in real time by using a strain sensor matrix, and carrying out supervised training on the deep network learning implemented by the deep learning network.
Preferably, the vision perception system 2 includes a vehicle-mounted camera, the vehicle-mounted camera includes 6 cameras, is two front binocular cameras, two left and right side cameras and two rear binocular cameras respectively, and is used for carrying out all-round environmental perception. When the vehicle runs, the vehicle-mounted camera monitors the conditions of surrounding roads in real time, sends images to the computer vision unit 4 in real time, and utilizes the images to perform vehicle positioning and object identification.
Preferably, the method for locating the vehicle by using the image is to locate the current position of the vehicle by using a visual SLAM algorithm, and includes:
reading surrounding environment data through a vehicle-mounted camera;
estimating relative motion between the front time and the rear time by using a visual odometer;
processing the accumulated error estimated by the visual odometer by using a filter and a graph optimization algorithm at the back end;
and constructing a three-dimensional map and eliminating space accumulated errors through loop detection so as to position the current position of the vehicle.
Preferably, the method for object recognition using an image includes:
identifying an object in the image by using a convolutional neural network;
after an image candidate frame is set, extracting a feature vector of the image by using a convolutional neural network;
identifying objects in the image using an SVM algorithm, the objects including vehicles, pedestrians, and/or obstacles;
transmitting the identified things in the traffic environment into the sensor fusion unit 6.
Preferably, the radar device 3 includes a radar and a laser radar, wherein the laser radar is used for high-precision map drawing, the laser radar includes a scanning component, an optical component and a photosensitive component, the laser radar transmits laser to the surroundings in real time while rotating at a constant speed, the scanning component and the optical component continuously collect the distance between a reflection point and the time and horizontal angle of the reflection point, and the photosensitive component continuously detects the intensity of return light, generates a cloud map of the surroundings and synthesizes to form a 3D map; the radar is used for estimating the distance and the speed of pedestrians and vehicles through frequency modulation continuous waves, the radar transmits high-frequency continuous waves with variable frequency and receives reflected signals, Fourier transformation is carried out on frequency difference values, the frequency and the phase angle of obtained frequency spectrums are analyzed, therefore distance measurement and speed measurement are carried out on the pedestrians and the vehicles, and data obtained by sensing the environment are sent to the sensor fusion unit 6 through the radar equipment 3.
Preferably, the sensor fusion unit 6 receives the sensing information from different sensors, performs deep fusion, and forms a comprehensive vehicle environment sensing system, including:
the sensor fusion unit 6 establishes a visual-auditory-tactile information correlation model based on data acquired by a camera, radar equipment and an intelligent tire;
sensing environmental parameters by different sensing elements consisting of the camera, the radar equipment and the intelligent tire, extracting characteristic vectors and reconstructing a time domain and a space domain of the environmental parameters, and generating a time and space information pair;
and (3) fusing the perception data by adopting a multi-Bayesian estimation method to generate a real-time high-precision three-dimensional environment map, thereby forming a comprehensive vehicle environment perception system.
The invention has the beneficial technical effects that:
1. the intelligent automobile sensing system based on the intelligent tire increases 'touch' sensing for the automobile, solves the problem of lack of road surface information acquisition, and can sense important information of the road surface timely and reliably.
2. The intelligent automobile sensing system based on the intelligent tire has the environmental sensing in the three aspects of vision, hearing and touch, and has more senses compared with other sensing systems, and the surrounding environment state can be acquired more comprehensively.
3. The intelligent automobile sensing system based on the intelligent tire can timely feed back environmental information to the whole automobile control unit, and timely remind the user when the road surface changes, so that the whole automobile control unit can timely change the control strategy, and the safety of automobile running is improved.
4. The intelligent tire is not limited by the types of vehicles and driving environments, has wide applicability and low cost, can be widely applied to various vehicles, and provides 'touch' perception for a perception system of the vehicles.
The tyre is used as a medium for contacting a vehicle and a road surface, and can well reflect the stress condition of the vehicle. When a vehicle runs on a road surface, the tire receives an isotropic force from the road surface. According to the invention, a piezoelectric strain sensor matrix is arranged on the inner wall of the tire, each strain sensor acquires strain data of the inner wall of the tire in real time, and according to the piezoelectric principle, sensors at different positions in the matrix can generate different electric signals. The voltage signals are transmitted to a data preprocessing unit positioned in the center of the hub through a high-speed slip ring arranged on the periphery of the rim to be amplified, and then the signal electric signals are transmitted to a touch sensing unit through a wireless signal transmission unit in a wireless mode. Tire inner wall strain data under a large number of different road surface states are collected through a tire bench test and a real vehicle test, the touch sensing unit is combined with a deep learning algorithm to analyze and process electric signals of all positions in the matrix, road surface information (such as roughness, friction factors and the like) and current vehicle running states (such as vehicle speed, wheel speed and slip rate) are further analyzed, and real-time touch sensing of the current road surface is provided for the vehicle. Meanwhile, the state of the tire is recorded and detected in real time, and warning is given out when the tire pressure is too low or too high, so that the driving safety is ensured. The tire pressure state of the tire is judged by monitoring the change of the electric signal in real time, and the driving safety is ensured. The piezoelectric type strain sensor matrix utilizes a piezoelectric principle, generates an electric signal by sensing tire strain and does not need to supply power; the single chip microcomputer positioned in the wheel hub is powered by the whole vehicle control unit through wireless transmission; the whole vehicle control unit is powered by a low-voltage power supply of the vehicle; the energy required by the whole system is low, and the electric quantity influence on the whole low-voltage power supply is small.
Drawings
Some specific embodiments of the invention are described in detail with reference to the accompanying drawings by way of illustration and not limitation. The same reference numbers in the drawings identify the same or similar elements or components. Those skilled in the art will appreciate that the drawings are not necessarily drawn to scale. The objects and features of the present invention will become more apparent in view of the following description taken in conjunction with the accompanying drawings, in which:
FIG. 1 is a schematic diagram of a smart car sensing system based on smart tire tactile sensing according to an embodiment of the present invention;
fig. 2 is a flowchart of a road surface tactile perception method based on an intelligent tire according to an embodiment of the invention.
Detailed Description
The following detailed description of the embodiments of the present invention is provided with reference to the accompanying drawings, but the present invention is not limited thereto.
The intelligent automobile sensing system based on intelligent tire touch sensing of the embodiment is shown in fig. 1 and comprises an intelligent tire 1, a visual sensing system 2, a radar device 3, a computer vision unit 4, a tire information processing unit 5, a sensor fusion unit 6, an electronic circuit 7, a wireless data transmission device 8 and an entire automobile control unit 9, wherein the intelligent tire 1 is mechanically connected with two ends of an axle, a plurality of strain sheets are arranged in the intelligent tire 1 along the circumferential direction to form a strain sensor matrix for sensing contact deformation of the intelligent tire 1 and the ground, and generated electric signals are communicated and transmitted to the tire information processing unit 5 through the wireless data transmission device 8; the visual perception system is arranged on the top of the vehicle and is electrically connected with the computer vision unit 4, and the visual perception system 2 is used for observing the road condition around the vehicle and transmitting the image to the computer vision unit 4 for analysis and processing; the radar equipment 3 is arranged on the top of the vehicle and used for speed measurement, positioning and obstacle monitoring, the radar equipment 3 is connected with the sensor fusion unit 6 through an electronic circuit, and monitored information is fed back to the sensor fusion unit 6; the computer vision unit 4 is positioned in an electronic controller of the vehicle and is electrically connected with the sensor fusion unit 6, and the computer vision unit 4 analyzes and processes the image acquired by the vehicle-mounted camera, identifies the surrounding environment and sends the analyzed and processed result to the sensor fusion unit 6; the tire information processing unit 5 is arranged in an electronic controller in the vehicle, is connected with the intelligent tire 1 through a wireless data transmission device 8, and is used for analyzing and processing signals acquired by the intelligent tire to obtain the road surface condition; the tire information processing unit 5 is also electrically connected with the sensor fusion unit 6 and feeds back the sensed road surface information to the sensor fusion unit 6 for fusion; the sensor fusion unit 6 is mechanically installed in an electronic controller in the vehicle and electrically connected with the radar equipment 3, the computer vision unit 4 and the tire information processing unit 5, and the sensor fusion unit 6 carries out deep fusion on the received vehicle environment information to form a comprehensive environment information sensing network and transmits the comprehensive environment information sensing network to the vehicle control unit 9 through an electronic circuit; the vehicle control unit 9 is installed in an electronic controller of the vehicle, and adjusts and optimizes a control strategy after receiving the environment sensing information of the sensor fusion unit 6, so that the driving power and safety of the vehicle are ensured.
In this embodiment, the strain sensor matrix is a piezoelectric strain sensor matrix, and it is within the scope of the present invention that a person skilled in the art may use other types of strain sensors or other types of load cells known in the art according to the application requirements.
Referring to fig. 2, the intelligent tire 1 of the present embodiment is used for tactile sensing of a road surface to provide a real-time three-dimensional road surface information sensing for a vehicle, and the working principle thereof includes:
1. a plurality of strain gauges at different positions in a strain sensor matrix acquire road surface signals;
2. preprocessing a road surface signal, comprising:
21, performing principal component analysis on the road surface signal to eliminate noise and drift of the signal caused by tire vibration;
and 22, carrying out filtering and wavelet transformation processing on the road surface signals obtained after the principal component analysis, and converting the road surface signals to a time domain and a frequency domain to carry out multi-scale detailed analysis.
23, performing supervised training on the deep network learning implemented by the deep learning network to obtain the current road surface parameters comprises: carrying out a tire bench test and a real vehicle test, carrying out a stress test on the tire, acquiring strain data of the inner wall of the tire at different positions under different stress states in real time by using a strain sensor matrix, and carrying out supervised training on deep network learning implemented by a deep learning network;
3. carrying out sensor position marking on the preprocessed signals to generate input quantity required by the deep learning network;
4. carrying out supervised training on deep network learning implemented by a deep learning network to obtain current road surface parameters, wherein the road surface comprises road surface types, friction factors and/or road surface gradients;
5. the method comprises the steps that the current motion state of a vehicle is obtained by comparing the time domain difference values of peak signals of strain sensors at different positions and combining the spatial distribution of each sensor in a strain sensor matrix through a deep learning algorithm, wherein the motion state of the vehicle comprises the rotating speed, the vehicle speed and/or the slip ratio of the vehicle;
6. the method comprises the steps of sensing the current tire pressure state of a tire by comparing the peak value change of a plurality of strain gauge acquisition signals in a strain sensor matrix in a time domain and combining a deep learning algorithm;
7. and (4) alarming: if the average peak value of the peak value signal of the strain sensor matrix increases along with the increase of time, the tire is considered to be in a state of increased tire pressure currently, and when the average peak value of the peak value signal of the strain sensor matrix is larger than a threshold value, an alarm of overlarge tire pressure is sent to a whole vehicle control unit; if the average peak value of the peak value signal of the strain sensor matrix is reduced along with the increase of time, the strain sensor matrix is considered to be in a state of reducing the tire pressure at present, and when the average peak value of the peak value signal of the strain sensor matrix is smaller than a threshold value, an alarm that the tire pressure is too small is sent to a whole vehicle control unit;
8. and (3) controlling and adjusting: when the intelligent tire 1 senses that the current road surface is an ice surface, the whole vehicle control unit timely makes a response, including reducing the vehicle speed and/or adjusting the oil pressure of a braking system; the current road surface friction factor is obtained by responding to the intelligent tire 1, the current road surface friction factor is fed back to the whole vehicle control unit, dynamic control is optimized and adjusted, the intelligent tire 1 monitors road surface information in real time, signals are reliable, and the intelligent tire is not influenced by the environment.
In this embodiment, the visual perception system 2 is implemented by using a vehicle-mounted camera, and may be implemented by using other types of visual perception devices. The vehicle-mounted camera comprises 6 cameras, namely two front-mounted binocular cameras, a left side camera, a right side camera and two rear-mounted binocular cameras, and is used for carrying out all-around environment perception. When the vehicle runs, the vehicle-mounted camera monitors the conditions of surrounding roads in real time, sends images to the computer vision unit 4 in real time, and utilizes the images to perform vehicle positioning and object identification.
The method for positioning the vehicle by using the image is to position the current position of the vehicle by using a visual SLAM algorithm, and comprises the following steps of:
1. reading surrounding environment data through a vehicle-mounted camera;
2. estimating relative motion between the front time and the rear time by using a visual odometer;
3. processing the accumulated error estimated by the visual odometer by using a filter and a graph optimization algorithm at the back end;
4. and constructing a three-dimensional map and eliminating space accumulated errors through loop detection so as to position the current position of the vehicle.
The method for object recognition by using the image comprises the following steps:
1. identifying an object in the image by using a convolutional neural network;
2. after setting an image candidate frame, extracting a feature vector of the image by using a convolutional neural network;
3. identifying objects in the image by using an SVM algorithm, wherein the objects comprise vehicles, pedestrians and/or obstacles;
4. the identified things in the traffic environment are sent to the sensor fusion unit 6.
In this embodiment, the radar device 3 is composed of at least a radar and a laser radar, and naturally, other types of radar device subcomponents can be added according to the test requirements, so as to improve the detection accuracy and the richness of the obtained information. The laser radar is used for high-precision map drawing, and comprises a scanning component, an optical component and a photosensitive component, wherein the laser radar sends laser to the surroundings in real time while rotating at a constant speed, meanwhile, the scanning component and the optical component continuously collect the distance of a reflection point and the time and horizontal angle of the reflection point, and the photosensitive component continuously detects the intensity of return light, generates a cloud map of the surrounding environment and synthesizes to form a 3D map; the radar is used for carrying out the estimation of distance and speed through frequency modulation continuous wave to pedestrian, vehicle, and the high frequency continuous wave that the radar transmission frequency changes receives the reflected signal, through carrying out Fourier transform to the frequency difference, and frequency and the phase angle of the obtained frequency spectrum of analysis to carry out range finding and speed measuring to pedestrian, vehicle, radar equipment 3 will be to sensor fusion unit 6 to the data transmission that the environmental perception obtained.
In this system, sensor fuses unit 6 and receives the perception information from different sensors, carries out the degree of depth and fuses, forms comprehensive vehicle environment perception system, and the theory of operation includes:
1. the sensor fusion unit 6 establishes a visual-auditory-tactile information correlation model based on data acquired by the camera, the radar equipment and the intelligent tire;
2. sensing environmental parameters by different sensing elements consisting of a camera, radar equipment and an intelligent tire, extracting characteristic vectors of the environmental parameters and reconstructing a time domain and a space domain to generate a time and space information pair;
3. and (3) fusing the perception data by adopting a multi-Bayesian estimation method to generate a real-time high-precision three-dimensional environment map, thereby forming a comprehensive vehicle environment perception system.
The intelligent automobile sensing system based on the intelligent tire is added with 'touch' sensing for the automobile, the problem of lack of acquisition of road surface information is solved, important information of the road surface can be sensed timely and reliably, environment sensing in three aspects of 'vision', 'hearing' and 'touch' is achieved simultaneously, compared with other sensing systems, more sensing organs are achieved, the surrounding environment state can be acquired more comprehensively, meanwhile, the environment information can be fed back to the whole automobile control unit timely, timely reminding is achieved when the road surface changes, the whole automobile control unit can change the control strategy timely, and the driving safety of the automobile is improved. The related intelligent tire is not limited by the types of vehicles and driving environments, has wide applicability and low cost, can be widely applied to various vehicles, and provides 'touch' perception for a perception system of the vehicles.
The technical solutions provided by the embodiments of the present invention are described in detail above, and the principles and embodiments of the present invention are explained herein by using specific examples, and the descriptions of the embodiments are only used to help understanding the principles of the embodiments of the present invention; meanwhile, a person skilled in the art may change the embodiments and the application scope according to the embodiments of the present invention, and in summary, the content of the present description should not be construed as limiting the present invention.
Claims (9)
1. The utility model provides an intelligent automobile perception system based on intelligent tire tactile sensation which characterized in that: the intelligent tire comprises an intelligent tire (1), a visual perception system (2), radar equipment (3), a computer vision unit (4), a tire information processing unit (5), a sensor fusion unit (6), an electronic circuit (7), a wireless data transmission device (8) and a whole vehicle control unit (9), wherein the intelligent tire (1) is mechanically connected with two ends of an axle, a plurality of strain gauges are arranged inside the intelligent tire (1) along the circumferential direction to form a strain sensor matrix for perceiving contact deformation of the intelligent tire (1) and the ground, and generated electric signals are communicated and transmitted to the tire information processing unit (5) through the wireless data transmission device (8); the visual perception system is arranged on the top of the vehicle and is electrically connected with the computer vision unit (4), and the visual perception system (2) is used for observing the road condition around the vehicle and transmitting the image to the computer vision unit (4) for analysis and processing; the radar equipment (3) is arranged on the top of the vehicle and used for speed measurement, positioning and obstacle monitoring, the radar equipment (3) is connected with the sensor fusion unit (6) through an electronic circuit, and monitored information is fed back to the sensor fusion unit (6); the computer vision unit (4) is positioned in an electronic controller of the vehicle and is electrically connected with the sensor fusion unit (6), and the computer vision unit (4) analyzes and processes images acquired by a vehicle-mounted camera of the vision perception system (2), identifies the surrounding environment and sends the analyzed and processed result to the sensor fusion unit (6); the tire information processing unit (5) is arranged in an electronic controller in the vehicle, is connected with the intelligent tire (1) through a wireless data transmission device (8), and is used for analyzing and processing signals acquired by the intelligent tire to obtain the road surface condition; the tire information processing unit (5) is also electrically connected with the sensor fusion unit (6) and feeds back sensed road surface information to the sensor fusion unit (6) for fusion; the sensor fusion unit (6) is mechanically installed in an electronic controller in the vehicle and is electrically connected with the radar equipment (3), the computer vision unit (4) and the tire information processing unit (5), and the sensor fusion unit (6) carries out deep fusion on received vehicle environment information to form a comprehensive environment information sensing network and transmits the environment information sensing network to the vehicle control unit (9) through an electronic circuit; the whole vehicle control unit (9) is arranged in an electronic controller of the vehicle, and adjusts and optimizes a control strategy after receiving the environment sensing information of the sensor fusion unit (6), so that the driving dynamic property and safety of the vehicle are ensured;
the intelligent tire (1) is used for tactile perception of a road surface and providing real-time three-dimensional road surface information perception for vehicles, and comprises:
a plurality of strain gauges at different positions in a strain sensor matrix acquire road surface signals;
preprocessing the road surface signal;
carrying out sensor position marking on the preprocessed signals to generate input quantity required by a deep learning network;
carrying out supervised training on the deep network learning implemented by the deep learning network to obtain current road surface parameters, wherein the road surface comprises road surface types, friction factors and/or road surface gradients;
the method comprises the steps that the current motion state of a vehicle is obtained by comparing the time domain difference values of peak signals of strain sensors at different positions and combining the spatial distribution of each sensor in a strain sensor matrix through a deep learning algorithm, wherein the motion state of the vehicle comprises the rotating speed, the vehicle speed and/or the slip ratio of the vehicle;
the method comprises the steps of sensing the current tire pressure state of a tire by comparing the peak value change of a plurality of strain gauge acquisition signals in a strain sensor matrix in a time domain and combining a deep learning algorithm;
and (4) alarming: if the average peak value of the peak value signal of the strain sensor matrix increases along with the increase of time, the tire is considered to be in a state of increased tire pressure currently, and when the average peak value of the peak value signal of the strain sensor matrix is larger than a threshold value, an alarm of overlarge tire pressure is sent to a whole vehicle control unit; if the average peak value of the peak value signal of the strain sensor matrix is reduced along with the increase of time, the strain sensor matrix is considered to be in a state of reducing the tire pressure at present, and when the average peak value of the peak value signal of the strain sensor matrix is smaller than a threshold value, an alarm that the tire pressure is too small is sent to a whole vehicle control unit;
and (3) controlling and adjusting: responding to the situation that the intelligent tire (1) senses that the current road surface is an ice surface, and enabling a whole vehicle control unit to timely react, wherein the reaction comprises the steps of reducing the vehicle speed and/or adjusting the oil pressure of a braking system; responding to the situation that the intelligent tire (1) acquires the current road surface friction factor, feeding the current road surface friction factor back to a whole vehicle control unit, optimizing and adjusting the dynamic control, and monitoring the road surface information in real time by the intelligent tire (1).
2. The intelligent vehicle sensing system based on intelligent tire tactile sensing of claim 1, wherein: the strain sensor matrix is a piezoelectric strain sensor matrix.
3. The intelligent automobile perception system based on intelligent tire tactile perception according to claim 1, wherein: the preprocessing the road surface signal comprises:
performing principal component analysis on the road surface signal to eliminate noise and drift of the signal caused by tire vibration;
and carrying out filtering and wavelet transformation processing on the road surface signals obtained after the principal component analysis, and converting the road surface signals to a time domain and a frequency domain to carry out multi-scale detailed analysis.
4. The intelligent automobile perception system based on intelligent tire tactile perception according to claim 1, wherein: the supervised training of the deep network learning implemented by the deep learning network to obtain the current road surface parameters comprises the following steps: and carrying out a tire bench test and a real vehicle test, carrying out a stress test on the tire, acquiring strain data of the inner wall of the tire under different stress states in real time by using a strain sensor matrix, and carrying out supervised training on the deep network learning implemented by the deep learning network.
5. The intelligent automobile perception system based on intelligent tire tactile perception according to claim 1, wherein: the vision perception system (2) comprises a vehicle-mounted camera, the vehicle-mounted camera comprises 6 cameras which are respectively two front binocular cameras, a left side camera, a right side camera and two rear binocular cameras and are used for carrying out omnibearing environment perception, when a vehicle runs, the vehicle-mounted camera monitors the conditions of surrounding roads in real time, sends images to a computer vision unit (4) in real time, and utilizes the images to carry out vehicle positioning and object recognition.
6. The intelligent automobile perception system based on intelligent tire tactile perception according to claim 5, wherein: the method for positioning the vehicle by using the image is to position the current position of the vehicle by using a visual SLAM algorithm, and comprises the following steps:
reading surrounding environment data through a vehicle-mounted camera;
estimating relative motion between the front time and the rear time by using a visual odometer;
processing the accumulated error estimated by the visual odometer by using a filter and a graph optimization algorithm at the back end;
and constructing a three-dimensional map and eliminating space accumulated errors through loop detection so as to position the current position of the vehicle.
7. The intelligent automobile perception system based on intelligent tire tactile perception according to claim 5, wherein: the method for object recognition by using images comprises the following steps:
identifying an object in the image by using a convolutional neural network;
after an image candidate frame is set, extracting a feature vector of the image by using a convolutional neural network;
identifying objects in the image using an SVM algorithm, the objects including vehicles, pedestrians, and/or obstacles;
transmitting the identified things in the traffic environment into the sensor fusion unit (6).
8. The intelligent automobile perception system based on intelligent tire tactile perception according to claim 1, wherein: the radar equipment (3) comprises a radar and a laser radar, wherein the laser radar is used for high-precision map drawing, the laser radar comprises a scanning component, an optical component and a photosensitive component, the laser radar transmits laser to the surroundings in real time while rotating at a constant speed, the scanning component and the optical component continuously collect the distance between a reflection point and the time and horizontal angle of the reflection point, and the photosensitive component continuously detects the intensity of return light, generates a cloud map of the surrounding environment and synthesizes to form a 3D map; the radar is used for estimating the distance and the speed of pedestrians and vehicles through frequency modulation continuous waves, the radar transmits the high-frequency continuous waves with variable frequency and receives reflected signals, Fourier transformation is carried out on frequency difference values, the frequency and the phase angle of obtained frequency spectrums are analyzed, therefore distance measurement and speed measurement are carried out on the pedestrians and the vehicles, and data obtained through environment perception are sent to the sensor fusion unit (6) through the radar equipment (3).
9. The intelligent automobile perception system based on intelligent tire tactile perception according to claim 1, wherein: the sensor fuses unit (6) and receives the perception information from different sensors, carries out the degree of depth and fuses, forms comprehensive vehicle environmental perception system, includes:
the sensor fusion unit (6) establishes a visual-auditory-tactile information association model based on data acquired by a camera, radar equipment and an intelligent tire;
sensing environmental parameters by different sensing elements consisting of the camera, the radar equipment and the intelligent tire, extracting characteristic vectors and reconstructing a time domain and a space domain of the environmental parameters, and generating a time and space information pair;
and (3) fusing the perception data by adopting a multi-Bayesian estimation method to generate a real-time high-precision three-dimensional environment map, thereby forming a comprehensive vehicle environment perception system.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011177626.1A CN111994066B (en) | 2020-10-29 | 2020-10-29 | Intelligent automobile sensing system based on intelligent tire touch sensing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011177626.1A CN111994066B (en) | 2020-10-29 | 2020-10-29 | Intelligent automobile sensing system based on intelligent tire touch sensing |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111994066A CN111994066A (en) | 2020-11-27 |
| CN111994066B true CN111994066B (en) | 2021-01-29 |
Family
ID=73475766
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011177626.1A Active CN111994066B (en) | 2020-10-29 | 2020-10-29 | Intelligent automobile sensing system based on intelligent tire touch sensing |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111994066B (en) |
Families Citing this family (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112223962A (en) * | 2020-12-08 | 2021-01-15 | 北京航空航天大学 | Intelligent vehicle control system and method based on road surface touch perception |
| CN112224212B (en) * | 2020-12-08 | 2021-05-11 | 北京航空航天大学 | Vehicle control system based on road surface condition tactile perception |
| CN113408459B (en) * | 2021-06-29 | 2022-08-30 | 吉林大学 | Vehicle multi-mode interaction method based on Bayesian theorem |
| CN113619485A (en) * | 2021-07-30 | 2021-11-09 | 北京信息职业技术学院 | Intelligent networking automobile multi-sensor fusion longitudinal control method and device |
| CN113696905A (en) * | 2021-10-08 | 2021-11-26 | 无锡物联网创新中心有限公司 | Vehicle safety driving warning system based on environment perception |
| CN114264356B (en) * | 2021-12-23 | 2024-04-26 | 东方世纪科技股份有限公司 | Dynamic vehicle weighing system and method based on edge calculation |
| CN114194195B (en) * | 2022-02-17 | 2022-07-22 | 北京航空航天大学 | Vehicle control system based on road condition auditory perception |
| CN116129553A (en) * | 2023-04-04 | 2023-05-16 | 北京理工大学前沿技术研究院 | Fusion sensing method and system based on multi-source vehicle-mounted equipment |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN2821727Y (en) * | 2005-08-25 | 2006-09-27 | 宁波市科技园区金榜汽车电子有限公司 | Intelligent automobile tire pressure monitor system |
| CN205930052U (en) * | 2016-08-17 | 2017-02-08 | 西安航空学院 | Intelligence automobile tire monitoring system |
| CN107116977A (en) * | 2017-05-23 | 2017-09-01 | 山东玲珑轮胎股份有限公司 | Intelligent tire |
| CN109612740A (en) * | 2017-10-18 | 2019-04-12 | 宁波轩悦行电动汽车服务有限公司 | Electronic vehicle attitude intelligent safety monitoring system |
| CN110962512A (en) * | 2019-12-24 | 2020-04-07 | 深圳供电局有限公司 | Intelligent monitoring system for automobile tire |
| CN111221342A (en) * | 2020-03-12 | 2020-06-02 | 湖南汽车工程职业学院 | Environment sensing system for automatic driving automobile |
-
2020
- 2020-10-29 CN CN202011177626.1A patent/CN111994066B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN2821727Y (en) * | 2005-08-25 | 2006-09-27 | 宁波市科技园区金榜汽车电子有限公司 | Intelligent automobile tire pressure monitor system |
| CN205930052U (en) * | 2016-08-17 | 2017-02-08 | 西安航空学院 | Intelligence automobile tire monitoring system |
| CN107116977A (en) * | 2017-05-23 | 2017-09-01 | 山东玲珑轮胎股份有限公司 | Intelligent tire |
| CN109612740A (en) * | 2017-10-18 | 2019-04-12 | 宁波轩悦行电动汽车服务有限公司 | Electronic vehicle attitude intelligent safety monitoring system |
| CN110962512A (en) * | 2019-12-24 | 2020-04-07 | 深圳供电局有限公司 | Intelligent monitoring system for automobile tire |
| CN111221342A (en) * | 2020-03-12 | 2020-06-02 | 湖南汽车工程职业学院 | Environment sensing system for automatic driving automobile |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111994066A (en) | 2020-11-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111994066B (en) | Intelligent automobile sensing system based on intelligent tire touch sensing | |
| CN111994068B (en) | Intelligent driving automobile control system based on intelligent tire touch perception | |
| CN111507233B (en) | Multi-mode information fusion intelligent vehicle pavement type identification method | |
| CN112562405A (en) | Radar video intelligent fusion and early warning method and system | |
| EP3784505A1 (en) | Device and method for determining a center of a trailer tow coupler | |
| US20200172110A1 (en) | Vehicle traveling control device, vehicle traveling control method, and program | |
| JP5758759B2 (en) | Vehicle road surface determination device and driving support device | |
| CN105184262A (en) | Road Surface Condition Detection With Recursive Adaptive Learning And Validation | |
| CN110103952B (en) | Vehicle driving method and device based on roadside sensing device and vehicle-road cooperative system | |
| CN113378741B (en) | Auxiliary sensing method and system for aircraft tractor based on multi-source sensor | |
| CN110682907B (en) | Automobile rear-end collision prevention control system and method | |
| CN113096387B (en) | Vehicle-mounted road surface monitoring and early warning method, system, terminal and storage medium | |
| CN109444872B (en) | Pedestrian and vehicle distinguishing method and device, computer equipment and storage medium | |
| CN110161488A (en) | The method and apparatus for executing visual token using vehicle camera system | |
| EP3779501A1 (en) | Vehicle systems and methods utilizing lidar data for road condition estimation | |
| JP6605176B1 (en) | Traffic information generation system | |
| Holzmann et al. | Predictive estimation of the road-tire friction coefficient | |
| Yang et al. | A survey of intelligent tires for tire-road interaction recognition toward autonomous vehicles | |
| Vosahlik et al. | Self-supervised learning of camera-based drivable surface friction | |
| CN112149560B (en) | Lane departure detection method | |
| CN112223962A (en) | Intelligent vehicle control system and method based on road surface touch perception | |
| CN113962301B (en) | Multi-source input signal fused pavement quality detection method and system | |
| Wilson et al. | Terrain roughness identification for high-speed UGVs | |
| Kumar et al. | Internet of Vehicles (IoV) based Framework for Vehicle Degradation using Multidimensional Dynamic Time Warping (MDTW) | |
| KR102501312B1 (en) | Continuous detecting system of road conditions using acceleration sensor, and method for the same |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |