CN116578816B - Two-wheeled balance car driving auxiliary method and system based on image recognition - Google Patents
Two-wheeled balance car driving auxiliary method and system based on image recognition Download PDFInfo
- Publication number
- CN116578816B CN116578816B CN202310529970.XA CN202310529970A CN116578816B CN 116578816 B CN116578816 B CN 116578816B CN 202310529970 A CN202310529970 A CN 202310529970A CN 116578816 B CN116578816 B CN 116578816B
- Authority
- CN
- China
- Prior art keywords
- sensing information
- balance car
- information
- image
- gesture
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 35
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 52
- 230000001133 acceleration Effects 0.000 claims abstract description 29
- 230000004927 fusion Effects 0.000 claims abstract description 27
- 238000013507 mapping Methods 0.000 claims abstract description 22
- 238000000605 extraction Methods 0.000 claims abstract description 13
- 238000003709 image segmentation Methods 0.000 claims abstract description 12
- 239000011159 matrix material Substances 0.000 claims description 34
- 230000006870 function Effects 0.000 claims description 30
- 238000005259 measurement Methods 0.000 claims description 23
- 238000001914 filtration Methods 0.000 claims description 19
- 238000005562 fading Methods 0.000 claims description 10
- 238000004364 calculation method Methods 0.000 claims description 6
- 230000010354 integration Effects 0.000 claims description 5
- 238000004590 computer program Methods 0.000 claims description 4
- 230000004069 differentiation Effects 0.000 claims description 4
- 229910052739 hydrogen Inorganic materials 0.000 claims description 3
- 230000007704 transition Effects 0.000 claims description 3
- 230000010355 oscillation Effects 0.000 description 8
- 230000008569 process Effects 0.000 description 8
- 230000001276 controlling effect Effects 0.000 description 7
- 230000004044 response Effects 0.000 description 5
- 238000012545 processing Methods 0.000 description 3
- 230000009471 action Effects 0.000 description 2
- 230000003044 adaptive effect Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000012888 cubic function Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 238000009825 accumulation Methods 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 230000008878 coupling Effects 0.000 description 1
- 238000010168 coupling process Methods 0.000 description 1
- 238000005859 coupling reaction Methods 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 238000005265 energy consumption Methods 0.000 description 1
- 230000005484 gravity Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000001105 regulatory effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B62—LAND VEHICLES FOR TRAVELLING OTHERWISE THAN ON RAILS
- B62K—CYCLES; CYCLE FRAMES; CYCLE STEERING DEVICES; RIDER-OPERATED TERMINAL CONTROLS SPECIALLY ADAPTED FOR CYCLES; CYCLE AXLE SUSPENSIONS; CYCLE SIDE-CARS, FORECARS, OR THE LIKE
- B62K11/00—Motorcycles, engine-assisted cycles or motor scooters with one or two wheels
- B62K11/007—Automatic balancing machines with single main ground engaging wheel or coaxial wheels supporting a rider
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/11—Complex mathematical operations for solving equations, e.g. nonlinear equations, general mathematical optimization problems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/16—Matrix or vector computation, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/90—Dynamic range modification of images or parts thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/277—Analysis of motion involving stochastic approaches, e.g. using Kalman filters
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20021—Dividing image into blocks, subimages or windows
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/60—Other road transportation technologies with climate change mitigation effect
- Y02T10/72—Electric energy management in electromobility
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Mathematical Physics (AREA)
- Data Mining & Analysis (AREA)
- Computational Mathematics (AREA)
- Pure & Applied Mathematics (AREA)
- Mathematical Analysis (AREA)
- Mathematical Optimization (AREA)
- Software Systems (AREA)
- Algebra (AREA)
- Databases & Information Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Multimedia (AREA)
- Operations Research (AREA)
- Mechanical Engineering (AREA)
- Gyroscopes (AREA)
Abstract
The invention provides a two-wheeled balance car driving auxiliary method and system based on image recognition, wherein the method comprises the steps of fusing first sensing information and second sensing information obtained by an acceleration sensor of a two-wheeled balance car and second sensing information obtained by a gyroscope sensor through a preset sensing information fusion algorithm to determine the gesture information of the two-wheeled balance car; based on image sensing information obtained by an image sensor of the two-wheeled balance car, performing image segmentation and feature extraction on the image sensing information, and determining balance control noise caused by a road surface on which the two-wheeled balance car runs through a mapping function; and according to the gesture information and the image sensing information, combining the balance control noise, cooperatively controlling the speed and gesture of the two-wheel balance car through a preset balance control algorithm, so as to realize balance control of the two-wheel balance car. The method disclosed by the invention can assist the driving of the two-wheeled balance car.
Description
Technical Field
The disclosure relates to the technical field of control, in particular to a two-wheeled balance car driving assisting method and system based on image recognition.
Background
The existing two-wheel balance car driving assistance only focuses on the balance of the car, but the situation of the road surface on which the two-wheel balance car is driven is not analyzed, so that the noise for balance control on uneven road surface is larger.
The patent with the application number of CN201520082909.6 and the name of an image-based two-wheel balance car control system discloses an image sensor arranged on a driving rod of a two-wheel balance car, a gyroscope arranged on a car body, two pressure sensors arranged on a car body pedal, an acceleration sensor arranged on the car body, a control unit arranged on the car body, a driving motor connected with the output end of the control unit and a braking unit connected with the output end of the control unit; the balance car can effectively ensure the driving safety in the driving process, and particularly can accurately control the action of the balance car according to the relevant information of the car body and the human body when the balance car has a certain slope and needs to turn.
The patent with the application number of CN201610269942.9 and the name of a two-wheel self-balancing trolley control system and method based on track information discloses a self-balancing control method for forming a trolley by combining a gyroscope and an acceleration sensor, so that the problems of large vibration interference and drift error of the sensor in the existing self-balancing trolley are well solved, and the accuracy and instantaneity of measuring the attitude dip angle are improved. Meanwhile, the control module can receive accurate and complete road image signals by capturing track information through the camera, so that the balance and the speed of the two-wheel self-balancing trolley at a curve and a narrow road can be controlled conveniently, the stability of the two-wheel self-balancing trolley is improved, and the robustness of a system is enhanced; and the motor driving chip is adopted, so that the peripheral circuit is simplified, the control of the MCU control circuit on the motor is facilitated, and the real-time performance of motor control is improved.
Disclosure of Invention
The embodiment of the disclosure provides a two-wheeled balance car driving assistance method and system based on image recognition, which can at least solve part of problems in the prior art, namely solve the problem that the noise for balance control is large on uneven road surfaces because analysis is not performed on the condition of the road surfaces on which the two-wheeled balance car runs.
In a first aspect of embodiments of the present disclosure,
The two-wheeled balance car driving assisting method based on image recognition comprises the following steps:
based on first sensing information acquired by an acceleration sensor of a two-wheel balance car and second sensing information acquired by a gyroscope sensor, fusing the first sensing information and the second sensing information through a preset sensing information fusion algorithm to determine the posture information of the two-wheel balance car, wherein the sensing information fusion algorithm is constructed based on an improved self-adaptive Kalman filtering algorithm;
Based on image sensing information obtained by an image sensor of the two-wheeled balance car, performing image segmentation and feature extraction on the image sensing information, and determining balance control noise caused by a road surface on which the two-wheeled balance car runs through a mapping function;
And according to the gesture information and the image sensing information, combining the balance control noise, cooperatively controlling the speed and gesture of the two-wheel balance car through a preset balance control algorithm, so as to realize balance control of the two-wheel balance car.
In an alternative embodiment of the present invention,
The fusing the first sensing information and the second sensing information through a preset sensing information fusion algorithm to determine the gesture information of the two-wheel balance car comprises the following steps:
Taking the first sensing information and the second sensing information as a first state variable and the first drift information of the first sensing information and the second drift information of the second sensing information as a second state variable respectively;
determining a measurement noise covariance matrix according to the first sensing information, the predicted acceleration information, the second sensor and the predicted angular velocity information, and updating a filtering gain equation of the sensing information fusion algorithm based on the measurement noise covariance;
And judging whether the filter gain equation is converged by combining the time-varying fading factor and the convergence adjustment coefficient, and if the filter gain equation is converged, fusing the first sensing information and the second sensing information to determine the attitude information of the two-wheeled balance car.
In an alternative embodiment of the present invention,
The measurement noise covariance matrix is determined according to the first sensing information, the predicted acceleration information, the second sensor and the predicted angular velocity information, and the measurement noise covariance matrix is shown in the following formula:
SC(t)＝A(t-1)SC(t-1)+B(t-1)H(t)
Wherein V (T) represents a measurement noise covariance matrix at time T, T represents a time period, Z (T) represents a measurement matrix at time T, H (T) represents a state estimation matrix at time T, and S C (T) represents a state variable matrix at time T;
a (t-1) represents a state transition matrix at the time of t-1, and B (t-1) represents an input control matrix at the time of t-1;
The filter gain equation for updating the sensing information fusion algorithm based on the measured noise covariance is shown in the following formula:
K(t)＝V(t)(1-dt)HT(t)K(t-1)
Wherein K (t) and K (t-1) respectively represent filter gain equations at the time t and the time t-1, d t represents an estimated mean square error value at the time t, and H T (t) represents a transposed matrix of a state estimation matrix at the time t;
And if the filter gain equation is smaller than or equal to a convergence criterion, judging that the filter gain equation converges, wherein the convergence criterion is represented by the following formula:
Wherein P CON represents a convergence criterion, time loss represents a time-varying fading factor, con c represents a convergence adjustment coefficient, P (t|t-1) represents a state estimation of the convergence of the time t-1 to the time t, and K 0 (t) represents an initial filter gain equation corresponding to the time t.
In an alternative embodiment of the present invention,
The image sensing information acquired based on the image sensor of the two-wheeled balance car is subjected to image segmentation and feature extraction, and balance control noise caused by the road surface on which the two-wheeled balance car runs is determined through a mapping function, wherein the method comprises the following steps of:
Converting the image sensing information into a gray image, and vertically and equally dividing the gray image to obtain a plurality of gray subareas;
dividing the gray subregions into a plurality of image intervals according to gray values, carrying out pixel point statistics on the image intervals, determining the global proportion of the pixel points in each image interval to the global pixel points, and constructing a pixel set array;
And carrying out differential absolute value calculation on the pixel set array to determine the probability density distribution value of the pixel set array, and determining balance control noise caused by the road surface on which the two-wheel balance car runs through a mapping function.
In an alternative embodiment of the present invention,
The determining the balance control noise caused by the road surface on which the two-wheeled balance car runs through the mapping function comprises the following steps:
the balance control noise is determined as shown in the following formula:
Wherein NOS BL denotes balance control noise, N, M denotes the number of gray subregions and the number of pixel points, PIX N,2M denotes probability density distribution values of the pixel set array, and C a、Cb、Cc denotes the first fitting coefficient, the second fitting coefficient, and the third fitting coefficient, respectively.
In an alternative embodiment of the present invention,
The step of cooperatively controlling the speed and the gesture of the two-wheel balance car through a preset balance control algorithm according to the gesture information and the image sensing information and combining the balance control noise comprises the following steps:
The speed and the gesture of the two-wheel balance car are controlled according to the following formula:
Wherein u represents the speed and the gesture of the two-wheel balance car, K P、KI、KD represents the nonlinear gain functions of three control links of proportion, integration and differentiation respectively, R represents the gesture information, NOS BL represents the balance control noise, e represents the error value of the input and the system of the controller, K I(R,NOSBL) +. edt represents the gesture controller, And the speed controller is represented by T, the output moment of the left wheel and the right wheel of the two-wheel balance car is represented by T, and the inclination angle of the two-wheel balance car compared with the ground is represented by theta.
In a second aspect of the embodiments of the present disclosure,
The utility model provides a two-wheeled balance car driving auxiliary system based on image recognition, includes:
The system comprises a first unit, a second unit and a third unit, wherein the first unit is used for fusing the first sensing information and the second sensing information through a preset sensing information fusion algorithm based on first sensing information acquired by an acceleration sensor of the two-wheel balance car and second sensing information acquired by a gyroscope sensor to determine the posture information of the two-wheel balance car, and the sensing information fusion algorithm is constructed based on an improved self-adaptive Kalman filtering algorithm;
The second unit is used for carrying out image segmentation and feature extraction on the image sensing information based on the image sensing information acquired by the image sensor of the two-wheeled balance car, and determining balance control noise caused by the road surface on which the two-wheeled balance car runs through a mapping function;
And the third unit is used for cooperatively controlling the speed and the gesture of the two-wheel balance car through a preset balance control algorithm according to the gesture information and the image sensing information and combining the balance control noise so as to realize balance control of the two-wheel balance car.
In a third aspect of the embodiments of the present disclosure,
There is provided an electronic device including:
A processor;
a memory for storing processor-executable instructions;
Wherein the processor is configured to invoke the instructions stored in the memory to perform the method described previously.
In a fourth aspect of embodiments of the present disclosure,
There is provided a computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, implement the method as described above.
The application combines the image information and the gesture information, has more information dimension and can control the vehicle more accurately; meanwhile, the speed and the gesture are controlled, the data processing dimension is reduced, the influence caused by two different control methods is fully considered, and finally the control effect on the two-wheel balance car is improved.
Drawings
Fig. 1 is a flowchart illustrating a two-wheeled balance car driving assistance method based on image recognition according to an embodiment of the present disclosure.
Fig. 2 is a schematic structural diagram of a two-wheeled balance car driving assistance system based on image recognition according to an embodiment of the present disclosure.
Detailed Description
For the purposes of making the objects, technical solutions and advantages of the embodiments of the present disclosure more apparent, the technical solutions of the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present disclosure, and it is apparent that the described embodiments are only some embodiments of the present disclosure, not all embodiments. Based on the embodiments in this disclosure, all other embodiments that a person of ordinary skill in the art would obtain without making any inventive effort are within the scope of protection of this disclosure.
The technical scheme of the present disclosure is described in detail below with specific examples. The following embodiments may be combined with each other, and some embodiments may not be repeated for the same or similar concepts or processes.
Fig. 1 is a schematic flow chart of a two-wheeled balance car driving assisting method based on image recognition according to an embodiment of the disclosure, as shown in fig. 1, the method includes:
s101, based on first sensing information acquired by an acceleration sensor of a two-wheeled balance car and second sensing information acquired by a gyroscope sensor, fusing the first sensing information and the second sensing information through a preset sensing information fusion algorithm to determine the attitude information of the two-wheeled balance car;
Illustratively, the first sensing information is used for indicating the acceleration information acquired by the acceleration sensor, the first sensing information is used for indicating the angular velocity information acquired by the gyroscope sensor, the sensing information fusion algorithm is constructed based on the improved Kalman filtering algorithm and is used for fusing the angular velocity information and the acceleration information and determining the gesture information, such as the inclination angle information, the speed information and the like, of the two-wheel balance car.
For example, the current inclination information of the vehicle body can be described after the data measured by the accelerometer sensor and the gyroscope sensor are converted. But both suffer from the disadvantage of being not negligible. The advantages and disadvantages of both sensors are described below: the accelerometer has good static performance, but when the balance car is in a motion state, an acceleration is generated in the horizontal direction. At this time, the accelerometer measures the combined acceleration of the component of the gravitational acceleration in the horizontal direction and the acceleration of the vehicle body in the horizontal direction. In addition, when the vehicle body is in a motion state, high-frequency noise interference can be generated, and the accuracy of the angle value acquired by the accelerometer is also affected.
The gyroscope has good short-time stability, but static drift can be generated due to the fact that the gyroscope is not perfect in structure and is influenced by factors such as temperature change, magnetic field interference and the like. Although the drift error value is extremely small, the integration operation causes an accumulated error in the angle value. Even very small drift values accumulate to give very large angular errors.
From the above analysis, if the accelerometer or the gyroscope sensor is used alone to collect the attitude information of the vehicle body, the obtained results have larger errors, so that the balance and the motion control of the vehicle body are directly affected. Therefore, the sensing information obtained by the two sensors needs to be fused, so that more accurate vehicle body posture information is obtained.
Illustratively, the sensing information fusion algorithm of the embodiments of the present application is constructed based on an improved adaptive Kalman filtering algorithm. The principle of the Kalman filtering algorithm can be known, the precondition for realizing the optimal estimation of the Kalman filtering algorithm is that the statistical properties of a system model and noise are known, the condition is often not satisfied in the actual application scene of the driving assistance of the two-wheeled balance car, the condition is limited by various uncertainty factors of the actual application environment, and the statistical properties of the noise are not invariable. Whereas the conventional kalman filtering algorithm does not have adaptive characteristics, it is difficult to cope with noise having time-varying characteristics.
The value of the kalman gain depends on the covariance of the a priori estimation error and the measurement noise covariance. When the system gradually enters a steady state, the covariance of the a priori estimation error set at the initial moment is gradually corrected and converged, and the gain K becomes a minimum value at this time because the measurement noise covariance R is a fixed value. If the system suddenly changes at this time, the residual value will increase, and the gain is still a minimum value, which may result in the degradation of the accuracy of the kalman filtering algorithm, or even divergence.
In an alternative embodiment of the present invention,
The fusing the first sensing information and the second sensing information through a preset sensing information fusion algorithm to determine the gesture information of the two-wheel balance car comprises the following steps:
Taking the first sensing information and the second sensing information as a first state variable and the first drift information of the first sensing information and the second drift information of the second sensing information as a second state variable respectively;
determining a measurement noise covariance matrix according to the first sensing information, the predicted acceleration information, the second sensor and the predicted angular velocity information, and updating a filtering gain equation of the sensing information fusion algorithm based on the measurement noise covariance;
And judging whether the filter gain equation is converged by combining the time-varying fading factor and the convergence adjustment coefficient, and if the filter gain equation is converged, fusing the first sensing information and the second sensing information to determine the attitude information of the two-wheeled balance car.
Illustratively, the first sensing information of the embodiments of the present disclosure may include acceleration information, the second sensing information may include angular velocity information, wherein the first sensing information and the second sensing information may be used as a first state variable, and the first drift information of the first sensing information and the second drift information of the second sensing information may be used as a second state variable, specifically,
The first state variable may be expressed asThe second state variable may be expressed asWhere acc represents acceleration information, ang represents angular velocity information, acc_bias represents acceleration drift information, ang_bias represents angular velocity drift information,
In practical application, the predicted sensing information of the next moment can be predicted based on the data acquired by the sensor, and an error value exists between the predicted sensing information and the actually acquired sensing information, and the error value can be used for updating a filtering gain equation to adjust the overall accuracy of the algorithm. Alternatively, the process may be carried out in a single-stage,
The determination of the measurement noise covariance matrix may be represented by the following formula:
SC(t)＝A(t-1)SC(t-1)+B(t-1)H(t)
wherein V (T) represents a measurement noise covariance matrix at the time T, T represents a time period, Z (T) represents a measurement matrix at the time T, H (T) represents a state estimation matrix at the time T, and SC (T) represents a state variable matrix at the time T;
a (t-1) represents a state transition matrix at the time of t-1, and B (t-1) represents an input control matrix at the time of t-1;
in an alternative embodiment of the present invention,
The filter gain equation for updating the sensing information fusion algorithm based on the measured noise covariance is shown in the following formula:
K(t)＝V(t)(1-dt)HT(t)K(t-1)
Wherein K (t) and K (t-1) respectively represent filter gain equations at the time t and the time t-1, d t represents an estimated mean square error value at the time t, and H T (t) represents a transposed matrix of a state estimation matrix at the time t;
And if the filter gain equation is smaller than or equal to a convergence criterion, judging that the filter gain equation converges, wherein the convergence criterion is represented by the following formula:
Wherein P CON represents a convergence criterion, time loss represents a time-varying fading factor, con c represents a convergence adjustment coefficient, P (t|t-1) represents a state estimation of the convergence of the time t-1 to the time t, and K 0 (t) represents an initial filter gain equation corresponding to the time t.
The measurement noise covariance matrix is used for indicating noise information caused in the actual measurement process of the sensor, errors caused by noise can be comprehensively considered through representation of the covariance matrix, whether a gain equation is converged or not is determined through combination of convergence criteria, and accuracy of finally obtained attitude information is improved.
In an alternative embodiment of the present invention,
Fusing the first sensing information and the second sensing information to determine the gesture information of the two-wheel balance car as shown in the following formula:
Wherein R represents the attitude information.
Optionally, the time-varying fading factor is greater than or equal to 1, and is used for adjusting the convergence of the convergence criterion, when the time-varying fading factor is equal to 1, the convergence criterion is more close to convergence, and when the time-varying fading factor is greater than 1, the convergence criterion affects the Kalman filtering gain to increase the Kalman filtering gain, so that the convergence criterion is more close to divergence. The convergence adjustment coefficient is used for further adjusting the convergence of the convergence criterion. By constructing the convergence judging condition of the filter, when the filter diverges, the value of the estimation error is several times higher than that of the theory, and information fusion is carried out on convergence of the filter gain equation, so that the real-time performance of the system is improved, and the stability of the system is ensured.
S102, based on image sensing information obtained by an image sensor of the two-wheeled balance car, performing image segmentation and feature extraction on the image sensing information, and determining balance control noise caused by a road surface on which the two-wheeled balance car runs through a mapping function;
Illustratively, the bumpy road surface also has great interference on the control of the two-wheeled balance car, uneven road surface can increase balance control noise caused by the road surface on which the two-wheeled balance car runs, and noise accumulation can cause final balance control deviation, so that the control effect is reduced.
In an alternative embodiment of the present invention,
The image sensing information acquired based on the image sensor of the two-wheeled balance car is subjected to image segmentation and feature extraction, and balance control noise caused by the road surface on which the two-wheeled balance car runs is determined through a mapping function, wherein the method comprises the following steps of:
Converting the image sensing information into a gray image, and vertically and equally dividing the gray image to obtain a plurality of gray subareas;
dividing the gray subregions into a plurality of image intervals according to gray values, carrying out pixel point statistics on the image intervals, determining the global proportion of the pixel points in each image interval to the global pixel points, and constructing a pixel set array;
And carrying out differential absolute value calculation on the pixel set array to determine the probability density distribution value of the pixel set array, and determining balance control noise caused by the road surface on which the two-wheel balance car runs through a mapping function.
For example, the conversion of the image sensing information into the gray image according to the embodiment of the present application may refer to the existing image binarization conversion, and the vertical equal division of the gray image may divide the image into a plurality of gray sub-areas according to a preset division ratio.
The application can acquire image sensing information through the image sensor of the two-wheeled balance car, and carry out image segmentation and feature extraction on the image sensing information, in particular:
The image sensor can acquire image sensing information and convert the image sensing information into a gray level image, and the gray level image is vertically and equally divided to divide the image into a plurality of gray sub-areas; the LBP histogram of each gray sub-area is counted by selecting a local dynamic threshold value method through an LBP feature extraction algorithm, the LBP histogram can be divided into a plurality of image areas according to gray distribution of 0 to 255, the image areas are counted again, the pixel points of each image area occupy the global proportion, a pixel set array is obtained, differential absolute value calculation is carried out on the pixel set array, the probability density distribution value of the LBP histogram is determined, and road information is represented through the probability density distribution value; the minimum value of the probability density distribution value is 0, at the moment, the texture distribution of the road surface is the minimum, the represented meaning is the state of the road surface which is the flattest, otherwise, the maximum value of the probability density distribution value is 1 (the texture contrast is the maximum), the road surface is rugged, and the balance control noise caused by the road surface on which the two-wheel balance car runs is determined by combining the mapping function.
The probability density distribution value is used as the size judgment basis of the filtering initial value, and the balance control noise is determined by combining the mapping function so as to optimize the sensor signal denoising performance.
In an alternative embodiment, the probability density distribution values for determining the array of pixel sets may be represented by the following formula:
Wherein N, M represents the number of gray sub-regions and the number of pixels, and PIX n,2m represents the probability density distribution value of the pixel set array when the number of gray sub-regions is n and the number of pixels is 2 m.
In an alternative embodiment of the present invention,
The determining the balance control noise caused by the road surface on which the two-wheeled balance car runs through the mapping function comprises the following steps:
the balance control noise is determined as shown in the following formula:
Wherein NOS BL denotes balance control noise, f () denotes a mapping function, N, M denotes the number of gray subregions and the number of pixels, PIX n,2m denotes the number of gray subregions n, and when the number of pixels is 2m, C a、Cb、Cc denotes the first, second, and third fitting coefficients, respectively.
By way of example, a measurement variance is derived by means of a functional map, which measures the extent of the road surface damage, i.e. the balancing control noise. And performing function fitting by least square under MATLAB according to the obtained balance control noise under different pavements to obtain the function of the most suitable initial value of the filter. To improve data accuracy, a cubic function fit may be performed, where C a、Cb、Cc represents the first, second, and third fit coefficients, respectively.
S103, according to the gesture information and the image sensing information, combining the balance control noise, cooperatively controlling the speed and gesture of the two-wheel balance car through a preset balance control algorithm, and realizing balance control of the two-wheel balance car.
For the two-wheel balance car, the nonlinear characteristic of the system is obvious, a state equation contains a plurality of nonlinear functions with high orders, meanwhile, due to the fact that underactuation and instability of the system are obvious, the two-wheel balance car is always in a small-range oscillation state near a balance point, the output of a controller oscillates in a small range along with the output of the controller near the balance point, the oscillation brings unnecessary trouble and difficulty to the balance control problem of the two-wheel balance car, meanwhile, unavoidable errors are generated in accurate control of gesture balance, and due to the fact that the continuous oscillation phenomenon, motor loss and energy consumption are greatly increased, and in the control process of a conventional PID controller, the two-wheel balance car is difficult to achieve high response speed and strong anti-interference characteristics. Therefore, the nonlinear characteristics of the two-wheeled balance car system can be well compensated by adding the proper nonlinear function characteristics into the controller, the performance of system control can be greatly improved, and the response speed and the robustness of the system are improved.
The two-wheeled balance car is a typical nonlinear unstable multivariable complex system, the main problem of motion control is motion balance control, and the two-wheeled balance car is a basic premise that other motion modes can be realized. Because the two-wheeled balance car has strong nonlinearity and instability, an accurate dynamic model of the two-wheeled balance car is a nonlinear equation set with a complex high order, and the two-wheeled balance car becomes extremely complicated in the calculation and processing solving processes. However, the conventional linear PID controller parameters are difficult to determine, or the application range of a certain parameter is limited, the anti-interference capability is weak and the robustness is poor, so that the conventional PID control method needs to be improved appropriately to solve the above problems. The application improves the conventional PID control by analyzing and researching the dynamic characteristics of the system, so that the controller is more in line with the dynamic characteristics of the two-wheeled balance car system and can be well applied to a physical system.
The existing PID controller has overlong response time mainly because the balance of the gesture needs to be ensured firstly in the control process of the running speed of the two-wheel balance car, so that the speed value needs to be regulated for a long time to prevent the gesture from changing greatly; in addition, the two-wheel balance car can cause the oscillation phenomenon of the inclination angle of the machine body when the speed is suddenly changed, the oscillation phenomenon is more obvious when the acceleration is larger, great difficulty is brought to the gesture control problem of the two-wheel balance car, and the gesture control and the speed control of the two-wheel balance car are strong coupling characteristics at the acceleration and deceleration time points, so that the response speed of the system is greatly reduced, namely, the machine body of the two-wheel balance car needs a period of time to adjust the self balance state at each sudden acceleration and deceleration moment.
The balance control algorithm of the application can cooperatively control the speed and the gesture of the two-wheel balance car, thereby realizing the balance control of the two-wheel balance car, reducing the data processing amount compared with the existing mode of respectively controlling the speed and the gesture, and comprehensively considering the integral influence of the speed and the gesture on the balance control of the two-wheel balance car. The balance control algorithm may include an algorithm combining cubic functions with PID control, among others.
In an alternative embodiment of the present invention,
The step of cooperatively controlling the speed and the gesture of the two-wheel balance car through a preset balance control algorithm according to the gesture information and the image sensing information and combining the balance control noise comprises the following steps:
The speed and the gesture of the two-wheel balance car are controlled according to the following formula:
Wherein u represents the speed and the gesture of the two-wheel balance car, K P、KI、KD represents the nonlinear gain functions of three control links of proportion, integration and differentiation respectively, R represents the gesture information, NOS BL represents the balance control noise, e represents the error value of the input and the system of the controller, K I(R,NOSBL) +. edt represents the gesture controller, And the speed controller is represented by T, the output moment of the left wheel and the right wheel of the two-wheel balance car is represented by T, and the inclination angle of the two-wheel balance car compared with the ground is represented by theta.
The application adjusts the traditional PID control into an optimized PID controller integrating the input and error values of the system and the inclination tangent value. The application applies the nonlinear characteristic of the tangent function to the two-wheel balance car gesture controller, can enable the output of the control quantity of the two-wheel balance car to accord with the response condition of the machine body under the action of gravity, and simultaneously, when the inclination angle of the machine body deviates from a larger range of the balance point, the output control quantity can quickly pull the machine body back to the balance state, and when the machine body approaches to the vicinity of the balance point, the control quantity is almost zero. In the control process of the running speed of the two-wheel balance car, attitude information is determined through the information fusion, so that the great change of the attitude is effectively prevented; in addition, the two-wheeled balance car can cause the oscillation phenomenon of the inclination angle of the car body when the speed is suddenly changed, the oscillation phenomenon is more obvious when the acceleration is larger, balance control noise is introduced, and the control quantity is increased in corresponding integral and differential control, so that the oscillation phenomenon can be effectively reduced.
In a second aspect of the embodiments of the present disclosure,
Fig. 2 is a schematic structural diagram of a two-wheeled balance car driving auxiliary system based on image recognition according to an embodiment of the disclosure, including:
The system comprises a first unit, a second unit and a third unit, wherein the first unit is used for fusing the first sensing information and the second sensing information through a preset sensing information fusion algorithm based on first sensing information acquired by an acceleration sensor of the two-wheel balance car and second sensing information acquired by a gyroscope sensor to determine the posture information of the two-wheel balance car, and the sensing information fusion algorithm is constructed based on an improved self-adaptive Kalman filtering algorithm;
The second unit is used for carrying out image segmentation and feature extraction on the image sensing information based on the image sensing information acquired by the image sensor of the two-wheeled balance car, and determining balance control noise caused by the road surface on which the two-wheeled balance car runs through a mapping function;
And the third unit is used for cooperatively controlling the speed and the gesture of the two-wheel balance car through a preset balance control algorithm according to the gesture information and the image sensing information and combining the balance control noise so as to realize balance control of the two-wheel balance car.
The present invention may be a method, apparatus, system, and/or computer program product. The computer program product may include a computer readable storage medium having computer readable program instructions embodied thereon for performing various aspects of the present invention.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present disclosure, and not for limiting the same; although the present disclosure has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present disclosure.
Claims (5)
1. The two-wheeled balance car driving assisting method based on image recognition is characterized by comprising the following steps of:
Based on first sensing information acquired by an acceleration sensor of a two-wheel balance car and second sensing information acquired by a gyroscope sensor, fusing the first sensing information and the second sensing information through a preset sensing information fusion algorithm to determine the posture information of the two-wheel balance car, wherein the sensing information fusion algorithm is constructed based on a self-adaptive Kalman filtering algorithm;
The fusing the first sensing information and the second sensing information through a preset sensing information fusion algorithm to determine the gesture information of the two-wheel balance car comprises the following steps:
Taking the first sensing information and the second sensing information as a first state variable and the first drift information of the first sensing information and the second drift information of the second sensing information as a second state variable respectively;
determining a measurement noise covariance matrix according to the first sensing information, the predicted acceleration information, the second sensing information and the predicted angular velocity information, and updating a filtering gain equation of the sensing information fusion algorithm based on the measurement noise covariance;
judging whether the filter gain equation is converged or not by combining the time-varying fading factor and the convergence adjustment coefficient, and if the filter gain equation is converged, fusing the first sensing information and the second sensing information to determine the attitude information of the two-wheeled balance car;
Based on image sensing information obtained by an image sensor of the two-wheeled balance car, performing image segmentation and feature extraction on the image sensing information, and determining balance control noise caused by a road surface on which the two-wheeled balance car runs through a mapping function;
The image sensing information acquired based on the image sensor of the two-wheeled balance car is subjected to image segmentation and feature extraction, and balance control noise caused by the road surface on which the two-wheeled balance car runs is determined through a mapping function, wherein the method comprises the following steps of:
Converting the image sensing information into a gray image, and vertically and equally dividing the gray image to obtain a plurality of gray subareas;
dividing the gray subregions into a plurality of image intervals according to gray values, carrying out pixel point statistics on the image intervals, determining the global proportion of the pixel points in each image interval to the global pixel points, and constructing a pixel set array;
Carrying out differential absolute value calculation on the pixel set array to determine probability density distribution values of the pixel set array, and determining balance control noise caused by a road surface on which the two-wheel balance car runs through a mapping function;
The determining the balance control noise caused by the road surface on which the two-wheeled balance car runs through the mapping function comprises the following steps:
the balance control noise is determined as shown in the following formula:
Wherein NOS BL represents balance control noise, N, M represents the number of gray subregions and the number of pixel points, respectively, and C a、Cb、Cc represents a first fitting coefficient, a second fitting coefficient, and a third fitting coefficient, respectively;
According to the gesture information and the image sensing information, the balance control noise is combined, and the speed and the gesture of the two-wheel balance car are cooperatively controlled through a preset balance control algorithm, so that balance control of the two-wheel balance car is realized;
the step of cooperatively controlling the speed and the gesture of the two-wheel balance car through a preset balance control algorithm according to the gesture information and the image sensing information and combining the balance control noise comprises the following steps:
The speed and the gesture of the two-wheel balance car are controlled according to the following formula:
Wherein u represents the speed and the gesture of the two-wheel balance car, K P、KI、KD represents the nonlinear gain functions of three control links of proportion, integration and differentiation respectively, R represents the gesture information, NOS BL represents the balance control noise, e represents the error value of the input and the system of the controller, K I(R,NOSBL) +. edt represents the gesture controller, And the speed controller is represented by T, the output moment of the left wheel and the right wheel of the two-wheel balance car is represented by T, and the inclination angle of the two-wheel balance car compared with the ground is represented by theta.
2. The method of claim 1, wherein determining a measurement noise covariance matrix from the first sensed information and predicted acceleration information, the second sensed information and predicted angular velocity information is represented by the formula:
SC(t)＝A(t-1)SC(t-1)+B(t-1)H(t)
Wherein V (t) represents a measurement noise covariance matrix at Time t, time represents a Time period, Z (t) represents a measurement matrix at Time t, H (t) represents a state estimation matrix at Time t, and S C (t) represents a state variable matrix at Time t;
a (t-1) represents a state transition matrix at the time of t-1, and B (t-1) represents an input control matrix at the time of t-1;
The filter gain equation for updating the sensing information fusion algorithm based on the measured noise covariance is shown in the following formula:
K(t)＝V(t)(1-dt)HT(t)K(t-1)
Wherein K (t) and K (t-1) respectively represent filter gain equations at the time t and the time t-1, d t represents an estimated mean square error value at the time t, and H T (t) represents a transposed matrix of a state estimation matrix at the time t;
And if the filter gain equation is smaller than or equal to a convergence criterion, judging that the filter gain equation converges, wherein the convergence criterion is represented by the following formula:
Wherein P CON represents a convergence criterion, time loss represents a time-varying fading factor, con c represents a convergence adjustment coefficient, P (t|t-1) represents a state estimation of the convergence of the time t-1 to the time t, and K 0 (t) represents an initial filter gain equation corresponding to the time t.
3. Two-wheeled balance car driving auxiliary system based on image recognition, characterized by comprising:
The system comprises a first unit, a second unit and a third unit, wherein the first unit is used for fusing the first sensing information and the second sensing information through a preset sensing information fusion algorithm based on first sensing information acquired by an acceleration sensor of the two-wheel balance car and second sensing information acquired by a gyroscope sensor to determine the posture information of the two-wheel balance car, and the sensing information fusion algorithm is constructed based on a self-adaptive Kalman filtering algorithm;
The fusing the first sensing information and the second sensing information through a preset sensing information fusion algorithm to determine the gesture information of the two-wheel balance car comprises the following steps:
Taking the first sensing information and the second sensing information as a first state variable and the first drift information of the first sensing information and the second drift information of the second sensing information as a second state variable respectively;
determining a measurement noise covariance matrix according to the first sensing information, the predicted acceleration information, the second sensing information and the predicted angular velocity information, and updating a filtering gain equation of the sensing information fusion algorithm based on the measurement noise covariance;
judging whether the filter gain equation is converged or not by combining the time-varying fading factor and the convergence adjustment coefficient, and if the filter gain equation is converged, fusing the first sensing information and the second sensing information to determine the attitude information of the two-wheeled balance car;
The second unit is used for carrying out image segmentation and feature extraction on the image sensing information based on the image sensing information acquired by the image sensor of the two-wheeled balance car, and determining balance control noise caused by the road surface on which the two-wheeled balance car runs through a mapping function;
The image sensing information acquired based on the image sensor of the two-wheeled balance car is subjected to image segmentation and feature extraction, and balance control noise caused by the road surface on which the two-wheeled balance car runs is determined through a mapping function, wherein the method comprises the following steps of:
Converting the image sensing information into a gray image, and vertically and equally dividing the gray image to obtain a plurality of gray subareas;
dividing the gray subregions into a plurality of image intervals according to gray values, carrying out pixel point statistics on the image intervals, determining the global proportion of the pixel points in each image interval to the global pixel points, and constructing a pixel set array;
Carrying out differential absolute value calculation on the pixel set array to determine probability density distribution values of the pixel set array, and determining balance control noise caused by a road surface on which the two-wheel balance car runs through a mapping function;
The determining the balance control noise caused by the road surface on which the two-wheeled balance car runs through the mapping function comprises the following steps:
the balance control noise is determined as shown in the following formula:
Wherein NOS BL represents balance control noise, N, M represents the number of gray subregions and the number of pixel points, respectively, and C a、Cb、Cc represents a first fitting coefficient, a second fitting coefficient, and a third fitting coefficient, respectively;
The third unit is used for cooperatively controlling the speed and the gesture of the two-wheel balance car through a preset balance control algorithm according to the gesture information and the image sensing information and combining the balance control noise so as to realize balance control of the two-wheel balance car;
the step of cooperatively controlling the speed and the gesture of the two-wheel balance car through a preset balance control algorithm according to the gesture information and the image sensing information and combining the balance control noise comprises the following steps:
The speed and the gesture of the two-wheel balance car are controlled according to the following formula:
Wherein u represents the speed and the gesture of the two-wheel balance car, K P、KI、KD represents the nonlinear gain functions of three control links of proportion, integration and differentiation respectively, R represents the gesture information, NOS BL represents the balance control noise, e represents the error value of the input and the system of the controller, K I(R,NOSBL) +. edt represents the gesture controller, And the speed controller is represented by T, the output moment of the left wheel and the right wheel of the two-wheel balance car is represented by T, and the inclination angle of the two-wheel balance car compared with the ground is represented by theta.
4. An electronic device, comprising:
A processor;
a memory for storing processor-executable instructions;
wherein the processor is configured to invoke the instructions stored in the memory to perform the method of any of claims 1 to 2.
5. A computer readable storage medium having stored thereon computer program instructions, which when executed by a processor, implement the method of any of claims 1 to 2.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310529970.XA CN116578816B (en) | 2023-05-11 | 2023-05-11 | Two-wheeled balance car driving auxiliary method and system based on image recognition |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310529970.XA CN116578816B (en) | 2023-05-11 | 2023-05-11 | Two-wheeled balance car driving auxiliary method and system based on image recognition |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116578816A CN116578816A (en) | 2023-08-11 |
| CN116578816B true CN116578816B (en) | 2024-08-13 |
Family
ID=87539047
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310529970.XA Active CN116578816B (en) | 2023-05-11 | 2023-05-11 | Two-wheeled balance car driving auxiliary method and system based on image recognition |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116578816B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105759823A (en) * | 2016-04-27 | 2016-07-13 | 长江大学 | Track-information-based control system and method for two-wheel self-balancing dolly |
| CN107340298A (en) * | 2017-06-29 | 2017-11-10 | 广西师范大学 | Balance car system monitoring method based on camera pavement detection |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107543546B (en) * | 2016-06-28 | 2021-03-05 | 沈阳新松机器人自动化股份有限公司 | Attitude calculation method and device for six-axis motion sensor |
| CN108733055A (en) * | 2018-05-18 | 2018-11-02 | 郑州万达科技发展有限公司 | A kind of method and AGV navigation positional devices of Fusion |
-
2023
- 2023-05-11 CN CN202310529970.XA patent/CN116578816B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105759823A (en) * | 2016-04-27 | 2016-07-13 | 长江大学 | Track-information-based control system and method for two-wheel self-balancing dolly |
| CN107340298A (en) * | 2017-06-29 | 2017-11-10 | 广西师范大学 | Balance car system monitoring method based on camera pavement detection |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116578816A (en) | 2023-08-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112758097B (en) | State prediction and estimation method for unmanned vehicle | |
| US8457832B2 (en) | Vehicle body speed estimating device | |
| CN108162955B (en) | A kind of vehicle active anti-rollover control system with time delay identification and compensation function | |
| WO2018173340A1 (en) | System and method for calibrating tire of vehicle | |
| CN108128308A (en) | A kind of vehicle state estimation system and method for distributed-driving electric automobile | |
| WO2022123826A1 (en) | Stochastic model-predictive control of uncertain system | |
| JP2001334921A (en) | Estimating device for surface friction coefficient of vehicle | |
| CN115298045A (en) | Vehicle control device, vehicle control method, and vehicle control system | |
| CN111746537B (en) | Self-adaptive cruise speed control system and method based on road surface flatness and vehicle | |
| CN113256679A (en) | Electronic image stabilization algorithm based on vehicle-mounted rearview mirror system | |
| CN114228431A (en) | Suspension pre-aiming control method, device, equipment and readable storage medium | |
| CN117922561B (en) | Intelligent rolling track deviation correcting system and method based on three-dimensional model | |
| JP2007233985A (en) | Optimal control method of system | |
| CN116578816B (en) | Two-wheeled balance car driving auxiliary method and system based on image recognition | |
| CN113811472A (en) | Touchdown load estimation device, control device, and touchdown load estimation method | |
| JP2019018773A (en) | Control system for suspension | |
| CN114435371A (en) | Road slope estimation method and device | |
| CN116901982B (en) | Switching strategy construction method for high-performance acquisition of vehicle centroid slip angle | |
| CN113853329A (en) | Touchdown load estimation device, control device, and touchdown load estimation method | |
| CN116620306B (en) | Intelligent integrated vehicle control method and system for electric motorcycle | |
| CN115402044B (en) | Vehicle body steady-state roll control method and system based on whole vehicle performance, electronic equipment and vehicle | |
| CN118747354A (en) | Online identification method and online identification system for pneumatic parameters | |
| JPH1095214A (en) | Suspension controlling device | |
| EP4389549A1 (en) | Road condition determining device, road condition determining method, and computer-readable storage medium | |
| US20240116325A1 (en) | Method for controlling variable damping suspensions |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |