CN111667742A - Intelligent auxiliary training method and system for learner-driven vehicle - Google Patents
Intelligent auxiliary training method and system for learner-driven vehicle Download PDFInfo
- Publication number
- CN111667742A CN111667742A CN202010696315.XA CN202010696315A CN111667742A CN 111667742 A CN111667742 A CN 111667742A CN 202010696315 A CN202010696315 A CN 202010696315A CN 111667742 A CN111667742 A CN 111667742A
- Authority
- CN
- China
- Prior art keywords
- learner
- driven vehicle
- training
- machine learning
- path
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
- G09B9/00—Simulators for teaching or training purposes
- G09B9/02—Simulators for teaching or training purposes for teaching control of vehicles or other craft
- G09B9/04—Simulators for teaching or training purposes for teaching control of vehicles or other craft for teaching control of land vehicles
- G09B9/042—Simulators for teaching or training purposes for teaching control of vehicles or other craft for teaching control of land vehicles providing simulation in a real vehicle
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
- G06V20/54—Surveillance or monitoring of activities, e.g. for recognising suspicious objects of traffic, e.g. cars on the road, trains or boats
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
- G09B19/00—Teaching not covered by other main groups of this subclass
- G09B19/16—Control of vehicles or other craft
- G09B19/167—Control of land vehicles
Abstract
The invention relates to an intelligent auxiliary training method and system for a learner-driven vehicle. The method comprises the following steps: acquiring image information of a training field in real time through a camera, and uploading the image information to a machine learning model; detecting the training field image through the model to judge the type of the training field and judge whether the training field image has the learner-driven vehicle; if the instructional car exists, generating a driving path through model calculation and analysis processing; the model judges whether the driving path deviates from a preset path or not by calculating and analyzing the driving path; if the deviation is detected, generating correction information through the model, and sending the information to the learner-driven vehicle; and displaying and playing the correction information in the vehicle, so that a student in the vehicle adjusts the driving action of the student according to the correction information, and the driving path is adjusted to be close to the preset path. The invention can replace coaches to a certain extent, provides intelligent driving practice assistance for trainees in the driving practice process, and improves the training level of the trainees.
Description
Technical Field
The embodiment of the invention relates to the technical field of intelligent assistance in driving school teaching, in particular to an intelligent assistance training method and system for a learner-driven vehicle.
Background
With the development of society progress, the living standard of people is gradually improved, more and more families have cars, so more and more people start to learn the driving technology, the number of coaches in a driving school is relatively small in the face of numerous examinees, the workload of the coaches in the driving school is very large, the same technical actions can be repeated continuously by the same coaches in one day, the psychological change that the coaches are intolerant to is probably caused, and the contradiction between students and the coaches in the driving school is caused. In addition, because the number of the students is large, the traditional teaching of face-to-face is limited in the time and times of learning, and the students cannot select flexibly, so that the improvement of the driving skill level of the students is influenced to a certain extent.
The intelligent products equipped in the existing driving school are basically virtual products, such as a driving training simulator, but the practice effect of the trainees is not obvious because the operation followability of the driving training simulator is different from that of a real vehicle greatly, the practicability of the matched teaching materials is poor and the like; and the driving teaching in different areas is different, and the products can not be changed according to the change of the driving school, so the training simulator of each driving school is almost designed.
In the study of driving skills, a second subject needs a student to train for a large amount of time, especially a backing-up and warehousing project in the second subject, because the training behavior is complex, the student needs to put more energy into practice to increase proficiency, and often needs to finely adjust the direction of a vehicle to enable the vehicle to enter a parking space in the process of backing-up and warehousing, but the student hardly realizes the fine-adjustment direction and the fine-adjustment angle in the beginning, and a coach hardly reminds in real time, and if the mood of the coach is not good, the tension of the student is aggravated, and the improvement of the driving level of the student is not facilitated. Accordingly, there is a need to ameliorate one or more of the problems with the related art solutions described above.
It is noted that this section is intended to provide a background or context to the embodiments of the invention that are recited in the claims. The description herein is not admitted to be prior art by inclusion in this section.
Disclosure of Invention
An object of the embodiments of the present invention is to provide an intelligent training assisting method and system for a learner-driven vehicle, so as to overcome one or more problems caused by the limitations and disadvantages of the related art at least to a certain extent.
According to a first aspect of the embodiment of the invention, an intelligent auxiliary training method for a learner-driven vehicle is provided, which comprises the following steps:
acquiring image information of the training field in real time through a camera arranged above the training field, and uploading the image information of the training field to a machine learning model;
detecting the training field image through a target detection model pre-established in the machine learning model to judge the type of the training field and judge whether a learner-driven vehicle exists in the training field image;
the machine learning model detects the training field image by adopting a pre-established deep learning-based learner-driven vehicle target detection model;
if the learner-driven vehicle exists, calculating and analyzing a plurality of moving positions of the learner-driven vehicle through the machine learning model so as to generate a driving path for the plurality of moving positions;
the machine learning model calculates and analyzes the driving path to judge whether the driving path deviates from a preset path;
the machine learning model is used by solidifying parameters of the machine learning model after learning and training of the driving path of the learner-driven vehicle for multiple times;
if the driving path deviates, generating correction information according to the deviation degree of the driving path through the machine learning model, and sending the correction information to a receiving module on the learner-driven vehicle;
the receiving module sends the received correction information to a control module in the learner-driven vehicle, and the control module controls the correction information to be displayed and played in the learner-driven vehicle, so that a learner in the learner-driven vehicle adjusts the driving action of the learner according to the correction information, and the driving path is adjusted by the learner to be close to the preset path;
the correction information comprises voice prompt information and video demonstration information, and the video demonstration information is demonstration animation information of the pre-rotation direction and angle of the steering wheel in the learner-driven vehicle;
after the learner-driven vehicle finishes a single training project, the machine learning model stores the actual driving path data of the learner-driven vehicle;
and comparing the stored actual driving path with a preset path to generate path comparison information, and sending the path comparison information to a terminal for the student to correct errors in time.
In an embodiment of the present invention, the method further includes:
monitoring the traveling speed of the learner-driven vehicle in real time in the process of finishing the training project by the learner-driven vehicle, and judging whether the traveling speed value is within a preset range or not;
and if the travelling speed exceeds the preset range, generating a prompt message and sending the prompt message to the learner-driven vehicle.
In an embodiment of the present invention, the method further includes:
and when the driving path of the learner-driven vehicle deviates and the preset path exceeds the preset deviation and the machine learning model identifies that the learner-driven vehicle is in a dangerous state, sending a brake signal to the learner-driven vehicle so as to control the learner-driven vehicle to stop in time.
In an embodiment of the present invention, the route comparison information is comparison video information between an actual driving video of the learner-driven vehicle and a driving video of a preset virtual learner-driven vehicle.
In an embodiment of the present invention, the types of the training fields include: the system comprises a backing and parking training item, a side parking training item, an S-turn training item and a quarter turn training item.
According to a second aspect of the embodiments of the present invention, there is provided an intelligent auxiliary training system for a learner-driven vehicle, the system including:
the camera module is arranged above the training field and used for acquiring image information of the training field in real time and uploading the image information of the training field to the machine learning model;
the machine learning model is used for detecting the training field image through a target detection model pre-established in the machine learning model so as to judge the type of the training field and judge whether a learner-driven vehicle exists in the training field image; the machine learning model detects the training field image by adopting a pre-established deep learning-based learner-driven vehicle target detection model; if the learner-driven vehicle exists, calculating and analyzing a plurality of moving positions of the learner-driven vehicle through the machine learning model so as to generate a driving path for the plurality of moving positions;
the machine learning model is used for calculating and analyzing the driving path to judge whether the driving path deviates from a preset path; the machine learning model is used by solidifying parameters of the machine learning model after learning and training of the driving path of the learner-driven vehicle for multiple times; if the driving path deviates, generating correction information according to the deviation degree of the driving path through the machine learning model, and sending the correction information to a receiving module on the learner-driven vehicle; after the learner-driven vehicle completes a single training project, the machine learning model is used for storing the actual driving path data of the learner-driven vehicle;
the learner-driven vehicle module comprises a receiving module and a control module, wherein the receiving module is used for sending the received correction information to the control module in the learner-driven vehicle; the control module is used for controlling the correction information to be displayed and played in the learner-driven vehicle, so that a learner in the learner-driven vehicle adjusts the driving action of the learner according to the correction information, and the driving path is adjusted by the learner to be close to the preset path;
the correction information comprises voice prompt information and video demonstration information, and the video demonstration information is demonstration animation information of the pre-rotation direction and angle of the steering wheel in the learner-driven vehicle;
and the comparison module is used for comparing the stored actual driving path with a preset path to generate path comparison information and sending the path comparison information to the terminal so as to allow the student to correct the error in time.
In an embodiment of the present invention, the machine learning module is configured to monitor a traveling speed of the learner-driven vehicle in real time during a process of completing the training project by the learner-driven vehicle, and determine whether the traveling speed value is within a preset range; and if the travelling speed exceeds the preset range, generating a prompt message and sending the prompt message to the learner-driven vehicle.
In an embodiment of the invention, the machine learning model is further configured to send a brake signal to the learner-driven vehicle to control the learner-driven vehicle to stop in time when the driving path of the learner-driven vehicle deviates from the preset path and exceeds a preset deviation amount and the learner-driven vehicle is identified to be in a dangerous state by the machine learning model.
In an embodiment of the present invention, the route comparison information is comparison video information between an actual driving video of the learner-driven vehicle and a driving video of a preset virtual learner-driven vehicle.
In an embodiment of the present invention, the types of the training fields include: the system comprises a backing and parking training item, a side parking training item, an S-turn training item and a quarter turn training item.
The technical scheme provided by the embodiment of the invention can have the following beneficial effects:
in the embodiment of the invention, according to the intelligent auxiliary training method and the system for the learner-driven vehicle, the image information is captured in real time by the camera arranged above the training field, the image information is uploaded to the machine learning model, then the actual driving path of the learner-driven vehicle is calculated and analyzed by the machine learning model, the offset between the actual driving path and the preset path is obtained, and the correction information is generated according to the offset, the correction information can be displayed and played in the learner-driven vehicle in real time, a learner can intuitively adjust the steering and the angle of the steering wheel according to the video demonstration in the correction information, meanwhile, the learner-driven vehicle can also carry out voice prompt, the learner can better correct the driving action of the learner by the combined action of the video demonstration and the voice prompt, the intelligent auxiliary training method can replace the coach to a certain extent, the intelligent driving training aid is provided for the trainee in the driving training process, and the training level of the trainee can be improved to a certain extent.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and together with the description, serve to explain the principles of the disclosure. It is to be understood that the drawings in the following description are merely exemplary of the disclosure, and that other drawings may be derived from those drawings by one of ordinary skill in the art without the exercise of inventive faculty.
FIG. 1 illustrates a flow chart of a method for intelligent assisted training of a learner-driven vehicle in an exemplary embodiment of the present invention;
FIG. 2 illustrates a block diagram of an intelligent auxiliary training system of a learner-driven vehicle in an exemplary embodiment of the present invention;
FIG. 3 is a schematic view of steering wheel steering and angle in a display screen of a learner-driven vehicle in an exemplary embodiment of the present invention;
FIG. 4 is a schematic diagram illustrating a predetermined path of travel of a learner-driven vehicle in a driving field in accordance with an exemplary embodiment of the present invention;
FIG. 5 is a schematic diagram illustrating an actual path of travel of a learner-driven vehicle through a training field in an exemplary embodiment of the present invention;
fig. 6 is a schematic diagram illustrating a comparison path between an actual travel path and a preset path of a learner-driven vehicle in a training field according to an exemplary embodiment of the present invention.
Detailed Description
Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
Furthermore, the drawings are merely schematic illustrations of embodiments of the invention, which are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus their repetitive description will be omitted. Some of the block diagrams shown in the figures are functional entities and do not necessarily correspond to physically or logically separate entities.
The embodiment of the example firstly provides an intelligent auxiliary training method for a learner-driven vehicle. Referring to fig. 1, the method may include:
and S101, acquiring image information of the training field in real time through a camera arranged above the training field, and uploading the image information of the training field to a machine learning model.
Step S102, detecting the training field image through a target detection model pre-established in the machine learning model to judge the type of the training field and judge whether a learner-driven vehicle exists in the training field image; the machine learning model detects the training field image by adopting a pre-established deep learning-based learner-driven vehicle target detection model.
Step S103, if there is a learner-driven vehicle, calculating and analyzing a plurality of moving positions of the learner-driven vehicle through the machine learning model, so as to generate a driving path for the plurality of moving positions.
Step S104, the machine learning model calculates and analyzes the driving path to judge whether the driving path deviates from a preset path; the machine learning model is used by solidifying parameters of the machine learning model after learning and training of the driving path of the learner-driven vehicle for multiple times.
And step S105, if the driving path deviates, generating correction information according to the deviation degree of the driving path through the machine learning model, and sending the correction information to a receiving module on the learner-driven vehicle.
Step S106, the receiving module sends the received correction information to a control module in the instructional car, and the control module controls the correction information to be displayed and played in the instructional car, so that a learner in the instructional car adjusts the driving action of the learner according to the correction information, and the driving path is adjusted by the learner to be close to the preset path; the correction information comprises voice prompt information and video demonstration information, and the video demonstration information is demonstration animation information of the pre-rotation direction and angle of the steering wheel in the learner-driven vehicle.
And S107, after the learner-driven vehicle finishes a single training project, storing the actual driving path data of the learner-driven vehicle by the machine learning model.
And S108, comparing the stored actual driving path with a preset path to generate path comparison information, and sending the path comparison information to a terminal for the student to correct errors in time.
According to the intelligent auxiliary training method for the learner-driven vehicle, image information is captured in real time through the camera arranged above the training field and uploaded to the machine learning model, then the actual driving path of the learner-driven vehicle is calculated and analyzed through the machine learning model to obtain the offset between the actual driving path and the preset path, and correction information is generated according to the offset, the correction information can be displayed and played in the learner-driven vehicle in real time, a learner can intuitively adjust the steering and the angle of the steering wheel according to video demonstration in the correction information, voice prompt can be carried out in the learner-driven vehicle, the learner can better correct the driving action of the learner by the combined action of the video demonstration and the voice prompt, the intelligent auxiliary training method can replace the trainer to a certain extent, and intelligent vehicle-driving assistance is provided for the learner in the vehicle-driving process, and the training level of the trainees can be improved to a certain extent.
Next, each part of the above-mentioned intelligent training assisting method for an instructional car in the present exemplary embodiment will be described in more detail with reference to fig. 1 to 6.
In step S101, the image information of the training field is obtained in real time by a camera disposed above the training field, and is uploaded to a machine learning model.
In an example, the cameras are fixedly installed above the training fields, when the trainees train the subject two-item, the cameras above the training fields are opened, the cameras can acquire image information of the training fields in real time, and the acquired image information of the training fields is uploaded to the machine learning model, where the image information can be understood as picture frame data, but the invention is not limited thereto. Firstly, reading data by a machine learning model, namely collecting the data; and processing and correcting the data in advance, such as feature extraction, feature dimension reduction, feature null value processing, feature conversion, feature normalization, target value null value processing, target value conversion and the like; in the model testing stage, a cross validation method can be selected firstly, and data are divided in advance; after the data is processed, a training model is built by using the data, and the evaluation of the model has a plurality of parameters, such as score, precision ratio, recall ratio and the like. The details can be understood with reference to the prior art, and are not described herein.
In step S102, detecting the training field image through a target detection model pre-established in the machine learning model to determine a type of the training field and determine whether a learner-driven vehicle exists in the training field image; the machine learning model detects the training field image by adopting a pre-established deep learning-based learner-driven vehicle target detection model.
For example, the step of establishing the learner-driven vehicle target detection model based on deep learning may be: constructing a target detection deep learning network of the learner-driven vehicle; the learner-driven vehicle target detection deep learning network comprises: the learner-driven vehicle comprises a learner-driven vehicle feature extractor consisting of 8 convolution layers, 8 ReLU active layers and 3 pooling layers, and a first convolution layer and a second convolution layer which are connected with the learner-driven vehicle feature extractor; acquiring training field image samples with different angles, illumination and image quality; marking the position of a training starting area of the learner-driven vehicle in the training field image sample by using a rectangular frame, and recording the coordinate information of the rectangular frame; generating a learner-driven vehicle target detection training data set by taking each training field image sample and coordinate information of a rectangular frame in the training field image sample as a group of training data; training the learner-driven vehicle target detection deep learning network by using the generated learner-driven vehicle target detection training data set, and obtaining a learner-driven vehicle target detection model based on deep learning after updating parameters. Reference may be made in particular to the prior art. After the target detection model completes the detection of the image of the training field, the type to which the training field belongs is obtained, and in one example, the type to which the training field belongs includes: the intelligent parking training system comprises a backing garage entry training project, a side parking training project, an S-turn training project and a quarter turn training project, wherein each project can be trained through the intelligent auxiliary training method provided by the embodiment.
In step S103, if there is a learner-driven vehicle, a plurality of moving positions of the learner-driven vehicle are calculated and analyzed by the machine learning model, so as to generate a driving path for the plurality of moving positions.
In an example, the machine learning model may receive image information of a training field shot in real time through a camera, because the training field has a training vehicle, and the machine learning model can be used by training through a large number of positive and negative samples of the training vehicle, so as to solidify parameters thereof for use, the positive sample is a sample corresponding to a category that is desired to be correctly classified. For example, when a learner trains in a backing and warehousing field, the camera sequentially uploads a frame of shot images to the machine learning module, the machine learning module calculates and processes each frame of received image, and a driving path is formed by the images and the previous frame of received image, so that the driving path is generated from the beginning of backing the learner-driven vehicle and changes in real time along with the movement of the learner-driven vehicle.
In step S104, the machine learning model calculates and analyzes the driving path to determine whether the driving path deviates from a preset path; the machine learning model is used by solidifying parameters of the machine learning model after learning and training of the driving path of the learner-driven vehicle for multiple times.
The example is that the machine learning model is used by solidifying parameters of a large number of learner-driven vehicles after learning and training the driving paths of the learner-driven vehicles, and can be understood as that the machine learning model can calculate how the learner-driven vehicles can successfully fall into a garage after training of a large number of samples, or how to adjust the steering and the angle of a steering wheel so that the learner-driven vehicles fall into the garage under the condition of deviating from an original route, and the adjusting mode is converted into voice for assisting a learner to train in the process of the learner-driven vehicles. As shown in fig. 4, when the learner drives the vehicle to perform the parking training, the machine learning model calculates and analyzes the driving path of the learner-driven vehicle in real time by capturing the obtained image information, and determines whether the driving path deviates from the preset path, the machine learning model is equivalent to that a coach stares at the learner to practice the vehicle in the driving school, and when the driving path of the learner is different from the regular path, the coach reminds the learner to perform direction adjustment timely so as to enable the learner-driven vehicle to approach the regular path.
In step S105, if the driving path deviates, generating a correction information according to the degree of deviation of the driving path through the machine learning model, and sending the correction information to a receiving module on the learner-driven vehicle.
For example, if the machine learning model calculates that the driving path of the learner-driven vehicle deviates or the learner-driven vehicle continues to drive along the driving path, and the training subject fails, the machine learning model generates correction information according to the deviation degree of the driving path relative to the preset path, and sends the correction information to the learner-driven vehicle. For example, when a learner is practicing backing up and warehousing, if the learner continues to drive along the route due to too early steering, the learner-driven vehicle is determined to press the garage line, and at the moment, the machine learning model generates correction information through calculation and analysis, and the correction information is sent to a receiving module on the learner-driven vehicle to prompt the learner to adjust the direction.
In step S106, the receiving module sends the received correction information to a control module in the learner-driven vehicle, and the control module controls the correction information to be displayed and played in the learner-driven vehicle, so that a learner in the learner-driven vehicle adjusts the driving action of the learner according to the correction information, and the driving path is adjusted by the learner to be close to the preset path; the correction information comprises voice prompt information and video demonstration information, and the video demonstration information is demonstration animation information of the pre-rotation direction and angle of the steering wheel in the learner-driven vehicle.
The instructional car is provided with a control module for controlling a display screen in the instructional car to display correction information and controlling a loudspeaker in the car to perform voice prompt. For example, when the trainee is practicing the backing garage entry project, if the trainee continues to drive along the route because the trainee drives the steering wheel too early, the trainee can press the garage line, thereby causing the failure of subject training, and at the moment, the machine learning model generates a correction information through calculation and analysis, the correction information is sent to a receiving module on the instructional car, the receiving module sends the correction information to a control module in the car, the control module controls a display screen to display the correction information, as shown in figure 3, the displayed content comprises video demonstration information in the correction information, the video demonstration information is demonstration animation information of the pre-rotation direction and angle of the steering wheel in the learner-driven vehicle, the rotating direction of the steering wheel can be marked by an arrow, and how to operate the steering wheel can be displayed to a student more intuitively through video display in the display screen, so that the training efficiency of the student is improved; simultaneously, the learner-driven vehicle sends out voice prompt according to the voice prompt information in the correction information, if the learner-driven vehicle returns to the disc for half a circle, after the learner-driven vehicle travels a distance according to the path, the machine learning model generates another correction information through calculation and analysis and sends the correction information to the learner-driven vehicle, at the moment, the display screen displays the steering wheel video animations in different angles and rotating directions, and meanwhile, the learner-driven vehicle is matched with the voice prompt, if the learner-driven vehicle is dead to the right, the learner-driven vehicle is stopped in the garage correctly through the video prompt and the voice prompt in the display screen. It should be noted that the content of the voice prompt message in the corrective message can be selectively set according to the actual situation, and is not limited to that described in this embodiment.
In steps S107 and S108, after the learner-driven vehicle completes a single training project, the machine learning model stores actual driving path data of the learner-driven vehicle; and comparing the stored actual driving path with a preset path to generate path comparison information, and sending the path comparison information to a terminal for the student to correct errors in time.
For example, as shown in fig. 5 and 6, after a trainee completes a training project, a storage module in the machine learning model stores an actual driving path of the training car and compares the actual driving path with a preset path to generate a path ratio pair information, in one example, the path ratio pair information is comparison video information between an actual driving video of the learner-driven car and a driving video of a preset virtual learner-driven car, specifically, a camera arranged above a training field can shoot the actual path of the training project of the learner-driven car and store the video, the video is the actual driving video of the learner-driven car, the actual driving video and the preset virtual driving video can be compared by the comparison module to generate a comparison video, and the comparison video information is sent to a terminal, such as a mobile phone of the trainee, the student can more intuitively see the problems of the student in training through watching the path comparison video, so that the rapid progress of the student is facilitated.
After step S103, the training method further includes:
and step S1031, monitoring the traveling speed of the learner-driven vehicle in real time in the process that the learner-driven vehicle completes the training project, and judging whether the traveling speed value is within a preset range.
And S1032, if the travelling speed exceeds the preset range, generating a prompt message and sending the prompt message to the learner-driven vehicle.
For example, in an examination of subject two, particularly when the vehicle is backed up and parked at a side, the slower the vehicle speed is, the easier the learner can adjust the state of the vehicle, so that the driving speed of the learner-driven vehicle can be monitored in real time in the process of completing the training project, and whether the driving speed value is within a preset range is judged, if the driving speed of the learner-driven vehicle is greater than 5km/h, the learner-driven vehicle will prompt the learner to slow down by voice, and the specific preset range can be set according to actual conditions, without limitation.
After step S104, the training method further includes:
and S1041, when the driving path of the learner-driven vehicle deviates from the preset path and exceeds the preset deviation and the learner-driven vehicle is identified to be in a dangerous state through the machine learning model, sending a brake signal to the learner-driven vehicle so as to control the learner-driven vehicle to stop in time.
The example, when the learner is at the in-process of driving practice, for preventing unexpected emergence, if the learner steps on the brake carelessly and makes the learner-driven vehicle emergence of the circumstances such as out of control, the route is predetermine in the skew of learner-driven vehicle is too much, or the skew is unusual, and think that this learner-driven vehicle is in dangerous state through the judgement of machine learning model, will send brake signal to the receiving module in the learner-driven vehicle, brake module in the learner-driven vehicle will receive the brake signal that receiving module sent to control this learner-driven vehicle and stop, with the safety of guaranteeing the learner in the driver's driving practice.
The embodiment of the example also provides an intelligent auxiliary training system of the learner-driven vehicle. Referring to fig. 2, the system may include a camera module, a machine learning model, a learner vehicle module, and a comparison module.
The camera module is arranged above the training field and used for acquiring image information of the training field in real time and uploading the image information of the training field to the machine learning model.
The machine learning model is used for detecting the training field image through a target detection model pre-established in the machine learning model so as to judge the type of the training field and judge whether a learner-driven vehicle exists in the training field image; the machine learning model detects the training field image by adopting a pre-established deep learning-based learner-driven vehicle target detection model; if the learner-driven vehicle exists, calculating and analyzing a plurality of moving positions of the learner-driven vehicle through the machine learning model so as to generate a driving path for the plurality of moving positions;
the machine learning model is used for calculating and analyzing the driving path to judge whether the driving path deviates from a preset path; the machine learning model is used by solidifying parameters of the machine learning model after learning and training of the driving path of the learner-driven vehicle for multiple times; if the driving path deviates, generating correction information according to the deviation degree of the driving path through the machine learning model, and sending the correction information to a receiving module on the learner-driven vehicle; and after the learner-driven vehicle finishes a single training project, the machine learning model is used for storing the actual driving path data of the learner-driven vehicle.
The learner-driven vehicle module is used for sending the received correction information to a control module in the learner-driven vehicle; the control module is used for controlling the correction information to be displayed and played in the learner-driven vehicle, so that a learner in the learner-driven vehicle adjusts the driving action of the learner according to the correction information, and the driving path is adjusted by the learner to be close to the preset path; the correction information comprises voice prompt information and video demonstration information, and the video demonstration information is demonstration animation information of the pre-rotation direction and angle of the steering wheel in the learner-driven vehicle.
The comparison module is used for comparing the stored actual driving path with a preset path to generate path comparison information and sending the path comparison information to the terminal so as to allow the student to correct the error in time.
The detailed description can be understood by referring to the above embodiments, which are not repeated herein.
In one example, the machine learning module is configured to monitor a traveling speed of the learner-driven vehicle in real time during a process of completing the training project by the learner-driven vehicle, and determine whether the traveling speed value is within a preset range; and if the travelling speed exceeds the preset range, generating a prompt message and sending the prompt message to the learner-driven vehicle.
In one example, the machine learning model is further configured to send a brake signal to the learner-driven vehicle to control the learner-driven vehicle to stop in time when the driving path of the learner-driven vehicle deviates from the preset path and exceeds the preset deviation and the learner-driven vehicle is identified to be in a dangerous state by the machine learning model.
In one example, the route comparison information is comparison video information of an actual driving video of the learner-driven vehicle and a driving video of a preset virtual learner-driven vehicle.
In one example, the type to which the training field belongs includes: the system comprises a backing and parking training item, a side parking training item, an S-turn training item and a quarter turn training item.
According to the intelligent auxiliary training method and the system for the learner-driven vehicle, the image information is captured in real time through the camera arranged above the training field, the image information is uploaded to the machine learning model, then the offset between the actual driving path and the preset path is obtained through calculation and analysis of the actual driving path of the learner-driven vehicle through the machine learning model, the correction information is generated according to the offset, the correction information can be displayed and played in the learner-driven vehicle in real time, a learner can intuitively adjust the steering and the angle of the steering wheel according to the video demonstration in the correction information, meanwhile, voice prompt is carried out in the learner-driven vehicle, the learner can better correct the driving action of the learner through the combined action of the video demonstration and the voice prompt, the intelligent auxiliary training method can replace a coach to a certain extent, and intelligent vehicle-driving assistance is provided for the learner in the vehicle-driving process, and the training level of the trainees can be improved to a certain extent.
It is to be understood that the terms "central," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," and the like in the foregoing description are used for indicating or indicating the orientation or positional relationship illustrated in the drawings, and are used merely for convenience in describing embodiments of the present invention and for simplifying the description, and do not indicate or imply that the device or element so referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore should not be construed as limiting the embodiments of the present invention.
Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of the embodiments of the present invention, "a plurality" means two or more unless specifically limited otherwise.
In the embodiments of the present invention, unless otherwise explicitly specified or limited, the terms "mounted," "connected," "fixed," and the like are to be construed broadly, e.g., as being fixedly connected, detachably connected, or integrated; can be mechanically or electrically connected; either directly or indirectly through intervening media, either internally or in any other relationship. The specific meanings of the above terms in the present invention can be understood by those skilled in the art according to specific situations.
In embodiments of the invention, unless expressly stated or limited otherwise, the first feature "on" or "under" the second feature may comprise the first and second features being in direct contact, or the first and second features being in contact, not directly, but via another feature therebetween. Also, the first feature being "on," "above" and "over" the second feature includes the first feature being directly on and obliquely above the second feature, or merely indicating that the first feature is at a higher level than the second feature. A first feature being "under," "below," and "beneath" a second feature includes the first feature being directly under and obliquely below the second feature, or simply meaning that the first feature is at a lesser elevation than the second feature.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples described in this specification can be combined and combined by those skilled in the art.
Other embodiments of the invention will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the invention and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the invention being indicated by the following claims.
Claims (10)
1. An intelligent auxiliary training method for a learner-driven vehicle is characterized by comprising the following steps:
acquiring image information of the training field in real time through a camera arranged above the training field, and uploading the image information of the training field to a machine learning model;
detecting the training field image through a target detection model pre-established in the machine learning model to judge the type of the training field and judge whether a learner-driven vehicle exists in the training field image;
the machine learning model detects the training field image by adopting a pre-established deep learning-based learner-driven vehicle target detection model;
if the learner-driven vehicle exists, calculating and analyzing a plurality of moving positions of the learner-driven vehicle through the machine learning model so as to generate a driving path for the plurality of moving positions;
the machine learning model calculates and analyzes the driving path to judge whether the driving path deviates from a preset path;
the machine learning model is used by solidifying parameters of the machine learning model after learning and training of the driving path of the learner-driven vehicle for multiple times;
if the driving path deviates, generating correction information according to the deviation degree of the driving path through the machine learning model, and sending the correction information to a receiving module on the learner-driven vehicle;
the receiving module sends the received correction information to a control module in the learner-driven vehicle, and the control module controls the correction information to be displayed and played in the learner-driven vehicle, so that a learner in the learner-driven vehicle adjusts the driving action of the learner according to the correction information, and the driving path is adjusted by the learner to be close to the preset path;
the correction information comprises voice prompt information and video demonstration information, and the video demonstration information is demonstration animation information of the pre-rotation direction and angle of the steering wheel in the learner-driven vehicle;
after the learner-driven vehicle finishes a single training project, the machine learning model stores the actual driving path data of the learner-driven vehicle;
and comparing the stored actual driving path with a preset path to generate path comparison information, and sending the path comparison information to a terminal for the student to correct errors in time.
2. The intelligent auxiliary training method for instructional cars according to claim 1, further comprising:
monitoring the traveling speed of the learner-driven vehicle in real time in the process of finishing the training project by the learner-driven vehicle, and judging whether the traveling speed value is within a preset range or not;
and if the travelling speed exceeds the preset range, generating a prompt message and sending the prompt message to the learner-driven vehicle.
3. The intelligent auxiliary training method for instructional cars according to claim 2, further comprising:
and when the driving path of the learner-driven vehicle deviates and the preset path exceeds the preset deviation and the machine learning model identifies that the learner-driven vehicle is in a dangerous state, sending a brake signal to the learner-driven vehicle so as to control the learner-driven vehicle to stop in time.
4. The intelligent auxiliary training method for the learner-driven vehicle as claimed in claim 3, wherein the path comparison information is comparison video information of an actual driving video of the learner-driven vehicle and a driving video of a preset virtual learner-driven vehicle.
5. The intelligent auxiliary training method for instructional cars of claim 4, wherein the types to which the training fields belong comprise: the system comprises a backing and parking training item, a side parking training item, an S-turn training item and a quarter turn training item.
6. The utility model provides a learner-driven vehicle intelligence training aid system which characterized in that, this system includes:
the camera module is arranged above the training field and used for acquiring image information of the training field in real time and uploading the image information of the training field to the machine learning model;
the machine learning model is used for detecting the training field image through a target detection model pre-established in the machine learning model so as to judge the type of the training field and judge whether a learner-driven vehicle exists in the training field image; the machine learning model detects the training field image by adopting a pre-established deep learning-based learner-driven vehicle target detection model; if the learner-driven vehicle exists, calculating and analyzing a plurality of moving positions of the learner-driven vehicle through the machine learning model so as to generate a driving path for the plurality of moving positions;
the machine learning model is used for calculating and analyzing the driving path to judge whether the driving path deviates from a preset path; the machine learning model is used by solidifying parameters of the machine learning model after learning and training of the driving path of the learner-driven vehicle for multiple times; if the driving path deviates, generating correction information according to the deviation degree of the driving path through the machine learning model, and sending the correction information to a receiving module on the learner-driven vehicle; after the learner-driven vehicle completes a single training project, the machine learning model is used for storing the actual driving path data of the learner-driven vehicle;
the learner-driven vehicle module comprises a receiving module and a control module, wherein the receiving module is used for sending the received correction information to the control module in the learner-driven vehicle; the control module is used for controlling the correction information to be displayed and played in the learner-driven vehicle, so that a learner in the learner-driven vehicle adjusts the driving action of the learner according to the correction information, and the driving path is adjusted by the learner to be close to the preset path;
the correction information comprises voice prompt information and video demonstration information, and the video demonstration information is demonstration animation information of the pre-rotation direction and angle of the steering wheel in the learner-driven vehicle;
and the comparison module is used for comparing the stored actual driving path with a preset path to generate path comparison information and sending the path comparison information to the terminal so as to allow the student to correct the error in time.
7. The intelligent auxiliary training system for the learner-driven vehicle as claimed in claim 6, wherein the machine learning module is configured to monitor the traveling speed of the learner-driven vehicle in real time and determine whether the traveling speed value is within a preset range in the process that the learner-driven vehicle completes the training program; and if the travelling speed exceeds the preset range, generating a prompt message and sending the prompt message to the learner-driven vehicle.
8. The intelligent auxiliary training system of the learner-driven vehicle as claimed in claim 7, wherein the machine learning model is further configured to send a brake signal to the learner-driven vehicle to control the learner-driven vehicle to stop in time when the driving path of the learner-driven vehicle deviates from the preset path and exceeds a preset deviation amount and the learner-driven vehicle is identified to be in a dangerous state by the machine learning model.
9. The intelligent auxiliary training system for the learner-driven vehicle as claimed in claim 8, wherein the path comparison information is comparison video information of an actual driving video of the learner-driven vehicle and a driving video of a preset virtual learner-driven vehicle.
10. The intelligent assistive training system of claim 9, wherein the type of training field comprises: the system comprises a backing and parking training item, a side parking training item, an S-turn training item and a quarter turn training item.
Priority Applications (5)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010696315.XA CN111667742A (en) | 2020-07-20 | 2020-07-20 | Intelligent auxiliary training method and system for learner-driven vehicle |
| PCT/CN2020/105268 WO2022016583A1 (en) | 2020-07-20 | 2020-07-28 | Intelligent training aiding method and system for learner-driven vehicle |
| SE2051569A SE2051569A1 (en) | 2020-07-20 | 2020-07-28 | Intelligent training aiding method and system for instructional cars |
| ZA2020/07882A ZA202007882B (en) | 2020-07-20 | 2020-12-17 | Intelligent training aiding method and system for instructional cars |
| AU2020104282A AU2020104282A4 (en) | 2020-07-20 | 2020-12-23 | Intelligent Training Aiding Method And System For Instructional Cars |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010696315.XA CN111667742A (en) | 2020-07-20 | 2020-07-20 | Intelligent auxiliary training method and system for learner-driven vehicle |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111667742A true CN111667742A (en) | 2020-09-15 |
Family
ID=72392693
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010696315.XA Pending CN111667742A (en) | 2020-07-20 | 2020-07-20 | Intelligent auxiliary training method and system for learner-driven vehicle |
Country Status (5)
| Country | Link |
|---|---|
| CN (1) | CN111667742A (en) |
| AU (1) | AU2020104282A4 (en) |
| SE (1) | SE2051569A1 (en) |
| WO (1) | WO2022016583A1 (en) |
| ZA (1) | ZA202007882B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112201113A (en) * | 2020-10-21 | 2021-01-08 | 广东星唯信息技术有限公司 | Method for monitoring backing and warehousing of learner-driven vehicle by RTK mapping technology |
| CN115578914A (en) * | 2022-11-23 | 2023-01-06 | 湖南视觉伟业智能科技有限公司 | Training system and method for backing in place during driving training |
| CN116729422A (en) * | 2023-06-07 | 2023-09-12 | 广州市德赛西威智慧交通技术有限公司 | Deviation correction method for vehicle track, vehicle driving assistance method and device |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114399932A (en) * | 2021-12-28 | 2022-04-26 | 南京财经大学 | Talent training simulation training system based on intelligent learning |
| CN114822169B (en) * | 2022-05-06 | 2023-06-09 | 辽宁科技大学 | Auxiliary driving exercise method and device for learner-driven vehicle |
| US11643108B1 (en) * | 2022-06-23 | 2023-05-09 | Motional Ad Llc | Generating corrected future maneuver parameters in a planner |
| CN116645848A (en) * | 2023-04-17 | 2023-08-25 | 武汉未来幻影科技有限公司 | Vehicle operation control method and related equipment |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105590449A (en) * | 2015-12-29 | 2016-05-18 | 南京邮电大学 | Instructional car driving monitoring system and monitoring method |
| CN108109481A (en) * | 2017-12-21 | 2018-06-01 | 云南冲浪科技有限公司 | A kind of vehicle driver training intelligent coach system |
| CN108389477A (en) * | 2018-03-05 | 2018-08-10 | 广州星唯信息科技有限公司 | A kind of correction guidance method for driving training field training |
| CN108806371A (en) * | 2018-08-31 | 2018-11-13 | 成都杏仁网络科技有限公司 | A kind of intelligent decision method and system based on Driving Test Subject training |
| CN109272821A (en) * | 2018-10-22 | 2019-01-25 | 广州星唯信息科技有限公司 | A kind of place driving evaluation method based on high-precision vision positioning |
| CN110599853A (en) * | 2019-08-05 | 2019-12-20 | 深圳华桥智能设备科技有限公司 | Intelligent teaching system and method for driving school |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8930227B2 (en) * | 2012-03-06 | 2015-01-06 | State Farm Mutual Automobile Insurance Company | Online system for training novice drivers and rating insurance products |
| CN208376741U (en) * | 2017-03-24 | 2019-01-15 | 多伦科技股份有限公司 | A kind of intelligent robot coach DAS (Driver Assistant System) |
| CN111047948B (en) * | 2019-11-27 | 2022-06-10 | 康忠文 | Auxiliary method and system for learning to train |
-
2020
- 2020-07-20 CN CN202010696315.XA patent/CN111667742A/en active Pending
- 2020-07-28 SE SE2051569A patent/SE2051569A1/en not_active Application Discontinuation
- 2020-07-28 WO PCT/CN2020/105268 patent/WO2022016583A1/en active Application Filing
- 2020-12-17 ZA ZA2020/07882A patent/ZA202007882B/en unknown
- 2020-12-23 AU AU2020104282A patent/AU2020104282A4/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105590449A (en) * | 2015-12-29 | 2016-05-18 | 南京邮电大学 | Instructional car driving monitoring system and monitoring method |
| CN108109481A (en) * | 2017-12-21 | 2018-06-01 | 云南冲浪科技有限公司 | A kind of vehicle driver training intelligent coach system |
| CN108389477A (en) * | 2018-03-05 | 2018-08-10 | 广州星唯信息科技有限公司 | A kind of correction guidance method for driving training field training |
| CN108806371A (en) * | 2018-08-31 | 2018-11-13 | 成都杏仁网络科技有限公司 | A kind of intelligent decision method and system based on Driving Test Subject training |
| CN109272821A (en) * | 2018-10-22 | 2019-01-25 | 广州星唯信息科技有限公司 | A kind of place driving evaluation method based on high-precision vision positioning |
| CN110599853A (en) * | 2019-08-05 | 2019-12-20 | 深圳华桥智能设备科技有限公司 | Intelligent teaching system and method for driving school |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112201113A (en) * | 2020-10-21 | 2021-01-08 | 广东星唯信息技术有限公司 | Method for monitoring backing and warehousing of learner-driven vehicle by RTK mapping technology |
| CN115578914A (en) * | 2022-11-23 | 2023-01-06 | 湖南视觉伟业智能科技有限公司 | Training system and method for backing in place during driving training |
| CN115578914B (en) * | 2022-11-23 | 2023-03-14 | 湖南视觉伟业智能科技有限公司 | Training system and method for backing in place during driving training |
| CN116729422A (en) * | 2023-06-07 | 2023-09-12 | 广州市德赛西威智慧交通技术有限公司 | Deviation correction method for vehicle track, vehicle driving assistance method and device |
| CN116729422B (en) * | 2023-06-07 | 2024-03-08 | 广州市德赛西威智慧交通技术有限公司 | Deviation correction method for vehicle track, vehicle driving assistance method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| AU2020104282A4 (en) | 2021-04-15 |
| SE2051569A1 (en) | 2022-01-27 |
| WO2022016583A1 (en) | 2022-01-27 |
| ZA202007882B (en) | 2021-09-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111667742A (en) | Intelligent auxiliary training method and system for learner-driven vehicle | |
| US11568755B1 (en) | Pre-license development tool | |
| US10255821B2 (en) | Driver training | |
| US10446047B1 (en) | Real-time driver observation and scoring for driver'S education | |
| CN106710360B (en) | Intelligent driving training system and method based on enhancing virtual reality human-computer interaction | |
| CN108806371B (en) | Intelligent judgment method and system based on driving test subject training | |
| CN106875781A (en) | A kind of intelligent robot trains auxiliary driving method and its system | |
| Jiménez et al. | Gaze fixation system for the evaluation of driver distractions induced by IVIS | |
| CN110688877B (en) | Danger early warning method, device, equipment and storage medium | |
| Tunçer et al. | Vision based lane keeping assistance control triggered by a driver inattention monitor | |
| CN102096930A (en) | Student standing and sitting detection method for intelligent recorded broadcasting system for teaching | |
| CN111402671A (en) | Driving school driving language audio and video teaching method and device | |
| CN110853452A (en) | Self-service system based on VR technique | |
| CN114627717A (en) | Virtual reality and big data analysis-based novice driver training system | |
| JP6086515B1 (en) | Driving skill evaluation device, server device, driving skill evaluation system, program, and driving skill evaluation method | |
| CN108806366B (en) | Intelligent automobile driving teaching method and system based on big data analysis | |
| JP2012155285A (en) | Drive hazard predictive learning support system | |
| JP2008257002A (en) | Parking evaluation method for parking drive operation and system thereof | |
| CN112215111A (en) | Method and related device for evaluating direction control capability of motor vehicle driver | |
| RU72779U1 (en) | VEHICLE DRIVING CONTROL SYSTEM (OPTIONS) | |
| CN115440107A (en) | VR virtual reality-based intelligent driving training system and method for deaf-mute | |
| CN113313982A (en) | Education system based on 5G network | |
| CN113052041A (en) | Intelligent teaching system and monitoring method | |
| CN114419950A (en) | Big data analysis-based driving training teaching optimization method and system | |
| CN113569628B (en) | Training posture inspection method, device and system for new energy automobile teaching |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200915 |
|
| RJ01 | Rejection of invention patent application after publication |