CN113205070B - Visual perception algorithm optimization method and system - Google Patents
Visual perception algorithm optimization method and system Download PDFInfo
- Publication number
- CN113205070B CN113205070B CN202110587249.7A CN202110587249A CN113205070B CN 113205070 B CN113205070 B CN 113205070B CN 202110587249 A CN202110587249 A CN 202110587249A CN 113205070 B CN113205070 B CN 113205070B
- Authority
- CN
- China
- Prior art keywords
- visual perception
- perception algorithm
- data
- simulation
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000016776 visual perception Effects 0.000 title claims abstract description 183
- 238000000034 method Methods 0.000 title claims abstract description 59
- 238000005457 optimization Methods 0.000 title claims abstract description 52
- 238000004088 simulation Methods 0.000 claims abstract description 149
- 238000012549 training Methods 0.000 claims abstract description 20
- 230000008447 perception Effects 0.000 claims abstract description 17
- 238000001514 detection method Methods 0.000 claims description 19
- 238000012360 testing method Methods 0.000 claims description 13
- 238000004590 computer program Methods 0.000 claims description 11
- 238000012545 processing Methods 0.000 claims description 6
- 238000012937 correction Methods 0.000 claims description 5
- 230000001133 acceleration Effects 0.000 claims description 3
- 238000012795 verification Methods 0.000 description 17
- 238000011156 evaluation Methods 0.000 description 6
- 238000004891 communication Methods 0.000 description 5
- 230000000694 effects Effects 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 238000007781 pre-processing Methods 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000004927 fusion Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000012805 post-processing Methods 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000007499 fusion processing Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/588—Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Multimedia (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention provides a visual perception algorithm optimization method and a system, wherein the visual perception algorithm optimization method comprises the following steps: acquiring real data and simulation data, and acquiring vehicle state simulation information; based on the real data, the simulation data and the vehicle state simulation information, loading a visual perception algorithm to generate a perception performance index; confirming that the perceived performance index is smaller than the performance index threshold, and training the visual perception algorithm based on the real data and the simulation data to obtain an updated visual perception algorithm; inputting the updated visual perception algorithm to a control and planning unit to obtain a feedback result; and (3) confirming that the feedback result is not in the feedback threshold range, iteratively loading the updated visual perception algorithm through the real data, the simulation data and the vehicle state simulation information until the feedback result is in the feedback threshold range, and outputting the target visual perception algorithm. The invention can improve the accuracy of the visual perception algorithm and optimize the updating efficiency.
Description
Technical Field
The invention relates to the technical field of artificial intelligence, in particular to a visual perception algorithm optimization method and system.
Background
With the development of automatic driving technology, for some working machines, such as a mixer truck, a muck truck and the like, requirements on performance of a visual perception algorithm are higher and higher, and how to verify and optimize the visual perception algorithm becomes a problem to be solved urgently.
At present, aiming at an automatic driving system, a method for carrying out system simulation through various simulation platforms appears to provide support for verification of various module algorithms so as to replace actual road tests, however, the current simulation verification method only can verify the effectiveness of the algorithms and cannot optimize the visual perception algorithm, and the simulation verification process is low in efficiency and cannot fully play the role of verification results.
Disclosure of Invention
The invention provides a visual perception algorithm optimization method and a visual perception algorithm optimization system, which are used for solving the defects that the visual perception algorithm cannot be optimized in the prior art, the efficiency of the simulation verification process is low, and the effect of a verification result cannot be fully exerted.
The invention provides a visual perception algorithm optimization method, which comprises the following steps: acquiring real data and simulation data, and acquiring vehicle state simulation information; loading a visual perception algorithm based on the real data, the simulation data and the vehicle state simulation information to generate a perception performance index; confirming that the perceived performance index is smaller than a performance index threshold, and training the visual perception algorithm based on the real data and the simulation data to obtain an updated visual perception algorithm; inputting the updated visual perception algorithm to a control and planning unit to obtain a feedback result; and confirming that the feedback result is not in the feedback threshold range, carrying out iterative loading on the updated visual perception algorithm through the real data, the simulation data and the vehicle state simulation information until the feedback result is in the feedback threshold range, and outputting a target visual perception algorithm.
According to the visual perception algorithm optimization method provided by the invention, the visual perception algorithm is trained based on the real data and the simulation data to obtain an updated visual perception algorithm, and the visual perception algorithm optimization method comprises the following steps: the real data and the simulation data are called in, fine-tune training is carried out on the visual perception algorithm, and parameters of the visual perception algorithm are updated; and obtaining the updated visual perception algorithm based on the updated parameters of the visual perception algorithm.
According to the visual perception algorithm optimization method provided by the invention, the acquisition of real data and simulation data comprises the following steps: acquiring original real data and original simulation data; and carrying out format correction processing on the original real data and the original simulation data to obtain the real data and the simulation data.
According to the visual perception algorithm optimization method provided by the invention, the acquisition of real data and simulation data comprises the following steps: collecting the real data and generating initial data; and carrying out error adjustment on the initial data based on physical parameters of the camera to obtain the simulation data.
According to the visual perception algorithm optimization method provided by the invention, the physical parameters comprise: at least one of sensor internal parameters, sensor external parameters, distortion coefficients, mounting height, mounting angle and mounting inclination.
According to the visual perception algorithm optimization method provided by the invention, the visual perception algorithm at least comprises the following steps: an obstacle detection algorithm and a lane line detection algorithm.
According to the visual perception algorithm optimization method provided by the invention, the vehicle state simulation information comprises chassis information, positioning information and control information.
According to the visual perception algorithm optimization method provided by the invention, the visual perception algorithm optimization method is executed by the simulation test platform.
The invention also provides a visual perception algorithm optimization system, which comprises: the acquisition module is used for acquiring real data and simulation data and acquiring vehicle state simulation information; the loading module is used for loading a visual perception algorithm based on the real data, the simulation data and the vehicle state simulation information to generate a perception performance index; the training module is used for confirming that the perception performance index is smaller than a performance index threshold value, training the visual perception algorithm based on the real data and the simulation data, and obtaining an updated visual perception algorithm; the feedback module is used for inputting the updated visual perception algorithm to the control and planning unit to obtain a feedback result; and the iteration module is used for confirming that the feedback result is not in the feedback threshold range, carrying out iteration loading on the updated visual perception algorithm through the real data, the simulation data and the vehicle state simulation information until the feedback result is in the feedback threshold range, and outputting a target visual perception algorithm.
The invention also provides an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the steps of the visual perception algorithm optimization method according to any one of the above are realized when the processor executes the computer program.
The present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the visual perception algorithm optimization method as described in any of the above.
According to the visual perception algorithm optimization method and system, the visual perception algorithm is verified and iterated based on the real data, the simulation data and the vehicle state simulation information, and the visual perception algorithm is updated on the basis of evaluation, so that the visual perception algorithm is optimized, the efficiency of the simulation verification process is improved, the effect of a verification result can be fully exerted, the accuracy of the visual perception algorithm is improved, the updating efficiency is optimized, and the automatic driving is promoted to be safer and more efficient.
Drawings
In order to more clearly illustrate the invention or the technical solutions of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic flow chart of a visual perception algorithm optimization method provided by the invention;
FIG. 2 is a schematic diagram of a visual perception algorithm optimization method provided by the invention;
FIG. 3 is a schematic diagram of a visual perception algorithm optimization system provided by the present invention;
fig. 4 is a schematic structural diagram of an electronic device provided by the present invention;
fig. 5 is a schematic flow chart of a visual perception algorithm verification iteration provided by the invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The visual perception algorithm optimization method and system of the present invention are described below in conjunction with fig. 1-5.
The visual perception algorithm is a computer program stored in a vehicle-mounted host computer of an automatic driving vehicle, can process images and videos shot by a camera of the vehicle, and perceives objects such as obstacles and lane lines, so that the vehicle can be controlled to make feedback in time.
As shown in fig. 1 and 2, the present invention provides a visual perception algorithm optimization method, which includes: steps 110 to 150 are as follows.
And 110, acquiring real data and simulation data, and acquiring vehicle state simulation information.
It may be appreciated that a simulation test platform may be provided, which may include a data generation unit, a data preprocessing unit, a vehicle status information unit, a perception algorithm verification and processing unit, and a control planning unit. The data generating unit can acquire real data and simulation data, the real data can be an environment real-time image or real-time video shot by a camera in an actual scene, and the simulation data is an environment image or environment video simulated by the data generating system.
The camera can be a short-focus camera, a long-focus camera or a fish-eye camera.
The vehicle state simulation information is related information which is simulated by the vehicle state information unit according to the vehicle state information which is provided by the vehicle running in the real scene, and the vehicle state information unit can generate the vehicle state simulation information which is in a simulated data form.
And step 120, loading a visual perception algorithm based on the real data, the simulation data and the vehicle state simulation information to generate a perception performance index.
It can be understood that the real data, the simulation data and the vehicle state simulation information can be input into the perception algorithm verification and processing unit, and meanwhile, the visual perception algorithm is loaded in the perception algorithm verification and processing unit, that is, the real data, the simulation data and the vehicle state simulation information are processed by using the visual perception algorithm, so that an actual process of the visual perception algorithm running in an actual driving scene is simulated.
After the visual perception algorithm is run, the obstacle information attribute can be obtained firstly, wherein the obstacle information attribute can comprise an object ID, an object category, the width and the height of the object, the confidence coefficient of the object and the center coordinate of the object, and the information attribute of the lane line comprises the quality (confidence coefficient) of the lane line, the type of the lane line and the fitting parameters (C0, C1, C2 and C3) of the cubic polynomial.
The obstacle information attribute may be converted into a perceptual performance index, which may include: AP for obstacle detection (single category evaluation accuracy), mAP (average value of a plurality of categories of APs), fps (frame rate), lane line detection index p (accuracy).
And 130, confirming that the perceived performance index is smaller than the performance index threshold, and training the visual perception algorithm based on the real data and the simulation data to obtain an updated visual perception algorithm.
It may be appreciated that the performance index threshold may be set, the perceived performance index may be compared with the performance index threshold, if the perceived performance index is greater than or equal to the performance index threshold, the visual perception algorithm is considered to be validated, and at this time, the visual perception algorithm may not be updated, and may be output to the fusion module and the post-processing unit, that is, the visual perception algorithm may be directly stored for subsequent use and operation.
If the perception performance index is smaller than the performance index threshold, the visual perception algorithm is considered to be required to be optimized and updated, and the visual perception algorithm is trained by using real data and simulation data, so that the parameters of the visual perception algorithm are adjusted, and the updated visual perception algorithm is obtained based on the adjusted parameters of the visual perception algorithm.
It should be noted that the performance index threshold is an evaluation index corresponding to the visual perception algorithm, different visual perception algorithms correspond to different performance index thresholds, and the performance index threshold may be set according to actual requirements, which is not particularly limited in this embodiment.
And 140, inputting the updated visual perception algorithm to a control and planning unit to obtain a feedback result.
It can be understood that the control and planning unit can simulate according to the working state of the executing element in the actual driving scene of the vehicle, the control and planning unit can respond to the input updated visual perception algorithm to obtain a feedback result, and the updated visual perception algorithm can be subjected to secondary evaluation according to the feedback result.
The feedback results may include: lane line width, detection distance, deviation error of obstacle transverse distance, etc., and these data obtained by control and planning unit are based on preset feedback threshold value to judge whether the result provided by updated visual perception algorithm is proper or not, if not, the perception algorithm is further optimized, and the corresponding parameters are optimized.
And 150, confirming that the feedback result is not in the feedback threshold range, and iteratively loading the updated visual perception algorithm through the real data, the simulation data and the vehicle state simulation information until the feedback result is in the feedback threshold range, and outputting the target visual perception algorithm.
It can be understood that a feedback threshold value can be preset, the feedback result is compared with the feedback threshold value, if the feedback result is within the feedback threshold value range, the updated visual perception algorithm is considered to meet the requirements, and the updated visual perception algorithm is output to the fusion and post-processing module as a target visual perception algorithm; if the feedback result is not within the feedback threshold range, the updated visual perception algorithm is considered to be not satisfactory, and at this time, the updated visual perception algorithm is iteratively loaded through the real data, the simulation data and the vehicle state simulation information, that is, the processes from step 120 to step 150 are repeatedly executed until the obtained feedback result is within the feedback threshold range, and at this time, the updated visual perception algorithm which is satisfactory is output as the target visual perception algorithm.
The feedback threshold may be a range value, and if the feedback result is not within the range, the iterative loading operation is performed until the feedback result obtained by iterative iteration is within the range.
Therefore, through the verification and iteration processes, multiple evaluations of the visual perception algorithm are realized, the output target visual perception algorithm is optimized, and the automatic driving is safer and more efficient.
It is worth mentioning that the visual perception algorithm optimization method can be directly executed in the simulation test platform, the simulation test platform can be operated in one terminal device for verification optimization of the visual perception algorithm, the simulation test platform does not need to be connected with a server or a mobile terminal when the visual perception algorithm is optimized, other devices are not needed to assist detection, the simulation test platform can directly execute steps of the method in sequence, and can finish optimization tasks by itself without depending on other equipment ends, so that the flow can be simplified, time consumed by information conversion and transmission is reduced, cost and time of actual testing are also reduced, and optimization efficiency of the visual perception algorithm can be greatly improved.
According to the visual perception algorithm optimization method provided by the invention, the visual perception algorithm is verified and iterated based on the real data, the simulation data and the vehicle state simulation information, and is updated on the basis of evaluation, so that the visual perception algorithm is optimized, the efficiency of the simulation verification process is improved, the effect of a verification result can be fully exerted, the accuracy of the visual perception algorithm is improved, the updating efficiency is optimized, and the automatic driving is promoted to be safer and more efficient.
In some embodiments, the training the visual perception algorithm based on the real data and the simulation data to obtain the updated visual perception algorithm in step 130 includes: the real data and the simulation data are called in, fine-tune training is carried out on the visual perception algorithm, and parameters of the visual perception algorithm are updated; and obtaining the updated visual perception algorithm based on the updated visual perception algorithm parameters.
It can be understood that fine-tune is a fine-tuning training mode in the deep learning technology, where real data and simulation data can be tuned in to perform fine-tuning training on the visual perception algorithm, parameters of the updated visual perception algorithm are adjusted, and the updated visual perception algorithm is obtained according to the parameters of the updated visual perception algorithm.
In some embodiments, step 110, obtaining the real data and the simulation data, further includes:
acquiring original real data and original simulation data; and carrying out format correction processing on the original real data and the original simulation data to obtain the real data and the simulation data.
It will be appreciated that the simulation platform may include a data preprocessing unit that may perform format corrections on the raw real data and the raw simulation data, such as performing unified format storage, such as CV: mat, image and other operations, wide and high alignment operations, correcting Image data, recording the frame number of the data, and obtaining real data and simulation data after the preprocessing operation.
In some embodiments, step 110, obtaining real data and simulation data, includes: collecting real data and generating initial data; and carrying out error adjustment on the initial data based on the physical parameters of the camera to obtain simulation data.
It will be appreciated that the data generation unit may generate initial data, which may have errors, and thus the initial data may be corrected, i.e. error adjusted, in the error correction unit using the actual data and the physical parameters of the camera. The physical parameters of the camera can influence the imaging effect, the physical parameters of the camera are used for correcting initial data, and after error adjustment is carried out on the initial data, simulation data are obtained, and the simulation data are closer to an environment image under the real condition.
In some embodiments, the physical parameters include: at least one of sensor internal parameters, sensor external parameters, distortion coefficients, mounting height, mounting angle and mounting inclination.
It is understood that the sensor physical parameters of the camera may include sensor internal parameters, sensor external parameters, distortion coefficients, mounting height, mounting angle, and mounting inclination. These physical parameters can be used to make error adjustments to the initial data to obtain simulated data.
It is worth noting that in the prior art, a sensor model building process, and parameter comparison and error analysis between initial data and real data generated virtually are absent, if the process is absent, the actual effectiveness of the generated simulation data cannot be guaranteed, and the verification and optimization process of a visual perception algorithm is difficult to have actual effects.
According to the embodiment, the sensor internal parameters, the sensor external parameters, the distortion coefficients, the mounting height, the mounting angle and the mounting inclination of the camera are introduced to perform error adjustment on the initial data, simulation data in different scenes can be simulated, simulation data in different working conditions close to an actual scene are generated, the simulation data are enabled to be closer to actual video data in the actual scene, the authenticity of the simulation data for detection is enabled to be higher, and then the accuracy of a visual perception algorithm obtained by detection and optimization based on the simulation data is also higher.
In some embodiments, the visual perception algorithm comprises at least: an obstacle detection algorithm and a lane line detection algorithm.
As shown in fig. 5, the present invention can implement simultaneous checksum iteration on multiple visual perception algorithms, so that each visual perception algorithm corresponds to a Type identifier Type, and corresponds to an obstacle detection algorithm when the Type is 0, and corresponds to a lane line detection algorithm when the Type is 1.
It will be appreciated that the obstacle detection algorithm can be used to extract obstacle information from an ambient image of the vehicle, and that the lane line detection algorithm can be used to extract lane line information from an ambient image of the vehicle, although the visual perception algorithm may include other relevant detection algorithms.
In some embodiments, the vehicle state simulation information includes chassis information, positioning information, and control information.
It is understood that the vehicle state simulation information is state information simulating a real running condition of the vehicle, and the vehicle state information may include chassis information reflecting information such as forward, backward, and steering of the chassis of the vehicle, positioning information reflecting a real-time position of the vehicle, and control information reflecting state information such as a deceleration state, an acceleration state, or a braking state of the vehicle when the vehicle is actually running.
In some embodiments, the visual perception algorithm optimization method is executed by the simulation test platform, that is, the visual perception algorithm optimization method can be automatically operated by the simulation test platform without other equipment for auxiliary detection, the simulation test platform can automatically complete detection and optimization updating of the visual perception algorithm, the time consumed by information conversion and transmission is reduced, the cost and time of actual testing are also reduced, and therefore the efficiency and the safety performance of automatic driving are further improved.
The visual perception algorithm optimization system provided by the invention is described below, and the visual perception algorithm optimization system described below and the visual perception algorithm optimization method described above can be correspondingly referred to each other.
As shown in fig. 3, the present invention further provides a visual perception algorithm optimization system, including: an acquisition module 310, a loading module 320, a training module 330, a feedback module 340, and an iteration module 350.
The acquiring module 310 is configured to acquire real data and simulation data, and acquire vehicle state simulation information.
The loading module 320 is configured to load the visual perception algorithm based on the real data, the simulation data and the vehicle state simulation information, and generate a perception performance index.
The training module 330 is configured to confirm that the perceived performance index is smaller than the performance index threshold, and train the visual perception algorithm based on the real data and the simulation data to obtain an updated visual perception algorithm.
And the feedback module 340 is configured to input the updated visual perception algorithm to the control and planning unit, so as to obtain a feedback result.
And the iteration module 350 is configured to confirm that the feedback result is not within the feedback threshold range, iteratively load the updated visual perception algorithm through the real data, the simulation data and the vehicle state simulation information until the feedback result is within the feedback threshold range, and output the target visual perception algorithm.
Fig. 4 illustrates a physical schematic diagram of an electronic device, as shown in fig. 4, which may include: processor 410, communication interface (Communications Interface) 420, memory 430 and communication bus 440, wherein processor 410, communication interface 420 and memory 430 communicate with each other via communication bus 440. The processor 410 may invoke logic instructions in the memory 430 to perform a visual perception algorithm optimization method comprising: acquiring real data and simulation data, and acquiring vehicle state simulation information; based on the real data, the simulation data and the vehicle state simulation information, loading a visual perception algorithm to generate a perception performance index; confirming that the perceived performance index is smaller than the performance index threshold, and training the visual perception algorithm based on the real data and the simulation data to obtain an updated visual perception algorithm; inputting the updated visual perception algorithm to a control and planning unit to obtain a feedback result; and (3) confirming that the feedback result is not in the feedback threshold range, iteratively loading the updated visual perception algorithm through the real data, the simulation data and the vehicle state simulation information until the feedback result is in the feedback threshold range, and outputting the target visual perception algorithm.
Further, the logic instructions in the memory 430 described above may be implemented in the form of software functional units and may be stored in a computer-readable storage medium when sold or used as a stand-alone product. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product comprising a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions which, when executed by a computer, are capable of performing the visual perception algorithm optimization method provided by the above methods, the method comprising: acquiring real data and simulation data, and acquiring vehicle state simulation information; based on the real data, the simulation data and the vehicle state simulation information, loading a visual perception algorithm to generate a perception performance index; confirming that the perceived performance index is smaller than the performance index threshold, and training the visual perception algorithm based on the real data and the simulation data to obtain an updated visual perception algorithm; inputting the updated visual perception algorithm to a control and planning unit to obtain a feedback result; and (3) confirming that the feedback result is not in the feedback threshold range, iteratively loading the updated visual perception algorithm through the real data, the simulation data and the vehicle state simulation information until the feedback result is in the feedback threshold range, and outputting the target visual perception algorithm.
In yet another aspect, the present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, is implemented to perform the visual perception algorithm optimization method provided above, the method comprising: acquiring real data and simulation data, and acquiring vehicle state simulation information; based on the real data, the simulation data and the vehicle state simulation information, loading a visual perception algorithm to generate a perception performance index; confirming that the perceived performance index is smaller than the performance index threshold, and training the visual perception algorithm based on the real data and the simulation data to obtain an updated visual perception algorithm; inputting the updated visual perception algorithm to a control and planning unit to obtain a feedback result; and (3) confirming that the feedback result is not in the feedback threshold range, iteratively loading the updated visual perception algorithm through the real data, the simulation data and the vehicle state simulation information until the feedback result is in the feedback threshold range, and outputting the target visual perception algorithm.
The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims (9)
1. A method for optimizing a visual perception algorithm, comprising:
acquiring real data and simulation data, and acquiring vehicle state simulation information;
loading a visual perception algorithm based on the real data, the simulation data and the vehicle state simulation information to generate a perception performance index; the real data are environment real-time images or real-time videos shot by a camera in an actual scene; the simulation data are environment images or environment videos simulated by the data generation system; the vehicle state simulation information is obtained by simulating vehicle state information units according to vehicle state information of a vehicle running in a real scene, the vehicle state simulation information comprises chassis information, positioning information and control information, the chassis information is used for reflecting forward, backward and steering information of a vehicle chassis, the positioning information is used for reflecting real-time positions of the vehicle, and the control information is used for reflecting deceleration state, acceleration state or braking state information of the vehicle; the acquiring of the simulation data comprises: correcting the initial data generated by the data generating unit by utilizing the real data and the physical parameters of the camera to obtain the simulation data;
confirming that the perceived performance index is smaller than a performance index threshold, and training the visual perception algorithm based on the real data and the simulation data to obtain an updated visual perception algorithm;
inputting the updated visual perception algorithm to a control and planning unit to obtain a feedback result; the control and planning unit is used for simulating according to the working state of the executive component in the actual driving scene of the vehicle, responding to the input updated visual perception algorithm and obtaining the feedback result;
and confirming that the feedback result is not in the feedback threshold range, carrying out iterative loading on the updated visual perception algorithm through the real data, the simulation data and the vehicle state simulation information until the feedback result is in the feedback threshold range, and outputting a target visual perception algorithm.
2. The visual perception algorithm optimization method according to claim 1, wherein the training the visual perception algorithm based on the real data and the simulation data to obtain an updated visual perception algorithm comprises:
the real data and the simulation data are called in, fine-tune training is carried out on the visual perception algorithm, and parameters of the visual perception algorithm are updated;
and obtaining the updated visual perception algorithm based on the updated parameters of the visual perception algorithm.
3. The visual perception algorithm optimization method of claim 1, wherein the acquiring real data and simulation data comprises:
acquiring original real data and original simulation data;
and carrying out format correction processing on the original real data and the original simulation data to obtain the real data and the simulation data.
4. The visual perception algorithm optimization method of claim 1, wherein the physical parameters include: at least one of sensor internal parameters, sensor external parameters, distortion coefficients, mounting height, mounting angle and mounting inclination.
5. The visual perception algorithm optimization method of any one of claims 1 to 4, wherein the visual perception algorithm comprises at least: an obstacle detection algorithm and a lane line detection algorithm.
6. The visual perception algorithm optimization method of any one of claims 1-4, wherein the visual perception algorithm optimization method is performed by a simulation test platform.
7. A visual perception algorithm optimization system, comprising:
the acquisition module is used for acquiring real data and simulation data and acquiring vehicle state simulation information;
the loading module is used for loading a visual perception algorithm based on the real data, the simulation data and the vehicle state simulation information to generate a perception performance index; the real data are environment real-time images or real-time videos shot by a camera in an actual scene; the simulation data are environment images or environment videos simulated by the data generation system; the vehicle state simulation information is obtained by simulating vehicle state information units according to vehicle state information of a vehicle running in a real scene, the vehicle state simulation information comprises chassis information, positioning information and control information, the chassis information is used for reflecting forward, backward and steering information of a vehicle chassis, the positioning information is used for reflecting real-time positions of the vehicle, and the control information is used for reflecting deceleration state, acceleration state or braking state information of the vehicle; the acquiring of the simulation data comprises: correcting the initial data generated by the data generating unit by utilizing the real data and the physical parameters of the camera to obtain the simulation data;
the training module is used for confirming that the perception performance index is smaller than a performance index threshold value, training the visual perception algorithm based on the real data and the simulation data, and obtaining an updated visual perception algorithm;
the feedback module is used for inputting the updated visual perception algorithm to the control and planning unit to obtain a feedback result; the control and planning unit is used for simulating according to the working state of the executive component in the actual driving scene of the vehicle, responding to the input updated visual perception algorithm and obtaining the feedback result;
and the iteration module is used for confirming that the feedback result is not in the feedback threshold range, carrying out iteration loading on the updated visual perception algorithm through the real data, the simulation data and the vehicle state simulation information until the feedback result is in the feedback threshold range, and outputting a target visual perception algorithm.
8. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the visual perception algorithm optimization method of any one of claims 1 to 6 when the program is executed by the processor.
9. A non-transitory computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when executed by a processor, implements the steps of the visual perception algorithm optimization method of any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110587249.7A CN113205070B (en) | 2021-05-27 | 2021-05-27 | Visual perception algorithm optimization method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110587249.7A CN113205070B (en) | 2021-05-27 | 2021-05-27 | Visual perception algorithm optimization method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113205070A CN113205070A (en) | 2021-08-03 |
| CN113205070B true CN113205070B (en) | 2024-02-20 |
Family
ID=77023313
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110587249.7A Active CN113205070B (en) | 2021-05-27 | 2021-05-27 | Visual perception algorithm optimization method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113205070B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114035545A (en) * | 2021-11-08 | 2022-02-11 | 苏州挚途科技有限公司 | Automatic driving algorithm simulation test method and device and electronic equipment |
| CN114802284A (en) * | 2022-02-16 | 2022-07-29 | 武汉路特斯汽车有限公司 | Vehicle perception performance evaluation method and system |
| CN114580575A (en) * | 2022-04-29 | 2022-06-03 | 中智行(苏州)科技有限公司 | Method for constructing sustainable closed-loop link capable of realizing automatic driving visual perception |
| CN114923523A (en) * | 2022-05-27 | 2022-08-19 | 中国第一汽车股份有限公司 | Method and device for acquiring sensing data, storage medium and electronic device |
| CN114911813B (en) * | 2022-06-27 | 2023-09-26 | 芯砺智能科技(上海)有限公司 | Updating method and device of vehicle-mounted perception model, electronic equipment and storage medium |
| CN115291726B (en) * | 2022-08-09 | 2023-11-07 | 中国电信股份有限公司 | Haptic simulation method, haptic simulation device, haptic simulation system, storage medium and electronic device |
| CN116451590B (en) * | 2023-06-09 | 2023-11-17 | 安徽深信科创信息技术有限公司 | Simulation method and device of automatic driving simulation test platform |
Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107506830A (en) * | 2017-06-20 | 2017-12-22 | 同济大学 | Towards the artificial intelligence training platform of intelligent automobile programmed decision-making module |
| WO2018002910A1 (en) * | 2016-06-28 | 2018-01-04 | Cognata Ltd. | Realistic 3d virtual world creation and simulation for training automated driving systems |
| CN107992016A (en) * | 2016-10-26 | 2018-05-04 | 法乐第(北京)网络科技有限公司 | A kind of automatic driving vehicle analog detection method |
| CN109407547A (en) * | 2018-09-28 | 2019-03-01 | 合肥学院 | Multi-cam assemblage on-orbit test method and system towards panoramic vision perception |
| CN109522825A (en) * | 2018-10-31 | 2019-03-26 | 蔚来汽车有限公司 | The Performance Test System and its performance test methods of visual perception system |
| CN109785418A (en) * | 2018-12-21 | 2019-05-21 | 广东工业大学 | The blinkpunkt of view-based access control model sensor model renders optimization algorithm |
| CN110070139A (en) * | 2019-04-28 | 2019-07-30 | 吉林大学 | Small sample towards automatic Pilot environment sensing is in ring learning system and method |
| CN110187647A (en) * | 2018-02-23 | 2019-08-30 | 北京京东尚科信息技术有限公司 | Model training method and system |
| WO2020079685A1 (en) * | 2018-10-17 | 2020-04-23 | Cognata Ltd. | System and method for generating realistic simulation data for training an autonomous driver |
| WO2020106908A1 (en) * | 2018-11-21 | 2020-05-28 | Amazon Technologies, Inc. | Reinforcement learning model training through simulation |
| EP3722977A1 (en) * | 2019-04-11 | 2020-10-14 | Siemens Aktiengesellschaft | Method and apparatus for generating a design for a technical system or product |
| CN111785085A (en) * | 2020-06-11 | 2020-10-16 | 北京航空航天大学 | Visual perception and perception network training method, device, equipment and storage medium |
| CN111832652A (en) * | 2020-07-14 | 2020-10-27 | 北京罗克维尔斯科技有限公司 | Training method and device of decision model |
| CN112198859A (en) * | 2020-09-07 | 2021-01-08 | 西安交通大学 | Method, system and device for testing automatic driving vehicle in vehicle ring under mixed scene |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP6832421B2 (en) * | 2018-03-08 | 2021-02-24 | バイドゥドットコム タイムズ テクノロジー （ベイジン） カンパニー リミテッドＢａｉｄｕ．ｃｏｍ Ｔｉｍｅｓ Ｔｅｃｈｎｏｌｏｇｙ （Ｂｅｉｊｉｎｇ） Ｃｏ．， Ｌｔｄ． | Simulation-based evaluation method for sensing requirements of self-driving cars |
| US11254312B2 (en) * | 2019-06-07 | 2022-02-22 | Tusimple, Inc. | Autonomous vehicle simulation system |
| US11467575B2 (en) * | 2019-06-27 | 2022-10-11 | Toyota Motor Engineering & Manufacturing North America, Inc. | Systems and methods for safety-aware training of AI-based control systems |
-
2021
- 2021-05-27 CN CN202110587249.7A patent/CN113205070B/en active Active
Patent Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018002910A1 (en) * | 2016-06-28 | 2018-01-04 | Cognata Ltd. | Realistic 3d virtual world creation and simulation for training automated driving systems |
| CN107992016A (en) * | 2016-10-26 | 2018-05-04 | 法乐第(北京)网络科技有限公司 | A kind of automatic driving vehicle analog detection method |
| CN107506830A (en) * | 2017-06-20 | 2017-12-22 | 同济大学 | Towards the artificial intelligence training platform of intelligent automobile programmed decision-making module |
| CN110187647A (en) * | 2018-02-23 | 2019-08-30 | 北京京东尚科信息技术有限公司 | Model training method and system |
| CN109407547A (en) * | 2018-09-28 | 2019-03-01 | 合肥学院 | Multi-cam assemblage on-orbit test method and system towards panoramic vision perception |
| WO2020079685A1 (en) * | 2018-10-17 | 2020-04-23 | Cognata Ltd. | System and method for generating realistic simulation data for training an autonomous driver |
| CN109522825A (en) * | 2018-10-31 | 2019-03-26 | 蔚来汽车有限公司 | The Performance Test System and its performance test methods of visual perception system |
| WO2020106908A1 (en) * | 2018-11-21 | 2020-05-28 | Amazon Technologies, Inc. | Reinforcement learning model training through simulation |
| CN109785418A (en) * | 2018-12-21 | 2019-05-21 | 广东工业大学 | The blinkpunkt of view-based access control model sensor model renders optimization algorithm |
| EP3722977A1 (en) * | 2019-04-11 | 2020-10-14 | Siemens Aktiengesellschaft | Method and apparatus for generating a design for a technical system or product |
| CN110070139A (en) * | 2019-04-28 | 2019-07-30 | 吉林大学 | Small sample towards automatic Pilot environment sensing is in ring learning system and method |
| CN111785085A (en) * | 2020-06-11 | 2020-10-16 | 北京航空航天大学 | Visual perception and perception network training method, device, equipment and storage medium |
| CN111832652A (en) * | 2020-07-14 | 2020-10-27 | 北京罗克维尔斯科技有限公司 | Training method and device of decision model |
| CN112198859A (en) * | 2020-09-07 | 2021-01-08 | 西安交通大学 | Method, system and device for testing automatic driving vehicle in vehicle ring under mixed scene |
Non-Patent Citations (2)
| Title |
|---|
| 牟洪波，戚大伟.基于BP和RBF神经网络的木材缺陷检测研究.哈尔滨工程大学出版社,2011,第92页. * |
| 邓伟文，任秉韬.汽车智能驾驶模拟仿真技术.机械工业出版社,2021,第120-122页. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113205070A (en) | 2021-08-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113205070B (en) | Visual perception algorithm optimization method and system | |
| CN107481292B (en) | Attitude error estimation method and device for vehicle-mounted camera | |
| CN109990790B (en) | Unmanned aerial vehicle path planning method and device | |
| CN110197027B (en) | Automatic driving test method and device, intelligent equipment and server | |
| US8489353B2 (en) | Methods and systems for calibrating vehicle vision systems | |
| CN111091739B (en) | Automatic driving scene generation method and device and storage medium | |
| CN112422936B (en) | Vehicle panoramic image calibration system, method, server and storage medium | |
| CN107862268A (en) | Look around Lane detection method, apparatus and computer-readable recording medium | |
| KR101843516B1 (en) | Method for determination of focal length for a zoom lens | |
| US11449707B2 (en) | Method for processing automobile image data, apparatus, and readable storage medium | |
| CN110211069B (en) | Image denoising model training method, system, equipment and computer medium | |
| CN112382165A (en) | Driving strategy generation method, device, medium, equipment and simulation system | |
| JP2020052687A (en) | Position estimation device | |
| EP4280573A1 (en) | Method for returning data on vehicle, controller on vehicle, cloud server, and vehicle | |
| CN116579409A (en) | Intelligent camera model pruning acceleration method and acceleration system based on re-parameterization | |
| CN116776288A (en) | Optimization method and device of intelligent driving perception model and storage medium | |
| CN114104005B (en) | Decision-making method, device and equipment of automatic driving equipment and readable storage medium | |
| CN115880142A (en) | Image generation method and device of trailer, storage medium and terminal | |
| CN113066185B (en) | Scene data creating method, device, medium and equipment of mine simulation system | |
| US20210174122A1 (en) | Probabilistic sampling acceleration and corner feature extraction for vehicle systems | |
| CN114398250A (en) | Vehicle test evaluation method and device based on functional scene | |
| CN114676781A (en) | Vehicle fault diagnosis and prediction method, device and equipment and vehicle | |
| CN112395695A (en) | Method and system for establishing simulation scene in real time | |
| CN118036693A (en) | Remote model training system | |
| CN116721168A (en) | Engineering vehicle panoramic image calibration method and device and engineering vehicle |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |