CN113205070A - Visual perception algorithm optimization method and system - Google Patents
Visual perception algorithm optimization method and system Download PDFInfo
- Publication number
- CN113205070A CN113205070A CN202110587249.7A CN202110587249A CN113205070A CN 113205070 A CN113205070 A CN 113205070A CN 202110587249 A CN202110587249 A CN 202110587249A CN 113205070 A CN113205070 A CN 113205070A
- Authority
- CN
- China
- Prior art keywords
- visual perception
- perception algorithm
- data
- simulation
- real data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000016776 visual perception Effects 0.000 title claims abstract description 185
- 238000000034 method Methods 0.000 title claims abstract description 62
- 238000005457 optimization Methods 0.000 title claims abstract description 56
- 238000004088 simulation Methods 0.000 claims abstract description 144
- 230000008447 perception Effects 0.000 claims abstract description 26
- 238000012549 training Methods 0.000 claims abstract description 21
- 238000001514 detection method Methods 0.000 claims description 18
- 238000012360 testing method Methods 0.000 claims description 14
- 238000009434 installation Methods 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 11
- 238000012545 processing Methods 0.000 claims description 6
- 238000012937 correction Methods 0.000 claims description 4
- 238000012795 verification Methods 0.000 description 18
- 230000000694 effects Effects 0.000 description 7
- 238000011156 evaluation Methods 0.000 description 6
- 238000004891 communication Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 238000007781 pre-processing Methods 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000004927 fusion Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000012805 post-processing Methods 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000007499 fusion processing Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000002893 slag Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/588—Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Multimedia (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention provides a visual perception algorithm optimization method and a system, wherein the visual perception algorithm optimization method comprises the following steps: acquiring real data and simulation data, and acquiring vehicle state simulation information; loading a visual perception algorithm based on the real data, the simulation data and the vehicle state simulation information to generate a perception performance index; confirming that the perception performance index is smaller than the performance index threshold value, training a visual perception algorithm based on real data and simulation data, and obtaining an updated visual perception algorithm; inputting the updated visual perception algorithm into a control and planning unit to obtain a feedback result; and confirming that the feedback result is not in the feedback threshold range, iteratively loading the updated visual perception algorithm through the real data, the simulation data and the vehicle state simulation information until the feedback result is in the feedback threshold range, and outputting the target visual perception algorithm. The invention can improve the accuracy of the visual perception algorithm and optimize the updating efficiency.
Description
Technical Field
The invention relates to the technical field of artificial intelligence, in particular to a visual perception algorithm optimization method and system.
Background
With the development of the automatic driving technology, the requirements on the performance of the visual perception algorithm for some working machines, such as a mixer truck, a slag car and the like, are higher and higher, and how to verify and optimize the visual perception algorithm becomes a problem to be solved urgently.
At present, aiming at an automatic driving system, a method for carrying out system simulation through various simulation platforms appears to provide support for verification of various module algorithms, so that actual road tests can be replaced, however, the existing simulation verification method can only verify the effectiveness of the algorithms and cannot optimize the visual perception algorithms, the efficiency of the simulation verification process is low, and the effect of verification results cannot be fully played.
Disclosure of Invention
The invention provides a visual perception algorithm optimization method and a visual perception algorithm optimization system, which are used for solving the defects that the visual perception algorithm cannot be optimized in the prior art, the simulation verification process has lower efficiency and cannot fully exert the effect of a verification result, the optimization of the visual perception algorithm is realized, the efficiency of the simulation verification process is improved, the effect of the verification result can be fully exerted, the accuracy and the optimization updating efficiency of the visual perception algorithm are improved, and the automatic driving is promoted to be safer and more efficient.
The invention provides a visual perception algorithm optimization method, which comprises the following steps: acquiring real data and simulation data, and acquiring vehicle state simulation information; loading a visual perception algorithm based on the real data, the simulation data and the vehicle state simulation information to generate a perception performance index; confirming that the perception performance index is smaller than a performance index threshold value, training the visual perception algorithm based on the real data and the simulation data, and obtaining an updated visual perception algorithm; inputting the updated visual perception algorithm to a control and planning unit to obtain a feedback result; and confirming that the feedback result is not in a feedback threshold range, iteratively loading the updated visual perception algorithm through the real data, the simulation data and the vehicle state simulation information until the feedback result is in the feedback threshold range, and outputting a target visual perception algorithm.
According to the visual perception algorithm optimization method provided by the invention, the training of the visual perception algorithm based on the real data and the simulation data to obtain the updated visual perception algorithm comprises the following steps: calling the real data and the simulation data, performing fine-tune training on the visual perception algorithm, and updating parameters of the visual perception algorithm; and obtaining the updated visual perception algorithm based on the updated parameters of the visual perception algorithm.
According to the visual perception algorithm optimization method provided by the invention, the acquiring of the real data and the simulation data comprises the following steps: acquiring original real data and original simulation data; and carrying out format correction processing on the original real data and the original simulation data to obtain the real data and the simulation data.
According to the visual perception algorithm optimization method provided by the invention, the acquiring of the real data and the simulation data comprises the following steps: collecting the real data and generating initial data; and carrying out error adjustment on the initial data based on physical parameters of a camera to obtain the simulation data.
According to the visual perception algorithm optimization method provided by the invention, the physical parameters comprise: at least one of sensor internal parameter, sensor external parameter, distortion coefficient, installation height, installation angle and installation gradient.
According to the visual perception algorithm optimization method provided by the invention, the visual perception algorithm at least comprises the following steps: an obstacle detection algorithm and a lane line detection algorithm.
According to the visual perception algorithm optimization method provided by the invention, the vehicle state simulation information comprises chassis information, positioning information and control information.
According to the visual perception algorithm optimization method provided by the invention, the visual perception algorithm optimization method is executed by a simulation test platform.
The invention also provides a visual perception algorithm optimizing system, which comprises: the acquisition module is used for acquiring real data and simulation data and acquiring vehicle state simulation information; the loading module is used for loading a visual perception algorithm based on the real data, the simulation data and the vehicle state simulation information to generate a perception performance index; the training module is used for confirming that the perception performance index is smaller than a performance index threshold value, and training the visual perception algorithm based on the real data and the simulation data to obtain an updated visual perception algorithm; the feedback module is used for inputting the updated visual perception algorithm to the control and planning unit to obtain a feedback result; and the iteration module is used for determining that the feedback result is not in a feedback threshold range, iteratively loading the updated visual perception algorithm through the real data, the simulation data and the vehicle state simulation information until the feedback result is in the feedback threshold range, and outputting a target visual perception algorithm.
The invention also provides an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of any of the visual perception algorithm optimization methods described above when executing the computer program.
The present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the visual perception algorithm optimization method according to any one of the above.
According to the visual perception algorithm optimization method and system provided by the invention, the visual perception algorithm is verified and iterated based on the real data, the simulation data and the vehicle state simulation information, and the visual perception algorithm is updated on the basis of evaluation, so that the optimization of the visual perception algorithm is realized, the efficiency of a simulation verification process is improved, the effect of a verification result can be fully exerted, the accuracy and the optimization updating efficiency of the visual perception algorithm are improved, and the automatic driving is promoted to be safer and more efficient.
Drawings
In order to more clearly illustrate the technical solutions of the present invention or the prior art, the drawings needed for the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.
FIG. 1 is a schematic flow chart of a visual perception algorithm optimization method provided by the present invention;
FIG. 2 is a schematic diagram illustrating a method for optimizing a visual perception algorithm according to the present invention;
FIG. 3 is a schematic structural diagram of a visual perception algorithm optimization system provided by the present invention;
FIG. 4 is a schematic structural diagram of an electronic device provided by the present invention;
fig. 5 is a schematic flow chart of verification iteration of the visual perception algorithm provided by the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The visual perception algorithm optimization method and system of the present invention are described below with reference to fig. 1-5.
The visual perception algorithm is a computer program stored in a vehicle-mounted host of the automatic driving vehicle, and can process images and videos shot by a camera of the vehicle and perceive objects such as obstacles, lane lines and the like, so that the vehicle can be controlled to feed back in time.
As shown in fig. 1 and 2, the present invention provides a visual perception algorithm optimization method, including: as follows from step 110 to step 150.
And step 110, acquiring real data and simulation data, and acquiring vehicle state simulation information.
It is understood that a simulation test platform may be provided, and the simulation test platform may include a data generation unit, a data preprocessing unit, a vehicle state information unit, a perception algorithm verification and processing unit, and a control planning unit. The data generation unit can acquire real data and simulation data, the real data can be an environment real-time image or a real-time video shot by a camera in an actual scene, and the simulation data is an environment image or an environment video simulated by the data generation system.
The camera can be a short-focus camera, a long-focus camera or a fisheye camera.
The vehicle state simulation information is related information which is simulated by the vehicle state information unit according to the vehicle state information possessed by the vehicle running in a real scene, the vehicle state information unit can generate vehicle state simulation information, and the vehicle state simulation information is a simulated data form.
And 120, loading a visual perception algorithm based on the real data, the simulation data and the vehicle state simulation information to generate a perception performance index.
It can be understood that the real data, the simulation data and the vehicle state simulation information can be input into the perception algorithm verifying and processing unit, and the visual perception algorithm is loaded in the perception algorithm verifying and processing unit, that is, the real data, the simulation data and the vehicle state simulation information are processed by using the visual perception algorithm, so as to simulate the actual process of the operation of the visual perception algorithm in the actual driving scene.
After the visual perception algorithm is operated, obstacle information attributes can be obtained firstly, the obstacle information attributes can comprise a target ID, a target category, the width and the height of the target, the confidence coefficient of the target and the center coordinate of the target, and the information attributes of the lane line comprise lane line quality (confidence coefficient), lane line type and cubic polynomial fitting parameters (C0, C1, C2 and C3).
The obstacle information attribute may be converted into a perceptual performance index, which may include: AP (single category evaluation accuracy) for obstacle detection, maps (average of multiple categories of APs), fps (frame rate), lane line detection index p (accuracy).
And step 130, confirming that the perception performance index is smaller than the performance index threshold value, and training the visual perception algorithm based on the real data and the simulation data to obtain an updated visual perception algorithm.
It can be understood that a performance index threshold value can be set, the perception performance index is compared with the performance index threshold value, if the perception performance index is greater than or equal to the performance index threshold value, the visual perception algorithm is considered to pass the verification, at this moment, the visual perception algorithm is not updated, the visual perception algorithm can be output to the fusion module and the post-processing unit, namely, the visual perception algorithm is directly stored for subsequent use and operation.
And if the perception performance index is smaller than the performance index threshold value, the visual perception algorithm is considered to need to be optimized and updated, the visual perception algorithm is trained by utilizing real data and simulation data, so that the parameters of the visual perception algorithm are adjusted, and the updated visual perception algorithm is obtained based on the adjusted parameters of the visual perception algorithm.
It should be noted that the performance index threshold is an evaluation index corresponding to the visual perception algorithm, different visual perception algorithms correspond to different performance index thresholds, and the performance index thresholds can be set according to actual requirements, which is not specifically limited in this embodiment.
And 140, inputting the updated visual perception algorithm to the control and planning unit to obtain a feedback result.
It can be understood that the control and planning unit can simulate the working state of the execution element in the actual driving scene of the vehicle, respond to the input updated visual perception algorithm to obtain a feedback result, and perform secondary evaluation on the updated visual perception algorithm according to the feedback result.
The feedback result may include: the data obtained by the control and planning unit based on the preset feedback threshold value determines whether the result provided by the updated visual perception algorithm is appropriate, and if not, the perception algorithm is further optimized, and corresponding parameters are optimized.
And 150, confirming that the feedback result is not in the feedback threshold range, iteratively loading the updated visual perception algorithm through the real data, the simulation data and the vehicle state simulation information until the feedback result is in the feedback threshold range, and outputting the target visual perception algorithm.
It can be understood that a feedback threshold value can be preset, the feedback result is compared with the feedback threshold value, if the feedback result is within the range of the feedback threshold value, the updated visual perception algorithm is considered to meet the requirement, and at the moment, the updated visual perception algorithm is output to the fusion and post-processing module as a target visual perception algorithm; if the feedback result is not within the feedback threshold range, the updated visual perception algorithm is not considered to meet the requirement, the updated visual perception algorithm is iteratively loaded through the real data, the simulation data and the vehicle state simulation information, namely the processes from the step 120 to the step 150 are repeatedly executed until the obtained feedback result is within the feedback threshold range, and the updated visual perception algorithm meeting the requirement is output as the target visual perception algorithm.
The feedback threshold may be an interval value, and if the feedback result is not within the interval range, the iterative loading operation is performed until the feedback result obtained by the iterative iteration is within the interval range.
Therefore, through the verification and iteration process, the multiple evaluation of the visual perception algorithm is realized, the optimization of the output target visual perception algorithm is realized, and the automatic driving is safer and more efficient.
It is worth mentioning that the visual perception algorithm optimization method can be directly executed in a simulation test platform, the simulation test platform can run in a terminal device and is used for verifying and optimizing the visual perception algorithm, when the visual perception algorithm is optimized, the simulation test platform does not need to be connected with a server or a mobile terminal, other devices are not needed for assisting detection, the simulation test platform can be directly executed in sequence according to the steps of the method, and the optimization task can be automatically completed without depending on other device terminals, so that the flow can be simplified, the time consumed by information conversion and transmission is reduced, the cost and the time of actual test are also reduced, and the optimization efficiency of the visual perception algorithm can be greatly improved.
According to the visual perception algorithm optimization method provided by the invention, the visual perception algorithm is verified and iterated based on the real data, the simulation data and the vehicle state simulation information, and the visual perception algorithm is updated on the basis of evaluation, so that the optimization of the visual perception algorithm is realized, the efficiency of a simulation verification process is improved, the effect of a verification result can be fully exerted, the accuracy and the optimization updating efficiency of the visual perception algorithm are improved, and the automatic driving is promoted to be safer and more efficient.
In some embodiments, the training the visual perception algorithm based on the real data and the simulation data in step 130 to obtain an updated visual perception algorithm includes: calling in real data and simulation data, performing fine-tune training on the visual perception algorithm, and updating parameters of the visual perception algorithm; and obtaining the updated visual perception algorithm based on the updated parameters of the visual perception algorithm.
It can be understood that the fine-tune is a fine tuning training mode in the deep learning technology, where real data and simulation data may be called to perform fine tuning training on the visual perception algorithm, parameters of the updated visual perception algorithm may be adjusted, and the updated visual perception algorithm may be obtained according to the parameters of the updated visual perception algorithm.
In some embodiments, the step 110 of obtaining real data and simulation data further comprises:
acquiring original real data and original simulation data; and carrying out format correction processing on the original real data and the original simulation data to obtain the real data and the simulation data.
It will be appreciated that the simulation platform may include a data pre-processing unit that may format correct the raw real data and the raw simulation data, such as to perform a uniform format store, such as CV: mat, Image and other operations, width and height alignment operations, Image data correction, data frame number recording, and real data and simulation data can be obtained after the preprocessing operation.
In some embodiments, step 110, obtaining real data and simulation data, comprises: collecting real data and generating initial data; and performing error adjustment on the initial data based on the physical parameters of the camera to obtain simulation data.
It is understood that the data generating unit may generate initial data, which may have errors, and thus the initial data may be corrected, i.e., error-adjusted, in the error correcting unit using the real data and the physical parameters of the camera. The physical parameters of the camera can influence the imaging effect, the physical parameters of the camera are used for correcting initial data, the initial data are subjected to error adjustment, simulation data are obtained, and the simulation data are closer to an environment image under a real condition.
In some embodiments, the physical parameters include: at least one of sensor internal parameter, sensor external parameter, distortion coefficient, installation height, installation angle and installation gradient.
It is understood that the sensor physical parameters of the camera may include sensor internal parameters, sensor external parameters, distortion coefficients, mounting height, mounting angle and mounting inclination. These physical parameters can be used to make error adjustment on the initial data to obtain simulation data.
It is worth noting that the prior art lacks a sensor model building process, parameter comparison and error analysis between initial data and real data which are generated virtually, if the process is not available, the actual validity of the generated simulation data cannot be guaranteed, and the actual effect on the verification and optimization process of the visual perception algorithm is difficult to achieve.
The sensor internal parameters, the sensor external parameters, the distortion coefficient, the installation height, the installation angle and the installation inclination of the camera are introduced into the embodiment, the sensor physical parameters are used for carrying out error adjustment on the initial data, simulation data under different scenes can be simulated, simulation data under different working conditions close to the actual scene are generated, the simulation data are closer to actual video data in the actual scene, the reality of the simulation data for detection is higher, and the accuracy of obtaining a visual perception algorithm based on the simulation data for detection and optimization is also higher.
In some embodiments, the visual perception algorithm comprises at least: an obstacle detection algorithm and a lane line detection algorithm.
As shown in fig. 5, the invention can implement simultaneous verification and iteration on a plurality of visual perception algorithms, so that each visual perception algorithm corresponds to a Type identifier Type, and when the Type is 0, corresponds to an obstacle detection algorithm, and when the Type is 1, corresponds to a lane line detection algorithm.
It is understood that the obstacle detection algorithm can be used to extract obstacle information from the environment image of the vehicle, and the lane line detection algorithm can be used to extract lane line information from the environment image of the vehicle, although the visual perception algorithm may also include other related detection algorithms.
In some embodiments, the vehicle state simulation information includes chassis information, positioning information, and control information.
It can be understood that the vehicle state simulation information is state information simulating a vehicle in a real driving situation, and when the vehicle is in the real driving situation, the vehicle state information may include chassis information, positioning information and control information, the chassis information is used for reflecting forward, backward and steering information of a chassis of the vehicle, the positioning information is used for reflecting a real-time position of the vehicle, and the control information is used for reflecting a deceleration state, an acceleration state or a braking state of the vehicle.
In some embodiments, the visual perception algorithm optimization method is executed by the simulation test platform, that is, the visual perception algorithm optimization method can be automatically run by the simulation test platform without the need of other devices for assisting detection, the simulation test platform can automatically complete detection and optimization updating of the visual perception algorithm, time consumed by information conversion and transmission is reduced, and cost and time of actual test are also reduced, so that efficiency can be improved, and safety performance of automatic driving is further improved.
The following describes the visual perception algorithm optimization system provided by the present invention, and the visual perception algorithm optimization system described below and the visual perception algorithm optimization method described above can be referred to correspondingly.
As shown in fig. 3, the present invention further provides a visual perception algorithm optimizing system, including: an acquisition module 310, a loading module 320, a training module 330, a feedback module 340, and an iteration module 350.
And the obtaining module 310 is used for obtaining the real data and the simulation data and obtaining the vehicle state simulation information.
And the loading module 320 is configured to load the visual perception algorithm based on the real data, the simulation data, and the vehicle state simulation information, so as to generate a perception performance index.
And the training module 330 is configured to confirm that the perceptual performance index is smaller than the performance index threshold, and train the visual perception algorithm based on the real data and the simulation data to obtain an updated visual perception algorithm.
And the feedback module 340 is configured to input the updated visual perception algorithm to the control and planning unit to obtain a feedback result.
And the iteration module 350 is configured to confirm that the feedback result is not within the feedback threshold range, iteratively load the updated visual perception algorithm through the real data, the simulation data and the vehicle state simulation information until the feedback result is within the feedback threshold range, and output the target visual perception algorithm.
Fig. 4 illustrates a physical structure diagram of an electronic device, which may include, as shown in fig. 4: a processor (processor)410, a communication Interface 420, a memory (memory)430 and a communication bus 440, wherein the processor 410, the communication Interface 420 and the memory 430 are communicated with each other via the communication bus 440. The processor 410 may invoke logic instructions in the memory 430 to perform a method of visual perception algorithm optimization, the method comprising: acquiring real data and simulation data, and acquiring vehicle state simulation information; loading a visual perception algorithm based on the real data, the simulation data and the vehicle state simulation information to generate a perception performance index; confirming that the perception performance index is smaller than the performance index threshold value, training a visual perception algorithm based on real data and simulation data, and obtaining an updated visual perception algorithm; inputting the updated visual perception algorithm into a control and planning unit to obtain a feedback result; and confirming that the feedback result is not in the feedback threshold range, iteratively loading the updated visual perception algorithm through the real data, the simulation data and the vehicle state simulation information until the feedback result is in the feedback threshold range, and outputting the target visual perception algorithm.
In addition, the logic instructions in the memory 430 may be implemented in the form of software functional units and stored in a computer readable storage medium when the software functional units are sold or used as independent products. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product comprising a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions which, when executed by a computer, enable the computer to perform a method for visual perception algorithm optimization provided by the above methods, the method comprising: acquiring real data and simulation data, and acquiring vehicle state simulation information; loading a visual perception algorithm based on the real data, the simulation data and the vehicle state simulation information to generate a perception performance index; confirming that the perception performance index is smaller than the performance index threshold value, training a visual perception algorithm based on real data and simulation data, and obtaining an updated visual perception algorithm; inputting the updated visual perception algorithm into a control and planning unit to obtain a feedback result; and confirming that the feedback result is not in the feedback threshold range, iteratively loading the updated visual perception algorithm through the real data, the simulation data and the vehicle state simulation information until the feedback result is in the feedback threshold range, and outputting the target visual perception algorithm.
In yet another aspect, the present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, is implemented to perform the visual perception algorithm optimization method provided above, the method comprising: acquiring real data and simulation data, and acquiring vehicle state simulation information; loading a visual perception algorithm based on the real data, the simulation data and the vehicle state simulation information to generate a perception performance index; confirming that the perception performance index is smaller than the performance index threshold value, training a visual perception algorithm based on real data and simulation data, and obtaining an updated visual perception algorithm; inputting the updated visual perception algorithm into a control and planning unit to obtain a feedback result; and confirming that the feedback result is not in the feedback threshold range, iteratively loading the updated visual perception algorithm through the real data, the simulation data and the vehicle state simulation information until the feedback result is in the feedback threshold range, and outputting the target visual perception algorithm.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (11)
1. A visual perception algorithm optimization method, comprising:
acquiring real data and simulation data, and acquiring vehicle state simulation information;
loading a visual perception algorithm based on the real data, the simulation data and the vehicle state simulation information to generate a perception performance index;
confirming that the perception performance index is smaller than a performance index threshold value, training the visual perception algorithm based on the real data and the simulation data, and obtaining an updated visual perception algorithm;
inputting the updated visual perception algorithm to a control and planning unit to obtain a feedback result;
and confirming that the feedback result is not in a feedback threshold range, iteratively loading the updated visual perception algorithm through the real data, the simulation data and the vehicle state simulation information until the feedback result is in the feedback threshold range, and outputting a target visual perception algorithm.
2. The method of claim 1, wherein the training the visual perception algorithm based on the real data and the simulation data to obtain an updated visual perception algorithm comprises:
calling the real data and the simulation data, performing fine-tune training on the visual perception algorithm, and updating parameters of the visual perception algorithm;
and obtaining the updated visual perception algorithm based on the updated parameters of the visual perception algorithm.
3. The visual perception algorithm optimization method of claim 1, wherein the obtaining real data and simulated data comprises:
acquiring original real data and original simulation data;
and carrying out format correction processing on the original real data and the original simulation data to obtain the real data and the simulation data.
4. The visual perception algorithm optimization method of claim 1, wherein the obtaining real data and simulated data comprises:
collecting the real data and generating initial data;
and carrying out error adjustment on the initial data based on physical parameters of a camera to obtain the simulation data.
5. The visual perception algorithm optimization method of claim 4, wherein the physical parameters include: at least one of sensor internal parameter, sensor external parameter, distortion coefficient, installation height, installation angle and installation gradient.
6. A visual perception algorithm optimization method according to any one of claims 1 to 5, wherein the visual perception algorithm includes at least: an obstacle detection algorithm and a lane line detection algorithm.
7. The visual perception algorithm optimization method according to any one of claims 1 to 5, wherein the vehicle state simulation information includes chassis information, positioning information, and control information.
8. The visual perception algorithm optimization method according to any one of claims 1 to 5, wherein the visual perception algorithm optimization method is performed by a simulation test platform.
9. A visual perception algorithm optimization system, comprising:
the acquisition module is used for acquiring real data and simulation data and acquiring vehicle state simulation information;
the loading module is used for loading a visual perception algorithm based on the real data, the simulation data and the vehicle state simulation information to generate a perception performance index;
the training module is used for confirming that the perception performance index is smaller than a performance index threshold value, and training the visual perception algorithm based on the real data and the simulation data to obtain an updated visual perception algorithm;
the feedback module is used for inputting the updated visual perception algorithm to the control and planning unit to obtain a feedback result;
and the iteration module is used for determining that the feedback result is not in a feedback threshold range, iteratively loading the updated visual perception algorithm through the real data, the simulation data and the vehicle state simulation information until the feedback result is in the feedback threshold range, and outputting a target visual perception algorithm.
10. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the steps of the visual perception algorithm optimization method according to any one of claims 1 to 8 are implemented when the program is executed by the processor.
11. A non-transitory computer readable storage medium having stored thereon a computer program, wherein the computer program when executed by a processor implements the steps of the visual perception algorithm optimization method according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110587249.7A CN113205070B (en) | 2021-05-27 | 2021-05-27 | Visual perception algorithm optimization method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110587249.7A CN113205070B (en) | 2021-05-27 | 2021-05-27 | Visual perception algorithm optimization method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113205070A true CN113205070A (en) | 2021-08-03 |
| CN113205070B CN113205070B (en) | 2024-02-20 |
Family
ID=77023313
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110587249.7A Active CN113205070B (en) | 2021-05-27 | 2021-05-27 | Visual perception algorithm optimization method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113205070B (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114035545A (en) * | 2021-11-08 | 2022-02-11 | 苏州挚途科技有限公司 | Automatic driving algorithm simulation test method and device and electronic equipment |
| CN114580575A (en) * | 2022-04-29 | 2022-06-03 | 中智行(苏州)科技有限公司 | Method for constructing sustainable closed-loop link capable of realizing automatic driving visual perception |
| CN114802284A (en) * | 2022-02-16 | 2022-07-29 | 武汉路特斯汽车有限公司 | Vehicle perception performance evaluation method and system |
| CN114911813A (en) * | 2022-06-27 | 2022-08-16 | 芯砺智能科技(上海)有限公司 | Vehicle-mounted perception model updating method and device, electronic equipment and storage medium |
| CN114923523A (en) * | 2022-05-27 | 2022-08-19 | 中国第一汽车股份有限公司 | Method and device for acquiring sensing data, storage medium and electronic device |
| CN115291726A (en) * | 2022-08-09 | 2022-11-04 | 中国电信股份有限公司 | Haptic simulation method, device, system, storage medium and electronic device |
| CN116451590A (en) * | 2023-06-09 | 2023-07-18 | 西安深信科创信息技术有限公司 | Simulation method and device of automatic driving simulation test platform |
Citations (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107506830A (en) * | 2017-06-20 | 2017-12-22 | 同济大学 | Towards the artificial intelligence training platform of intelligent automobile programmed decision-making module |
| WO2018002910A1 (en) * | 2016-06-28 | 2018-01-04 | Cognata Ltd. | Realistic 3d virtual world creation and simulation for training automated driving systems |
| CN107992016A (en) * | 2016-10-26 | 2018-05-04 | 法乐第(北京)网络科技有限公司 | A kind of automatic driving vehicle analog detection method |
| CN109407547A (en) * | 2018-09-28 | 2019-03-01 | 合肥学院 | Multi-cam assemblage on-orbit test method and system towards panoramic vision perception |
| CN109522825A (en) * | 2018-10-31 | 2019-03-26 | 蔚来汽车有限公司 | The Performance Test System and its performance test methods of visual perception system |
| CN109785418A (en) * | 2018-12-21 | 2019-05-21 | 广东工业大学 | The blinkpunkt of view-based access control model sensor model renders optimization algorithm |
| CN110070139A (en) * | 2019-04-28 | 2019-07-30 | 吉林大学 | Small sample towards automatic Pilot environment sensing is in ring learning system and method |
| CN110187647A (en) * | 2018-02-23 | 2019-08-30 | 北京京东尚科信息技术有限公司 | Model training method and system |
| US20190278290A1 (en) * | 2018-03-08 | 2019-09-12 | Baidu Usa Llc | Simulation-based method to evaluate perception requirement for autonomous driving vehicles |
| WO2020079685A1 (en) * | 2018-10-17 | 2020-04-23 | Cognata Ltd. | System and method for generating realistic simulation data for training an autonomous driver |
| WO2020106908A1 (en) * | 2018-11-21 | 2020-05-28 | Amazon Technologies, Inc. | Reinforcement learning model training through simulation |
| EP3722977A1 (en) * | 2019-04-11 | 2020-10-14 | Siemens Aktiengesellschaft | Method and apparatus for generating a design for a technical system or product |
| CN111785085A (en) * | 2020-06-11 | 2020-10-16 | 北京航空航天大学 | Visual perception and perception network training method, device, equipment and storage medium |
| CN111832652A (en) * | 2020-07-14 | 2020-10-27 | 北京罗克维尔斯科技有限公司 | Training method and device of decision model |
| US20200384998A1 (en) * | 2019-06-07 | 2020-12-10 | Tusimple, Inc. | Autonomous vehicle simulation system |
| US20200409359A1 (en) * | 2019-06-27 | 2020-12-31 | Toyota Motor Engineering & Manufacturing North America, Inc. | Systems and methods for safety-aware training of ai-based control systems |
| CN112198859A (en) * | 2020-09-07 | 2021-01-08 | 西安交通大学 | Method, system and device for testing automatic driving vehicle in vehicle ring under mixed scene |
-
2021
- 2021-05-27 CN CN202110587249.7A patent/CN113205070B/en active Active
Patent Citations (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018002910A1 (en) * | 2016-06-28 | 2018-01-04 | Cognata Ltd. | Realistic 3d virtual world creation and simulation for training automated driving systems |
| CN107992016A (en) * | 2016-10-26 | 2018-05-04 | 法乐第(北京)网络科技有限公司 | A kind of automatic driving vehicle analog detection method |
| CN107506830A (en) * | 2017-06-20 | 2017-12-22 | 同济大学 | Towards the artificial intelligence training platform of intelligent automobile programmed decision-making module |
| CN110187647A (en) * | 2018-02-23 | 2019-08-30 | 北京京东尚科信息技术有限公司 | Model training method and system |
| US20190278290A1 (en) * | 2018-03-08 | 2019-09-12 | Baidu Usa Llc | Simulation-based method to evaluate perception requirement for autonomous driving vehicles |
| CN109407547A (en) * | 2018-09-28 | 2019-03-01 | 合肥学院 | Multi-cam assemblage on-orbit test method and system towards panoramic vision perception |
| WO2020079685A1 (en) * | 2018-10-17 | 2020-04-23 | Cognata Ltd. | System and method for generating realistic simulation data for training an autonomous driver |
| CN109522825A (en) * | 2018-10-31 | 2019-03-26 | 蔚来汽车有限公司 | The Performance Test System and its performance test methods of visual perception system |
| WO2020106908A1 (en) * | 2018-11-21 | 2020-05-28 | Amazon Technologies, Inc. | Reinforcement learning model training through simulation |
| CN109785418A (en) * | 2018-12-21 | 2019-05-21 | 广东工业大学 | The blinkpunkt of view-based access control model sensor model renders optimization algorithm |
| EP3722977A1 (en) * | 2019-04-11 | 2020-10-14 | Siemens Aktiengesellschaft | Method and apparatus for generating a design for a technical system or product |
| CN110070139A (en) * | 2019-04-28 | 2019-07-30 | 吉林大学 | Small sample towards automatic Pilot environment sensing is in ring learning system and method |
| US20200384998A1 (en) * | 2019-06-07 | 2020-12-10 | Tusimple, Inc. | Autonomous vehicle simulation system |
| US20200409359A1 (en) * | 2019-06-27 | 2020-12-31 | Toyota Motor Engineering & Manufacturing North America, Inc. | Systems and methods for safety-aware training of ai-based control systems |
| CN111785085A (en) * | 2020-06-11 | 2020-10-16 | 北京航空航天大学 | Visual perception and perception network training method, device, equipment and storage medium |
| CN111832652A (en) * | 2020-07-14 | 2020-10-27 | 北京罗克维尔斯科技有限公司 | Training method and device of decision model |
| CN112198859A (en) * | 2020-09-07 | 2021-01-08 | 西安交通大学 | Method, system and device for testing automatic driving vehicle in vehicle ring under mixed scene |
Non-Patent Citations (1)
| Title |
|---|
| 邓伟文，任秉韬, 哈尔滨工程大学出版社, pages: 120 - 122 * |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114035545A (en) * | 2021-11-08 | 2022-02-11 | 苏州挚途科技有限公司 | Automatic driving algorithm simulation test method and device and electronic equipment |
| CN114802284A (en) * | 2022-02-16 | 2022-07-29 | 武汉路特斯汽车有限公司 | Vehicle perception performance evaluation method and system |
| CN114580575A (en) * | 2022-04-29 | 2022-06-03 | 中智行(苏州)科技有限公司 | Method for constructing sustainable closed-loop link capable of realizing automatic driving visual perception |
| CN114923523A (en) * | 2022-05-27 | 2022-08-19 | 中国第一汽车股份有限公司 | Method and device for acquiring sensing data, storage medium and electronic device |
| CN114911813A (en) * | 2022-06-27 | 2022-08-16 | 芯砺智能科技(上海)有限公司 | Vehicle-mounted perception model updating method and device, electronic equipment and storage medium |
| CN114911813B (en) * | 2022-06-27 | 2023-09-26 | 芯砺智能科技(上海)有限公司 | Updating method and device of vehicle-mounted perception model, electronic equipment and storage medium |
| CN115291726A (en) * | 2022-08-09 | 2022-11-04 | 中国电信股份有限公司 | Haptic simulation method, device, system, storage medium and electronic device |
| CN115291726B (en) * | 2022-08-09 | 2023-11-07 | 中国电信股份有限公司 | Haptic simulation method, haptic simulation device, haptic simulation system, storage medium and electronic device |
| CN116451590A (en) * | 2023-06-09 | 2023-07-18 | 西安深信科创信息技术有限公司 | Simulation method and device of automatic driving simulation test platform |
| CN116451590B (en) * | 2023-06-09 | 2023-11-17 | 安徽深信科创信息技术有限公司 | Simulation method and device of automatic driving simulation test platform |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113205070B (en) | 2024-02-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113205070B (en) | Visual perception algorithm optimization method and system | |
| CN110120024B (en) | Image processing method, device, equipment and storage medium | |
| CN111091739B (en) | Automatic driving scene generation method and device and storage medium | |
| CN113869293B (en) | Lane line recognition method and device, electronic equipment and computer readable medium | |
| CN107862268A (en) | Look around Lane detection method, apparatus and computer-readable recording medium | |
| CN111627001B (en) | Image detection method and device | |
| CN110211069B (en) | Image denoising model training method, system, equipment and computer medium | |
| CN112422936B (en) | Vehicle panoramic image calibration system, method, server and storage medium | |
| CN113787521B (en) | Robot grabbing method, system, medium and electronic device based on deep learning | |
| CN111695421A (en) | Image recognition method and device and electronic equipment | |
| US20200082641A1 (en) | Three dimensional representation generating system | |
| CN109635639B (en) | Method, device, equipment and storage medium for detecting position of traffic sign | |
| JP6996455B2 (en) | Detector generator, monitoring device, detector generator and detector generator | |
| CN113885496B (en) | Intelligent driving simulation sensor model and intelligent driving simulation method | |
| CN115880142A (en) | Image generation method and device of trailer, storage medium and terminal | |
| CN115713678A (en) | Arrow picture data augmentation method and system, electronic device and storage medium | |
| CN114398250A (en) | Vehicle test evaluation method and device based on functional scene | |
| CN112561836B (en) | Method and device for acquiring point cloud set of target object | |
| JP7074723B2 (en) | Learning equipment and programs | |
| US20220101500A1 (en) | Evaluation apparatus for camera system and evaluation method | |
| CN113887290A (en) | Monocular 3D detection method and device, electronic equipment and storage medium | |
| CN108603809A (en) | Automobile Test System, method and computer program product | |
| CN112395695A (en) | Method and system for establishing simulation scene in real time | |
| CN118036693A (en) | Remote model training system | |
| JP2020184735A (en) | Verification method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |