CN108319249A - Unmanned algorithm synthesis evaluation system based on driving simulator and method - Google Patents
Unmanned algorithm synthesis evaluation system based on driving simulator and method Download PDFInfo
- Publication number
- CN108319249A CN108319249A CN201711362553.1A CN201711362553A CN108319249A CN 108319249 A CN108319249 A CN 108319249A CN 201711362553 A CN201711362553 A CN 201711362553A CN 108319249 A CN108319249 A CN 108319249A
- Authority
- CN
- China
- Prior art keywords
- test
- data
- appraisal
- module
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B23/00—Testing or monitoring of control systems or parts thereof
- G05B23/02—Electric testing or monitoring
- G05B23/0205—Electric testing or monitoring by means of a monitoring system capable of detecting and responding to faults
- G05B23/0218—Electric testing or monitoring by means of a monitoring system capable of detecting and responding to faults characterised by the fault detection method dealing with either existing or incipient faults
- G05B23/0221—Preprocessing measurements, e.g. data collection rate adjustment; Standardization of measurements; Time series or signal analysis, e.g. frequency analysis or wavelets; Trustworthiness of measurements; Indexes therefor; Measurements using easily measured parameters to estimate parameters difficult to measure; Virtual sensor creation; De-noising; Sensor fusion; Unconventional preprocessing inherently present in specific fault detection methods like PCA-based methods
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Automation & Control Theory (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Traffic Control Systems (AREA)
Abstract
Present disclose provides a kind of unmanned algorithm synthesis evaluation system and method based on driving simulator, the model of place with real world traffic scene is established by scenario simulation module, the design that comprehensive test module passes through three kinds of Job evaluation modes, in conjunction with the reproduction of test and appraisal scene, the comparison for providing evaluating result under same scene, foundation is provided for algorithm improvement direction.It is single that the disclosure solves existing unmanned assessment method test and appraisal scene, the problem of cannot reliably evaluating；Multi-level, systematization test and appraisal automatic driving vehicle algorithm performance is realized by three kinds of Job evaluation modes；Multi-layer multi-angle test and appraisal are realized, gos deep into inside algorithm, obtains reliable evaluating result；Testing and assessing, efficient, experimentation is reproducible, and the test and appraisal of cost relative test field are greatly lowered.
Description
Technical field
This disclosure relates to which unmanned Function Appraising technical field more particularly to a kind of nobody based on driving simulator drive
Sail algorithm synthesis evaluation system and method.
Background technology
Automatic driving vehicle integrates the functions such as environment sensing, decision rule and Driving control, can autonomous, safety,
Reliably travel in certain circumstances.With the development of automatic driving vehicle technology in recent years, the test and appraisal to unmanned performance
Demand also becomes more urgent.
At present correlation is proposed for the system in unmanned test and appraisal field mainly with the relevant test and appraisal of ADAS systems
Energy index, for characterizing system quality；This test and assess test site for needing infrastructure perfect and numerous test and appraisal equipment, cost
Height, vehicle shows under the test and appraisal period is long and the extreme case that cannot test and assess, and test and appraisal reliability needs to be investigated.
Therefore, truer, abundant traffic scene how is provided, to promote test and appraisal efficiency, ensure evaluating result can
It is that those skilled in the art need to continue the direction of research by property.
Disclosure
(1) technical problems to be solved
Present disclose provides a kind of unmanned algorithm synthesis evaluation system and method based on driving simulator, at least
Part solves technical problem set forth above.
(2) technical solution
According to one aspect of the disclosure, a kind of unmanned algorithm synthesis test and appraisal system based on driving simulator is provided
System, including：Comprehensive test module, including：Test and appraisal design cell, for formulating test and appraisal design requirement；Scenario simulation module, according to
Test and appraisal design requirement is modeled to obtain model of place to actual traffic scene；Model of place operational process on driving simulator
In, scenario simulation module acquires in the image data and model of place operational process of performance real-time scene of institute's Run-time scenario model
The contextual data and human-machine interaction data for defining scene/description vehicle-state are generated, information management module is sent to；And it is comprehensive
Test and appraisal module further include：Measurement and management unit, for being divided to Job evaluation mode；Job evaluation mode is selected according to demand, is obtained
Selected Job evaluation mode corresponds to index, and assessment work is carried out under scene model running, obtains evaluating result.
In some embodiments of the present disclosure, further include：Automatic driving vehicle communication module, for receiving information management mould
The image data and contextual data that block is sent, are sent to automatic driving vehicle；The perception that automatic driving vehicle algorithm generates is determined
Plan data, actuator director data and human-machine interaction data are sent to information management module；Information management module is used for scene mould
Quasi- communication between module, comprehensive test module and automatic driving vehicle communication module；Record the data of assessment process generation simultaneously
It preserves.
In some embodiments of the present disclosure, the Job evaluation mode of measurement and management dividing elements includes：Module level is tested and assessed, for tool
The min algorithm unit of a standby function system, for providing guidance for algorithm deficiency and algorithm improvement；System-level test and appraisal are obtaining
On the basis of obtaining module level index, by aggregation, establishes system-level index and complete test and appraisal；For quantifying comprising several calculations
The system performance of method shows, joining relation between research system；And the test and appraisal of vehicle grade, for evaluating vehicle in virtual traffic environment
Vehicle grade index is established in external overall performance in scene.
In some embodiments of the present disclosure, scenario simulation module includes：Scene modeling platform, for being designed according to test and appraisal
The model of place for meeting real world, or the scene for generating combination are established in the requirement of unit by virtual scene software
Model；Scene operation platform, for ensureing that the model of place generated realizes real time execution on driving simulator, and to information
Management module provides image data, contextual data and the human-machine interaction data of established model；Receiving actuator instructs, and controls vehicle
Model moves in model of place；And driving simulator, for acquiring the driving data in model of place, and will drive
Data are sent to information management module by scene operation platform and store.
In some embodiments of the present disclosure, information management module includes：Information centre's unit：For completing scenario simulation
The information exchange of module, automatic driving vehicle communication module and comprehensive test intermodule；And information memory cell：For recording
Experimentation, preserves the data generated in assessment process and storage in advance perceives Truth data and human-machine interaction data, and provides
Data query.
In some embodiments of the present disclosure, including：Step A：Test and appraisal design requirement is determined in test and appraisal design cell, and will
Test and appraisal design requirement is sent to scenario simulation module by information management module；Pass through virtual scene software in scene analog module
Build model of place；Step B：In measurement and management Unit selection Job evaluation mode；Automatic driving vehicle communication module is managed according to test and appraisal
The requirement of module is managed, correspondence opens automatic driving vehicle PORT COM；Step C：Under original state, according to selected Job evaluation mode,
Complete corresponding data processing；Step D：Model of place brings into operation in scene operation platform, automatic driving vehicle communication module,
Scenario simulation module and comprehensive test module will be interacted by information management module start information；Step E：Measurement and management unit according to
Evaluating result is obtained according to internal algorithm and interaction data, completes test and appraisal, and the data generated during test and appraisal will be automatically saved to letter
Cease management module；Check the evaluating result that measurement and management unit generates, analysis test and appraisal data.
In some embodiments of the present disclosure, under original state, the corresponding data processing method packet of Job evaluation mode in step C
It includes：Sub-step C1：When selecting the test and appraisal of vehicle grade, the driving of model of place is completed by the mankind, scene operation platform will drive
Member's driving data is sent to information memory cell and is stored, and driver's driving data is transferred by information centre's unit, sends
To measurement and management unit；And sub-step C2：When selecting module level test and appraisal/system-level test and appraisal, pass through information centre's unit tune
The perception Truth data being previously stored is taken, measurement and management unit is sent to.
In some embodiments of the present disclosure, after selection sub-step C1 in the process of running, the data of vehicle grade test and appraisal are come
Source includes：The image data and contextual data of the model of place for test transmitted by scene operation platform；Automatic driving car
Human-machine interaction data；And driver's driving data of acquisition.
In some embodiments of the present disclosure, after selection sub-step C2 in the process of running, module level test and appraisal/system-level survey
The data source commented includes：The image data and contextual data of the model of place for test transmitted by scene operation platform；
The perception decision data and actuator director data that automatic driving vehicle algorithm generates；Automatic driving vehicle human-machine interaction data；
And it is previously stored in the perception Truth data in information memory cell.
In some embodiments of the present disclosure, the model of place built in step A is the scene mould for meeting real world
Type, or to be required to combine existing model of place according to test and appraisal.
(3) advantageous effect
It can be seen from the above technical proposal that unmanned algorithm synthesis evaluation system of the disclosure based on driving simulator
And method, at least have the advantages that one of them：
(1) the abundant traffic scene for utilizing scenario simulation module to provide, evaluates the unmanned performance of vehicle in all directions.
(2) the multi-level test and appraisal of module level test and appraisal, system-level test and appraisal and the test and appraisal of vehicle grade is utilized to design, it can be external from vehicle
The expansion of overall performance and internal algorithm performance is tested and assessed, and broad covered area of testing and assessing, evaluating result is relatively reliable, can more system it is effective
Evaluate the performance of automatic driving vehicle algorithm in ground.
(3) in measurement and management unit, the reproduction for scene of testing and assessing provides the comparison of evaluating result under same scene, to calculate
Method improvement direction provides foundation.
(4) integrated information unit is utilized, the information exchange between modules is completed, system is easy to operate, the degree of automation
Height, test and appraisal are efficient.
It is single that the disclosure solves existing unmanned assessment method test and appraisal scene, the problem of cannot reliably evaluating.Pass through
Three kinds of Job evaluation modes realize multi-level, systematization test and appraisal automatic driving vehicle algorithm performance.It functionally sees, the disclosure realizes
Multi-layer multi-angle is tested and assessed, and is goed deep into inside algorithm, is obtained reliable evaluating result；In terms of whole angle, disclosure test and appraisal efficiency
High, experimentation reproduce, and the test and appraisal of cost relative test field are greatly lowered.
Description of the drawings
Fig. 1 is unmanned algorithm synthesis evaluation system total frame diagram of the disclosure based on driving simulator.
Fig. 2 is unmanned algorithm synthesis evaluation system frame diagram of the disclosure based on driving simulator.
Fig. 3 is the assessment method flow chart element of unmanned algorithm synthesis evaluation system of the disclosure based on driving simulator
Figure.
Specific implementation mode
Present disclose provides a kind of unmanned algorithm synthesis evaluation system and method based on driving simulator.Pass through field
Scape analog module establishes the model of place with real world traffic scene, and the data generated during scene model running
It is sent to information management module, to provide abundant traffic scene, is conducive to evaluate the unmanned performance of vehicle in all directions.It is comprehensive
Design of the module by three kinds of Job evaluation modes of testing and assessing provides pair of evaluating result under same scene in conjunction with the reproduction of test and appraisal scene
Than providing foundation for algorithm improvement direction, realizing the performance of multi-level systematization test and appraisal automatic driving vehicle algorithm.Message tube
Reason module realizes the data interaction between automatic driving vehicle by automatic driving vehicle communication module.Automatic driving vehicle communicates
Module, scenario simulation module and comprehensive test module carry out information exchange by information management unit；Pass through information management module
Record the data that test process generates and preservation.
To make the purpose, technical scheme and advantage of the disclosure be more clearly understood, below in conjunction with specific embodiment, and reference
The disclosure is further described in attached drawing.
Disclosure some embodiments will be done with reference to appended attached drawing in rear and more comprehensively describe to property, some of but not complete
The embodiment in portion will be shown.In fact, the various embodiments of the disclosure can be realized in many different forms, and should not be construed
To be limited to this several illustrated embodiment；Relatively, these embodiments are provided so that the disclosure meets applicable legal requirement.
In first exemplary embodiment of the disclosure, a kind of unmanned algorithm based on driving simulator is provided
Compositive appraisement system.Fig. 1 is unmanned algorithm synthesis evaluation system total frame diagram of the disclosure based on driving simulator.Such as figure
Shown in 1, including：Comprehensive test module is responsible for completing test and appraisal design requirement；Job evaluation mode is divided；On model of place
Assessment work is carried out, gathered data is simultaneously sent to information management module.Automatic driving vehicle communication module receives information management mould
The image data and contextual data that block is sent, are sent to automatic driving vehicle；The perception that automatic driving vehicle algorithm generates is determined
Plan data and actuator director data are sent to information management module.Scenario simulation module, to the traffic scene of real world into
Row modeling；In virtual scene operational process, scenario simulation module acquires the image data and contextual data of institute's moving model, hair
It send to information management module, information management module is using the image data and contextual data received as vehicle performance quality
Judging basis；It receives the actuator director data sent from information management module and executes, control auto model is in the scene
Various actions are completed such as to turn, brake, accelerate.When being unfolded to test and assess, data acquisition can be driven by true driver and be obtained
Take also can drive acquisition altogether by man-machine, be not specially limited here.Information management module communicates mould for automatic driving vehicle
Communication between block, scenario simulation module and comprehensive test module；Record the data that assessment process generates and preservation.
Fig. 2 is unmanned algorithm synthesis evaluation system frame diagram of the disclosure based on driving simulator.As shown in Fig. 2,
Comprehensive test module includes：Test and appraisal design cell completes test and appraisal design requirement, specifically includes the establishment to assessment process, edits
And execution, designed process can also be achieved and the process that has preserved of heavy duty, and be sent to information management list
Member.Measurement and management unit selects Job evaluation mode according to demand, obtains selected Job evaluation mode and corresponds to index, runs detecting and evaluating algorithms, exhibition
Show evaluating result and preservation.Job evaluation mode specifically includes in measurement and management unit：Module level is tested and assessed, to have a certain function system
Min algorithm unit, for providing guidance for algorithm deficiency and algorithm improvement.Such as the vehicle detection institute in sensory perceptual system is right
The index answered, index can be vehicle detection accuracy, track of vehicle prediction accuracy etc..System-level test and appraisal are obtaining module
On the basis of grade index, by aggregation, establishes system-level index and complete test and appraisal；System for quantifying to include polyalgorithm
Performance, joining relation between research system.Such as the static-obstacle analyte detection accuracy and dynamic barrier of sensory perceptual system
Detect accuracy and detection used time etc.；It is man-machine to drive the man-machine of system altogether and drive harmonious degree and degree of safety altogether.Vehicle grade is tested and assessed, for commenting
External overall performance of the valence vehicle in virtual traffic environment scene, establishes vehicle grade index.Such as accident rate, comfortable etc.
Grade, power categories, fuel-economizing percentage etc..
Scenario simulation module includes：Scene modeling platform passes through virtual scene software according to the requirement of test and appraisal design cell
The model of place for meeting reality is established, or generates the model of place of combination according to the requirement of test and appraisal design cell.Here field
Scape model is the model of place established according to real roads environment, traffic environment, weather environment.Scene operation platform ensures life
At model of place real time execution is realized on driving simulator；The picture number of established model is provided to information management module
According to, contextual data and human-machine interaction data；Receive the execution from unmanned communication module forwarded by information management module
The actions such as device instructs, and control auto model completes turning, brake in model of place, accelerates.Driving simulator is acquired in scene
Driving data in model, it can also be man-machine driving data that driving data here, which can be driver's driving data, and will
Driving data is sent to information management module by scene operation platform and preserves.The driving simulator that the disclosure provides includes true
The hardware such as real accelerator pedal, brake pedal, clutch pedal, shift handle, steering wheel, instrument board, key switch, wherein accelerating
Pedal is furnished with steering angle sensor and pressure sensing equipped with linear movement pick-up, brake pedal equipped with hydrostatic sensor, steering wheel
Device (is located at the easy contact position of steering wheel both hands).
Information management module includes：Information memory cell：For recording experimentation, the number generated in assessment process is preserved
According to the perception Truth data and driver's driving data that store in advance and provide data query.And information centre's unit：With
In completion scenario simulation module, the information exchange of automatic driving vehicle communication module and comprehensive test intermodule.It is information below
The responsible specific works of center cell：Image data and contextual data that scenario simulation module is sent are acquired, which is sent
To automatic driving vehicle communication module.Perception decision data that automatic driving vehicle communication module is sent is acquired to information management mould
Block.The actuator director data of acquisition automatic driving vehicle communication module transmission is simultaneously sent to scenario simulation module.Acquire nobody
The warning message to driver that vehicle communication module is sent is driven to scenario simulation module and information management module.Acquisition test and appraisal
The test and appraisal design requirement of design cell is simultaneously sent to scenario simulation module, makes it according to the test and appraisal manual/auto generation of requirement of experiment
Model of place.The steering wheel pressure sensor information from scenario simulation module is acquired to information management module.Work as selecting module
When grade test and appraisal/system-level test and appraisal, transfers the perception Truth data that test and appraisal information management module has been previously stored and be sent to test and appraisal pipe
Reason unit is for testing and assessing.When selecting to transfer test and appraisal under the test and appraisal of vehicle grade, the driver that gathering information management module has stored drives
It sails data and is sent to measurement and management unit for the test and appraisal of vehicle grade.
Modules are connected by Ethernet, and different function modules is distributed in different network nodes, form small-sized point
Cloth network.The sensor information of driving simulator is related to displacement signal, pressure signal etc., and scene operation platform is to such signal
It completes conversion and information management module is sent to by Ethernet.
The unmanned algorithm synthesis evaluation system introduction based on driving simulator provided above with respect to the disclosure finishes.
Individually below pair this disclosure relates to software, hardware configuration and specific data be described in detail.
Virtual product model software involved by the disclosure mainly uses：By the vehicle emulation and perception of French RJ Reynolds Tobacco Holdings ,Inc.
The SCANeR DT softwares of research group and OKTAL companies joint development or UC-win/Road softwares by Japan's research and development.
Driving simulator involved in the disclosure, including true accelerator pedal, brake pedal, clutch pedal, shift hand
The hardware such as handle, steering wheel, instrument board, key switch, and it is mounted with respective sensor acquisition hardware status information, the disclosure carries
The driving simulator of confession may be implemented it is man-machine drive altogether, can also be driven by true driver, pass through scene operation platform provide people
Driving data of the class driver in model of place is simultaneously sent to comprehensive test module.
Image data involved by the disclosure includes mainly：The band timestamp acquired in real time during scene model running
Scene image, i.e., the virtual preceding camera in source, left camera, right camera and rear camera image.
Contextual data involved by the disclosure includes mainly：In model of place operational process generate define scene data and
The data of vehicle-state are described；It specifically includes：The information such as natural environment data in model of place, i.e. rainfall, snowfall, air quantity；
The information such as road environment data in scape model, i.e. road number, road width, road curvature, road grade, lane line line style；
Traffic environment data in model of place, including the information such as traffic signs and signals lamp；Driving behavior data, i.e. scene operation platform
The information such as throttle position, brake pedal position, gear, the steering wheel angle of middle auto model；Vehicle status data, i.e. vehicle mould
The information such as speed, acceleration, attitude angle, rotating speed, the distance away from surrounding vehicles and barrier, the wheel tracks lane line distance of type.
Perception decision data involved by the disclosure includes mainly：Decision data is perceived, mould is communicated for automatic driving vehicle
The data that generate when block intermediate link is run, for example, the lane line, traffic sign and the traffic lights that detect as a result, the row of planning
Sail the data such as path.
Actuator director data involved by the disclosure includes mainly：Actuator director data.
Perception Truth data involved in the disclosure includes mainly following four classes：The first kind is to indicate lane line region
Picture, the data of the traffic lights color of the corresponding scene number of the second class of the representation, third class indicates corresponding scene number traffic mark
The data of will meaning, the 4th class indicate the data of the Obstacle Position under corresponding scene number.
Human-machine interaction data involved in the disclosure includes mainly：It to the warning message of driver and is driven including communication software
Sail the pressure sensor information of simulator steering wheel.
Above pair this disclosure relates to the detailed description introduction of software, hardware configuration and specific data finish.
In first exemplary embodiment of the disclosure, a kind of unmanned calculation based on driving simulator is additionally provided
The assessment method of method compositive appraisement system.Fig. 3 is unmanned algorithm synthesis evaluation system of the disclosure based on driving simulator
Assessment method flow diagram.As shown in figure 3,
Step A：Test and appraisal design requirement is determined in test and appraisal design cell, and test and appraisal design requirement is passed through into information management module
It is sent to scenario simulation module；Model of place is built by virtual scene software in scene analog module.
Step B：In measurement and management Unit selection Job evaluation mode；Automatic driving vehicle communication module is according to measurement and management module
Requirement, correspondence open automatic driving vehicle PORT COM.The PORT COM specifically opened includes：For receive image data and
The communication structure of contextual data；It is used for transmission the communication interface of actuator director data；Human-machine interaction data interface.
Step C：When selecting the test and appraisal of vehicle grade, the driving of model of place is completed by the mankind, scene operation platform will be driven
The person's of sailing driving data is sent to information memory cell and is stored, and driver's driving data is transferred by information centre's unit, hair
It send to measurement and management unit.
Step D：Model of place brings into operation in scene operation platform, automatic driving vehicle communication module, scenario simulation mould
Block and comprehensive test module will be interacted by information management module start information.
Step E：Measurement and management unit obtains evaluating result according to internal algorithm and interaction data, completes test and appraisal, tests and assesses the phase
Between the data that generate will be automatically saved to information management module.The data source of vehicle grade test and appraisal includes mainly that scene operation is flat
The image data and contextual data of the model of place for test transmitted by platform；Automatic driving vehicle human-machine interaction data；It adopts
Driver's driving data of collection.Check the evaluating result that measurement and management unit generates, analysis test and appraisal data.
It is real above with respect to the first of the unmanned algorithm synthesis assessment method based on driving simulator of disclosure offer
Example introduction is applied to finish.
In second exemplary embodiment of the disclosure, a kind of unmanned calculation based on driving simulator is additionally provided
The assessment method of method compositive appraisement system.It selects to be that module level is surveyed with one embodiment difference lies in Job evaluation mode
Comment/system-level test and appraisal.Fig. 3 is the assessment method stream of unmanned algorithm synthesis evaluation system of the disclosure based on driving simulator
Journey block diagram.As shown in figure 3,
Step A：Test and appraisal design requirement is determined in test and appraisal design cell, and test and appraisal design requirement is passed through into information management module
It is sent to scenario simulation module；Model of place is built by virtual scene software in scene analog module.
Step B：In measurement and management Unit selection Job evaluation mode；Automatic driving vehicle communication module is according to measurement and management module
Requirement, correspondence open automatic driving vehicle PORT COM.The PORT COM specifically opened includes：For receive image data and
The communication structure of contextual data；It is used for transmission the communication interface of actuator director data；Human-machine interaction data interface and for connecing
Receive the communication interface of perception decision data.
Step C：When selecting module level test and appraisal/system-level test and appraisal, is transferred and be previously stored by information centre's unit
Truth data is perceived, measurement and management unit is sent to.
Step D：Model of place brings into operation in scene operation platform, automatic driving vehicle communication module, scenario simulation mould
Block and comprehensive test module will be interacted by information management module start information.
Step E：Measurement and management unit obtains evaluating result according to internal algorithm and interaction data, completes test and appraisal, tests and assesses the phase
Between the data that generate will be automatically saved to information management module.The wherein data source master of module level test and appraisal/system-level test and appraisal
To include the image data and contextual data of the model of place for test transmitted by scene operation platform；Automatic driving vehicle
The perception decision data and actuator director data that algorithm generates；Automatic driving vehicle human-machine interaction data；It is previously stored in number
According to the perception Truth data in administrative unit.Check the evaluating result that measurement and management unit generates, analysis test and appraisal data.
It is real above with respect to the second of the unmanned algorithm synthesis assessment method based on driving simulator of disclosure offer
Example introduction is applied to finish.
So far, attached drawing is had been combined the embodiment of the present disclosure is described in detail.It should be noted that in attached drawing or saying
In bright book text, the realization method for not being painted or describing is form known to a person of ordinary skill in the art in technical field, and
It is not described in detail.In addition, the above-mentioned definition to each element and method be not limited in mentioning in embodiment it is various specific
Structure, shape or mode, those of ordinary skill in the art simply can be changed or replaced to it.
According to above description, those skilled in the art should be comprehensive to unmanned algorithm of the disclosure based on driving simulator
Closing evaluation system and method has clear understanding.
In conclusion to solve existing unmanned assessment method test and appraisal scene single for the disclosure, cannot reliably evaluate
Problem.Multi-level, systematization test and appraisal automatic driving vehicle algorithm performance is realized by three kinds of Job evaluation modes.Functionally see, this
It is open to realize multi-layer multi-angle test and appraisal, go deep into inside algorithm, obtains reliable evaluating result；In terms of whole angle, this public affairs
It opens efficient, experimentation of testing and assessing reproduce, the test and appraisal of cost relative test field are greatly lowered.
It should also be noted that, the direction term mentioned in embodiment, for example, "upper", "lower", "front", "rear", " left side ",
" right side " etc. is only the direction of refer to the attached drawing, not is used for limiting the protection domain of the disclosure.Through attached drawing, identical element by
Same or similar reference numeral indicates.When that understanding of this disclosure may be caused to cause to obscure, conventional structure will be omitted
Or construction.
And the shape and size of each component do not reflect actual size and ratio in figure, and only illustrate the embodiment of the present disclosure
Content.In addition, in the claims, any reference mark between bracket should not be configured to the limit to claim
System.
Furthermore word "comprising" does not exclude the presence of element or step not listed in the claims.Before element
Word "a" or "an" does not exclude the presence of multiple such elements.
The word of specification and ordinal number such as " first ", " second ", " third " etc. used in claim, with modification
Corresponding element, itself is not meant to that the element has any ordinal number, does not also represent the suitable of a certain element and another element
Sequence in sequence or manufacturing method, the use of those ordinal numbers are only used for enabling the element with certain name and another tool
There is the element of identical name that can make clear differentiation.
In addition, unless specifically described or the step of must sequentially occur, there is no restriction in the above institute for the sequence of above-mentioned steps
Row, and can change or rearrange according to required design.And above-described embodiment can be based on the considerations of design and reliability, that
This mix and match is used using or with other embodiment mix and match, i.e., the technical characteristic in different embodiments can be freely combined
Form more embodiments.
Algorithm and display be not inherently related to any certain computer, virtual system or miscellaneous equipment provided herein.
Various general-purpose systems can also be used together with enlightenment based on this.As described above, it constructs required by this kind of system
Structure be obvious.In addition, the disclosure is not also directed to any certain programmed language.It should be understood that can utilize various
Programming language realizes content of this disclosure described here, and the description done above to language-specific is to disclose this public affairs
The preferred forms opened.
The disclosure can by means of include several different elements hardware and by means of properly programmed computer come
It realizes.The all parts embodiment of the disclosure can be with hardware realization, or to run on one or more processors
Software module is realized, or is realized with combination thereof.It will be understood by those of skill in the art that can use in practice micro-
Processor or digital signal processor (DSP) are some or all in the relevant device according to the embodiment of the present disclosure to realize
The some or all functions of component.The disclosure be also implemented as a part for executing method as described herein or
Whole equipment or program of device (for example, computer program and computer program product).Such journey for realizing the disclosure
Sequence can may be stored on the computer-readable medium, or can be with the form of one or more signal.Such signal can
It is obtained with being downloaded from internet website, either provided on carrier signal or provided in any other forms.
Those skilled in the art, which are appreciated that, to carry out adaptively the module in the equipment in embodiment
Change and they are arranged in the one or more equipment different from the embodiment.It can be the module or list in embodiment
Member or component be combined into a module or unit or component, and can be divided into addition multiple submodule or subelement or
Sub-component.Other than such feature and/or at least some of process or unit exclude each other, it may be used any
Combination is disclosed to all features disclosed in this specification (including adjoint claim, abstract and attached drawing) and so to appoint
Where all processes or unit of method or equipment are combined.Unless expressly stated otherwise, this specification (including adjoint power
Profit requires, abstract and attached drawing) disclosed in each feature can be by providing the alternative features of identical, equivalent or similar purpose come generation
It replaces.Also, in the unit claims listing several devices, several in these devices can be by same hard
Part item embodies.
Similarly, it should be understood that in order to simplify the disclosure and help to understand one or more of each open aspect,
Above in the description of the exemplary embodiment of the disclosure, each feature of the disclosure is grouped together into single implementation sometimes
In example, figure or descriptions thereof.However, the method for the disclosure should be construed to reflect following intention：It is i.e. required to protect
The disclosure of shield requires features more more than the feature being expressly recited in each claim.More precisely, as following
Claims reflect as, open aspect is all features less than single embodiment disclosed above.Therefore,
Thus the claims for following specific implementation mode are expressly incorporated in the specific implementation mode, wherein each claim itself
All as the separate embodiments of the disclosure.
Particular embodiments described above has carried out further in detail the purpose, technical solution and advantageous effect of the disclosure
It describes in detail bright, it should be understood that the foregoing is merely the specific embodiment of the disclosure, is not limited to the disclosure, it is all
Within the spirit and principle of the disclosure, any modification, equivalent substitution, improvement and etc. done should be included in the guarantor of the disclosure
Within the scope of shield.
Claims (10)
1. a kind of unmanned algorithm synthesis evaluation system based on driving simulator, including：
Comprehensive test module, including：
Test and appraisal design cell, for formulating test and appraisal design requirement；
Scenario simulation module is modeled to obtain model of place according to test and appraisal design requirement to actual traffic scene；Model of place
On driving simulator in operational process, scenario simulation module acquires the picture number of the performance real-time scene of institute's Run-time scenario model
According to the contextual data and human-machine interaction data for defining scene/description vehicle-state with generation in model of place operational process, send
To information management module；And
The comprehensive test module further includes：
Measurement and management unit, for being divided to Job evaluation mode；Job evaluation mode is selected according to demand, obtains selected Job evaluation mode
Corresponding index, assessment work is carried out under scene model running, obtains evaluating result.
2. unmanned algorithm synthesis evaluation system according to claim 1, further includes：
Automatic driving vehicle communication module, image data and contextual data for receiving information management module transmission, is sent to
Automatic driving vehicle；Perception decision data, actuator director data and the human-computer interaction number that automatic driving vehicle algorithm is generated
According to being sent to information management module；And
Information management module, for the scenario simulation module, the comprehensive test module and automatic driving vehicle communication
Communication between module；Record the data that assessment process generates and preservation.
3. unmanned algorithm synthesis evaluation system according to claim 1, wherein the measurement and management dividing elements
The Job evaluation mode includes：
Module level is tested and assessed, to have the min algorithm unit of a function system, for being provided for algorithm deficiency and algorithm improvement
Guidance；
System-level test and appraisal, by aggregation, establish system-level index and complete test and appraisal on the basis of obtaining module level index；
It is showed for quantifying the system performance comprising several algorithms, joining relation between research system；And
Vehicle grade is tested and assessed, and for evaluating external overall performance of the vehicle in virtual traffic environment scene, establishes vehicle grade index.
4. unmanned algorithm synthesis evaluation system according to claim 1, wherein the scenario simulation module includes：
Scene modeling platform meets real world for the requirement according to test and appraisal design cell by the foundation of virtual scene software
Model of place, or for generate combination model of place；
Scene operation platform, for ensureing that the model of place generated realizes real time execution on driving simulator, and to information
Management module provides image data, contextual data and the human-machine interaction data of established model；Receiving actuator instructs, and controls vehicle
Model moves in model of place；And
Driving simulator for acquiring the driving data in model of place, and driving data is sent out by scene operation platform
It send to information management module and stores.
5. unmanned algorithm synthesis evaluation system according to claim 1, wherein described information management module include：
Information centre's unit：For completing scenario simulation module, automatic driving vehicle communication module and comprehensive test intermodule
Information exchange；And
Information memory cell：For recording experimentation, preserves the data generated in assessment process and storage in advance perceives true value
Data and driver's driving data, and data query is provided.
6. a kind of assessment method of the unmanned algorithm synthesis evaluation system based on driving simulator, including：
Step A：Test and appraisal design requirement is determined in test and appraisal design cell, and test and appraisal design requirement is sent by information management module
To scenario simulation module；Model of place is built by virtual scene software in scene analog module；
Step B：In measurement and management Unit selection Job evaluation mode；Automatic driving vehicle communication module is wanted according to measurement and management module
It asks, correspondence opens automatic driving vehicle PORT COM；
Step C：Under original state, according to selected Job evaluation mode, corresponding data processing is completed；
Step D：Model of place brings into operation in scene operation platform, automatic driving vehicle communication module, scenario simulation module and
Comprehensive test module will be interacted by information management module start information；And
Step E：Measurement and management unit obtains evaluating result according to internal algorithm and interaction data, completes test and appraisal, is produced during test and appraisal
Raw data will be automatically saved to information management module；Check the evaluating result that measurement and management unit generates, analysis test and appraisal number
According to.
7. the assessment method of unmanned algorithm synthesis evaluation system according to claim 6, wherein under original state,
The corresponding data processing method of Job evaluation mode includes in the step C：
Sub-step C1：When selecting the test and appraisal of vehicle grade, the driving of model of place is completed by the mankind, scene operation platform will drive
Member's driving data is sent to information memory cell and is stored, and driver's driving data is transferred by information centre's unit, sends
To measurement and management unit；And
Sub-step C2：When selecting module level test and appraisal/system-level test and appraisal, the sense being previously stored is transferred by information centre's unit
Know Truth data, is sent to measurement and management unit.
8. the assessment method of unmanned algorithm synthesis evaluation system according to claim 7, wherein selection sub-step C1
Afterwards in the process of running, the data source of the vehicle grade test and appraisal includes：
The image data and contextual data of the model of place for test transmitted by scene operation platform；
Automatic driving vehicle human-machine interaction data；And
Driver's driving data of acquisition.
9. the assessment method of unmanned algorithm synthesis evaluation system according to claim 7, wherein selection sub-step C2
Afterwards in the process of running, the data source of the module level test and appraisal/system-level test and appraisal includes：
The image data and contextual data of the model of place for test transmitted by scene operation platform；
The perception decision data and actuator director data that automatic driving vehicle algorithm generates；
Automatic driving vehicle human-machine interaction data；And
The perception Truth data being previously stored in information memory cell.
10. the assessment method of unmanned algorithm synthesis evaluation system according to claim 6, taken in wherein step A
The model of place built is to meet the model of place of real world, or to be required to combine existing model of place according to test and appraisal.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711362553.1A CN108319249B (en) | 2017-12-15 | 2017-12-15 | Unmanned driving algorithm comprehensive evaluation system and method based on driving simulator |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711362553.1A CN108319249B (en) | 2017-12-15 | 2017-12-15 | Unmanned driving algorithm comprehensive evaluation system and method based on driving simulator |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108319249A true CN108319249A (en) | 2018-07-24 |
| CN108319249B CN108319249B (en) | 2020-10-30 |
Family
ID=62892336
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711362553.1A Active CN108319249B (en) | 2017-12-15 | 2017-12-15 | Unmanned driving algorithm comprehensive evaluation system and method based on driving simulator |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108319249B (en) |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109470498A (en) * | 2018-09-27 | 2019-03-15 | 交控科技股份有限公司 | Fully automatic operation system integration testing method |
| CN109557904A (en) * | 2018-12-06 | 2019-04-02 | 百度在线网络技术（北京）有限公司 | A kind of test method, device, equipment and medium |
| CN109614886A (en) * | 2018-11-23 | 2019-04-12 | 中国人民解放军63966部队 | It is a kind of nobody/performance evaluation methodology and device of autonomous driving system |
| CN109765060A (en) * | 2018-12-29 | 2019-05-17 | 同济大学 | A kind of automatic driving vehicle traffic coordinating virtual test system and method |
| CN109782737A (en) * | 2019-01-28 | 2019-05-21 | 平安科技（深圳）有限公司 | Unmanned vehicle analog detection method and system |
| CN109816014A (en) * | 2019-01-22 | 2019-05-28 | 天津大学 | Generate method of the deep learning target detection network training with labeled data collection |
| CN110046833A (en) * | 2019-05-13 | 2019-07-23 | 吉林大学 | A kind of traffic congestion auxiliary system virtual test system |
| CN110673590A (en) * | 2019-10-21 | 2020-01-10 | 安徽江淮汽车集团股份有限公司 | Method, device and equipment for judging reliability of automatic driving system and storage medium |
| CN111006879A (en) * | 2019-12-25 | 2020-04-14 | 畅加风行(苏州)智能科技有限公司 | Port environment-oriented man-machine hybrid driving automatic driving test method and test platform thereof |
| CN111859597A (en) * | 2019-04-18 | 2020-10-30 | 北京京东尚科信息技术有限公司 | Evaluation method and system of automatic driving algorithm |
| CN111983935A (en) * | 2020-08-19 | 2020-11-24 | 北京京东叁佰陆拾度电子商务有限公司 | Performance evaluation method and device |
| CN112924185A (en) * | 2021-01-22 | 2021-06-08 | 大连理工大学 | Human-computer co-driving test method based on digital twin virtual-real interaction technology |
| CN113420952A (en) * | 2021-05-17 | 2021-09-21 | 同济大学 | Automobile human-computer interaction testing and evaluating system based on simulated driving |
| CN114755035A (en) * | 2022-06-15 | 2022-07-15 | 中汽信息科技(天津)有限公司 | Intelligent driving multidimensional test method based on vehicle-mounted terminal |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102495631A (en) * | 2011-12-09 | 2012-06-13 | 中国科学院合肥物质科学研究院 | Intelligent control method of driverless vehicle tracking desired trajectory |
| CN202815590U (en) * | 2012-07-30 | 2013-03-20 | 中国航天科工集团第三研究院第八三五七研究所 | Control system for mini self-driving unmanned vehicle |
| US8920345B2 (en) * | 2008-12-07 | 2014-12-30 | Apdm, Inc. | System and apparatus for continuous monitoring of movement disorders |
| CN105388021A (en) * | 2015-10-21 | 2016-03-09 | 重庆交通大学 | ADAS virtual development and test system |
| CN106153352A (en) * | 2016-07-04 | 2016-11-23 | 江苏大学 | A kind of automatic driving vehicle test and verification platform and method of testing thereof |
| CN106873397A (en) * | 2017-01-23 | 2017-06-20 | 同济大学 | Intelligent network joins automobile " hardware in loop " accelerated loading emulation test system |
-
2017
- 2017-12-15 CN CN201711362553.1A patent/CN108319249B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8920345B2 (en) * | 2008-12-07 | 2014-12-30 | Apdm, Inc. | System and apparatus for continuous monitoring of movement disorders |
| CN102495631A (en) * | 2011-12-09 | 2012-06-13 | 中国科学院合肥物质科学研究院 | Intelligent control method of driverless vehicle tracking desired trajectory |
| CN202815590U (en) * | 2012-07-30 | 2013-03-20 | 中国航天科工集团第三研究院第八三五七研究所 | Control system for mini self-driving unmanned vehicle |
| CN105388021A (en) * | 2015-10-21 | 2016-03-09 | 重庆交通大学 | ADAS virtual development and test system |
| CN106153352A (en) * | 2016-07-04 | 2016-11-23 | 江苏大学 | A kind of automatic driving vehicle test and verification platform and method of testing thereof |
| CN106873397A (en) * | 2017-01-23 | 2017-06-20 | 同济大学 | Intelligent network joins automobile " hardware in loop " accelerated loading emulation test system |
Non-Patent Citations (2)
| Title |
|---|
| GUANGMING XIONG: "《Research on the Quantitative Evaluation System for Unmanned Ground Vehicles》", 《2010 IEEE INTELLIGENT VEHICLES SYMPOSIUM》 * |
| 高洪波等: "《基于云模型的智能驾驶车辆变粒度测评研究》", 《电子学报》 * |
Cited By (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109470498A (en) * | 2018-09-27 | 2019-03-15 | 交控科技股份有限公司 | Fully automatic operation system integration testing method |
| CN109614886A (en) * | 2018-11-23 | 2019-04-12 | 中国人民解放军63966部队 | It is a kind of nobody/performance evaluation methodology and device of autonomous driving system |
| CN109557904B (en) * | 2018-12-06 | 2020-07-10 | 百度在线网络技术（北京）有限公司 | Test method, device, equipment and medium |
| CN109557904A (en) * | 2018-12-06 | 2019-04-02 | 百度在线网络技术（北京）有限公司 | A kind of test method, device, equipment and medium |
| CN109765060A (en) * | 2018-12-29 | 2019-05-17 | 同济大学 | A kind of automatic driving vehicle traffic coordinating virtual test system and method |
| CN109816014A (en) * | 2019-01-22 | 2019-05-28 | 天津大学 | Generate method of the deep learning target detection network training with labeled data collection |
| CN109782737A (en) * | 2019-01-28 | 2019-05-21 | 平安科技（深圳）有限公司 | Unmanned vehicle analog detection method and system |
| CN111859597A (en) * | 2019-04-18 | 2020-10-30 | 北京京东尚科信息技术有限公司 | Evaluation method and system of automatic driving algorithm |
| CN111859597B (en) * | 2019-04-18 | 2024-04-05 | 北京京东乾石科技有限公司 | Evaluation method and system of automatic driving algorithm |
| CN110046833A (en) * | 2019-05-13 | 2019-07-23 | 吉林大学 | A kind of traffic congestion auxiliary system virtual test system |
| CN110673590A (en) * | 2019-10-21 | 2020-01-10 | 安徽江淮汽车集团股份有限公司 | Method, device and equipment for judging reliability of automatic driving system and storage medium |
| CN110673590B (en) * | 2019-10-21 | 2021-02-05 | 安徽江淮汽车集团股份有限公司 | Method, device and equipment for judging reliability of automatic driving system and storage medium |
| CN111006879A (en) * | 2019-12-25 | 2020-04-14 | 畅加风行(苏州)智能科技有限公司 | Port environment-oriented man-machine hybrid driving automatic driving test method and test platform thereof |
| CN111006879B (en) * | 2019-12-25 | 2021-11-02 | 畅加风行(苏州)智能科技有限公司 | Port environment-oriented man-machine hybrid driving automatic driving test method and test platform thereof |
| CN111983935A (en) * | 2020-08-19 | 2020-11-24 | 北京京东叁佰陆拾度电子商务有限公司 | Performance evaluation method and device |
| CN111983935B (en) * | 2020-08-19 | 2024-04-05 | 北京京东叁佰陆拾度电子商务有限公司 | Performance evaluation method and device |
| CN112924185A (en) * | 2021-01-22 | 2021-06-08 | 大连理工大学 | Human-computer co-driving test method based on digital twin virtual-real interaction technology |
| CN112924185B (en) * | 2021-01-22 | 2021-11-30 | 大连理工大学 | Human-computer co-driving test method based on digital twin virtual-real interaction technology |
| CN113420952A (en) * | 2021-05-17 | 2021-09-21 | 同济大学 | Automobile human-computer interaction testing and evaluating system based on simulated driving |
| CN114755035A (en) * | 2022-06-15 | 2022-07-15 | 中汽信息科技(天津)有限公司 | Intelligent driving multidimensional test method based on vehicle-mounted terminal |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108319249B (en) | 2020-10-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108319249A (en) | Unmanned algorithm synthesis evaluation system based on driving simulator and method | |
| CN109327695A (en) | The test drives scene database system of virtual test Driving Scene for height reality | |
| US11755396B2 (en) | Generating autonomous vehicle simulation data from logged data | |
| Kaur et al. | A survey on simulators for testing self-driving cars | |
| Chen et al. | A novel integrated simulation and testing platform for self-driving cars with hardware in the loop | |
| CN110427827A (en) | It is a kind of it is multiple dimensioned perception and Global motion planning under autonomous driving network | |
| CN108921200A (en) | Method, apparatus, equipment and medium for classifying to Driving Scene data | |
| CN109765803A (en) | A kind of the simulation hardware test macro and method of the synchronic sky of the more ICU of autonomous driving vehicle | |
| CN114879631A (en) | Automatic driving test system and method based on digital twin cloud control platform | |
| CN107782564A (en) | A kind of automatic driving vehicle evaluation system and method | |
| CN107479532A (en) | The domain controller test system and method for a kind of intelligent automobile | |
| CN107015483A (en) | Method and apparatus for generating the test case for autonomous vehicle | |
| KR102139513B1 (en) | Autonomous driving control apparatus and method based on ai vehicle in the loop simulation | |
| Sun et al. | Acclimatizing the operational design domain for autonomous driving systems | |
| Mahr et al. | The contre (continuous tracking and reaction) task: A flexible approach for assessing driver cognitive workload with high sensitivity | |
| WO2019065409A1 (en) | Automatic driving simulator and map generation method for automatic driving simulator | |
| US20220198107A1 (en) | Simulations for evaluating driving behaviors of autonomous vehicles | |
| EP3849867A1 (en) | Assessing ride quality for autonomous vehicles | |
| US20220204009A1 (en) | Simulations of sensor behavior in an autonomous vehicle | |
| CN115291515A (en) | Automatic driving simulation test system and evaluation method based on digital twinning | |
| Chen et al. | NADS at the University of IOWA: A tool for driving safety research | |
| CN112781888A (en) | System and method for testing vehicle | |
| CN110426215B (en) | Model establishing method for vehicle ride comfort test and intelligent driving system | |
| Zhao et al. | Virtual traffic simulator for connected and automated vehicles | |
| Zhao et al. | Corroborative evaluation of the real-world energy saving potentials of inforich eco-autonomous driving (iread) system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |