CN103823747A - Automatic regression testing method - Google Patents
Automatic regression testing method Download PDFInfo
- Publication number
- CN103823747A CN103823747A CN201210462280.9A CN201210462280A CN103823747A CN 103823747 A CN103823747 A CN 103823747A CN 201210462280 A CN201210462280 A CN 201210462280A CN 103823747 A CN103823747 A CN 103823747A
- Authority
- CN
- China
- Prior art keywords
- regression
- test
- regression test
- level
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Abstract
The invention discloses an automatic regression testing method. The automatic regression testing method includes a first step, performing regression starting and running, in other words, respectively managing regression tests on different kinds of test vectors in a classified and graded manner according to specific conditions of projects, respectively selectively performing the module-level, subsystem-level or system-level regression tests for different stages of hierarchical verification and generating conventional information files and error information files; a second step, performing regression information post-processing, in other words, statistically analyzing each grade of regression test results, generating project regression home pages, generating module or regression classification branch pages and generating detailed regression result branch pages of each module. The project regression home pages contain project information, regression versions and coverage rates. The module or regression classification branch pages contain module classification type lists and pass or fail test case summaries. The detailed regression result branch pages of each module contain each test case name, simulation running time, random frequencies, case passing information, fail type statistics and simulation result conventional information file indexes. The automatic regression testing method has the advantage that the design verification process efficiency and the verification completeness can be improved by the aid of the automatic regression testing method.
Description
Technical field
The present invention relates to a kind of method of carrying out automatic regression test in the checking work of chip Front-end Design.
Background technology
Regression test occupies very important status in the checking of chip front end.Because along with chip complexity is more and more higher, the head end test vector needing is constantly refinement and increasing also, if the effective regression testing method of neither one is effectively managed it, whole checking work likely can be absorbed in a kind of duty of poor efficiency.
Along with the development of verification technique and method and complicated, for the regression test that occupies vital role in checking, new demand and challenge are also constantly proposed, how within the limited time, guarantee as far as possible the quality of checking work, improve Front-end Design checking flow process scientific and efficient, for the flow that chip is finally guaranteed the quality is given security on time.
Traditional regression testing method is normally by all test vector order arrangements successively that need operation, be equivalent to the each test vector of batch processing isolated operation, be separated by the long period at every turn, or move a regression test after making larger renewal, and in the end carry out check result by artificial or simple order.
Such regression testing method is lower in checking complexity, in the situation of test vector negligible amounts, can meet the demands; But along with becoming, design and verification environment become increasingly complex, the emphasis of test vector, and in the situation that quantity all improves constantly, traditional regression test is more and more difficult to meet different designs Qualify Phase to regression test requirement.As increasing of test vector quantity, make to be difficult to complete the regression test of all test vectors within an evening or short period, when problem can not obtain feeding back as early as possible like this, will hide some dangers for for further Change In Design.In addition, the regression test of long period operation of being separated by, also can make problem not feed back as early as possible, make to design the state of the function (performance) of already passing through in continuous modification, and new modification always likely brings new problem, so be absorbed in vicious cycle, finally cause design effort to be absorbed in the circulation of poor efficiency.Also have, if each by artificial or simple command check result, can only check roughly result, can not provide the key message of each test vector, unless checked successively the simulation result of each test vector, if and in the many situation of failed test vector, such way is also difficult to implement; And also may produce artificial error-detecting result by hand inspection result.
In order to make up traditional regression tested deficiency, meet the challenge that project proposes checking work, need regression test to accomplish every day latest update to be carried out to regression test at the critical stage of design verification, by possible defect or Failure elimination in bud, regression test simultaneously can provide the key message of failure testing vector, meet the needs that accelerate debugging, improve the efficiency of design verification work with this.
Summary of the invention
The problem to be solved in the present invention is to provide a kind of method of automatic regression test, can improve the completeness of design verification flow path efficiency and checking.
For solving the problems of the technologies described above, the method for automatic regression test of the present invention, comprises the steps:
Step 1, recurrence Start-up and operating performance, according to project concrete condition, the regression test of variety classes test vector is carried out to classifying type classification and don't bother about reason, for project reality, in the different phase of stratification checking, select respectively to carry out module level, subsystem irrespective of size or system-level regression test, and produce routine information file and error message file;
Step 2, the aftertreatment of recurrence information, carry out statistical study for the regression test result of every kind of rank, and generation project respectively returns homepage, comprises project information and return version, coverage rate; Generation module classification paging or return classification paging, comprise module classification list of types and by or failed test case gather; The detailed regression result paging that produces each module, comprises each test case title, and the simulation run time, whether random number of times, pass through, Fail Type statistics, simulation result routine information file index.
The present invention uses full-automatic script operation, by configuration regression tested parameter, starts regression test; Checking slip-stick artist, by the configuration file that returns startup or the aftertreatment of the information of recurrence is carried out to simple modification, just can be relatively easy between disparity items multiplexing; Variety classes test vector can be carried out to classify and grading and don't bother about reason, and adopt the regression test of different stage, the regression test of reasonable arrangement different phase for project reality in the different phase of stratification checking.After regression test finishes, regression test result is processed in robotization, can produce the good recurrence report of user interface, this recurrence report comprises historical record and returns at every turn, the key message of each module and each test vector, facilitate the quick location defect of slip-stick artist or debug, can effectively improve the checking flow path efficiency of chip Front-end Design and the completeness of checking; Can with verification environment collaborative work, match with the design verification automated environment of third party's instrument and subsidy exploitation, easily with third party's tool interface, become a part for whole design verification environment, form unified design verification robotization platform, what automaticity was higher completes test result inspection.
Accompanying drawing explanation
Below in conjunction with accompanying drawing and embodiment, the present invention is further detailed explanation:
Accompanying drawing is the method control flow chart of described automatic regression test.
Embodiment
The method of described automatic regression test is based on the method such as script and HTML(Hypertext Markup Language) exploitation, applied to as if carry out automatic returning management for the checking flow process of chip Front-end Design.In the situation that verification method and test and excitation are constantly complicated, can all test vectors of more effective management, improve the checking flow process of chip Front-end Design, efficiently carry out design verification work.
Because automatized script is carried out the demand for reusability, need to carry out parametrization processing as much as possible to project and verification environment; Method the dependent of described automatic regression test are present in verification environment, but need to carry out some necessary base conditioning.In short, mainly comprise two parts: be first that project starts the front parameter configuration that project is normalized, relevant design verification environment variable is set and calls configuration, to implement in the future the required information of method of described automatic regression test by parameter extraction; Secondly need to aspect verification platform and excitation, carry out rule and set, comprise bibliographic structure, naming rule etc.Putting and can realize automatic regression test before this.
Shown in accompanying drawing, the method for described automatic regression test, comprises the steps: to return Start-up and operating performance, returns information aftertreatment.
Described recurrence Start-up and operating performance step, the parameter configuration before project starts, project being normalized, configuration design verification environmental correclation parameter and variable, arrange relevant design verification environment and call configuration; Start automatic regression test by single command row or periodicity fill order (CRONTAB); Call Version command synchronization design verification environment and design source code, and record relevant item recurrence version number; According to project concrete condition, the regression test of test vector is carried out to classifying type management, according to startup command row, carry out regression test type analysis, and select to enter corresponding regression test type according to regression test type analysis result; According to startup command row and the regression test type entering, carry out regression test grade analysis, and select to enter appropriate level regression test according to regression test grade analysis result.
Described regression test type, comprises basic recurrence (SANITY), returns fast (QUICK), daily recurrence (NIGHTLY), and entirety returns (ROTATE), and single excitation returns (TEST) five types of regression tests.
Wherein:
The regression test of described SANITY type, being mainly used in Method at Register Transfer Level code makes larger modification and is confirmed whether fast afterwards to produce the regression test of additional defect, after design engineer's Method at Register Transfer Level code update, be confirmed whether the regression test that can exert an influence to other partial design.
The regression test of described QUICK type, is often used in design engineer at every turn to the quick regression test after Method at Register Transfer Level code update, with described SANITY type regression test function class seemingly, but regression test scale is less.
The regression test of described NIGHTLY type, is a kind of regression test the most frequently used in project proof procedure, and carry out regression test every night checking middle and later periods that is generally used for chip Front-end Design, produces regression test report every day, tracking and testing process.
The regression test of described ROTATE type, be used for time-consuming long (being the recurrence that once all excitations are carried out in chip flow before) that before chip flow, need operation to confirm, operation or need last confirmation and there is no need the back and forth test vector of operation, before chip flow or carry out the regression test of several times ROTATE type vacation long period.
The regression test of described TEST type is used for supporting each test vector to carry out single artificial debugging.
Every kind of regression test type comprises module level, subsystem irrespective of size, the regression test of system-level three ranks.
For described module level regression test and the regression test of subsystem irrespective of size, need to carry out successively test vector configuration, random number configuration, contoured configuration, the parameters such as coverage rate configuration.
Need to carry out subclass analysis judgement for described system-level regression test, then select corresponding subclass regression test.
System-level regression test comprises Method at Register Transfer Level (RTL) regression test, field programmable gate array level (FL) regression test and three subclass of net table level regression test (GSIM).Method at Register Transfer Level comprises: module level (BL), subsystem irrespective of size (SS), four sub-rank regression tests of system-level (SL) and software-hardware synergism level (IC); Described Method at Register Transfer Level regression test is carried out after sub level do not select, entering the other regression test of corresponding sub level.If net table level, needing to analyze is the regression test that belongs to that sequential kind.The regression test of net table level comprises: the regression test (PRE) of pre-emulation before placement-and-routing, worst case Regression test (WC), best-case Regression test (BC), low temperature regression test (BCLT) under regression test in typical case (TC) and best-case, the sequential kind of the standard cell lib that its classification situation should provide according to wafer production factory increases or reduces.On this basis system-level regression test is carried out to interface configuration, encryption configuration, contoured configuration, coverage rate configuration.According to all configurations of carrying out before, operation regression test, and produce routine information file and error message file.
Before returning startup, need the excitation file designation form of emulation to standardize, shown in specifically seeing table.
Wherein, item domains can directly obtain from design verification environmental variance; Level domain need to be set according to pre-specified naming rule according to different regression test ranks; Module territory need to be set according to the naming rule of module and the function of verifying; Functional domain need to simply be set according to the unique point that will verify, without the rule of predetermining.
Before returning Start-up and operating performance, in verification platform, need the type information of emulation to standardize, mainly comprise:
Module start information, is used for recording specific incentives file and belongs to some specific modules.
Module ending message, is used for recording the whether normal termination of a module testing.
Excitation start information, is used as the beginning of an excitation.
Excitation-off information, is used for recording the normal termination of an excitation, by or failure information, record an excitation and whether pass through.
Simulation run temporal information, is used for recording a specific incentives consuming time.
If module level and subsystem level verification, adopt random number as excitation, also need multiplicity as random number of times information etc.Example below as explanation:
Dv:suite SCI7816 STARTED representation module returns and starts;
Dv:test SCI7816_SEND STARTED represents to encourage emulation to start;
. start the intermediate information of emulation;
. start the intermediate information of emulation;
. start the intermediate information of emulation;
Dv:test SCI7816_SEND PASSED represents that emulation passes through;
CPU time 114.70 represents the simulation run time;
Dv:test SCI7816_SEND repeat mumber=10 represents random number of times;
Dv:test SCI7816_SEND FINISHED represents to encourage emulation to finish;
Dv:suite SCI7816 FINISHED representation module returns and finishes.
Above-mentioned is mainly the necessary standardization processing that naming rule and type information are carried out, and need to before recurrence starts, complete, while carrying out regression test, directly select certain module of customization to return according to input parameter, certain type returns, or full chip recurrence, gives an example as follows:
If carry out the recurrence of SCI7816 module level, only need action command row: regress.cshSCI7816.Be not with other parameters acquiescence to produce wave file, and random number of times be set for once.
If carry out the operation of the full chip of subsystem irrespective of size, need action command row: regress.cshsubsystem off1 wherein " off " represents to close and produces waveform option, and random number of times is set for once.
If carry out the recurrence of SANITY type, only need action command row: regress.csh sanity, project regression tested booting script can select typical excitation to return fast according to the option of internal configurations automatically.
It should be noted that, just start and return here, return report as produced, must start the regression test result treatment script of mentioning below and produce report.
Described recurrence Start-up and operating performance, adopts project regression tested booting script, starts, and can be common to module level, subsystem irrespective of size and system-level by single command row or periodicity fill order.Described project regression tested booting script, supports automatic synchronization Version, and can judge it is module level regression test, the regression test of subsystem irrespective of size or system-level regression test according to input parameter; For every kind of rank regression test, support to produce routine information file directory and error message file directory, collect respectively general information and error message, and it is carried out to difference management in the different phase returning.Support according to input parameter select whether the source code of storer to be encrypted.Support to determine whether producing coverage rate report according to coverage rate input parameter.Support according to Waveform switch parameter, whether configuration needs to produce waveform, and supports to make mistakes help information etc.
Described project regression tested booting script, comprises project information, the checking of operation or recurrence rank, and necessary path parameter setting, instrument calls and version setting, with the collaborative work of design verification robotization platform.If when having new projects to create or need to multiplexingly transferring to new projects, can carry out easy configuration to project regression tested booting script, after this only need to or enable project regression tested booting script described in periodically fill order control in operation every day, just can realize regression test and move and produce final regression test result.
Because can complete all regression tests and interpretation of result process by simple command is capable, therefore can be convenient to use periodically fill order and carry out automatic cycling operating, coordinate dissimilar regression test, can arrange in pairs or groups flexibly, meet the needs of various regression test types.
Described module level regression test adopts described project regression test booting script to start by configuration parameter, module difference simultaneously that can test as required, and option and installment module name is realized the regression test of concrete certain module as input.The main SCI7816 that supports in two interface series products, the checking of the module level such as controller (EECTRL) and radio frequency interface controller (RFIF) and the regression test thereof of EEPROM, need to configure described project regression test booting script if there is new module to add.According to the stray parameter of input, the quantity of the packet of configuration random test, the packet by the random number in the stray parameter replacement test vector of input as random test; If there is no the stray parameter of input, automatically produce the quantity of the packet of random test according to the random number in test vector.According to the stray parameter of input, be configured for regression tested test vector, if there is no the stray parameter of input, the test vector in all configuration files of acquiescence operation, carries out regression test according to the test vector of listing.
The regression test of described subsystem irrespective of size adopts described project regression test booting script to start by configuration parameter, can set by input stray parameter the quantity of the packet of random test, if do not input the value of random test, call the packet numerical value that test vector carries; Can specify the test vector that needs recurrence by input test vector lists file, if omit input test vector lists file, the test vector list in Automatically invoked configuration file.
Described system-level regression test adopts described project regression test booting script to start by configuration parameter, different from module level regression test and the regression test of subsystem irrespective of size is, system-level regression test is because adopt firmware directly to carry out emulation, the mechanism that does not have the packet of random test to produce in test vector, does not need to input stray parameter yet.But the regression test of the different subclass such as Method at Register Transfer Level, field programmable gate array level and net table level that system-level regression test is integrated.If regression test is for Method at Register Transfer Level, described project regression test booting script does not need to configure net table sequential type parameter, and the regression test of register transfer level system and software-hardware synergism level regression tested design verification environment and script configuration parameter are completely multiplexing.As regression test is used for netting table level, so need to be in the order line of described project regression test booting script using sequential type as input parameter, as previously described best, the worst, typical case, low temperature regression test etc. under best-case.Described recurrence booting script be responsible for by all need test excitations move, and after end of run according to the difference of input timing type, produce different routine information daily records and error message daily record.Conventional and error message file designation rule adds that according to excitation the configuration of sequential type names automatically, as, move the system-level net table test and excitation of a SCI7816, reactionary slogan, anti-communist poster type is WC, the journal file name producing is called " TC.2901M4.SS.SCI7816_RECEIVE_sdfwc ", to facilitate subsequent treatment.
The regression test actuating section of more than supporting for the present invention, complete when regression test operation, need to process regression test result, enter recurrence information post-processing step, specifically comprise the following steps:
Step 1, according to input parameter judge be the regression test of that rank.Support of the present invention comprises module level, subsystem irrespective of size, system-level regression test.
Step 2, the source code version number of collecting the Version (or other Versions) that this regression test uses, debug or reappear after convenient.
Step 3, according to current date, produce file take current time as title, after this all processing and report will move in this file.
Step 4, operation regression test result treatment script are processed the routine information file of collecting and error message file, and are produced regression test detailed report.
Step 5, determine whether certain regression tested coverage rate result is merged according to input parameter, produce coverage rate report structure.
Step 6, according to the result of all regression tests report under all regression test catalogues, produce the top level page of this project regression test report, this top level page comprises entry name, historical regression test report results link, and coverage rate link etc.
Described recurrence information post-processing stages only need to be inputted regression test class information, this class information can be system-level, subsystem or module level, also can be one of them sequential type of net table level, the entry program of recurrence information aftertreatment can check and record together with system information current project information, and regression test type parameter is passed to routine information documentor.
Described routine information documentor comprises top layer routine information documentor and bottom routine information documentor.
Described top layer routine information documentor be used for generation project return homepage (comprising: project information, return version, coverage rate), after having moved, recurrence comprises recurrence analysis of version as shown in FIG., return version generation and produce the meetings such as coverage rate link and complete in this stage, be to call third party's emulation tool automatically according to returning script startup parameter generating and produce coverage rate, the present invention just directly calls the coverage rate result having produced.Described top layer routine information documentor is classified to the module of required processing according to the regression test type parameter of input, and module classification is set in regression test result treatment script, therefore can adjust according to project difference.In top layer routine information documentor header file, generation project regression tested is reported top layer HTML(HTML (Hypertext Markup Language) automatically) form web page files, this web page files comprises project name and links according to the recurrence version information producing working time and coverage rate, and calls bottom routine information documentor and process in detail.Can gather regression tested old version, link the regression test report of different levels in each version, produce the total page of project regression test (html format) simultaneously.
Described bottom routine information documentor is used for producing the total page of regression test, comprise module classification paging or return classification paging (comprise module classification list of types and by or failed test case gather), and the detailed simulation result page of test vector that will generation regression test total page subordinate.Need advanced line retrace type analysis in this stage as shown in the figure, return grade analysis.Bottom routine information documentor is processed according to regression test type parameter and the module information of input, generation module classification paging returns link, the type parameter that wherein regression test type parameter is received with top layer is consistent, and module information is generated by analysis by top layer routine information documentor.Then the labor that carries out each recurrence excitation result, analytic routines (log) message file/mistake (err) message file as shown in the figure, comprises and carries out regression test grade analysis, analysis of simulation result, simulation run time series analysis, random number analysis; The detailed regression result paging that produces each module, comprises each test case title, and the simulation run time, whether random number of times, pass through, Fail Type statistics, simulation result routine information file index.Produce the detailed simulation result page that each test vector comprises key message and according to the total page of the regression test of module classification with this; According to the simulation result of each test vector, produce vector report, as certain test vector emulation failure, enter error analysis branch, carry out the preliminary location automatically of defect according to routine information file and error message file content.
As a part for design verification automated environment, described automatic regression test method can or be subsidized the collaborative work of developing instrument script with other third party's instruments, automatic regression test need to support to comprise that different emulators rise scientific and technological emulation tool as NCverilog(clang), VCS(Synopsys emulator) etc.; Different wave produces instrument as VPD(Synopsys acquiescence generation wave file), FSDB(source sheng, a reed pipe wind instrument science and technology wave file, is now purchased by Synopsys), the reference waveform file that the most of instruments of VCD(provide commercial city to support) etc.; Described automatic regression test method is supported Distributed Calculation instrument, as the Distributed Calculation instrument of SGE(Sun Company Inc.) and the Distributed Calculation instrument that proposes of LSF(International Business Machines Corporation); In the time that regression test moves, can call executed in parallel instrument (LSF or SGE) and carry out Distributed Calculation, improve regression test efficiency, guarantee that daily type recurrence can complete; And can with Synopsys(Synopsys) emulation produce coverage rate (coverage) combine, for each recurrence version produces corresponding coverage rate situation report ".The html format form web page report in the end generating can cross-platformly conduct interviews.
More than pass through detailed description of the invention; the principle of the invention and using method are introduced; do not departing from principle of the invention situation; those skilled in the art can also make many distortion and improvement; as for different verification platforms, different verification environments, need to configure different parameters information; or the report producing can comprise different key message etc., these optimizations and change also should be considered as protection scope of the present invention.
Claims (17)
1. the method for an automatic regression test, it is characterized in that, comprise the steps: step 1, return Start-up and operating performance, according to project concrete condition, the regression test of variety classes test vector is carried out to classifying type classification and don't bother about reason, different phase for project reality in stratification checking, selects to carry out respectively module level, subsystem irrespective of size or system-level regression test; After regression test operation, produce routine information file and error message file;
Step 2, the aftertreatment of recurrence information, carry out statistical study for the regression test result of every kind of rank, and generation project respectively returns homepage, comprises project information and return version, coverage rate; Generation module classification paging or return classification paging, comprise module classification list of types and by or failed test case gather; The detailed regression result paging that produces each module, comprises each test case title, and the simulation run time, whether random number of times, pass through, Fail Type statistics, simulation result routine information file index.
2. the method for claim 1, it is characterized in that: described in implementation step one, return Start-up and operating performance in the following way, the parameter configuration before project starts, project being normalized, configuration design verification environmental correclation parameter and variable, relevant design verification environment is set and calls configuration, start automatic regression test by single command row or periodicity fill order; Call Version command synchronization design verification environment and design source code, and record relevant item recurrence version number; According to startup command row, carry out regression test type analysis, and select to enter corresponding regression test type according to regression test type analysis result; According to startup command row and the regression test type entering, carry out regression test grade analysis, and select to enter appropriate level regression test according to regression test grade analysis result.
3. method as claimed in claim 1 or 2, is characterized in that, regression test type described in step 1, comprises following five kinds, respectively:
Basic recurrence, after Method at Register Transfer Level code update, can exert an influence to other partial design for being confirmed whether;
Return fast, after at every turn to Method at Register Transfer Level code update, can exert an influence to other partial design for being confirmed whether, its regression test scale is less than described basic recurrence type;
Daily recurrence, in the checking middle and later periods of chip Front-end Design, for carrying out regression test every night, produces regression test report every day, tracking and testing process;
Entirety returns, and for carrying out the recurrence of once all excitations before chip flow, operation or need last confirmation and there is no need the back and forth test vector of operation, before chip flow or carry out repeatedly entirety vacation and return type regression test;
Single excitation returns, for supporting each test vector to carry out single artificial debugging;
Every kind of regression test type comprises module level, subsystem irrespective of size, system-level three ranks.
4. the method for claim 1, is characterized in that: before described module level regression test and subsystem irrespective of size regression test operation, carry out successively test vector configuration, random number configuration, contoured configuration and coverage rate configuration.
5. the method for claim 1, is characterized in that: described system-level regression test comprises: Method at Register Transfer Level, field programmable gate array level and three subclass regression tests of net table level; Described system-level regression test is carried out after subclass selection, entered corresponding subclass regression test;
Described Method at Register Transfer Level regression test comprises: module level, subsystem irrespective of size, system-level and software-hardware synergism level regression test; Described Method at Register Transfer Level regression test is carried out after sub level do not select, entering the other regression test of corresponding sub level;
The regression test of described net table level, comprising: before placement-and-routing, the regression test PRE of pre-emulation, worst case Regression are tested five kinds of sequential types of low temperature regression test BCLT under WC, best-case Regression test b C, in typical case regression test TC and best-case; To described net table level, regression test is carried out after sequential type selecting, enters the regression test of corresponding time sequence type;
Before described system-level regression test operation, carry out successively interface configuration, encryption configuration, contoured configuration and coverage rate configuration.
6. the method for claim 1, is characterized in that: before returning startup, the excitation file designation form of emulation is standardized; Wherein, item domains directly obtains from design verification environmental variance; Level domain, for different regression test ranks, sets according to pre-specified naming rule; Module territory, sets according to the naming rule of module and the function verified; Functional domain, sets according to verified unique point, without the rule of predetermining.
7. the method for claim 1, is characterized in that: before returning Start-up and operating performance, the type information to emulation in verification platform is standardized; Comprise:
Module start information, is used for recording specific incentives file and belongs to some specific modules;
Module ending message, is used for recording the whether normal termination of a module testing;
Excitation start information, is used as the beginning of an excitation;
Excitation-off information, is used for recording the normal termination of an excitation, by or failure information, whether an excitation is passed through;
Simulation run temporal information, is used for recording a specific incentives consuming time;
If module level and the regression test of subsystem irrespective of size adopt random number as excitation, need multiplicity as random number of times information.
8. the method for claim 1, is characterized in that: described in step 1, return Start-up and operating performance, adopt project regression tested booting script, start by single command row or periodicity fill order; This project regression tested booting script, supports automatic synchronization Version, judges it is module level regression test, the regression test of subsystem irrespective of size or system-level regression test according to input parameter; For every kind of rank regression test, support to produce routine information file directory and error message file directory, collect respectively general information and error message, and it is carried out to difference management in the different phase returning; Support according to input parameter select whether the source code of storer to be encrypted; Support to determine whether producing coverage rate report according to coverage rate input parameter; Support according to Waveform switch parameter, whether configuration needs to produce the type of waveform and waveform; The support help information of makeing mistakes.
9. method as claimed in claim 8, is characterized in that: project regression tested booting script, comprises project information, the checking of operation or recurrence rank, necessary path parameter setting, instrument calls and version setting, with the collaborative work of design verification robotization platform;
If when having new projects to create or need to multiplexingly transferring to new projects, can carry out easy configuration to project regression tested booting script, after this only need to or enable project regression tested booting script described in periodically fill order control in operation every day, just can realize regression test and move and produce final regression test result.
10. the method as described in claim 1,8 or 9, it is characterized in that: described module level regression test adopts described project regression test booting script to start by configuration parameter, the module difference of test as required, option and installment module name is realized the regression test of concrete module as input; Need to configure described project regression test booting script if there is new module to add;
According to the stray parameter of input, the quantity of the packet of configuration random test, the packet by the random number in the stray parameter replacement test vector of input as random test; If there is no the stray parameter of input, automatically produce the quantity of the packet of random test according to the random number in test vector;
According to the stray parameter of input, be configured for regression tested test vector, if there is no the stray parameter of input, the test vector in all configuration files of acquiescence operation, carries out regression test according to the test vector of listing.
11. the method as described in claim 1,8 or 9, is characterized in that: described subsystem irrespective of size regression test adopts project regression test booting script to start by configuration parameter, with the stray parameter of input, set the quantity of the packet of random test; If do not input stray parameter, call the packet numerical value that test vector carries; Specify the test vector that needs recurrence with input test vector lists file, if omit input test vector lists file, the test vector list in Automatically invoked configuration file.
12. methods as described in claim 5,8 or 9, is characterized in that: described system-level regression test adopts described project regression test booting script to start by configuration parameter; Wherein,
Method at Register Transfer Level regression test and software-hardware synergism level regression tested design verification environment and script configuration parameter are completely multiplexing;
The regression test of net table level, in the order line of described project regression test booting script using sequential type as input parameter, described project returns booting script and is responsible for all excitations that need test to move, and after end of run according to the difference of input timing type, produce different routine information daily records and error message daily record; Routine information file and error message file designation rule add that according to excitation name the configuration of sequential type names automatically.
13. the method for claim 1, it is characterized in that: support with third party's instrument or subsidize the collaborative work of developing instrument script, support different emulators, different wave produces instrument, support Distributed Calculation instrument, in the time that regression test moves, Selection and call executed in parallel instrument carries out Distributed Calculation as required; And can combine with the coverage rate that Synopsys Synopsys emulation produces, for each recurrence version produces corresponding coverage rate situation report; The top layer HTML (Hypertext Markup Language) html format form web page report in the end generating can cross-platformly conduct interviews.
14. the method for claim 1, is characterized in that: described in step 2, return information aftertreatment, specifically comprise the following steps:
Step 1, according to input parameter judge be the regression test of that rank;
Step 2, collect the Version that this regression test is used, debug or reappear after convenient;
Step 3, according to current date, produce file take current time as title, after this all processing and report will move in this file;
Step 4, operation regression test result treatment script are processed the routine information file of collecting and error message file, and are produced regression test detailed report;
Step 5, determine whether certain regression tested coverage rate result is merged according to input parameter, produce coverage rate report structure;
Step 6, according to the result of all regression tests report under all regression test catalogues, produce the top level page of this project regression test report, this top level page comprises entry name, historical regression test report results link, the link of machine coverage rate.
15. methods as described in claim 1 or 14, it is characterized in that: described in step 2, return information aftertreatment, only need to input regression test class information, this class information can be system-level, subsystem or module level can be also one of them sequential types of system-level middle net table level; The entry program of recurrence information aftertreatment can check and record together with system information current project information, and regression test rank parameter is passed to routine information documentor;
Described routine information documentor comprises top layer routine information documentor and bottom routine information documentor.
16. methods as claimed in claim 15, is characterized in that: described top layer routine information documentor generation project returns homepage; According to the regression test type parameter of input, the module of required processing is classified, and module classification is set in regression test result treatment script, adjusts according to project difference;
In top layer routine information documentor header file, generation project regression tested is reported top layer HTML (Hypertext Markup Language) html format web page files automatically, this web page files comprises project name and links according to the recurrence version information producing working time and coverage rate, and calls bottom routine information documentor and process in detail; Gather regression tested old version simultaneously, link the regression test report of different levels in each version, produce the total page of project regression test of HTML (Hypertext Markup Language) html format.
17. methods as claimed in claim 16, it is characterized in that: described bottom routine information documentor produces the total page of regression test, comprise module classification paging or return classification paging, and will produce the total page subordinate's of regression test the detailed simulation result page of test vector;
Process according to regression test type parameter and the module information of input, generation module classification paging returns link, the type parameter that wherein regression test type parameter is received with top layer routine information documentor is consistent, and module information is to be produced by analysis by top layer routine information documentor; The labor that carries out each recurrence excitation result, comprises and carries out regression test grade analysis, analysis of simulation result, simulation run time series analysis, random number analysis; Produce the detailed regression result paging of each module; Produce the detailed simulation result page that each test vector comprises key message and according to the total page of the regression test of module classification; According to the simulation result of each test vector, produce vector report, as certain test vector emulation failure, enter error analysis branch, carry out the preliminary location automatically of defect according to routine information file and error message file content.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210462280.9A CN103823747B (en) | 2012-11-16 | 2012-11-16 | The method of automatic regression test |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210462280.9A CN103823747B (en) | 2012-11-16 | 2012-11-16 | The method of automatic regression test |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103823747A true CN103823747A (en) | 2014-05-28 |
| CN103823747B CN103823747B (en) | 2017-09-15 |
Family
ID=50758828
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210462280.9A Expired - Fee Related CN103823747B (en) | 2012-11-16 | 2012-11-16 | The method of automatic regression test |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103823747B (en) |
Cited By (25)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104679652A (en) * | 2015-02-15 | 2015-06-03 | 北京宇航系统工程研究所 | Markov method-based simulation system test case generating method |
| CN104794053A (en) * | 2015-04-22 | 2015-07-22 | 烽火通信科技股份有限公司 | Logic simulation verification regression testing system and method based on random use cases |
| CN105158681A (en) * | 2015-08-07 | 2015-12-16 | 广州中大微电子有限公司 | Radio frequency identification reader chip verification method and system |
| CN105389200A (en) * | 2015-12-16 | 2016-03-09 | 浪潮(北京)电子信息产业有限公司 | Multi-group stimulation batch simulation method and system applied to super-large scale chip verification |
| CN105487966A (en) * | 2014-09-17 | 2016-04-13 | 腾讯科技（深圳）有限公司 | Program testing method, device and system |
| CN105677996A (en) * | 2016-01-13 | 2016-06-15 | 浪潮集团有限公司 | Method for determining regression test version abnormity source in chip verification |
| CN105975924A (en) * | 2016-04-29 | 2016-09-28 | 杭州当虹科技有限公司 | Regression testing method for precisely recognizing ad content based on video frame statistics |
| CN106291312A (en) * | 2015-06-12 | 2017-01-04 | 超威半导体(上海)有限公司 | A kind of method emulating for speed-up chip and debugging and chip test system |
| CN106897485A (en) * | 2017-01-09 | 2017-06-27 | 北京中电华大电子设计有限责任公司 | Intelligence returns verification management system |
| CN107885925A (en) * | 2017-11-03 | 2018-04-06 | 中国电子科技集团公司第五十四研究所 | A kind of register testing method of EDA Qualify Phases |
| CN109240872A (en) * | 2018-08-08 | 2019-01-18 | 福州瑞芯微电子股份有限公司 | A kind of chip checking key signal coverage rate statistical analysis technique and storage medium |
| CN110232017A (en) * | 2019-04-30 | 2019-09-13 | 北京云迹科技有限公司 | Robot regression testing method and system under simulated environment |
| CN110348217A (en) * | 2019-05-28 | 2019-10-18 | 深圳壹账通智能科技有限公司 | Interface test method, device, electronic equipment and storage medium |
| CN111427794A (en) * | 2020-04-03 | 2020-07-17 | 天津飞腾信息技术有限公司 | Method, system and medium for accelerating simulation of storage component netlist |
| CN111428431A (en) * | 2020-02-28 | 2020-07-17 | 中科亿海微电子科技（苏州）有限公司 | Method and system for supporting automatic test and recording of EDA (electronic design automation) software |
| CN111859830A (en) * | 2020-06-30 | 2020-10-30 | 山东云海国创云计算装备产业创新中心有限公司 | Verification plan and report generation method, device, equipment and storage medium |
| CN112015665A (en) * | 2020-09-30 | 2020-12-01 | 北京百度网讯科技有限公司 | Test verification backtracking method and device, electronic equipment and storage medium |
| CN112131827A (en) * | 2020-09-11 | 2020-12-25 | 山东云海国创云计算装备产业创新中心有限公司 | Chip testing method, system, equipment and storage medium |
| CN112131512A (en) * | 2020-11-20 | 2020-12-25 | 中国人民解放军国防科技大学 | Method and system for website management script safety certification |
| CN112380127A (en) * | 2020-11-16 | 2021-02-19 | 海光信息技术股份有限公司 | Test case regression method, device, equipment and storage medium |
| CN112464596A (en) * | 2020-11-27 | 2021-03-09 | 海光信息技术股份有限公司 | Regression testing method, system, equipment and readable storage medium |
| CN112613255A (en) * | 2020-12-10 | 2021-04-06 | 海光信息技术股份有限公司 | Multi-environment collaborative regression method, device, equipment and storage medium |
| CN116306388A (en) * | 2023-05-23 | 2023-06-23 | 苇创微电子(上海)有限公司 | Automatic UVM verification platform free of path connection and construction method thereof |
| CN116776793A (en) * | 2023-08-22 | 2023-09-19 | 成都翌创微电子有限公司 | Multi-period path constraint verification method combining static time sequence analysis and pre-simulation |
| CN117034821A (en) * | 2023-10-09 | 2023-11-10 | 芯耀辉科技有限公司 | Regression verification method and medium for chip design front-end simulation verification |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040060044A1 (en) * | 2002-09-20 | 2004-03-25 | International Business Machines Corporation | Method and apparatus for automatic updating and testing of software |
| CN101178687A (en) * | 2007-11-26 | 2008-05-14 | 中兴通讯股份有限公司 | Software regression testing method |
| CN101329652A (en) * | 2008-07-30 | 2008-12-24 | 中兴通讯股份有限公司 | Regression test automatic system and method |
| CN102095448A (en) * | 2010-12-08 | 2011-06-15 | 江苏天泽信息产业股份有限公司 | Automatic testing system of production line of vehicle information terminal and testing method of system |
-
2012
- 2012-11-16 CN CN201210462280.9A patent/CN103823747B/en not_active Expired - Fee Related
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040060044A1 (en) * | 2002-09-20 | 2004-03-25 | International Business Machines Corporation | Method and apparatus for automatic updating and testing of software |
| CN101178687A (en) * | 2007-11-26 | 2008-05-14 | 中兴通讯股份有限公司 | Software regression testing method |
| CN101329652A (en) * | 2008-07-30 | 2008-12-24 | 中兴通讯股份有限公司 | Regression test automatic system and method |
| CN102095448A (en) * | 2010-12-08 | 2011-06-15 | 江苏天泽信息产业股份有限公司 | Automatic testing system of production line of vehicle information terminal and testing method of system |
Non-Patent Citations (1)
| Title |
|---|
| 郭惠玲等: "《中国科技论文在线www.paper.edu.cn/html/releasepaper/2010/12/1057/》", 24 December 2010 * |
Cited By (38)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105487966A (en) * | 2014-09-17 | 2016-04-13 | 腾讯科技（深圳）有限公司 | Program testing method, device and system |
| CN104679652A (en) * | 2015-02-15 | 2015-06-03 | 北京宇航系统工程研究所 | Markov method-based simulation system test case generating method |
| CN104679652B (en) * | 2015-02-15 | 2017-12-22 | 北京宇航系统工程研究所 | A kind of analogue system method for generating test case based on Markov methods |
| CN104794053B (en) * | 2015-04-22 | 2017-12-01 | 烽火通信科技股份有限公司 | A kind of logical simulation checking regression test system and method based on random use-case |
| CN104794053A (en) * | 2015-04-22 | 2015-07-22 | 烽火通信科技股份有限公司 | Logic simulation verification regression testing system and method based on random use cases |
| CN106291312A (en) * | 2015-06-12 | 2017-01-04 | 超威半导体(上海)有限公司 | A kind of method emulating for speed-up chip and debugging and chip test system |
| CN105158681A (en) * | 2015-08-07 | 2015-12-16 | 广州中大微电子有限公司 | Radio frequency identification reader chip verification method and system |
| CN105389200A (en) * | 2015-12-16 | 2016-03-09 | 浪潮(北京)电子信息产业有限公司 | Multi-group stimulation batch simulation method and system applied to super-large scale chip verification |
| CN105677996A (en) * | 2016-01-13 | 2016-06-15 | 浪潮集团有限公司 | Method for determining regression test version abnormity source in chip verification |
| CN105677996B (en) * | 2016-01-13 | 2018-10-23 | 浪潮集团有限公司 | A kind of method that regression test version exception source is determined in chip checking |
| CN105975924A (en) * | 2016-04-29 | 2016-09-28 | 杭州当虹科技有限公司 | Regression testing method for precisely recognizing ad content based on video frame statistics |
| CN106897485A (en) * | 2017-01-09 | 2017-06-27 | 北京中电华大电子设计有限责任公司 | Intelligence returns verification management system |
| CN107885925A (en) * | 2017-11-03 | 2018-04-06 | 中国电子科技集团公司第五十四研究所 | A kind of register testing method of EDA Qualify Phases |
| CN109240872A (en) * | 2018-08-08 | 2019-01-18 | 福州瑞芯微电子股份有限公司 | A kind of chip checking key signal coverage rate statistical analysis technique and storage medium |
| CN109240872B (en) * | 2018-08-08 | 2021-12-10 | 瑞芯微电子股份有限公司 | Statistical analysis method for coverage rate of chip verification key signals and storage medium |
| CN110232017A (en) * | 2019-04-30 | 2019-09-13 | 北京云迹科技有限公司 | Robot regression testing method and system under simulated environment |
| CN110348217A (en) * | 2019-05-28 | 2019-10-18 | 深圳壹账通智能科技有限公司 | Interface test method, device, electronic equipment and storage medium |
| CN111428431A (en) * | 2020-02-28 | 2020-07-17 | 中科亿海微电子科技（苏州）有限公司 | Method and system for supporting automatic test and recording of EDA (electronic design automation) software |
| CN111428431B (en) * | 2020-02-28 | 2024-02-02 | 中科亿海微电子科技（苏州）有限公司 | Automatic test and recording method and system supporting EDA software |
| CN111427794A (en) * | 2020-04-03 | 2020-07-17 | 天津飞腾信息技术有限公司 | Method, system and medium for accelerating simulation of storage component netlist |
| CN111427794B (en) * | 2020-04-03 | 2023-05-16 | 飞腾信息技术有限公司 | Method, system and medium for accelerating simulation of memory component netlist |
| CN111859830A (en) * | 2020-06-30 | 2020-10-30 | 山东云海国创云计算装备产业创新中心有限公司 | Verification plan and report generation method, device, equipment and storage medium |
| CN112131827B (en) * | 2020-09-11 | 2023-03-28 | 山东云海国创云计算装备产业创新中心有限公司 | Chip testing method, system, equipment and storage medium |
| CN112131827A (en) * | 2020-09-11 | 2020-12-25 | 山东云海国创云计算装备产业创新中心有限公司 | Chip testing method, system, equipment and storage medium |
| CN112015665A (en) * | 2020-09-30 | 2020-12-01 | 北京百度网讯科技有限公司 | Test verification backtracking method and device, electronic equipment and storage medium |
| CN112015665B (en) * | 2020-09-30 | 2023-04-11 | 北京百度网讯科技有限公司 | Test verification backtracking method and device, electronic equipment and storage medium |
| CN112380127A (en) * | 2020-11-16 | 2021-02-19 | 海光信息技术股份有限公司 | Test case regression method, device, equipment and storage medium |
| CN112380127B (en) * | 2020-11-16 | 2022-11-22 | 海光信息技术股份有限公司 | Test case regression method, device, equipment and storage medium |
| CN112131512A (en) * | 2020-11-20 | 2020-12-25 | 中国人民解放军国防科技大学 | Method and system for website management script safety certification |
| CN112131512B (en) * | 2020-11-20 | 2021-02-09 | 中国人民解放军国防科技大学 | Method and system for website management script safety certification |
| CN112464596A (en) * | 2020-11-27 | 2021-03-09 | 海光信息技术股份有限公司 | Regression testing method, system, equipment and readable storage medium |
| CN112613255B (en) * | 2020-12-10 | 2023-01-24 | 海光信息技术股份有限公司 | Multi-environment collaborative regression method, device, equipment and storage medium |
| CN112613255A (en) * | 2020-12-10 | 2021-04-06 | 海光信息技术股份有限公司 | Multi-environment collaborative regression method, device, equipment and storage medium |
| CN116306388A (en) * | 2023-05-23 | 2023-06-23 | 苇创微电子(上海)有限公司 | Automatic UVM verification platform free of path connection and construction method thereof |
| CN116776793A (en) * | 2023-08-22 | 2023-09-19 | 成都翌创微电子有限公司 | Multi-period path constraint verification method combining static time sequence analysis and pre-simulation |
| CN116776793B (en) * | 2023-08-22 | 2023-11-03 | 成都翌创微电子有限公司 | Multi-period path constraint verification method combining static time sequence analysis and pre-simulation |
| CN117034821A (en) * | 2023-10-09 | 2023-11-10 | 芯耀辉科技有限公司 | Regression verification method and medium for chip design front-end simulation verification |
| CN117034821B (en) * | 2023-10-09 | 2023-12-19 | 芯耀辉科技有限公司 | Regression verification method and medium for chip design front-end simulation verification |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103823747B (en) | 2017-09-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103823747A (en) | Automatic regression testing method | |
| CN110309071B (en) | Test code generation method and module, and test method and system | |
| CN101241467B (en) | Automatized white box test system and method facing to WEB application | |
| US7320090B2 (en) | Methods, systems, and media for generating a regression suite database | |
| CN102156784B (en) | Verifying environment patterned chip verifying method and device | |
| Mariani et al. | Compatibility and regression testing of COTS-component-based software | |
| CN105095059B (en) | A kind of method and apparatus of automatic test | |
| CN102122265B (en) | System and method for verifying computer software test results | |
| CN101316185B (en) | Method for positioning system resource bottleneck by analysis result based on log file | |
| CN104407971A (en) | Method for automatically testing embedded software | |
| CN106909510A (en) | A kind of method and server for obtaining test case | |
| CN102521700A (en) | Electrical network informatization evaluation rapid test system | |
| US20070061641A1 (en) | Apparatus and method for generating test driver | |
| CN104794057A (en) | Automatic testing method and device for crossed event | |
| CN102541723A (en) | Automated testing device for software testing | |
| CN102508779A (en) | Automatic performance test script generating system based on web crawler logs and automatic performance test script generating method based on same | |
| CN111240968A (en) | Automatic test management method and system | |
| CN109491922A (en) | One kind being based on model-driven test method and apparatus | |
| CN112540924A (en) | Interface automation test method, device, equipment and storage medium | |
| CN101765129A (en) | Network access test system and method for data value-added service | |
| CN104123397A (en) | Automatic test device and method for Web page | |
| CN101894073A (en) | Defect automatic positioning device based on control flow intersection and automatic positioning method thereof | |
| CN114239459B (en) | Processing method, device, equipment and medium for FPGA prototype design file | |
| Luo et al. | Clustering and tailoring user session data for testing web applications | |
| CN111782557B (en) | Method and system for testing web application permission |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20170915 Termination date: 20191116 |