CN104022913B - For method of testing and the device of data cluster - Google Patents
For method of testing and the device of data cluster Download PDFInfo
- Publication number
- CN104022913B CN104022913B CN201310697802.8A CN201310697802A CN104022913B CN 104022913 B CN104022913 B CN 104022913B CN 201310697802 A CN201310697802 A CN 201310697802A CN 104022913 B CN104022913 B CN 104022913B
- Authority
- CN
- China
- Prior art keywords
- test
- service order
- service
- existing network
- result data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Landscapes
- Debugging And Monitoring (AREA)
Abstract
The invention discloses a kind of method of testing for data cluster and device, belong to data cluster field.Described method comprises: obtain the test authority of isolating with the authority of data cluster process business; The computational resource used according to data cluster process business, obtains measuring and calculation resource; Obtain the existing network operations flows to business and existing network result data; Reduction treatment is carried out to existing network operations flows, obtains test service stream; Based on measuring and calculation resource and test authority, the operation that each service order in execution test service stream is corresponding, obtains test result data; Existing network result data and test result data are carried out consistency contrast, exports comparing result.The present invention is tested by the production environment of direct usage data cluster, without the need to additionally building test scene, reduces human input and cost, full dose covering is carried out to the production environment of data cluster, enhances coverage, further, improve test mass, reduce false pass rate.
Description
Technical field
The present invention relates to data cluster field, particularly a kind of method of testing for data cluster and device.
Background technology
Large data sets group refers to that thousands of computer connects computer cluster that is formed, that jointly can complete the tasks such as data storage, data calculating by network.
The data volume that large data sets group carries is huge, and business complexity is also very high, and therefore, in order to normal and stable operation, large data sets group must have high stability.And in order to ensure the high stability of large data sets group, need to test the production environment of large data sets group.
But in relevant method of testing, be all generally construct the test scene similar with the production environment of large data sets group, test under this test scene, the cost that structure test scene expends is very high; Because the production environment of the business diversity of constructed test scene, the parameter such as business complexity and data volume and large data sets group has very large gap, easily cause test result inaccurate, be not easy to the defect existing for production environment finding large data sets group, false pass rate is high.
Summary of the invention
In order to solve the problem of prior art, embodiments provide a kind of method of testing for data cluster.Described technical scheme is as follows:
First aspect, provides a kind of method of testing for data cluster, and described method comprises:
Obtain test authority, the authority of described test authority and data cluster process business is isolated;
The computational resource used according to data cluster process business, obtains measuring and calculation resource;
Obtain described data cluster to the existing network operations flows of described business and existing network result data, described existing network result data comprises the existing network actual parameter of the operating result data of existing network service order and the indices of process business procedure;
Carry out reduction treatment to described existing network operations flows, obtain test service stream, described test service stream comprises service order received in the described business procedure of process;
Based on described measuring and calculation resource and described test authority, perform the operation that each service order in described test service stream is corresponding, obtain the test result data of described test service stream, described test result data comprises the test result parameter of operating result data corresponding to each service order in test service stream and described indices;
Described existing network result data and described test result data are carried out consistency contrast, exports comparing result.
Second aspect, provides a kind of testing apparatus for data cluster, and described device comprises:
Test authority acquiring module, for obtaining test authority, the authority of described test authority and data cluster process business is isolated;
Source obtaining module, for the computational resource used according to data cluster process business, obtains measuring and calculation resource;
Existing network business acquisition module, for obtaining described data cluster to the existing network operations flows of described business and existing network result data, described existing network result data comprises the existing network actual parameter of the operating result data of existing network service order and the indices of process business procedure;
Recovery module, for carrying out reduction treatment to described existing network operations flows, obtains test service stream, and described test service stream comprises service order received in the described business procedure of process;
Test result data acquisition module, for based on described measuring and calculation resource and described test authority, perform the operation that each service order in described test service stream is corresponding, obtain the test result data of described test service stream, described test result data comprises the test result parameter of operating result data corresponding to each service order in test service stream and described indices;
Contrast module, for described existing network result data and described test result data are carried out consistency contrast, exports comparing result.
The beneficial effect that the technical scheme that the embodiment of the present invention provides is brought is:
The method and apparatus that the embodiment of the present invention provides, obtain processing the service order received in business procedure by carrying out reduction treatment to existing network operations flows in data cluster, again perform the operation that each service order is corresponding, obtain the test result data of indices, by the real data of indices and test result data are carried out consistency contrast, export comparing result.The production environment of the direct usage data cluster of this method of testing is tested, and without the need to additionally building test scene, significantly reduce the human input in test process and cost, and full dose covering is carried out to the production environment of data cluster, enhance coverage, further, improve test mass, reduce false pass rate.
Accompanying drawing explanation
In order to be illustrated more clearly in the technical scheme in the embodiment of the present invention, below the accompanying drawing used required in describing embodiment is briefly described, apparently, accompanying drawing in the following describes is only some embodiments of the present invention, for those of ordinary skill in the art, under the prerequisite not paying creative work, other accompanying drawing can also be obtained according to these accompanying drawings.
Fig. 1 is the flow chart of a kind of method of testing for data cluster that the embodiment of the present invention provides;
Fig. 2 a is the flow chart of a kind of method of testing for data cluster that the embodiment of the present invention provides;
Fig. 2 b is the authority schematic diagram that the embodiment of the present invention provides;
Fig. 2 c is the schematic diagram of the acquisition test service stream that the embodiment of the present invention provides;
Fig. 2 d is the adjustment service order schematic diagram that the embodiment of the present invention provides;
Fig. 2 e is the output comparing result schematic diagram that the embodiment of the present invention provides;
Fig. 2 f is the high-level schematic functional block diagram that the embodiment of the present invention provides;
Fig. 2 g is the data cluster frame diagram that the embodiment of the present invention provides;
Fig. 2 h is the defect number design sketch that the embodiment of the present invention provides;
Fig. 2 i is the false pass rate design sketch that the embodiment of the present invention provides;
Fig. 3 is a kind of testing apparatus structural representation for data cluster that the embodiment of the present invention provides.
Embodiment
Below in conjunction with the accompanying drawing in the embodiment of the present invention, be clearly and completely described the technical scheme in the embodiment of the present invention, obviously, described embodiment is the present invention's part embodiment, instead of whole embodiments.Based on the embodiment in the present invention, those of ordinary skill in the art, not making the every other embodiment obtained under creative work prerequisite, belong to the scope of protection of the invention.
For convenience of explanation, will be introduced data cluster.For this data cluster for Hadoop cluster, Hadoop cluster comprises HDFS(Hadoop Distributed File System, distributed file system) and map stipulations Map Reduce system, HDFS system is for storing the file in Hadoop cluster, provide the data of high transmission rates access application, be applicable to the application program with super large data set, Map Reduce system is used for realizing Distributed Calculation.HDFS and Map Reduce system can run on large commercial machine cluster.
Fig. 1 is the flow chart of a kind of method of testing for data cluster that the embodiment of the present invention provides, and see Fig. 1, described method comprises:
100, obtain test authority, the authority of this test authority and data cluster process business is isolated;
101, computational resource used according to data cluster process business, obtains measuring and calculation resource;
102, obtain this data cluster to the existing network operations flows of this business and existing network result data, this existing network result data comprises the existing network actual parameter of the operating result data of existing network service order and the indices of process business procedure;
103, carry out reduction treatment to this existing network operations flows, obtain test service stream, this test service stream comprises service order received in this business procedure of process;
104, based on this measuring and calculation resource and this test authority, perform the operation that each service order in this test service stream is corresponding, obtain the test result data of this test service stream, this test result data comprises the test result parameter of operating result data corresponding to each service order in test service stream and this indices;
105, this existing network result data and this test result data are carried out consistency contrast, export comparing result.
The method that the embodiment of the present invention provides, obtain processing the service order received in business procedure by carrying out reduction treatment to existing network operations flows in data cluster, again perform the operation that each service order is corresponding, obtain the test result data of indices, by the real data of indices and test result data are carried out consistency contrast, export comparing result.The production environment of the direct usage data cluster of this method of testing is tested, and without the need to additionally building test scene, significantly reduce the human input in test process and cost, and full dose covering is carried out to the production environment of data cluster, enhance coverage, further, improve test mass, reduce false pass rate.
Alternatively, based on this measuring and calculation resource and this test resource, perform the operation that each service order in this test service stream is corresponding, before obtaining the test result data of this indices, the method also comprises:
Regulate the service order in this test service stream, perform test process based on the service order after regulating.
Alternatively, regulate the service order in this test service stream, comprise based on the service order execution test process after regulating:
Regulate the concurrency of the service order in this test service stream, this concurrency comprises at least one item in instruction concurrency and mapping/stipulations MR concurrency;
According to the concurrency after adjustment, issuing service instruction, performs test process based on the service order issued.
Alternatively, according to the concurrency after adjustment, issuing service instruction comprises:
Determine the priority of each service order in this existing network operations flows in this test service stream;
According to priority order from high to low, obtain the service order in this test service stream successively, until the service order number got arrives the instruction concurrency after regulating, issue the service order got;
And/or, according to priority order from high to low, obtain the service order in this test service stream successively, until the number of the mapping Map task that generates in this existing network operations flows of the service order got and stipulations Reduce task and value arrive the MR concurrency after adjustment, issue the service order got.
Alternatively, determine that in this test service stream, the priority of each service order in this existing network operations flows comprises:
The operation duration shared in this existing network operations flows according to each service order in this test service stream, determines the priority of each service order in this existing network operations flows; Or,
The mapping Map task generated in this existing network operations flows according to each service order in this test service stream and the number of stipulations Reduce task, determine the priority of each service order in this existing network operations flows.
Alternatively, the method also comprises:
Operating result data corresponding for each service order in this test service stream be stored in the default storehouse table of this data cluster, the different sink table for same data cluster is shown in this default storehouse table and the storehouse of the operating result data storing this existing network service order; Or,
Under operating result data corresponding for each service order in this test service stream are stored in the predetermined directory of this data cluster, this predetermined directory and the operating result data storing existing network service order are the different directories of same data cluster.
Alternatively, the method also comprises:
The test result data of indices in the test result data of this indices and the data cluster of old version is carried out consistency contrast, exports the comparing result between old version.
Alternatively, this test authority comprises the authority of data of read-only current process business, the authority of the real data of read-only storage and the authority of read/write test result data.
Above-mentioned all alternatives, can adopt and combine arbitrarily formation optional embodiment of the present invention, this is no longer going to repeat them.
Fig. 2 a is the flow chart of a kind of method of testing for data cluster that the embodiment of the present invention provides, and see Fig. 2 a, described method comprises:
200, obtain test authority, the authority of this test authority and data cluster process business is isolated.
In embodiments of the present invention, before this measuring and calculation resource of acquisition, obtain test authority.Tester can log in test account, obtains the test authority that this test account is corresponding, thus obtains measuring and calculation resource, based on this test authority and this measuring and calculation resource, tests this data cluster.
Wherein, the project one_to_one corresponding that the authority of the project that this test authority comprises and this process business comprises and mutually isolated, to avoid damaging existing network resource.Test authority can be identical with the priority assignation of each project in the authority of process business, also can be different.
As, see Fig. 2 b, this data cluster comprises multiple user pond, user pond comprises multiple user's group, user's group comprises multiple user, the computational resource that each user pond is corresponding certain, and this user corresponding to user pond can based on the computational resource process business in user pond.The authority list of this process business understands the authority of each user when the business of process.According to multiple user ponds that this data cluster comprises, multiple test subscriber pond is set, and obtain this test authority, this test authority shows the operation that each test subscriber can perform when testing, and test subscriber can carry out based on the measuring and calculation resource corresponding to test subscriber pond when testing.
Wherein, the authority of this process business comprises the authority of the data of the current process business of read/write, and the authority in the real data write database of real data current process business obtained or catalogue.Accordingly, this test authority comprises the authority of data of read-only current process business, the authority of the real data of read-only storage and the authority of read/write test result data, causes damage to prevent the real data of test process to this data cluster.
201, computational resource used according to data cluster process business, obtains measuring and calculation resource.
The embodiment of the present invention is applied in the production environment of data cluster, under the scene of test this data cluster.This data cluster needs consumption calculations resource in the process of process business, accordingly, is carrying out, in the process of testing, needing consumption calculations resource equally, then first obtain measuring and calculation resource, according to this measuring and calculation resource, to test this data cluster.
Wherein, this computational resource refers to that as MapReduce resource, the embodiment of the present invention is not construed as limiting this for providing the resource of traffic handing capacity in data cluster.
Particularly, obtain the computational resource one to one computational resource used with this data cluster process business, as this measuring and calculation resource, this measuring and calculation resource computational resource size used with this data cluster process business is identical, to process business procedure to carry out based on identical computational resource with test process, further, be convenient to test the stability of this data cluster.Alternatively, this measuring and calculation resource and this data cluster process business computational resource used are isolated, and avoid the resource contention between process business procedure and test process.
202, obtain this data cluster to the existing network operations flows of this business and existing network result data, this existing network result data comprises the existing network actual parameter of the operating result data of existing network service order and the indices of process business procedure.
Wherein, this existing network operations flows have recorded the service order that this data cluster receives in the process of process business.In order to obtain the practical operation situation of data cluster, in process business procedure, gather this existing network result data, this existing network result data comprises the existing network actual parameter of the operating result data of existing network service order and the indices of process business procedure, and preserves this existing network operations flows and this existing network result data.Further, from the platforms such as ganglia monitor supervision platform, system process, network management platform, dynamically gather the existing network actual parameter of the operating result data of existing network service order and the indices of process business procedure, and the operating result data collected (or statistics data of operating result data) and existing network actual parameter are kept in catalogue corresponding to this indices or storehouse table.
Wherein, this indices comprises business key index, system core index and result data index etc., and can preset or set according to testing requirement, the embodiment of the present invention does not limit this.
Alternatively, in this data cluster, reserved one allows the interface obtaining existing network operations flows, can obtain the existing network operations flows of this data cluster to this business by this interface.In addition, this interface can comprise the group system operation informations such as self-defined water operation instruction stream, database flowing water, log recording, and the embodiment of the present invention does not limit this.
In actual application, the data volume of the operating result data of real existing network service order is very large, be not easy to directly preserve, now can adopt the mode of statistics, each field of these operating result data is counted respectively, calculates and is worth, the statistic processes such as calculating mean value, obtain the statistics data of this existing network service order, using the operating result data of these statistics data as existing network service order.In subsequent process, the statistical identical with the operating result data of this existing network service order can be adopted, the operating result data of this test service instruction are added up, obtain the statistics data of this test service instruction, the statistics data of this test service instruction are as the operating result data of test service instruction, and when consistency contrasts, can by carrying out the operating result data of existing network service order and the operating result data of test service instruction, that is to say, when the statistics data of test service instruction and the statistics data consistent of existing network service order, then think the operating result data of existing network service order and the operating result data consistent of test service instruction.
203, carry out reduction treatment to this existing network operations flows, obtain test service stream, this test service stream comprises service order received in this business procedure of process.
Particularly, this existing network operations flows have recorded multiple service orders that this data cluster receives in the process of process business, the form of each service order may be different, then this existing network operations flows is filtered and form collator, with the form of the service order in this existing network operations flows unified, analyze the performance objective of each service order in this existing network operations flows, this performance objective is replaced with test target, obtain test operation stream, scenario reduction is carried out to this test operation stream, obtain this test service stream, this test service stream comprises the service order that receives in process business procedure and test target corresponding to each service order, so that follow-up, operation corresponding to each service order is performed to each test target.
In embodiments of the present invention, this existing network operations flows to be filtered and form collator includes but not limited to that the spcial character this existing network operations flows comprised replaces with space or deletion.Wherein, this additional character is newline, carriage return character, multiple space and annotation etc.
Alternatively, according to the corresponding relation of the performance objective preset and test target, the performance objective of service order is replaced with test target.The performance objective of process business procedure and test target one_to_one corresponding, the service order received in business procedure as processed is read the data in the directory A of this data cluster, this performance objective is the data in directory A, then corresponding test target is with the test result data of real data logic isolation, the data in the catalogue a corresponding with directory A.
Further alternatively, according to the corresponding relation between the test operation stream preset and test service scene, be the test service scene that this test operation stream is corresponding by the business scenario reduction of this data cluster, to test under the test service scene after reduction.
Such as, see Fig. 2 c, this Fig. 2 c is depicted as filtration to service order and form collator process.This existing network operations flows is kept in log recording or data-base recording, and a service order is wherein:
“Hadoop jar\n
Xxx.jar\t…-
Dmapred output.dir-
OnlineTargetDirl ... #-annotation-
--user " time,
By the deletion such as the additional character " n " in this service order, " t ", space, annotation, obtain:
“Hadoop jar
Xxx.jar-
Dmapred output.dir-
OnlineTargetDirl-
--user”
Analyzing the performance objective obtaining service order is " OnlineTargetDirl ", according to this performance objective and test target mapping table, performance objective is replaced with test target " TestTargetDir1 ", and user is replaced with test subscriber " TestUser ", obtain the service order in test operation stream:
“Hadoop jar
Xxx.jar-
Dmapred output.dir-
TestTargetDir1-
--TestUser”
By in the test operation stream write into Databasce that obtains, scenario reduction is carried out to this test operation stream, obtains test service stream, by this test service stream write into Databasce.
Wherein, the mapping table of performance objective and test target is as shown in table 1, comprise five fields: the database " test_db " of the database " online_db " of traffic ID (sequence number) " query_id ", real data, performance objective " online_t ", test result data and test target " test_table ", the test target that performance objective " Onlineta " is corresponding is " T table1 ".The test operation stream record sheet that target obtains after replacing is as shown in table 2, comprises two fields: traffic ID " query_id " and test operation stream " OperateFlow ".Mapping table between test operation stream and test service scene is as shown in table 3, comprises three fields: test service scene ID " scene_id ", reduction sequence " step_id " and traffic ID " query_id ".
Table 1
| query_id | online_db | online_t | test_db | test_table |
| 01039131000100019853 | Onlinedb | Onlineta | Testdb | T table1 |
Table 2
| query_id | OperateFlow |
| … | … |
Table 3
| scene_id | step_id | query_id |
| … | … | … |
204, regulate the concurrency of the service order in this test service stream, this concurrency comprises instruction concurrency and MR(Map/Reduce, mapping/stipulations) at least one item in concurrency.
Wherein, this instruction concurrency refers on a time point, this data cluster can treatable service order number, this MR concurrency refers on a time point, this data cluster can the number of treatable Map task and the number of Reduce task, or the number of Map task and Reduce task and value.The embodiment of the present invention is described with value for the number of this MR concurrency for Map task and Reduce task.
In embodiments of the present invention, this test service stream comprises the service order that receives of process business procedure, by issuing the service order in this test service stream, and performs operation corresponding to the service order that issues, to process business process simulation, obtain the test result data of indices.Wherein, the concurrency of the service order in this test service stream is different, then the pressure that this data cluster is subject in test process is just different, by regulating the pressure of this concurrency and this data cluster adjustable.In the embodiment of the present invention, by regulating the concurrency of service order, simulate the pressure in various degree suffered by this data cluster, to test the stability of this data cluster.
Alternatively, regulate the concurrency of the service order in this test service stream from small to large, so that according to the concurrency after adjustment, issuing service instruction, until the force value of this data cluster is the preset multiple of original force value.This preset multiple can be determined according to testing requirement, and the embodiment of the present invention does not limit this.
By regulating this test service stream, regulate the pressure of this data cluster, directly apply pressure to this data cluster, in conjunction with the pressure of this data cluster reality, form the controlled pressure being greater than this data cluster actual pressure, to verify the stability performance at various pressures of this data cluster.
205, according to the concurrency after adjustment, issuing service instruction.
In embodiments of the present invention, according to the concurrency after adjustment, determine the service order at every turn issued, and issue this service order determined, the operation that the service order issued with execution is corresponding, after this executable operations completes, determines the service order next time issued, to continue issuing service instruction, until all service orders in this test service stream have been issued.
Alternatively, this step 205 comprises step (1) and step (2):
(1) priority of each service order in this existing network operations flows in this test service stream is determined.
In embodiments of the present invention, in order to test the stability of this data cluster, when simulating the service order received in this test service stream, the execution sequence of service order, concurrency etc. can be regulated, obtain the test result data under different scene, then first can determine the priority of each service order in this existing network operations flows, determine to wait to simulate the service order performed, issuing service instruction according to this priority.
Wherein, this step (1) can comprise following any one:
(1-1) shared in this existing network operations flows according to each service order in this test service stream operation duration, determines the priority of each service order in this existing network operations flows.
In embodiments of the present invention, operation duration that can be shared in this existing network operations flows according to each service order in this test service stream, determine the priority of each service order, specifically can be defined as the service order that shared operation duration is longer, priority is higher, also can be defined as the service order that shared operation duration is shorter, priority is higher.
This determines that the mode of priority can be determined according to testing requirement, when hope is tested the Pressure maximum value that this data cluster can bear, the service order that shared operation duration is longer can be determined, priority is higher, so that during according to the priority issuing service instruction determined, larger pressure is produced to this data cluster.When the Business Processing speed of hope to this data cluster is tested, the service order that shared operation duration is shorter can be determined, priority is higher, so that during according to the priority issuing service instruction determined, less pressure is produced, to test the Business Processing speed of this data cluster to this data cluster.
(1-2) the Map task generated in this existing network operations flows according to each service order in this test service stream and the number of Reduce task, determine the priority of each service order in this existing network operations flows.
In embodiments of the present invention, each service order can generate multiple Map task and multiple Reduce task, the Map task that different service orders generates is different with the number of Reduce task, the pressure produced this data cluster is also different, then according to the number of Map task and Reduce task, the priority of service order can be determined.Specifically can be defined as the larger service order of MR number, priority is higher, and also can be defined as the less service order of MR number, priority is higher, and wherein MR number is number and the value of Map task and Reduce task.
With the operation duration determination priority class shared by each service order seemingly, the mode according to MR number determination priority can be determined according to testing requirement, does not repeat them here.
(2) according to priority order from high to low, obtain the service order in this test service stream successively, until the service order number got arrives the instruction concurrency after regulating, issue the service order got.
In embodiments of the present invention, regulate the concurrency of the service order in this test service stream, after each adjustment, according to priority order from high to low, obtain the service order in this test service stream successively, when the instruction concurrency after the service order number got and this regulate is equal, stops obtaining service order, issue the service order got.
As, this test service stream comprises 10,000 service orders, and this instruction concurrency is 500, then according to priority order from high to low, obtain 500 service orders and issue.
In another embodiment that the embodiment of the present invention provides, this step (2) can be replaced by step (3) or step (4):
(3) according to priority order from high to low, obtain the service order in this test service stream successively, until the number of the mapping Map task that generates in this existing network operations flows of the service order got and stipulations Reduce task and value arrive the MR concurrency after adjustment, issue the service order got.
In embodiments of the present invention, regulate the concurrency of the service order in this test service stream, after each adjustment, according to priority order from high to low, obtain the service order in this test service stream successively, when getting service order at every turn, calculate the MR number of this service order, and add up the MR number of service order got and value, MR concurrency after the regulating with value and this of MR number of the service order got is equal, or the MR number of the service order got with value be greater than this regulate after MR concurrency time, stop obtaining service order, and issue the service order got.
(4) according to priority order from high to low, obtain the service order in this test service stream successively, until the service order number got arrives the instruction concurrency after regulating, whether what judge the MR number of the service order got arrives the MR concurrency after adjustment with value, if, issue the service order got, if not, continue the service order obtained according to priority order from high to low in this test service stream, until the number of the Map task that generates in this existing network operations flows of the service order got and Reduce task and value arrive the MR concurrency after adjustment, issue the service order got.
Based on Fig. 2 d, the operation duration shared in this existing network operations flows according to each service order in this test service stream, or the Map task that in this test service stream, each service order generates in this existing network operations flows and the number of Reduce task and value, determine the priority of each service order, according to priority order from high to low, obtain the service order in this test service stream successively, judge the number of service order got whether arrive adjustment after instruction concurrency, if, whether that continues the MR number judging the service order got arrives the MR concurrency after adjustment with value, when this MR number with value arrive regulate after MR concurrency time, stop obtaining service order, issue the service order got.
As, this test service stream comprises 10,000 service orders, and this instruction concurrency is 500, this MR concurrency is 20,000, then according to priority order from high to low, obtain 500 service orders, judge that whether the summation of the MR number of 500 service orders obtained is more than 20,000, if, stop obtaining service order, issue the service order got, if not, continue acquisition service order, until the summation of the MR number of the service order got is more than 20,000.
It should be noted that, when this MR concurrency is the number of the number of Map task and Reduce task, namely this MR concurrency comprises Map concurrency and Reduce concurrency, then judge the number of the Map task of the service order got and value whether arrive adjustment after Map concurrency, and judge the number of the Reduce task of service order got and value whether arrive adjustment after Reduce concurrency, when whether the number of the Map task of the service order got arrives the Map concurrency after adjustment with value, and whether the number of Reduce task and value be when arriving the Reduce concurrency after adjustment, stop obtaining service order, and issue the service order got.
206, based on this measuring and calculation resource and this test authority, the operation that each service order that execution issues is corresponding, obtain the test result data of this test service stream, this test result data comprises the test result parameter of operating result data corresponding to each service order in test service stream and this indices.
Namely the embodiment of the present invention is issue the service order in this test service stream, based on this measuring and calculation resource, the operation that each service order that execution issues is corresponding, with the process simulation to this data cluster process business, obtain the test result parameter of operating result data corresponding to each service order in test service stream and this indices.Because this test process carries out based on this test authority, therefore when the operation that each service order of execution is corresponding, only perform the operation corresponding to test authority, avoid damaging the data in data cluster.
In embodiments of the present invention, the test result parameter of in collecting test Business Stream, each service order is corresponding in test process operating result data and this indices is needed.Particularly, the dynamically test result parameter of acquisition operations result data and indices from the platforms such as ganglia monitor supervision platform, system process, network management platform, and the test result parameter of these operating result data and indices is kept in corresponding catalogue or storehouse table.
Alternatively, operating result data corresponding for each service order in this test service stream be stored in the default storehouse table of this data cluster, the different sink table for same data cluster is shown in this default storehouse table and the storehouse of the operating result data storing this existing network service order; Or under operating result data corresponding for each service order in this test service stream are stored in the predetermined directory of this data cluster, this predetermined directory and the operating result data storing existing network service order are the different directories of same data cluster.
In embodiments of the present invention, in each catalogue that the existing network result data of this data cluster is distributed in whole cluster or each storehouse table, and for the ease of managing test result data, store operating result data that in this test service stream, each service order is corresponding and the catalogue of operating result data and the Ku Biao logic isolation that store this existing network service order, ensure that the fail safe of the operating result data of the existing network service order of data cluster, make to carry out when consistency contrasts convenient to existing network result data and test result data, be convenient to the follow-up cleaning carrying out production environment.
207, this existing network result data and this test result data are carried out consistency contrast, export comparing result.
Particularly, this existing network result data and the output record number of this test result data, the computational resource of consumption are carried out consistency contrast, export comparing result.In addition, just this existing network result data and this test result data carry out consistency contrast, can sample to by this existing network result data and this test result data, and the existing network sample data extracted and test sample book data are contrasted, or contrast again after adding up.As for the existing network sample data of value type and test sample book data, respectively calculating mean value, contrasts the mean value obtained, or for the existing network sample data of character type and test sample book data, calculate non-repetitive character number respectively, the character number obtained is contrasted.
In addition, when having carried out renewal to this data cluster, need to test the data cluster of redaction, then when obtaining this test result data, the test result data of the test result data of current version and the data cluster of old version is carried out consistency contrast, can better test the stability of the functional module that this upgrades.Accordingly, the method also comprises: the test result data in the data cluster of this test result data and old version is carried out consistency contrast, exports the comparing result between old version.
Further, can set the threshold value of indices, when the existing network actual parameter of a certain item index and the difference of test result parameter are beyond the threshold value set, consistency contrast is not passed through.
Wherein, for quantifiable index, directly the test result data of test result data and old version can be carried out consistency contrast, as full GC number, average RPC(Remote Procedure Call, remote call program) duration etc., and for not quantifiable index, can carry out curve fitting to the test result data of the test result data of current version and old version, to the two kinds of curves obtained carry out consistency contrast a little, relatively on two kinds of curves side-play amount a little, when this side-play amount has exceeded predetermined threshold value, consistency contrast is not passed through.
In the embodiment of the present invention, when exporting comparing result, whole comparing results can be exported, also can when consistency contrast not be passed through, just output-consistence contrasts the comparing result of unsanctioned index, and the embodiment of the present invention does not limit this.Wherein, export the mode of comparing result can be index with color alarm, the tinkle of bells alarm or mail alarm etc., the embodiment of the present invention does not also limit this.
See Fig. 2 e, testing apparatus from ganglia monitor supervision platform, JVM(Java Virtual Machine, Java Virtual Machine), gather existing network result data in the platform such as system process, network management platform, and to preserve in a database.The record number of existing network result data and test result data, resource consumption are contrasted, line sampling of going forward side by side, contrasts the sample data of extraction, and adds up sample data, contrast statistics.Quantifiable indicator in the test result data of current version and the test result data of old version is contrasted, the curve degree of not quantifiable indicator and resource utilization are contrasted.When having contrasted, export the comparing result that difference exceedes the index of threshold value, Output rusults or mail alarm can be represented on monitoring foreground.
After this step 207, the method can also comprise: when test completes, and can clear up the test result data of this indices, and reclaims test authority.
See Fig. 2 f, the embodiment of the present invention can be cleared up these five functional modules of module by secure processing module, Service recovery module, test module, results analyses module and environment and be performed.Wherein, secure processing module is used for rights of using isolation and obtains the test authority of isolating with the authority of process business, and use result data isolation that this existing network result data and test result data are carried out logic isolation, ensure that the existing network result data of this data cluster can not be destroyed.Service recovery module is used for carrying out Operations Analyst to existing network operations flows, extracts useful operations flows information, and carries out suitably Safe Transformation and scenario reduction, obtain the test service stream that can be directly used in test.Test module is used for by regulating this test service stream, regulates the pressure of data cluster, under pressure after adjustment, performs the operation corresponding to service order issued, simulates the process of this data cluster process business, obtain this test result data.Results analyses module is used for this existing network result data and this test result data to carry out consistency contrast, export comparing result, and the test result data of the test result data of current version and old version is carried out consistency contrast, export the comparing result between old version.Environment cleaning module is used for when test completes, and clears up unwanted test result data, and reclaims test authority.The method that the embodiment of the present invention provides can based on the frame diagram shown in Fig. 2 g.
The method that the embodiment of the present invention provides is applicable to large data, large cluster and cannot assesses software or the system of large cluster real quality situation under the test environment, adopt technical scheme of the present invention, the restriction testing the factors such as diversity, business complexity, data volume, cluster resource in large data sets group system testing can be broken through, find the defect of large data sets group system up hill and dale, the capacity of assessment large data sets group, has filled up the blank of large data sets group method of testing.
The method that the embodiment of the present invention provides, obtain processing the service order received in business procedure by carrying out reduction treatment to existing network operations flows in data cluster, again perform the operation that each service order is corresponding, obtain the test result data of indices, by the real data of indices and test result data are carried out consistency contrast, export comparing result.The production environment of the direct usage data cluster of this method of testing is tested, and without the need to additionally building test scene, significantly reduce the human input in test process and cost, and full dose covering is carried out to the production environment of data cluster, enhance coverage, improve test mass, reduce false pass rate.Further, regulate the pressure of this data cluster, this data cluster stability is under a large pressure tested, be convenient to find this data cluster defect under a large pressure, reduce further false pass rate.
In order to the beneficial effect of the more particularly bright embodiment of the present invention, provide following test result.Fig. 2 h is the defect number design sketch that the embodiment of the present invention provides.Implemented by the method for testing provided the embodiment of the present invention, obtain comparing result, find the defect existing for this data cluster according to comparing result, as shown in fig. 2h.After finding the defect of this data cluster some through the traditional test methods such as static scanning, craft, automation, special project, even tune, the method that the application embodiment of the present invention provides, still can find a large amount of defects to greatly reduce false pass rate.
Fig. 2 i is the false pass rate design sketch that the embodiment of the present invention provides, and is implemented by the method for testing provided the embodiment of the present invention, finds defect and evaluation problem, and calculates false pass rate.Can find out, from year October in December, 2012 to 2013, false pass rate is reduced to 0.76% from 13.30%, is reduced to 2/35ths of former false pass rate, greatly reduces false pass rate.
Fig. 3 is a kind of testing apparatus structural representation for data cluster that the embodiment of the present invention provides, see Fig. 3, this device comprises: test authority acquiring module 300, source obtaining module 301, existing network business acquisition module 302, recovery module 303, test result data acquisition module 304, contrast module 305
Wherein, test authority acquiring module 300 is for obtaining test authority, and the authority of this test authority and data cluster process business is isolated; Source obtaining module 301 is connected with test authority acquiring module 300 and is used for the computational resource used according to data cluster process business, obtains measuring and calculation resource; Existing network business acquisition module 302 is connected with source obtaining module 301, for obtaining this data cluster to the existing network operations flows of this business and existing network result data, this existing network result data comprises the existing network actual parameter of the operating result data of existing network service order and the indices of process business procedure; Recovery module 303 is connected with existing network business acquisition module 302, for carrying out reduction treatment to this existing network operations flows, obtains test service stream, and this test service stream comprises service order received in this business procedure of process; Test result data acquisition module 304 is connected with recovery module 303, for based on this measuring and calculation resource and this test authority, perform the operation that each service order in this test service stream is corresponding, obtain the test result data of this test service stream, this test result data comprises the test result parameter of operating result data corresponding to each service order in test service stream and this indices; Contrast module 305 is connected with test result data acquisition module 304, for this existing network result data and this test result data are carried out consistency contrast, exports comparing result.
Alternatively, this device also comprises:
Adjustment module, for regulating the service order in this test service stream, performs test process based on the service order after regulating.
Alternatively, this adjustment module comprises:
Concurrency regulon, for regulating the concurrency of the service order in this test service stream, this concurrency comprises at least one item in instruction concurrency and mapping/stipulations MR concurrency;
Instruction issues unit, for according to regulate after concurrency, issuing service instruction;
Test execution unit, for performing test process based on the service order issued.
Alternatively, this instruction issues unit and comprises:
Priority determination subelement, for determining the priority of each service order in this existing network operations flows in this test service stream;
Instruction issues subelement, for according to priority order from high to low, obtains the service order in this test service stream successively, until the service order number got arrives the instruction concurrency after regulating, issues the service order got;
And/or, instruction issues subelement, also for according to priority order from high to low, obtain the service order in this test service stream successively, until the number of the mapping Map task that generates in this existing network operations flows of the service order got and stipulations Reduce task and value arrive the MR concurrency after adjustment, issue the service order got.
Alternatively, this priority determination subelement is used for the operation duration shared in this existing network operations flows according to each service order in this test service stream, determines the priority of each service order in this existing network operations flows; Or,
This priority determination subelement is used for the number of mapping Map task and the stipulations Reduce task generated in this existing network operations flows according to each service order in this test service stream, determines the priority of each service order in this existing network operations flows.
Alternatively, this device also comprises:
First memory module, for operating result data corresponding for each service order in this test service stream being stored in the default storehouse table of this data cluster, the different sink table for same data cluster is shown in this default storehouse table and the storehouse of the operating result data storing this existing network service order; Or,
Second memory module, under operating result data corresponding for each service order in this test service stream are stored in the predetermined directory of this data cluster, this predetermined directory and the operating result data storing existing network service order are the different directories of same data cluster.
Alternatively, this device also comprises:
Old version contrast module, for the test result data of indices in the test result data of this indices and the data cluster of old version is carried out consistency contrast, exports the comparing result between old version.
Alternatively, this test authority comprises the authority of data of read-only current process business, the authority of the real data of read-only storage and the authority of read/write test result data.
The device that the embodiment of the present invention provides, obtain processing the service order received in business procedure by carrying out reduction treatment to existing network operations flows in data cluster, again perform the operation that each service order is corresponding, obtain the test result data of indices, by the real data of indices and test result data are carried out consistency contrast, export comparing result.The production environment of the direct usage data cluster of this method of testing is tested, and without the need to additionally building test scene, significantly reduce the human input in test process and cost, and full dose covering is carried out to the production environment of data cluster, enhance coverage, further, improve test mass, reduce false pass rate.
It should be noted that: the testing apparatus for data cluster that above-described embodiment provides is when the test for data cluster, only be illustrated with the division of above-mentioned each functional module, in practical application, can distribute as required and by above-mentioned functions and be completed by different functional modules, internal structure by equipment is divided into different functional modules, to complete all or part of function described above.In addition, the testing apparatus for data cluster that above-described embodiment provides belongs to same design with the method for testing embodiment for data cluster, and its specific implementation process refers to embodiment of the method, repeats no more here.
One of ordinary skill in the art will appreciate that all or part of step realizing above-described embodiment can have been come by hardware, the hardware that also can carry out instruction relevant by program completes, described program can be stored in a kind of computer-readable recording medium, the above-mentioned storage medium mentioned can be read-only memory, disk or CD etc.
The foregoing is only preferred embodiment of the present invention, not in order to limit the present invention, within the spirit and principles in the present invention all, any amendment done, equivalent replacement, improvement etc., all should be included within protection scope of the present invention.
Claims (16)
1. for a method of testing for data cluster, it is characterized in that, described method comprises:
Obtain test authority, the authority of described test authority and data cluster process business is isolated;
The computational resource used according to data cluster process business, obtains measuring and calculation resource;
Obtain described data cluster to the existing network operations flows of described business and existing network result data, described existing network result data comprises the existing network actual parameter of the operating result data of existing network service order and the indices of process business procedure;
Carry out reduction treatment to described existing network operations flows, obtain test service stream, described test service stream comprises service order received in the described business procedure of process;
Based on described measuring and calculation resource and described test authority, perform the operation that each service order in described test service stream is corresponding, obtain the test result data of described test service stream, described test result data comprises the test result parameter of operating result data corresponding to each service order in test service stream and described indices;
Described existing network result data and described test result data are carried out consistency contrast, exports comparing result.
2. method according to claim 1, it is characterized in that, based on described measuring and calculation resource and described test authority, perform the operation that each service order in described test service stream is corresponding, before obtaining the test result data of described indices, described method also comprises:
Regulate the service order in described test service stream, perform test process based on the service order after regulating.
3. method according to claim 2, is characterized in that, regulates the service order in described test service stream, comprises based on the service order execution test process after regulating:
Regulate the concurrency of the service order in described test service stream, described concurrency comprises at least one item in instruction concurrency and mapping/stipulations Map/Reduce concurrency;
According to the concurrency after adjustment, issuing service instruction;
Test process is performed based on the service order issued.
4. method according to claim 3, is characterized in that, according to the concurrency after adjustment, issuing service instruction comprises:
Determine the priority of each service order in described existing network operations flows in described test service stream;
According to priority order from high to low, obtain the service order in described test service stream successively, until the service order number got arrives the instruction concurrency after regulating, issue the service order got;
And/or, according to priority order from high to low, obtain the service order in described test service stream successively, until the number of the mapping Map task that generates in described existing network operations flows of the service order got and stipulations Reduce task and value arrive the Map/Reduce concurrency after adjustment, issue the service order got.
5. method according to claim 4, is characterized in that, determines that in described test service stream, the priority of each service order in described existing network operations flows comprises:
The operation duration shared in described existing network operations flows according to each service order in described test service stream, determines the priority of each service order in described existing network operations flows; Or,
The mapping Map task generated in described existing network operations flows according to each service order in described test service stream and the number of stipulations Reduce task, determine the priority of each service order in described existing network operations flows.
6. method according to claim 1, is characterized in that, described method also comprises:
Operating result data corresponding for each service order in described test service stream be stored in the default storehouse table of described data cluster, the different sink table for same data cluster is shown in described default storehouse table and the storehouse of the operating result data storing described existing network service order; Or,
Under operating result data corresponding for each service order in described test service stream are stored in the predetermined directory of described data cluster, described predetermined directory and the operating result data storing existing network service order are the different directories of same data cluster.
7. method according to claim 1, is characterized in that, described method also comprises:
The test result data of indices in the test result data of described indices and the data cluster of old version is carried out consistency contrast, exports the comparing result between old version.
8. method according to claim 1, is characterized in that, described test authority comprises the authority of data of read-only current process business, the authority of the real data of read-only storage and the authority of read/write test result data.
9. for a testing apparatus for data cluster, it is characterized in that, described device comprises:
Test authority acquiring module, for obtaining test authority, the authority of described test authority and data cluster process business is isolated;
Source obtaining module, for the computational resource used according to data cluster process business, obtains measuring and calculation resource;
Existing network business acquisition module, for obtaining described data cluster to the existing network operations flows of described business and existing network result data, described existing network result data comprises the existing network actual parameter of the operating result data of existing network service order and the indices of process business procedure;
Recovery module, for carrying out reduction treatment to described existing network operations flows, obtains test service stream, and described test service stream comprises service order received in the described business procedure of process;
Test result data acquisition module, for based on described measuring and calculation resource and described test authority, perform the operation that each service order in described test service stream is corresponding, obtain the test result data of described test service stream, described test result data comprises the test result parameter of operating result data corresponding to each service order in test service stream and described indices;
Contrast module, for described existing network result data and described test result data are carried out consistency contrast, exports comparing result.
10. device according to claim 9, is characterized in that, described device also comprises:
Adjustment module, for regulating the service order in described test service stream, performs test process based on the service order after regulating.
11. devices according to claim 10, is characterized in that, described adjustment module comprises:
Concurrency regulon, for regulating the concurrency of the service order in described test service stream, described concurrency comprises at least one item in instruction concurrency and mapping/stipulations Map/Reduce concurrency;
Instruction issues unit, for according to regulate after concurrency, issuing service instruction;
Test execution unit, for performing test process based on the service order issued.
12. devices according to claim 11, is characterized in that, described instruction issues unit and comprises:
Priority determination subelement, for determining the priority of each service order in described existing network operations flows in described test service stream;
Instruction issues subelement, for according to priority order from high to low, obtains the service order in described test service stream successively, until the service order number got arrives the instruction concurrency after regulating, issues the service order got;
And/or, instruction issues subelement, also for according to priority order from high to low, obtain the service order in described test service stream successively, until the number of the mapping Map task that generates in described existing network operations flows of the service order got and stipulations Reduce task and value arrive the Map/Reduce concurrency after adjustment, issue the service order got.
13. devices according to claim 12, it is characterized in that, described priority determination subelement is used for the operation duration shared in described existing network operations flows according to each service order in described test service stream, determines the priority of each service order in described existing network operations flows; Or,
Described priority determination subelement is used for the number of mapping Map task and the stipulations Reduce task generated in described existing network operations flows according to each service order in described test service stream, determines the priority of each service order in described existing network operations flows.
14. devices according to claim 9, is characterized in that, described device also comprises:
First memory module, for operating result data corresponding for each service order in described test service stream being stored in the default storehouse table of described data cluster, the different sink table for same data cluster is shown in described default storehouse table and the storehouse of the operating result data storing described existing network service order; Or,
Second memory module, under operating result data corresponding for each service order in described test service stream are stored in the predetermined directory of described data cluster, described predetermined directory and the operating result data storing existing network service order are the different directories of same data cluster.
15. devices according to claim 9, is characterized in that, described device also comprises:
Old version contrast module, for the test result data of indices in the test result data of described indices and the data cluster of old version is carried out consistency contrast, exports the comparing result between old version.
16. devices according to claim 9, is characterized in that, described test authority comprises the authority of data of read-only current process business, the authority of the real data of read-only storage and the authority of read/write test result data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310697802.8A CN104022913B (en) | 2013-12-18 | 2013-12-18 | For method of testing and the device of data cluster |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310697802.8A CN104022913B (en) | 2013-12-18 | 2013-12-18 | For method of testing and the device of data cluster |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104022913A CN104022913A (en) | 2014-09-03 |
| CN104022913B true CN104022913B (en) | 2015-09-09 |
Family
ID=51439495
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310697802.8A Active CN104022913B (en) | 2013-12-18 | 2013-12-18 | For method of testing and the device of data cluster |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104022913B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104539614B (en) * | 2014-12-27 | 2018-03-27 | 北京奇安信科技有限公司 | A kind of method and apparatus of website detection number of concurrent control |
| CN105243168B (en) * | 2015-11-11 | 2019-08-30 | 中国建设银行股份有限公司 | A kind of data migration method and system |
| CN105577470A (en) * | 2015-12-12 | 2016-05-11 | 天津南大通用数据技术股份有限公司 | Cluster abnormity orthogonal test method based on distributed resource control |
| CN107085552A (en) * | 2016-02-15 | 2017-08-22 | 中兴通讯股份有限公司 | Method of testing and device |
| CN106453549B (en) * | 2016-10-08 | 2019-09-13 | 中国工商银行股份有限公司 | A kind of distributed software version dissemination method and system |
| CN109976997B (en) * | 2017-12-28 | 2022-12-27 | 北京京东尚科信息技术有限公司 | Test method and device |
| CN112463850A (en) * | 2020-11-20 | 2021-03-09 | 中国建设银行股份有限公司 | Data processing method, device and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8321454B2 (en) * | 2009-09-14 | 2012-11-27 | Myspace Llc | Double map reduce distributed computing framework |
| CN103176903A (en) * | 2013-03-12 | 2013-06-26 | 百度在线网络技术（北京）有限公司 | Testing method and testing device of MapReduce distributed system program |
| CN103338132A (en) * | 2013-06-21 | 2013-10-02 | 安科智慧城市技术（中国）有限公司 | Method and system for verifying access of equipment in management platform |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090199047A1 (en) * | 2008-01-31 | 2009-08-06 | Yahoo! Inc. | Executing software performance test jobs in a clustered system |
-
2013
- 2013-12-18 CN CN201310697802.8A patent/CN104022913B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8321454B2 (en) * | 2009-09-14 | 2012-11-27 | Myspace Llc | Double map reduce distributed computing framework |
| CN103176903A (en) * | 2013-03-12 | 2013-06-26 | 百度在线网络技术（北京）有限公司 | Testing method and testing device of MapReduce distributed system program |
| CN103338132A (en) * | 2013-06-21 | 2013-10-02 | 安科智慧城市技术（中国）有限公司 | Method and system for verifying access of equipment in management platform |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104022913A (en) | 2014-09-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104022913B (en) | For method of testing and the device of data cluster | |
| CN107957957B (en) | Test case obtaining method and device | |
| US20200013065A1 (en) | Method and Apparatus of Identifying a Transaction Risk | |
| CN110309071B (en) | Test code generation method and module, and test method and system | |
| CN106897322B (en) | A kind of access method and device of database and file system | |
| KR101559206B1 (en) | Method of processing log data, and system operating the same | |
| DE112012005037B4 (en) | Manage redundant immutable files using deduplications in storage clouds | |
| DE112019002948T5 (en) | DETERMINING AN OPTIMAL STORAGE ENVIRONMENT FOR DATA RECORDS AND FOR THE MIGRATION OF DATA RECORDS | |
| CN106980669A (en) | Data storage and acquisition method and device | |
| CN104331487B (en) | The processing method and processing device of log | |
| CN105550270B (en) | Data base query method and device | |
| CN110377519B (en) | Performance capacity test method, device and equipment of big data system and storage medium | |
| US11321318B2 (en) | Dynamic access paths | |
| US20140207754A1 (en) | Gathering Index Statistics Using Sampling | |
| CN103019855A (en) | Method for forecasting executive time of Map Reduce operation | |
| CN104778123A (en) | Method and device for detecting system performance | |
| US11182386B2 (en) | Offloading statistics collection | |
| CN104376397B (en) | A kind of data real-time analysis method and device | |
| CN111639016A (en) | Big data log analysis method and device and computer storage medium | |
| CN109446167A (en) | A kind of storage of daily record data, extracting method and device | |
| CN111552956A (en) | Role authority control method and device for background management | |
| CN110019229B (en) | Database configuration system | |
| CN107894942B (en) | Method and device for monitoring data table access amount | |
| CN109656945A (en) | A kind of information processing method | |
| CN108873781A (en) | A kind of Full-automatic digital equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |