CN111459815B - Real-time computing engine testing method and system - Google Patents
Real-time computing engine testing method and system Download PDFInfo
- Publication number
- CN111459815B CN111459815B CN202010236464.8A CN202010236464A CN111459815B CN 111459815 B CN111459815 B CN 111459815B CN 202010236464 A CN202010236464 A CN 202010236464A CN 111459815 B CN111459815 B CN 111459815B
- Authority
- CN
- China
- Prior art keywords
- test
- real
- time
- data
- testing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 238000012360 testing method Methods 0.000 title claims abstract description 328
- 238000000034 method Methods 0.000 claims abstract description 62
- 238000004364 calculation method Methods 0.000 claims abstract description 44
- 238000013112 stability test Methods 0.000 claims abstract description 15
- 238000011056 performance test Methods 0.000 claims abstract description 11
- 238000012545 processing Methods 0.000 claims description 36
- 230000002159 abnormal effect Effects 0.000 claims description 35
- 230000006870 function Effects 0.000 claims description 19
- 230000004622 sleep time Effects 0.000 claims description 10
- 238000003672 processing method Methods 0.000 claims description 4
- 238000012217 deletion Methods 0.000 claims description 3
- 230000037430 deletion Effects 0.000 claims description 3
- 238000002715 modification method Methods 0.000 claims description 3
- 230000003993 interaction Effects 0.000 claims description 2
- 238000010998 test method Methods 0.000 abstract description 3
- 230000008569 process Effects 0.000 description 28
- 238000010586 diagram Methods 0.000 description 12
- 238000004590 computer program Methods 0.000 description 7
- 238000011990 functional testing Methods 0.000 description 4
- 238000013461 design Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000004883 computer application Methods 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 239000012634 fragment Substances 0.000 description 2
- 230000007774 longterm Effects 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 238000012430 stability testing Methods 0.000 description 2
- 230000005856 abnormality Effects 0.000 description 1
- 238000009825 accumulation Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 239000000306 component Substances 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 239000008358 core component Substances 0.000 description 1
- 230000009193 crawling Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002035 prolonged effect Effects 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 230000009897 systematic effect Effects 0.000 description 1
- 230000001131 transforming effect Effects 0.000 description 1
- 230000007306 turnover Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3684—Test management for test design, e.g. generating new test cases
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention discloses a real-time computing engine test method and a system, wherein the method comprises the following steps: selecting a test direction; the test direction includes: three major directions of performance test, function test and stability test are provided; adding test cases in the corresponding test directions; the test case comprises: name and belonging scene; determining whether the test case needs parameters, and if the test case needs the parameters, acquiring input parameters; otherwise, continuing to execute; calling a real-time calculation engine to realize the test content related to the test case; calculating and acquiring result data of the test case request; the result data is a test result; and displaying the test result. According to the method, the strength of the engine is calculated in real time and the overall performance is analyzed through simulating the load and testing different pressure types, so that the testing efficiency can be effectively improved.
Description
Technical Field
The invention relates to the technical field of data processing, in particular to a real-time computing engine testing method and system.
Background
Data is the basis of calculation, and data processing widely exists in various computer applications and is an important function and a component of the computer applications. However, most of the research on data processing is focused on specific methods and technologies of data processing, and a systematic and generalized data processing method is still lacking.
According to IBM's statistical reports, more than 90% of the world's data production today is due to the emergence of new equipment, sensors, and technology, and the data growth rate is accelerated. Technically, this means that large data fields are becoming more complex and challenging to process. The real-time computing engine is a core component for running real-time computing, and is used for providing various tools required by various operations, so that the real-time computing engine is concerned by vast developers.
In the existing real-time calculation engine testing method in the market, all testing modules cannot be used in common, and the maintenance workload is large. Moreover, a tester who takes over the work newly needs to learn a new calculation engine frame, the learning cost is high, and the working period is prolonged.
The invention patent CN 106649119B discloses a method and an apparatus for testing a flow calculation engine, which is to test the flow calculation engine, wherein the flow calculation emphasizes the calculation of data in a certain time period, such as the turnover of a day, the next unit of a day, and the like. The patent document also has the following problems:
1) the stream computing engine only analyzes the flowing data in real time, selects a multithreading parallel mode for a scene with large data volume, adopts distributed computing for sample data, and distributes computing logic to a plurality of computing nodes to process the data in parallel. If the real-time data change is small, the system can still repeatedly perform a large amount of calculation when the actually used calculation model does not need to be changed, the real-time performance is poor, and resources are wasted;
2) because the data of the flow calculation is generally dynamic and has no boundary, the controllability is poor;
3) the system does not make related webpage end interfaces at the steps of inputting parameters, displaying calculation results and the like which need interaction, is not friendly to users, particularly non-professional users, and greatly reduces the usability of the system.
The above are the main problems faced by the real-time computing engine test, and these problems result in a long implementation period and a high learning cost of the real-time computing engine test project. No effective solution has been proposed to the above problems.
Disclosure of Invention
In view of the above problems, the present invention is proposed to provide a method and a system for testing a real-time computing engine, which overcome or at least partially solve the above problems, simulate a load, test the strength and analyze the overall performance of the real-time computing engine through different pressure classes, and can effectively improve the testing efficiency.
In a first aspect, an embodiment of the present invention provides a method for testing a real-time computing engine, including:
selecting a test direction; the test direction includes: three major directions of performance test, function test and stability test are provided;
adding test cases in the corresponding test directions; the test case comprises: name and belonging scene;
determining whether the test case needs parameters, and if the test case needs the parameters, acquiring input parameters; otherwise, continuing to execute;
calling a real-time calculation engine to realize the test content related to the test case;
calculating and acquiring result data of the test case request; the result data is a test result;
and displaying the test result.
Further, invoking a real-time computing engine to implement the test content related to the test case, including:
under the performance testing direction, when the real-time computing engine executes different program codes of different languages, the throughput, the memory, the CPU, the JVM and the real-time performance are respectively tested.
Further, invoking a real-time computing engine to implement the test content related to the test case, including:
and under the function test direction, when the real-time computing engine executes different program codes of different languages, the abnormal data processing, the data caching and the data format are tested.
Further, invoking a real-time computing engine to implement the test content related to the test case, including:
and under the stability test direction, continuously operating for a preset time by loading a load, and detecting whether stable operation can be performed or not.
In a second aspect, an embodiment of the present invention further provides a real-time computing engine testing system, including:
the test direction selecting module is used for selecting a test direction; the test direction includes: three major directions of performance test, function test and stability test are provided;
the test case adding module is used for adding test cases in the corresponding test direction; the test case comprises: name and belonging scene;
the test parameter acquisition module is used for determining whether the test case needs parameters or not, and acquiring input parameters if the test case needs the parameters; otherwise, continuing to execute;
the execution test case module is used for calling a real-time calculation engine and realizing the test content related to the test case;
the test result calculating module is used for calculating and acquiring result data of the test case request; the result data is a test result;
and the test result display module is used for displaying the test result.
Further, the test case execution module is specifically configured to test throughput, a memory, a CPU, a JVM, and real-time performance, respectively, when the real-time computing engine executes different program codes in different languages in the performance testing direction.
Further, the test case execution module is specifically configured to perform testing on abnormal data processing, data caching, and data formats when the real-time computing engine executes different program codes in different languages in the functional testing direction.
Further, the test case execution module is specifically configured to continuously run for a preset time by loading a load in the stability test direction, and detect whether the test case execution module can stably run.
The technical scheme provided by the embodiment of the invention has the beneficial effects that at least:
the embodiment of the invention provides a real-time computing engine testing method, which comprises the following steps:
(1) unlike the nearest convective calculation engine test, streaming calculation represents one way to process data, while real-time calculation represents a time consuming case to process data. The real-time computing engine provided by the embodiment of the invention processes the data set with the boundary, and finishes processing the limited data, so that the controllability is stronger;
(2) the invention can subdivide the testing requirement aspect of the real-time computing engine, can meet the testing of different aspects of the performance, the function and the stability of the real-time computing engine, and can reflect the computing capability of the real-time computing engine more comprehensively and more accurately according to the testing result.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
The technical solution of the present invention is further described in detail by the accompanying drawings and embodiments.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention and not to limit the invention. In the drawings:
FIG. 1 is a flow chart of a method for testing a real-time computing engine according to an embodiment of the present invention;
FIG. 2 is a diagram of test items included in three test directions provided by an embodiment of the present invention;
fig. 3 is a schematic diagram illustrating exchange between two processes of a memory according to an embodiment of the present invention;
FIG. 4 is a test flow diagram provided by an embodiment of the present invention;
FIG. 5 is a block diagram of a real-time computing engine testing system provided by an embodiment of the present invention;
fig. 6 is an overall flowchart of a real-time computing engine testing system according to an embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
In order to facilitate accurate understanding of the real-time computing engine testing method provided by the invention, some related terms are specifically explained:
1) and (3) calculating in real time: the method is carried out aiming at mass data, and generally requires the second level. The method mainly comprises two steps of real-time data storage and real-time data calculation;
2) JVM: the abbreviation of Java Virtual Machine, a specification for computing devices, is an imaginary computer implemented by emulating various computer functions on a real computer;
3) real-time performance: the reaction capacity of the system within a specified time;
4) abnormal data: data indicating an abnormality occurring during the program operation or abnormal data in the program operation result;
5) operating time: time in run state;
6) throughput: refers to the amount of information processed in a unit of time;
7) data caching: refers to a high-speed memory inside a hard disk, which temporarily stores some data for reading and re-reading in a computer like a buffer. The method has great advantages for the hard disk of the big data cache when accessing scattered files;
8) physical memory: relative to virtual memory. The physical memory refers to a memory space obtained by a physical memory bank, and the virtual memory refers to a region of the hard disk divided as a memory;
9) memory exchange: the process must be in memory in order to execute. However, the process may be temporarily swapped from memory to backup storage, and recalled to memory when executed again;
10) memory usage: refers to the memory spent by the process;
11) CPU occupation: the CPU resource required by program operation;
12) data set: also known as a data set, a data set or a data set, is a set composed of data;
13) data processing: collecting, storing, retrieving, processing, transforming and transmitting data;
14) test case: refers to the description of testing tasks performed on a particular software product, embodying test schemes, methods, techniques and strategies. The content of the method comprises a test target, a test environment, input data, test steps, expected results and the like, and finally a document is formed;
15) and (3) performance testing: various normal, peak and abnormal load conditions are simulated through an automatic testing tool to test various performance indexes of the system;
16) and (4) functional test: testing the characteristics and operational behavior of a product to determine that they meet design requirements, according to a test plan;
17) and (3) stability testing: the long-term stable running capability of the test system needs time accumulation to be measured.
Under general conditions, the real-time calculation data volume is small, rapid reaction is emphasized, and the real-time calculation is completed in a memory. For example, when dealing with mobile application advertising, fraud detection, taxi booking and patient monitoring scenarios, real-time data needs to be processed in real-time in order to make a fast and feasible decision.
In a first aspect, referring to fig. 1, a method for testing a real-time computing engine according to an embodiment of the present invention includes: S1-S6;
s1, selecting a test direction; the test direction includes: three major directions of performance test, function test and stability test are provided;
s2, adding test cases in the corresponding test directions; the test case comprises: name and belonging scene;
s3, determining whether the test case needs parameters or not, and if the test case needs the parameters, acquiring input parameters; otherwise, continuing to execute;
s4, calling a real-time calculation engine to realize the test content related to the test case;
s5, calculating and obtaining result data of the test case request; the result data is a test result;
and S6, displaying the test result.
In the embodiment, the test direction is selected firstly, and the test can be performed in three directions of performance test, function test and stability test; adding test cases in the corresponding test direction, wherein the test cases comprise names and scenes of the added test cases; after the test case is added, judging whether the test case needs parameters or not, if so, inputting, otherwise, continuing to execute; then, calling a real-time calculation engine to realize the test content related to the test case; after the real-time calculation engine is called, calculating and acquiring result data of the test case request; and finally, displaying the test result. According to the method, the time for computing and emphasizing the computing process in real time is short, namely the current found result is given, the computing magnitude is relatively small, and the emphasis is on the timeliness, such as scenes of killing a user in seconds, promoting the user in seconds and the like. The test case library covers the test case library with functional integrity, reliability and safety, and the test efficiency can be effectively improved.
The above steps will be described in detail below.
1. Selecting a test direction: each test case has a corresponding test direction, and the embodiment of the invention is carried out in three directions of performance test, functional test and stability test. The three test directions include test items as shown in FIG. 2. The performance test is to test various performance indexes of the real-time computing system; the functional test determines whether the design requirements are met according to the test scheme; the stability test is to test the long-term stable operation capability of the real-time computing system.
In one embodiment, in the performance testing direction, when the real-time computing engine executes different program codes of different languages, the throughput, the memory, the CPU, the JVM and the real-time performance are respectively tested.
The throughput was tested: throughput, which means the amount of information processed per unit time, has different calculation methods from different traffic perspectives. Throughput is specified as the amount of processing data per unit time in the embodiment of the present invention. If the program runs for N seconds and processes M bytes of data together, the throughput is M/N (bytes/second). Because the time required by the same program is different when the same program is run at different times, the embodiment of the invention tests the corresponding throughput when the program is run at different times under the condition of the same processing data volume. In the running process of the program, the sleep time can be set, the running of the program is interrupted, and after the sleep time is set, the corresponding throughput can be tested. Therefore, under the condition of adding the program running times or the sleep time, the specific running times and the sleep time are required to be input as parameters when testing the throughput.
And testing the exchange memory and the physical memory: the exchange of two processes of memory is schematically illustrated in fig. 3, where the processes must be in memory in order to execute. However, the process may be temporarily swapped from memory to backup storage and recalled to memory when executed again. The embodiment of the invention tests the size of the exchange memory, the used memory for exchange, the unused memory for exchange, the occupancy rate of the exchange memory, the accumulated number of bytes exchanged from the disk by the system and the accumulated number of bytes exchanged from the disk by the system. The embodiment of the invention can test the size of the physical memory, the occupancy rate of the physical memory, the physical used memory, the physical unused memory, the physical memory currently used or recently used, the marked unused memory, the cache used by the buffer area, the cache used by the cache area, the size of the physical memory displayed to be shared and the cache of the memory data structure.
And (3) testing the CPU: the test items specifically include CPU time spent by the process executing the user mode code, percentage of CPU occupancy when executing at a user level with high priority, CPU time used by the kernel to execute the system call, percentage of time that the CPU is idle and the system has no outstanding disk I/O requests, time that the CPU waits for I/O input/output, percentage of time that the CPU spends maintaining hardware interrupts, percentage of time that the CPU spends servicing software interrupts, percentage of time that the CPU is used to run virtual processors, other operating systems running in the virtualization environment, percentage of CPU occupancy when executing at a user level with high priority, utilization of the CPU in the elapsed time of the last call, and utilization of the CPU in the current 1S.
Testing the JVM: the test items specifically include memory garbage collection and stack memory exception detection. The JVM adopts the concept of generation collection to manage the memory, each object is provided with an age counter, and when the objects are collected, the objects which need to be placed in the new generation and the objects which need to be placed in the old generation can be identified. And if the stack depth of the thread request is greater than the maximum depth allowed by the JVM, throwing a stack overflow exception. And when the heap memory reaches the set maximum memory parameter value, throwing out the overflow exception of the heap memory.
And (3) testing the real-time performance: in real-time computing, the input data is input and processed one by one in a sequential manner, i.e., not all of the input data need be known before the program runs. The real-time computing framework requires immediate return of the computation results and rapid response to the request. Typical metrics for real-time performance include preemption time, interrupt delay time, semaphore shuffle time, deadlock resolution time, and packet throughput rate. In the embodiment of the invention, the message queue Kafka is used in the storage process and the frame Spark Streaming is used in the processing process. The Kafka cluster contains one or more servers, which are called brookers. Each message issued to the Kafka cluster has a category, which is called Topic. The specific process is as follows: the Producer responsible for issuing the message sends the crawled data to the Kafka browser according to a certain time interval to simulate real-time data acquisition, the Consumer serves as a message Consumer, the message is read from the Kafka browser and processed, and the Spark stores the processed data in a designated position.
In a specific embodiment, in the functional testing direction, when the real-time computing engine executes different program codes of different languages, a test case is designed and implemented for the parts of exception data processing, data caching and data format.
And (3) testing abnormal data processing: in the process of real-time data processing, almost all data received by Spark Streaming has some abnormal data, and these abnormal data include attribute missing, value out-of-range, data format error, etc. in order to minimize the influence of these abnormal values on the data calculation result, before the data is calculated, a program will process these abnormal data. The inventive examples were tested in three ways:
(1) and recording the data volume to be cleaned before abnormal data processing is carried out each time. After the abnormal data are detected, recording the abnormal data amount, and then calculating the proportion of the abnormal data amount in the total data amount;
(2) recording the running time of each abnormal data processing part, and calculating the proportion of the abnormal data processing time to the total running time of the program after the program runs;
(3) after the program is processed by abnormal data, the data amount is recorded again and compared with the data before processing, so that the data processing method used by the program is deduced, if the data amount is obviously reduced, the deletion method is deduced, and if the data amount is not obviously reduced, the filling method or the modification method is deduced.
The embodiment of the invention tests the influence of the number of cache-free fragments on the program running time and the influence of the number of cache-containing fragments on the program running time.
The embodiment of the invention can test various application scenes of the real-time computing engine, so that different types of data can be processed. The format of the input data of the program is different, and the use condition and the access speed of the storage space are influenced. Therefore, the embodiment of the invention can test the influence of the data sets with different data formats on the program operation.
In one embodiment, in the stability testing direction, the system is continuously operated for a period of time (for example, generally 7 × 24 hours) by loading a certain service pressure to the system, and whether the system can operate stably is detected, so as to check the reliability of the system. The embodiment of the invention takes throughput as an example, and tests whether the throughput can be correctly calculated when the system runs for a long time;
2. and adding test cases, namely adding the test cases in the corresponding test directions, wherein the test cases comprise names and belonged scenes of the added test cases. The whole test system comprises a plurality of test cases, so a plurality of test tasks need to be added. The test case of the embodiment of the invention comprises a test name, a test scene, input data, a test step, an expected result and a test script. For example, the embodiment of the invention only embodies the test name and the test scene at the webpage end, and the contents of other test cases are expressed in the document. The test flow for executing the test is shown in fig. 4, and a plurality of paths can be selected in the process of executing the test, without being limited to one test path;
3. and test parameter acquisition, namely judging whether the test case needs parameters or not after the test case is added, if so, inputting, and otherwise, continuously executing. For example, when testing throughput with sleep time, a specific sleep time needs to be input at a web page end as a parameter;
4. executing the test case, calling a real-time calculation engine, and realizing the test content related to the test case;
5. calculating a test result, and calculating and acquiring result data of the test case request after the real-time calculation engine is called;
6. and displaying the test result, returning the calculation result to the webpage by the real-time calculation engine, and displaying the test result in a visualized manner by the webpage.
The real-time calculation engine provided by the embodiment of the invention emphasizes that the time of the calculation process is short, namely the current result is found, the calculation magnitude is relatively small, the timeliness is emphasized, and the input data is generated in real time, so that the input data can be changed all the time. Such as user kill, promotion, etc.
The real-time computing engine provided by the embodiment of the invention has two realization methods, one is a real-time simulation method, and the other is a real-time method. The method of simulating real time is a method in which all data is known in advance, but is input to a real time calculation engine at a certain time interval, and the data is constantly changing from the viewpoint of the calculation engine. The calculation engine is tested, for example, by a program that analyzes men and women's number of purchases per second in real time. For example, the data set compression package used is the transaction data of the first 6 months of the two 11 years of the Taobao 2015. And reading the shopping log data, and sending the data to the calculation engine at the same interval. This is a way of simulating real time. While true real-time is where data is acquired and input to the compute engine in real-time. For example, when crawling website data in real time, the data is not fully in place until the program runs. When the real-time computing engine processes the real-time data, the real-time computing engine respectively tests the real-time data in three major directions of performance, function and stability, test items are determined by the characteristics of the processed real-time data, and the test items are shown in fig. 2.
The real-time computing engine test method provided by the embodiment of the invention has the following advantages:
(1): unlike the nearest convection calculation engine test, flow calculation represents one way to process data, while real-time calculation represents a time consuming case to process data. The real-time computing engine processes the data set with the boundary, and finishes processing the limited data, so that the controllability is stronger;
(2): the invention can subdivide the testing requirement aspect of the real-time computing engine, can meet the testing of different aspects of performance, function and stability which focus on the real-time computing engine, and more finely divides the testing parameters in various directions, so that the testing result can more comprehensively and more accurately reflect the computing capability of the real-time computing engine.
Based on the same inventive concept, the embodiment of the invention also provides a real-time computing engine test system, and as the principle of the problem solved by the system is similar to that of a real-time computing engine test method, the implementation of the system can refer to the implementation of the method, and repeated parts are not repeated.
In a second aspect, the present invention further provides a real-time computing engine testing system, as shown in fig. 5, including:
the test direction selecting module is used for selecting a test direction; the test direction includes: three major directions of performance test, function test and stability test are provided;
the test case adding module is used for adding test cases in the corresponding test direction; the test case comprises: name and belonging scene;
the test parameter acquisition module is used for determining whether the test case needs parameters or not, and acquiring input parameters if the test case needs the parameters; otherwise, continuing to execute;
the execution test case module is used for calling a real-time calculation engine and realizing the test content related to the test case;
the test result calculating module is used for calculating and acquiring result data of the test case request; the result data is a test result;
and the test result display module is used for displaying the test result.
In this embodiment, the system is presented in the form of a website. The user can interact with the webpage end, including selecting a test direction, adding a test case and inputting test parameters, and the webpage end can also display the result of the test of the real-time computing engine for the user. After the webpage end submits the testing task, the webpage firstly sends a request to the server, and then the server requests the real-time computing engine to be tested. The real-time computing engine can execute the program file through the program operation interface, thereby computing the test result and returning the test result to the webpage end. For convenient understanding, the system is divided into six modules, namely a test direction selecting module, a test case adding module, a test parameter obtaining module, a test case executing module, a test result calculating module and a test result displaying module.
Firstly, executing a test direction selection module, wherein the test direction selection module is executed in three directions of performance test, functional test and stability test, so that the test direction of the test case needs to be judged before adding the test case; executing a test case adding module, after selecting a test direction, adding test cases in the corresponding test direction, wherein the test cases comprise names and scenes of the added test cases; then executing a test parameter acquisition module, after adding the test case, judging whether the test case needs test parameters, if so, inputting, otherwise, continuing to execute; next, a module for executing the test case is used, wherein the module is the most critical module in the whole test flow, and a real-time computing engine needs to be called to realize the test content related to the test case; then executing a calculation test result module, calling a real-time calculation engine, and calculating and acquiring result data of the test case request; and finally, a test result display module is used for displaying the test result in a visual mode on the webpage side.
The system is presented in the form of a website, and the overall flow chart of the system is shown in fig. 6. The user can interact with the webpage end, including selecting a test direction, adding a test case and inputting test parameters, and the webpage end can also display the result of the test of the real-time computing engine for the user. After the webpage end submits the testing task, the webpage sends a request to the server, and the server requests the real-time computing engine to be tested. The real-time computing engine can execute the program file through the program operation interface, thereby computing the test result and returning the test result to the webpage end.
In this embodiment, in addition to the advantages of the above method embodiment, in order to improve the user experience degree, in the embodiment of the present invention, a test website system may be designed as a front end interacting with a real-time computing engine, which is more user-friendly, and a user may design a test scheme using simpler and easier operations, and may also visually and clearly show a test result to the user.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, and the like) having computer-usable program code embodied therein.
The present invention has been described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present invention without departing from the spirit and scope of the invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include such modifications and variations.
Claims (4)
1. A real-time computing engine testing method is characterized by comprising the following steps:
selecting a test direction; the test direction includes: three major directions of performance test, function test and stability test are provided;
adding test cases in the corresponding test directions; the test case comprises: name and belonging scene;
determining whether the test case needs parameters, and if the test case needs the parameters, acquiring input parameters; otherwise, continuing to execute;
calling a real-time calculation engine to realize the test content related to the test case;
calculating and acquiring result data of the test case request; the result data is a test result;
displaying the test result;
calling a real-time computing engine to realize the test content related to the test case, wherein the method comprises the following steps:
under the performance testing direction, when the real-time computing engine executes different program codes of different languages, respectively testing throughput, memory, CPU, JVM and instantaneity;
in the function testing direction, when the real-time computing engine executes different program codes of different languages, abnormal data processing, data caching and data formats are tested;
wherein the throughput is tested: under the condition of adding program running times or sleep time, when testing throughput, inputting specific running times and sleep time as parameters;
the testing of the memory includes: exchanging the memory and the physical memory for testing; testing the size of the exchange memory, the used memory for exchange, the unused memory for exchange, the occupancy rate of the exchange memory, the accumulated byte number exchanged from the disk by the system and the accumulated byte number exchanged from the disk by the system;
testing the size of a physical memory, the occupancy rate of the physical memory, a physically used memory, a physically unused memory, a currently used or recently used physical memory, a marked unused memory, a buffer used by a buffer area, a buffer used by the buffer area, a physical memory size and a memory data structure buffer used by shared display;
and (3) testing abnormal data processing: (1) recording the data volume to be cleaned before abnormal data processing; after the abnormal data are detected, recording the abnormal data amount, and then calculating the proportion of the abnormal data amount in the total data amount;
recording the running time of each abnormal data processing part, and calculating the proportion of the abnormal data processing time to the total running time of the program after the program runs;
after the program is processed by abnormal data, the data amount is recorded again and compared with the data before processing, so as to deduce the data processing method used by the program, if the data amount is obviously reduced, the method is deduced to be a deletion method, if the data amount is not obviously reduced, the method is deduced to be a filling method or a modification method,
and displaying the test result, and interacting with a webpage end by a user through webpage presentation, wherein the interaction comprises the steps of selecting a test direction, adding a test case and inputting test parameters.
2. The method for testing a real-time computing engine according to claim 1, wherein invoking the real-time computing engine to implement the test content related to the test case comprises:
and under the stability test direction, continuously running for a preset time by loading a load, and detecting whether stable running can be performed or not.
3. A real-time computing engine testing system, comprising:
the test direction selecting module is used for selecting a test direction; the test direction includes: three major directions of performance test, function test and stability test are provided;
the test case adding module is used for adding test cases in the corresponding test direction; the test case comprises: name and belonging scene;
the test parameter acquisition module is used for determining whether the test case needs parameters or not, and acquiring input parameters if the test case needs the parameters; otherwise, continuing to execute;
the execution test case module is used for calling a real-time calculation engine and realizing the test content related to the test case;
the test result calculating module is used for calculating and acquiring result data of the test case request; the result data is a test result;
the test result display module is used for displaying the test result; the test result is displayed and presented through a webpage, and a user interacts with a webpage end, including selecting a test direction, adding a test case and inputting test parameters;
the execution test case module is specifically used for respectively testing throughput, memory, CPU, JVM and instantaneity when the real-time computing engine executes different program codes of different languages in the performance testing direction; wherein the throughput is tested: under the condition of adding program running times or sleep time, when testing throughput, inputting specific running times and sleep time as parameters;
the execution test case module is specifically used for testing abnormal data processing, data caching and data formats when the real-time computing engine executes different program codes of different languages in the function test direction; and (3) testing abnormal data processing: (1) recording the data volume to be cleaned before abnormal data processing; after abnormal data are detected, recording the abnormal data quantity, and then calculating the proportion of the abnormal data quantity in the total data quantity;
(2) recording the running time of each abnormal data processing part, and calculating the proportion of the abnormal data processing time to the total running time of the program after the program runs;
(3) after the program is processed by abnormal data, the data amount is recorded again and compared with the data before processing, so that the data processing method used by the program is deduced, if the data amount is obviously reduced, the deletion method is deduced, and if the data amount is not obviously reduced, the filling method or the modification method is deduced.
4. The system according to claim 3, wherein the test case execution module is further configured to detect whether stable operation is possible by loading a load and continuously operating for a preset time in the stability test direction.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010236464.8A CN111459815B (en) | 2020-03-30 | 2020-03-30 | Real-time computing engine testing method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010236464.8A CN111459815B (en) | 2020-03-30 | 2020-03-30 | Real-time computing engine testing method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111459815A CN111459815A (en) | 2020-07-28 |
| CN111459815B true CN111459815B (en) | 2022-06-14 |
Family
ID=71680108
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010236464.8A Expired - Fee Related CN111459815B (en) | 2020-03-30 | 2020-03-30 | Real-time computing engine testing method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111459815B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112882910B (en) * | 2021-01-22 | 2024-09-10 | 中国建设银行股份有限公司 | Interface pressure testing method and device of workflow engine |
| CN113254344A (en) * | 2021-06-07 | 2021-08-13 | 吉林大学 | Novel computing engine test platform, device and system |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101853206A (en) * | 2010-06-18 | 2010-10-06 | 北京九合创胜网络科技有限公司 | Software performance detection method and device |
| CN102902621A (en) * | 2011-07-28 | 2013-01-30 | 塔塔咨询服务有限公司 | Application performance measurement and reporting |
| CN104182340A (en) * | 2014-08-21 | 2014-12-03 | 浪潮(北京)电子信息产业有限公司 | Database compatibility test method and system |
| CN106502898A (en) * | 2016-10-25 | 2017-03-15 | 广州神马移动信息科技有限公司 | Method of testing and test driver and test platform |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TW201321973A (en) * | 2011-11-25 | 2013-06-01 | Askey Technology Jiangsu Ltd | Handheld electronic device testing method |
-
2020
- 2020-03-30 CN CN202010236464.8A patent/CN111459815B/en not_active Expired - Fee Related
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101853206A (en) * | 2010-06-18 | 2010-10-06 | 北京九合创胜网络科技有限公司 | Software performance detection method and device |
| CN102902621A (en) * | 2011-07-28 | 2013-01-30 | 塔塔咨询服务有限公司 | Application performance measurement and reporting |
| CN104182340A (en) * | 2014-08-21 | 2014-12-03 | 浪潮(北京)电子信息产业有限公司 | Database compatibility test method and system |
| CN106502898A (en) * | 2016-10-25 | 2017-03-15 | 广州神马移动信息科技有限公司 | Method of testing and test driver and test platform |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111459815A (en) | 2020-07-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Lu et al. | Log-based abnormal task detection and root cause analysis for spark | |
| US8631401B2 (en) | Capacity planning by transaction type | |
| Oaks | Java Performance: The Definitive Guide: Getting the Most Out of Your Code | |
| US8938729B2 (en) | Two pass automated application instrumentation | |
| US10691571B2 (en) | Obtaining application performance data for different performance events via a unified channel | |
| CN111143286B (en) | Cloud platform log management method and system | |
| US7836435B2 (en) | Checking for memory access collisions in a multi-processor architecture | |
| CN103109276B (en) | System detection method | |
| US9442817B2 (en) | Diagnosis of application server performance problems via thread level pattern analysis | |
| CN108153587B (en) | Slow task reason detection method for big data platform | |
| CN111459815B (en) | Real-time computing engine testing method and system | |
| CA2948700A1 (en) | Systems and methods for websphere mq performance metrics analysis | |
| CN111563014A (en) | Interface service performance test method, device, equipment and storage medium | |
| Ousterhout | Always measure one level deeper | |
| CN104919427B (en) | For handling the method, computing system and computer readable storage medium of data | |
| US8286192B2 (en) | Kernel subsystem for handling performance counters and events | |
| CN104937548A (en) | Dynamic graph performance monitoring | |
| CN1279443C (en) | Test method of overall server performance | |
| CN112905431A (en) | Method, device and equipment for automatically positioning system performance problem | |
| CN114327963A (en) | Anomaly detection method and device | |
| CN115629953B (en) | Performance benchmark evaluation method suitable for domestic basic software and hardware environment | |
| CN111444102A (en) | Method and system for testing off-line computing engine | |
| EP3995966A1 (en) | System and method for automatic application log messages grouping using logging framework code instrumentation | |
| JP2014178865A (en) | Bottleneck analysis device, method, and program | |
| CN107861802A (en) | Method for managing resource and device in cloud data system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20220614 |