CN110727615B - Method and system for realizing acceleration of interface data request - Google Patents
Method and system for realizing acceleration of interface data request Download PDFInfo
- Publication number
- CN110727615B CN110727615B CN201910964034.5A CN201910964034A CN110727615B CN 110727615 B CN110727615 B CN 110727615B CN 201910964034 A CN201910964034 A CN 201910964034A CN 110727615 B CN110727615 B CN 110727615B
- Authority
- CN
- China
- Prior art keywords
- request
- script
- data
- interface
- execution
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
- G06F13/10—Program control for peripheral devices
- G06F13/102—Program control for peripheral devices where the programme performs an interfacing function, e.g. device driver
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
- G06F13/38—Information transfer, e.g. on bus
- G06F13/382—Information transfer, e.g. on bus using universal interface adapter
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Debugging And Monitoring (AREA)
- Stored Programmes (AREA)
Abstract
The invention provides a method for realizing an accelerated interface data request, which comprises the following steps: step S1, configuring a script name, configuring the script request mode, configuring the data source of each interface, configuring the report display mode, configuring the memory and CPU requirement; step S2, when executing any script, monitoring the script execution environment of the current service, when the service task content is more, performing time delay operation, and when the environment allows, entering step S3; step S3, adjusting the optimal number of concurrent threads according to the expected time of the user, dividing the data of the data source of the interface into each thread for execution, monitoring the execution state, and pushing the task when the execution is finished; step S4, reading a data source, and storing the request data, the response result, the response time and the response code according to the data structure of the json structure; the high concurrent execution of the interface request is realized, and the script execution time is effectively reduced.
Description
Technical Field
The invention relates to the technical field of automatic testing, in particular to a method and a system for realizing an accelerated interface data request.
Background
In the present day with the change of big data development, a plurality of services realize front-end separation, and the API of restful is provided by the back end for the front end to call. There are many application scenarios in which data verification is required. Among them are for example: in the fields of intelligent dialogue, handwriting recognition and the like, huge data sets are required to be verified. The conventional verification method has two methods: 1) performing function test acceptance through the application front end; 2) directly reading a certain text or initiating an interface request of a database through a script, and then making condition judgment according to returned data.
The first disadvantage is that if the test is accepted through the function test, although the test is intuitive, the test is very weak when the data magnitude is high, and the test cannot be completed at all. The efficiency of the request by the script is higher than that of the functional test, but is limited by the response time of the API interface request, and if the request magnitude is large, for example, 100000 data requests, calculated by the interface response speed of 1/second, may require 100000 seconds, that is, about 27.78 hours. In the internet era of agile iteration, this time is undoubtedly too long.
The second disadvantage is that the current relatively mature script test generally makes an interface request through an automation script and inputs the interface request into a certain file report through condition judgment, and the whole automation script is generally required to complete the test so as to see the result. One of the current executions is not known during the test. Meanwhile, if concurrent requests are frequently sent through a large number of script execution interfaces, the occupation of the CPU and the memory is relatively large, and if the CPU or the memory is full, the program may be crashed, or even the fault of the script execution server may occur.
And thirdly, most of the execution results of the interface scripts are reports generated by means of file storage and the like through rules required by program reservation at present. The method has certain realization difficulty for some testers without script capability, meanwhile, the test still needs to carry out personal analysis or report arrangement aiming at a large amount of data, and the process cost is relatively high.
Disclosure of Invention
In order to overcome the above problems, an object of the present invention is to provide a method for accelerating interface data requests, so as to implement a high concurrent execution of interface requests, and effectively reduce the time for executing scripts.
The invention is realized by adopting the following scheme: a method for implementing accelerated interface data requests, the method comprising the steps of:
step S1, configuring a script name, configuring the script request mode, configuring the data source of each interface, configuring the report display mode, configuring the memory and CPU requirement;
step S2, when executing any script, monitoring the script execution environment of the current service, when the service task content is more, performing time delay operation, and when the environment allows, entering step S3;
step S3, adjusting the optimal number of concurrent threads according to the expected time of the user, dividing the data of the data source of the interface into each thread for execution, monitoring the execution state, and pushing the task when the execution is finished;
and step S4, reading the data source, and storing the request data, the response result, the response time and the response code according to the data structure of the json structure.
Furthermore, the data structure of the json structure comprises three substructures request, response and other; the request is used for storing data from a data source, the response is used for storing result data returned after the request interface request data is requested, and the other is used for storing some characteristic information in the interface request process, wherein the characteristic information comprises response time and response codes.
Further, the step S1 is further specifically: step 11, creating a script name;
step 12, configuring the request mode of the script, namely configuring the request type and the request address;
step 13, configuring the data source of each interface, and loading the data source needing to execute the request; the method comprises the steps that database table information corresponding to a data source is configured, a corresponding database query code is configured at last, and when a script is executed, data to be tested can be known;
step 14, after configuring the data source, configuring the condition of report output, namely configuring the report output condition, wherein the configuration is divided into two parts, one part is the configuration condition, and the other part is the configuration display, so that the user can input the concerned content in a self-defined manner, but not output all executed data and return information;
and step 15, configuring an available memory and a CPU (Central processing Unit) value required by script execution for environment monitoring and judgment before the script execution.
Further, the step S2 is further specifically: step 21, when any script is executed, acquiring a required memory and a CPU value configured by a current script;
step 22, acquiring the actual occupation condition of one memory and CPU of the service;
step 23, obtaining whether the actual memory meets the required memory of the script according to an actual memory occupation condition of the service, if not, waiting for a set time, and modifying the state of the script execution task into waiting;
step 24, under the condition that the actual memory is satisfied, acquiring whether the serving CPU satisfies the CPU required by the script, and if not, waiting for the set time, and recalculating the current state;
and step 25, under the condition that the CPU and the actual memory of the current service are met, modifying the state of the script execution task to be in execution, and entering step 3.
Further, the step S3 is further specifically: step 31, the first stage of script execution is to obtain the configured data source and obtain the total data volume DateCount under the data source;
step 32, obtaining expected completion time expecteTime, wherein a main program of the service during script execution can initiate a pre-judging request for obtaining response time ResponseTime of the current interface;
step 33, the main program calculates how many concurrent threads ThreadCount are needed if the execution task needs to be completed within the ExpectTime, ThreadCount = DateCount ResponseTime/ExpectTime; when the calculated number of concurrent threads ThreadCount is greater than the upper line threshold ThreadLimit, ThreadCount = ThreadLimit, and at the same time, the actual estimated time expectedTimeLine is given, expectedTimeLine = DateCount @ ResponseTime/ThreadLimit;
step 34, dividing the data source, and respectively allocating a data source Date [ (i-1) × limit, i × limit-1], limit = DateCount/ThreadCount +1 to the corresponding thread, wherein i refers to the second thread;
and step 35, distributing the data source, executing the interface request by the multiple threads respectively, monitoring the thread execution condition by the main program, changing the script execution state to be completed when all the threads are executed, and pushing information.
Further, the step S4 is further specifically: step 41, circularly initiating an interface request according to a data list sent by a main program of the service;
step 42, after each interface request is initiated, the main program assigns the requested data request _ body and the response result _ body to the variable data, request _ body and variable data, response _ body, respectively, and assigns the response time and the response code status to the variable data, other, time and variable data, other, status;
step 43, after all data requests are completed, marks the end of the thread.
In addition, the invention also provides a system for realizing the interface data acceleration request, which comprises a script configuration module, an environment monitoring module, a script execution module and an interface request module;
the script configuration module is used for configuring the data source of each interface, configuring a script name, configuring the script request mode, configuring the memory and CPU requirements and configuring the report display mode;
the environment monitoring module is used for monitoring the script execution environment of the current service when any script is executed, carrying out delay operation when the service task content is more, and entering the script execution module when the environment allows;
the script execution module is used for adjusting the optimal number of concurrent threads according to the expected time of a user, dividing data of a data source of an interface into each thread for execution, monitoring an execution state, and pushing a task when the execution is finished;
and the interface request module is used for reading a data source and storing request data, response results, response time and response codes according to a data structure of the json structure.
Furthermore, the data structure of the json structure comprises three substructures request, response and other; the request is used for storing data from a data source, the response is used for storing result data returned after the request interface request data is requested, and the other is used for storing some characteristic information in the interface request process, wherein the characteristic information comprises response time and response codes.
Further, the script configuration module further comprises: the method comprises the following steps of creating a script unit, a configuration script request unit, a configuration interface data source unit, a configuration report output unit and a configuration memory and CPU unit;
the script creating unit is used for creating a script name;
the configuration script request unit is used for configuring the request mode of the script, namely the type of the request and the address of the request are required to be configured;
the configuration interface data source unit is used for configuring the data source of each interface and loading the data source needing to execute the request; the method comprises the steps that database table information corresponding to a data source is configured, a corresponding database query code is configured at last, and when a script is executed, data to be tested can be known;
the configuration report output unit is used for configuring the condition of report output after the data source is configured, namely the configuration of the report output condition is carried out for enabling a user to input concerned content in a self-defined manner instead of outputting all executed data and return information, and the configuration is divided into two parts, wherein one part is the configuration condition and the other part is the configuration display;
the configuration memory and CPU unit is used for configuring an available memory and a CPU value required by script execution and used for environment monitoring and judgment before the script execution.
Further, the environment monitoring module further comprises: the system comprises an acquisition unit, an acquisition service execution unit, a memory judgment unit, a CPU judgment unit and a state modification unit;
the acquiring unit is used for acquiring the required memory and CPU value configured by the current script when any script is executed;
the service acquisition execution unit is used for acquiring the actual occupation condition of a memory and a CPU of the service;
the memory judgment unit is used for obtaining whether the actual memory meets the required memory of the script according to an actual memory occupation condition of the service, if not, waiting for a set time, and modifying the state of the script execution task into waiting;
the CPU judgment unit is used for acquiring whether the serving CPU meets the CPU required by the script under the condition that the actual memory meets the requirement, and if the serving CPU does not meet the requirement, the CPU judgment unit waits for the set time and recalculates the current state;
and the state modification unit is used for modifying the state of the script execution task into the execution state under the condition that the CPU and the actual memory of the current service are both satisfied, and entering the script execution module.
Further, the script execution module further comprises: the device comprises a data quantity acquisition unit, a pre-judgment unit, a concurrent thread calculation unit, a data division unit and a monitoring unit;
the data volume obtaining unit is used for obtaining the configured data source in the first stage of script execution to obtain the total data volume DateCount under the data source;
the pre-judging unit is used for acquiring expected finished time expecteTime, and a main program served during the script execution period can initiate a pre-judging request for acquiring response time ResponseTime of the current interface;
the concurrent thread calculating unit is used for calculating how many concurrent threads are needed to complete the execution task within the ExpectTime, and the concurrent threads are ThreadCount = DateCount × ResponseTime/expectTime; when the calculated number of concurrent threads ThreadCount is greater than the upper line threshold ThreadLimit, ThreadCount = ThreadLimit, and at the same time, the actual estimated time expectedTimeLine is given, expectedTimeLine = DateCount @ ResponseTime/ThreadLimit;
the data source partitioning unit is used for partitioning a data source, corresponding threads are respectively allocated with a data source Date [ (i-1) × limit, i × limit-1], limit = DateCount/ThreadCount +1, and i refers to the number of threads;
the monitoring unit is used for distributing data sources, executing interface requests by a plurality of threads respectively, monitoring the execution condition of the threads by the main program, changing the execution state of the script into the completed state when all the threads are executed, and pushing information.
Further, the interface request module further includes: the system comprises an initiating request unit, an assignment unit and a mark unit;
the initiating request unit is used for circularly initiating an interface request according to a data list sent by a main program of the service;
the assignment unit is configured to assign a request _ body of the requested data and a response _ body of the response to a variable data, the request _ body and the variable data, the response _ body, and assign a response time and a response code state to the variable data, other, time and other, states, respectively, after each interface request is initiated;
and the marking unit is used for marking the end of the thread after all data requests are completed.
The invention has the beneficial effects that: 1. the interface script configuration is carried out by the mode provided by the invention, and the function of interface data verification can be completed by simple configuration.
2. The interface data request is carried out in the mode provided by the invention, so that high concurrent execution of the interface request can be realized, and the script execution time is effectively reduced.
3. The interface data management is carried out in the mode provided by the invention, the presentation of the custom report can be realized, the report display is realized in a configuration mode, and the development investment is reduced.
4. The environment monitoring module provided by the invention can reasonably perform script execution management, and avoid resource preemption, server instability and the like caused by multi-script execution.
5. By analyzing the interface data request result data (such as concurrency number, average response time, abnormal state code ratio and the like) in the mode provided by the invention, a performance outline of the service interface can be visually seen.
Drawings
FIG. 1 is a schematic flow diagram of the process of the present invention.
Fig. 2 is a block diagram of the system architecture of the present invention.
Fig. 3 is a schematic diagram of a service main flow according to an embodiment of the system of the present invention.
FIG. 4 is a functional block diagram of the script configuration module of the system of the present invention.
FIG. 5 is a block diagram of the environmental monitoring module of the system of the present invention.
FIG. 6 is a functional block diagram of the script execution module of the system of the present invention.
FIG. 7 is a block diagram of the operation of the interface request module of the system of the present invention.
Detailed Description
The invention is further described below with reference to the accompanying drawings.
Referring to fig. 1, the present invention implements management of addition, deletion, modification and check of a function script, manages a request method of a certain interface by configuring the type, address, request data and other contents of the request interface, and configures a condition of a data source and a report display that are required to be used by the request method. Secondly, when the script needs to be executed, an environment monitoring stage needs to be entered, when the service environment is monitored to have the execution condition, the script is entered to be executed, otherwise, the service environment enters a waiting state until the service environment has the execution condition. After the script enters the execution state, the number of parallel threads is adjusted according to a desired time of a user, when the service cannot allocate enough multiple threads, an actual optimal execution time is given and fed back, and after all data are executed, a currently executed result can be displayed according to an expected set report display condition.
The invention relates to a method for realizing acceleration of interface data request, which comprises the following steps:
step S1, configuring a script name, configuring the script request mode, configuring the data source of each interface, configuring the report display mode, configuring the memory and CPU requirement;
step S2, when executing any script, monitoring the script execution environment of the current service, when the service task content is more, performing time delay operation, and when the environment allows, entering step S3;
step S3, adjusting the optimal number of concurrent threads according to the expected time of the user, dividing the data of the data source of the interface into each thread for execution, monitoring the execution state, and pushing the task when the execution is finished;
and step S4, reading the data source, and storing the request data, the response result, the response time and the response code according to the data structure of the json structure.
The data structure of the json structure comprises three substructures request, response and other; the request is used for storing data from a data source, the response is used for storing result data returned after the request interface request data is requested, and the other is used for storing some characteristic information in the interface request process, wherein the characteristic information comprises response time and response codes.
Step 11, creating a script name; for example, the name [ dialog system execution ] script;
step 12, configuring the request mode of the script, namely configuring the request type and the request address; only the request type [ e.g. GET, POST ] and the address of the request [ e.g.: https://127.0.0.1/v0.1/chat ]. Of course, the present invention also provides a module for attempting, the user may enter a temporary data structure for attempting to request the results. For example: request _ body = { "qustion": how much "1 plus 1, etc.", "answer": 2}, the body is transmitted to the corresponding configuration service interface, and the obtained response content (for example, response content is response _ body = { "answer": 2 }) is output on the interface. Of course, if a certain interface requires header authorization, an extensible custom header field is still provided for incoming. For example: a header: "get _ token";
step 13, configuring the data source of each interface, and loading the data source needing to execute the request; configuring database table information corresponding to the data source, and finally configuring a corresponding database query code (for example, select query, answer from table), wherein when the script is executed, the data to be tested can be known;
step 14, after configuring the data source, configuring the condition of report output, namely configuring the report output condition, wherein the configuration is divided into two parts, one part is the configuration condition, and the other part is the configuration display, so that the user can input the concerned content in a self-defined manner, but not output all executed data and return information; similarly, the dialog interface is used as an example, and information that the answer actually returned is inconsistent with the expectation is required to be output. Then [ condition ] is configured as request. answer = = response.answer, and [ shown ] is configured as request. At the same time, the default herein calculates the average time avetime of all requests and the ratio of different status codes status (e.g., 200,404,500, etc.).
And step 15, configuring an available memory and a CPU (Central processing Unit) value required by script execution for environment monitoring and judgment before the script execution.
Step 21, when executing any script, first obtaining the required memory and CPU value configured by the current script;
step 22, acquiring the actual occupation condition of one memory and CPU of the service;
step 23, obtaining whether the actual memory meets the required memory of the script according to an actual memory occupation condition of the service, if not, waiting for a set time, such as 10 seconds, and modifying the state of the script execution task into waiting;
step 24, under the condition that the actual memory is satisfied, acquiring whether the serving CPU satisfies the CPU required by the script, and if not, waiting for the set time, and recalculating the current state;
and step 25, under the condition that the CPU and the actual memory of the current service are met, modifying the state of the script execution task to be in execution, and entering step 3.
The step S3 further includes: step 31, the first stage of script execution is to obtain the configured data source and obtain the total data volume DateCount under the data source;
step 32, obtaining expected completion time expecteTime, wherein a main program of the service during script execution can initiate a pre-judging request for obtaining response time ResponseTime of the current interface;
step 33, the main program calculates how many concurrent threads ThreadCount, ThreadCount = DateCount ResponseTime/ExpectTime (the calculation result is one by one at the end) are needed to complete the execution task within the ExpectTime; when the calculated number of concurrent threads ThreadCount is greater than the upper line threshold ThreadLimit, ThreadCount = ThreadLimit, and at the same time, the actual estimated time expectedTimeLine is given, expectedTimeLine = DateCount @ ResponseTime/ThreadLimit;
step 34, dividing the data source, and respectively allocating a data source Date [ (i-1) × limit, i × limit-1], limit = DateCount/ThreadCount +1 to the corresponding thread, wherein i refers to the second thread; for example, the 1 st thread allocates data as a data source Date [0, limit-1], and the same goes by analogy, the 2 nd thread allocates data as a data source Date [ limit, limit × 2-1 ];
and step 35, distributing the data source, executing the interface request by the multiple threads respectively, monitoring the thread execution condition by the main program, changing the script execution state to be completed when all the threads are executed, and pushing information.
The step S4 further includes: step 41, circularly initiating an interface request according to a data list sent by a main program of the service;
step 42, after each interface request is initiated, the main program assigns the requested data request _ body and the response result _ body to the variable data, request _ body and variable data, response _ body, respectively, and assigns the response time and the response code status to the variable data, other, time and variable data, other, status;
step 43, after all data requests are completed, marks the end of the thread.
Referring to fig. 2 to fig. 7, a system for implementing interface data acceleration request according to the present invention includes a script configuration module, an environment monitoring module, a script execution module, and an interface request module;
the script configuration module is used for configuring the data source of each interface, configuring a script name, configuring the script request mode, configuring the memory and CPU requirements and configuring the report display mode;
the environment monitoring module is used for monitoring the script execution environment of the current service when any script is executed, carrying out delay operation when the service task content is more, and entering the script execution module when the environment allows;
the script execution module is used for adjusting the optimal number of concurrent threads according to the expected time of a user, dividing data of a data source of an interface into each thread for execution, monitoring an execution state, and pushing a task when the execution is finished;
and the interface request module is used for reading a data source and storing request data, response results, response time and response codes according to a data structure of the json structure.
The data structure of the json structure comprises three substructures request, response and other; the request is used for storing data from a data source, the response is used for storing result data returned after the request interface request data is requested, and the other is used for storing some characteristic information in the interface request process, wherein the characteristic information comprises response time and response codes.
Wherein the script configuration module further comprises: the method comprises the following steps of creating a script unit, a configuration script request unit, a configuration interface data source unit, a configuration report output unit and a configuration memory and CPU unit;
the script creating unit is used for creating a script name;
the configuration script request unit is used for configuring the request mode of the script, namely the type of the request and the address of the request are required to be configured;
the configuration interface data source unit is used for configuring the data source of each interface and loading the data source needing to execute the request; the method comprises the steps that database table information corresponding to a data source is configured, a corresponding database query code is configured at last, and when a script is executed, data to be tested can be known;
the configuration report output unit is used for configuring the condition of report output after the data source is configured, namely the configuration of the report output condition is carried out for enabling a user to input concerned content in a self-defined manner instead of outputting all executed data and return information, and the configuration is divided into two parts, wherein one part is the configuration condition and the other part is the configuration display;
the configuration memory and CPU unit is used for configuring an available memory and a CPU value required by script execution and used for environment monitoring and judgment before the script execution.
Additionally, the environmental monitoring module further comprises: the system comprises an acquisition unit, an acquisition service execution unit, a memory judgment unit, a CPU judgment unit and a state modification unit;
the acquiring unit is used for acquiring the required memory and CPU value configured by the current script when any script is executed;
the service acquisition execution unit is used for acquiring the actual occupation condition of a memory and a CPU of the service;
the memory judgment unit is used for obtaining whether the actual memory meets the required memory of the script according to an actual memory occupation condition of the service, if not, waiting for a set time, and modifying the state of the script execution task into waiting;
the CPU judgment unit is used for acquiring whether the serving CPU meets the CPU required by the script under the condition that the actual memory meets the requirement, and if the serving CPU does not meet the requirement, the CPU judgment unit waits for the set time and recalculates the current state;
and the state modification unit is used for modifying the state of the script execution task into the execution state under the condition that the CPU and the actual memory of the current service are both satisfied, and entering the script execution module.
The script execution module further comprises: the device comprises a data quantity acquisition unit, a pre-judgment unit, a concurrent thread calculation unit, a data division unit and a monitoring unit;
the data volume obtaining unit is used for obtaining the configured data source in the first stage of script execution to obtain the total data volume DateCount under the data source;
the pre-judging unit is used for acquiring expected finished time expecteTime, and a main program served during the script execution period can initiate a pre-judging request for acquiring response time ResponseTime of the current interface;
the concurrent thread calculating unit is used for calculating how many concurrent threads are needed to complete the execution task within the ExpectTime, and the concurrent threads are ThreadCount = DateCount × ResponseTime/expectTime; when the calculated number of concurrent threads ThreadCount is greater than the upper line threshold ThreadLimit, ThreadCount = ThreadLimit, and at the same time, the actual estimated time expectedTimeLine is given, expectedTimeLine = DateCount @ ResponseTime/ThreadLimit;
the data source partitioning unit is used for partitioning a data source, corresponding threads are respectively allocated with a data source Date [ (i-1) × limit, i × limit-1], limit = DateCount/ThreadCount +1, and i refers to the number of threads;
the monitoring unit is used for distributing data sources, executing interface requests by a plurality of threads respectively, monitoring the execution condition of the threads by the main program, changing the execution state of the script into the completed state when all the threads are executed, and pushing information.
The interface request module further comprises: the system comprises an initiating request unit, an assignment unit and a mark unit;
the initiating request unit is used for circularly initiating an interface request according to a data list sent by a main program of the service;
the assignment unit is configured to assign a request _ body of the requested data and a response _ body of the response to a variable data, the request _ body and the variable data, the response _ body, and assign a response time and a response code state to the variable data, other, time and other, states, respectively, after each interface request is initiated;
and the marking unit is used for marking the end of the thread after all data requests are completed.
The invention is further illustrated below with reference to a specific embodiment: scene one:
the student B is responsible for testing an intelligent dialogue system, the dialogue system operates and maintains millions of linguistic data so far, the student B divides the linguistic data into a plurality of parts in order to improve the execution speed of the linguistic data, and executes the linguistic data on a server through a plurality of scripts, hopes that a result feedback can be obtained within a set time, but because the current CPU and memory occupation condition is not considered, and meanwhile, too many scripts are started to cause the CPU to be full, the service is automatically shut down. The retry process is unaware that the progress of the test verification has been affected, and at the same time, other application processes on the service are affected.
According to the scheme of the patent, when the classmate B uses the service to execute the script, the system detects a resource occupation condition of the current service, performs a task delay of the script, enters the script execution after waiting for several minutes, adjusts the optimal number of concurrent threads according to the expected time of a user, divides the data of a data source into each thread to execute, monitors the execution state, and pushes the task when the execution is completed; the whole progress situation is basically in one grasp of small B. And after the test verification is finished, reading the data source, and storing the request data, the response result, the response time and the response code according to the data structure of the json structure.
Scene two:
as the student C is in charge of an FAQ system (namely an FAQ information retrieval system), the work needs frequent verification to judge whether the actual content returned by the interface is the same as expected, the work content is quite boring, so the student C learns the computer programming language python, and hopefully, the automatic verification can be completed in a script automation mode. As small C wishes, small C validates faq the system through automated scripts written in its own efforts. But as faq systems become more and more populated, the small C in turn begins to optimize the speed of script execution and corresponding report generation, etc.
According to the scheme of the patent, the classmate C only needs to configure faq request interface addresses, import commonly maintained use cases into a database, configure a condition and display of data source information and report display on a platform, adjust the optimal number of concurrent threads according to expected time of a user, divide data of a data source into each thread for execution, monitor an execution state, and push tasks when execution is completed; an efficient verification can be achieved.
The above description is only a preferred embodiment of the present invention, and all equivalent changes and modifications made in accordance with the claims of the present invention should be covered by the present invention.
Claims (12)
1. A method for realizing acceleration of interface data request is characterized in that: the method comprises the following steps:
step S1, configuring a script name, and configuring the script request mode, i.e. the type of the request interface and the address of the request interface need to be configured; configuring a data source of each interface and configuring a report display mode; configuring a memory and CPU requirement required by script execution;
step S2, when executing any script, monitoring the script execution environment of the current service, when the service task content is more, performing time delay operation, and when the environment allows, entering step S3;
step S3, adjusting the optimal number of concurrent threads according to the expected time of the user, dividing the data of the data source of the interface into each thread for execution, monitoring the execution state, and pushing the task when the execution is finished;
and step S4, reading the data source, and storing the request data, the response result, the response time and the response code according to the data structure of the json structure.
2. The method of claim 1, wherein the method further comprises: the data structure of the json structure comprises three substructures request, response and other; the request is used for storing data from a data source, the response is used for storing result data returned after the request interface request data is requested, and the other is used for storing some characteristic information in the interface request process, wherein the characteristic information comprises response time and response codes.
3. The method of claim 1, wherein the method further comprises: the step S1 further includes: step 11, creating a script name;
step 12, configuring the request mode of the script, namely the type of the request interface and the address of the request interface are required to be configured;
step 13, configuring the data source of each interface, and loading the data source needing to execute the request; the method comprises the steps that database table information corresponding to a data source is configured, a corresponding database query code is configured at last, and when a script is executed, data to be tested can be known;
step 14, after configuring the data source, configuring the condition of report output, namely configuring the report output condition, wherein the configuration is divided into two parts, one part is the configuration condition, and the other part is the configuration display, so that the user can input the concerned content in a self-defined manner, but not output all executed data and return information;
and step 15, configuring an available memory and a CPU (Central processing Unit) value required by script execution for environment monitoring and judgment before the script execution.
4. The method of claim 1, wherein the method further comprises: the step S2 further includes: step 21, when any script is executed, acquiring a required memory and a CPU value configured by a current script;
step 22, acquiring the actual occupation condition of one memory and CPU of the service;
step 23, obtaining whether the actual memory meets the required memory of the script according to an actual memory occupation condition of the service, if not, waiting for a set time, and modifying the state of the script execution task into waiting;
step 24, under the condition that the actual memory is satisfied, acquiring whether the serving CPU satisfies the CPU required by the script, and if not, waiting for the set time, and recalculating the current state;
and step 25, under the condition that the CPU and the actual memory of the current service are met, modifying the state of the script execution task to be in execution, and entering step 3.
5. The method of claim 1, wherein the method further comprises: the step S3 further includes: step 31, the first stage of script execution is to obtain the configured data source and obtain the total data volume DateCount under the data source;
step 32, obtaining expected completion time expecteTime, wherein a main program of the service during script execution can initiate a pre-judging request for obtaining response time ResponseTime of the current interface;
step 33, the main program calculates how many concurrent threads ThreadCount are needed if the execution task needs to be completed within the ExpectTime, ThreadCount = DateCount ResponseTime/ExpectTime; when the calculated number of concurrent threads ThreadCount is greater than the upper line threshold ThreadLimit, ThreadCount = ThreadLimit, and at the same time, the actual estimated time expectedTimeLine is given, expectedTimeLine = DateCount @ ResponseTime/ThreadLimit;
step 34, dividing the data source, and respectively allocating a data source Date [ (i-1) × limit, i × limit-1], limit = DateCount/ThreadCount +1 to the corresponding thread, wherein i refers to the second thread;
and step 35, distributing the data source, executing the interface request by the multiple threads respectively, monitoring the thread execution condition by the main program, changing the script execution state to be completed when all the threads are executed, and pushing information.
6. The method of claim 1, wherein the method further comprises: the step S4 further includes: step 41, circularly initiating an interface request according to a data list sent by a main program of the service;
step 42, after each interface request is initiated, the main program assigns the requested data request _ body and the response result _ body to the variable data, request _ body and variable data, response _ body, respectively, and assigns the response time and the response code status to the variable data, other, time and variable data, other, status;
step 43, after all data requests are completed, marks the end of the thread.
7. A system for implementing accelerated interface data requests, comprising: the system comprises a script configuration module, an environment monitoring module, a script execution module and an interface request module;
the script configuration module is used for configuring the data source of each interface, configuring a script name and configuring the script request mode, namely the type of the request interface and the address of the request interface are required to be configured; configuring a memory and CPU requirement required by script execution, and configuring a report display mode;
the environment monitoring module is used for monitoring the script execution environment of the current service when any script is executed, carrying out delay operation when the service task content is more, and entering the script execution module when the environment allows;
the script execution module is used for adjusting the optimal number of concurrent threads according to the expected time of a user, dividing data of a data source of an interface into each thread for execution, monitoring an execution state, and pushing a task when the execution is finished;
and the interface request module is used for reading a data source and storing request data, response results, response time and response codes according to a data structure of the json structure.
8. The system of claim 7, wherein the acceleration request is from a mobile device to a mobile device, and wherein the acceleration request comprises: the data structure of the json structure comprises three substructures request, response and other; the request is used for storing data from a data source, the response is used for storing result data returned after the request interface request data is requested, and the other is used for storing some characteristic information in the interface request process, wherein the characteristic information comprises response time and response codes.
9. The system of claim 7, wherein the acceleration request is from a mobile device to a mobile device, and wherein the acceleration request comprises: the script configuration module further comprises: the method comprises the following steps of creating a script unit, a configuration script request unit, a configuration interface data source unit, a configuration report output unit and a configuration memory and CPU unit;
the script creating unit is used for creating a script name;
the configuration script request unit is used for configuring the request mode of the script, namely the type of the request interface and the address of the request interface are required to be configured;
the configuration interface data source unit is used for configuring the data source of each interface and loading the data source needing to execute the request; the method comprises the steps that database table information corresponding to a data source is configured, a corresponding database query code is configured at last, and when a script is executed, data to be tested can be known;
the configuration report output unit is used for configuring the condition of report output after the data source is configured, namely the configuration of the report output condition is carried out for enabling a user to input concerned content in a self-defined manner instead of outputting all executed data and return information, and the configuration is divided into two parts, wherein one part is the configuration condition and the other part is the configuration display;
the configuration memory and CPU unit is used for configuring an available memory and a CPU value required by script execution and used for environment monitoring and judgment before the script execution.
10. The system of claim 7, wherein the acceleration request is from a mobile device to a mobile device, and wherein the acceleration request comprises: the environmental monitoring module further includes: the system comprises an acquisition unit, an acquisition service execution unit, a memory judgment unit, a CPU judgment unit and a state modification unit;
the acquiring unit is used for acquiring the required memory and CPU value configured by the current script when any script is executed;
the service acquisition execution unit is used for acquiring the actual occupation condition of a memory and a CPU of the service;
the memory judgment unit is used for obtaining whether the actual memory meets the required memory of the script according to an actual memory occupation condition of the service, if not, waiting for a set time, and modifying the state of the script execution task into waiting;
the CPU judgment unit is used for acquiring whether the serving CPU meets the CPU required by the script under the condition that the actual memory meets the requirement, and if the serving CPU does not meet the requirement, the CPU judgment unit waits for the set time and recalculates the current state;
and the state modification unit is used for modifying the state of the script execution task into the execution state under the condition that the CPU and the actual memory of the current service are both satisfied, and entering the script execution module.
11. The system of claim 7, wherein the acceleration request is from a mobile device to a mobile device, and wherein the acceleration request comprises: the script execution module further comprises: the device comprises a data quantity acquisition unit, a pre-judgment unit, a concurrent thread calculation unit, a data division unit and a monitoring unit;
the data volume obtaining unit is used for obtaining the configured data source in the first stage of script execution to obtain the total data volume DateCount under the data source;
the pre-judging unit is used for acquiring expected finished time expecteTime, and a main program served during the script execution period can initiate a pre-judging request for acquiring response time ResponseTime of the current interface;
the concurrent thread calculating unit is used for calculating how many concurrent threads are needed to complete the execution task within the ExpectTime, and the concurrent threads are ThreadCount = DateCount × ResponseTime/expectTime; when the calculated number of concurrent threads ThreadCount is greater than the upper line threshold ThreadLimit, ThreadCount = ThreadLimit, and at the same time, the actual estimated time expectedTimeLine is given, expectedTimeLine = DateCount @ ResponseTime/ThreadLimit;
the data source partitioning unit is used for partitioning a data source, corresponding threads are respectively allocated with a data source Date [ (i-1) × limit, i × limit-1], limit = DateCount/ThreadCount +1, and i refers to the number of threads;
the monitoring unit is used for distributing data sources, executing interface requests by a plurality of threads respectively, monitoring the execution condition of the threads by the main program, changing the execution state of the script into the completed state when all the threads are executed, and pushing information.
12. The system of claim 7, wherein the acceleration request is from a mobile device to a mobile device, and wherein the acceleration request comprises: the interface request module further comprises: the system comprises an initiating request unit, an assignment unit and a mark unit;
the initiating request unit is used for circularly initiating an interface request according to a data list sent by a main program of the service;
the assignment unit is configured to assign a request _ body of the requested data and a response _ body of the response to a variable data, the request _ body and the variable data, the response _ body, and assign a response time and a response code state to the variable data, other, time and other, states, respectively, after each interface request is initiated;
and the marking unit is used for marking the end of the thread after all data requests are completed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910964034.5A CN110727615B (en) | 2019-10-11 | 2019-10-11 | Method and system for realizing acceleration of interface data request |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910964034.5A CN110727615B (en) | 2019-10-11 | 2019-10-11 | Method and system for realizing acceleration of interface data request |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110727615A CN110727615A (en) | 2020-01-24 |
| CN110727615B true CN110727615B (en) | 2020-12-25 |
Family
ID=69221074
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910964034.5A Active CN110727615B (en) | 2019-10-11 | 2019-10-11 | Method and system for realizing acceleration of interface data request |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110727615B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112214400A (en) * | 2020-09-17 | 2021-01-12 | 浙江数链科技有限公司 | Automatic testing method and device, electronic equipment and readable storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101394334A (en) * | 2007-09-19 | 2009-03-25 | 北京中电华大电子设计有限责任公司 | Data packet scheduling transmitting device used for WLAN |
| CN105338086A (en) * | 2015-11-04 | 2016-02-17 | 浪潮软件股份有限公司 | Distributed message forwarding method |
| CN106227782A (en) * | 2016-07-15 | 2016-12-14 | 广东亿迅科技有限公司 | A kind of method being inserted into data base based on multi-data source |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102790683A (en) * | 2011-05-16 | 2012-11-21 | 北大方正集团有限公司 | Real-time market data processing method, server and system |
| US8799988B2 (en) * | 2012-01-25 | 2014-08-05 | Microsoft Corporation | Document communication runtime interfaces |
| CN103973594B (en) * | 2014-05-09 | 2017-07-28 | 清华大学 | Method for scheduling task based on multiple resource |
| CN105677486B (en) * | 2016-01-08 | 2019-03-22 | 上海交通大学 | Data parallel processing method and system |
-
2019
- 2019-10-11 CN CN201910964034.5A patent/CN110727615B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101394334A (en) * | 2007-09-19 | 2009-03-25 | 北京中电华大电子设计有限责任公司 | Data packet scheduling transmitting device used for WLAN |
| CN105338086A (en) * | 2015-11-04 | 2016-02-17 | 浪潮软件股份有限公司 | Distributed message forwarding method |
| CN106227782A (en) * | 2016-07-15 | 2016-12-14 | 广东亿迅科技有限公司 | A kind of method being inserted into data base based on multi-data source |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110727615A (en) | 2020-01-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102868573B (en) | Method and device for Web service load cloud test | |
| CN110362409A (en) | Based on a plurality of types of resource allocation methods, device, equipment and storage medium | |
| CN114546738B (en) | Universal test method, system, terminal and storage medium for server | |
| CN111625331A (en) | Task scheduling method, device, platform, server and storage medium | |
| CN111143143B (en) | Performance test method and device | |
| CN112286806A (en) | Automatic testing method and device, storage medium and electronic equipment | |
| CN111831191A (en) | Workflow configuration method and device, computer equipment and storage medium | |
| US11169910B2 (en) | Probabilistic software testing via dynamic graphs | |
| CN112052082B (en) | Task attribute optimization method, device, server and storage medium | |
| CN111104304A (en) | Multi-task scene performance testing method, storage medium, electronic device and system | |
| CN112988185A (en) | Cloud application updating method, device and system, electronic equipment and storage medium | |
| CN114818565A (en) | Simulation environment management platform, method, equipment and medium based on python | |
| CN113127356A (en) | Pressure measurement method and device, electronic equipment and storage medium | |
| CN113157411A (en) | Reliable configurable task system and device based on Celery | |
| CN110727615B (en) | Method and system for realizing acceleration of interface data request | |
| CN110716960B (en) | Database query method, device, equipment and storage medium | |
| CN110750453B (en) | HTML 5-based intelligent mobile terminal testing method, system, server and storage medium | |
| CN111813758A (en) | Distributed analysis method and device for database files, server and storage medium | |
| CN111294377A (en) | Network request sending method of dependency relationship, terminal device and storage medium | |
| CN112363914A (en) | Parallel test resource configuration optimization method, computing device and storage medium | |
| CN107885593B (en) | User authentication method and device | |
| CN115391204A (en) | Test method and device for automatic driving service, electronic equipment and storage medium | |
| CN114185656A (en) | Test task processing method, device, equipment and storage medium | |
| CN114297042A (en) | Interface pressure performance testing method, device, equipment and storage medium | |
| CN112346979A (en) | Software performance testing method, system and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |