CN116383021A

CN116383021A - Software package performance testing method, system, computing device and readable storage medium

Info

Publication number: CN116383021A
Application number: CN202310205864.6A
Authority: CN
Inventors: 郑威; 郭斌; 汪晓刚
Original assignee: Uniontech Software Technology Co Ltd
Current assignee: Uniontech Software Technology Co Ltd
Priority date: 2023-03-03
Filing date: 2023-03-03
Publication date: 2023-07-04

Abstract

The invention discloses a method, a system, a computing device and a readable storage medium for testing software package performance, wherein the method comprises the following steps: when a software package monitoring service running on the test end monitors that the software package to be tested is delivered, installing the software package to be tested; judging whether the binary flag bit of the software package to be tested is set or not; if the binary flag bit is not set, configuring the test environment and service of the reference test program, and stopping the software package monitoring service; testing the software package to be tested through a reference test program; comparing the first test data with baseline data corresponding to the software package to be tested; if the first test data is degraded compared with the baseline data, the part to be retested in the baseline test program recorded to be degraded is sent to the server side, so that the server side carries out binary construction on the software package to be tested, and the software package to be tested after binary construction is sent to the test side for continuous test. The scheme of the invention improves the testing efficiency of the software package.

Description

Software package performance testing method, system, computing device and readable storage medium

Technical Field

The present invention relates to the field of computer technologies, and in particular, to a method, a system, a computing device, and a readable storage medium for testing performance of a software package.

Background

At present, performance test is taken as one of evaluation points existing in cores in core test, and is closely focused by developers and vast users, and for cores, bug repair, novel drive integration, drive update, memory scheduling management policy change, security policy, characteristic modification, core function integration and the like can cause risks to core performance. At present, problems introduced by risk submission points are analyzed, and all the submission points can be eliminated one by adopting a dichotomy. However, the existing kernel test scheme needs to manually retest and analyze data by a test engineer, and has the problems of long period of locating risk lifting points, high labor cost and the like. Therefore, a solution for testing the performance of a software package is needed to solve the problems in the prior art.

Disclosure of Invention

To this end, the present invention provides a method, system, computing device and readable storage medium for testing the performance of a software package to solve or at least alleviate the above-identified problems.

According to a first aspect of the present invention, there is provided a method for testing performance of a software package, executed in a testing terminal, the method comprising: when a software package monitoring service running on the test end monitors that a software package to be tested is sent, installing the software package to be tested; judging whether the binary flag bit of the software package to be tested is set or not; if the binary flag bit is not set, configuring the test environment and service of the reference test program, and stopping the software package monitoring service; testing the software package to be tested through the reference test program to obtain first test data; comparing the first test data with baseline data corresponding to the software package to be tested; determining whether the first test data is degraded compared to the baseline data; if the first test data is degraded compared with the baseline data, recording a part to be retested in a reference test program with degradation, sending the part to be retested to a server side, setting a binary flag bit of the software package to be tested so that the server side carries out binary construction on the software package to be tested, and transmitting the binary constructed software package to be tested to a test side for continuous test; if the first test data is not degraded compared with the baseline data, the test is completed and the software package listening service is started.

Optionally, in the method for testing the performance of the software package according to the present invention, the method further includes: if the binary flag bit is set, configuring the test environment and service of the part to be retested in the reference test program; testing the software package to be tested through the part to be retested in the reference test program to obtain second test data; comparing the second test data with the baseline data corresponding to the software package to be tested; determining whether the second test data is degraded compared to the baseline data; if the second test data is degraded compared with the baseline data, judging whether the local commit point record file contains content or not; if the local submitting point record file has content, prompting that the software package to be tested has problems, clearing the binary flag bit of the software package to be tested and the local submitting point record file, starting the software package monitoring service, otherwise, sending a binary construction request in the current direction to a server side; and if the second test data is not degraded compared with the baseline data, clearing the local commit point record file, and sending a halving construction request in the opposite direction to a server side.

Optionally, in the method for testing the performance of a software package according to the present invention, comparing the first test data with baseline data corresponding to the software package to be tested includes: and comparing the first test data with the baseline data corresponding to the version number of the software package to be tested.

Optionally, in the method for testing the performance of a software package according to the present invention, comparing the second test data with baseline data corresponding to the software package to be tested includes: and comparing the second test data with the baseline data corresponding to the version number of the software package to be tested.

Optionally, in the method for testing the performance of a software package according to the present invention, the testing end is connected to a database, and the baseline data is stored in the database, and the method further includes: and writing the first test data into a database.

Optionally, in the method for testing the performance of a software package according to the present invention, the testing end is connected to a database, and the baseline data is stored in the database, and the method further includes: and writing the second test data into a database.

Optionally, in the method for testing the performance of the software package according to the present invention, the method further includes: and if the installation fails, prompting that the installation of the software package to be tested fails.

According to a second aspect of the present invention, there is provided a method for testing performance of a software package, executed at a server side, the method comprising: constructing and compiling codes submitted in a preset time period and containing a plurality of submitted points, and sending the compiled software package to be tested to a testing end, so that the testing end executes a testing method for testing the performance of the software package to be tested; when a binary construction request is received, all the extraction points constructed in the previous time are obtained; performing binary construction according to the request direction and all the submitting points to obtain the submitting point of the construction and the software package to be tested of the construction; judging whether the number of the submitting points constructed at the time is one; if the number of the submitting points constructed at this time is one, sending the submitting points constructed at this time and the software package to be tested constructed at this time to a testing end so that the testing end writes the submitting points constructed at this time into a local submitting point record file; if the number of the submitting points constructed at the time is not one, sending the software package to be tested constructed at the time to the testing end.

According to a second aspect of the present invention, there is provided a system for testing the performance of a software package, comprising: the test terminal is suitable for installing the software package to be tested when the software package monitoring service running on the test terminal monitors that the software package to be tested is delivered, judging whether the two-part flag bit of the software package to be tested is set, if the two-part flag bit is not set, configuring the test environment and service of the reference test program, stopping the software package monitoring service, testing the software package to be tested through the reference test program to obtain first test data, comparing the first test data with baseline data corresponding to the software package to be tested, judging whether the first test data is degraded with the baseline data, if the first test data is degraded with the baseline data, recording a part to be retested in the reference test program with degradation, sending the part to the server terminal, setting the two-part flag bit of the software package to be tested, if the first test data is not degraded with the baseline data, completing the test, and starting the software package monitoring service. The server side is suitable for constructing and compiling codes which are submitted in a preset time period and contain a plurality of point of submission and sending the compiled software package to be tested to the testing side, when a binary construction request is received, all the point of submission constructed in the previous time are obtained, construction is carried out according to the request direction and the binary points of all the point of submission, the point of submission constructed in this time and the software package to be tested constructed in this time are obtained, whether the number of the point of submission constructed in this time is one or not is judged, if yes, the point of submission constructed in this time and the software package to be tested constructed in this time are sent to the testing side, so that the testing side writes the point of submission constructed in the local point of submission record file, and otherwise, the software package to be tested constructed in this time is sent to the testing side.

Optionally, in the system for testing the performance of a software package according to the present invention, the testing end is further adapted to: when the binary flag bit is set, configuring a test environment and service of a part to be retested in a reference test program, testing the software package to be tested through the part to be retested in the reference test program to obtain second test data, comparing the second test data with baseline data corresponding to the software package to be tested, judging whether the second test data is degraded compared with the baseline data, judging whether contents exist in a local submitting point record file if the second test data is degraded compared with the baseline data, prompting that the software package to be tested has problems if the contents exist in the local submitting point record file, clearing the binary flag bit and the local submitting point record file of the software package to be tested, starting the software package monitoring service, otherwise, sending a binary construction request in the current direction to a server, clearing the local point record file if the second test data is not degraded compared with the submitted baseline data, and sending a binary construction request in the opposite direction to the server.

Optionally, in the test system for software package performance according to the present invention, the test system further includes: and the database is suitable for storing the first test data and the second test data.

According to a third aspect of the present invention, there is provided a system for testing the performance of a software package, comprising: the construction management server is suitable for constructing codes which are submitted in a preset time period and contain a plurality of point of submission, when a binary construction request is received, all the point of submission constructed in the previous time are obtained, construction is carried out according to the request direction and the binary points of all the point of submission, the point of submission constructed at this time and a software package to be tested constructed at this time are obtained, whether the number of the point of submission constructed at this time is one or not is judged, if so, the point of submission constructed at this time and the software package to be tested constructed at this time are sent to the testing end, so that the testing end writes the point of submission constructed at this time into a local point of submission record file, otherwise, the software package to be tested constructed at this time is sent to the testing end; the software package compiling server is suitable for compiling codes submitted in a preset time period and comprising a plurality of point of submission and sending the compiled software package to be tested to the testing end; the device to be tested is suitable for installing the software package to be tested when the software package monitoring service running on the test end monitors that the software package to be tested is delivered, judging whether the two-part flag bit of the software package to be tested is set, if the two-part flag bit is not set, configuring the test environment and service of the reference test program, stopping the software package monitoring service, testing the software package to be tested through the reference test program to obtain first test data, comparing the first test data with baseline data corresponding to the software package to be tested, judging whether the first test data is degraded with the baseline data, if the first test data is degraded with the baseline data, recording a part to be retested in the reference test program with degradation, sending the part to be retested to the server end, setting the two-part flag bit of the software package to be tested, if the first test data is not degraded with the baseline data, completing the test, and starting the software package monitoring service. And the database is suitable for storing the first test data.

Optionally, in the system for testing the performance of a software package according to the present invention, the device to be tested is further adapted to configure a test environment and a service of a portion to be retested in a benchmark test program when a binary flag bit is set, test the software package to be tested through the portion to be retested in the benchmark test program to obtain second test data, compare the second test data with baseline data corresponding to the software package to be tested, determine whether degradation occurs in the second test data compared with the baseline data, if degradation occurs in the second test data compared with the baseline data, determine whether content exists in a local commit point record file, if content exists in the local commit point record file, prompt that the software package to be tested has a problem, clear the binary flag bit and the local commit point record file of the software package to be tested, and start the software package monitoring service, and if no degradation occurs in the second test data compared with the baseline data, clear the local commit point record file, and otherwise send a binary request in a reverse direction to the server; the database is further adapted to store second test data.

Optionally, in the test system for software package performance according to the present invention, the test system further includes: and the user management device is suitable for presenting the first test data and the second test data.

According to a fourth aspect of the present invention there is provided a computing device comprising: at least one processor; a memory storing program instructions, wherein the program instructions are configured to be executed by the at least one processor, the program instructions comprising instructions for performing the method as described above.

According to a fifth aspect of the present invention there is provided a readable storage medium storing program instructions which, when read and executed by a computing device, cause the computing device to perform the method as described above.

According to the technical scheme, whether the corresponding test environment and service are set and configured and the more targeted test is operated through whether the binary flag bit of the software package to be tested is set or not, whether the test data is degraded compared with the baseline data or not is judged, and the part needing retest is determined, so that the server carries out binary construction on the code according to the condition of whether the degradation occurs, the software package is continuously tested by the test end until the test is completed, the automatic software package test diagnosis is realized, the positioning period and the labor input cost are greatly shortened, and the test efficiency of the software package performance is improved. The method is applied to a kernel test scene, and can realize completely unattended operation.

The foregoing description is only an overview of the present invention, and is intended to be implemented in accordance with the teachings of the present invention in order that the same may be more clearly understood and to make the same and other objects, features and advantages of the present invention more readily apparent.

Drawings

To the accomplishment of the foregoing and related ends, certain illustrative aspects are described herein in connection with the following description and the annexed drawings, which set forth the various ways in which the principles disclosed herein may be practiced, and all aspects and equivalents thereof are intended to fall within the scope of the claimed subject matter. The above, as well as additional objects, features, and advantages of the present disclosure will become more apparent from the following detailed description when read in conjunction with the accompanying drawings. Like reference numerals generally refer to like parts or elements throughout the present disclosure.

FIG. 1 illustrates a block diagram of the physical components (i.e., hardware) of a computing device 100;

FIG. 2 illustrates a flow diagram of a method 200 of testing the performance of a software package according to one embodiment of the invention;

FIG. 3 is a flow chart of a method 300 for testing the performance of a software package according to another embodiment of the invention;

FIG. 4 is a flow chart of a method 400 for testing the performance of a software package according to yet another embodiment of the invention;

FIG. 5 shows a flow diagram of a method 500 for testing the performance of a software package according to yet another embodiment of the invention;

FIG. 6 shows a schematic diagram of a test system 600 for software package performance according to one embodiment of the invention;

FIG. 7 shows a schematic diagram of a test system 700 for software package performance according to one embodiment of the invention;

FIG. 8 is a flow chart of a method 800 for testing the performance of a seed package according to yet another embodiment of the invention;

fig. 9 is a schematic diagram illustrating a process flow of a software package listening service of a device under test according to one embodiment of the present invention.

Detailed Description

Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

Fig. 1 illustrates a block diagram of the physical components (i.e., hardware) of a computing device 100. In a basic configuration, computing device 100 includes at least one processing unit 102 and system memory 104. According to one aspect, the processing unit 102 may be implemented as a processor, depending on the configuration and type of computing device. The system memory 104 includes, but is not limited to, volatile storage (e.g., random access memory), non-volatile storage (e.g., read only memory), flash memory, or any combination of such memories. According to one aspect, the system memory 104 includes an operating system 105 and program modules 106, and the program modules 106 include the test system 600 or 700 of the software package capabilities of the present invention.

According to one aspect, operating system 105 is suitable, for example, for controlling the operation of computing device 100. Further, examples are practiced in connection with a graphics library, other operating systems, or any other application program and are not limited to any particular application or system. This basic configuration is illustrated in fig. 1 by those components within dashed line 108. According to one aspect, computing device 100 has additional features or functionality. For example, according to one aspect, computing device 100 includes additional data storage devices (removable and/or non-removable) such as, for example, magnetic disks, optical disks, or tape. Such additional storage is illustrated in fig. 1 by removable storage device 109 and non-removable storage device 110.

As set forth hereinabove, according to one aspect, program modules 106 are stored in system memory 104. According to one aspect, program modules 106 may include one or more applications, the invention is not limited in the type of application, for example, the application may include: email and contacts applications, word processing applications, spreadsheet applications, database applications, slide show applications, drawing or computer-aided application, web browser applications, etc.

According to one aspect, the examples may be practiced in a circuit comprising discrete electronic components, a packaged or integrated electronic chip containing logic gates, a circuit utilizing a microprocessor, or on a single chip containing electronic components or a microprocessor. For example, examples may be practiced via a system on a chip (SOC) in which each or many of the components shown in fig. 1 may be integrated on a single integrated circuit. According to one aspect, such SOC devices may include one or more processing units, graphics units, communication units, system virtualization units, and various application functions, all of which are integrated (or "burned") onto a chip substrate as a single integrated circuit. When operating via an SOC, the functionality described herein may be operated via dedicated logic integrated with other components of computing device 100 on a single integrated circuit (chip). Embodiments of the invention may also be practiced using other techniques capable of performing logical operations (e.g., AND, OR, AND NOT), including but NOT limited to mechanical, optical, fluidic, AND quantum techniques. In addition, embodiments of the invention may be practiced within a general purpose computer or in any other circuit or system.

According to one aspect, the computing device 100 may also have one or more input devices 112, such as a keyboard, mouse, pen, voice input device, touch input device, and the like. Output device(s) 114 such as a display, speakers, printer, etc. may also be included. The foregoing devices are examples and other devices may also be used. Computing device 100 may include one or more communication connections 116 that allow communication with other computing devices 118. Examples of suitable communication connections 116 include, but are not limited to: RF transmitter, receiver and/or transceiver circuitry; universal Serial Bus (USB), parallel and/or serial ports.

The term computer readable media as used herein includes computer storage media. Computer storage media may include volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information (e.g., computer readable instructions, data structures, or program modules). System memory 104, removable storage 109, and non-removable storage 110 are all examples of computer storage media (i.e., memory storage). Computer storage media may include Random Access Memory (RAM), read Only Memory (ROM), electrically erasable read only memory (EEPROM), flash memory or other memory technology, CD-ROM, digital Versatile Disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other article of manufacture that can be used to store information and that can be accessed by computer device 100. According to one aspect, any such computer storage media may be part of computing device 100. Computer storage media does not include a carrier wave or other propagated data signal.

According to one aspect, communication media is embodied by computer readable instructions, data structures, program modules, or other data in a modulated data signal (e.g., carrier wave or other transport mechanism) and includes any information delivery media. According to one aspect, the term "modulated data signal" describes a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, radio Frequency (RF), infrared, and other wireless media.

In one embodiment of the invention, computing device 100 includes one or more processors and one or more readable storage media storing program instructions. The program instructions, when configured to be executed by one or more processors, cause a computing device to perform a method of testing the performance of a software package in an embodiment of the invention.

In some embodiments, the software package related to the method for testing the performance of the software package of the present invention may be a kernel package (deb-package), and the method for testing the performance of the kernel may be used for testing the performance of the kernel.

FIG. 2 shows a flow diagram of a method 200 of testing the performance of a software package, according to one embodiment of the invention, the method 200 beginning at step 210.

210. The method comprises the steps of a kernel timing construction task, automatically constructing codes submitted by a kernel in a preset time at a server side, automatically transmitting the codes to equipment to be tested (Dev i ce Under Test, DUT for short) after construction, wherein the DUT is a test carrier for testing the performance of a software package and can be used for deploying a monitoring service to execute data acquisition and diagnosis work of a daily benchmark. Here, the predetermined time may be daily, and the kernel daily submission code may be structured, or may be a period of time of every half day, every two days, every week, or the like.

220. And deploying a packet monitoring service at the test end, polling whether the monitoring server has the software packet to be sent to the test end, if so, starting to install the software packet, performing performance test configuration according to the request, and stopping the monitoring service. Here, the software package may be a kernel package.

230. And starting the performance test service, performing performance test according to the kernel performance test configuration, automatically sending test data to the database and the server after the test is completed, and restarting the packet monitoring server. The database may be used to store test data, provide presentation and review functions for the test data.

240. Starting performance test data processing comparison service, comparing test data obtained by performance test with baseline data to judge whether a degraded benchmark test item appears, and submitting a bipartite construction request to a server for bipartite construction aiming at the degraded benchmark test item.

250. And starting the bipartite construction service, and starting to enter a bipartite construction flow after the server side receives the bipartite construction request, wherein the kernel stops the construction task at fixed time.

In order to more clearly illustrate the software package performance test scheme of the present invention, the following describes the technical scheme of the present invention with reference to specific embodiments. Fig. 3 shows a flow chart of a method 300 for testing the performance of a software package according to another embodiment of the invention, and the method 300 is performed by the test end, the service end and the database. As shown in fig. 3, method 300 begins at step 301.

301. The server side constructs and compiles codes submitted in a preset time period and comprising a plurality of submitted points, and sends the compiled software package to be tested to the test side. Here, the predetermined time may be daily, and the kernel daily submission code may be structured, or may be a period of time of every half day, every two days, every week, or the like. Here, the software package may be a kernel package.

Optionally, the compiled software package to be tested is sent to a Device Under Test (DUT) at a test end, where the DUT is a test carrier for testing performance of the software package, and may be used to deploy a listening service to perform data collection and diagnosis of a daily benchmark test (benchmark).

302. And when the software package monitoring service running on the test end monitors that the software package to be tested is delivered, installing the software package to be tested.

Optionally, if the installation of the software package to be tested is successful, the subsequent flow is continued, and if the installation of the software package to be tested is failed, the installation failure of the software package to be tested is prompted, for example: and sending a software package installation failure warning mail.

303. And judging whether the binary flag bit of the software package to be tested is set or not. Step 304 is entered if the binary flag bit is not set (i.e., the flag bit is 0) and step 310 is entered if the binary flag bit is set (i.e., the flag bit is 1).

Judging whether the binary flag bit of the software package to be tested is set or not represents judging whether the software package to be tested is a binary testing package or not; and if the binary flag bit of the software package to be tested is not set, the software package to be tested is not retested.

304. And if the binary flag bit is not set, configuring the test environment and service of the benchmark test program, and stopping the software package from monitoring the service.

According to one embodiment of the invention, if the software package to be tested is not a retest package, the kernel benchmark test environment and service configuration is carried out, the package monitoring service is stopped, the package monitoring service is set to be started and not to be started automatically, and the computer is restarted for testing. Alternatively, if the software package under test is not a retest package, the configured benchmark program includes the complete test tool.

305. And testing the software package to be tested through a reference test program to obtain first test data.

According to one embodiment of the invention, the first test data is automatically written into the database for recording and storage, so that the test data can be consulted and displayed later when needed. Alternatively, the database may be a MySQL open source database, or other databases may be selected.

306. And comparing the first test data with the baseline data corresponding to the software package to be tested.

Optionally, the data comparison is performed by taking a tool included in the benchmark test program as a unit, and when a test item lower than the performance degradation standard appears, the test tool name is returned, that is, when the benchmark test is performed on the software package to be tested, since a plurality of test tools are included in the benchmark test, when a certain test tool is tested, the first test data appears degradation, the name of the test tool is returned, so that the tool can be retested later in retesting. Optionally, the name of the returned test tool is written into the configuration file of the benchmark test, so that the test environment and the service are configured by adopting the configuration file in the subsequent test, and only the test tool written in the configuration file is tested in the test.

Optionally, the baseline data are stored in a database, and the baseline data corresponding to the software package to be tested are obtained from the database.

307. It is determined whether the first test data is degraded compared to the baseline data.

Optionally, baseline data obtained by a previous test (a previous version test, for example, a previous day test) corresponding to the version number of the software package to be tested is obtained from the database, the first test data is compared with the baseline data corresponding to the version number of the software package to be tested, and whether the first test data is worse than the baseline data is judged. The version number may be the version number of the current test, for example: version number 2.3.

Optionally, baseline data obtained by a previous test (a previous version of test, for example, a previous day of test) corresponding to the same model of the to-be-tested device and the version number of the to-be-tested software package are obtained from the database, the first test data is compared with the baseline data corresponding to the version number of the to-be-tested software package, and whether the first test data is worse than the baseline data is judged.

Optionally, the first test data is written into the database so that the first test data can be subsequently compared as baseline data with the test data of the following day.

308. If the first test data is degraded compared with the baseline data, recording a part to be retested in a reference test program with degradation, sending the part to be retested to a server side, setting a binary flag bit of a software package to be tested so that the server side carries out binary construction on the software package to be tested, and transmitting the software package to be tested after binary construction to the test side for continuous test.

According to one embodiment of the invention, when a test item lower than the performance degradation standard appears, the name of the test tool (namely the part to be retested in the benchmark test program) is recorded, and the binary construction request and the name of the test tool to be retested are sent to the server side. Optionally, according to different test suites, inquiring a corresponding data table, comparing each small item in the test suite one by one, and when the performance score is compared with the degradation, recording the name of the test suite to which the degraded small item belongs, and informing the server side that a new software package to be tested needs to be built for testing.

When the server side receives the bipartite construction request, all the point of intersection constructed in the previous time are obtained, construction is carried out according to the request direction and the bipartite points of all the point of intersection, and the point of intersection constructed at the time and the software package to be tested constructed at the time are obtained. And judging whether the number of the submitting points constructed at the time is one. If the number of the submitting points constructed at this time is one, sending the submitting points constructed at this time and the software package to be tested constructed at this time to a testing end so that the testing end writes the submitting points constructed at this time into a local submitting point record file; if the number of the submitting points constructed at the time is not one, sending the software package to be tested constructed at the time to the testing end.

According to one embodiment of the invention, the step executed after degradation is implemented as a binary construction interface, the part to be retested in the reference test program recorded with degradation is sent to the server side, and the binary flag bit of the software package to be tested is set, so that the server side carries out binary construction on the software package to be tested, and the software package to be tested after binary construction is transmitted to the test side to execute logic for continuous test, and the execution logic is implemented as the binary construction interface. And if the software package is degraded, calling a bipartite construction interface to wait for the to-be-tested software package of the retest version to be sent to the test end.

309. If the first test data is not degraded compared to the baseline data, the test is completed and the package listening service is started.

According to one embodiment of the invention, the test is terminated if the test result (first test data) is not degraded and is not a retest packet. And starting the software package monitoring service to wait for the next kernel test.

310. And if the binary flag bit is set, configuring the test environment and service of the part to be retested in the benchmark test program.

According to one embodiment of the invention, if the software package to be tested is a retested package, the part to be retested can be determined according to the test tool written in the configuration file of the reference test, namely, only the test tool to be retested is tested aiming at the software package to be tested with the binary flag bit, and the test environment and the service are configured aiming at the test tool to be retested. The test of a test tool with complete reference test program is not required to be carried out on the software package to be tested with the binary flag bit, and the efficiency of kernel test is improved.

311. And testing the software package to be tested through the part to be retested in the reference test program to obtain second test data.

According to one embodiment of the invention, the second test data is automatically written into the database for data record storage so as to be convenient for subsequent reference and display of the test data when needed.

312. And comparing the second test data with the baseline data corresponding to the software package to be tested.

Optionally, the data comparison is performed by taking a tool as a unit, and when a test item lower than the performance degradation standard appears, the name of the test tool is returned, that is, when the test item is subjected to the benchmark test on the software package to be tested, because a plurality of test tools are included in the benchmark test, when a certain test tool is tested, the name of the test tool is returned, so that the tool can be retested later. Optionally, the name of the returned testing tool is written into the configuration file of the benchmark test, so that the testing environment and the service are configured by adopting the configuration file in the subsequent test, and only the tool written in the configuration file is tested in the test.

Optionally, the baseline data are stored in a database, and the baseline data corresponding to the software package to be tested are obtained from the database. 313. It is determined whether the second test data is degraded compared to the baseline data.

Optionally, the baseline data obtained by the previous test (the previous test of the previous version, for example, the previous test of the day) corresponding to the version number of the software package to be tested is obtained from the database, the second test data is compared with the baseline data corresponding to the version number of the software package to be tested, and whether the second test data is worse than the baseline data is judged. The version number may be the version number of the current test, for example: version number 2.3.

Optionally, baseline data obtained by a previous test (a previous version of test, for example, a previous day of test) corresponding to the same model of the to-be-tested device and the version number of the to-be-tested software package are obtained from the database, the second test data is compared with the baseline data corresponding to the version number of the to-be-tested software package, and whether the second test data is worse than the baseline data is judged.

Optionally, the second test data is written into the database so that the second test data can be subsequently compared as baseline data with the test data of the following day.

314. If the second test data is degraded compared with the baseline data, judging whether the local commit point record file contains contents or not.

According to an embodiment of the present invention, it may also be determined whether the number of current direction code submissions in the local submission point record file is 1, or whether the number of merge records in the current direction in the local submission point record file is 1. If the content in the local commit point record file/the number of code commit in the current direction in the local commit point record file is 1/the number of merge records in the current direction in the local commit point record file is 1, the fact that the software package to be tested has a problem is indicated, and otherwise, the problem does not exist.

Alternatively, the local commit point record file may be stored in the test end or in a database.

315. If the local submitting point record file has content, prompting that the software package to be tested has a problem, clearing the binary flag bit of the software package to be tested and the local submitting point record file, starting the software package monitoring service, otherwise executing 316, and sending a binary construction request in the current direction to the server side.

The content in the local commit point record file indicates that the software package test is finished, the binary flag bit of the software package to be tested and the local commit point record file are cleared, and the software package monitoring service is started so as to wait for the core test of the next round.

Optionally, a mail containing the problematic point of submission is pushed to alert the software package to be tested to the problem. Other means of prompting may also be employed.

If the local commit point record file has no content, the number of current direction code commit in the local commit point record file is not 1, and the number of merging records in the current direction in the local commit point record file is not 1, which indicates that the test is not finished, and the test needs to be continued in the current direction, for example: the former bisection construction divides the commit point into 1-6 and 7-12, the software package to be tested comprises the commit point of 1-6, if the local commit point records the content in the file, the bisection construction is still carried out in the current direction, namely, the commit point of 1-6 is continued to be bisected, for example, the commit point can be further divided into 1-3 and 4-6. In the case of this example, the opposite direction bipartite construction is to bipartite construct commit points 7-12.

317. If the second test data is not degraded compared with the baseline data, the local commit point record file is emptied, and a reverse binary construction request is sent to the server side.

When the server side receives the binary construction request sent by the test side, all the lifting points constructed in the previous time are acquired. And carrying out binary construction according to the request direction and all the submitting points to obtain the submitting point of the construction and the software package to be tested of the construction. And judging whether the number of the submitting points constructed at the time is one. If the number of the submitting points constructed at this time is one, the submitting points constructed at this time and the software package to be tested constructed at this time are sent to the testing end, so that the testing end writes the submitting points constructed at this time into a local submitting point record file. If the number of the submitting points constructed at the time is not one, sending the software package to be tested constructed at the time to the testing end.

Alternatively, when the bisection is performed according to the request direction and all the commit points of the previous construction, the commit points included in the software package to be tested may be bisected in the request direction. For example: all the point of submission of the last time bipartite construction is 1-12, the point of submission is divided into 1-6 and 7-12 by the last time bipartite construction, the software package to be tested of the last time construction comprises the point of submission of 1-6, if the request direction is the current direction, the bipartite construction can be carried out on the bipartite points of the point of submission comprised by the software package to be tested of the last time construction, for example: dividing the submitting points of 1-6 into 1-3 and 4-6, and obtaining the submitting points constructed at this time as the submitting points 1-6, wherein the software package to be tested constructed at this time can be 1-3 or 4-6; the software package to be tested constructed in the previous time can also be constructed in a bisection mode without selecting a bisection point, for example: commit points of 1-6 are divided into 1-2 and 3-6. If the request direction is the opposite direction, selecting the commit points except the commit point contained in the software package to be tested constructed in the previous time from all the commit points in the previous time to perform binary construction, and performing binary construction on the binary middle points of the commit points except the commit point contained in the software package to be tested constructed in the previous time, in this example, performing binary construction on the commit point of 7-12, and dividing the commit point into 7-9 and 10-12, so that the commit point of this construction is 7-12, and the software package to be tested of this construction can be 7-9 or 10-12; the bipartite point may be omitted for bipartite construction, for example, the commit point of 7-12 is divided into 7-8 and 9-12.

If the number of the submitting points constructed at this time is one, the submitting points contained in the software package to be tested are consistent with the submitting points constructed at this time, for example, the submitting point constructed at this time is submitting point 3, and the software package to be tested is also submitting point 3, then the submitting points constructed at this time and the software package to be tested constructed locally are sent to the testing end, and after the testing end receives the submitting points constructed at this time, the testing end writes the submitting points into the local submitting point record file so as to be used for judging whether the performance test of the software package is finished or not in the testing process. If the number of the submitting points constructed at this time is not one, the submitting points constructed at this time are not sent to the testing end, and the sample submitting point record file is empty and can be used for determining that the test is not finished when the testing end performs the test.

Fig. 4 shows a flow diagram of a method 400 of testing the performance of a software package according to yet another embodiment of the invention, the method 400 being adapted to be executed in a testing terminal. As shown in fig. 4, method 400 begins at step 401. 401. And judging whether the software package to be tested is delivered. The software package may be a kernel package. If so, then 402 is entered.

402. And installing the software package to be tested.

403. And judging whether the software package to be tested is successfully installed. If the installation fails, it proceeds to 404, and if the installation is successful, it proceeds to 405.

404. And sending prompt mail and performing environment check.

405. And judging whether the binary flag bit of the software package to be tested is set or not. If the binary flag bit of the software package under test is set, then 406 is entered, and if the binary flag bit of the software package under test is not set, then 407 is entered.

406. And configuring a reference test program and service to be retested, stopping the software package monitoring service, restarting the test equipment of the execution method 400 of the test terminal and starting the test.

407. The test environment and services of the benchmark test program are configured, the software package listening service is stopped, the test equipment of the execution method 400 of the test terminal is restarted, and the test is started.

408. The test is completed.

409. And automatically acquiring test data, writing the test data into a database, and comparing the test data with baseline data.

410. And judging whether the binary flag bit of the software package to be tested is set or not. If the binary flag of the software package under test is not set, then 411 is performed, and if the binary flag of the software package under test is set, 414 is performed.

411. It is determined whether the test data is degraded compared to the baseline data. If degraded, 412 is performed, and if not, 413 is performed.

412. And recording a reference test program suite in which degradation occurs, triggering a two-half test flow, and setting two-half flag bits.

413. And sending information prompting the completion of the test, and starting the software package monitoring service to wait for the next round of test.

414. It is determined whether the test data is degraded compared to the baseline data. If not, 415 is performed, and if so, 416 is performed.

415. And clearing the local commit point record file, and sending a halving construction request in the opposite direction to the server side.

416. A determination is made as to whether there is content in the local commit point record file, if not, proceed to 417, if so, proceed to 418.

417. And sending a binary construction request continuing to the current direction to the server side.

418. Pushing the mail comprising the node with the merging problem, clearing the binary flag bit and the local commit point record file, and starting the software package monitoring service to wait for the next round of testing.

Fig. 5 shows a flow diagram of a method 500 of testing the performance of a software package according to yet another embodiment of the invention, the method 500 being adapted to be executed in a server side. As shown in fig. 5, method 500 begins at step 501.

501. When a bipartite construction request is received, all the retrieval points of the previous construction are obtained, and the bipartite points are taken according to the request direction for construction.

502. Judging whether the number of the extraction points constructed by the current bipartite is 1. If 1, 503 is executed, and if not 1, 504 is executed.

503. And issuing the submitting point of the current construction and the software package to be tested of the current construction to the testing end.

504. And issuing the software package to be tested constructed at the time to the test terminal.

It should be noted that the specific execution logic of the

test methods

400 and 500 for the performance of the software package is referred to in the description of the corresponding portions of the method 300, and will not be repeated here.

The present invention also provides a kernel testing system, fig. 6 is a schematic diagram of a testing system 600 for testing performance of a software package according to an embodiment of the present invention, where the system 600 includes a testing end 610 and a server end 620, and the system 600 may further include a database 630.

The test end 610 is adapted to install the software package to be tested when the software package monitoring service running on the test end monitors that the software package to be tested is delivered, determine whether a binary flag bit of the software package to be tested is set, if the binary flag bit is not set, configure a test environment and service of a reference test program, stop the software package monitoring service, test the software package to be tested through the reference test program to obtain first test data, compare the first test data with baseline data corresponding to the software package to be tested, determine whether degradation occurs between the first test data and the baseline data, if the first test data is degraded compared with the baseline data, record a portion to be retested in the reference test program with degradation, send the portion to the server, set the binary flag bit of the software package to be tested, if the first test data is not degraded compared with the baseline data, complete the test, and start the software package monitoring service.

The test end 610 is further adapted to configure a test environment and a service of a portion to be retested in the reference test program when the binary flag bit is set, test the software package to be tested through the portion to be retested in the reference test program to obtain second test data, compare the second test data with baseline data corresponding to the software package to be tested, judge whether the second test data is degraded compared with the baseline data, judge whether contents exist in a local commit point record file if the second test data is degraded compared with the baseline data, prompt that the software package to be tested has a problem if the contents exist in the local commit point record file, clear the binary flag bit and the local commit point record file of the software package to be tested, and start the software package monitoring service, otherwise send a binary construction request in a current direction to the server, empty the local point record file if the second test data is not degraded compared with the baseline data, and send a binary construction request in a reverse direction to the server.

The server 620 is adapted to construct and compile codes including a plurality of commit points submitted in a predetermined time period, send the compiled to-be-tested software package to the testing end, when receiving a binary construction request, obtain all the commit points constructed in the previous time, construct according to the request direction and the binary points of all the commit points, obtain the constructed commit point and the constructed to-be-tested software package, determine whether the number of the constructed commit points is one, if yes, send the constructed commit point and the constructed to-be-tested software package to the testing end, so that the testing end writes the constructed commit point into a local commit point record file, otherwise send the constructed to-be-tested software package to the testing end.

The database 630 is adapted to store the first test data and the second test data, and is further adapted to store baseline data corresponding to the software package to be tested.

The specific execution logic of the test terminal 610, the server terminal 620, and the database 630 is referred to in the corresponding parts of the foregoing methods 300 to 500, and will not be described herein.

The present invention also provides another system for testing the performance of a software package, fig. 7 is a schematic diagram of a system 700 for testing the performance of a software package according to an embodiment of the present invention, where the system 700 includes a build management server 710, a software package compiling server 720, a device under test 730, and a database 740. The system 700 may also include a user management device 750. One or more devices under test 730 may be configured to be communicatively connected to the management server 710, the software package compiling server 720, the devices under test 730, the database 740, and the user management device 750 via a TCP/I P network.

The construction management server 710 is adapted to construct codes including a plurality of construction points submitted in a predetermined time period, when a binary construction request is received, acquire all the construction points of the previous construction, construct the construction points and the software packages to be tested according to the request direction and the binary points of all the construction points, judge whether the number of the construction points to be tested is one, if so, send the construction points and the construction software packages to be tested to the testing end, so that the testing end writes the construction points to a local construction point record file, otherwise, send the construction software packages to be tested to the testing end;

The software package compiling server 720 is adapted to compile codes submitted in a predetermined time period and including a plurality of point of submission, and send the compiled software package to be tested to the testing end;

and the device to be tested 730 is adapted to install the software package to be tested when the software package monitoring service running on the test end monitors that the software package to be tested is delivered, judge whether the binary flag bit of the software package to be tested is set, if the binary flag bit is not set, configure the test environment and service of the reference test program, stop the software package monitoring service, test the software package to be tested through the reference test program, obtain first test data, compare the first test data with the baseline data corresponding to the software package to be tested, judge whether the first test data is degraded with the baseline data, if the first test data is degraded with the baseline data, record the part to be retested in the reference test program with degradation, send the part to the server end, set the binary flag bit of the software package to be tested, if the first test data is not degraded with the baseline data, complete the test, and start the software package monitoring service.

The device under test 730 is further adapted to configure a test environment and a service of a portion to be retested in the reference test program when the binary flag bit is set, test the software package under test through the portion to be retested in the reference test program to obtain second test data, compare the second test data with baseline data corresponding to the software package under test, judge whether the second test data is degraded compared with the baseline data, judge whether contents are in a local commit point record file if the second test data is degraded compared with the baseline data, prompt the software package under test to have problems if the contents are in the local commit point record file, clear the binary flag bit and the local commit point record file of the software package under test, and start the software package monitoring service, otherwise send a binary construction request in the current direction to a server, empty the local point record file if the second test data is not degraded compared with the baseline data, and send a binary construction request in the opposite direction to the server;

a database 740 adapted to store first test data; and is further adapted to store second test data.

A user management device 750 is adapted to present said first test data and said second test data.

The specific execution logic of the build management server 710, the software package compiling server 720, the device under test 730, the database 740, and the user management device 750 refers to the descriptions in the corresponding parts of the foregoing methods 300 to 500, and will not be repeated here.

The method for testing the performance of a software package provided by the present invention will be described with reference to a specific embodiment. FIG. 8 shows a flow diagram of a method 800 for testing the performance of a seed package according to yet another embodiment of the invention. The method 800 involves building a management server, a software package compilation server, a device under test, a database, and a user management device.

The build management server includes a deployment platform (e.g., jenk ins) for continuous integration (Cont i nuous i ntegrat i on, abbreviated CI) build event management and a deployment platform (e.g., gerr it) for code management.

The software package compiling server is used for deploying a compiling environment constructed by the kernel, compiling the kernel code, compiling the software package through Jenk ins and sending the compiled software package to a specified test machine type (equipment to be tested).

The device to be tested, a test carrier (DUT) for testing the performance of the software package, can be used for deploying a software package monitoring service and realizes data acquisition work and diagnosis work for a benchmark test program. The software package monitoring service can be used for identifying the received software package and completing the work of data entry, data diagnosis, drive retest and the like of the benchmark test program.

The database is used for storing test data (such as first test data and second test data) obtained by the test.

The user management device is used for accessing the data display platform and checking the details of the test data, the data display platform can be deployed by the data display server, and the data display is realized by analyzing the test data stored in the database.

As shown in fig. 8, method 800 begins at step 810.

810. Jenk ins obtains the code corresponding to the latest code submission i d (commit t i d) on Gerr it for use in building the software package.

820. The Jenk ins creates a task of constructing the software package to be tested, and sends the software package to be tested to the device to be tested. Optionally, the task of constructing the software package to be tested constructs the software package to be tested on the construction management server through ssh protocol based on the latest code submission i d, and sends the constructed software package to be tested to the appointed directory of the equipment to be tested through FTP protocol.

830. When a software package monitoring service (Mon i tor Server) running on the device to be tested monitors that the software package to be tested is delivered, the software package to be tested is tested, and test data are written into a database. Optionally, after the software package monitoring service monitors that a new software package to be tested exists in the designated directory of the device to be tested, the new software package to be tested is installed to cover the original kernel, and the test logic in the method 300 is adopted to complete the test, so as to obtain test data and write the test data into the database.

840. The test data is presented through the user management device, optionally, the data display platform is accessed through the user management device, and the details of the test data are checked. The data display platform can display the test data by adopting the UTP platform, and a user can access the data display platform on a webpage through the HTTP protocol.

Steps 850-880 are loop phases and steps 850-880 will be repeated until the test is completed.

850. And the software package monitoring service running on the equipment to be tested analyzes the test data, and when the degradation of the test data is found, a bipartite construction interface is called to drive the Jenk ins to split according to all the submitting points constructed in the previous time, the software package to be tested is reconstructed, and the newly constructed software package to be tested is sent to the equipment to be tested so as to test the newly constructed software package to be tested.

860. When the binary build interface of Jenk ins is triggered, all codes submitted over a predetermined time period (e.g., the current day) on Gerr i t are acquired.

870. The Jenk ins filters codes submitted in a preset time period to construct a new software package to be tested.

880. After the software package monitoring service running on the equipment to be tested identifies the software package to be tested with the binary flag bit, the software package to be tested with the binary flag bit is copied, the result of each retest is informed to Jenk ins through a binary construction interface, jenk ins is judged according to the feedback result, and finally, after the circulation is finished, the feedback is given to the commit i d of the submitting point of the software package monitoring service with related risk.

890. After the test is completed, the software package monitoring service writes the obtained commit id of the at-risk commit point into the database.

As shown in fig. 9, the software package monitoring service of the device under test regularly identifies whether a new software package under test is added locally and whether a retest identifier exists (the binary flag bit indicates that the retest identifier exists) and if the software package under test is added newly and the retest identifier exists, the environment in which the benchmark tool containing degradation data operates is configured and operated, and the benchmark tool without degradation does not participate in retest (the part to be retested in the operation benchmark test program). All test data are diagnosed one by comparing historical data (baseline data), and after confirming degradation according to a threshold, for example: and when the degradation degree of the test data is smaller than a certain threshold value compared with the baseline data, calling the binary construction interface, writing a tool corresponding to the degradation condition into the configuration file, uploading the synchronously returned intersection point with risk to the database when the binary construction interface returns a binary construction ending mark, and clearing the configuration of the configuration file about retest. Wherein the binary construct interface may perform the flow of the method 500 described above. Finally, the test data and the point of mention with risk are written into a database.

If the software package to be tested is newly added but the retest mark does not exist, judging whether the software package to be tested is the latest kernel, and if not, not processing; if so, the environment in which the benchmark tool is running is configured and run. All test data are diagnosed one by comparing historical data (baseline data), and after confirming degradation according to a threshold, for example: and when the degradation degree of the test data is smaller than a certain threshold value compared with the baseline data, calling the binary construction interface, writing a tool corresponding to the degradation condition into the configuration file, uploading the synchronously returned intersection point with risk to the database when the binary construction interface returns a binary construction ending mark, and clearing the configuration of the configuration file about retest. Finally, the test data and the point of mention with risk are written into a database.

Further, the current version and the previous version of the database are obtained through the version number and the model of the tested software package to be compared, and the part to be retested in the degraded reference test program is recorded, so that the subsequent part to be retested is continuously tested, the whole reference test program is not required to be retested, and the test efficiency of the software package is improved.

In the field of kernel testing, the scheme of the invention can improve the testing efficiency of kernel performance.

The various techniques described herein may be implemented in connection with hardware or software or, alternatively, with a combination of both. Thus, the methods and apparatus of the present invention, or certain aspects or portions of the methods and apparatus of the present invention, may take the form of program code (i.e., instructions) embodied in tangible media, such as removable hard drives, U-drives, floppy diskettes, CD-ROMs, or any other machine-readable storage medium, wherein, when the program is loaded into and executed by a machine, such as a computer, the machine becomes an apparatus for practicing the invention.

In the case of program code execution on programmable computers, the mobile terminal will generally include a processor, a storage medium readable by the processor (including volatile and non-volatile memory and/or storage elements), at least one input device, and at least one output device. Wherein the memory is configured to store program code; the processor is configured to execute the method of testing the performance of the software package of the invention in accordance with instructions in said program code stored in the memory.

By way of example, and not limitation, readable media comprise readable storage media and communication media. The readable storage medium stores information such as computer readable instructions, data structures, program modules, or other data. Communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. Combinations of any of the above are also included within the scope of readable media.

In the description provided herein, algorithms and displays are not inherently related to any particular computer, virtual system, or other apparatus. Various general-purpose systems may also be used with examples of the invention. The required structure for a construction of such a system is apparent from the description above. In addition, the present invention is not directed to any particular programming language. It will be appreciated that the teachings of the present invention described herein may be implemented in a variety of programming languages, and the above description of specific languages is provided for disclosure of enablement and best mode of the present invention.

In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.

Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects.

Those skilled in the art will appreciate that the modules or units or components of the devices in the examples disclosed herein may be arranged in a device as described in this embodiment, or alternatively may be located in one or more devices different from the devices in this example. The modules in the foregoing examples may be combined into one module or may be further divided into a plurality of sub-modules.

Those skilled in the art will appreciate that the modules in the apparatus of the embodiments may be adaptively changed and disposed in one or more apparatuses different from the embodiments. The modules or units or components of the embodiments may be combined into one module or unit or component and, furthermore, they may be divided into a plurality of sub-modules or sub-units or sub-components. Any combination of all features disclosed in this specification, and all processes or units of any method or apparatus so disclosed, may be employed, except that at least some of such features and/or processes or units are mutually exclusive. Each feature disclosed in this specification may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.

Furthermore, those skilled in the art will appreciate that while some embodiments described herein include some features but not others included in other embodiments, combinations of features of different embodiments are meant to be within the scope of the invention and form different embodiments.

Furthermore, some of the embodiments are described herein as methods or combinations of method elements that may be implemented by a processor of a computer system or by other means of performing the functions. Thus, a processor with the necessary instructions for implementing the described method or method element forms a means for implementing the method or method element. Furthermore, the elements of the apparatus embodiments described herein are examples of the following apparatus: the apparatus is for carrying out the functions performed by the elements for carrying out the objects of the invention.

As used herein, unless otherwise specified the use of the ordinal terms "first," "second," "third," etc., to describe a general object merely denote different instances of like objects, and are not intended to imply that the objects so described must have a given order, either temporally, spatially, in ranking, or in any other manner.

While the invention has been described with respect to a limited number of embodiments, those skilled in the art, having benefit of the above description, will appreciate that other embodiments are contemplated within the scope of the invention as described herein. Furthermore, it should be noted that the language used in the specification has been principally selected for readability and instructional purposes, and may not have been selected to delineate or circumscribe the inventive subject matter.

Claims

1. A method for testing performance of a software package, executed in a testing terminal, the method comprising:

when a software package monitoring service running on the test end monitors that a software package to be tested is sent, installing the software package to be tested; judging whether the binary flag bit of the software package to be tested is set or not;

if the binary flag bit is not set, configuring the test environment and service of the reference test program, and stopping the software package monitoring service;

testing the software package to be tested through the reference test program to obtain first test data;

comparing the first test data with baseline data corresponding to the software package to be tested;

determining whether the first test data is degraded compared to the baseline data;

If the first test data is degraded compared with the baseline data, recording a part to be retested in a reference test program with degradation, sending the part to be retested to a server side, setting a binary flag bit of the software package to be tested so that the server side carries out binary construction on the software package to be tested, and transmitting the software package to be tested after binary construction to a test side for continuous test;

if the first test data is not degraded compared with the baseline data, the test is completed and the software package listening service is started.

2. The method of claim 1, further comprising:

if the binary flag bit is set, configuring the test environment and service of the part to be retested in the reference test program;

testing the software package to be tested through the part to be retested in the reference test program to obtain second test data;

comparing the second test data with the baseline data corresponding to the software package to be tested;

determining whether the second test data is degraded compared to the baseline data;

if the second test data is degraded compared with the baseline data, judging whether the local commit point record file contains content or not;

If the local submitting point record file has content, prompting that the software package to be tested has problems, clearing the binary flag bit of the software package to be tested and the local submitting point record file, starting the software package monitoring service, otherwise, sending a binary construction request in the current direction to a server side;

and if the second test data is not degraded compared with the baseline data, clearing the local commit point record file, and sending a halving construction request in the opposite direction to a server side.

3. The method according to claim 1 or 2, wherein the comparing the first test data with baseline data corresponding to the software package under test comprises:

and comparing the first test data with the baseline data corresponding to the version number of the software package to be tested.

4. The method of claim 2, wherein the comparing the second test data with baseline data corresponding to the software package under test comprises:

and comparing the second test data with the baseline data corresponding to the version number of the software package to be tested.

5. The method of any of claims 1 to 4, wherein the test end is connected to a database, the baseline data being stored in the database, the method further comprising:

And writing the first test data into a database.

6. The method of claim 2, wherein the test end is coupled to a database, the baseline data being stored in the database, the method further comprising:

and writing the second test data into a database.

7. The method of any one of claims 1 to 6, further comprising:

and if the installation fails, prompting that the installation of the software package to be tested fails.

8. A method for testing performance of a software package, executed at a server, the method comprising:

constructing and compiling codes submitted in a preset time period and containing a plurality of submitted points, and sending the compiled software package to be tested to a testing end so that the testing end can execute the method according to any one of claims 1 to 7 to test the software package to be tested;

when a binary construction request is received, all the extraction points constructed in the previous time are obtained;

performing binary construction according to the request direction and all the submitting points to obtain the submitting point of the construction and the software package to be tested of the construction;

judging whether the number of the submitting points constructed at the time is one;

if the number of the submitting points constructed at this time is one, sending the submitting points constructed at this time and the software package to be tested constructed at this time to a testing end so that the testing end writes the submitting points constructed at this time into a local submitting point record file;

If the number of the submitting points constructed at the time is not one, sending the software package to be tested constructed at the time to the testing end.

9. A system for testing the performance of a software package, comprising:

the test terminal is suitable for installing the software package to be tested when the software package monitoring service running on the test terminal monitors that the software package to be tested is delivered, judging whether the two-part flag bit of the software package to be tested is set, if the two-part flag bit is not set, configuring the test environment and service of the reference test program, stopping the software package monitoring service, testing the software package to be tested through the reference test program to obtain first test data, comparing the first test data with baseline data corresponding to the software package to be tested, judging whether the first test data is degraded with the baseline data, if the first test data is degraded with the baseline data, recording a part to be retested in the reference test program with degradation, sending the part to the server terminal, setting the two-part flag bit of the software package to be tested, if the first test data is not degraded with the baseline data, completing the test, and starting the software package monitoring service.

The server side is suitable for constructing and compiling codes which are submitted in a preset time period and contain a plurality of point of submission and sending the compiled software package to be tested to the testing side, when a binary construction request is received, all the point of submission constructed in the previous time are obtained, construction is carried out according to the request direction and the binary points of all the point of submission, the point of submission constructed in this time and the software package to be tested constructed in this time are obtained, whether the number of the point of submission constructed in this time is one or not is judged, if yes, the point of submission constructed in this time and the software package to be tested constructed in this time are sent to the testing side, so that the testing side writes the point of submission constructed in the local point of submission record file, and otherwise, the software package to be tested constructed in this time is sent to the testing side.

10. The system of claim 9, wherein the test end is further adapted to: when the binary flag bit is set, configuring a test environment and service of a part to be retested in a reference test program, testing the software package to be tested through the part to be retested in the reference test program to obtain second test data, comparing the second test data with baseline data corresponding to the software package to be tested, judging whether the second test data is degraded compared with the baseline data, judging whether contents exist in a local submitting point record file if the second test data is degraded compared with the baseline data, prompting that the software package to be tested has problems if the contents exist in the local submitting point record file, clearing the binary flag bit and the local submitting point record file of the software package to be tested, starting the software package monitoring service, otherwise, sending a binary construction request in the current direction to a server, clearing the local point record file if the second test data is not degraded compared with the submitted baseline data, and sending a binary construction request in the opposite direction to the server.

11. The system of claim 10, further comprising:

and the database is suitable for storing the first test data and the second test data.

12. A system for testing the performance of a software package, comprising:

the construction management server is suitable for constructing codes which are submitted in a preset time period and contain a plurality of point of submission, when a binary construction request is received, all the point of submission constructed in the previous time are obtained, construction is carried out according to the request direction and the binary points of all the point of submission, the point of submission constructed at this time and a software package to be tested constructed at this time are obtained, whether the number of the point of submission constructed at this time is one or not is judged, if so, the point of submission constructed at this time and the software package to be tested constructed at this time are sent to the testing end, so that the testing end writes the point of submission constructed at this time into a local point of submission record file, otherwise, the software package to be tested constructed at this time is sent to the testing end;

the software package compiling server is suitable for compiling codes submitted in a preset time period and comprising a plurality of point of submission and sending the compiled software package to be tested to the testing end;

the device to be tested is suitable for installing the software package to be tested when the software package monitoring service running on the test end monitors that the software package to be tested is delivered, judging whether the two-part flag bit of the software package to be tested is set, if the two-part flag bit is not set, configuring the test environment and service of the reference test program, stopping the software package monitoring service, testing the software package to be tested through the reference test program to obtain first test data, comparing the first test data with baseline data corresponding to the software package to be tested, judging whether the first test data is degraded with the baseline data, if the first test data is degraded with the baseline data, recording a part to be retested in the reference test program with degradation, sending the part to be retested to the server end, setting the two-part flag bit of the software package to be tested, if the first test data is not degraded with the baseline data, completing the test, and starting the software package monitoring service.

And the database is suitable for storing the first test data.

13. The system of claim 12, wherein:

the device to be tested is further adapted to configure a test environment and service of a part to be retested in a reference test program when a binary flag bit is set, test the software package to be tested through the part to be retested in the reference test program to obtain second test data, compare the second test data with baseline data corresponding to the software package to be tested, judge whether the second test data is degraded compared with the baseline data, judge whether contents exist in a local submitting point record file if the second test data is degraded compared with the baseline data, prompt the software package to be tested to have problems if the contents exist in the local submitting point record file, clear the binary flag bit and the local submitting point record file of the software package to be tested, start the software package monitoring service, otherwise, send a binary construction request in the current direction to a server, clear the local submitting point record file if the second test data is not degraded compared with the baseline data, and send a binary construction request in the opposite direction to the server;

The database is further adapted to store second test data.

14. The system of claim 12 or 13, further comprising:

and the user management device is suitable for presenting the first test data and the second test data.

15. A computing device, comprising:

at least one processor; and

a memory storing program instructions, wherein the program instructions are configured to be adapted to be executed by the at least one processor, the program instructions comprising instructions for performing the method of any one of claims 1 to 8.

16. A readable storage medium storing program instructions which, when read and executed by a computing device, cause the computing device to perform the method of any one of claims 1 to 8.