CN116225911A - Function test method and device for observability platform - Google Patents

Function test method and device for observability platform Download PDF

Info

Publication number
CN116225911A
CN116225911A CN202310045082.0A CN202310045082A CN116225911A CN 116225911 A CN116225911 A CN 116225911A CN 202310045082 A CN202310045082 A CN 202310045082A CN 116225911 A CN116225911 A CN 116225911A
Authority
CN
China
Prior art keywords
test
case
observability
version
platform
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310045082.0A
Other languages
Chinese (zh)
Inventor
李宇飞
张国山
陆宏鸣
蒋烁淼
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Observation Future Information Technology Co ltd
Original Assignee
Shanghai Observation Future Information Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Observation Future Information Technology Co ltd filed Critical Shanghai Observation Future Information Technology Co ltd
Priority to CN202310045082.0A priority Critical patent/CN116225911A/en
Publication of CN116225911A publication Critical patent/CN116225911A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/36—Preventing errors by testing or debugging software
    • G06F11/3668—Software testing
    • G06F11/3672—Test management
    • G06F11/3684—Test management for test design, e.g. generating new test cases
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/36—Preventing errors by testing or debugging software
    • G06F11/3668—Software testing
    • G06F11/3672—Test management
    • G06F11/368—Test management for test version control, e.g. updating test cases to a new software version
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/36—Preventing errors by testing or debugging software
    • G06F11/3668—Software testing
    • G06F11/3672—Test management
    • G06F11/3688—Test management for test execution, e.g. scheduling of test suites
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/36—Preventing errors by testing or debugging software
    • G06F11/3668—Software testing
    • G06F11/3672—Test management
    • G06F11/3692—Test management for test results analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/36—Preventing errors by testing or debugging software
    • G06F11/3668—Software testing
    • G06F11/3696—Methods or tools to render software testable
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
    • G06F13/38—Information transfer, e.g. on bus
    • G06F13/382—Information transfer, e.g. on bus using universal interface adapter
    • G06F13/385—Information transfer, e.g. on bus using universal interface adapter for adaptation of a particular data processing system to different peripheral devices
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

The invention provides a function test method and device of an observability platform, wherein the method comprises the following steps: s1, compiling and storing a self-defined test case; s2, inputting a version to be regressed, calculating a regressive case of the version according to version information of the version and the test case, and forming a regressive case set; s3, applying for computing resources according to the regression use case set, and creating Pod in real time according to the computing test resources; s4, executing the regression use case in the Pod, testing the regression use case, obtaining a test result and caching the test result; s5, reading the test result, summarizing the test related information, converting the test result into an inline protocol format, and outputting the inline protocol format to an observability platform for secondary processing and analysis. The method integrates the observability data and results generated in the testing process with the observability platform, and outputs the observability data and results to the observability platform through a standardized data format for secondary processing and analysis.

Description

Function test method and device for observability platform
Technical Field
The invention belongs to the technical field of software testing, and particularly relates to a function testing method and device of an observability platform.
Background
The observability platform is an open platform based on cloud primordia, provides a series of observability functions including log, index, tracking, debugging and the like, and the functions can help a user to quickly locate problems, so that the stability and usability of the system are improved.
The function test of the observability platform is to perform a black box test on each function of the observability platform through an API interface, and whether the function of the observability platform is normal is verified.
In the traditional API function test method, a developer is required to write test cases containing one or more API call sequences, parameters and response expectations, then the test cases are executed through a test framework, and whether the functions of the observability platform are normal is verified.
The existing function test method of the observability platform has the following limitations:
1) The test framework integrated with the programming language often needs to be developed and fully cooperated with a tester, so that the cost for writing the test case is high, the writing and maintenance workload of the test case is high, and the quick iteration of the test case is not facilitated;
2) The client test programming tool with the graphical interface improves test programming efficiency, but is not suitable for a continuous test scene because the tool needs to be installed on the client to trigger execution. For example, when a version regression test is triggered, thousands of different test cases often need to be executed, and at this time, a single client often needs tens of hours to complete the task;
3) Observability data and results generated in the testing process often circulate inside the platform and are difficult to use by other external observability platforms, so the method is not completely suitable for testing and integrating the observability platforms.
In view of this, the present invention has been made.
Disclosure of Invention
In view of the above, the invention discloses a functional test method and device for an observability platform, which realize centralized management and distributed execution of test cases at a server side and improve the execution efficiency of the test cases; and integrating the observability data and results generated in the testing process with an observability platform, outputting the observability data and results to an external observability platform through a standardized data format, and carrying out secondary processing and analysis.
Specifically, the invention is realized by the following technical scheme:
in a first aspect, the invention discloses a method for testing functions of an observability platform, which comprises the following steps:
s1, compiling and storing a self-defined test case;
specifically, the test cases can be written by a common client graphical tool, the test cases of the JMeter can be written by a GUI tool of the JMeter, and submitted in the form of JMX files for centralized storage and calculation.
S2, inputting a version to be regressed, calculating a regressive case of the version according to version information of the version and the test case, and forming a regressive case set;
s3, applying for computing resources according to the regression use case set, and creating Pod in real time according to the computing test resources;
s4, executing the regression use case in the Pod, testing the regression use case, obtaining a test result and caching the test result;
s5, reading the test result, summarizing the test related information, converting the test result into an inline protocol format, and outputting the inline protocol format to an observability platform for secondary processing and analysis.
Specifically, the data is transmitted in an InfluxDB compatible line protocol format, and is output to an external observability platform for secondary processing, in the process, the data acquisition module acquires information such as a request address, parameters, a status code, response time, error prompt and the like of each API call, and meanwhile, the data acquisition module adds additional marks to the acquired data, including the name of a test case, the computing instance to which an execution environment belongs, and information such as a custom tag.
Further, in the step S3, the method for applying for computing resources includes dynamically calling Virtual Kubelet of Kubernetes to apply for computing resources.
Further, in the step S3, the test resource includes a name of the test resource, a type of the test resource, a configuration of the test resource, and environmental information of the test resource.
Wherein the name of the test resource is a unique identifier of the test resource;
types of test resources include Pod, deployment of Kubernetes, etc.;
the configuration of the test resources comprises CPU, memory, disk and the like of Pod;
the environment information of the test resource includes mirror image, mirror image version, etc. of the Pod.
Further, in the step S1, the definition information of the test case includes a test device, test case information, version control information and a custom tag.
The testing device is a set of a plurality of testing cases;
the test case information comprises the name of the test case, the unique identification of the test case, the address of the API, parameters and response expectations;
the version control information comprises the version information of the test case and the version information of the observability platform corresponding to the test case;
the custom tag is a custom tag transmitted by a user and is used for distinguishing different test cases and application scenes.
Further, the test related information includes definition information of the test case, operation environment information of the test case, and execution information of the test case.
Further, in the step S4, the test result includes an API address, a parameter, an error message, a response time, and whether it is expected.
Further, in the step S5, the secondary processing and analysis includes verifying whether the function of the observability platform is normal by calling an API.
In a second aspect, the present invention discloses a functional testing apparatus for an observability platform, comprising:
test case management module: storing a custom test case;
regression use case calculation module: inputting a version to be regressed, calculating a regressive case of the version according to version information of the version and the test case, and forming a regressive case set;
and (3) a Pod scheduling module: applying for computing resources according to the regression use case set, and creating Pod in real time according to the computing test resources;
the test result processing module: executing the regression use case in the Pod, testing the regression use case, obtaining a test result and caching;
and the data output module is used for: and reading the test result, summarizing the test related information, converting the test result into a row protocol format, and outputting the row protocol format to an observability platform for secondary processing and analysis.
In a third aspect, the present invention discloses a computer-readable storage medium, on which a computer program is stored, which program, when being executed by a processor, implements the steps of the functional test method according to the first aspect.
In a fourth aspect, the present invention discloses a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, said processor implementing the steps of the functional test method according to the first aspect when said program is executed.
Compared with the prior art, the invention has the beneficial effects that:
the function test method of the observability platform provided by the invention realizes the function test of the observability platform, verifies whether the function of the observability platform is abnormal by calling the API, and can complete the writing of test cases, the execution of the test cases, the collection of test results and the analysis of the test results under the condition that a tester does not need the intervention of a developer; the test cases can be executed in a distributed mode at the server, so that large-scale version regression test is not required to be manually attended, and labor is saved; observability data and results generated in the testing process are automatically output to the observability platform, and dependency locking with the testing platform is released.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to designate like parts throughout the figures. In the drawings:
FIG. 1 is a flow chart of a method for testing functions of an observability platform according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a functional test device of an observability platform according to an embodiment of the invention;
FIG. 3 is a schematic diagram of a resource allocation flow according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of regression testing interactions provided by an embodiment of the present invention;
fig. 5 is a schematic structural diagram of a computer device according to an embodiment of the present invention.
Detailed Description
The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings and detailed description, but it will be understood by those skilled in the art that the examples described below are some, but not all, examples of the present invention, and are intended to be illustrative of the present invention only and should not be construed as limiting the scope of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
In order to more clearly illustrate the technical scheme of the invention, the following description is given by way of specific examples.
Examples
Referring to fig. 1, the invention discloses a function test method of an observability platform, which comprises the following steps:
s1, compiling and storing a self-defined test case;
s2, inputting a version to be regressed, calculating a regressive case of the version according to version information of the version and the test case, and forming a regressive case set;
s3, applying for computing resources according to the regression use case set, and creating Pod in real time according to the computing test resources;
s4, executing the regression use case in the Pod, testing the regression use case, obtaining a test result and caching the test result;
s5, reading the test result, summarizing the test related information, converting the test result into an inline protocol format, and outputting the inline protocol format to an observability platform for secondary processing and analysis.
The actual operation of the invention is as follows:
the user uses the JMeter client tool to write the following test cases, see fig. 4:
1. sending an index data to the 'write/metrics' interface of the observability platform, wherein the index value is: vcpu=1, vmem=10mb, labeled: somekey=somevalue.
2. Inquiring index data from an observability platform, wherein the inquiring conditions are as follows: somekey=somevalue.
3. The test case version is hash ai 8a9sd, the associated observability platform version is v1.0.0, and the joint version number is v1.0.0-ai 8a9sd.
The user submits the test cases to the testing device, inputs the version v1.0.0 to be regressed, calculates all regression cases meeting the conditions under v1.0.0 according to the version information of the version and the test cases, forms a regression case set, applies for calculation resources according to the regression case set, calls Kubernetes Virtual Kubelet, distributes independent operation environments for the test cases, and starts execution, and is shown in FIG. 3. After the test cases are executed, the test device uploads the test results to the cache file, a user can screen the test cases with version v1.0.0 on the observability platform, check the test results, and finally complete the version regression test.
The test results contained the following:
1) The definition information of the test case comprises the name of the test case, the version of the test case, the associated observability platform version, the joint version number and a custom label;
2) The running environment information of the test case comprises the name, type, CPU, memory, disk and grid of the running environment of the test case;
3) The execution information of the test case comprises an execution result, execution time, an execution log and the like of the test case.
The invention also provides a function test device of the observability platform, as shown in fig. 2, which specifically comprises:
test case management module: storing a custom test case;
regression use case calculation module: inputting a version to be regressed, calculating a regressive case of the version according to version information of the version and the test case, and forming a regressive case set;
and (3) a Pod scheduling module: applying for computing resources according to the regression use case set, and creating Pod in real time according to the computing test resources;
the test result processing module: executing the regression use case in the Pod, testing the regression use case, obtaining a test result and caching;
and the data output module is used for: and reading the test result, summarizing the test related information, converting the test result into a row protocol format, and outputting the row protocol format to an observability platform for secondary processing and analysis. .
The device mainly comprises the five modules, and the purpose of parallel operation can be realized by well constructing the system and simultaneously mounting the same file system.
In the implementation, each module may be implemented as an independent entity, or may be combined arbitrarily, and implemented as the same entity or several entities, and the implementation of each unit may be referred to the foregoing method embodiment, which is not described herein again.
Fig. 5 is a schematic structural diagram of a computer device according to the present disclosure. Referring to FIG. 5, the computer device 400 includes at least a memory 402 and a processor 401; the memory 402 is connected to the processor through a communication bus 403, and is configured to store computer instructions executable by the processor 401, and the processor 401 is configured to read the computer instructions from the memory 402 to implement the steps of the functional test method described in the foregoing embodiments.
For the above-described device embodiments, reference is made to the description of the method embodiments for the relevant points, since they essentially correspond to the method embodiments. The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the objectives of the disclosed solution. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
Computer readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices including, for example, semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices), magnetic disks (e.g., internal magnetic disks or removable disks), magneto-optical disks, and CD-ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.
Finally, it should be noted that: while this specification contains many specific implementation details, these should not be construed as limitations on the scope of any invention or of what may be claimed, but rather as descriptions of features of specific embodiments of particular inventions. Certain features that are described in this specification in the context of separate embodiments can also be implemented in combination in a single embodiment. On the other hand, the various features described in the individual embodiments may also be implemented separately in the various embodiments or in any suitable subcombination. Furthermore, although features may be acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.
Similarly, although operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In some cases, multitasking and parallel processing may be advantageous. Moreover, the separation of various system modules and components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.
Thus, particular embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve desirable results. Furthermore, the processes depicted in the accompanying drawings are not necessarily required to be in the particular order shown, or sequential order, to achieve desirable results. In some implementations, multitasking and parallel processing may be advantageous.
The foregoing description of the preferred embodiments of the present invention should not be taken as limiting the invention, but rather should be understood to cover all modifications, equivalents, alternatives, and improvements falling within the spirit and principles of the present invention.

Claims (10)

1. The function test method of the observability platform is characterized by comprising the following steps of:
s1, compiling and storing a self-defined test case;
s2, inputting a version to be regressed, calculating a regressive case of the version according to version information of the version and the test case, and forming a regressive case set;
s3, applying for computing resources according to the regression use case set, and creating Pod in real time according to the computing test resources;
s4, executing the regression use case in the Pod, testing the regression use case, obtaining a test result and caching the test result;
s5, reading the test result, summarizing the test related information, converting the test result into an inline protocol format, and outputting the inline protocol format to an observability platform for secondary processing and analysis.
2. The method according to claim 1, wherein in the step S3, the method for applying computing resources includes dynamically calling Kubernetes' virtual kubelet application computing resources.
3. The function test method according to claim 1, wherein in the step S3, the test resources include a name of the test resource, a type of the test resource, a configuration of the test resource, and environmental information of the test resource.
4. The method according to claim 1, wherein in the step S1, the definition information of the test case includes a test device, test case information, version control information, and custom tags.
5. The function test method according to claim 4, wherein the test-related information includes definition information of the test case, running environment information of the test case, and execution information of the test case.
6. The function test method according to claim 1, wherein in the step S4, the test result includes an API address, a parameter, an error message, a response time, and whether or not it is expected.
7. The function test method according to claim 1, wherein in the step S5, the secondary processing and analysis includes verifying whether the function of the observability platform is normal by calling an API.
8. An apparatus for observability platform function testing using a method according to any one of claims 1 to 7, comprising:
test case management module: storing a custom test case;
regression use case calculation module: inputting a version to be regressed, calculating a regressive case of the version according to version information of the version and the test case, and forming a regressive case set;
and (3) a Pod scheduling module: applying for computing resources according to the regression use case set, and creating Pod in real time according to the computing test resources;
the test result processing module: executing the regression use case in the Pod, testing the regression use case, obtaining a test result and caching;
and the data output module is used for: and reading the test result, summarizing the test related information, converting the test result into a row protocol format, and outputting the row protocol format to an observability platform for secondary processing and analysis.
9. A computer readable storage medium having stored thereon a computer program, characterized in that the program when executed implements the steps of the functional test method according to any of claims 1-7.
10. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the functional test method according to any of claims 1-7 when the program is executed by the processor.
CN202310045082.0A 2023-01-30 2023-01-30 Function test method and device for observability platform Pending CN116225911A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310045082.0A CN116225911A (en) 2023-01-30 2023-01-30 Function test method and device for observability platform

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310045082.0A CN116225911A (en) 2023-01-30 2023-01-30 Function test method and device for observability platform

Publications (1)

Publication Number Publication Date
CN116225911A true CN116225911A (en) 2023-06-06

Family

ID=86583678

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310045082.0A Pending CN116225911A (en) 2023-01-30 2023-01-30 Function test method and device for observability platform

Country Status (1)

Country Link
CN (1) CN116225911A (en)

Similar Documents

Publication Publication Date Title
US10083027B2 (en) Systems and methods for managing software development environments
CN108287694B (en) Application program construction method, system, computer device and storage medium
US9294296B2 (en) Automated test execution in a shared virtualized resource pool
Singh et al. Comparison of different CI/CD tools integrated with cloud platform
US10073683B2 (en) System and method for providing software build violation detection and self-healing
US7870169B2 (en) Method for enabling traceability and recovery from errors during migration of software applications
US20170123777A1 (en) Deploying applications on application platforms
US10387295B1 (en) Application testing using multiple context-aware threads
CN107783816A (en) The method and device that creation method and device, the big data cluster of virtual machine create
US9959157B1 (en) Computing instance migration
CN111290941A (en) Method and device for testing multiple interfaces, computing equipment and medium
US10552280B2 (en) In-band monitor in system management mode context for improved cloud platform availability
CN113448678A (en) Application information generation method, deployment method, device, system and storage medium
JP5400873B2 (en) Method, system, and computer program for identifying software problems
US20210191842A1 (en) Software bug reproduction
US9354962B1 (en) Memory dump file collection and analysis using analysis server and cloud knowledge base
CN116225911A (en) Function test method and device for observability platform
KR20170122874A (en) Apparatus for managing log of application based on data distribution service
Haeri et al. VNE-Sim: a virtual network embedding simulator
CN115617668A (en) Compatibility testing method, device and equipment
US10481993B1 (en) Dynamic diagnostic data generation
CN115237441A (en) Upgrade test method, device and medium based on cloud platform
CN114895916A (en) Code deployment method, device, storage medium and electronic equipment
CN114115982A (en) Code issuing method, device, equipment and storage medium
CN110908918A (en) Unit testing method and device for multiple interdependent node

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination