CN111694724A - Testing method and device of distributed table system, electronic equipment and storage medium - Google Patents
Testing method and device of distributed table system, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN111694724A CN111694724A CN201910197457.9A CN201910197457A CN111694724A CN 111694724 A CN111694724 A CN 111694724A CN 201910197457 A CN201910197457 A CN 201910197457A CN 111694724 A CN111694724 A CN 111694724A
- Authority
- CN
- China
- Prior art keywords
- abnormal
- file system
- local file
- exception
- operation function
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
Abstract
The embodiment of the invention discloses a testing method, a testing device, electronic equipment and a storage medium of a distributed form system, wherein the method comprises the following steps: responding to the call of a process to be tested in the distributed table system to a target rewriting operation function in the preloading library, and acquiring abnormal parameters of the local file system; acquiring local file system abnormal operation information corresponding to a target rewriting operation function from the abnormal parameters; and feeding back the abnormal operation information of the local file system to the process to be tested in the distributed table system. According to the test scheme of the distributed table system provided by the embodiment of the invention, the high-cost distributed file system abnormal structure is converted into the low-cost local file system abnormal structure, the cost of the abnormal structure is reduced, the accuracy of the abnormal structure is improved, and the scene coverage of the automatic abnormal test scheme is enlarged.
Description
Technical Field
The embodiment of the invention relates to the technical field of computers, in particular to a method and a device for testing a distributed table system, electronic equipment and a storage medium.
Background
With the continuous development of internet technology, a stand-alone Database is gradually unable to meet the requirements of high-performance and large-capacity storage of mass data, so that a scheme of a distributed table storage system adopting a layered architecture system is gradually known, for example, HBase (Hadoop Database) is one of open source implementations, the distributed table system of such a layered architecture is usually deployed on a distributed file system, all system metadata and actual data are stored in a bottom distributed file system, and a table layer only needs to pay attention to distributed index service. For example: both the Master and the RegionServer of the HBase use resources of a bottom-layer distributed file system HDFS (Hadoop file system).
However, the hierarchical architecture system causes the stability of the distributed table system to depend strongly on the stability of the underlying distributed file system, and the risk is amplified layer by layer, and a point of the underlying distributed file system fluctuates and may have a great influence on the upper table service. For example: if a small amount of read-write delay occurs in the HDFS, the HBase service may not be available in the entire system. Therefore, various abnormal conditions may occur in the bottom distributed file system, and the upper table service needs to be subjected to fault-tolerant processing as much as possible, otherwise the system cannot stably run for a long time. For example: after the capacity of the distributed file system is full, the table layer should be prohibited from writing; after the expansion is complete, the form level service should automatically resume. Therefore, the distributed table system with the layered architecture and the anomaly test of the lower storage layer are a big core in the whole quality assurance work.
However, in the existing test schemes of the distributed table system, some are difficult to test all the possible exceptions of the coverage distributed file system, some are configured with exceptions of specific scenes, the cost is high, even some scenes cannot be accurately configured, and the test process cannot be completely automated.
Disclosure of Invention
The embodiment of the invention provides a testing method and device of a distributed form system, electronic equipment and a storage medium, and aims to solve the technical problems that the existing testing scheme of the distributed form system is difficult to comprehensively test possible exceptions of a coverage distributed file system, the cost of constructing exceptions in a specific scene is high, parts of scenes cannot be accurately constructed, the testing process cannot be completely automated, and the like.
In a first aspect, an embodiment of the present invention provides a method for testing a distributed table system, including:
responding to the call of a process to be tested in the distributed table system to a target rewriting operation function in the preloading library, and acquiring abnormal parameters of the local file system;
acquiring local file system abnormal operation information corresponding to a target rewriting operation function from the abnormal parameters;
feeding back the abnormal operation information of the local file system to the process to be tested in the distributed table system;
the distributed table system is started in a preloading library mode and is deployed in the local file system, the preloading library comprises a plurality of rewriting operation functions, and each rewriting operation function is obtained by rewriting the local file system operation function based on an application layer.
In a second aspect, an embodiment of the present invention further provides a testing apparatus for a distributed table system, including:
the abnormal parameter acquisition module is used for responding to the call of a process to be tested in the distributed table system to a target rewriting operation function in the preloading library and acquiring the abnormal parameters of the local file system;
the abnormal operation information acquisition module is used for acquiring the local file system abnormal operation information corresponding to the target rewriting operation function from the abnormal parameters;
the first feedback module is used for feeding back the abnormal operation information of the local file system to the process to be tested in the distributed table system;
the distributed table system is started in a preloading library mode and is deployed in the local file system, the preloading library comprises a plurality of rewriting operation functions, and each rewriting operation function is obtained by rewriting the local file system operation function based on an application layer.
In a third aspect, an embodiment of the present invention further provides an electronic device, including:
one or more processors;
a memory for storing one or more programs,
when the one or more programs are executed by the one or more processors, the one or more processors implement the method for testing a distributed table system according to any embodiment of the present invention.
In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements a testing method for a distributed table system according to any embodiment of the present invention.
The embodiment of the invention provides a test method, a test device, electronic equipment and a storage medium of a distributed table system, which are characterized in that a process to be tested in the distributed table system is started based on a starting mode of a preloading library, the process to be tested calls a target rewriting operation function in the preloading library, a plurality of rewriting operation functions in the preloading library are obtained by rewriting a local file system operation function based on a system application layer, and the aim is to enable the target rewriting operation function to respond to the call of the process to be tested, obtain abnormal parameters of the local file system specified in advance by a tester, obtain abnormal operation information of the local file system corresponding to the target rewriting operation function from the abnormal parameters and feed the abnormal operation information back to the process to be tested, so that the abnormal process to be tested is processed by the distributed table system. Therefore, when the distributed table system is subjected to exception test, only the local file system needs to be subjected to exception construction, the exception construction cost is reduced, and any storage resource of the local file system can be subjected to accurate exception construction according to the test requirement, so that the scene coverage of an automatic exception test scheme is enlarged, and the automatic test is realized.
Drawings
Fig. 1 is a schematic flowchart illustrating a testing method of a distributed table system according to an embodiment of the present invention;
fig. 2 is a schematic flowchart of a testing method of a distributed table system according to a second embodiment of the present invention;
fig. 3 is a schematic structural diagram of a testing apparatus of a distributed table system according to a third embodiment of the present invention;
fig. 4 is a schematic structural diagram of an electronic device according to a fourth embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.
Example one
Fig. 1 is a flowchart of a testing method for a distributed table system according to an embodiment of the present invention, where the method is applicable to a case of performing an automatic exception test on a distributed table system with a hierarchical architecture, and the method may be executed by a corresponding testing apparatus for the distributed table system, where the apparatus may be implemented in a software and/or hardware manner, and may be configured on an electronic device, such as a computer terminal or a server.
As shown in fig. 1, the testing method of the distributed table system provided in the embodiment of the present invention may include:
and S110, responding to the call of the process to be tested in the distributed table system to the target rewriting operation function in the preloading library, and acquiring the abnormal parameters of the local file system.
The distributed table system is started in a preloading library mode and is deployed in the local file system, the preloading library comprises a plurality of rewriting operation functions, and each rewriting operation function is obtained by rewriting the local file system operation function based on an application layer.
Specifically, in order to adapt to different underlying storage resources, the distributed table system encapsulates a storage adaptation layer, and uses the same interface to implement operation invocation of different storages. Taking HBase as an example, the storage adaptation layer may be deployed on an HDFS cluster, or may be deployed in a stand-alone system using a local hard disk, and the directory structure and interface calling manner of the stand-alone environment and the cluster environment are completely consistent, and there are some differences only in a few characteristics. In the embodiment of the invention, the storage adaptation layer is utilized to deploy the distributed table system in the local file system, for example, a local disk of an electronic device of the distributed table system is deployed, so that the high-cost distributed file system abnormal structure is converted into the low-cost local file system abnormal structure, and the test cost is reduced.
In the Linux environment, the application layer uses a series of system functions, such as a local file system operation function, to actually operate the local file system, for example, a system function create is used to create a file, and a system function write is used to write data. The distributed table system is started based on a preloading library mode, namely the preloading library is automatically loaded when the distributed table system is started, the preloading library comprises a plurality of rewriting operation functions, each rewriting operation function is obtained by rewriting based on the local file system operation function, and the purpose is to enable the rewriting operation function to respond to the call of the process to be tested to obtain the abnormal parameters of the local file system.
Preferably, the rewrite operation function is a HOOK system function corresponding to the local file system operation function obtained by rewriting the local file system operation function of the application layer by using a HOOK mechanism. HOOK is a segment of a program that handles messages and is hooked into the system through a system call. Whenever a particular message is sent, the hook program captures the message before the destination window is reached, i.e. the hook function gets control. In this case, the hook function may process (change) the message, may continue to transfer the message without processing, or may forcibly end the transfer of the message. Through the HOOK mechanism, the rewrite operation function can respond to the call of the process to be tested to acquire the abnormal parameters of the local file system to complete the abnormal construction, so that the abnormal test of the distributed table system is realized.
The exception parameters are preset for the service scenario and the test requirements. The exception parameters may include at least an exception construction condition and at least one local file system exception operation information associated with the local file system operation function. For example, in a specific service scenario, different processes in the tested distributed table system need to be processed, and then according to the test requirements, for example, it may be specified which process or processes are to be constructed with an exception, a specific scenario corresponding to when an exception is constructed, that is, when what condition is satisfied, an exception is constructed, and condition information such as who constructs an exception, which may constitute the exception construction condition. For example, the exception parameter may be written in terms of "name of exception function-exception probability-exception error code or operation tamp down time" or "name of exception function-inode remaining percentage or memory space remaining percentage".
Common local file system exceptions include capacity fullness, inode (index node) exhaustion, read-write failure, read-write data errors, read-write slowness, and the like, so different exception parameters can be written for different test requirements, and the local file system operation functions of the same type can also correspond to a plurality of exception operation information, wherein the exception operation information refers to exception information fed back to a process to be tested, and for example, the local file system exception operation information related to the local file system operation function open can include two types of file opening errors and file opening and tamping. Preferably, it can also be written as an exception parameter in JSON format.
In an embodiment, the abnormal parameter may be written into a local shared memory of the electronic device through an HTTP server, and the rewrite operation function responds to a call of the process to be tested to obtain the abnormal parameter from the shared memory. The method is simple and convenient to operate and is convenient to integrate into an automatic test.
Specifically, the obtaining of the abnormal parameter of the local file system from the shared memory includes:
acquiring a file lock of the shared memory, wherein the file lock is used for locking read operation and write operation of programs except a target rewriting operation function on the shared memory;
acquiring abnormal parameters of a local file system from the shared memory;
and releasing the file lock.
It should be noted here that there is a possibility that the process of writing the abnormal parameter into the shared memory through the HTTP server and the process of obtaining the abnormal parameter from the shared memory by the target rewrite operation function may occur simultaneously in time. Therefore, in order to ensure the orderliness of the abnormal structure, it is necessary to lock the read operation and the write operation of the program other than the target rewrite operation function to the shared memory when acquiring the abnormal parameter according to the way of the file lock, then read the abnormal parameter from the shared memory, and then release the file lock, so that other abnormal parameters can be continuously written into the shared memory through the HTTP server.
And S120, acquiring the abnormal operation information of the local file system corresponding to the target rewriting operation function from the abnormal parameters.
Illustratively, if the target rewrite operation function is a rewritten read function, then the local file system abnormal operation information corresponding to the rewritten read function is obtained from the abnormal parameter, for example, read _ error and error code 5, or read _ ringing and ringing time 0.2 s; if the target rewrite operation function is a rewritten statfs function, then the local file system abnormal operation information corresponding to the target rewrite operation function is obtained from the abnormal parameters, such as the storage space remaining percentage and the inode remaining percentage of the local file system. The target rewrite operation function is rewritten read and rewritten statfs, which are only examples, and may also be other rewrite operation functions corresponding to the local file system operation function based on the application layer, and is not limited in detail herein.
S130, feeding back the abnormal operation information of the local file system to the process to be tested in the distributed table system.
Exemplarily, the local file system abnormal operation information corresponding to the target rewrite operation function acquired in S120 is fed back to the process to be tested, so as to implement the local file system abnormal structure for the process to be tested. And then the distributed table system performs corresponding exception handling for the abnormal process to be tested. Therefore, whether the processing of the distributed form system in the abnormal period is in accordance with the expectation can be verified according to the processing result of the distributed form system on the abnormal process to be tested.
It should be noted here that, for the problem of stability test of the distributed table system with a layered architecture, the following 3 schemes are mainly adopted in the prior art:
according to the scheme A, various management operations and abnormal conditions possibly occurring on line are released in a distributed file system layer, a use scene of covering upper-layer table service by an automatic test Case is compiled, and a scene with damage to the distributed table system service is combed;
according to the scheme B, a storage system is built together to integrate a test environment, service scenes and tolerable abnormal conditions of respective systems are put in layers, a certain background pressure is put in at the same time, and the stability of the system is tested and verified from the overall perspective;
and the scheme C is used for making an operation plan and performing drilling regularly aiming at the file system abnormity which cannot be covered by the distributed table system.
However, the three system integration test schemes mainly have the following problems: (1) according to the first scheme, possible abnormity of a coverage distributed file system is difficult to test comprehensively, due to the fact that the system has large-degree fault tolerance in design, and due to the abnormity of a small number of nodes or modules, the service of an upper-layer table system cannot be influenced under normal conditions; (2) in the second scheme, the construction of the exception of a specific scenario is costly, for example: constructing a scene of error check of the underlying data crc, reading correct data through a specific tool, constructing a file path of the previous data after backup, wherein part of the data is abnormal, and restoring the backup data after the abnormal test is finished; (3) some scenes cannot be constructed accurately, for example: the reading and writing of the file system are slow, the overtime time limit of the table layer is exceeded, and the scene cannot be accurately constructed in a mode of increasing the pressure through a pressure measuring tool; (4) the testing process cannot be completely automated.
Compared with the existing distributed table system stability test scheme aiming at the layered architecture, the technical scheme of the embodiment of the invention starts the process to be tested in the distributed table system based on the starting mode of the preloading library, so that the process to be tested calls the target rewriting operation function in the preloading library, and the plurality of rewriting operation functions in the preloading library are obtained by rewriting the local file system operation function based on the system application layer, so that the target rewriting operation function responds to the call of the process to be tested, the abnormal parameters of the local file system specified in advance by a tester are obtained, the abnormal operation information of the local file system corresponding to the target rewriting operation function is obtained from the abnormal parameters and fed back to the process to be tested, and the distributed table system can perform abnormal processing on the abnormal process to be tested. Therefore, when the distributed table system is subjected to exception test, only the local file system needs to be subjected to exception construction, the exception construction cost is reduced, and all storage resources of the local file system can be subjected to accurate exception construction according to the test requirement, so that the scene coverage of an automatic exception test scheme is enlarged.
Example two
Fig. 2 is a flowchart illustrating a testing method of a distributed table system according to a second embodiment of the present invention. In this embodiment, optimization is performed based on the above embodiment, and as shown in fig. 2, the test method of the distributed table system provided in the embodiment of the present invention may include:
s210, responding to the call of the process to be tested in the distributed table system to the target rewriting operation function in the preloading library, and obtaining the abnormal parameters of the local file system.
The distributed table system is started in a preloading library mode and is deployed in the local file system, the preloading library comprises a plurality of rewriting operation functions, and each rewriting operation function is obtained by rewriting the local file system operation function based on an application layer.
In the embodiment of the present invention, the exception parameter includes at least one local file system exception operation information related to a local file system operation function, and also includes an exception configuration condition. In a specific business scenario, many processes are required for processing, so that a target process which needs to be abnormally constructed in the multiple processes can be judged through an abnormal construction condition. The abnormal construction condition comprises an abnormal effective process identification or an abnormal effective process name and process path, an abnormal operation type, an abnormal file list and an abnormal occurrence probability. The process ID which takes effect abnormally is preferably a process ID, and as the process ID can uniquely identify one process, the process ID can be used for designating the process which needs to construct the abnormality; the process name and the process path which take effect abnormally can jointly identify a process, so that the process which needs to be constructed abnormally can be specified through the process name and the process path which take effect abnormally; the abnormal operation types comprise write failure, write slow, read failure, table building failure and the like; the exception file list is used for storing files needing exception processing of the process to be tested and can be set according to actual test requirements.
S220, judging whether the process to be tested in the distributed table system meets the abnormal construction condition or not according to the abnormal parameters, if so, executing S230-S240, and otherwise, executing S250.
And judging whether the process to be detected meets the abnormal construction condition according to the abnormal parameters, wherein the process to be detected can be sequentially matched with the process identifier which takes effect abnormally or the process name which takes effect abnormally and the process path, as well as the abnormal operation type, the abnormal file list and the abnormal occurrence probability.
Specifically, the determination may be performed according to the following determination procedure:
s1, acquiring the process ID of the process to be tested or the process name and the process path of the process to be tested, matching the process ID with the abnormally effective process identifier or the abnormally effective process name with the process path, if the matching is successful, indicating that the current process to be tested is the target process to be constructed abnormally, and continuing to execute S2, and if the matching is failed, indicating that the process to be tested does not need the abnormal construction, and directly executing S250.
S2, determining whether the abnormal operation types are matched, for example, if the operation executed by the current process to be tested is a read operation, and if the abnormal operation type matched with the process path through the process identifier or the process name in effect is a write failure, the two are not matched, and executing S250; if the abnormal operation types match, execution continues with S3.
And S3, judging whether the file in the local file system to be operated by the process to be tested is located in the abnormal file list, if so, continuing to execute S4, otherwise, executing S250.
And S4, judging whether the abnormal occurrence probability meets the parameter setting, such as the abnormal occurrence probability or the storage space residual percentage. If yes, determining that the process to be tested meets the abnormal construction condition, and executing S230-S240 to complete the abnormal construction; otherwise, S250 is performed.
And S230, acquiring the abnormal operation information of the local file system corresponding to the target rewriting operation function from the abnormal parameters.
S240, feeding back the abnormal operation information of the local file system to the process to be tested in the distributed table system.
And S250, feeding back the normal operation information of the local file system corresponding to the target rewriting operation function to the process to be tested in the distributed table system.
And if the process to be tested does not meet the abnormal construction condition, feeding back the normal operation information of the local file system corresponding to the target rewriting operation function to the process to be tested, for example, if the process to be tested reads a certain file in the local file system, feeding back the detailed information corresponding to the file to the process to be tested.
On the basis of the above, two specific abnormal test cases are provided as follows:
scheme one, fuzz testing.
Simultaneously and randomly putting various operations in a distributed table system to actually operate a local file system, such as table creation, data reading and writing, table deletion, data sorting, snapshot, splitting, merging and the like; the method comprises the steps of randomly selecting to put exceptions into metadata of a distributed table system, WAL (Write-Ahead Logging) and files of actual data, randomly selecting the thrown exceptions from various system operation functions, and selecting exception types and exception values in a reasonable range, wherein the exception types and the exception values are selected, for example, the files of/home/disk 1/data/master data in a local file system are written and tamped, and the tamping time is 0.02s (namely, the exception values). In the whole fuzzy test, whether the system has the problems of system crash and the like under various service scenes and abnormal scenes is checked, whether the system can be normally recovered after abnormal recovery is verified, and a defect point of system fault-tolerant processing is found.
The test scheme has the advantages that the input-output ratio is high, more and more new problems can be solved through long-term automatic test, and the defect of the scheme is that regression test of the problems cannot be accurately carried out.
And a second scheme is precise exception testing.
The scheme is a supplement of the fuzz test and is used for accurately verifying whether the problems found by the fuzz test, the on-line problems and the like are repaired. Taking HBase as an example, a typical precise anomaly testing procedure is as follows: performing a table building operation in the distributed table system, responding to the call of a table building process to a Linux system function related to the table building operation, enabling the function to select an abnormal parameter with the delay of 30s and the duration of 2 minutes from the abnormal parameters of the local file system, wherein the path corresponding to the parameter is the path of the Wal log, so that the purpose of putting the abnormality with the write delay of 30s and the duration of 2 minutes to the path of the Wal log is realized, and reading and writing data from a table layer for verification: verifying whether the treatment of the HBase in the abnormal period is in accordance with expectation or not and verifying whether the HBase is recovered to be normal or not after the abnormal recovery.
The test scheme can finely control the abnormal files, abnormal operation types, abnormal values and duration of the storage layer, and is convenient for quick regression of known problems.
According to the technical scheme of the embodiment of the invention, the high-cost distributed file system abnormal structure is converted into the low-cost local file system abnormal structure, and whether the process to be tested meets the abnormal structure condition is judged, so that the influence range can be accurately controlled in the abnormal structure process, the abnormal information can be fed back to the specified tested service, no influence is caused on an operating system, hardware or other services on the same machine, and the process to be tested does not need to be changed.
EXAMPLE III
Fig. 3 is a schematic structural diagram of a testing apparatus of a distributed table system according to a third embodiment of the present invention. As shown in fig. 3, the apparatus includes:
the abnormal parameter obtaining module 310 is used for responding to the call of the process to be tested in the distributed table system to the target rewriting operation function in the preloading library and obtaining the abnormal parameters of the local file system;
an abnormal operation information obtaining module 320, configured to obtain, from the abnormal parameter, local file system abnormal operation information corresponding to the target rewrite operation function;
the first feedback module 330 is configured to feed back the abnormal operation information of the local file system to the process to be tested in the distributed table system.
The distributed table system is started in a preloading library mode and is deployed in the local file system, the preloading library comprises a plurality of rewriting operation functions, and each rewriting operation function is obtained by rewriting the local file system operation function based on an application layer.
In the embodiment of the invention, when the process to be tested calls the target rewriting operation function in the preloading library, the abnormal operation information acquisition module acquires the local file system abnormal operation information corresponding to the target rewriting operation function from the abnormal parameter acquisition module and feeds the local file system abnormal operation information back to the process to be tested through the first feedback module, so that the abnormal structure of the process to be tested is realized, and the high-cost distributed file system abnormal structure is changed into the low-cost and accurate local file system abnormal structure.
On the basis of the above embodiment, the rewrite operation function is a HOOK system function corresponding to a local file system operation function of an application layer, which is obtained by rewriting the local file system operation function by using a HOOK mechanism.
On the basis of the above embodiment, the abnormal parameter acquiring module includes:
the device comprises an acquisition unit, a storage unit and a processing unit, wherein the acquisition unit is used for acquiring abnormal parameters of a local file system from a shared memory, and the abnormal parameters are written into the shared memory through an HTTP server.
On the basis of the foregoing embodiment, the obtaining unit is specifically configured to:
acquiring a file lock of the shared memory, wherein the file lock is used for locking read operation and write operation of processes except a target rewriting operation function on the shared memory;
acquiring abnormal parameters of a local file system from the shared memory;
and releasing the file lock.
On the basis of the above embodiment, the exception parameters at least include an exception construction condition and at least one local file system exception operation information related to the local file system operation function;
correspondingly, the abnormal operation information obtaining module includes:
the judging unit is used for judging whether the process to be detected in the distributed table system meets the abnormal construction condition or not according to the abnormal parameters;
and the abnormal operation information acquisition unit is used for acquiring the local file system abnormal operation information corresponding to the target rewriting operation function from the abnormal parameters when the judgment unit determines that the abnormal construction condition is met.
On the basis of the above embodiment, the exception construction condition at least includes an exception-enabled process identifier or an exception-enabled process name and process path, and an exception operation type, an exception file list, and an exception occurrence probability.
On the basis of the above embodiment, the apparatus further includes:
and the second feedback module is used for feeding back the normal operation information of the local file system corresponding to the target rewriting operation function to the process to be tested in the distributed table system when the judging unit determines that the abnormal construction condition is not met.
The testing device of the distributed form system provided by the embodiment of the invention can execute the testing method of the distributed form system provided by any embodiment of the invention, and has corresponding functional modules and beneficial effects of the execution method.
Example four
Fig. 4 is a schematic structural diagram of an electronic device according to a fourth embodiment of the present invention. FIG. 4 illustrates a block diagram of an exemplary electronic device 12 suitable for use in implementing embodiments of the present invention. The electronic device 12 shown in fig. 4 is only an example and should not bring any limitation to the function and the scope of use of the embodiment of the present invention.
As shown in FIG. 4, electronic device 12 is embodied in the form of a general purpose computing device. The components of electronic device 12 may include, but are not limited to: one or more processors or processors 16, a memory 28, and a bus 18 that connects the various system components (including the memory 28 and the processors 16).
The memory 28 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM)30 and/or cache memory 32. The electronic device 12 may further include other removable/non-removable, volatile/nonvolatile computer system storage media. By way of example only, storage system 34 may be used to read from and write to non-removable, nonvolatile magnetic media (not shown in FIG. 4, and commonly referred to as a "hard drive"). Although not shown in FIG. 4, a magnetic disk drive for reading from and writing to a removable, nonvolatile magnetic disk (e.g., a "floppy disk") and an optical disk drive for reading from or writing to a removable, nonvolatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 by one or more data media interfaces. Memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of embodiments of the invention.
A program/utility 40 having a set (at least one) of program modules 42 may be stored, for example, in memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which examples or some combination thereof may comprise an implementation of a network environment. Program modules 42 generally carry out the functions and/or methodologies of the described embodiments of the invention.
The processor 16 executes various functional applications and data processing by executing programs stored in the memory 28, for example, to implement the test method of the distributed table system provided by the embodiment of the present invention, including:
responding to the call of a process to be tested in the distributed table system to a target rewriting operation function in the preloading library, and acquiring abnormal parameters of the local file system;
acquiring local file system abnormal operation information corresponding to a target rewriting operation function from the abnormal parameters;
feeding back the abnormal operation information of the local file system to the process to be tested in the distributed table system;
the distributed table system is started in a preloading library mode and is deployed in the local file system, the preloading library comprises a plurality of rewriting operation functions, and each rewriting operation function is obtained by rewriting the local file system operation function based on an application layer.
EXAMPLE five
In an embodiment of the invention, there is provided a storage medium containing computer executable instructions which, when executed by a computer processor, perform a method of testing a distributed table system, the method comprising:
responding to the call of a process to be tested in the distributed table system to a target rewriting operation function in the preloading library, and acquiring abnormal parameters of the local file system;
acquiring local file system abnormal operation information corresponding to a target rewriting operation function from the abnormal parameters;
feeding back the abnormal operation information of the local file system to the process to be tested in the distributed table system;
the distributed table system is started in a preloading library mode and is deployed in the local file system, the preloading library comprises a plurality of rewriting operation functions, and each rewriting operation function is obtained by rewriting the local file system operation function based on an application layer.
Of course, the storage medium provided in the embodiment of the present invention contains computer-executable instructions, and the computer-executable instructions are not limited to the method operations described above, and may also perform related operations in the test of the distributed table system provided in any embodiment of the present invention.
Computer storage media for embodiments of the invention may employ any combination of one or more computer-readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims (16)
1. A method of testing a distributed table system, the method comprising:
responding to the call of a process to be tested in the distributed table system to a target rewriting operation function in the preloading library, and acquiring abnormal parameters of the local file system;
acquiring local file system abnormal operation information corresponding to a target rewriting operation function from the abnormal parameters;
feeding back the abnormal operation information of the local file system to the process to be tested in the distributed table system;
the distributed table system is started in a preloading library mode and is deployed in the local file system, the preloading library comprises a plurality of rewriting operation functions, and each rewriting operation function is obtained by rewriting the local file system operation function based on an application layer.
2. The method of claim 1, wherein the rewrite operation function is a HOOK system function corresponding to a native file system operation function of an application layer by rewriting the native file system operation function using a HOOK mechanism.
3. The method according to claim 1 or 2, wherein the obtaining of the exception parameter of the local file system comprises:
obtaining abnormal parameters of a local file system from a shared memory, wherein the abnormal parameters are written into the shared memory through an HTTP server.
4. The method of claim 3, wherein the obtaining the exception parameter of the local file system from the shared memory comprises:
acquiring a file lock of the shared memory, wherein the file lock is used for locking read operation and write operation of processes except a target rewriting operation function on the shared memory;
acquiring abnormal parameters of a local file system from the shared memory;
and releasing the file lock.
5. The method according to claim 1 or 2, wherein the exception parameters at least comprise exception construction conditions and at least one local file system exception operation information related to the local file system operation function;
correspondingly, obtaining the local file system abnormal operation information corresponding to the target rewrite operation function from the abnormal parameter includes:
judging whether the process to be tested in the distributed table system meets the abnormal construction condition or not according to the abnormal parameters;
and if so, acquiring the abnormal operation information of the local file system corresponding to the target rewriting operation function from the abnormal parameters.
6. The method of claim 5, wherein the exception construction condition comprises at least an identification of the process in which the exception is valid, or a name and a path of the process in which the exception is valid, and an exception operation type, an exception file list, and an exception occurrence probability.
7. The method of claim 5, further comprising:
and if the abnormal construction condition is not met, feeding back the normal operation information of the local file system corresponding to the target rewriting operation function to the process to be tested in the distributed table system.
8. An apparatus for testing a distributed table system, the apparatus comprising:
the abnormal parameter acquisition module is used for responding to the call of a process to be tested in the distributed table system to a target rewriting operation function in the preloading library and acquiring the abnormal parameters of the local file system;
the abnormal operation information acquisition module is used for acquiring the local file system abnormal operation information corresponding to the target rewriting operation function from the abnormal parameters;
the first feedback module is used for feeding back the abnormal operation information of the local file system to the process to be tested in the distributed table system;
the distributed table system is started in a preloading library mode and is deployed in the local file system, the preloading library comprises a plurality of rewriting operation functions, and each rewriting operation function is obtained by rewriting the local file system operation function based on an application layer.
9. The apparatus of claim 8, wherein the rewrite operation function is a HOOK system function corresponding to a native file system operation function of an application layer by rewriting the native file system operation function using a HOOK mechanism.
10. The apparatus according to claim 8 or 9, wherein the anomaly parameter obtaining module comprises:
the device comprises an acquisition unit, a storage unit and a processing unit, wherein the acquisition unit is used for acquiring abnormal parameters of a local file system from a shared memory, and the abnormal parameters are written into the shared memory through an HTTP server.
11. The apparatus according to claim 10, wherein the obtaining unit is specifically configured to:
acquiring a file lock of the shared memory, wherein the file lock is used for locking read operation and write operation of processes except a target rewriting operation function on the shared memory;
acquiring abnormal parameters of a local file system from the shared memory;
and releasing the file lock.
12. The apparatus according to claim 8 or 9, wherein the exception parameters at least include an exception construction condition and at least one local file system exception operation information related to the local file system operation function;
correspondingly, the abnormal operation information obtaining module includes:
the judging unit is used for judging whether the process to be detected in the distributed table system meets the abnormal construction condition or not according to the abnormal parameters;
and the abnormal operation information acquisition unit is used for acquiring the local file system abnormal operation information corresponding to the target rewriting operation function from the abnormal parameters when the judgment unit determines that the abnormal construction condition is met.
13. The apparatus of claim 12, wherein the exception construction condition at least comprises an identification of the process in which the exception takes effect or a name and a path of the process in which the exception takes effect, and an exception operation type, an exception file list, and an exception occurrence probability.
14. The apparatus of claim 12, further comprising:
and the second feedback module is used for feeding back the normal operation information of the local file system corresponding to the target rewriting operation function to the process to be tested in the distributed table system when the judging unit determines that the abnormal construction condition is not met.
15. An electronic device, comprising:
one or more processors;
a memory for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement a method of testing a distributed table system as claimed in any one of claims 1 to 7.
16. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out a method of testing a distributed table system according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910197457.9A CN111694724B (en) | 2019-03-15 | 2019-03-15 | Test method and device of distributed form system, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910197457.9A CN111694724B (en) | 2019-03-15 | 2019-03-15 | Test method and device of distributed form system, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111694724A true CN111694724A (en) | 2020-09-22 |
| CN111694724B CN111694724B (en) | 2023-05-05 |
Family
ID=72475898
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910197457.9A Active CN111694724B (en) | 2019-03-15 | 2019-03-15 | Test method and device of distributed form system, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111694724B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113849098A (en) * | 2021-11-29 | 2021-12-28 | 北京蔚领时代科技有限公司 | Hook-based method and device for remotely setting cursor |
| CN116069638A (en) * | 2023-01-19 | 2023-05-05 | 蔷薇大树科技有限公司 | Method for simulating distributed abnormal state based on kernel mode |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2728482A1 (en) * | 2012-11-05 | 2014-05-07 | Realworld Holding B.V. | Method and arrangement for collecting timing data related to a computer application |
| CN104008420A (en) * | 2014-05-26 | 2014-08-27 | 中国科学院信息工程研究所 | Distributed outlier detection method and system based on automatic coding machine |
| US20140289564A1 (en) * | 2012-06-29 | 2014-09-25 | Vmware, Inc. | System and method for injecting faults into code for testing thereof |
| CN104461865A (en) * | 2014-11-04 | 2015-03-25 | 哈尔滨工业大学 | Cloud environment distributed file system reliability test suite |
| CN105426310A (en) * | 2015-11-27 | 2016-03-23 | 北京奇虎科技有限公司 | Method and apparatus for detecting performance of target process |
| CN106227656A (en) * | 2016-07-18 | 2016-12-14 | 中国科学院软件研究所 | A kind of data contention sample detecting method based on hardware and thread-local sequential |
| CN107239405A (en) * | 2017-08-03 | 2017-10-10 | 四川长虹电器股份有限公司 | A kind of automated testing method based on many browser kernels of Android |
| CN108334447A (en) * | 2017-12-31 | 2018-07-27 | 江苏易润信息技术有限公司 | A kind of system and method for test processes computer software exception |
| CN108563562A (en) * | 2018-03-22 | 2018-09-21 | 平安科技（深圳）有限公司 | Test method, device, computer equipment and the storage medium of distributed system |
| CN108959055A (en) * | 2018-07-25 | 2018-12-07 | 郑州云海信息技术有限公司 | Readwrite performance test method, device and the equipment of xfs file system |
-
2019
- 2019-03-15 CN CN201910197457.9A patent/CN111694724B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140289564A1 (en) * | 2012-06-29 | 2014-09-25 | Vmware, Inc. | System and method for injecting faults into code for testing thereof |
| EP2728482A1 (en) * | 2012-11-05 | 2014-05-07 | Realworld Holding B.V. | Method and arrangement for collecting timing data related to a computer application |
| CN104008420A (en) * | 2014-05-26 | 2014-08-27 | 中国科学院信息工程研究所 | Distributed outlier detection method and system based on automatic coding machine |
| CN104461865A (en) * | 2014-11-04 | 2015-03-25 | 哈尔滨工业大学 | Cloud environment distributed file system reliability test suite |
| CN105426310A (en) * | 2015-11-27 | 2016-03-23 | 北京奇虎科技有限公司 | Method and apparatus for detecting performance of target process |
| CN106227656A (en) * | 2016-07-18 | 2016-12-14 | 中国科学院软件研究所 | A kind of data contention sample detecting method based on hardware and thread-local sequential |
| CN107239405A (en) * | 2017-08-03 | 2017-10-10 | 四川长虹电器股份有限公司 | A kind of automated testing method based on many browser kernels of Android |
| CN108334447A (en) * | 2017-12-31 | 2018-07-27 | 江苏易润信息技术有限公司 | A kind of system and method for test processes computer software exception |
| CN108563562A (en) * | 2018-03-22 | 2018-09-21 | 平安科技（深圳）有限公司 | Test method, device, computer equipment and the storage medium of distributed system |
| CN108959055A (en) * | 2018-07-25 | 2018-12-07 | 郑州云海信息技术有限公司 | Readwrite performance test method, device and the equipment of xfs file system |
Non-Patent Citations (2)
| Title |
|---|
| BINBIN QU等: "Memory Leak Dynamic Monitor Based On HOOK Technique", 《2009 INTERNATIONAL CONFERENCE ON COMPUTATIONAL INTELLIGENCE AND SOFTWARE ENGINEERING》 * |
| 牟宗磊等: "分布式测试系统时间同步技术的研究", 《计算机测量与控制》 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113849098A (en) * | 2021-11-29 | 2021-12-28 | 北京蔚领时代科技有限公司 | Hook-based method and device for remotely setting cursor |
| CN116069638A (en) * | 2023-01-19 | 2023-05-05 | 蔷薇大树科技有限公司 | Method for simulating distributed abnormal state based on kernel mode |
| CN116069638B (en) * | 2023-01-19 | 2023-09-01 | 蔷薇大树科技有限公司 | Method for simulating distributed abnormal state based on kernel mode |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111694724B (en) | 2023-05-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7908521B2 (en) | Process reflection | |
| CN111610989B (en) | Application publishing/updating method and system for offline container cloud environment | |
| US20170147469A1 (en) | Correlation of source code with system dump information | |
| CN110088744B (en) | Database maintenance method and system | |
| CN110063042B (en) | Database fault response method and terminal thereof | |
| CN111158741B (en) | Method and device for monitoring dependency relationship change of service module on third party class library | |
| CN114443332B (en) | Storage pool detection method and device, electronic equipment and storage medium | |
| US10635575B2 (en) | Testing of enterprise resource planning systems | |
| CN110851471A (en) | Distributed log data processing method, device and system | |
| CN111694684B (en) | Abnormal construction method and device of storage device, electronic device and storage medium | |
| CN111694724B (en) | Test method and device of distributed form system, electronic equipment and storage medium | |
| CN112256532A (en) | Test interface generation method and device, computer equipment and readable storage medium | |
| CN111324668B (en) | Database data synchronous processing method, device and storage medium | |
| CN113467941A (en) | Method and device for sharing information | |
| US20160275002A1 (en) | Image capture in application lifecycle management for documentation and support | |
| CN112751782B (en) | Flow switching method, device, equipment and medium based on multi-activity data center | |
| CN113094251A (en) | Embedded system testing method and device, computer equipment and storage medium | |
| KR101449202B1 (en) | System and method for data copy and restoration in steel process middleware | |
| CN110008114B (en) | Configuration information maintenance method, device, equipment and readable storage medium | |
| KR102145403B1 (en) | Method for application monitoring in smart devices by big data analysis of excption log | |
| CN116149707B (en) | Method and device for detecting and avoiding upgrading risk of distributed system | |
| US20050108194A1 (en) | System for verifying a state of an environment | |
| CN117407460A (en) | Cross-environment data synchronization method and device, storage medium and electronic equipment | |
| CN114077630A (en) | ETL task execution method, system, device, server and storage medium | |
| CN113986593A (en) | Method for checking fault of electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |