CN112699046B - Application program testing method and device, electronic equipment and storage medium - Google Patents
Application program testing method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN112699046B CN112699046B CN202110035303.7A CN202110035303A CN112699046B CN 112699046 B CN112699046 B CN 112699046B CN 202110035303 A CN202110035303 A CN 202110035303A CN 112699046 B CN112699046 B CN 112699046B
- Authority
- CN
- China
- Prior art keywords
- test
- page
- current page
- distribution
- actions
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012360 testing method Methods 0.000 claims abstract description 437
- 238000009826 distribution Methods 0.000 claims abstract description 194
- 230000009471 action Effects 0.000 claims abstract description 166
- 238000000034 method Methods 0.000 claims abstract description 81
- 230000008569 process Effects 0.000 claims abstract description 47
- 230000006399 behavior Effects 0.000 claims description 179
- 230000001960 triggered effect Effects 0.000 claims description 21
- 230000006870 function Effects 0.000 description 16
- 101100055496 Arabidopsis thaliana APP2 gene Proteins 0.000 description 5
- 101100016250 Saccharomyces cerevisiae (strain ATCC 204508 / S288c) GYL1 gene Proteins 0.000 description 5
- 238000013461 design Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 5
- 101150053844 APP1 gene Proteins 0.000 description 4
- 101100189105 Homo sapiens PABPC4 gene Proteins 0.000 description 4
- 102100039424 Polyadenylate-binding protein 4 Human genes 0.000 description 4
- 230000004044 response Effects 0.000 description 4
- 238000004891 communication Methods 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- FFBHFFJDDLITSX-UHFFFAOYSA-N benzyl N-[2-hydroxy-4-(3-oxomorpholin-4-yl)phenyl]carbamate Chemical compound OC1=C(NC(=O)OCC2=CC=CC=C2)C=CC(=C1)N1CCOCC1=O FFBHFFJDDLITSX-UHFFFAOYSA-N 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000010295 mobile communication Methods 0.000 description 2
- 238000010998 test method Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 239000006185 dispersion Substances 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 238000009827 uniform distribution Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3688—Test management for test execution, e.g. scheduling of test suites
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
- G06F18/232—Non-hierarchical techniques
- G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The invention provides an application program testing method and device, electronic equipment and a storage medium, wherein the method comprises the following steps: acquiring a test behavior sequence generated in an automatic test process aiming at a target application, wherein the test behavior sequence comprises a history page and test actions executed on the history page; according to the test behavior sequence and the current page, predicting probability distribution of a plurality of test actions supported by the current page through a pre-constructed multi-mode user behavior prediction model; and controlling to execute corresponding test actions on the current page of the target application according to the probability distribution. By the technical scheme, the accuracy of the application test result can be improved.
Description
[ field of technology ]
The present invention relates to the field of information processing technologies, and in particular, to an application program testing method and apparatus, an electronic device, and a storage medium.
[ background Art ]
The application test based on the user behavior can adopt a manual test mode or a script test mode. There are also tests by modeling the user, etc. The manual test mode has a high test range, but consumes a great deal of labor cost and time cost, and has low test efficiency. The script test mode needs to write the script manually, the more abundant the applied functions, the more complex the user behaviors possibly occur, the higher the complexity of the required script, and the lower the efficiency and accuracy of writing the script manually, thereby influencing the test efficiency and the accuracy of the test result.
Therefore, how to efficiently and high-quality complete the application test work is a technical problem to be solved at present.
[ invention ]
The embodiment of the invention provides an application program testing method and device, electronic equipment and storage medium, and aims to solve the technical problems of low efficiency, low accuracy of testing results and the like in application testing in related technologies.
In a first aspect, an embodiment of the present invention provides an application program testing method, including: acquiring a test behavior sequence generated in an automatic test process aiming at a target application, wherein the test behavior sequence comprises a history page and test actions executed on the history page; according to the test behavior sequence and the current page, predicting probability distribution of a plurality of test actions supported by the current page through a pre-constructed multi-mode user behavior prediction model; and controlling to execute corresponding test actions on the current page of the target application according to the probability distribution.
In the above embodiment of the present invention, optionally, the multi-modal user behavior prediction model takes a target test behavior sequence including three consecutive pages and actions and a next page associated with the three consecutive pages as model inputs, and the multi-modal user behavior prediction model takes probability distribution of test actions supported by the next page associated with the three consecutive pages as model outputs; the test behavior sequence includes three pages tested before the current page and test actions performed on the three pages.
In the foregoing embodiment of the present invention, optionally, predicting, according to the test behavior sequence and the current page, a probability distribution of a plurality of test actions supported by the current page by a multi-mode user behavior prediction model constructed in advance includes: outputting a plurality of groups of Gaussian distributions through the pre-constructed multi-mode user behavior prediction model according to the test behavior sequence and the current page, wherein each group of Gaussian distributions comprises weights, expectations and variances; the controlling, according to the probability distribution, the execution of the corresponding test action on the current page of the target application includes: taking the weight of each group of Gaussian distribution as the probability of selecting each group of Gaussian distribution, and selecting any group of Gaussian distribution from the plurality of groups of Gaussian distribution; and determining a predicted value corresponding to the selected set of Gaussian distributions based on the test behavior sequence, the current page and the expected and variance of the selected set of Gaussian distributions, wherein the predicted value is used for reflecting the possibility that the test action supported by the current page and corresponding to the selected set of Gaussian distributions is triggered.
In the foregoing embodiment of the present invention, optionally, the controlling, according to the probability distribution, performing a corresponding test action on the current page of the target application includes: distributing different test actions to each test client in a plurality of test clients according to the probability distribution; and controlling the plurality of test clients to execute the allocated test actions aiming at the current page of the target application in parallel.
In the foregoing embodiment of the present invention, optionally, the controlling, according to the probability distribution, performing a corresponding test action on the current page of the target application includes: creating a plurality of sub-processes for the current test process by copying the test process; distributing different test actions to the plurality of sub-process distributions according to the probability distribution; and controlling the plurality of subprocesses to execute corresponding test actions aiming at the current page of the target application in parallel.
In a second aspect, an embodiment of the present invention provides an application program testing apparatus, including: the system comprises a test behavior sequence acquisition unit, a test behavior sequence generation unit and a test execution unit, wherein the test behavior sequence acquisition unit is used for acquiring a test behavior sequence generated in an automatic test process aiming at a target application, and the test behavior sequence comprises a history page and test actions executed on the history page; the probability distribution determining unit is used for predicting probability distribution of a plurality of test actions supported by the current page through a pre-constructed multi-mode user behavior prediction model according to the test behavior sequence and the current page; and the test action control unit is used for controlling the execution of the corresponding test action on the current page of the target application according to the probability distribution.
In the above embodiment of the present invention, optionally, the multi-modal user behavior prediction model takes a target test behavior sequence including three consecutive pages and actions and a next page associated with the three consecutive pages as model inputs, and the multi-modal user behavior prediction model takes probability distribution of test actions supported by the next page associated with the three consecutive pages as model outputs; the test behavior sequence includes three pages tested before the current page and test actions performed on the three pages.
In the above embodiment of the present invention, optionally, the probability distribution determining unit is configured to: outputting a plurality of groups of Gaussian distributions through the pre-constructed multi-mode user behavior prediction model according to the test behavior sequence and the current page, wherein each group of Gaussian distributions comprises weights, expectations and variances; the test action control unit is used for: and selecting any group of Gaussian distribution in the plurality of groups of Gaussian distribution by taking the weight of each group of Gaussian distribution as the probability of selecting each group of Gaussian distribution, and determining a predicted value corresponding to the selected Gaussian distribution based on the test behavior sequence, the current page and the expected and variance of the selected Gaussian distribution, wherein the predicted value is used for reflecting the possibility that the test action supported by the current page and corresponding to the selected group of Gaussian distribution is triggered.
In the above embodiment of the present invention, optionally, the test action control unit is configured to: distributing different test actions to each test client in a plurality of test clients according to the probability distribution; and controlling the plurality of test clients to execute the allocated test actions aiming at the current page of the target application in parallel.
In the above embodiment of the present invention, optionally, the test action control unit is configured to: creating a plurality of sub-processes for the current test process by copying the test process; distributing different test actions to the plurality of sub-process distributions according to the probability distribution; and controlling the plurality of subprocesses to execute corresponding test actions aiming at the current page of the target application in parallel.
In a third aspect, an embodiment of the present invention provides an electronic device, including: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being arranged to perform the method of any of the first aspects above.
In a fourth aspect, an embodiment of the present invention provides a storage medium storing computer-executable instructions for performing the method flow of any one of the first aspects above.
According to the technical scheme, firstly, a test behavior sequence generated in an automatic test process aiming at a target application is obtained. The test behavior sequence is a sequence of one or more test behaviors triggered in an automatic test process for a target application, and comprises a history page and test actions executed on the history page.
For example, after the page a of APP1 executes the test action x, the page b is entered, and at this time, the test action sequence includes the history page a and the test action x executed on the history page a. For another example, after the page a of APP2 executes the test behavior x, the page b is entered, and after the page b executes the test behavior y, the page c is entered, where the test behavior sequence is the test behavior x executed on the history page a, the test behavior b executed on the history page a, and the test behavior y executed on the history page b.
And predicting probability distribution of a plurality of test actions supported by the current page through a pre-constructed multi-mode user behavior prediction model according to the test behavior sequence and the current page.
The input information of the pre-constructed multi-mode user behavior prediction model is a test behavior sequence and a current page, the output information is probability distribution of a plurality of test actions supported by the current page, and the multi-mode user behavior prediction model reflects the test behavior sequence and the association relation between the current page and the probability distribution. Further, the multi-mode user behavior prediction model is trained by taking a historical test behavior sequence, a historical current page after the historical test behavior sequence and a plurality of test behaviors supported by the historical current page as sample data.
For example, after the page a of APP1 executes the test behavior x, the page b is entered, where the test behavior sequence includes the history page a and the test behavior x executed on the history page a, and the current page is the page b. Therefore, the historical page a, the test behavior x and the current page b can be used as the input of the multi-mode user behavior prediction model, and finally, the probability distribution of a plurality of test actions supported by the current page b is output. For another example, after the page a of the APP2 executes the test behavior x, the page b is entered, and after the page b executes the test behavior y, the page c is entered, where the test behavior sequence is the history page a, the test behavior x executed on the history page a, the history page b, and the test behavior y executed on the history page b, and the current page is the page c. Therefore, the historical page a, the test behavior x, the historical page b, the test behavior y and the current page c can be used as inputs of the multi-mode user behavior prediction model, and finally, probability distribution of a plurality of test actions supported by the current page c is output.
The probability distribution of the plurality of test actions supported by the current page reflects the likelihood that each of the plurality of test actions supported by the current page is performed, and thus, based on this probability distribution, it may be determined which test action of the plurality of test actions is performed on the current page, i.e., the corresponding test action is performed on the current page of the target application according to the probability distribution control.
In conclusion, the technical scheme of the method and the device can automatically predict the executable test action on the current page obtained after the test action sequence is executed based on the test action sequence generated in the automatic test process aiming at the target application, reduce the manual intervention in the test process, improve the test efficiency, reduce the error rate and help to improve the accuracy of the action prediction. Meanwhile, a plurality of test actions supported by the current page can be displayed one by one during prediction, so that a prediction result is more comprehensive and real, and higher accuracy is achieved.
[ description of the drawings ]
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 illustrates a flow chart of an application testing method according to one embodiment of the invention;
FIG. 2 illustrates a flow chart of an application testing method according to another embodiment of the invention;
FIG. 3 illustrates a flow chart of an application testing method according to yet another embodiment of the invention;
FIG. 4 shows a flow chart of an application testing method according to yet another embodiment of the invention;
FIG. 5 shows a schematic diagram of a test procedure according to one embodiment of the invention;
FIG. 6 illustrates a block diagram of an application testing apparatus according to one embodiment of the invention;
fig. 7 shows a block diagram of an electronic device according to an embodiment of the invention.
[ detailed description ] of the invention
For a better understanding of the technical solution of the present invention, the following detailed description of the embodiments of the present invention refers to the accompanying drawings.
It should be understood that the described embodiments are merely some, but not all, embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The terminology used in the embodiments of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in this application and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
Example 1
FIG. 1 illustrates a flow chart of an application testing method according to one embodiment of the invention.
As shown in fig. 1, the flow of the application program testing method according to one embodiment of the present invention includes:
step 102, a test behavior sequence generated in an automatic test process for a target application is obtained.
The test behavior sequence is a sequence of one or more test behaviors triggered in an automatic test process for a target application, and comprises a history page and test actions executed on the history page.
For example, after the page a of APP1 executes the test action x, the page b is entered, and at this time, the test action sequence includes the history page a and the test action x executed on the history page a.
For another example, after the page a of APP2 executes the test behavior x, the page b is entered, and after the page b executes the test behavior y, the page c is entered, where the test behavior sequence is the test behavior x executed on the history page a, the test behavior b executed on the history page a, and the test behavior y executed on the history page b.
And 104, predicting probability distribution of a plurality of test actions supported by the current page through a pre-constructed multi-mode user behavior prediction model according to the test behavior sequence and the current page.
The input information of the pre-constructed multi-mode user behavior prediction model is a test behavior sequence and a current page, the output information is probability distribution of a plurality of test actions supported by the current page, and the multi-mode user behavior prediction model reflects the test behavior sequence and the association relation between the current page and the probability distribution. Further, the multi-mode user behavior prediction model is trained by taking a historical test behavior sequence, a historical current page after the historical test behavior sequence and a plurality of test behaviors supported by the historical current page as sample data.
For example, after the page a of APP1 executes the test behavior x, the page b is entered, where the test behavior sequence includes the history page a and the test behavior x executed on the history page a, and the current page is the page b. Therefore, the historical page a, the test behavior x and the current page b can be used as the input of the multi-mode user behavior prediction model, and finally, the probability distribution of a plurality of test actions supported by the current page b is output.
For another example, after the page a of the APP2 executes the test behavior x, the page b is entered, and after the page b executes the test behavior y, the page c is entered, where the test behavior sequence is the history page a, the test behavior x executed on the history page a, the history page b, and the test behavior y executed on the history page b, and the current page is the page c. Therefore, the historical page a, the test behavior x, the historical page b, the test behavior y and the current page c can be used as inputs of the multi-mode user behavior prediction model, and finally, probability distribution of a plurality of test actions supported by the current page c is output.
And step 106, controlling to execute corresponding test actions on the current page of the target application according to the probability distribution.
The probability distribution of the plurality of test actions supported by the current page reflects the likelihood that each of the plurality of test actions supported by the current page is performed, and thus, based on this probability distribution, a determination may be made as to which test action of the plurality of test actions is performed on the current page.
In conclusion, the technical scheme of the method and the device can automatically predict the executable test action on the current page obtained after the test action sequence is executed based on the test action sequence generated in the automatic test process aiming at the target application, reduce the manual intervention in the test process, improve the test efficiency, reduce the error rate and help to improve the accuracy of the action prediction. Meanwhile, a plurality of test actions supported by the current page can be displayed one by one during prediction, so that a prediction result is more comprehensive and real, and higher accuracy is achieved.
On the basis of the first embodiment, in one possible design, the probability distribution includes, but is not limited to, a distribution of discrete random variables and a distribution of continuous random variables, wherein the distribution of discrete random variables includes, but is not limited to, a single-point distribution, a two-point distribution, a binomial distribution, a geometric distribution, a negative binomial distribution, a super-geometric distribution, a poisson distribution, and the like, and the distribution of continuous random variables includes, but is not limited to, a uniform distribution, a gaussian distribution, a cauchy distribution, a lognormal distribution, an exponential distribution, a gamma (Γ) distribution, a beta (beta) distribution, an x2 distribution, a student distribution, and an F distribution.
Example two
On the basis of the first embodiment, the multi-mode user behavior prediction model takes a target test behavior sequence comprising three continuous pages and actions and a next page associated with the three continuous pages as model inputs, and takes probability distribution of test actions supported by the next page associated with the three continuous pages as model output; the test behavior sequence includes three pages tested before the current page and test actions performed on the three pages.
For example, in the target application APP2, after the page a executes the test behavior x, the page b is entered, after the page b executes the test behavior y, the page c is entered, and after the page c executes the test behavior z, the page d is entered. At this time, the target test behavior sequence including three consecutive pages and actions is: the method comprises the steps of a history page a, a test behavior x executed on the history page a, a history page b, a test behavior y executed on the history page b, a history page c and a test behavior z executed on the history page c, wherein the next page associated with three continuous pages is a page d.
Then, taking the history page a, the test behavior x, the history page b, the test behavior y, the history page c, the test behavior z and the page d as inputs of the multi-mode user behavior prediction model, and outputting probability distribution of the test actions supported by the page d through the multi-mode user behavior prediction model.
Example III
Optionally, if the probability distribution is gaussian, as shown in fig. 2, a flow of an application program testing method according to another embodiment of the present invention includes:
step 202, a test behavior sequence generated in an automatic test process for a target application is obtained, wherein the test behavior sequence comprises a history page and test actions executed on the history page.
And 204, outputting a plurality of groups of Gaussian distributions through the pre-constructed multi-mode user behavior prediction model according to the test behavior sequence and the current page, wherein each group of Gaussian distributions comprises weights, expectations and variances.
The multi-mode user behavior prediction model can output a plurality of groups of Gaussian distributions, each group of Gaussian distribution corresponds to one test action executable on the current page, and the difference of weights among the plurality of groups of Gaussian distributions reflects the difference of possibility that the test actions corresponding to the plurality of groups of Gaussian distributions are triggered to a certain extent. The desire of a gaussian distribution represents the distribution symmetry axis of the data therein, the variance of the gaussian distribution represents the degree of dispersion of the data distribution thereof, and the higher the weight of the gaussian distribution, the closer the desire is to the variance, and the greater the likelihood that the corresponding test action will be triggered.
And 206, selecting any Gaussian distribution among the plurality of groups of Gaussian distributions by taking the weight of each group of Gaussian distribution as the probability of selecting each group of Gaussian distribution.
In one possible design, multiple sets of gaussian distributions output by a multimodal user behavior prediction model are sampled, one set of gaussian distributions being selected at a time that is different from the previous sample.
Taking the example given in the second embodiment as a reference, taking a history page a, a test behavior x, a history page b, a test behavior y, a history page c, a test behavior z and a page d as inputs of a multi-mode user behavior prediction model, outputting k groups of Gaussian distributions (weight pi, expected mu and variance sigma) of test actions supported by the page d through the multi-mode user behavior prediction model, taking the weight pi of the k groups of Gaussian distributions as a selection probability, sampling the k groups of Gaussian distributions, and selecting any one group of Gaussian distributions each time.
Step 208, determining a predicted value corresponding to the selected set of gaussian distributions based on the test behavior sequence, the current page, and the expected and variance of the selected set of gaussian distributions, where the predicted value is used to reflect the possibility that the test action supported by the current page and corresponding to the selected set of gaussian distributions is triggered.
The test action is actually triggered and is not only related to the corresponding gaussian distribution, but is also limited by the current page itself and the test action sequence occurring before triggering the current page. Therefore, the weight of each group of Gaussian distribution can be used as the probability of selecting each group of Gaussian distribution, any group of Gaussian distribution is selected from the plurality of groups of Gaussian distribution, and a predicted value corresponding to the group of Gaussian distribution is calculated by combining a test behavior sequence, a current page and expected and variance in the group of Gaussian distribution, wherein the predicted value reflects the probability that the test action corresponding to the group of Gaussian distribution is actually triggered in a plurality of test actions supported by the current page.
In one possible design, a probability density corresponding to the set of gaussian distributions is calculated as a predictor corresponding to the set of gaussian distributions based on the test behavior sequence, the current page, and the expectations and variances in the set of gaussian distributions. Specifically, with the test behavior sequence and the current page as variables, the probability density corresponding to the set of gaussian distributions is calculated based on the following formula.
Wherein pi, mu and sigma are the weight, the expected and the variance of the Gaussian distribution respectively, the dependent variable x is the test behavior sequence and the current page, and y is the probability density corresponding to the Gaussian distribution, namely the predicted value. The higher the resulting probability density, the higher the likelihood that the test action corresponding to the set of gaussian distributions will be triggered. Therefore, on the basis of reducing manual intervention in the test process and improving the test automation level, a plurality of test action triggering conditions possibly occurring in the current page can be accurately known, so that the prediction result is more comprehensive and real.
Example IV
In addition, on the basis of the first to third embodiments, an application program testing method according to still another embodiment of the present invention includes:
step 302, a test behavior sequence generated in an automatic test process for a target application is obtained, wherein the test behavior sequence comprises a history page and test actions executed on the history page.
And step 304, predicting probability distribution of a plurality of test actions supported by the current page through a pre-constructed multi-mode user behavior prediction model according to the test behavior sequence and the current page.
And step 306, distributing different test actions to each test client in the plurality of test clients according to the probability distribution.
Step 308, controlling the multiple test clients to execute the allocated test actions on the current page of the target application in parallel.
In the above technical solution, different test actions are allocated to each test client in the plurality of test clients according to the probability distribution, so that each test client can execute the test actions, specifically, each test client still executes the allocated test actions on the current page in its own target application. In other words, after obtaining multiple probability distributions based on the test behavior sequence and the current page, since each probability distribution corresponds to a test action executable by the current page, in order to obtain a comprehensive and accurate test result, a test action corresponding to the probability distribution can be respectively executed by multiple test clients, so that different trigger results brought by test actions corresponding to different probability distributions can be respectively displayed in different test clients, and multiple test results which may occur are obtained through parallel operation of multiple test clients, and the reliability of the test is improved.
Example five
On the basis of the first to third embodiments, an application program testing method according to another embodiment of the present invention includes:
step 402, a test behavior sequence generated in an automatic test process for a target application is obtained, wherein the test behavior sequence comprises a history page and test actions executed on the history page.
And step 404, predicting probability distribution of a plurality of test actions supported by the current page through a pre-constructed multi-mode user behavior prediction model according to the test behavior sequence and the current page.
In step 406, a plurality of sub-processes are created for the current test process by copying the test process.
Step 408, allocating different test actions to the plurality of sub-process distributions according to the probability distribution.
Step 410, controlling the multiple sub-processes to execute corresponding test actions in parallel for the current page of the target application.
Compared with the parallel mode of the multiple test clients provided in the fourth embodiment, in order to reduce the number of the test clients and save test resources, the current test process can be directly copied into multiple sub-processes, and test actions corresponding to the multiple probability distributions are respectively distributed to each sub-process, so that the test actions corresponding to the multiple probability distributions are respectively executed through the multiple sub-processes. In this way, different trigger results brought by test actions corresponding to different probability distributions can be respectively displayed in different subprocesses, and various possible test results are obtained through parallel operation of the multiprocess, so that the reliability of the test is improved.
In one possible design, the test procedure may be replicated by a Fork function. The Fork function is a Fork function in UNIX or UNIX-like, which divides a running program into multiple identical processes, each of which starts a thread that starts executing from the same location in the code. The threads in the multiple processes continue to execute as if two users have initiated two copies of the application at the same time.
Compared with the related art, the technical scheme of the embodiment omits the step of manually restarting the test process, improves the test efficiency and reduces the error rate. Meanwhile, multi-process prediction is performed based on multiple kinds of second simulation behavior information possibly generated, so that the operation is closer to the operation of a real user, more actual scenes are covered, and the test coverage is improved.
As shown in fig. 5, in the test process 1, three test actions a, b, c that may be triggered in the first page may be predicted based on the first page and the test behavior sequence z corresponding to the first page.
At this time, the test process 1 is copied to 11, 12, 13 by the Fork function, and the test actions a, b, c are assigned to the three sub-processes 11, 12, 13, respectively.
In the subprocess 11, a test action a is triggered, then a second page is entered, and when the second page is taken as a current page, a test action sequence corresponding to the second page is { z+the first page+the test action a }. Based on { z+first page+test action a } corresponding to the second page and the first page, three test actions h, i and j which are possibly triggered in the second page are predicted.
Similarly, in the subprocess 12, the test action b is triggered, and then a third page is entered, and when the third page is taken as the current page, the test action sequence corresponding to the third page is { z+the first page+the test action b }. And predicting two test actions k and l which are possibly triggered in the third page based on { z+the first page+the test action b } corresponding to the third page and the third page.
Similarly, in the sub-process 13, the test action c is triggered, and then a fourth page is entered, where the test behavior sequence corresponding to the fourth page is { z+the first page+the test action c }, when the fourth page is taken as the current page. And predicting three test actions m, n and o which are possibly triggered in the fourth page based on { z+the first page+the test action c } corresponding to the fourth page and the fourth page.
Next, the sub-process 11 is copied to 111, 112, 113 by the Fork function, and the test actions h, i, j are assigned to 111, 112, 113, respectively, and so on, the sub-process 12 is copied to 121, 122 by the Fork function, and the test actions k, l are assigned to 121, 122, respectively, and the sub-process 13 is copied to 131, 132, 133 by the Fork function, and the test actions m, n, o are assigned to 131, 132, 133, respectively. And (5) predicting circularly until the test is finished.
In the test scheme of the related art, after one prediction is performed based on the test behavior sequence and a prediction result is obtained, the test process needs to be restarted, and because the test action and the page are very complicated in application, the generated test behavior sequence is various, and the test scheme which is performed independently needs to restart the test process in a large amount to cover enough test behavior sequences. However, restarting the test process continuously consumes a lot of time and cost, and also causes the consumption of test equipment, which is unfavorable for the smooth performance of the test.
In this regard, referring to fig. 5, after performing a prediction based on the first test behavior sequence and obtaining a plurality of prediction results, the plurality of prediction results are directly added to the inputs of the plurality of sub-processes, so as to cycle down, thereby avoiding time cost loss and test equipment cost loss caused by restarting the test process. In addition, as can be seen from fig. 5, a plurality of test processes can run in parallel, and as the test progresses, more and more parallel test processes are performed, the more and more test behavior sequences are tested, the larger and more test ranges are obtained, so that the time cost is saved, and the test efficiency is improved.
In one possible design, all test processes are terminated if the number of all test processes currently running reaches a predetermined number, which is determined based on the test capability information and/or the number of test devices.
The predetermined number is determined based on the testing capability information and/or the number of the testing devices, the testing capability information and/or the number of the testing devices determine the upper limit of the testing capability, and the predetermined number refers to the maximum number of testing processes which can be borne by the testing capability information and/or the number of the testing devices before reaching the upper limit of the testing capability. Therefore, if the number of all the currently running test processes reaches the preset number, the maximum number of test processes which can be borne by the test equipment before the test equipment reaches the upper limit of the test capability is indicated, that is, the current test has enabled the test equipment to realize the maximum load, at this time, the test can be ended, the test processes are not increased any more, and the problems of process breakdown, test result errors and the like are avoided.
In addition, on the basis of any of the above embodiments, the multi-modal user behavior prediction model may be updated based on the resulting probability distribution. In particular, parameters of the multimodal user behavior prediction model may be updated by deriving and back-propagating a predetermined loss function CostFunction in conjunction with the probability distribution. The predetermined loss function is:
wherein x is a test behavior sequence, y is a test action which is actually triggered, K is the total group number of Gaussian distribution, K is the serial number of the Gaussian distribution of the current group, pi k (x) Mu as a function of x as a variable associated with the weight pi of the kth Gaussian distribution k (x) Sigma, the expectation of the k-th group Gaussian distribution k (x) For the variance of the k-th set of gaussian distributions,in terms of y, mu k (x)、σ k (x) As a function of the variables.
By CostFunction of a predetermined loss function k Deriving and back-propagating, can be applied to pi k (x) Andand updating parameters of the model, thereby realizing the updating of the multi-mode user behavior prediction model.
In summary, through the technical scheme of the application, the method and the device can output a plurality of prediction results based on the test behavior sequence generated in the automatic test process aiming at the target application, and can more accurately draw and predict the operation behaviors of the real user, so that the multi-mode user behavior prediction model is closer to the real user, and the accuracy of test action prediction in the test process is improved. Meanwhile, the test actions can be predicted in a multi-process parallel manner, a large number of restarting behaviors are avoided, the repeated execution paths in the test can be reduced, more comprehensive test coverage is realized on the basis of being close to the actual user operation, the time for the test is shortened, and the utilization rate of physical resources in the test is improved.
FIG. 6 illustrates a block diagram of an application testing apparatus according to one embodiment of the invention.
As shown in fig. 6, an application testing apparatus 600 according to an embodiment of the present invention includes:
a test behavior sequence obtaining unit 602, configured to obtain a test behavior sequence generated in an automated test process for a target application, where the test behavior sequence includes a history page and a test action executed on the history page; the probability distribution determining unit 604 is configured to predict, according to the test behavior sequence and the current page, probability distributions of a plurality of test actions supported by the current page through a multi-mode user behavior prediction model that is constructed in advance; and a test action control unit 606, configured to control, according to the probability distribution, execution of a corresponding test action on the current page of the target application.
In the above embodiment of the present invention, optionally, the multi-modal user behavior prediction model takes a target test behavior sequence including three consecutive pages and actions and a next page associated with the three consecutive pages as model inputs, and the multi-modal user behavior prediction model takes probability distribution of test actions supported by the next page associated with the three consecutive pages as model outputs; the test behavior sequence includes three pages tested before the current page and test actions performed on the three pages.
In the above-described embodiment of the present invention, optionally, the probability distribution determining unit 604 is configured to: outputting a plurality of groups of Gaussian distributions through the pre-constructed multi-mode user behavior prediction model, wherein each group of Gaussian distributions comprises weights, expectations and variances; the test action control unit 606 is configured to: and selecting any group of Gaussian distribution in the plurality of groups of Gaussian distribution by taking the weight of each group of Gaussian distribution as the probability of selecting each group of Gaussian distribution, and determining a predicted value corresponding to the selected Gaussian distribution based on the test behavior sequence, the current page and the expected and variance of the selected Gaussian distribution, wherein the predicted value is used for reflecting the possibility that the test action supported by the current page and corresponding to the selected group of Gaussian distribution is triggered.
In the above embodiment of the present invention, optionally, the test action control unit 606 is configured to: distributing different test actions to each test client in a plurality of test clients according to the probability distribution; and controlling the plurality of test clients to execute the allocated test actions aiming at the current page of the target application in parallel.
In the above embodiment of the present invention, optionally, the test action control unit 606 is configured to: creating a plurality of sub-processes for the current test process by copying the test process; distributing different test actions to the plurality of sub-process distributions according to the probability distribution; and controlling the plurality of subprocesses to execute corresponding test actions aiming at the current page of the target application in parallel.
The application testing apparatus 600 has the same technical effects as those in any of the above embodiments, and will not be described herein.
Fig. 7 shows a block diagram of an electronic device according to an embodiment of the invention.
As shown in fig. 7, an electronic device 700 of one embodiment of the invention includes at least one memory 702; and a processor 704 communicatively coupled to the at least one memory 702; wherein the memory stores instructions executable by the at least one processor 704, the instructions configured to perform the aspects described in any of the embodiments above. Therefore, the electronic device 700 has the same technical effects as those of any of the above embodiments, and will not be described herein.
The electronic device of the embodiments of the present invention exists in a variety of forms including, but not limited to:
(1) Mobile communication devices, which are characterized by mobile communication functionality and are aimed at providing voice, data communication. Such terminals include smart phones (e.g., iPhone), multimedia phones, functional phones, and low-end phones, among others.
(2) Ultra mobile personal computer equipment, which belongs to the category of personal computers, has the functions of calculation and processing and generally has the characteristic of mobile internet surfing. Such terminals include PDA, MID and UMPC devices, etc., such as iPad.
(3) Portable entertainment devices such devices can display and play multimedia content. Such devices include audio, video players (e.g., iPod), palm game consoles, electronic books, and smart toys and portable car navigation devices.
(4) The server is similar to a general computer architecture in that the server is provided with high-reliability services, and therefore, the server has high requirements on processing capacity, stability, reliability, safety, expandability, manageability and the like.
(5) Other electronic devices with data interaction function.
In addition, an embodiment of the present invention provides a storage medium storing computer executable instructions for performing the method flow described in any one of the above embodiments.
The technical scheme of the invention is explained in detail by combining the drawings, and through the technical scheme of the invention, the accuracy of the test result is improved, the labor cost and the time cost are reduced, and the efficiency of the test work is improved.
It should be understood that the term "and/or" as used herein is merely one relationship describing the association of the associated objects, meaning that there may be three relationships, e.g., a and/or B, may represent: a exists alone, A and B exist together, and B exists alone. In addition, the character "/" herein generally indicates that the front and rear associated objects are an "or" relationship.
Depending on the context, the word "if" as used herein may be interpreted as "at … …" or "at … …" or "in response to a determination" or "in response to detection". Similarly, the phrase "if determined" or "if detected (stated condition or event)" may be interpreted as "when determined" or "in response to determination" or "when detected (stated condition or event)" or "in response to detection (stated condition or event), depending on the context.
In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of the elements is merely a logical function division, and there may be additional divisions when actually implemented, e.g., multiple elements or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.
In addition, each functional unit in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in hardware plus software functional units.
The integrated units implemented in the form of software functional units described above may be stored in a computer readable storage medium. The software functional unit is stored in a storage medium, and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a Processor (Processor) to perform part of the steps of the methods according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
The foregoing description of the preferred embodiments of the invention is not intended to be limiting, but rather to enable any modification, equivalent replacement, improvement or the like to be made within the spirit and principles of the invention.
Claims (10)
1. An application testing method, comprising:
acquiring a test behavior sequence generated in an automatic test process aiming at a target application, wherein the test behavior sequence comprises a history page and test actions executed on the history page;
according to the test behavior sequence and the current page, predicting probability distribution of a plurality of test actions supported by the current page through a pre-constructed multi-mode user behavior prediction model;
controlling to execute corresponding test actions on the current page of the target application according to the probability distribution;
wherein the probability distribution comprises a distribution of discrete random variables or a distribution of continuous random variables; the controlling, according to the probability distribution, the execution of the corresponding test action on the current page of the target application includes:
based on the test behavior sequence, the current page and the expected and variance of a selected set of probability distributions in each set of probability distributions, a predicted value corresponding to the selected set of probability distributions is determined, wherein the predicted value is used for reflecting the possibility that a test action supported by the current page and corresponding to the selected set of probability distributions is triggered.
2. The application testing method of claim 1, wherein,
the multi-modal user behavior prediction model takes a target test behavior sequence containing three continuous pages and actions and a next page associated with the three continuous pages as model inputs, and takes probability distribution of test actions supported by the next page associated with the three continuous pages as model output;
the test behavior sequence includes three pages tested before the current page and test actions performed on the three pages.
3. The application testing method of claim 1, wherein the probability distribution comprises a gaussian distribution; the controlling, according to the probability distribution, the execution of the corresponding test action on the current page of the target application includes:
and selecting one group of Gaussian distribution from multiple groups of Gaussian distribution by taking the weight of each group of Gaussian distribution as the probability of selecting each group of Gaussian distribution.
4. An application testing method according to any one of claims 1 to 3, wherein said controlling, according to said probability distribution, the execution of a corresponding testing action on the current page of the target application comprises:
Distributing different test actions to each test client in a plurality of test clients according to the probability distribution;
and controlling the plurality of test clients to execute the allocated test actions aiming at the current page of the target application in parallel.
5. The application testing method according to claim 1, wherein the controlling the execution of the corresponding test action on the current page of the target application according to the probability distribution includes:
creating a plurality of sub-processes for the current test process by copying the test process;
distributing different test actions to the plurality of sub-process distributions according to the probability distribution;
and controlling the plurality of subprocesses to execute corresponding test actions aiming at the current page of the target application in parallel.
6. An application testing apparatus, comprising:
the system comprises a test behavior sequence acquisition unit, a test behavior sequence generation unit and a test execution unit, wherein the test behavior sequence acquisition unit is used for acquiring a test behavior sequence generated in an automatic test process aiming at a target application, and the test behavior sequence comprises a history page and test actions executed on the history page;
the probability distribution determining unit is used for predicting probability distribution of a plurality of test actions supported by the current page through a pre-constructed multi-mode user behavior prediction model according to the test behavior sequence and the current page;
The test action control unit is used for controlling the execution of a corresponding test action on the current page of the target application according to the probability distribution;
wherein the probability distribution comprises a distribution of discrete random variables or a distribution of continuous random variables; the test action control unit is used for: based on the test behavior sequence, the current page and the expected and variance of a selected set of probability distributions in each set of probability distributions, a predicted value corresponding to the selected set of probability distributions is determined, wherein the predicted value is used for reflecting the possibility that a test action supported by the current page and corresponding to the selected set of probability distributions is triggered.
7. The application testing apparatus of claim 6, wherein,
the multi-modal user behavior prediction model takes a target test behavior sequence containing three continuous pages and actions and a next page associated with the three continuous pages as model inputs, and takes probability distribution of test actions supported by the next page associated with the three continuous pages as model output;
the test behavior sequence includes three pages tested before the current page and test actions performed on the three pages.
8. The application testing apparatus of claim 6, wherein the probability distribution comprises a gaussian distribution; the test action control unit is used for:
and selecting any one group of Gaussian distribution from multiple groups of Gaussian distribution by taking the weight of each group of Gaussian distribution as the probability of selecting each group of Gaussian distribution.
9. An electronic device, comprising: at least one processor; and a memory communicatively coupled to the at least one processor;
wherein the memory stores instructions executable by the at least one processor, the instructions being arranged to perform the method of any of the preceding claims 1 to 7.
10. A storage medium storing computer executable instructions for performing the method flow of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110035303.7A CN112699046B (en) | 2021-01-12 | 2021-01-12 | Application program testing method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110035303.7A CN112699046B (en) | 2021-01-12 | 2021-01-12 | Application program testing method and device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112699046A CN112699046A (en) | 2021-04-23 |
| CN112699046B true CN112699046B (en) | 2024-03-29 |
Family
ID=75514076
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110035303.7A Active CN112699046B (en) | 2021-01-12 | 2021-01-12 | Application program testing method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112699046B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113220598B (en) * | 2021-06-21 | 2023-10-03 | 中国农业银行股份有限公司 | System test method, device, equipment, medium and program product |
| CN115037655B (en) * | 2022-05-19 | 2024-03-12 | 支付宝(杭州)信息技术有限公司 | Pressure measurement method and system |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108550279A (en) * | 2018-04-03 | 2018-09-18 | 同济大学 | Vehicle drive behavior prediction method based on machine learning |
| CN109977029A (en) * | 2019-04-09 | 2019-07-05 | 科大讯飞股份有限公司 | A kind of training method and device of page jump model |
| CN111078579A (en) * | 2019-12-31 | 2020-04-28 | 河北省科学院应用数学研究所 | Monkey testing method and device and terminal equipment |
| CN111124916A (en) * | 2019-12-23 | 2020-05-08 | 北京云聚智慧科技有限公司 | Model training method based on motion semantic vector and electronic equipment |
| CN111651361A (en) * | 2020-06-10 | 2020-09-11 | 宋文韬 | Script-free automatic testing method based on visual page |
| CN111857719A (en) * | 2020-07-31 | 2020-10-30 | 北京字节跳动网络技术有限公司 | Data processing method, device, electronic equipment and medium |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107577522B (en) * | 2017-09-30 | 2020-04-21 | Oppo广东移动通信有限公司 | Application control method, device, storage medium and electronic equipment |
-
2021
- 2021-01-12 CN CN202110035303.7A patent/CN112699046B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108550279A (en) * | 2018-04-03 | 2018-09-18 | 同济大学 | Vehicle drive behavior prediction method based on machine learning |
| CN109977029A (en) * | 2019-04-09 | 2019-07-05 | 科大讯飞股份有限公司 | A kind of training method and device of page jump model |
| CN111124916A (en) * | 2019-12-23 | 2020-05-08 | 北京云聚智慧科技有限公司 | Model training method based on motion semantic vector and electronic equipment |
| CN111078579A (en) * | 2019-12-31 | 2020-04-28 | 河北省科学院应用数学研究所 | Monkey testing method and device and terminal equipment |
| CN111651361A (en) * | 2020-06-10 | 2020-09-11 | 宋文韬 | Script-free automatic testing method based on visual page |
| CN111857719A (en) * | 2020-07-31 | 2020-10-30 | 北京字节跳动网络技术有限公司 | Data processing method, device, electronic equipment and medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112699046A (en) | 2021-04-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112699046B (en) | Application program testing method and device, electronic equipment and storage medium | |
| CN110164435A (en) | Audio recognition method, device, equipment and computer readable storage medium | |
| CN110378346B (en) | Method, device and equipment for establishing character recognition model and computer storage medium | |
| CN109473121B (en) | Voice synthesis quality testing method and device | |
| CN111582375A (en) | Data enhancement strategy searching method, device, equipment and storage medium | |
| CN109165286A (en) | Automatic question-answering method, device and computer readable storage medium | |
| CN114662676A (en) | Model optimization method and device, electronic equipment and computer-readable storage medium | |
| CN112286485B (en) | Method and device for controlling application through voice, electronic equipment and storage medium | |
| CN112395182A (en) | Automatic testing method, device, equipment and computer readable storage medium | |
| CN115018081A (en) | Feature selection method, application program prediction method and device | |
| CN111309421B (en) | Interface resource loading method and device, terminal and storage medium | |
| CN111046156B (en) | Method, device and server for determining rewarding data | |
| CN111475388A (en) | Data push test method and device, computer equipment and storage medium | |
| CN115328891A (en) | Data migration method and device, storage medium and electronic equipment | |
| CN113032278B (en) | Application running mode, and method and device for confirming grade of terminal equipment | |
| CN115544224A (en) | Machine reading understanding method, model training method, device and medium | |
| CN109522187B (en) | Method and device for quickly extracting state information | |
| CN114816719A (en) | Training method and device of multi-task model | |
| CN113836005A (en) | Virtual user generation method and device, electronic equipment and storage medium | |
| CN114530073A (en) | Training method and device based on virtual reality | |
| CN110647314B (en) | Skill generation method and device and electronic equipment | |
| CN114356512A (en) | Data processing method, data processing equipment and computer readable storage medium | |
| CN112329943A (en) | Combined index selection method and device, computer equipment and medium | |
| CN110968519A (en) | Game testing method, device, server and storage medium | |
| CN117251118B (en) | Virtual NVMe simulation and integration supporting method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 100041 B-0035, 2 floor, 3 building, 30 Shixing street, Shijingshan District, Beijing. Applicant after: Douyin Vision Co.,Ltd. Address before: 100041 B-0035, 2 floor, 3 building, 30 Shixing street, Shijingshan District, Beijing. Applicant before: Tiktok vision (Beijing) Co.,Ltd. Address after: 100041 B-0035, 2 floor, 3 building, 30 Shixing street, Shijingshan District, Beijing. Applicant after: Tiktok vision (Beijing) Co.,Ltd. Address before: 100041 B-0035, 2 floor, 3 building, 30 Shixing street, Shijingshan District, Beijing. Applicant before: BEIJING BYTEDANCE NETWORK TECHNOLOGY Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |