CN111694753A - Application program testing method and device and computer storage medium - Google Patents
Application program testing method and device and computer storage medium Download PDFInfo
- Publication number
- CN111694753A CN111694753A CN202010754358.9A CN202010754358A CN111694753A CN 111694753 A CN111694753 A CN 111694753A CN 202010754358 A CN202010754358 A CN 202010754358A CN 111694753 A CN111694753 A CN 111694753A
- Authority
- CN
- China
- Prior art keywords
- candidate
- value
- executed
- profit
- operations
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000012360 testing method Methods 0.000 claims abstract description 131
- 238000000034 method Methods 0.000 claims abstract description 55
- 230000008569 process Effects 0.000 claims abstract description 33
- 230000006870 function Effects 0.000 claims description 57
- 238000012512 characterization method Methods 0.000 claims description 33
- 238000004422 calculation algorithm Methods 0.000 claims description 20
- 230000002787 reinforcement Effects 0.000 claims description 18
- 238000004590 computer program Methods 0.000 claims description 12
- 238000004364 calculation method Methods 0.000 claims description 6
- 230000009471 action Effects 0.000 description 156
- 230000000875 corresponding effect Effects 0.000 description 116
- 230000008901 benefit Effects 0.000 description 7
- 238000010586 diagram Methods 0.000 description 4
- 238000004891 communication Methods 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 238000011161 development Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 238000001545 Page's trend test Methods 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000010998 test method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3676—Test management for coverage analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3684—Test management for test design, e.g. generating new test cases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3688—Test management for test execution, e.g. scheduling of test suites
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The present disclosure provides an application program testing method, apparatus and computer storage medium, wherein the method comprises: determining a plurality of candidate operations of a target test page in the test process of an application program; aiming at each candidate operation, calculating a multi-step operation profit value corresponding to the candidate operation; and selecting a target operation from the plurality of candidate operations as an operation to be tested and executed next step based on the executed times and the multi-step operation profit values respectively corresponding to the plurality of candidate operations. In the embodiment of the disclosure, after a plurality of candidate operations of a target test page are determined, according to the multi-step operation income value and the executed times respectively corresponding to each candidate operation, an operation with a large corresponding multi-step operation income value and a small executed number is selected as an operation for next test execution, so that invalid operations and repeated operations in the test process of the current test page are reduced, the value return of each test is improved, and the test speed and efficiency are improved.
Description
Technical Field
The present disclosure relates to the field of internet technologies, and in particular, to a method and an apparatus for testing an application program, and a computer storage medium.
Background
With the continuous development of internet technology, more and more applications are developed. The user can perform different operations on various application programs to meet the needs of the user, for example, playing games by operating a game application, watching videos by operating a video application, and the like.
After the development of each application program is completed, before the application program is online, testing each application program; and when the test result of the application program has no problem, the application program can be formally online. At present, depth-first search and breadth-first search are generally adopted to test an application program, and the application program can be comprehensively tested only by traversing each control of each page in the application program, so that the testing speed is low, and the testing efficiency is low.
Disclosure of Invention
The embodiment of the disclosure at least provides an application program testing method, an application program testing device and a computer storage medium.
In a first aspect, an embodiment of the present disclosure provides an application program testing method, where the method includes:
determining a plurality of candidate operations of a target test page in the test process of an application program;
calculating a multi-step operation profit value corresponding to each candidate operation; the profit value of the multi-step operation is the profit value brought by the execution of the multi-step operation from the execution of the candidate operation; the profit value is used for representing the contribution value of the multi-step operation on the test coverage growth rate of the page function;
and selecting a target operation from the candidate operations as an operation to be tested and executed next step based on the executed times and the multi-step operation profit values respectively corresponding to the candidate operations.
In a possible implementation manner, after determining the plurality of candidate operations, before calculating, for each of the candidate operations, a multi-step operation profit value corresponding to the candidate operation, the method further includes:
when at least one unexecuted operation exists in the candidate operations of the target test page, selecting the target operation from the at least one unexecuted operation as an operation of next test execution.
In one possible implementation, for each of the candidate operations, calculating a multi-step operation profit value corresponding to the candidate operation includes:
determining n report values corresponding to the candidate operation based on the page element attribute information after each step of the n steps of operations is executed; the n return values comprise an ith return value obtained after the operation of the ith step is executed, wherein the operation of the 1 st step is the candidate operation; i is a positive integer from 1 to n; the return value is used for representing an initial contribution value of each step of operation on the test coverage growth rate of the page function;
and calculating the multi-step operation income value by adopting a reinforcement learning algorithm based on the n return values.
In one possible implementation, the page element attribute information includes at least one of:
whether a new page component appears after the operation of the ith step is executed;
whether a new state appears after the operation of the ith step is executed;
whether a new operation exists after the operation of the ith step is executed;
the number of executed times corresponding to the operation of the ith step.
In a possible embodiment, the calculating the multi-step operation profit value by using a reinforcement learning algorithm based on the n return values includes:
based on the n return values, preset attenuation coefficients and a preset learning rate, performing repeated iteration updating on the multi-step operation profit values by adopting the reinforcement learning algorithm until preset convergence conditions are met, and obtaining final multi-step operation profit values corresponding to the candidate operation;
the preset convergence condition means that the difference value between the multi-step operation income values obtained by the last two times of iteration updating is smaller than a set threshold value.
In a possible implementation manner, based on the n return values and preset attenuation coefficients and learning rates, performing multiple iterative updates on the multi-step operation profit value by using the reinforcement learning algorithm until a preset convergence condition is met, including:
constructing n steps of discount income representing functions based on preset attenuation coefficients and the n return values; the n-step discount income characterization function comprises n addition terms, wherein the first n-1 addition terms comprise products of i-1 power of the attenuation coefficient gamma and the ith return value, and the nth addition term is a characterization function which is corresponding to the nth step operation and used for updating the previous multi-step operation income value;
constructing a characterization function of the updated multi-step operation profit value corresponding to the candidate operation based on the n-step discount profit characterization function, a preset learning rate and the characterization function of the updated multi-step operation profit value corresponding to the candidate operation;
and performing repeated iteration updating on the basis of the characterization function of the updated multi-step operation profit value corresponding to the candidate operation until the preset convergence condition is met.
In a possible implementation manner, selecting a target operation from the multiple candidate operations as an operation to be tested and executed next based on the executed times and the profit value of the multi-step operation respectively corresponding to the multiple candidate operations includes:
for each candidate operation in the multiple candidate operations, determining a first access return value corresponding to the candidate operation based on the multi-step operation profit value corresponding to the candidate operation and the number of times of access corresponding to the current state of the target test page; determining a second access report value corresponding to the candidate operation based on the executed times corresponding to the candidate operation, the accessed times corresponding to the current state and a preset balance coefficient;
determining a total access return value for the candidate operation as a sum of the first access return value and the second access return value;
and selecting the candidate operation with the maximum corresponding total access return value from the plurality of candidate operations as the target operation.
In a second aspect, an embodiment of the present disclosure further provides an application testing apparatus, where the apparatus includes:
the first determining module is used for determining a plurality of candidate operations of the target test page in the test process of the application program.
The calculation module is used for calculating a multi-step operation profit value corresponding to each candidate operation; the profit value of the multi-step operation is the profit value brought by the execution of the multi-step operation from the execution of the candidate operation; the profit value is used for characterizing the contribution value of the multi-step operation on the test coverage growth rate of the page function.
And the second determining module is used for selecting a target operation from the multiple candidate operations as an operation to be tested and executed next step based on the executed times and the multi-step operation profit values respectively corresponding to the multiple candidate operations.
In a possible implementation manner, the second determining module is further configured to, when it is determined that there is at least one unexecuted operation in the plurality of candidate operations of the target test page, select a target operation from the at least one unexecuted operation as an operation to be tested and executed next.
In a possible implementation manner, the calculation module is specifically configured to determine n report values corresponding to the candidate operation based on the page element attribute information after each of the n steps of operations is performed; the n return values comprise an ith return value obtained after the operation of the ith step is executed, wherein the operation of the 1 st step is the candidate operation; i is a positive integer from 1 to n; the return value is used for representing an initial contribution value of each step of operation on the test coverage growth rate of the page function; and calculating the multi-step operation income value by adopting a reinforcement learning algorithm based on the n return values.
In one possible implementation, the page element attribute information includes at least one of: whether a new page component appears after the operation of the ith step is executed; whether a new state appears after the operation of the ith step is executed; whether a new operation exists after the operation of the ith step is executed; the number of executed times corresponding to the operation of the ith step.
In a possible implementation manner, the calculation module is further specifically configured to perform multiple iterative updates on the multi-step operation profit value by using the reinforcement learning algorithm based on the n return values and a preset attenuation coefficient and a preset learning rate until a preset convergence condition is met, so as to obtain a final multi-step operation profit value corresponding to the candidate operation; the preset convergence condition means that the difference value between the multi-step operation income values obtained by the last two times of iteration updating is smaller than a set threshold value.
In a possible implementation manner, the calculation module is further specifically configured to construct an n-step discount income representation function based on a preset attenuation coefficient and the n return values; the n-step discount income characterization function comprises n addition terms, wherein the first n-1 addition terms comprise products of i-1 power of the attenuation coefficient gamma and the ith return value, and the nth addition term is a characterization function which is corresponding to the nth step operation and used for updating the previous multi-step operation income value; constructing a characterization function of the updated multi-step operation profit value corresponding to the candidate operation based on the n-step discount profit characterization function, a preset learning rate and the characterization function of the updated multi-step operation profit value corresponding to the candidate operation; and performing repeated iteration updating on the basis of the characterization function of the updated multi-step operation profit value corresponding to the candidate operation until the preset convergence condition is met.
In a possible implementation manner, the second determining module is specifically configured to determine, for each candidate operation in the multiple candidate operations, a first access return value corresponding to the candidate operation based on the multi-step operation profit value corresponding to the candidate operation and the number of times visited corresponding to the current state of the target test page; determining a second access report value corresponding to the candidate operation based on the executed times corresponding to the candidate operation, the accessed times corresponding to the current state and a preset balance coefficient; determining a total access return value for the candidate operation as a sum of the first access return value and the second access return value; and selecting the candidate operation with the maximum corresponding total access return value from the plurality of candidate operations as the target operation.
In a third aspect, an embodiment of the present disclosure further provides a computer device, including: a processor, a memory and a bus, the memory storing machine-readable instructions executable by the processor, the processor and the memory communicating via the bus when the computer device is running, the machine-readable instructions when executed by the processor performing the steps of the first aspect described above, or any possible implementation of the first aspect.
In a fourth aspect, this disclosed embodiment also provides a computer-readable storage medium, on which a computer program is stored, where the computer program is executed by a processor to perform the steps in the first aspect or any one of the possible implementation manners of the first aspect.
According to the application program testing method, device and computer storage medium provided by the embodiment of the disclosure, in the application program testing process, a plurality of candidate operations (actions) selectable in the next test are determined for a current test page, a multi-step operation profit value corresponding to each candidate action is calculated, a target action is selected from the plurality of candidate actions as an action executed in the next test based on the executed times and the multi-step operation profit values respectively corresponding to each candidate action, and here, according to the multi-step operation profit value and the executed times respectively corresponding to each candidate action, an action with a larger corresponding multi-step operation profit value and a smaller executed times can be selected as an action executed in the next test, so that invalid operations and repeated operations in the current test page testing process are reduced, the value return of each test is improved, and the testing speed and efficiency are improved.
For the description of the effects of the application testing apparatus, the electronic device, and the computer-readable storage medium, reference is made to the description of the application testing method, which is not repeated herein.
In order to make the aforementioned objects, features and advantages of the present disclosure more comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings required for use in the embodiments will be briefly described below, and the drawings herein incorporated in and forming a part of the specification illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the technical solutions of the present disclosure. It is appreciated that the following drawings depict only certain embodiments of the disclosure and are therefore not to be considered limiting of its scope, for those skilled in the art will be able to derive additional related drawings therefrom without the benefit of the inventive faculty.
FIG. 1 is a flow chart illustrating a method for testing an application provided by an embodiment of the present disclosure;
FIG. 2 is a schematic diagram of an application testing apparatus provided in an embodiment of the present disclosure;
fig. 3 shows a schematic diagram of a computer device provided by an embodiment of the present disclosure.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present disclosure more clear, the technical solutions of the embodiments of the present disclosure will be described clearly and completely with reference to the drawings in the embodiments of the present disclosure, and it is obvious that the described embodiments are only a part of the embodiments of the present disclosure, not all of the embodiments. The components of the embodiments of the present disclosure, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present disclosure, presented in the figures, is not intended to limit the scope of the claimed disclosure, but is merely representative of selected embodiments of the disclosure. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the disclosure without making creative efforts, shall fall within the protection scope of the disclosure.
Research shows that in the process of testing each application program, the application program is generally tested by adopting depth-first search and breadth-first search, and when the application program is tested by adopting the depth-first search and the breadth-first search, the application program can be comprehensively tested only by traversing each control of each page in the application program, so that the testing speed is low, and the testing efficiency is low.
Based on the above research, the present disclosure provides an application testing method, apparatus, and computer storage medium, in the testing process of the application program, a plurality of candidate operations (actions) which can be selected in the next test are determined aiming at the current test page, the multi-step operation income value corresponding to each candidate action is calculated, and a target action is selected from the candidate actions as the action executed in the next test based on the executed times and the multi-step operation income value respectively corresponding to each candidate action, according to the multi-step operation profit value and the executed times respectively corresponding to each candidate action, the corresponding actions with larger multi-step operation profit values and less executed times can be selected as the actions executed in the next test, so that invalid operations and repeated operations in the current test page test process are reduced, the value return of each test is improved, and the test speed and efficiency are improved.
The above-mentioned drawbacks are the results of the inventor after practical and careful study, and therefore, the discovery process of the above-mentioned problems and the solutions proposed by the present disclosure to the above-mentioned problems should be the contribution of the inventor in the process of the present disclosure.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.
To facilitate understanding of the present embodiment, first, an application testing method disclosed in the embodiments of the present disclosure is described in detail, where an execution subject of the application testing method provided in the embodiments of the present disclosure is generally a computer device with certain computing capability, and the computer device includes, for example: a terminal device, which may be a User Equipment (UE), a mobile device, a User terminal, a cellular phone, a cordless phone, a Personal Digital Assistant (PDA), a handheld device, a computing device, a vehicle mounted device, a wearable device, or a server or other processing device. In some possible implementations, the application test method may be implemented by a processor calling computer readable instructions stored in a memory.
The following describes an application testing method provided by the embodiment of the present disclosure by taking an execution subject as a server.
Example one
Referring to fig. 1, a flowchart of an application program testing method provided in the embodiment of the present disclosure is shown, where the method includes steps S101 to S103, where:
s101, in the process of testing the application program, determining a plurality of candidate operations (actions) of a target test page.
The target test page is a currently tested page, and the target test page can contain at least one control; a plurality of candidate operations corresponding to the target test page may be determined based on the state of each control in the target test page; the candidate actions can be operations performed on each control in the target test page, and the operations can include click operations, sliding operations, long-time pressing operations and the like.
In the concrete implementation, in the application program testing process, each page can be abstracted into a state node (state), each state has a plurality of corresponding candidate actions, and the candidate actions executed in the next testing process are determined according to the state of the current testing page; for example, the target test page includes 3 operable controls, which are control 1, control 2, and control 3; in the testing process, the state of the target test page corresponding to the clicked target test page after the control 1 in the target test page is clicked is state a, and it can be determined that the candidate actions under the state (i.e., state a) of the target test page can include long-time pressing of the control 1, clicking of the control 2, long-time pressing of the control 2, clicking of the control 3, long-time pressing of the control 3, and the like.
In a specific implementation, after determining a plurality of candidate actions under the state of the target test page, the target action may be selected from the plurality of candidate actions by calculating a multi-step profit value corresponding to each candidate action and the number of executed times corresponding to each candidate action, which is described in steps S102 to S103.
In an optional implementation manner, when at least one unexecuted action exists in the candidate actions of the target test page, the target action is selected from the at least one unexecuted action as an operation of next test execution.
For example, if the candidate actions under the state of the target test page include long-press control 1, click control 2, long-press control 2, click control 3, and long-press control 3, and the operation of clicking control 2 is an operation that is not tested in the test process (i.e., the operation of clicking control 2 is an unexecuted operation), the next test executed operation may be: clicking a control 2; if the candidate actions under the state of the target test page comprise a long press control 1, a click control 2, a long press control 2, a click control 3 and a long press control 3, and the operations of the click control 2 and the click control 3 are the operations which are not tested in the test process (namely the operations of the click control 2 and the click control 3 are the unexecuted operations), one of the two unexecuted operations is randomly selected as the next test execution operation.
S102, aiming at each candidate action, calculating a multi-step operation profit value corresponding to the candidate action.
Wherein, the profit value of the multi-step operation refers to the profit value brought by the execution of the multi-step operation from the execution of the candidate action. The profit value is used for representing the contribution value of the multi-step operation on the test coverage growth rate of the page function; here, when the tested page functions are more after the multi-step operation is performed, the contribution value to the test coverage growth rate of the page functions is high (i.e. the profit value is high); when the tested page functions are less after the multi-step operation is performed, the contribution value of the test coverage growth rate of the page functions is low (namely, the profit value is low).
In a specific implementation, the value of the benefit of the multi-step operation brought by the execution of the multi-step action from the execution of the candidate action may be calculated by an n-step sarsa algorithm in the reinforcement learning algorithm, which is specifically described as follows:
determining n return (Reward) values corresponding to the candidate action based on the page element attribute information after each action in the n-step operation is executed; and calculating the multi-step operation income value by adopting a reinforcement learning algorithm based on the n rewarded values.
Here, the Reward value is used to characterize the initial contribution value of each operation to the test coverage growth rate for the page function. The initial contribution value may be understood as a benefit value before iteratively updating the multi-step operation benefit value based on a reinforcement learning algorithm.
Here, the n rewarded values include the ith rewarded value obtained after the ith action is executed, wherein the 1 st action is the candidate action; i is a positive integer from 1 to n.
The page attribute information may include whether a new page component appears after the ith action is executed, whether a new state appears after the ith action is executed, whether a new action exists after the ith action is executed, and the number of times of execution corresponding to the ith action.
Here, whether a new page component appears after the ith action is executed may indicate whether to jump to a new page after the ith action is executed in the test process; whether a new state appears after the ith action is executed can indicate whether a new state appears in the target test page after the ith action is executed in the test process; whether a new action exists after the ith action is executed can indicate whether a new operation is generated after the ith action is executed in the test process.
Here, the larger the Reward value is, the smaller the number of times of execution of the ith action is, the more page components appear after the ith action is executed, the more state appears, and the more action appears after the ith action is executed; when no new page component appears after the ith step action is executed, no new state appears, no new action appears after the ith step action is executed, and the execution times of the ith step action are more, the smaller the Reward value is.
Illustratively, when a new page component appears after the operation of the ith step, a new state does not appear, and a new action does not appear after the operation of the ith step, the Reward value is 100; when no new page component appears, no new state appears and no new action appears after the operation of the ith step is executed, the Reward value is 100; when no new page component appears after the operation of the ith step, no new state appears, and a new action appears after the action of the ith step is executed, the Reward value is 100; and when no new page component appears, no new state appears and no new action appears after the operation of the ith step is executed, the Reward value is-100.
In a specific implementation, a reinforcement learning algorithm may be adopted to determine the multi-step operation profit value corresponding to each candidate action based on the n rewarded values corresponding to each candidate action, which is specifically described as follows: constructing n steps of discount income representation functions by adopting the reinforcement learning algorithm based on the n calculated Reward values and a preset attenuation coefficient; constructing a characterization function of the updated multi-step operation profit value corresponding to the candidate action based on the n-step discount profit characterization function, the preset learning rate and the characterization function of the pre-update multi-step operation profit value corresponding to each candidate action; and performing repeated iteration updating on the basis of the characterization function of the updated multi-step operation profit value corresponding to each candidate action until the preset convergence condition is met, so as to obtain the final multi-step operation profit value corresponding to each candidate action.
The n-step discount income characterization function comprises n addition items, wherein the first n-1 addition items comprise products of i-1 power of the attenuation coefficient gamma and the ith Reward value, and the nth addition item is a characterization function of the multi-step operation income value before updating corresponding to the nth step action; here, i is a positive integer from 1 to n.
The preset convergence condition means that the difference value between the multi-step operation income values obtained by the last two times of iteration updating is smaller than a set threshold value.
Here, the formula for calculating the multi-step operation profit value corresponding to each candidate action may be as shown in formula (i) and formula (ii):
Gt:t+n＝Rt+1+γRt+2+γ2Rt+3+…+γnQt+n-1(St+n，At+n) Formula (I)
Qt+n(St，At)＝Qt+n-1(St，At)+α[Gt:t+n-Qt+n-1(St，At) Formula 2
Wherein γ represents a preset attenuation coefficient; gt:t+nRepresenting n steps of discount income representing functions; rt+1State (S) indicated in the target test paget) The next execution candidate action jumps to the next state of the target test page (or jumps to a new test page) St+1The Reward value obtained; rt+2Is shown at St+1Execution of action jump to St+1Corresponding next state St+2The Reward value obtained; rt+3Is shown at St+2Execution of action jump to St+2Corresponding next state St+3The Reward value obtained; qt+n-1(St+n，At+n) Is shown at St+nAnd executing a characterization function of the yield value of the multi-step operation before updating corresponding to the action.
Wherein Q ist+n(St，At) Is shown at StExecuting a characterization function of the updated multi-step operation profit value corresponding to the candidate action; qt+n-1(St，At) Is shown at StAnd (8) executing a characterization function of the yield value of the multi-step operation before updating corresponding to the candidate action, wherein α is a preset learning rate.
In specific implementation, the n-step sarsa algorithm in the reinforcement learning algorithm is adopted, according to the formula (i) and the formula (ii), the multi-step operation profit value corresponding to each candidate action is iteratively updated for multiple times based on the n rewarded values corresponding to each candidate action, the preset attenuation coefficient and the preset learning rate until the preset convergence condition is satisfied, so as to obtain the final multi-step operation profit value corresponding to the candidate action, after the multi-step operation profit value is obtained, an action with a larger multi-step operation profit value and a smaller number of executed times can be selected from the multiple candidate actions as the target action, and the specific description is shown as step S103.
S103, selecting a target action from the plurality of candidate actions as an action for next test execution based on the executed times corresponding to the plurality of candidate actions and the multi-step operation profit value.
In a specific implementation, for each of the plurality of candidate actions, a first access report value corresponding to the candidate action may be determined based on the multi-step operation profit value corresponding to the candidate action and the number of times of access corresponding to the state; determining a second access report value corresponding to the candidate action based on the executed times corresponding to the candidate action, the visited times corresponding to the state and a preset balance coefficient; taking the sum of the first access report value and the second access report value as the total access report value of the candidate action; and selecting the candidate action with the maximum corresponding total access return value from the plurality of candidate actions as the target action.
Wherein, the formula expression can be shown as formula (three) as follows:
wherein, the UCB value of visited Action represents the total access return value, which is the total access return value corresponding to each candidate Action; ptRepresenting the multi-step operation profit value corresponding to each candidate action; vtIndicating the number of times the state corresponds to accessed.
Wherein, the first access report value is the ratio of the multi-step operation profit value corresponding to the candidate action to the number of accessed times corresponding to the state, that is to say
Wherein, VCRepresenting the number of executed times corresponding to the candidate action; c represents a preset balance coefficient and is used for balancing positive feedback brought by the multi-step operation income value corresponding to each candidate action and negative feedback brought by the executed times corresponding to each candidate action; where C may take the value of 1.4; second access return value in formula (III)
And (4) showing.
Exemplarily, it is determined according to step S103 that the total access report value corresponding to the long press control 1 operation, the total access report value corresponding to the click control 2 operation, the total access report value corresponding to the long press control 2 operation, the total access report value corresponding to the click control 3 operation, the total access report value corresponding to the long press control 2 operation, and the total access report value corresponding to the long press control 3 operation in the candidate action under the state of the target test page are 10, 20, 5, and 15; the operation click control 2 with the maximum total access return value is selected as the operation of the next test execution.
Illustratively, when it is determined according to S101 that the candidate actions under the state of the target test page are long press control 1, click control 2, long press control 2, click control 3, and long press control 3, the operation of long press control 2 is an unexecuted operation, the total access report value corresponding to the operation of long press control 1 is 5, the total access report value corresponding to the operation of click control 2 is 20, the total access report value corresponding to the operation of click control 3 is 10, and the total access report value corresponding to the operation of long press control 3 is 15; the operation to be not performed: and long pressing the control 2 as the next test execution operation.
In the application program testing process, multiple candidate operations (actions) selectable in the next testing process are determined for a current testing page, a multi-step operation profit value corresponding to each candidate action is calculated, a target action is selected from the multiple candidate actions as an action executed in the next testing process based on the executed times and the multi-step operation profit values corresponding to the candidate actions, and the corresponding action with a larger multi-step operation profit value and a smaller executed times can be selected as an action executed in the next testing process according to the multi-step operation profit value and the executed times corresponding to the candidate actions, so that invalid operations and repeated operations in the testing process of the current testing page are reduced, the value return of each testing process is improved, and the testing speed and efficiency are improved.
It will be understood by those skilled in the art that in the method of the present invention, the order of writing the steps does not imply a strict order of execution and any limitations on the implementation, and the specific order of execution of the steps should be determined by their function and possible inherent logic.
Based on the same inventive concept, an application program testing device corresponding to the application program testing method is also provided in the embodiments of the present disclosure, and since the principle of solving the problem of the device in the embodiments of the present disclosure is similar to the application program testing method in the embodiments of the present disclosure, the implementation of the device may refer to the implementation of the method, and repeated details are not described again.
Example two
Referring to fig. 2, a schematic diagram of an application testing apparatus 200 according to an embodiment of the present disclosure is provided, where the apparatus includes: a first determination module 201, a calculation module 202 and a second determination module 203; wherein: the first determining module 201 is configured to determine a plurality of candidate operation actions of a target test page in an application test process.
A calculating module 202, configured to calculate, for each candidate action, a multi-step operation profit value corresponding to the candidate action; the profit value of the multi-step operation is the profit value brought by starting to execute the multi-step action from the execution of the candidate action; the profit value is used for characterizing the contribution value of the multi-step operation on the test coverage growth rate of the page function.
The second determining module 203 is configured to select a target action from the multiple candidate actions as an action to be tested and executed next step based on the number of executed times and the multi-step operation profit value corresponding to the multiple candidate actions, respectively.
In the embodiment of the disclosure, in the application program testing process, a plurality of candidate operations (actions) selectable in the next test are determined for a current test page, a multi-step operation profit value corresponding to each candidate action is calculated, and based on the executed times and the multi-step operation profit values respectively corresponding to each candidate action, a target action is selected from the plurality of candidate actions as an action executed in the next test.
In a possible implementation manner, the second determining module 203 is further configured to select a target operation as an operation for next test execution from at least one unexecuted operation when it is determined that there is at least one unexecuted operation in the multiple candidate operations of the target test page.
In a possible implementation manner, the calculating module 202 is specifically configured to determine n Reward values corresponding to the candidate action based on the attribute information of the page element after each step of the n-step operation is executed; the n rewarded values comprise the ith rewarded value obtained after the ith action is executed, wherein the 1 st action is the candidate action; i is a positive integer from 1 to n; the Reward value is used for representing an initial contribution value of each step of operation on the test coverage growth rate of the page function; and calculating the multi-step operation income value by adopting a reinforcement learning algorithm based on the n rewarded values.
In one possible embodiment, the page element attribute information includes at least one of: whether a new page component appears after the ith action is executed; whether a new state appears after the ith action is executed; whether a new action exists after the action of the ith step is executed; the number of times of execution corresponding to the ith action.
In a possible implementation manner, the calculating module 202 is further specifically configured to perform multiple iterative updates on the multi-step operation profit value by using the reinforcement learning algorithm based on the n rewarded values and a preset attenuation coefficient and a preset learning rate until a preset convergence condition is met, so as to obtain a final multi-step operation profit value corresponding to the candidate action; the preset convergence condition means that the difference value between the multi-step operation income values obtained by the last two times of iteration updating is smaller than a set threshold value.
In a possible implementation manner, the calculating module 202 is further specifically configured to construct an n-step discount income representing function based on a preset attenuation coefficient and the n rewarded values; the n-step discount income characterization function comprises n addition items, wherein the first n-1 addition items comprise products of i-1 power of the attenuation coefficient gamma and the ith Reward value, and the nth addition item is a characterization function of the multi-step operation income value before updating corresponding to the nth step action; constructing a characterization function of the updated multi-step operation profit value corresponding to the candidate action based on the n-step discount profit characterization function, a preset learning rate and the characterization function of the updated multi-step operation profit value corresponding to the candidate action; and performing repeated iteration updating on the basis of the characterization function of the updated multi-step operation profit value corresponding to the candidate action until the preset convergence condition is met.
In a possible implementation manner, the second determining module 203 is specifically configured to determine, for each of the multiple candidate actions, a first access report value corresponding to the candidate action based on the multi-step operation benefit value corresponding to the candidate action and the number of times of access corresponding to the state; determining a second access report value corresponding to the candidate action based on the executed times corresponding to the candidate action, the visited times corresponding to the state and a preset balance coefficient; taking the sum of the first access report value and the second access report value as the total access report value of the candidate action; and selecting the candidate action with the maximum corresponding total access return value from the plurality of candidate actions as the target action.
The description of the processing flow of each module in the device and the interaction flow between the modules may refer to the related description in the above method embodiments, and will not be described in detail here.
Based on the same technical concept, the embodiment of the application also provides computer equipment. Referring to fig. 3, a schematic structural diagram of a computer device 300 provided in the embodiment of the present application includes a processor 301, a memory 302, and a bus 303. The memory 302 is used for storing execution instructions and includes a memory 3021 and an external memory 3022; the memory 3021 is also referred to as an internal memory, and is used for temporarily storing operation data in the processor 301 and data exchanged with an external memory 3022 such as a hard disk, the processor 301 exchanges data with the external memory 3022 through the memory 3021, and when the computer device 300 is operated, the processor 301 communicates with the memory 302 through the bus 303, so that the processor 301 executes the following instructions:
determining a plurality of candidate operations of a target test page in the test process of an application program; calculating a multi-step operation profit value corresponding to each candidate operation; the profit value of the multi-step operation is the profit value brought by the execution of the multi-step operation from the execution of the candidate operation; the profit value is used for representing the contribution value of the multi-step operation on the test coverage growth rate of the page function; and selecting a target operation from the candidate operations as an operation to be tested and executed next step based on the executed times and the multi-step operation profit values respectively corresponding to the candidate operations.
The embodiments of the present disclosure also provide a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the computer program performs the steps of the application program testing method in the above method embodiments. The storage medium may be a volatile or non-volatile computer-readable storage medium.
The computer program product of the application program testing method provided in the embodiments of the present disclosure includes a computer-readable storage medium storing a program code, where instructions included in the program code may be used to execute the steps of the application program testing method described in the above method embodiments, which may be referred to in the above method embodiments specifically, and are not described herein again.
The embodiments of the present disclosure also provide a computer program, which when executed by a processor implements any one of the methods of the foregoing embodiments. The computer program product may be embodied in hardware, software or a combination thereof. In an alternative embodiment, the computer program product is embodied in a computer storage medium, and in another alternative embodiment, the computer program product is embodied in a Software product, such as a Software Development Kit (SDK), or the like.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the system and the apparatus described above may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again. In the several embodiments provided in the present disclosure, it should be understood that the disclosed system, apparatus, and method may be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one logical division, and there may be other divisions when actually implemented, and for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or units through some communication interfaces, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present disclosure may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a non-volatile computer-readable storage medium executable by a processor. Based on such understanding, the technical solution of the present disclosure may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present disclosure. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
Finally, it should be noted that: the above-mentioned embodiments are merely specific embodiments of the present disclosure, which are used for illustrating the technical solutions of the present disclosure and not for limiting the same, and the scope of the present disclosure is not limited thereto, and although the present disclosure is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: any person skilled in the art can modify or easily conceive of the technical solutions described in the foregoing embodiments or equivalent technical features thereof within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the embodiments of the present disclosure, and should be construed as being included therein. Therefore, the protection scope of the present disclosure shall be subject to the protection scope of the claims.
Claims (10)
1. An application testing method, comprising:
determining a plurality of candidate operations of a target test page in the test process of an application program;
calculating a multi-step operation profit value corresponding to each candidate operation; the profit value of the multi-step operation is the profit value brought by the execution of the multi-step operation from the execution of the candidate operation; the profit value is used for representing the contribution value of the multi-step operation on the test coverage growth rate of the page function;
and selecting a target operation from the candidate operations as an operation to be tested and executed next step based on the executed times and the multi-step operation profit values respectively corresponding to the candidate operations.
2. The method of claim 1, wherein after determining the plurality of candidate operations, before calculating, for each of the candidate operations, a multi-step operation profit value corresponding to the candidate operation, further comprising:
when at least one unexecuted operation exists in the candidate operations of the target test page, selecting the target operation from the at least one unexecuted operation as an operation of next test execution.
3. The method of claim 1, wherein calculating, for each of the candidate operations, a multi-step operation profit value corresponding to the candidate operation comprises:
determining n report values corresponding to the candidate operation based on the page element attribute information after each step of the n steps of operations is executed; the n return values comprise an ith return value obtained after the operation of the ith step is executed, wherein the operation of the 1 st step is the candidate operation; i is a positive integer from 1 to n; the return value is used for representing an initial contribution value of each step of operation on the test coverage growth rate of the page function;
and calculating the multi-step operation income value by adopting a reinforcement learning algorithm based on the n return values.
4. The method of claim 3, wherein the page element attribute information comprises at least one of:
whether a new page component appears after the operation of the ith step is executed;
whether a new state appears after the operation of the ith step is executed;
whether a new operation exists after the operation of the ith step is executed;
the number of executed times corresponding to the operation of the ith step.
5. The method of claim 3, wherein calculating the multi-step operational yield value based on the n return values using a reinforcement learning algorithm comprises:
based on the n return values, preset attenuation coefficients and a preset learning rate, performing repeated iteration updating on the multi-step operation profit values by adopting the reinforcement learning algorithm until preset convergence conditions are met, and obtaining final multi-step operation profit values corresponding to the candidate operation;
the preset convergence condition means that the difference value between the multi-step operation income values obtained by the last two times of iteration updating is smaller than a set threshold value.
6. The method of claim 5, wherein the iteratively updating the multi-step operation profit value for a plurality of times by using the reinforcement learning algorithm based on the n return values and preset attenuation coefficients and learning rates until a preset convergence condition is met comprises:
constructing n steps of discount income representing functions based on preset attenuation coefficients and the n return values; the n-step discount income characterization function comprises n addition terms, wherein the first n-1 addition terms comprise products of i-1 power of the attenuation coefficient gamma and the ith return value, and the nth addition term is a characterization function which is corresponding to the nth step operation and used for updating the previous multi-step operation income value;
constructing a characterization function of the updated multi-step operation profit value corresponding to the candidate operation based on the n-step discount profit characterization function, a preset learning rate and the characterization function of the updated multi-step operation profit value corresponding to the candidate operation;
and performing repeated iteration updating on the basis of the characterization function of the updated multi-step operation profit value corresponding to the candidate operation until the preset convergence condition is met.
7. The method of claim 1, wherein selecting a target operation from the plurality of candidate operations as a next test execution operation based on the number of executed times and the profit value of the multi-step operation respectively corresponding to the plurality of candidate operations comprises:
for each candidate operation in the multiple candidate operations, determining a first access return value corresponding to the candidate operation based on the multi-step operation profit value corresponding to the candidate operation and the number of times of access corresponding to the current state of the target test page; and the number of the first and second groups,
determining a second access report value corresponding to the candidate operation based on the executed times corresponding to the candidate operation, the accessed times corresponding to the current state and a preset balance coefficient;
determining a total access return value for the candidate operation as a sum of the first access return value and the second access return value;
and selecting the candidate operation with the maximum corresponding total access return value from the plurality of candidate operations as the target operation.
8. An application testing apparatus, comprising:
the first determination module is used for determining a plurality of candidate operations of a target test page in the test process of the application program;
the calculation module is used for calculating a multi-step operation profit value corresponding to each candidate operation; the profit value of the multi-step operation is the profit value brought by the execution of the multi-step operation from the execution of the candidate operation; the profit value is used for representing the contribution value of the multi-step operation on the test coverage growth rate of the page function;
and the second determining module is used for selecting a target operation from the multiple candidate operations as an operation to be tested and executed next step based on the executed times and the multi-step operation profit values respectively corresponding to the multiple candidate operations.
9. A computer device, comprising: a processor, a memory and a bus, the memory storing machine-readable instructions executable by the processor, the processor and the memory communicating over the bus when a computer device is running, the machine-readable instructions when executed by the processor performing the steps of the application testing method of any of claims 1 to 7.
10. A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, which computer program, when being executed by a processor, performs the steps of the application testing method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010754358.9A CN111694753B (en) | 2020-07-30 | 2020-07-30 | Application program testing method and device and computer storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010754358.9A CN111694753B (en) | 2020-07-30 | 2020-07-30 | Application program testing method and device and computer storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111694753A true CN111694753A (en) | 2020-09-22 |
| CN111694753B CN111694753B (en) | 2023-04-11 |
Family
ID=72487108
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010754358.9A Active CN111694753B (en) | 2020-07-30 | 2020-07-30 | Application program testing method and device and computer storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111694753B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022227943A1 (en) * | 2021-04-30 | 2022-11-03 | 北京字节跳动网络技术有限公司 | Application program testing method and apparatus, computer device, and storage medium |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030208744A1 (en) * | 2002-05-06 | 2003-11-06 | Microsoft Corporation | Method and system for generating test matrices for software programs |
| CN104915290A (en) * | 2015-05-28 | 2015-09-16 | 小米科技有限责任公司 | Application testing method and device |
| CN107357725A (en) * | 2017-06-29 | 2017-11-17 | 腾讯科技（深圳）有限公司 | Application testing method and device |
| CN107562613A (en) * | 2016-06-30 | 2018-01-09 | 阿里巴巴集团控股有限公司 | Program testing method, apparatus and system |
| US20180039568A1 (en) * | 2016-08-03 | 2018-02-08 | International Business Machines Corporation | Test case generation for uncovered code paths |
| CN108038059A (en) * | 2017-12-22 | 2018-05-15 | 广州酷狗计算机科技有限公司 | Interface traversal method and device |
| CN108304324A (en) * | 2018-01-22 | 2018-07-20 | 百度在线网络技术（北京）有限公司 | Method for generating test case, device, equipment and storage medium |
| CN108334439A (en) * | 2018-03-14 | 2018-07-27 | 百度在线网络技术（北京）有限公司 | A kind of method for testing pressure, device, equipment and storage medium |
| US20190065182A1 (en) * | 2017-08-24 | 2019-02-28 | International Business Machines Corporation | Automatic machine-learning high value generator |
| US20190196943A1 (en) * | 2017-12-26 | 2019-06-27 | Hitachi, Ltd. | Coverage test support device and coverage test support method |
| CN109977029A (en) * | 2019-04-09 | 2019-07-05 | 科大讯飞股份有限公司 | A kind of training method and device of page jump model |
| CN111008154A (en) * | 2019-12-31 | 2020-04-14 | 南京大学 | Android application automatic test method and system based on reinforcement learning |
| CN111078579A (en) * | 2019-12-31 | 2020-04-28 | 河北省科学院应用数学研究所 | Monkey testing method and device and terminal equipment |
-
2020
- 2020-07-30 CN CN202010754358.9A patent/CN111694753B/en active Active
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030208744A1 (en) * | 2002-05-06 | 2003-11-06 | Microsoft Corporation | Method and system for generating test matrices for software programs |
| CN104915290A (en) * | 2015-05-28 | 2015-09-16 | 小米科技有限责任公司 | Application testing method and device |
| CN107562613A (en) * | 2016-06-30 | 2018-01-09 | 阿里巴巴集团控股有限公司 | Program testing method, apparatus and system |
| US20180039568A1 (en) * | 2016-08-03 | 2018-02-08 | International Business Machines Corporation | Test case generation for uncovered code paths |
| CN107357725A (en) * | 2017-06-29 | 2017-11-17 | 腾讯科技（深圳）有限公司 | Application testing method and device |
| US20190065182A1 (en) * | 2017-08-24 | 2019-02-28 | International Business Machines Corporation | Automatic machine-learning high value generator |
| CN108038059A (en) * | 2017-12-22 | 2018-05-15 | 广州酷狗计算机科技有限公司 | Interface traversal method and device |
| US20190196943A1 (en) * | 2017-12-26 | 2019-06-27 | Hitachi, Ltd. | Coverage test support device and coverage test support method |
| CN108304324A (en) * | 2018-01-22 | 2018-07-20 | 百度在线网络技术（北京）有限公司 | Method for generating test case, device, equipment and storage medium |
| CN108334439A (en) * | 2018-03-14 | 2018-07-27 | 百度在线网络技术（北京）有限公司 | A kind of method for testing pressure, device, equipment and storage medium |
| CN109977029A (en) * | 2019-04-09 | 2019-07-05 | 科大讯飞股份有限公司 | A kind of training method and device of page jump model |
| CN111008154A (en) * | 2019-12-31 | 2020-04-14 | 南京大学 | Android application automatic test method and system based on reinforcement learning |
| CN111078579A (en) * | 2019-12-31 | 2020-04-28 | 河北省科学院应用数学研究所 | Monkey testing method and device and terminal equipment |
Non-Patent Citations (2)
| Title |
|---|
| ARAVIND MACHIRY: "Dynodroid: An Input Generation System for Android Apps" * |
| PRIYAM PATEL: "On the Effectiveness of Random Testing for Android" * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022227943A1 (en) * | 2021-04-30 | 2022-11-03 | 北京字节跳动网络技术有限公司 | Application program testing method and apparatus, computer device, and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111694753B (en) | 2023-04-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106023015B (en) | Course learning path recommendation method and device | |
| CN112085172B (en) | Method and device for training graph neural network | |
| CN108833458B (en) | Application recommendation method, device, medium and equipment | |
| CN109685204B (en) | Image processing method and device, storage medium and electronic equipment | |
| CN111582479B (en) | Distillation method and device for neural network model | |
| CN106327240A (en) | Recommendation method and recommendation system based on GRU neural network | |
| CN110619082B (en) | Project recommendation method based on repeated search mechanism | |
| CN109388674A (en) | Data processing method, device, equipment and readable storage medium storing program for executing | |
| CN113127727A (en) | Information recommendation model determination method, information recommendation method and device | |
| CN108667877B (en) | Method and device for determining recommendation information, computer equipment and storage medium | |
| CN111694753B (en) | Application program testing method and device and computer storage medium | |
| US11468521B2 (en) | Social media account filtering method and apparatus | |
| CN115455280A (en) | Recommendation list determining method and server | |
| CN109117442A (en) | A kind of application recommended method and device | |
| CN113963248A (en) | Method, device, equipment and storage medium for neural network training and scene decision | |
| CN106919693B (en) | Method and device for improving hot word exposure coverage rate | |
| JP2020079991A (en) | Optimization apparatus, control method of optimization apparatus, and control program of optimization apparatus | |
| CN111652673A (en) | Intelligent recommendation method, device, server and storage medium | |
| CN113449176A (en) | Recommendation method and device based on knowledge graph | |
| US10872121B2 (en) | Systems and methods for matching users | |
| CN108848152A (en) | A kind of method and server of object recommendation | |
| CN112084412B (en) | Information pushing method, device, equipment and storage medium | |
| CN111506382B (en) | Progress bar curve determination method and device, storage medium and electronic equipment | |
| CN114201696A (en) | Message pushing method and device, storage medium and computer equipment | |
| WO2022227943A1 (en) | Application program testing method and apparatus, computer device, and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP01 | Change in the name or title of a patent holder | ||
| CP01 | Change in the name or title of a patent holder |
Address after: 100041 B-0035, 2 floor, 3 building, 30 Shixing street, Shijingshan District, Beijing. Patentee after: Douyin Vision Co.,Ltd. Address before: 100041 B-0035, 2 floor, 3 building, 30 Shixing street, Shijingshan District, Beijing. Patentee before: Tiktok vision (Beijing) Co.,Ltd. Address after: 100041 B-0035, 2 floor, 3 building, 30 Shixing street, Shijingshan District, Beijing. Patentee after: Tiktok vision (Beijing) Co.,Ltd. Address before: 100041 B-0035, 2 floor, 3 building, 30 Shixing street, Shijingshan District, Beijing. Patentee before: BEIJING BYTEDANCE NETWORK TECHNOLOGY Co.,Ltd. |