CN110209581A - Trigger action analysis method and device based on application program - Google Patents
Trigger action analysis method and device based on application program Download PDFInfo
- Publication number
- CN110209581A CN110209581A CN201910468276.5A CN201910468276A CN110209581A CN 110209581 A CN110209581 A CN 110209581A CN 201910468276 A CN201910468276 A CN 201910468276A CN 110209581 A CN110209581 A CN 110209581A
- Authority
- CN
- China
- Prior art keywords
- video image
- record
- determining
- picture
- application program
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000004458 analytical method Methods 0.000 title claims abstract description 46
- 238000000034 method Methods 0.000 claims abstract description 49
- 238000006073 displacement reaction Methods 0.000 claims description 45
- 230000008859 change Effects 0.000 claims description 28
- 230000006870 function Effects 0.000 claims description 18
- 238000012545 processing Methods 0.000 claims description 13
- 230000008569 process Effects 0.000 claims description 12
- 238000009432 framing Methods 0.000 claims description 5
- 238000012360 testing method Methods 0.000 abstract description 16
- 238000010586 diagram Methods 0.000 description 13
- 238000004590 computer program Methods 0.000 description 7
- 238000013500 data storage Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 5
- 230000000694 effects Effects 0.000 description 4
- 238000012986 modification Methods 0.000 description 4
- 230000004048 modification Effects 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 4
- 238000004891 communication Methods 0.000 description 3
- 238000011161 development Methods 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3688—Test management for test execution, e.g. scheduling of test suites
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Television Signal Processing For Recording (AREA)
Abstract
A kind of trigger action analysis method, device, electronic equipment and computer readable storage medium based on application program of the disclosure.Wherein method includes: the video image for receiving terminal and sending；Wherein, the video image includes the operation note that application program is operated on the terminal screen；Operation note is obtained according to the video image；Operation information relevant to the operation is determined according to the operation note；The operation information is analyzed to obtain operating result.The embodiment of the present disclosure obtains operation note by the video image sent according to terminal；Operation information relevant to the operation is determined according to the operation note；The operation information is analyzed to obtain operating result, so that when obtaining the operation for being directed to application program, it is not necessary to modify the source codes of application program, reduce testing cost, and what the disclosure obtained is operated outside application program, the test of all application programs can be used, there is general applicability.
Description
Technical Field
The present disclosure relates to the field of mobile terminal technologies, and in particular, to a method and an apparatus for analyzing trigger operations based on an application program, and a computer-readable storage medium.
Background
The application test technology is an important component in the application development process, and is an activity process for verifying and confirming application products (including staged products) throughout the whole application development life cycle, and the purpose of the technology is to discover various problems existing in the application products, namely, inconsistency with user requirements and predefined inconsistency as soon as possible. And checking the bug of the application program product, writing a test report, and submitting the test report to a developer for modification. The basic goal of application testers is to discover errors in an application.
In the process of the application program, there is usually a need to acquire a trigger operation (for example, a click position or a slide) of a user for the application program, and at present, a common scheme is to modify an application program source code and output the trigger operation for the application program in a log form.
Disclosure of Invention
The technical problem to be solved by the present disclosure is to provide an application-based trigger operation analysis method, so as to at least partially solve the technical problems in the prior art that source codes need to be modified, the test cost is high, external operations of the application cannot be obtained, and certain limitations are present. In addition, an application-based trigger operation analysis device, an application-based trigger operation analysis hardware device, a computer-readable storage medium and an application-based trigger operation analysis terminal are also provided.
In order to achieve the above object, according to one aspect of the present disclosure, the following technical solutions are provided:
an application-based trigger operation analysis method comprises the following steps:
receiving a video image sent by a terminal; wherein the video image contains an operation record of operating an application program on the terminal screen;
acquiring an operation record according to the video image;
determining operation information related to the operation according to the operation record;
and analyzing the operation information to obtain an operation result.
Further, the acquiring the operation record according to the video image includes:
performing framing processing on the video image according to a video frame rate, and obtaining a picture sequence according to the time sequence relation of each frame of picture;
and acquiring an operation record according to the picture sequence.
Further, the determining operation information related to the operation according to the operation record includes:
traversing the picture sequence, and determining a picture and an operation initial position corresponding to the start of operation according to a first change in the current picture operation record;
and determining the corresponding picture and the displacement offset when the operation is finished according to the second change in the subsequent picture operation record of the current picture.
Further, the analyzing the operation information to obtain an operation result includes:
determining operation time according to the picture corresponding to the operation starting time and the picture corresponding to the operation ending time; and/or
Determining an operation mode according to the displacement offset; and/or
And determining an operation position according to the operation starting position and the displacement offset.
Further, the determining the operation time according to the picture corresponding to the operation start time and the picture corresponding to the operation end time includes:
determining the interval time of two adjacent pictures in the picture sequence;
determining the number of frames from the corresponding picture at the beginning of the operation to the corresponding picture at the end of the operation according to the time sequence relation;
and taking the product of the frame number and the interval time as the operation time.
Further, the interval time is an inverse of the video frame rate.
Further, the determining an operation mode according to the displacement offset includes:
if the abscissa and the ordinate of the displacement offset are both 0, determining that the operation is a click operation; and if the abscissa and the ordinate of the displacement offset are not 0, determining that the operation is a sliding operation.
Further, the determining an operation position according to the operation start position and the displacement offset includes:
if the operation is a click operation, the operation position is the operation starting position; and if the operation is a sliding operation, the operation position is the operation starting position plus the displacement offset.
In order to achieve the above object, according to one aspect of the present disclosure, the following technical solutions are provided:
an application-based trigger operation analysis method comprises the following steps:
acquiring a video image; wherein the video image comprises an operation record of operating an application program on a terminal screen;
and sending the video image to a server so that the server acquires an operation record according to the video image, determines operation information related to the operation according to the operation record, and analyzes the operation information to obtain an operation result.
Further, a recording function is provided in the setting options of the terminal;
correspondingly, the acquiring the video image comprises the following steps:
and starting to record the terminal screen according to the starting operation of the recording function, and stopping recording the terminal screen according to the recording stopping operation to obtain the video image.
Further, the method further comprises:
in the recording process, detecting the operation of the application program on the terminal screen and generating an operation record;
including the operation record in the video image.
Further, the operation record contains at least one of the following information: a first change, an operation start time, an operation start position, a second change, an operation end time, and a displacement offset amount.
In order to achieve the above object, according to one aspect of the present disclosure, the following technical solutions are provided:
an application-based trigger operation analysis apparatus comprising:
the image receiving module is used for receiving the video image sent by the terminal; wherein the video image contains an operation record of operating an application program on the terminal screen;
the record acquisition module is used for acquiring operation records according to the video images;
the information determining module is used for determining operation information related to the operation according to the operation record;
and the operation analysis module is used for analyzing the operation information to obtain an operation result.
Further, the record obtaining module is specifically configured to: performing framing processing on the video image according to a video frame rate, and obtaining a picture sequence according to the time sequence relation of each frame of picture; and acquiring an operation record according to the picture sequence.
Further, the information determination module is specifically configured to: traversing the picture sequence, and determining a picture and an operation initial position corresponding to the start of operation according to a first change in the current picture operation record; and determining the corresponding picture and the displacement offset when the operation is finished according to the second change in the subsequent picture operation record of the current picture.
Further, the operation analysis module is specifically configured to: determining operation time according to the picture corresponding to the operation starting time and the picture corresponding to the operation ending time; and/or determining an operation mode according to the displacement offset; and/or determining an operation position according to the operation starting position and the displacement offset.
Further, the operation analysis module is specifically configured to: determining the interval time of two adjacent pictures in the picture sequence; determining the number of frames from the corresponding picture at the beginning of the operation to the corresponding picture at the end of the operation according to the time sequence relation; and taking the product of the frame number and the interval time as the operation time.
Further, the interval time is an inverse of the video frame rate.
Further, the operation analysis module is specifically configured to: if the abscissa and the ordinate of the displacement offset are both 0, determining that the operation is a click operation; and if the abscissa and the ordinate of the displacement offset are not 0, determining that the operation is a sliding operation.
Further, the operation analysis module is specifically configured to: if the operation is a click operation, the operation position is the operation starting position; and if the operation is a sliding operation, the operation position is the operation starting position plus the displacement offset.
In order to achieve the above object, according to one aspect of the present disclosure, the following technical solutions are provided:
an application-based trigger operation analysis apparatus comprising:
the image acquisition module is used for acquiring a video image; wherein the video image comprises an operation record of operating an application program on a terminal screen;
and the image sending module is used for sending the video image to a server so that the server can obtain an operation record according to the video image, determine operation information related to the operation according to the operation record, and analyze the operation information to obtain an operation result.
Further, a recording function is provided in the setting options of the terminal;
correspondingly, the image acquisition module is specifically configured to: and starting to record the terminal screen according to the starting operation of the recording function, and stopping recording the terminal screen according to the recording stopping operation to obtain the video image.
Further, the apparatus further comprises:
the recording module is used for detecting the operation of the terminal screen on the application program and generating an operation record in the recording process; including the operation record in the video image.
Further, the operation record contains at least one of the following information: a first change, an operation start time, an operation start position, a second change, an operation end time, and a displacement offset amount.
In order to achieve the above object, according to one aspect of the present disclosure, the following technical solutions are provided:
a server, comprising:
a memory for storing non-transitory computer readable instructions; and
a processor for executing the computer readable instructions, so that the processor implements any one of the above-mentioned trigger operation analysis methods when executed.
In order to achieve the above object, according to one aspect of the present disclosure, the following technical solutions are provided:
a computer-readable storage medium storing non-transitory computer-readable instructions that, when executed by a computer, cause the computer to perform any one of the above-described application-based trigger operation analysis methods.
In order to achieve the above object, according to one aspect of the present disclosure, the following technical solutions are provided:
an electronic device, comprising:
a memory for storing non-transitory computer readable instructions; and
a processor for executing the computer readable instructions, so that the processor implements any of the above-mentioned trigger operation analysis methods based on the application program when executing the computer readable instructions.
In order to achieve the above object, according to one aspect of the present disclosure, the following technical solutions are provided:
a computer readable storage medium storing non-transitory computer readable instructions which, when executed by a computer, cause the computer to perform any of the application-based trigger operation analysis methods described above.
In order to achieve the above object, according to still another aspect of the present disclosure, the following technical solutions are also provided:
an application-based trigger operation analysis terminal comprises any one of the application-based trigger operation analysis devices.
The embodiment of the disclosure acquires the operation record according to the video image sent by the terminal; determining operation information related to the operation according to the operation record; the operation information is analyzed to obtain the operation result, so that the source code of the application program does not need to be modified when the operation aiming at the application program is obtained, the test cost is reduced, the operation outside the application program is obtained by the method, the test on all the application programs can be used, and the method has universal applicability.
The foregoing is a summary of the present disclosure, and for the purposes of promoting a clear understanding of the technical means of the present disclosure, the present disclosure may be embodied in other specific forms without departing from the spirit or essential attributes thereof.
Drawings
FIG. 1a is a schematic flow chart diagram of a method for application-based analysis of trigger operations according to an embodiment of the present disclosure;
FIG. 1b is a schematic diagram illustrating an example of operation start and process in an application-based trigger operation analysis method according to an embodiment of the present disclosure;
fig. 1c is a schematic flowchart of an example after a click operation in an application-based trigger operation analysis method according to an embodiment of the present disclosure is completed;
FIG. 1d is a schematic diagram illustrating an example of a sliding operation in an application-based trigger operation analysis method according to an embodiment of the present disclosure;
fig. 2 is a flowchart illustrating a table data storage method based on a mobile terminal according to an embodiment of the present disclosure;
FIG. 3 is a schematic structural diagram of an apparatus for analyzing trigger operation based on an application according to an embodiment of the present disclosure;
FIG. 4 is a schematic structural diagram of an apparatus for analyzing trigger operation based on an application according to an embodiment of the present disclosure;
FIG. 5 is a schematic block diagram of a server according to one embodiment of the present disclosure;
fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure.
Detailed Description
The embodiments of the present disclosure are described below with specific examples, and other advantages and effects of the present disclosure will be readily apparent to those skilled in the art from the disclosure in the specification. It is to be understood that the described embodiments are merely illustrative of some, and not restrictive, of the embodiments of the disclosure. The disclosure may be embodied or carried out in various other specific embodiments, and various modifications and changes may be made in the details within the description without departing from the spirit of the disclosure. It is to be noted that the features in the following embodiments and examples may be combined with each other without conflict. All other embodiments, which can be derived by a person skilled in the art from the embodiments disclosed herein without making any creative effort, shall fall within the protection scope of the present disclosure.
It is noted that various aspects of the embodiments are described below within the scope of the appended claims. It should be apparent that the aspects described herein may be embodied in a wide variety of forms and that any specific structure and/or function described herein is merely illustrative. Based on the disclosure, one skilled in the art should appreciate that one aspect described herein may be implemented independently of any other aspects and that two or more of these aspects may be combined in various ways. For example, an apparatus may be implemented and/or a method practiced using any number of the aspects set forth herein. Additionally, such an apparatus may be implemented and/or such a method may be practiced using other structure and/or functionality in addition to one or more of the aspects set forth herein.
It should be noted that the drawings provided in the following embodiments are only for illustrating the basic idea of the present disclosure, and the drawings only show the components related to the present disclosure rather than the number, shape and size of the components in actual implementation, and the type, amount and ratio of the components in actual implementation may be changed arbitrarily, and the layout of the components may be more complicated.
In addition, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, it will be understood by those skilled in the art that the aspects may be practiced without these specific details.
Example one
In order to solve the technical problems that in the prior art, source codes need to be modified, testing cost is high, external operations of an application program cannot be obtained, and certain limitations exist, the embodiment of the disclosure provides an application-based trigger operation analysis method. The execution subject of the present embodiment is a server. As shown in fig. 1a, the method for analyzing trigger operation based on application mainly includes the following steps S11 to S14.
Step S11: receiving a video image sent by a terminal; wherein the video image contains an operation record of an operation application program on the terminal screen.
The terminal may be a mobile terminal, such as a tablet computer, a smart phone, a wearable device, or a fixed terminal, such as a desktop computer.
The video image may be a video image of a user operation application program recorded by the terminal, or a video image of a user operation application program shot by an external video recording device. The video image includes an operation record.
The operation record is presented at a preset position of a terminal screen in a table form, wherein the preset position can be the top of the terminal. The operation record includes information such as a first change, an operation start position, an operation start time, a second change, an operation end time, and a displacement offset.
Wherein the first change indicates the start of the operation, for example, the position identifier (dx, dy) in the operation record is changed to (x, y); the second change signifies the end of the operation, for example, the position indicator (x, y) in the operation record changes to (dx, dy).
The application program may be any application program, such as a short video application, a WeChat application, and the like.
Step S12: and acquiring operation records according to the video images.
Since the operation record is included in the video in the form of an image, the operation record needs to be parsed from the video image.
Step S13: and determining operation information related to the operation according to the operation record.
The operation information may be information such as a corresponding picture at the start of the operation, and/or an operation start position, and/or a corresponding picture at the end of the operation, and/or a displacement offset.
Step S14: and analyzing the operation information to obtain an operation result.
The operation result may be an operation time, an operation mode, and/or an operation position.
The operation mode may be clicking or sliding.
The embodiment acquires the operation record according to the video image sent by the terminal; determining operation information related to the operation according to the operation record; the operation information is analyzed to obtain the operation result, so that the source code of the application program does not need to be modified when the operation aiming at the application program is obtained, the test cost is reduced, the operation outside the application program is obtained by the method, the test on all the application programs can be used, and the method has universal applicability.
In an optional embodiment, step S12 specifically includes:
step S121: and performing framing processing on the video image according to the video frame rate, and obtaining a picture sequence according to the time sequence relation of each frame of picture.
The video frame rate is a measure for measuring the number of display frames, and the measurement unit is the number of display frames per second.
Specifically, the ffmpeg command or opencv command may be used to obtain the video frame rate, the ffmpeg command is used to decompose the video into pictures according to the video frame rate, and the pictures are ordered according to the time sequence to form a picture sequence and stored.
Step S122: and acquiring an operation record according to the picture sequence.
Specifically, the pictures are traversed according to the picture sequence, the pictures are cut in the traversing process for facilitating subsequent processing, the pictures only containing operation records are obtained, and operation record texts on the pictures are obtained through a text recognition technology. Fig. 1b shows the operation recording texts at the beginning and during the operation, where (dx, dy) in the texts is (x, y), fig. 1c shows the operation recording texts after the click operation is completed (dx is 0.0 and dy is 0.0), and fig. 1d shows the operation recording texts after the slide operation is completed (dx is 254.8 and dy is-289.8).
In an optional embodiment, step S13 specifically includes:
step S131: and traversing the picture sequence, and determining a picture and an operation initial position corresponding to the start of operation according to the first change in the current picture operation record.
Wherein the first change is indicative of the start of operation. For example, as shown in fig. 1b, the position indicator (dx, dy) in the operation record changes to (x, y) as the operation starts and progresses.
Step S132: and determining the corresponding picture and the displacement offset when the operation is finished according to the second change in the subsequent picture operation record of the current picture.
Wherein the second change indicates the end of the operation, for example, the position indicator (x, y) in the operation record is changed to (dx, dy).
For example, as shown in fig. 1c (dx is 0.0 and dy is 0.0), the corresponding displacement offset is (0,0), and as shown in fig. 1d (dx is 254.8 and dy is-289.8), the corresponding displacement offset is (254.8, -289.80).
In an optional embodiment, step S14 specifically includes:
determining operation time according to the picture corresponding to the operation starting time and the picture corresponding to the operation ending time; and/or determining an operation mode according to the displacement offset; and/or determining an operation position according to the operation starting position and the displacement offset.
Further, the determining the operation time according to the picture corresponding to the operation start time and the picture corresponding to the operation end time includes:
determining the interval time of two adjacent pictures in the picture sequence;
determining the number of frames from the corresponding picture at the beginning of the operation to the corresponding picture at the end of the operation according to the time sequence relation;
and taking the product of the frame number and the interval time as the operation time.
Wherein the interval time is the reciprocal of the video frame rate.
Further, the determining an operation mode according to the displacement offset includes:
if the abscissa and the ordinate of the displacement offset are both 0, determining that the operation is a click operation; and if the abscissa and the ordinate of the displacement offset are not 0, determining that the operation is a sliding operation.
Further, the determining an operation position according to the operation start position and the displacement offset includes:
if the operation is a click operation, the operation position is the operation starting position; and if the operation is a sliding operation, the operation position is the operation starting position plus the displacement offset.
Example two
In order to solve the technical problems that in the prior art, source codes need to be modified, the test cost is high, external operations of an application program cannot be obtained, and certain limitations exist, an embodiment of the present disclosure further provides an application-based trigger operation analysis method, and an execution main body of the embodiment is a terminal. As shown in fig. 2, the method specifically includes:
s21, acquiring a video image; wherein the video image contains an operation record of operating an application program on a terminal screen.
The terminal may be a mobile terminal, such as a tablet computer, a smart phone, a wearable device, or a fixed terminal, such as a desktop computer.
The video image may be a video image of a user operation application program recorded by the terminal, or a video image of a user operation application program shot by an external video recording device. The video image includes an operation record.
The operation record is presented at a preset position of a terminal screen in a table form, wherein the preset position can be the top of the terminal. Wherein, the operation record contains at least one item of information of: a first change, an operation start time, an operation start position, a second change, an operation end time, and a displacement offset amount.
Wherein the first change indicates the start of the operation, for example, the position identifier (dx, dy) in the operation record is changed to (x, y); the second change signifies the end of the operation, for example, the position indicator (x, y) in the operation record changes to (dx, dy).
The application program may be any application program, such as a short video application, a WeChat application, and the like.
And S22, sending the video image to a server so that the server can obtain an operation record according to the video image, determine operation information related to the operation according to the operation record, and analyze the operation information to obtain an operation result.
In the embodiment, the video image is acquired and sent to the server, so that the server acquires the operation record according to the video image; determining operation information related to the operation according to the operation record; the operation information is analyzed to obtain the operation result, so that the source code of the application program does not need to be modified when the operation aiming at the application program is obtained, the test cost is reduced, the operation outside the application program is obtained by the method, the test on all the application programs can be used, and the method has universal applicability.
In an optional embodiment, a recording function is provided in a setting option of the terminal;
correspondingly, step S21 specifically includes:
and starting to record the terminal screen according to the starting operation of the recording function, and stopping recording the terminal screen according to the recording stopping operation to obtain the video image.
Specifically, a pointer position, namely a recording function, can be provided in developer options set by the terminal, the terminal screen starts recording by starting the pointer position, a user performs clicking and sliding operations on the terminal screen, and the screen recording is stopped after the operation is completed, so that a video image is obtained.
In an optional embodiment, the method further comprises:
step S23: and in the recording process, detecting the operation of the terminal screen on the application program and generating an operation record.
Step S24: including the operation record in the video image.
The form of the generated operation record may refer to the above embodiments, and is not described herein again.
It will be appreciated by those skilled in the art that obvious modifications (e.g., combinations of the enumerated modes) or equivalents may be made to the above-described embodiments.
In the above, although the steps in the embodiment of the method for analyzing trigger operation based on the application program are described in the above sequence, it should be clear to those skilled in the art that the steps in the embodiment of the present disclosure are not necessarily performed in the above sequence, and may also be performed in other sequences such as reverse, parallel, and cross, and further, on the basis of the above steps, those skilled in the art may also add other steps, and these obvious modifications or equivalents should also be included in the protection scope of the present disclosure, and are not described herein again.
For convenience of description, only the relevant parts of the embodiments of the present disclosure are shown, and details of the specific techniques are not disclosed, please refer to the embodiments of the method of the present disclosure.
EXAMPLE III
In order to solve the technical problems that source codes need to be modified, testing cost is high, external operation of an application program cannot be obtained, and certain limitations exist in the prior art, the embodiment of the disclosure provides an application program-based trigger operation analysis device. The apparatus may perform the steps in the embodiment of the method for analyzing trigger operation based on application program according to the first embodiment, and the apparatus may be configured in a server. As shown in fig. 3, the apparatus mainly includes: an image receiving module 31, a record acquiring module 32, an information determining module 33 and an operation analyzing module 34; wherein,
the image receiving module 31 is used for receiving a video image sent by a terminal; wherein the video image contains an operation record of operating an application program on the terminal screen;
the record acquiring module 32 is configured to acquire an operation record according to the video image;
the information determining module 33 is configured to determine operation information related to the operation according to the operation record;
the operation analysis module 34 is configured to analyze the operation information to obtain an operation result.
Further, the record obtaining module 32 is specifically configured to: performing framing processing on the video image according to a video frame rate, and obtaining a picture sequence according to the time sequence relation of each frame of picture; and acquiring an operation record according to the picture sequence.
Further, the information determining module 33 is specifically configured to: traversing the picture sequence, and determining a picture and an operation initial position corresponding to the start of operation according to a first change in the current picture operation record; and determining the corresponding picture and the displacement offset when the operation is finished according to the second change in the subsequent picture operation record of the current picture.
Further, the operation analysis module 34 is specifically configured to: determining operation time according to the picture corresponding to the operation starting time and the picture corresponding to the operation ending time; and/or determining an operation mode according to the displacement offset; and/or determining an operation position according to the operation starting position and the displacement offset.
Further, the operation analysis module 34 is specifically configured to: determining the interval time of two adjacent pictures in the picture sequence; determining the number of frames from the corresponding picture at the beginning of the operation to the corresponding picture at the end of the operation according to the time sequence relation; and taking the product of the frame number and the interval time as the operation time.
Further, the interval time is an inverse of the video frame rate.
Further, the operation analysis module 34 is specifically configured to: if the abscissa and the ordinate of the displacement offset are both 0, determining that the operation is a click operation; and if the abscissa and the ordinate of the displacement offset are not 0, determining that the operation is a sliding operation.
Further, the operation analysis module 34 is specifically configured to: if the operation is a click operation, the operation position is the operation starting position; and if the operation is a sliding operation, the operation position is the operation starting position plus the displacement offset.
For detailed descriptions of the working principle, the achieved technical effect, and the like of the embodiment of the trigger operation analysis device based on the application program, reference may be made to the related descriptions in the foregoing embodiment of the trigger operation analysis method based on the application program, and no further description is given here.
Example four
In order to solve the technical problems that in the prior art, source codes need to be modified, the testing cost is high, external operations of application programs cannot be obtained, and certain limitations are caused, the embodiment of the disclosure provides a table data storage device based on a mobile terminal. The apparatus may perform the steps in the mobile terminal-based table data storage method embodiment described in the second embodiment, and the apparatus may be configured in a terminal. As shown in fig. 4, the apparatus mainly includes: an image acquisition module 41 and an image transmission module 42; wherein,
the image obtaining module 41 is used for obtaining video images; wherein the video image comprises an operation record of operating an application program on a terminal screen;
the image sending module 42 is configured to send the video image to a server, so that the server obtains an operation record according to the video image, determines operation information related to the operation according to the operation record, and analyzes the operation information to obtain an operation result.
Further, a recording function is provided in the setting options of the terminal;
correspondingly, the image obtaining module 41 is specifically configured to: and starting to record the terminal screen according to the starting operation of the recording function, and stopping recording the terminal screen according to the recording stopping operation to obtain the video image.
Further, the apparatus further comprises: a recording module 43; wherein,
the recording module 43 is configured to detect an operation on the application program on the terminal screen in a recording process, and generate an operation record; including the operation record in the video image.
Further, the operation record contains at least one of the following information: a first change, an operation start time, an operation start position, a second change, an operation end time, and a displacement offset amount.
For detailed descriptions of the working principle, the realized technical effect, and the like of the embodiment of the table data storage device based on the mobile terminal, reference may be made to the related descriptions in the foregoing embodiment of the table data storage method based on the mobile terminal, and no further description is given here.
EXAMPLE five
Fig. 5 is a block diagram illustrating an apparatus 500 for application-based trigger operation analysis, according to an example embodiment. For example, the apparatus 500 may be provided as a server. Referring to fig. 5, the apparatus 500 includes a processing component 522 that further includes one or more processors and memory resources, represented by memory 532, for storing instructions, such as applications, that are executable by the processing component 522. The application programs stored in memory 532 may include one or more modules that each correspond to a set of instructions. Further, the processing component 522 is configured to execute instructions to perform the above-described method: receiving a video image sent by a terminal; wherein the video image contains an operation record of operating an application program on the terminal screen; acquiring an operation record according to the video image; determining operation information related to the operation according to the operation record; and analyzing the operation information to obtain an operation result.
The apparatus 500 may also include a power component 526 configured to perform power management of the apparatus 500, a wired or wireless network interface 550 configured to connect the apparatus 500 to a network, and an input/output (I/O) interface 558. The apparatus 500 may operate based on an operating system stored in the memory 532, such as Windows Server, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, or the like.
EXAMPLE six
Referring now to FIG. 6, shown is a schematic diagram of an electronic device suitable for use in implementing embodiments of the present disclosure. The electronic devices in the embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., car navigation terminals), and the like, and fixed terminals such as digital TVs, desktop computers, and the like. The electronic device shown in fig. 6 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present disclosure.
As shown in fig. 6, the electronic device may include a processing means (e.g., a central processing unit, a graphics processor, etc.) 601, which may perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM)602 or a program loaded from a storage means 608 into a Random Access Memory (RAM) 603. In the RAM 603, various programs and data necessary for the operation of the electronic apparatus are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.
Generally, the following devices may be connected to the I/O interface 605: input devices 606 including, for example, a touch screen, touch pad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 606 including, for example, a Liquid Crystal Display (LCD), speakers, vibrators, or the like; storage 608 including, for example, tape, hard disk, etc.; and a communication device 609. The communication means 609 may allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. While fig. 6 illustrates an electronic device having various means, it is to be understood that not all illustrated means are required to be implemented or provided. More or fewer devices may alternatively be implemented or provided.
In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as a computer application program. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication means 609, or may be installed from the storage means 608, or may be installed from the ROM 602. The computer program, when executed by the processing device 601, performs the above-described functions defined in the methods of the embodiments of the present disclosure.
It should be noted that the computer readable medium in the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In contrast, in the present disclosure, a computer readable signal medium may comprise a propagated data signal with computer readable program code embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.
The computer readable medium may be embodied in the electronic device; or may exist separately without being assembled into the electronic device.
The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: acquiring a video image; wherein the video image comprises an operation record of operating an application program on a terminal screen; and sending the video image to a server so that the server acquires an operation record according to the video image, determines operation information related to the operation according to the operation record, and analyzes the operation information to obtain an operation result.
Computer program code for carrying out operations for aspects of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone application package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units described in the embodiments of the present disclosure may be implemented by an application program or by hardware. Where the name of an element does not in some cases constitute a limitation on the element itself.
The foregoing description is only exemplary of the preferred embodiments of the disclosure and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the disclosure herein is not limited to the particular combination of features described above, but also encompasses other embodiments in which any combination of the features described above or their equivalents does not depart from the spirit of the disclosure. For example, the above features and (but not limited to) the features disclosed in this disclosure having similar functions are replaced with each other to form the technical solution.
Claims (18)
1. A trigger operation analysis method based on an application program is characterized by comprising the following steps:
receiving a video image sent by a terminal; wherein the video image contains an operation record of operating an application program on the terminal screen;
acquiring an operation record according to the video image;
determining operation information related to the operation according to the operation record;
and analyzing the operation information to obtain an operation result.
2. The method of claim 1, wherein said recording from said video image capture operation comprises:
performing framing processing on the video image according to a video frame rate, and obtaining a picture sequence according to the time sequence relation of each frame of picture;
and acquiring an operation record according to the picture sequence.
3. The method of claim 2, wherein determining operation information related to the operation from the operation record comprises:
traversing the picture sequence, and determining a picture and an operation initial position corresponding to the start of operation according to a first change in the current picture operation record;
and determining the corresponding picture and the displacement offset when the operation is finished according to the second change in the subsequent picture operation record of the current picture.
4. The method of claim 3, wherein analyzing the operation information to obtain an operation result comprises:
determining operation time according to the picture corresponding to the operation starting time and the picture corresponding to the operation ending time; and/or
Determining an operation mode according to the displacement offset; and/or
And determining an operation position according to the operation starting position and the displacement offset.
5. The method according to claim 4, wherein the determining an operation time according to the picture corresponding to the operation start time and the picture corresponding to the operation end time comprises:
determining the interval time of two adjacent pictures in the picture sequence;
determining the number of frames from the corresponding picture at the beginning of the operation to the corresponding picture at the end of the operation according to the time sequence relation;
and taking the product of the frame number and the interval time as the operation time.
6. The method of claim 5, wherein the interval time is the inverse of the video frame rate.
7. The method of claim 4, wherein determining the operating mode based on the displacement offset comprises:
if the abscissa and the ordinate of the displacement offset are both 0, determining that the operation is a click operation; and if the abscissa and the ordinate of the displacement offset are not 0, determining that the operation is a sliding operation.
8. The method of claim 7, wherein determining an operating position from the operating start position and the displacement offset comprises:
if the operation is a click operation, the operation position is the operation starting position; and if the operation is a sliding operation, the operation position is the operation starting position plus the displacement offset.
9. A trigger operation analysis method based on an application program is characterized by comprising the following steps:
acquiring a video image; wherein the video image comprises an operation record of operating an application program on a terminal screen;
and sending the video image to a server so that the server acquires an operation record according to the video image, determines operation information related to the operation according to the operation record, and analyzes the operation information to obtain an operation result.
10. The method according to claim 9, wherein a recording function is provided in a setting option of the terminal;
correspondingly, the acquiring the video image comprises the following steps:
and starting to record the terminal screen according to the starting operation of the recording function, and stopping recording the terminal screen according to the recording stopping operation to obtain the video image.
11. The method of claim 10, further comprising:
in the recording process, detecting the operation of the application program on the terminal screen and generating an operation record;
including the operation record in the video image.
12. The method according to any of claims 9-11, wherein the operation record comprises at least one of the following information: a first change, an operation start time, an operation start position, a second change, an operation end time, and a displacement offset amount.
13. An application-based trigger action analysis apparatus, comprising:
the image receiving module is used for receiving the video image sent by the terminal; wherein the video image contains an operation record of operating an application program on the terminal screen;
the record acquisition module is used for acquiring operation records according to the video images;
the information determining module is used for determining operation information related to the operation according to the operation record;
and the operation analysis module is used for analyzing the operation information to obtain an operation result.
14. An application-based trigger action analysis apparatus, comprising:
the image acquisition module is used for acquiring a video image; wherein the video image comprises an operation record of operating an application program on a terminal screen;
and the image sending module is used for sending the video image to a server so that the server can obtain an operation record according to the video image, determine operation information related to the operation according to the operation record, and analyze the operation information to obtain an operation result.
15. A server, comprising:
a memory for storing non-transitory computer readable instructions; and
a processor for executing the computer readable instructions such that the processor when executing implements the method for application-based trigger operation analysis according to any of claims 1-8.
16. A computer-readable storage medium storing non-transitory computer-readable instructions that, when executed by a computer, cause the computer to perform the application-based trigger operation analysis method of any one of claims 1-8.
17. An electronic device, comprising:
a memory for storing non-transitory computer readable instructions; and
a processor for executing the computer readable instructions such that the processor when executing implements the application-based trigger operation analysis method according to any of claims 9-12.
18. A computer-readable storage medium storing non-transitory computer-readable instructions that, when executed by a computer, cause the computer to perform the application-based trigger operation analysis method of any of claims 9-12.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910468276.5A CN110209581A (en) | 2019-05-31 | 2019-05-31 | Trigger action analysis method and device based on application program |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910468276.5A CN110209581A (en) | 2019-05-31 | 2019-05-31 | Trigger action analysis method and device based on application program |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110209581A true CN110209581A (en) | 2019-09-06 |
Family
ID=67789984
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910468276.5A Pending CN110209581A (en) | 2019-05-31 | 2019-05-31 | Trigger action analysis method and device based on application program |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110209581A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20230126502A1 (en) * | 2021-10-25 | 2023-04-27 | International Business Machines Corporation | Video-based user interface aplication testing |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103237108A (en) * | 2013-05-13 | 2013-08-07 | 百度在线网络技术（北京）有限公司 | Test method and test terminal for mobile terminal |
| CN104956339A (en) * | 2013-01-31 | 2015-09-30 | 惠普发展公司，有限责任合伙企业 | Generating software test script from video |
| US20160357576A1 (en) * | 2015-06-02 | 2016-12-08 | International Business Machines Corporation | Generating customized on-demand videos from automated test scripts |
| CN108874673A (en) * | 2018-06-21 | 2018-11-23 | 腾讯科技（深圳）有限公司 | The test method and device of application program |
| CN108900776A (en) * | 2018-08-22 | 2018-11-27 | 北京百度网讯科技有限公司 | Method and apparatus for determining the response time |
| CN109561271A (en) * | 2018-11-19 | 2019-04-02 | 维沃移动通信有限公司 | A kind of guidance method of terminal operation, first terminal and second terminal |
| CN109783365A (en) * | 2018-12-15 | 2019-05-21 | 深圳壹账通智能科技有限公司 | Automated testing method, device, computer equipment and storage medium |
-
2019
- 2019-05-31 CN CN201910468276.5A patent/CN110209581A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104956339A (en) * | 2013-01-31 | 2015-09-30 | 惠普发展公司，有限责任合伙企业 | Generating software test script from video |
| CN103237108A (en) * | 2013-05-13 | 2013-08-07 | 百度在线网络技术（北京）有限公司 | Test method and test terminal for mobile terminal |
| US20160357576A1 (en) * | 2015-06-02 | 2016-12-08 | International Business Machines Corporation | Generating customized on-demand videos from automated test scripts |
| CN108874673A (en) * | 2018-06-21 | 2018-11-23 | 腾讯科技（深圳）有限公司 | The test method and device of application program |
| CN108900776A (en) * | 2018-08-22 | 2018-11-27 | 北京百度网讯科技有限公司 | Method and apparatus for determining the response time |
| CN109561271A (en) * | 2018-11-19 | 2019-04-02 | 维沃移动通信有限公司 | A kind of guidance method of terminal operation, first terminal and second terminal |
| CN109783365A (en) * | 2018-12-15 | 2019-05-21 | 深圳壹账通智能科技有限公司 | Automated testing method, device, computer equipment and storage medium |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20230126502A1 (en) * | 2021-10-25 | 2023-04-27 | International Business Machines Corporation | Video-based user interface aplication testing |
| US11822464B2 (en) * | 2021-10-25 | 2023-11-21 | International Business Machines Corporation | Video-based user interface application testing |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR102158557B1 (en) | Method and device for determining response time | |
| CN111625473B (en) | Interface test case generation method and device, storage medium and electronic equipment | |
| CN110213614B (en) | Method and device for extracting key frame from video file | |
| CN112684968A (en) | Page display method and device, electronic equipment and computer readable medium | |
| CN110347875B (en) | Video scene classification method and device, mobile terminal and storage medium | |
| CN111813685A (en) | Automatic testing method and device | |
| CN113449310A (en) | Application program vulnerability detection method, device and equipment | |
| CN110059064B (en) | Log file processing method and device and computer readable storage medium | |
| CN113144620B (en) | Method, device, platform, readable medium and equipment for detecting frame synchronous game | |
| CN112954056B (en) | Method and device for processing monitoring data, electronic equipment and storage medium | |
| CN112291121B (en) | Data processing method and related equipment | |
| CN110177270A (en) | Video head frame test method and device | |
| CN110209581A (en) | Trigger action analysis method and device based on application program | |
| CN109492163B (en) | List display recording method and device, terminal equipment and storage medium | |
| CN109408387B (en) | Page testing method and device | |
| CN116662193A (en) | Page testing method and device | |
| CN110134601A (en) | A kind of software pressure surveys coverage measure method, system, medium and electronic equipment | |
| CN114499709B (en) | Antenna radio test method and device, electronic equipment and storage medium | |
| CN111382017A (en) | Fault query method, device, server and storage medium | |
| CN112379967B (en) | Simulator detection method, device, equipment and medium | |
| CN117667663A (en) | Control positioning path determining method, device, equipment, storage medium and product | |
| CN117251355A (en) | Performance test method, device, equipment, computer readable storage medium and product | |
| CN111125015B (en) | Method, apparatus, terminal and medium for dump file classification | |
| CN114116480A (en) | Method, device, medium and equipment for determining application program test coverage rate | |
| CN113849416A (en) | Test method, test device, storage medium and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190906 |