WO2025082001A1 - 一种响应时延测试方法、装置、计算机设备及存储介质 - Google Patents

一种响应时延测试方法、装置、计算机设备及存储介质 Download PDF

Info

Publication number
WO2025082001A1
WO2025082001A1 PCT/CN2024/112144 CN2024112144W WO2025082001A1 WO 2025082001 A1 WO2025082001 A1 WO 2025082001A1 CN 2024112144 W CN2024112144 W CN 2024112144W WO 2025082001 A1 WO2025082001 A1 WO 2025082001A1
Authority
WO
WIPO (PCT)
Prior art keywords
frame
video
video frames
response
response frame
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
PCT/CN2024/112144
Other languages
English (en)
French (fr)
Inventor
蒋恒
徐斌
郑高挺
黄韦嘉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Zitiao Network Technology Co Ltd
Original Assignee
Beijing Zitiao Network Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Zitiao Network Technology Co Ltd filed Critical Beijing Zitiao Network Technology Co Ltd
Publication of WO2025082001A1 publication Critical patent/WO2025082001A1/zh
Anticipated expiration legal-status Critical
Pending legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/442—Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
    • H04L43/0852—Delays
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21—Server components or server architectures
    • H04N21/218—Source of audio or video content, e.g. local disk arrays
    • H04N21/2181—Source of audio or video content, e.g. local disk arrays comprising remotely distributed storage units, e.g. when movies are replicated over a plurality of video servers
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/478—Supplemental services, e.g. displaying phone caller identification, shopping application
    • H04N21/4781—Games

Definitions

  • the present disclosure relates to the field of computer technology, and in particular to a response delay test method, device, computer equipment and storage medium.
  • Cloud gaming has become a popular new gaming genre because it does not require installation and updates, and does not require high-performance equipment.
  • cloud gaming has high requirements for network performance, and the quality of service is greatly affected by response latency.
  • the mainstream method of current black box testing is to use a high-speed camera to shoot the game screen, and manually count the frames to get the response delay.
  • the number of frames captured by the high-speed camera is very large (more than 250 frames per second), and each frame needs to be manually checked to see if it is an operation frame or a response frame, resulting in high labor costs for this method.
  • the embodiments of the present disclosure at least provide a response delay test method, apparatus, computer equipment, and storage medium.
  • an embodiment of the present disclosure provides a response delay testing method, including:
  • the first display area of two adjacent video frames of the test video is detected to determine The operation frame of the test video; wherein the first display area is used to represent the trigger information of the current screen interface;
  • an embodiment of the present disclosure further provides a response delay testing device, including:
  • An acquisition module is used to acquire the test video of the program to be tested
  • a first detection module is used to detect the first display area of two adjacent video frames of the test video to determine the operation frame of the test video; wherein the first display area is used to represent the trigger information of the current screen interface;
  • a second detection module is used to detect the second display area of two adjacent video frames of the test video to determine the response frame of the test video; wherein the second display area is used to display the main screen of the program to be tested;
  • a calculation module is used to calculate the response delay of the program to be tested based on the operation frame and the response frame.
  • the first detection module when detecting the first display area of two adjacent video frames of the test video to determine the operation frame of the test video, is used to:
  • the latter video frame of the two adjacent video frames is used as the operation frame.
  • the first detection module when determining the first difference information between the first display areas of the two adjacent video frames, uses At:
  • the first difference information satisfies a first preset condition, including:
  • the image energy of the first binary image exceeds a first preset threshold; wherein the image energy of the first binary image is determined by the number of pixel points in the first binary image whose pixel values are the first target value.
  • the second detection module detects the second display area of two adjacent video frames of the test video to determine the response frame of the test video, and is used to:
  • a response frame is screened out from the suspected response frames.
  • the second detection module when determining the second difference information between the second display areas of the two adjacent video frames, is used to:
  • a second binarized image corresponding to the second difference image is determined; and the second difference information is represented by the second binarized image.
  • the second detection module when determining the second difference map When the second binarized image corresponding to the image is used for:
  • the image contour whose value is the second target value in the intermediate binary image is filtered to obtain the second binary image.
  • the second detection module when determining the suspected response frame of the test video based on the second difference information, is configured to:
  • the latter video frame of the two adjacent video frames is used as the suspected response frame.
  • the second detection module when determining the second difference information between the second display areas of the two adjacent video frames, is used to:
  • Coordinate difference information between position coordinates of the corner point in each of the two video frames is determined, and the second difference information is represented by the coordinate difference information.
  • the second detection module when determining the suspected response frame of the test video based on the second difference information, is configured to:
  • a suspected response frame of the test video is determined.
  • the second detection module when filtering out a response frame from the suspected response frame based on a time difference between the operation frame and the suspected response frame, is configured to:
  • a suspected response frame that follows the frame operation frame and has the shortest time interval with the operation frame and whose time interval exceeds a preset time interval is taken as a response frame corresponding to the operation frame.
  • an embodiment of the present disclosure further provides a computer device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the above-mentioned first aspect, or any possible implementation of the first aspect are performed.
  • an embodiment of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored.
  • a computer program is stored.
  • the steps of the above-mentioned first aspect or any possible implementation of the first aspect are executed.
  • FIG1 shows a flow chart of a response delay testing method provided by an embodiment of the present disclosure
  • FIG. 2 shows a test video image in the response delay test method provided by an embodiment of the present disclosure. Schematic diagram of the surface
  • FIG3 shows an overall flow chart of a response delay testing method provided by an embodiment of the present disclosure
  • FIG4 shows a flowchart of a specific method for determining an operation frame in a response delay test method provided by an embodiment of the present disclosure
  • FIG5 is a flowchart showing a specific method for determining a suspected response frame in the response delay test method provided by an embodiment of the present disclosure
  • FIG6 shows a flowchart of another specific method for determining a suspected response frame in the response delay test method provided by an embodiment of the present disclosure
  • FIG7 shows a schematic diagram of a response delay testing device provided by an embodiment of the present disclosure
  • FIG8 shows a schematic diagram of the structure of a computer device provided by an embodiment of the present disclosure.
  • existing black box testing technologies mainly include the following three methods to test response delay:
  • Method 1 High-speed camera shooting plus manual frame counting.
  • an external high-speed camera can be used to capture the screen of the user's device, and then the frames can be counted manually to determine the operation frame and the response frame frame by frame.
  • the number of picture frames is extremely large (more than 250 frames per second), and it is necessary to manually confirm whether each frame is an operation frame or a response frame, resulting in high labor costs for this method.
  • Method 2 Video signal detection.
  • the game platform can be installed on the user's device, and then the game program can be run on the game platform, and the game screen can be detected in real time through the game platform to determine the operation frame and the response frame.
  • the delay data of different platforms cannot be compared, so the video signal detection method can only be used on specific platforms.
  • a user can record the game screen while playing the game on the device to obtain a game video, and use CNN to detect the video screen to determine the operation frame and response frame.
  • CNN is more suitable for detecting a single action, but it is not effective for detecting continuous actions that often appear in games. It is difficult to detect a variety of cloud game projects and has poor generalization ability.
  • the existing black box testing technology has the problems of high labor cost and poor generalization ability.
  • the present disclosure provides a response delay test method, device, computer equipment and storage medium. After obtaining the test video of the program to be tested, the first display area and the second display area of the two adjacent frames of the test video are detected respectively, and the operation frame and the response frame of the test video are determined. Based on the operation frame and the response frame, the response delay of the program to be tested is calculated.
  • the response delay test of all cloud game projects can be completed on one device.
  • the test process does not require human participation. On the basis of ensuring accurate recognition, it reduces labor costs and has strong generalization capabilities.
  • a and/or B can mean: A exists alone, A and B exist at the same time, and B exists alone.
  • at least one in this article means any one of a plurality of Or any combination of at least two of the multiple types, for example, including at least one of A, B, and C, can represent any one or more elements selected from the set consisting of A, B, and C.
  • a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information.
  • the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.
  • the prompt information in response to receiving an active request from the user, may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form.
  • the pop-up window may also carry a selection control for the user to choose "agree” or “disagree” to provide personal information to the electronic device.
  • the execution subject of the response delay test method provided in the embodiment of the present disclosure is generally a computer device with certain computing capabilities, and the computer device includes, for example: a terminal device or a server or other processing device.
  • the terminal device can be a user equipment (User Equipment, UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (Personal Digital Assistant, PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc.
  • the response delay test method can be implemented by a processor calling a computer-readable instruction stored in a memory.
  • FIG. 1 is a flow chart of a response delay test method provided in an embodiment of the present disclosure, the method includes steps 101 to 104, wherein:
  • Step 101 Obtain a test video of the program to be tested.
  • Step 102 Detect the first display area of two adjacent video frames of the test video to determine the operation frame of the test video; wherein the first display area is used to represent the trigger information of the current screen interface.
  • Step 103 Detect the second display area of two adjacent video frames of the test video to determine the response frame of the test video; wherein the second display area is used to display the main screen of the program to be tested.
  • Step 104 Calculate the response delay of the program to be tested based on the operation frame and the response frame.
  • the program to be tested is a program that interacts with the user, that is, after the user operates the program to be tested, the program to be tested will perform a response action based on the user operation.
  • the program to be tested may be a cloud game program.
  • the program to be tested may be a program that requires black box testing, such as a mini-program.
  • the program to be tested may be an application developed by a third party, or a cloud game.
  • the program to be tested when the program to be tested is an application program, the program to be tested may be For shopping software, when the response delay is too high, users may repeatedly click buttons such as ordering, resulting in problems such as duplicate orders. In order to prevent similar problems from occurring, black box testing of the shopping software is required to ensure that the response delay of the shopping software is low.
  • the program to be tested is a cloud game
  • cloud games since cloud games have high requirements for network performance, a high response delay will cause the user's operations to not be responded to by the cloud game program or to be responded to only after a long period of time. It is difficult for users to complete the operations they want to complete when the response delay is high, which has a great impact on the user's gaming experience. Therefore, in order to ensure the user's gaming experience, cloud games need to be black-box tested to ensure that the response delay is low.
  • the test video can be obtained in a variety of ways, such as receiving a test video uploaded by a user, or recording a test video of the program to be tested by screen recording.
  • the present disclosure does not limit other ways of obtaining the test video.
  • the process of obtaining the test video is authorized by the user, and the specific authorization process is described above.
  • the user's trigger operation can be simulated by installing a script on the client.
  • the program to be tested can make a corresponding response to the trigger operation. For example, if the trigger operation is a trigger operation to control the character to move forward, the program to be tested can control the character on the screen to move accordingly after detecting the trigger operation.
  • the script can run at a certain frequency. Each time the script runs, it can simulate a trigger operation by the user. During different runs of the script, the trigger operations performed can be different (for example, different function buttons can be triggered).
  • step 102 and step 103 Regarding step 102 and step 103:
  • the test video may include a first display area and a second display area.
  • the first display area is an area for displaying the trigger information of the current screen interface, such as the current trigger position coordinates, horizontal sliding speed, vertical sliding speed, etc.
  • the second display area is an area for displaying the main screen of the program to be tested, and the main screen may be, for example, the main program interface of the program to be tested.
  • the main screen is a screen that displays operation buttons and control objects.
  • the main screen may be a screen that displays product information.
  • the first display area can be a fixed area of the screen interface, for example, can be located at the upper boundary of the screen interface, and the second display area can be other areas of the display interface of the program to be tested except the first display area when displaying the program to be tested.
  • the display interface of the program to be tested can be displayed in full screen when displayed.
  • the display positions of the first display area and the second display area can be as shown in Figure 2.
  • the user before recording a test video, the user can turn on the "Show pointer position" option in the developer options of the terminal device, so that the terminal device can display it in the first display area when performing any triggering operation.
  • step 102 and step 103 The specific method for determining the operation frame and the response frame in step 102 and step 103 will be introduced below.
  • Step a1 for any two adjacent video frames, determine first difference information between first display areas of the two adjacent video frames.
  • the first difference information is represented by a first binary image. Therefore, in order to determine the first difference information between the first display areas of two adjacent video frames, the first binary image needs to be obtained.
  • the first binary image for any two adjacent video frames, firstly subtract the first area images of the first display area of the two video frames to obtain a first difference image, and then determine the corresponding first binary image according to the first difference image.
  • Subtracting the first area images of the first display area of the two video frames can be understood as subtracting the pixel values of each pixel point in the first area images of the first display area of the two video frames. The difference should be calculated to obtain the first difference image.
  • the first difference image When determining the first binary image corresponding to the first difference image, that is, when binarizing the first difference image, the first difference image can be firstly grayed, and then the pixel values of the pixels whose gray values are less than a preset threshold are set to the third target value, and the pixel values of the pixels whose pixel values exceed the preset threshold are set to the first target value.
  • the first target value can be 1/255
  • the third target value can be 0.
  • Step a2 When the first difference information satisfies a first preset condition, the latter video frame of the two adjacent video frames is used as the operation frame.
  • the first difference information satisfies a first preset condition, which may be, for example, that the image energy of the first binary image exceeds a first preset threshold.
  • the image energy of the first binary image is determined by the number of pixels in the first binary image whose pixel values are the first target value.
  • the first preset threshold is 150
  • the first pixel value is 1, and the number of pixels with a pixel value of 1 in the first binary image corresponding to the first difference image between the third video frame and the fourth video frame is detected to be 182 since the product of the first pixel value and the number of pixels with a pixel value of 1 is 182, which is greater than the first preset threshold of 150, the first preset condition is met, and the fourth video frame, which is the later of the two video frames, is used as the operation frame.
  • the detection video cannot be used to test the response delay, and steps 101 and 102 need to be re-executed until the operation frame can be detected in the detection video; the same test video may also contain multiple operation frames. After completing the detection of one operation frame, other operation frames can be continued to be detected after the operation frame.
  • the first display area displays a value related to the trigger information
  • OCR optical character recognition
  • Step b1 for any two adjacent video frames, determine second difference information between the second display areas of the two adjacent video frames.
  • the second difference information may be determined in any one of the following two ways:
  • the second difference information can be represented by a second binary image.
  • the second area images of the second display areas of the two adjacent video frames can be subtracted from each other to obtain a second difference image, and the second difference image can be binarized to obtain an intermediate binary image; then, the image contour whose value is the second target value in the intermediate binary image is filtered to obtain the second binary image.
  • the second target value when binarizing the second difference image, may be the same as the first target value or different from the first target value. For example, when the first target value is 1, the second target value may be set to 1, or alternatively, the second target value may be set to 255.
  • the image area with the second target value is the area where there is a difference between the two adjacent video frames.
  • the contours whose corresponding image areas are smaller than the preset area in the image contours with the second target value in the intermediate binary image can be filtered.
  • the pixel values of the image regions whose corresponding image areas are smaller than the preset area can be adjusted from the second target value to a fourth target value, and the fourth target value is the value of the middle target value.
  • the second target value may be 1 and the fourth target value may be 0.
  • Method 2 When the video frame includes a control object, since the action of the control object may cause the position of the character to change, the second difference information may also be represented by the coordinate difference information of the control object.
  • the corner points of the controlled objects in the two video frames can be marked, the position coordinates of the corner points in each of the two video frames can be determined, and the coordinate difference information between the position coordinates of the corner points in each of the two video frames can be determined.
  • the marked corner points may be positions where the control object is prone to change.
  • the marked corner points may include points at the positions of fingers, feet, knees, cuffs, skirts, etc.
  • the number of corner points and the positions of the corner points marked for different control objects may be different.
  • the second difference information can be calculated using the above method one, and for video frames with smaller picture changes, the second difference information can be calculated using the above method two.
  • the size of the picture change can be distinguished by the action type of the trigger operation, and trigger operations for larger picture changes and smaller picture changes can be pre-defined.
  • Step b2 Determine the suspected response frame of the test video based on the second difference information.
  • the image energy of the second binarized image can be determined, and when the image energy of the second binarized image exceeds a second preset threshold, the latter of the two adjacent video frames is used as the suspected response frame.
  • the image energy of the second binary image can also be determined by the number of pixels whose pixel values are the second target value in the second binary image. Specifically, the image energy can be the product of the second target value and the number of pixels whose pixel values are the second target value. If the second preset threshold is 15000 and the second pixel value is 255, when it is detected that the 15th video frame is different from the 15th video frame, the image energy can be determined by the number of pixels whose pixel values are the second target value. The number of pixel points whose pixel values are the second pixel value in the second binary image corresponding to the second difference image of the 16 frames of video frames is 100.
  • the 16th frame of the video frame that is later than the two frames of video frames is taken as the suspected response frame.
  • method 2 determines the suspected response frame of the test video, then when the second difference information satisfies the second preset condition, the movement information of the target video frame that is closer to the latter of the two adjacent video frames can be marked as movement; and the first movement information of the first N frames of video frames before the target video frame and the second movement information of the last M frames of video frames are obtained; wherein N and M are preset positive integers; and then based on the first movement information and the second movement information, the suspected response frame of the test video is determined.
  • the second preset condition may be that the number of corner points that produce coordinate differences is greater than a moving intersection threshold.
  • the moving corner point threshold is 8 when it is detected that the number of corner points that produce coordinate differences between the 23rd frame of video and the 24th frame of video is 11, since the number of corner points that produce coordinate differences is greater than the moving intersection threshold of 8, the second preset condition is met, and the movement information of the 24th frame of video, which is the later of the two frames of video, is marked as moving.
  • the movement information of each frame of the test video can be marked according to the above method.
  • each video frame in the detection video can be traversed, and for any traversed video frame, the video frame is used as a target video frame, and then the first motion information of the first N video frames and the second motion information of the last M video frames of the target video frame are obtained; wherein the values of N and M can be the same;
  • the first movement threshold and the second movement threshold can be set respectively, and based on the first movement threshold, the second movement threshold, the first movement information and the second movement information, the suspected response frame of the test video can be determined.
  • the first movement information and the second movement information may be represented by numerical values.
  • the number “1” may represent “movement” in the movement information
  • the number “0” may represent “stationary” in the movement information.
  • the first motion threshold is used to limit the first motion information
  • the second motion threshold is used to limit the second motion information.
  • the first motion threshold can be the product of the number N of video frames before the target video frame and a first preset value
  • the second motion threshold can be the product of the number M of video frames after the target video frame and a second preset value.
  • Case 1 if the sum of the first motion information is less than the first motion threshold, and the sum of the second motion information is greater than the second motion threshold, the video frame is taken as a suspected response frame.
  • Case 2 If the first motion information and the second motion information do not satisfy Case 1, since the recognition accuracy cannot be guaranteed, no marking is made for the video frame.
  • the reason why the sum of the first motion information needs to be less than the first motion threshold and the sum of the second motion information needs to be greater than the second motion threshold is that if the motion information of the video frames before a certain video frame is mostly static and the motion information of the video frames after the video frame is mostly moving, then the frame is likely to be a response frame.
  • the suspected response frame determined here refers to the video frame at the beginning of the response.
  • the end response frame can also be determined to assist in detecting the next operation frame/suspected response frame.
  • a third movement threshold and a fourth movement threshold can also be set, wherein the first movement threshold and the third movement threshold are used to limit the first movement information, and the second movement threshold and the fourth movement threshold are used to limit the second movement information.
  • the third movement threshold may be the product of the number N of video frames before the target video frame and a third preset value
  • the fourth movement threshold may be the product of the number M of video frames after the target video frame and a fourth preset value.
  • the third movement threshold is greater than the first movement threshold, and the fourth movement threshold is less than the second movement threshold.
  • the first preset value is 1/10
  • the second preset value is 8/10
  • the third preset value is 8/10
  • the fourth preset value is 1/10
  • the first moving threshold is 1
  • the second moving threshold is 8
  • the third moving threshold is 8, and the fourth moving threshold is 1.
  • the method for determining the suspected response frame may be the same as the above method, and the method for determining the end response frame may be divided into the following cases:
  • Case 1 if the sum of the first motion information is greater than the third motion threshold, and if the sum of the second motion information is less than the fourth motion threshold, then the video frame is used as an end response frame.
  • Case 2 If the first movement information and the second movement information do not satisfy case 1, no marking is performed.
  • the sum of the first motion information is 9, which is greater than the third motion threshold 8, and the sum of the fourth motion information is 0, which is less than the second motion threshold 1, which satisfies the first situation, so the frame of video is taken as the end response frame; if the first motion information is 1111111111, the second motion information is 1100000000, the third preset value is 8/10, and the fourth preset value is 1/10, it can be known that the third motion threshold is 8 and the fourth motion threshold is 1.
  • the sum of the first motion information is 10, which is greater than the third motion threshold 8, but the sum of the fourth motion information is 2, which is greater than the second motion threshold 1, which does not meet the first situation, so no mark is made for the frame of video.
  • the reason why the sum of the first motion information needs to be greater than the third motion threshold and the sum of the second motion information needs to be less than the fourth motion threshold is that if the motion information of the video frames before a certain video frame is mostly moving, and the motion information of the video frames after the video frame is mostly still, then the frame is likely to be an end response frame.
  • One of the purposes of determining the end response frame is to assist in detecting the next response frame. After a suspected response frame is detected, if the end response frame is not detected, even if a video frame that meets the suspected response frame judgment conditions is detected, the video frame cannot be used as a suspected response frame.
  • Step b3 based on the time difference between the operation frame and the suspected response frame, select a response frame from the suspected response frame.
  • the time interval between the operation frame and the corresponding response frame cannot be too long.
  • the time interval between the operation frame and the corresponding response frame cannot be too long. short, therefore, there is a certain range of the time interval between the operation frame and the corresponding response frame, that is, when filtering out the response frame from the suspected response frame, for any frame operation frame, the suspected response frame after the frame operation frame and whose time interval meets the preset time interval condition is taken as the response frame corresponding to the operation frame.
  • the preset time interval condition may be a time interval greater than 50 milliseconds and less than 200 milliseconds.
  • the suspected response frame with the shortest time interval with the operation frame is used as the response frame corresponding to the operation frame.
  • the operation frame or response frame detection may fail due to calculation accuracy, and steps 101 to 103 need to be re-executed until the response frame can be detected in the detection video.
  • step 104
  • Mtp is a single delay
  • P(i) is the time of the response frame of the i-th frame
  • P(j) is the time of the operation frame corresponding to the response frame of the i-th frame.
  • the average value of all single delays can be used as response delay.
  • FIG. 3 is an overall flow chart of a response delay test method provided by an embodiment of the present disclosure, which mainly includes the following steps:
  • a test video of the program to be tested is obtained, and then the screen of the test video is divided into a first display area and a second display area, and the two areas are processed separately.
  • the operation frame of the test video can be determined.
  • the size of the picture change is first distinguished. If the picture change is relatively large, a suspected response frame can be determined based on the second difference image between two adjacent video frames.
  • the suspected response frame can be determined based on the position difference information of the pre-marked corner points.
  • a response frame is screened out from the suspected response frame based on a time difference between the operation frame and the suspected response frame.
  • the time difference between the response frame and the corresponding operation frame is calculated to obtain the response delay.
  • the first area images of the first display areas of the two video frames are subtracted to obtain a first difference image, and then the first difference image is grayed and binarized to determine the corresponding first binarized image.
  • the image energy of the first binarized image can be obtained.
  • the image energy of the first binarized image is compared with the first preset threshold. If the image energy of the first binarized image is greater than the first preset threshold, the latter frame of the two video frames can be determined as the operation frame. Since the same test video may contain multiple operation frames, after completing the detection of one operation frame, other operation frames can be continued to be detected after the operation frame.
  • the suspected response frame can be determined based on the second difference image of two adjacent video frames, and the flowchart shown in FIG5 can be referred to, including the following steps:
  • the second display area of the two video frames is first
  • the regional images are subtracted to obtain a second difference image
  • the first difference image is grayed and binarized to determine the corresponding intermediate binarized image.
  • the contours in the intermediate binarized image with an area smaller than a preset area are filtered to obtain a second binarized image.
  • the image energy of the second binarized image can be obtained.
  • the image energy of the second binarized image is compared with the second preset threshold. If the image energy of the second binarized image is greater than the second preset threshold, the latter frame of the two video frames can be determined as a suspected response frame.
  • the suspected response frame can be determined based on the position difference information of the pre-marked corner points.
  • the flowchart shown in FIG. 6 can be referred to, which includes the following steps:
  • the control object in the video screen is obtained, and the corner points of the control object are marked. Then, two adjacent frames in the detection video are compared to obtain the number of corner points that produce coordinate differences. If the number is greater than the moving corner point threshold, the movement information of the latter frame of the two video frames is marked as moving, otherwise the movement information of the latter frame of the two video frames is marked as still, and the first movement information of the N frames before the frame and the second movement information of the M frames after the frame are obtained.
  • the relationship between the first movement information and the first movement threshold and the third movement threshold is compared, and the relationship between the second movement information and the second movement threshold and the fourth movement threshold is compared to determine whether the frame is a suspected response frame or an end response frame.
  • the frame is determined as a suspected response frame; if the sum of the first movement information is greater than the third movement threshold and the sum of the second movement information is less than the fourth movement threshold, the frame is determined as an end response frame.
  • the end response frame can assist in determining the suspected response frame. Specifically, after the suspected response frame is detected, if the end response frame is not detected, even if a video frame that meets the suspected response frame determination condition is detected, the video frame cannot be used as a suspected response frame.
  • a response delay test device corresponding to the response delay test method is also provided in the embodiment of the present disclosure. Since the principle of solving the problem by the device in the embodiment of the present disclosure is similar to the above-mentioned response delay test method in the embodiment of the present disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.
  • the device includes: an acquisition module 701, a first detection module 702, a second detection module 703, and a calculation module 704; wherein,
  • the acquisition module 701 is used to acquire the test video of the program to be tested
  • a first detection module 702 is used to detect the first display area of two adjacent video frames of the test video to determine the operation frame of the test video; wherein the first display area is used to represent the trigger information of the current screen interface;
  • the second detection module 703 is used to detect the second display area of two adjacent video frames of the test video to determine the response frame of the test video; wherein the second display area is used to display the main screen of the program to be tested;
  • the calculation module 704 is used to calculate the response delay of the program to be tested based on the operation frame and the response frame.
  • the first detection module 702 when detecting the first display area of two adjacent video frames of the test video to determine the operation frame of the test video, is used to:
  • the latter video frame of the two adjacent video frames is used as the operation frame.
  • the first detection model Block 702 when determining first difference information between the first display areas of the two adjacent video frames, is used to:
  • the first difference information satisfies a first preset condition, including:
  • the image energy of the first binary image exceeds a first preset threshold; wherein the image energy of the first binary image is determined by the number of pixel points in the first binary image whose pixel values are the first target value.
  • the second detection module 703 detects the second display area of two adjacent video frames of the test video to determine the response frame of the test video, for:
  • a response frame is screened out from the suspected response frames.
  • the second detection module 703 when determining the second difference information between the second display areas of the two adjacent video frames, is used to:
  • a second binarized image corresponding to the second difference image is determined; and the second difference information is represented by the second binarized image.
  • the second detection module 703 when determining the second binarized image corresponding to the second difference image, is configured to:
  • the image contour whose value is the second target value in the intermediate binary image is filtered to obtain the second binary image.
  • the second detection module 703 when determining the suspected response frame of the test video based on the second difference information, is configured to:
  • the latter video frame of the two adjacent video frames is used as the suspected response frame.
  • the second detection module 703 when determining the second difference information between the second display areas of the two adjacent video frames, is used to:
  • Coordinate difference information between position coordinates of the corner point in each of the two video frames is determined, and the second difference information is represented by the coordinate difference information.
  • the second detection module 703 when determining the suspected response frame of the test video based on the second difference information, is configured to:
  • a suspected response frame of the test video is determined.
  • the second detection module 703 when filtering out a response frame from the suspected response frame based on the time difference between the operation frame and the suspected response frame, is configured to:
  • a suspected response frame that follows the frame operation frame and has the shortest time interval with the operation frame and whose time interval exceeds a preset time interval is taken as a response frame corresponding to the operation frame.
  • the embodiment of the present disclosure also provides a computer device.
  • a schematic diagram of the structure of the computer device 800 provided in the embodiment of the present disclosure is shown.
  • the computer device in the embodiment of the present disclosure may include but is not limited to mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc., or various forms of servers, such as independent servers or server clusters.
  • the computer device shown in FIG8 is only an example and should not bring any limitations to the functions and scope of use of the embodiment of the present disclosure.
  • the computer device 800 may include a processing device (e.g., a central processing unit, a response delay tester, etc.) 801, which can perform various appropriate actions and processes according to a program stored in a read-only storage device (ROM) 802 or a program loaded from a storage device 805 to a random access storage device (RAM) 803.
  • ROM read-only storage device
  • RAM random access storage device
  • Various programs and data required for the operation of the computer device 800 are also stored in the RAM 803.
  • the processing device 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804.
  • An input/output (I/O) interface 805 is also connected to the bus 804.
  • the following devices may be connected to the I/O interface 805: an input device 806 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 807 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 808 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 809.
  • the communication device 809 may allow the computer to The device 800 communicates with other devices wirelessly or by wire to exchange data.
  • FIG. 8 shows a computer device 800 having various devices, it should be understood that it is not required to implement or have all the devices shown. More or fewer devices may be implemented or have alternatively.
  • an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the response delay test method.
  • the computer program can be downloaded and installed from the network through the communication device 809, or installed from the storage device 808, or installed from the ROM 802.
  • the processing device 801 the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.
  • the present disclosure also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the response delay test method described in the above method embodiment are executed.
  • the storage medium can be a volatile or non-volatile computer-readable storage medium.
  • the embodiments of the present disclosure also provide a computer program product, which carries a program code.
  • the instructions included in the program code can be used to execute the steps of the response delay testing method described in the above method embodiment. For details, please refer to the above method embodiment, which will not be repeated here.
  • the computer program product may be implemented in hardware, software or a combination thereof.
  • the computer program product is embodied as a computer storage medium.
  • the computer program product is embodied as a software product, such as a software development kit (SDK).
  • SDK software development kit
  • the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
  • each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
  • the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that can be executed by a processor.
  • the computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present disclosure.
  • the aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.

Landscapes

  • Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Multimedia (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Databases & Information Systems (AREA)
  • Environmental & Geological Engineering (AREA)
  • Image Analysis (AREA)

Abstract

本公开提供了一种响应时延测试方法、装置、计算机设备及存储介质，其中，该方法包括：获取待测试程序的测试视频；对所述测试视频的相邻两帧视频帧的第一显示区域进行检测，确定所述测试视频的操作帧；其中，所述第一显示区域用于表征当前屏幕界面的触发信息；对所述测试视频的相邻两帧视频帧的第二显示区域进行检测，确定所述测试视频的响应帧；基于所述操作帧和所述响应帧，计算所述待测试程序的响应时延。

Description

一种响应时延测试方法、装置、计算机设备及存储介质
相关申请的交叉引用
本申请要求于2023年10月16日提交的申请号202311340337.2的中国专利的权益。以上申请的全部教导通过引用并入本文。
技术领域
本公开涉及计算机技术领域，具体而言，涉及一种响应时延测试方法、装置、计算机设备及存储介质。
背景技术
云游戏由于其无需安装和更新，无需高性能设备的特点，已经成为一种广受欢迎的新兴游戏类型，但是云游戏对网络性能的要求较高，服务质量受响应时延的影响较大。由于大部分情况下无法获取包含响应时延信息的日志数据，只能进行黑盒测试来获取响应时延信息，因此，如何在黑盒测试过程中测得精准的时延信息成为云游戏测试中非常重要的步骤。
现有的进行黑盒测试的主流方式为采用高速相机拍摄游戏画面，并通过人工对画面进行数帧，从而得到响应时延。由于这种方式中，高速相机拍摄得到的画面帧数极多(每秒超过250帧画面)，而且需要人工查看每一帧是否为操作帧或响应帧，导致这种方式的人工成本较高。
发明内容
本公开实施例至少提供一种响应时延测试方法、装置、计算机设备及存储介质。
第一方面，本公开实施例提供了一种响应时延测试方法，包括：
获取待测试程序的测试视频；
对所述测试视频的相邻两帧视频帧的第一显示区域进行检测，确定所 述测试视频的操作帧；其中，所述第一显示区域用于表征当前屏幕界面的触发信息；
对所述测试视频的相邻两帧视频帧的第二显示区域进行检测，确定所述测试视频的响应帧；其中，所述第二显示区域用于显示所述待测试程序的主体画面；
基于所述操作帧和所述响应帧，计算所述待测试程序的响应时延。
第二方面，本公开实施例还提供一种响应时延测试装置，包括：
获取模块，用于获取待测试程序的测试视频；
第一检测模块，用于对所述测试视频的相邻两帧视频帧的第一显示区域进行检测，确定所述测试视频的操作帧；其中，所述第一显示区域用于表征当前屏幕界面的触发信息；
第二检测模块，用于对所述测试视频的相邻两帧视频帧的第二显示区域进行检测，确定所述测试视频的响应帧；其中，所述第二显示区域用于显示所述待测试程序的主体画面；
计算模块，用于基于所述操作帧和所述响应帧，计算所述待测试程序的响应时延。
一种可能的实施方式中，所述第一检测模块，在对所述测试视频的相邻两帧视频帧的第一显示区域进行检测，确定所述测试视频的操作帧时，用于：
针对任意相邻两帧视频帧，确定该相邻两帧视频帧的第一显示区域之间的第一差异信息；
在所述第一差异信息满足第一预设条件的情况下，将所述相邻两帧视频帧中靠后的视频帧作为所述操作帧。
一种可能的实施方式中，针对任意相邻两帧视频帧，所述第一检测模块，在确定该相邻两帧视频帧的第一显示区域之间的第一差异信息时，用 于：
针对任意相邻两帧视频帧，将该两帧视频帧的第一显示区域的第一区域图像相减，得到第一差异图像；
确定所述第一差异图像对应的第一二值化图像；所述第一差异信息通过所述第一二值化图像表征；
所述第一差异信息满足第一预设条件，包括：
所述第一二值化图像的图像能量超过第一预设阈值；其中，所述第一二值化图像的图像能量通过所述第一二值化图像中像素值为第一目标值的像素点个数来确定。
一种可能的实施方式中，所述第二检测模块，在对所述测试视频的相邻两帧视频帧的第二显示区域进行检测，确定所述测试视频的响应帧，用于：
针对任意相邻两帧视频帧，确定该相邻两帧视频帧的第二显示区域之间的第二差异信息；
基于所述第二差异信息，确定所述测试视频的疑似响应帧；
基于所述操作帧与所述疑似响应帧之间的时间差，从所述疑似响应帧中筛选出响应帧。
一种可能的实施方式中，针对任意相邻两帧视频帧，所述第二检测模块，在确定该相邻两帧视频帧的第二显示区域之间的第二差异信息时，用于：
针对任意相邻两帧视频帧，将该两帧视频帧的第二显示区域的第二区域图像相减，得到第二差异图像；
确定所述第二差异图像对应的第二二值化图像；所述第二差异信息通过所述第二二值化图像表征。
一种可能的实施方式中，所述第二检测模块，在确定所述第二差异图 像对应的第二二值化图像时，用于：
将所述第二差异图像进行二值化处理，得到中间二值化图像；
对所述中间二值化图像中的取值为第二目标值的图像轮廓进行过滤处理，得到所述第二二值化图像。
一种可能的实施方式中，所述第二检测模块，在基于所述第二差异信息，确定所述测试视频的疑似响应帧时，用于：
确定所述第二二值化图像的图像能量；
在所述第二二值化图像的图像能量超过第二预设阈值的情况下，将所述相邻两帧视频帧中靠后的视频帧作为所述疑似响应帧。
一种可能的实施方式中，在所述视频帧中包括控制对象的情况下，针对任意相邻两帧视频帧，所述第二检测模块，在确定该相邻两帧视频帧的第二显示区域之间的第二差异信息时，用于：
针对任意相邻两帧视频帧，对该两帧视频帧中的控制对象进行角点标记，确定所述角点在所述两帧视频帧中各视频帧中的位置坐标；
确定所述角点在所述两帧视频帧中各视频帧中的位置坐标之间的坐标差异信息，所述第二差异信息通过所述坐标差异信息表征。
一种可能的实施方式中，所述第二检测模块，在基于所述第二差异信息，确定所述测试视频的疑似响应帧时，用于：
在所述第二差异信息满足第二预设条件的情况下，将所述相邻两帧视频帧中靠后的目标视频帧的移动信息标记为移动；
获取所述目标视频帧的前N帧视频帧的第一移动信息，以及后M帧视频帧的第二移动信息；其中，N、M为预设正整数；
基于所述第一移动信息和所述第二移动信息，确定所述测试视频的疑似响应帧。
一种可能的实施方式中，所述第二检测模块，在基于所述操作帧与所述疑似响应帧之间的时间差，从所述疑似响应帧中筛选出响应帧时，用于：
针对任一帧操作帧，将该帧操作帧之后的、与该操作帧之间的时间间隔最短的、且时间间隔超过预设时间间隔的疑似响应帧，作为与该操作帧对应的响应帧。
第三方面，本公开实施例还提供一种计算机设备，包括：处理器、存储器和总线，所述存储器存储有所述处理器可执行的机器可读指令，当计算机设备运行时，所述处理器与所述存储器之间通过总线通信，所述机器可读指令被所述处理器执行时执行上述第一方面，或第一方面中任一种可能的实施方式中的步骤。
第四方面，本公开实施例还提供一种计算机可读存储介质，该计算机可读存储介质上存储有计算机程序，该计算机程序被处理器运行时执行上述第一方面，或第一方面中任一种可能的实施方式中的步骤。
为使本公开的上述目的、特征和优点能更明显易懂，下文特举较佳实施例，并配合所附附图，作详细说明如下。
附图说明
为了更清楚地说明本公开实施例的技术方案，下面将对实施例中所需要使用的附图作简单地介绍，此处的附图被并入说明书中并构成本说明书中的一部分，这些附图示出了符合本公开的实施例，并与说明书一起用于说明本公开的技术方案。应当理解，以下附图仅示出了本公开的某些实施例，因此不应被看作是对范围的限定，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他相关的附图。
图1示出了本公开实施例所提供的一种响应时延测试方法的流程图；
图2示出了本公开实施例所提供的响应时延测试方法中，测试视频画 面的示意图；
图3示出了本公开实施例所提供的一种响应时延测试方法的整体流程图；
图4示出了本公开实施例所提供的响应时延测试方法中，确定操作帧具体方法的流程图；
图5示出了本公开实施例所提供的响应时延测试方法中，确定疑似响应帧的具体方法的流程图；
图6示出了本公开实施例所提供的响应时延测试方法中，另一种确定疑似响应帧的具体方法的流程图；
图7示出了本公开实施例所提供的一种响应时延测试装置的示意图；
图8示出了本公开实施例所提供的一种计算机设备的结构示意图。
具体实施方式
为使本公开实施例的目的、技术方案和优点更加清楚，下面将结合本公开实施例中附图，对本公开实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本公开一部分实施例，而不是全部的实施例。通常在此处附图中描述和示出的本公开实施例的组件可以以各种不同的配置来布置和设计。因此，以下对在附图中提供的本公开的实施例的详细描述并非旨在限制要求保护的本公开的范围，而是仅仅表示本公开的选定实施例。基于本公开的实施例，本领域技术人员在没有做出创造性劳动的前提下所获得的所有其他实施例，都属于本公开保护的范围。
经研究发现，现有的黑盒测试技术主要包括以下三种方式来测试响应时延：
方式一、高速相机拍摄加人工数帧。
具体的，可以通过外置高速相机拍摄用户设备画面，然后再通过人工去数帧，逐帧确定操作帧和响应帧。然而这种方式中，高速相机拍摄得到 的画面帧数极多(每秒超过250帧画面)，而且需要人工确认每一帧是否为操作帧或响应帧，导致这种方式的人工成本较高。
方式二、视频信号检测。
具体的，可以是在用户设备上安装游戏平台，然后在游戏平台上运行游戏程序，并通过游戏平台对游戏画面进行实时检测，确定操作帧与响应帧。但是由于不同的平台对视频信号检测的标准并不相同，不同平台的时延数据无法比较，因此视频信号检测方法只能在特定平台使用。
方式三、CNN(Convolutional Neural Network，卷积神经网络)检测。
具体的，可以是用户利用设备进行游玩的同时利用该设备对游戏画面进行录制，得到游戏视频，并利用CNN对视频画面进行检测，确定操作帧与响应帧。然而，CNN较为适合对单一动作进行检测，而对游戏中经常出现的连续动作检测效果不佳，难以检测多种云游戏项目，泛化能力较差。
综上所述，现有的黑盒测试技术存在人工成本高，泛化能力差的问题。
基于上述研究，本公开提供了一种响应时延测试方法、装置、计算机设备及存储介质，在获取待测试程序的测试视频后，分别对所述测试视频的相邻两帧视频帧的第一显示区域以及第二显示区域进行检测，确定所述测试视频的操作帧以及响应帧，并基于所述操作帧和所述响应帧，计算所述待测试程序的响应时延。通过这种方法，在一台设备上即可对完成所有云游戏项目的响应时延测试，测试过程无需人工参与，在保证识别准确的基础上，降低了人工成本且具有较强的泛化能力。
应注意到：相似的标号和字母在下面的附图中表示类似项，因此，一旦某一项在一个附图中被定义，则在随后的附图中不需要对其进行进一步定义和解释。
本文中术语“和/或”，仅仅是描述一种关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。另外，本文中术语“至少一种”表示多种中的任意一种 或多种中的至少两种的任意组合，例如，包括A、B、C中的至少一种，可以表示包括从A、B和C构成的集合中选择的任意一个或多个元素。
可以理解的是，在使用本公开各实施例公开的技术方案之前，均应当依据相关法律法规通过恰当的方式对本公开所涉及个人信息的类型、使用范围、使用场景等告知用户并获得用户的授权。
例如，在响应于接收到用户的主动请求时，向用户发送提示信息，以明确地提示用户，其请求执行的操作将需要获取和使用到用户的个人信息。从而，使得用户可以根据提示信息来自主地选择是否向执行本公开技术方案的操作的电子设备、应用程序、服务器或存储介质等软件或硬件提供个人信息。
作为一种可选的但非限定性的实现方式，响应于接收到用户的主动请求，向用户发送提示信息的方式例如可以是弹窗的方式，弹窗中可以以文字的方式呈现提示信息。此外，弹窗中还可以承载供用户选择“同意”或者“不同意”向电子设备提供个人信息的选择控件。
可以理解的是，上述通知和获取用户授权过程仅是示意性的，不对本公开的实现方式构成限定，其它满足相关法律法规的方式也可应用于本公开的实现方式中。
为便于对本实施例进行理解，首先对本公开实施例所公开的一种响应时延测试方法进行详细介绍，本公开实施例所提供的响应时延测试方法的执行主体一般为具有一定计算能力的计算机设备，该计算机设备例如包括：终端设备或服务器或其它处理设备，终端设备可以为用户设备(User Equipment，UE)、移动设备、用户终端、终端、蜂窝电话、无绳电话、个人数字助理(Personal Digital Assistant，PDA)、手持设备、计算设备、车载设备、可穿戴设备等。在一些可能的实现方式中，该响应时延测试方法可以通过处理器调用存储器中存储的计算机可读指令的方式来实现。
下面以执行主体为终端设备为例对本公开实施例提供的响应时延测试 方法加以说明。
参见图1所示，为本公开实施例提供的响应时延测试方法的流程图，所述方法包括步骤101～步骤104，其中：
步骤101、获取待测试程序的测试视频。
步骤102、对所述测试视频的相邻两帧视频帧的第一显示区域进行检测，确定所述测试视频的操作帧；其中，所述第一显示区域用于表征当前屏幕界面的触发信息。
步骤103、对所述测试视频的相邻两帧视频帧的第二显示区域进行检测，确定所述测试视频的响应帧；其中，所述第二显示区域用于显示所述待测试程序的主体画面。
步骤104、基于所述操作帧和所述响应帧，计算所述待测试程序的响应时延。
以下是对上述步骤的详细描述。
针对步骤101：
所述待测试程序为与用户存在交互行为的程序，即待测试程序在用户进行操作之后，会基于用户操作执行响应动作，示例性的，所述待测试程序可以为云游戏程序。
一种可能的场景中，所述待测试程序可以为需要进行黑盒测试的程序，例如可以是小程序，又或者，所述待测试程序可以为第三方开发的应用程序，或者云游戏等。
具体的，在所述待测试程序为小程序的情况下，所述待测试程序可以为题库，所述题库在用户作答后可以立刻对用户的回答给出评价，此时，为保证用户的答题体验，需要对题库进行黑盒测试，保证题库的响应时延处于合理的区间。
或者，在所述待测试程序为应用程序的情况下，所述待测试程序可以 为购物软件，响应时延过高时，用户可能会重复点击例如下单等按钮，导致重复下单等问题的出现，为了防止类似问题的发生，需要对购物软件进行黑盒测试，保证购物软件的响应时延较低。
又或者，在所述待测试程序为云游戏的情况下，由于云游戏对网络性能的要求较高，响应时延过高会导致用户的操作无法被云游戏程序响应或者在较长一段时间后才能够得到响应，用户难以在响应时延较高的情况下完成想要完成的操作，对用户的游戏体验影响极大，因此，为了保证用户的游戏体验，需要对云游戏进行黑盒测试，保证响应时延较低。
所述测试视频可以通过多种方式获取，例如接收用户上传的测试视频，或录通过屏幕录制的方式获取所述待测试程序的测试视频。对于其他的可以获取测试视频的方式，本公开并不限制。本公开中，获取测试视频的过程为经过用户授权的，具体授权过程参照上方描述。
一种可能的实施方式中，所述测试视频在录制过程中，可以通过在客户端安装脚本的方式来模拟用户的触发操作，在执行触发操作之后，待测试程序可以对触发操作做出对应的响应，例如若触发操作为控制人物向前移动的触发操作，则待测试程序可以在检测到该触发操作之后，控制屏幕中的人物进行相应的移动。
具体的，客户端在安装脚本之后，脚本可以按照一定的频率运行，每次脚本运行之后，可以模拟用户一次的触发操作，脚本在不同次运行过程中，执行的触发操作可以是不同的(例如可以触发不同的功能按钮)。
针对步骤102和步骤103：
所述测试视频中可以包括第一显示区域和第二显示区域，所述第一显示区域为用于显示当前屏幕界面的触发信息的区域，例如可以显示当前触发位置坐标、横向滑动速度、纵向滑动速度等。所述第二显示区域为用于显示所述待测试程序的主体画面的区域，所述主体画面例如可以是待测试程序的主程序界面。示例性的，在所述待测试程序为游戏的情况下，所述 主体画面为显示操作按钮以及控制对象的画面。或者，在所述待测试程序为购物软件的情况下，所述主体画面可以为展示商品信息的画面。
可选地，所述第一显示区域可以屏幕界面的固定区域，例如可以位于屏幕界面的上边界位置处，所述第二显示区域可以为在展示所述待测试程序时，所述待测试程序的展示界面中除所述第一显示区域外的其他区域，一般地，所述待测试程序的展示界面在展示时可以全屏展示，示例性的，所述第一显示区域和所述第二显示区域的显示位置可以如图2所示。
具体的，用户可以在录制测试视频之前，在终端设备的开发者选项中打开“显示指针位置”的选项，这样，终端设备在执行任何触发操作时，都可以在所述第一显示区域显示。或者，在执行本公开所提供的方法之前，可以申请“显示指针位置”的权限，在获得授权之后，可以自动打开指针显示权限。
下面将对步骤102和步骤103中确定操作帧和响应帧的具体方法进行介绍。
一种可能的实施方式中，在对所述测试视频的相邻两帧视频帧的第一显示区域进行检测，确定所述测试视频的操作帧时，可以参照如下步骤：
步骤a1、针对任意相邻两帧视频帧，确定该相邻两帧视频帧的第一显示区域之间的第一差异信息。
其中，所述第一差异信息通过第一二值化图像表征，因此为了确定相邻两帧视频帧的第一显示区域之间的第一差异信息，需要得到所述第一二值化图像。
具体的，在确定所述第一二值化图像时，针对任意相邻两帧视频帧，先将该两帧视频帧的第一显示区域的第一区域图像相减，得到第一差异图像，再根据所述第一差异图像确定与其对应的第一二值化图像。
所述将两帧视频帧的第一显示区域的第一区域图像相减可以理解为，将两帧视频帧的第一显示区域的第一区域图像中，各个像素点的像素值对 应求差值，得到所述第一差异图像。
在根据所述第一差异图像确定与其对应的第一二值化图像，即进行第一差异图像的二值化时，可以先将第一差异图像进行灰度化处理，然后将灰度值小于预设阈值的像素点的像素值设置为第三目标值，将像素值超过预设阈值的像素点的像素值设置为第一目标值，示例性的，所述第一目标值可以为1/255，所述第三目标值可以为0。
步骤a2、在所述第一差异信息满足第一预设条件的情况下，将所述相邻两帧视频帧中靠后的视频帧作为所述操作帧。
其中，所述第一差异信息满足第一预设条件，示例性的可以是所述第一二值化图像的图像能量超过第一预设阈值。所述第一二值化图像的图像能量是通过所述第一二值化图像中像素值为第一目标值的像素点个数来确定的。
示例性的，若所述第一预设阈值为150，第一像素值为1，在检测到第3帧视频帧与第4帧视频帧的第一差异图像对应的第一二值化图像中的像素值为1的像素点个数为182个，由于第一像素值与像素值为1的像素点个数的乘积为182，大于第一预设阈值150，满足第一预设条件，将两帧视频帧中靠后的第4帧视频帧作为所述操作帧。
此处，若检测视频中无法检测到操作帧，则该检测视频中不能用于测试响应时延，需要重新执行步骤101以及步骤102，直至检测视频中能够检测到操作帧；同一段测试视频中也可能包含多帧操作帧，在完成一帧操作帧检测之后，可以在该操作帧之后继续检测其他操作帧。
在另外一种可能的实施方式中，由于所述第一显示区域显示的是与触发信息相关的数值，因此，在对所述测试视频的相邻两帧视频帧的第一显示区域进行检测，确定所述测试视频的操作帧时，针对任意相邻两帧视频帧，还可以通过光学字符识别(Optical Character Recognition，OCR)技术识别所述第一显示区域显示的数值，然后将该相邻两帧视频帧显示的数值 进行比较，若数值发生变化，则可以将该相邻两帧视频帧中靠后的视频帧作为所述操作帧。
一种可能的实施方式中，在对所述测试视频的相邻两帧视频帧的第二显示区域进行检测，确定所述测试视频的响应帧时，可以参照如下步骤：
步骤b1、针对任意相邻两帧视频帧，确定该相邻两帧视频帧的第二显示区域之间的第二差异信息。
其中，确定所述第二差异信息可以通过以下两种方式的任意一种进行：
方法一、所述第二差异信息可以通过第二二值化图像表征，在针对任意相邻两帧视频帧，确定该相邻两帧视频帧的第二显示区域之间的第二差异信息时，可以先针对任意相邻两帧视频帧，将该两帧视频帧的第二显示区域的第二区域图像相减，得到第二差异图像，并将所述第二差异图像进行二值化处理，得到中间二值化图像；然后对所述中间二值化图像中的取值为第二目标值的图像轮廓进行过滤处理，得到所述第二二值化图像。
其中，在进行第二差异图像的二值化时，所述第二目标值可以与第一目标值相同，也可以与第一目标值不同，示例性的，在第一目标值为1的情况下，第二目标值可以设置为1，又或者，所述第二目标值可以设置为255。
所述中间二值化图像中，取值为第二目标值的图像区域为所述相邻的两帧视频帧中存在差异的区域。然而为了避免视频画面中出现的例如落叶、水花等与操作无关的要素对第二差异信息的判断准确性产生影响，还需要对所述中间二值化图像中取值为第二目标值的图像轮廓进行过滤处理。示例性的，可以将所述中间二值化图像中，取值为第二目标值的图像轮廓中，对应的图像面积小于预设面积的轮廓过滤。
具体的，在将取值为第二目标值图像轮廓中，对应的图像面积小于预设面积的轮廓过滤时，可以将对应的图像面积小于预设面积的图像区域的像素值，由所述第二目标值调整为第四目标值，所述第四目标值为所述中 间二值化图像中，除所述第二目标值外的像素值，示例性的，所述第二目标值可以为1，所述第四目标值可以为0。
方法二、在所述视频帧中包括控制对象的情况下，由于控制对象的动作会导致角色的位置变化，第二差异信息还可以通过控制对象的坐标差异信息表征。
具体的，在确定控制对象的坐标差异信息时，针对任意相邻两帧视频帧，可以对该两帧视频帧中的控制对象进行角点标记，确定所述角点在所述两帧视频帧中各视频帧中的位置坐标，并确定所述角点在所述两帧视频帧中各视频帧中的位置坐标之间的坐标差异信息。
实际应用中，标记的角点可以是控制对象容易发生变化的位置点，例如若所述控制对象为游戏人物，则标记的角点可以包括手指、脚、膝盖、袖口、裙摆等位置的点。不同控制对象标记的角点的数量和角点的位置可以是不同的。
实际应用中，针对画面变化较大的视频帧，可以通过上述方法一计算第二差异信息，针对画面变化较小的视频帧，可以通过上述方法二计算第二差异信息，而画面变化的大小可以通过触发操作的动作类型来区分，可以预先定义画面变化较大的触发操作，以及画面变化较小的触发操作。
步骤b2、基于所述第二差异信息，确定所述测试视频的疑似响应帧。
若采用方法一来确定所述测试视频的疑似响应帧，则可以确定所述第二二值化图像的图像能量，并在所述第二二值化图像的图像能量超过第二预设阈值的情况下，将所述相邻两帧视频帧中靠后的视频帧作为所述疑似响应帧。
其中，所述第二二值化图像的图像能量同样可以通过所述第二二值化图像中像素值为第二目标值的像素点个数来确定的。具体的，所述图像能量可以是第二目标值与像素值为第二目标值的像素点个数的乘积，若所述第二预设阈值为15000，第二像素值为255，在检测到第15帧视频帧与第 16帧视频帧的第二差异图像对应的第二二值化图像中的像素值为第二像素值的像素点个数为100个，由于第一像素值与像素值为第二像素值的像素点个数的乘积为25500，大于第二预设阈值15000，将两帧视频帧中靠后的第16帧视频帧作为所述疑似响应帧。
若采用方法二来确定所述测试视频的疑似响应帧，则可以在所述第二差异信息满足第二预设条件的情况下，将所述相邻两帧视频帧中靠后的目标视频帧的移动信息标记为移动；并获取所述目标视频帧的前N帧视频帧的第一移动信息，以及后M帧视频帧的第二移动信息；其中，N、M为预设正整数；然后基于所述第一移动信息和所述第二移动信息，确定所述测试视频的疑似响应帧。
其中，所述第二预设条件可以是产生坐标差异的角点数量大于移动交点阈值。
示例性的，若所述移动角点阈值为8，在检测到第23帧视频帧与第24帧视频帧产生坐标差异的角点数量为11个的情况下，由于产生坐标差异的角点数量大于移动交点阈值8，满足第二预设条件，将两帧视频帧中靠后的第24帧视频帧的移动信息标记为移动。
实际应用中，由于需要逐帧检测操作帧和响应帧，因此针对所述测试视频的每一帧，都可以按照上述方法进行移动信息的标记。
在完成检测视频中各视频帧的移动信息的标记后，可以遍历所述检测视频中的每一帧视频帧，针对遍历的任一帧视频帧，都将该帧视频帧作为目标视频帧，然后获取所述目标视频帧的前N帧视频帧的第一移动信息，以及后M帧视频帧的第二移动信息；其中，N、M的取值可以相同；
此处，需要说明的是，在游戏过程中，即使并未进行操作，角色也可能会产生一定动作，此时，动作并非是响应操作产生的，而是因为角色在闲置时也会产生例如晃动身体等动作，这就导致一部分视频帧的移动信息可能被错误标记为移动，且无法通过上文介绍的过滤轮廓的方法来避免。
因此，为提高识别准确率，在进行疑似响应帧的确定时，可以分别设置第一移动阈值和第二移动阈值，并基于第一移动阈值、第二移动阈值、所述第一移动信息和所述第二移动信息，确定所述测试视频的疑似响应帧。
其中，所述第一移动信息和所述第二移动信息可以通过数值来表征，示例性的，数字“1”可以表征移动信息中的“移动”，数字“0”可以表征移动信息中的“静止”。
若N和M的取值均为7，且目标视频帧的前7帧视频帧的移动信息均为静止，目标视频帧的移动信息为移动，目标视频帧的后7帧视频帧的移动信息为移动，则所述第一移动信息为“0000000”，所述第二移动信息为“1111111”。
第一移动阈值用于限定第一移动信息，所述第二移动阈值用于限定第二移动信息，所述第一移动阈值可以为目标视频帧前的视频帧的数量N与第一预设值的乘积，第二移动阈值可以为目标视频帧后的视频帧的数量M与第二预设值的乘积。
基于第一移动阈值、第二移动阈值、所述第一移动信息和所述第二移动信息，确定所述测试视频的疑似响应帧时，可以分为以下两种情况：
情况一、若所述第一移动信息之和小于所述第一移动阈值，且所述第二移动信息之和大于所述第二移动阈值，则将该帧视频帧作为疑似响应帧。
情况二、若所述第一移动信息和所述第二移动信息不满足情况一，由于不能够保证其识别准确率，对于该帧视频帧不做任何标记。
示例性的，若N和M的取值均为7，所述第一移动信息为100000，第二移动信息为1111110，第一预设值为2/7，第二预设值为5/7，则可知第一移动阈值为2、第二移动阈值为5。此时，所述第一移动信息之和为1，小于所述第一移动阈值2，且所述第二移动信息之和为6，大于所述第二移动阈值5，满足情况一，所以将该帧视频帧作为疑似响应帧；若所述第一移动信息为111000，第二移动信息为1111111，第一预设值为2/7，第二预设值 为5/7，则可知第一移动阈值为2、第二移动阈值为5。此时，虽然所述第二移动信息之和为7，大于所述第二移动阈值5，但是所述第一移动信息之和为3，大于所述第一移动阈值2，不满足情况一，所以对于该帧视频帧不做任何标记。
这里，所述第一移动信息之和需要小于第一移动阈值，且第二移动信息之和需要大于所述第二移动阈值的原因在于，若某一视频帧之前的视频帧的移动信息大部分为静止，且该视频帧之后的视频帧的移动信息大部分为移动，则该帧很有可能为响应帧。
这里确定的疑似响应帧是指响应开始的视频帧，在另外一种可能的实施方式中，由于动作需要连贯执行的，而下一次的操作帧必定是在响应结束之后执行的，因此在确定疑似响应帧之后，还可以确定结束响应帧，以辅助检测下一操作帧/疑似响应帧。
具体的，在确定所述结束响应帧时，除了设置上述第一移动阈值、第二移动阈值外，还可以设置第三移动阈值和第四移动阈值，其中，所述第一移动阈值和所述第三移动阈值用于限定第一移动信息，所述第二移动阈值和所述第四移动阈值用于限定第二移动信息。
其中，所述第三移动阈值可以为目标视频帧前的视频帧的数量N与第三预设值的乘积，第四移动阈值可以为目标视频帧后的视频帧的数量M与第四预设值的乘积。
所述第三移动阈值大于所述第一移动阈值，所述第四移动阈值小于所述第二移动阈值。
示例性的，若M和N的取值均为10，第一预设值为1/10，第二预设值为8/10，第三预设值为8/10，第四预设值为1/10，则所述第一移动阈值为1，第二移动阈值为8，第三移动阈值为8，第四移动阈值为1。
具体的，在确定所述疑似响应帧时与上述方法可以相同，在确定结束响应帧时，可以分为以下几种情况：
情况一、若所述第一移动信息之和大于所述第三移动阈值，且若所述第二移动信息之和小于所述第四移动阈值，则将该帧视频帧作为结束响应帧。
情况二、若所述第一移动信息和所述第二移动信息不满足情况一，则不做任何标记。
示例性的，在M和N的取值均为10的情况下，若所述第一移动信息为1111111110，第二移动信息为0000000000，第三预设值为8/10，第四预设值为1/10，则可知第三移动阈值为8，第四移动阈值为1。此时，所述第一移动信息之和为9，大于所述第三移动阈值8，且所述第四移动信息之和为0，小于所述第二移动阈值1，满足情况一，所以将该帧视频帧作为结束响应帧；若所述第一移动信息为1111111111，第二移动信息为1100000000，第三预设值为8/10，第四预设值为1/10，则可知第三移动阈值为8，第四移动阈值为1。此时，所述第一移动信息之和为10，大于所述第三移动阈值8，但是所述第四移动信息之和为2，大于所述第二移动阈值1，不满足情况一，所以不对该帧视频帧做任何标记。
这里，所述第一移动信息之和需要大于第三移动阈值，且第二移动信息之和需要小于第四移动阈值的原因在于，若某一视频帧之前的视频帧的移动信息大部分为移动，且该视频帧之后的视频帧的移动信息大部分为静止，则该帧很有可能为结束响应帧。
确定结束响应帧的目的之一是辅助检测下一响应帧，在检测到疑似响应帧之后，在没有检测到所述结束响应帧的情况下，即使检测到满足疑似响应帧判断条件的视频帧，也不能将该帧视频帧作为疑似响应帧。
步骤b3、基于所述操作帧与所述疑似响应帧之间的时间差，从所述疑似响应帧中筛选出响应帧。
为了保证游戏体验，操作帧与对应响应帧之间的时间间隔不能过长，同时，由于云游戏的特性，操作帧与对应响应帧之间的时间间隔不可能过 短，因此，操作帧与对应响应帧之间的时间间隔存在一定范围，也就是说，在从所述疑似响应帧中筛选出响应帧时，针对任一帧操作帧，将该帧操作帧之后，且时间间隔满足预设时间间隔条件的疑似响应帧，作为与该操作帧对应的响应帧，示例性的，所述预设时间间隔条件可以为时间间隔大于50毫秒且小于200毫秒。
存在多帧与同一操作帧之间的时间间隔满足预设时间间隔条件的疑似响应帧的情况下，将与该操作帧之间的时间间隔最短的疑似响应帧，作为与该操作帧对应的响应帧。
此处，若检测视频中无法检测到疑似响应帧或无法从疑似响应帧中确定与操作帧对应的响应帧，则可能会因为计算精度导致操作帧或响应帧检测失误，则需要重新执行步骤101～步骤103，直至检测视频中能够检测到响应帧。
针对步骤104：
在计算所述待测试程序的响应时延时，计算响应帧和与其对应的操作帧的时间的差值，并将差值作为单次时延，示例性的，单次时延的计算方式可以如下：
Mtp＝P(i)-P(j)
其中，Mtp为单次时延，P(i)为第i帧响应帧的时间，P(j)为与第i帧响应帧对应的操作帧的时间。
另外，为保证时延数据的准确性，避免由部分极端单次时延数据代表响应时延，导致时延数据不准确情况的发生，可以将全部单次时延的平均值作为响应时延。
下面将结合整体流程图，对上述响应时延测试方法进行整体介绍，参照图3所示为本公开实施例提供的一种响应时延测试方法的整体流程图，主要包括以下几个步骤：
先获取待测试程序的测试视频，然后将测试视频的画面分为第一显示区域以及第二显示区域，并分别进行处理。
对所述测试视频的相邻两帧视频帧的第一显示区域进行检测，可以确定所述测试视频的操作帧。
对于第二显示区域，首先对画面变化的大小进行区分，对于画面变化较大的情况，可以基于相邻两帧视频帧的第二差异图像确定疑似响应帧。
而对于画面变化较小的情况，可以基于预先标注的角点的位置差异信息确定疑似响应帧。
在得到操作帧以及疑似响应帧之后，基于所述操作帧与所述疑似响应帧之间的时间差，从所述疑似响应帧中筛选出响应帧。
然后计算响应帧和与其对应的操作帧的时间的差值，得到响应时延。
其中，在检测操作帧时，可以参照如图4所示的流程图，包括如下几个步骤:
针对任意相邻两帧视频帧，先将该两帧视频帧的第一显示区域的第一区域图像相减，得到第一差异图像，再对所述第一差异图像进行灰度化以及二值化处理，确定与其对应的第一二值化图像。
通过对第一二值化图像进行计算，可以得到第一二值化图像的图像能量，此时，将第一二值化图像的图像能量与第一预设阈值进行比较，若第一二值化图像的图像能量大于第一预设阈值，则可以将这两帧视频帧的后一帧确定为操作帧，由于同一段测试视频中可能包含多帧操作帧，在完成一帧操作帧检测之后，可以在该操作帧之后继续检测其他操作帧。
检测疑似响应帧时，在画面变化较大的情况下，可以基于相邻两帧视频帧的第二差异图像确定疑似响应帧，可以参照如图5所示的流程图，包括如下几个步骤:
针对任意相邻两帧视频帧，先将该两帧视频帧的第二显示区域的第二 区域图像相减，得到第二差异图像，再对所述第一差异图像进行灰度化以及二值化处理，确定与其对应的中间二值化图像。对中间二值化图像中面积小于预设面积的轮廓进行过滤，得到第二二值化图像。
通过对第二二值化图像进行计算，可以得到第二二值化图像的图像能量，此时，将第二二值化图像的图像能量与第二预设阈值进行比较，若第二二值化图像的图像能量大于第二预设阈值，则可以将这两帧视频帧的后一帧确定为疑似响应帧。
而对于画面变化较小的情况，可以基于预先标注的角点的位置差异信息确定疑似响应帧，在检测疑似响应帧时，可以参照如图6所示的流程图，包括如下几个步骤:
首先获取视频画面中的控制对象，并对控制对象进行角点标注，然后取检测视频中相邻两帧进行对比，得到产生坐标差异的角点数量，若数量大于移动角点阈值，则将两帧视频帧中的后一帧的移动信息标记为移动，否则将两帧视频帧中的后一帧的移动信息标记为静止，并获取该帧前N帧视频帧的第一移动信息以及后M帧视频帧的第二移动信息。
比较第一移动信息与第一移动阈值以及第三移动阈值的关系，并比较第二移动信息与第二移动阈值以及第四移动阈值的关系，确定该帧是否为疑似响应帧或结束响应帧。
若第一移动信息之和小于第一移动阈值且第二移动信息之和大于第二移动阈值，则将该帧确定为疑似响应帧；若第一移动信息之和大于第三移动阈值且第二移动信息之和小于第四移动阈值，则将该帧确定为结束响应帧。其中，结束响应帧可以辅助判断疑似响应帧，具体的，在检测到疑似响应帧之后，在没有检测到所述结束响应帧的情况下，即使检测到满足疑似响应帧判断条件的视频帧，也不能将该帧视频帧作为疑似响应帧。
本领域技术人员可以理解，在具体实施方式的上述方法中，各步骤的撰写顺序并不意味着严格的执行顺序而对实施过程构成任何限定，各步骤 的具体执行顺序应当以其功能和可能的内在逻辑确定。
基于同一发明构思，本公开实施例中还提供了与响应时延测试方法对应的响应时延测试装置，由于本公开实施例中的装置解决问题的原理与本公开实施例上述响应时延测试方法相似，因此装置的实施可以参见方法的实施，重复之处不再赘述。
参照图7所示，为本公开实施例提供的一种响应时延测试装置的架构示意图，所述装置包括：获取模块701、第一检测模块702、第二检测模块703、计算模块704；其中，
获取模块701，用于获取待测试程序的测试视频；
第一检测模块702，用于对所述测试视频的相邻两帧视频帧的第一显示区域进行检测，确定所述测试视频的操作帧；其中，所述第一显示区域用于表征当前屏幕界面的触发信息；
第二检测模块703，用于对所述测试视频的相邻两帧视频帧的第二显示区域进行检测，确定所述测试视频的响应帧；其中，所述第二显示区域用于显示所述待测试程序的主体画面；
计算模块704，用于基于所述操作帧和所述响应帧，计算所述待测试程序的响应时延。
一种可能的实施方式中，所述第一检测模块702，在对所述测试视频的相邻两帧视频帧的第一显示区域进行检测，确定所述测试视频的操作帧时，用于：
针对任意相邻两帧视频帧，确定该相邻两帧视频帧的第一显示区域之间的第一差异信息；
在所述第一差异信息满足第一预设条件的情况下，将所述相邻两帧视频帧中靠后的视频帧作为所述操作帧。
一种可能的实施方式中，针对任意相邻两帧视频帧，所述第一检测模 块702，在确定该相邻两帧视频帧的第一显示区域之间的第一差异信息时，用于：
针对任意相邻两帧视频帧，将该两帧视频帧的第一显示区域的第一区域图像相减，得到第一差异图像；
确定所述第一差异图像对应的第一二值化图像；所述第一差异信息通过所述第一二值化图像表征；
所述第一差异信息满足第一预设条件，包括：
所述第一二值化图像的图像能量超过第一预设阈值；其中，所述第一二值化图像的图像能量通过所述第一二值化图像中像素值为第一目标值的像素点个数来确定。
一种可能的实施方式中，所述第二检测模块703，在对所述测试视频的相邻两帧视频帧的第二显示区域进行检测，确定所述测试视频的响应帧，用于：
针对任意相邻两帧视频帧，确定该相邻两帧视频帧的第二显示区域之间的第二差异信息；
基于所述第二差异信息，确定所述测试视频的疑似响应帧；
基于所述操作帧与所述疑似响应帧之间的时间差，从所述疑似响应帧中筛选出响应帧。
一种可能的实施方式中，针对任意相邻两帧视频帧，所述第二检测模块703，在确定该相邻两帧视频帧的第二显示区域之间的第二差异信息时，用于：
针对任意相邻两帧视频帧，将该两帧视频帧的第二显示区域的第二区域图像相减，得到第二差异图像；
确定所述第二差异图像对应的第二二值化图像；所述第二差异信息通过所述第二二值化图像表征。
一种可能的实施方式中，所述第二检测模块703，在确定所述第二差异图像对应的第二二值化图像时，用于：
将所述第二差异图像进行二值化处理，得到中间二值化图像；
对所述中间二值化图像中的取值为第二目标值的图像轮廓进行过滤处理，得到所述第二二值化图像。
一种可能的实施方式中，所述第二检测模块703，在基于所述第二差异信息，确定所述测试视频的疑似响应帧时，用于：
确定所述第二二值化图像的图像能量；
在所述第二二值化图像的图像能量超过第二预设阈值的情况下，将所述相邻两帧视频帧中靠后的视频帧作为所述疑似响应帧。
一种可能的实施方式中，在所述视频帧中包括控制对象的情况下，针对任意相邻两帧视频帧，所述第二检测模块703，在确定该相邻两帧视频帧的第二显示区域之间的第二差异信息时，用于：
针对任意相邻两帧视频帧，对该两帧视频帧中的控制对象进行角点标记，确定所述角点在所述两帧视频帧中各视频帧中的位置坐标；
确定所述角点在所述两帧视频帧中各视频帧中的位置坐标之间的坐标差异信息，所述第二差异信息通过所述坐标差异信息表征。
一种可能的实施方式中，所述第二检测模块703，在基于所述第二差异信息，确定所述测试视频的疑似响应帧时，用于：
在所述第二差异信息满足第二预设条件的情况下，将所述相邻两帧视频帧中靠后的目标视频帧的移动信息标记为移动；
获取所述目标视频帧的前N帧视频帧的第一移动信息，以及后M帧视频帧的第二移动信息；其中，N、M为预设正整数；
基于所述第一移动信息和所述第二移动信息，确定所述测试视频的疑似响应帧。
一种可能的实施方式中，所述第二检测模块703，在基于所述操作帧与所述疑似响应帧之间的时间差，从所述疑似响应帧中筛选出响应帧时，用于：
针对任一帧操作帧，将该帧操作帧之后的、与该操作帧之间的时间间隔最短的、且时间间隔超过预设时间间隔的疑似响应帧，作为与该操作帧对应的响应帧。
关于装置中的各模块的处理流程、以及各模块之间的交互流程的描述可以参照上述方法实施例中的相关说明，这里不再详述。
基于同一技术构思，本公开实施例还提供了一种计算机设备。参照图8所示，为本公开实施例提供的计算机设备800的结构示意图。本公开实施例中的计算机设备可以包括但不限于诸如移动电话、笔记本电脑、数字广播接收器、PDA(个人数字助理)、PAD(平板电脑)、PMP(便携式多媒体播放器)、车载终端(例如车载导航终端)等等的移动终端以及诸如数字TV、台式计算机等等的固定终端，或者各种形式的服务器，如独立服务器或者服务器集群。图8示出的计算机设备仅仅是一个示例，不应对本公开实施例的功能和使用范围带来任何限制。
如图8所示，计算机设备800可以包括处理装置(例如中央处理器、响应时延测试器等)801，其可以根据存储在只读存储装置(ROM)802中的程序或者从存储装置805加载到随机访问存储装置(RAM)803中的程序而执行各种适当的动作和处理。在RAM 803中，还存储有计算机设备800操作所需的各种程序和数据。处理装置801、ROM 802以及RAM 803通过总线804彼此相连。输入/输出(I/O)接口805也连接至总线804。
通常，以下装置可以连接至I/O接口805：包括例如触摸屏、触摸板、键盘、鼠标、摄像头、麦克风、加速度计、陀螺仪等的输入装置806；包括例如液晶显示器(LCD)、扬声器、振动器等的输出装置807；包括例如磁带、硬盘等的存储装置808；以及通信装置809。通信装置809可以允许计算机 设备800与其他设备进行无线或有线通信以交换数据。虽然图8示出了具有各种装置的计算机设备800，但是应理解的是，并不要求实施或具备所有示出的装置。可以替代地实施或具备更多或更少的装置。
特别地，根据本公开的实施例，上文参考流程图描述的过程可以被实现为计算机软件程序。例如，本公开的实施例包括一种计算机程序产品，其包括承载在计算机可读介质上的计算机程序，该计算机程序包含用于执行响应时延测试方法的程序代码。在这样的实施例中，该计算机程序可以通过通信装置809从网络上被下载和安装，或者从存储装置808被安装，或者从ROM 802被安装。在该计算机程序被处理装置801执行时，执行本公开实施例的方法中限定的上述功能。
本公开实施例还提供一种计算机可读存储介质，该计算机可读存储介质上存储有计算机程序，该计算机程序被处理器运行时执行上述方法实施例中所述的响应时延测试方法的步骤。其中，该存储介质可以是易失性或非易失的计算机可读取存储介质。
本公开实施例还提供一种计算机程序产品，该计算机产品承载有程序代码，所述程序代码包括的指令可用于执行上述方法实施例中所述的响应时延测试方法的步骤，具体可参见上述方法实施例，在此不再赘述。
其中，上述计算机程序产品可以具体通过硬件、软件或其结合的方式实现。在一个可选实施例中，所述计算机程序产品具体体现为计算机存储介质，在另一个可选实施例中，计算机程序产品具体体现为软件产品，例如软件开发包(Software Development Kit，SDK)等等。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的系统和装置的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。在本公开所提供的几个实施例中，应该理解到，所揭露的系统、装置和方法，可以通过其它的方式实现。以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分， 实际实现时可以有另外的划分方式，又例如，多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些通信接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本公开各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。
所述功能如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个处理器可执行的非易失的计算机可读取存储介质中。基于这样的理解，本公开的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本公开各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(Read-Only Memory，ROM)、随机存取存储器(Random Access Memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
最后应说明的是：以上所述实施例，仅为本公开的具体实施方式，用以说明本公开的技术方案，而非对其限制，本公开的保护范围并不局限于此，尽管参照前述实施例对本公开进行了详细的说明，本领域的普通技术人员应当理解：任何熟悉本技术领域的技术人员在本公开揭露的技术范围内，其依然可以对前述实施例所记载的技术方案进行修改或可轻易想到变化，或者对其中部分技术特征进行等同替换；而这些修改、变化或者替换， 并不使相应技术方案的本质脱离本公开实施例技术方案的精神和范围，都应涵盖在本公开的保护范围之内。因此，本公开的保护范围应所述以权利要求的保护范围为准。

Claims (14)

  1. 一种响应时延测试方法，包括：
    获取待测试程序的测试视频；
    对所述测试视频的相邻两帧视频帧的第一显示区域进行检测，确定所述测试视频的操作帧；其中，所述第一显示区域用于表征当前屏幕界面的触发信息；
    对所述测试视频的相邻两帧视频帧的第二显示区域进行检测，确定所述测试视频的响应帧；其中，所述第二显示区域用于显示所述待测试程序的主体画面；
    基于所述操作帧和所述响应帧，计算所述待测试程序的响应时延。
  2. 根据权利要求1所述的方法，其中所述对所述测试视频的相邻两帧视频帧的第一显示区域进行检测，确定所述测试视频的操作帧，包括：
    针对任意相邻两帧视频帧，确定该相邻两帧视频帧的第一显示区域之间的第一差异信息；
    在所述第一差异信息满足第一预设条件的情况下，将所述相邻两帧视频帧中靠后的视频帧作为所述操作帧。
  3. 根据权利要求2所述的方法，其中所述针对任意相邻两帧视频帧，确定该相邻两帧视频帧的第一显示区域之间的第一差异信息，包括：
    针对任意相邻两帧视频帧，将该两帧视频帧的第一显示区域的第一区域图像相减，得到第一差异图像；
    确定所述第一差异图像对应的第一二值化图像；所述第一差异信息通过所述第一二值化图像表征；
    所述第一差异信息满足第一预设条件，包括：
    所述第一二值化图像的图像能量超过第一预设阈值；其中，所述第一 二值化图像的图像能量通过所述第一二值化图像中像素值为第一目标值的像素点个数来确定。
  4. 根据权利要求1所述的方法，其中所述对所述测试视频的相邻两帧视频帧的第二显示区域进行检测，确定所述测试视频的响应帧，包括：
    针对任意相邻两帧视频帧，确定该相邻两帧视频帧的第二显示区域之间的第二差异信息；
    基于所述第二差异信息，确定所述测试视频的疑似响应帧；
    基于所述操作帧与所述疑似响应帧之间的时间差，从所述疑似响应帧中筛选出响应帧。
  5. 根据权利要求4所述的方法，其中所述针对任意相邻两帧视频帧，确定该相邻两帧视频帧的第二显示区域之间的第二差异信息，包括：
    针对任意相邻两帧视频帧，将该两帧视频帧的第二显示区域的第二区域图像相减，得到第二差异图像；
    确定所述第二差异图像对应的第二二值化图像；所述第二差异信息通过所述第二二值化图像表征。
  6. 根据权利要求5所述的方法，其中所述确定所述第二差异图像对应的第二二值化图像，包括：
    将所述第二差异图像进行二值化处理，得到中间二值化图像；
    对所述中间二值化图像中的取值为第二目标值的图像轮廓进行过滤处理，得到所述第二二值化图像。
  7. 根据权利要求5或6所述的方法，其中所述基于所述第二差异信息，确定所述测试视频的疑似响应帧，包括：
    确定所述第二二值化图像的图像能量；
    在所述第二二值化图像的图像能量超过第二预设阈值的情况下，将所 述相邻两帧视频帧中靠后的视频帧作为所述疑似响应帧。
  8. 根据权利要求4所述的方法，其中在所述视频帧中包括控制对象的情况下，所述针对任意相邻两帧视频帧，确定该相邻两帧视频帧的第二显示区域之间的第二差异信息，包括：
    针对任意相邻两帧视频帧，对该两帧视频帧中的控制对象进行角点标记，确定所述角点在所述两帧视频帧中各视频帧中的位置坐标；
    确定所述角点在所述两帧视频帧中各视频帧中的位置坐标之间的坐标差异信息，所述第二差异信息通过所述坐标差异信息表征。
  9. 根据权利要求8所述的方法，其中所述基于所述第二差异信息，确定所述测试视频的疑似响应帧，包括：
    在所述第二差异信息满足第二预设条件的情况下，将所述相邻两帧视频帧中靠后的目标视频帧的移动信息标记为移动；
    获取所述目标视频帧的前N帧视频帧的第一移动信息，以及后M帧视频帧的第二移动信息；其中，N、M为预设正整数；
    基于所述第一移动信息和所述第二移动信息，确定所述测试视频的疑似响应帧。
  10. 根据权利要求4所述的方法，其中所述基于所述操作帧与所述疑似响应帧之间的时间差，从所述疑似响应帧中筛选出响应帧，包括：
    针对任一帧操作帧，将该帧操作帧之后的、与该操作帧之间的时间间隔最短的、且时间间隔超过预设时间间隔的疑似响应帧，作为与该操作帧对应的响应帧。
  11. 根据权利要求1所述的方法，其中在确定所述测试视频的响应帧之后，所述方法还包括：
    确定与所述响应帧对应的结束响应帧；其中，所述结束响应帧用于辅助检测下一操作帧/疑似响应帧。
  12. 一种响应时延测试装置，包括：
    获取模块，用于获取待测试程序的测试视频；
    第一检测模块，用于对所述测试视频的相邻两帧视频帧的第一显示区域进行检测，确定所述测试视频的操作帧；其中，所述第一显示区域用于表征当前屏幕界面的触发信息；
    第二检测模块，用于对所述测试视频的相邻两帧视频帧的第二显示区域进行检测，确定所述测试视频的响应帧；其中，所述第二显示区域用于显示所述待测试程序的主体画面；
    计算模块，用于基于所述操作帧和所述响应帧，计算所述待测试程序的响应时延。
  13. 一种计算机设备，包括：处理器、存储器和总线，所述存储器存储有所述处理器可执行的机器可读指令，当计算机设备运行时，所述处理器与所述存储器之间通过总线通信，所述机器可读指令被所述处理器执行时执行如权利要求1至11任一项所述的响应时延测试方法的步骤。
  14. 一种计算机可读存储介质，其中该计算机可读存储介质上存储有计算机程序，该计算机程序被处理器运行时执行如权利要求1至11任一项所述的响应时延测试方法的步骤。
PCT/CN2024/112144 2023-10-16 2024-08-14 一种响应时延测试方法、装置、计算机设备及存储介质 Pending WO2025082001A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202311340337.2 2023-10-16
CN202311340337.2A CN117354590A (zh) 2023-10-16 2023-10-16 一种响应时延测试方法、装置、计算机设备及存储介质

Publications (1)

Publication Number Publication Date
WO2025082001A1 true WO2025082001A1 (zh) 2025-04-24

Family

ID=89366311

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2024/112144 Pending WO2025082001A1 (zh) 2023-10-16 2024-08-14 一种响应时延测试方法、装置、计算机设备及存储介质

Country Status (2)

Country Link
CN (1) CN117354590A (zh)
WO (1) WO2025082001A1 (zh)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117354590A (zh) * 2023-10-16 2024-01-05 北京字跳网络技术有限公司 一种响应时延测试方法、装置、计算机设备及存储介质
CN119254950B (zh) * 2024-02-06 2025-09-19 荣耀终端股份有限公司 视频帧检测方法、设备及存储介质

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104468916A (zh) * 2014-10-24 2015-03-25 天津市康凯特软件科技有限公司 基于手机反应时间测试方法
CN106951346A (zh) * 2016-01-06 2017-07-14 阿里巴巴集团控股有限公司 一种响应时间的测试方法和装置
JP2018036687A (ja) * 2016-08-29 2018-03-08 株式会社日立製作所 情報処理装置および情報処理方法
CN108376117A (zh) * 2018-02-07 2018-08-07 网易（杭州）网络有限公司 交互响应的测试方法和设备
CN113141433A (zh) * 2021-03-23 2021-07-20 网易（杭州）网络有限公司 测试屏幕灵敏度的方法、装置和处理器
CN115695851A (zh) * 2022-12-28 2023-02-03 海马云（天津）信息技术有限公司 端到端延时计算方法、装置、存储介质及电子设备
CN117354590A (zh) * 2023-10-16 2024-01-05 北京字跳网络技术有限公司 一种响应时延测试方法、装置、计算机设备及存储介质

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104468916A (zh) * 2014-10-24 2015-03-25 天津市康凯特软件科技有限公司 基于手机反应时间测试方法
CN106951346A (zh) * 2016-01-06 2017-07-14 阿里巴巴集团控股有限公司 一种响应时间的测试方法和装置
JP2018036687A (ja) * 2016-08-29 2018-03-08 株式会社日立製作所 情報処理装置および情報処理方法
CN108376117A (zh) * 2018-02-07 2018-08-07 网易（杭州）网络有限公司 交互响应的测试方法和设备
CN113141433A (zh) * 2021-03-23 2021-07-20 网易（杭州）网络有限公司 测试屏幕灵敏度的方法、装置和处理器
CN115695851A (zh) * 2022-12-28 2023-02-03 海马云（天津）信息技术有限公司 端到端延时计算方法、装置、存储介质及电子设备
CN117354590A (zh) * 2023-10-16 2024-01-05 北京字跳网络技术有限公司 一种响应时延测试方法、装置、计算机设备及存储介质

Also Published As

Publication number Publication date
CN117354590A (zh) 2024-01-05

Similar Documents

Publication Publication Date Title
WO2025082001A1 (zh) 一种响应时延测试方法、装置、计算机设备及存储介质
CN112101305B (zh) 多路图像处理方法、装置及电子设备
US11058948B1 (en) Predictive modeling for multiplayer gaming
US20170192500A1 (en) Method and electronic device for controlling terminal according to eye action
CN108596955B (zh) 一种图像检测方法、图像检测装置及移动终端
CN107885645B (zh) 计算页面首屏渲染时长的方法、装置及电子设备
US9785201B2 (en) Controlling images at mobile devices using sensors
US20140354540A1 (en) Systems and methods for gesture recognition
CN107626105B (zh) 游戏画面显示方法及装置、存储介质、电子设备
CN110347608B (zh) 一种交互式应用的测试方法及相关装置
CN111294253B (zh) 测试数据处理方法、装置、计算机设备及存储介质
CN113259592A (zh) 拍摄方法、装置、电子设备及存储介质
CN114466182B (zh) 帧率检测方法、装置、计算机设备、存储介质和产品
CN112001442A (zh) 特征检测方法、装置、计算机设备及存储介质
CN114547905A (zh) 一种行人轨迹仿真方法、装置、设备及可读存储介质
WO2015176376A1 (zh) 自动调整有效触点的方法、装置及计算机存储介质
CN107831896B (zh) 音频信息播放方法、装置、存储介质及电子设备
CN103793053B (zh) 一种便携式终端的手势投影方法及装置
CN117492632A (zh) 触控响应方法、装置及设备
CN110213655A (zh) 一种浏览资源的曝光概率检测方法及曝光概率检测装置
CN109908584A (zh) 一种游戏信息的获取方法、装置及电子设备
CN116631056A (zh) 跑步机上的跌倒检测方法、装置、计算机设备和存储介质
CN111263084B (zh) 基于视频的手势抖动检测方法、装置、终端和介质
CN109933517A (zh) 基于安卓系统的测试方法、装置及设备
US20160239168A1 (en) Method and system of gui functionality management

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 24878653

Country of ref document: EP

Kind code of ref document: A1