CN115543714B - Remote desktop delay detection method and device, electronic equipment and system - Google Patents
Remote desktop delay detection method and device, electronic equipment and system Download PDFInfo
- Publication number
- CN115543714B CN115543714B CN202211534192.5A CN202211534192A CN115543714B CN 115543714 B CN115543714 B CN 115543714B CN 202211534192 A CN202211534192 A CN 202211534192A CN 115543714 B CN115543714 B CN 115543714B
- Authority
- CN
- China
- Prior art keywords
- time
- desktop
- image
- remote
- local
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/22—Detection or location of defective computer hardware by testing during standby operation or during idle time, e.g. start-up testing
- G06F11/2205—Detection or location of defective computer hardware by testing during standby operation or during idle time, e.g. start-up testing using arrangements specific to the hardware being tested
- G06F11/221—Detection or location of defective computer hardware by testing during standby operation or during idle time, e.g. start-up testing using arrangements specific to the hardware being tested to test buses, lines or interfaces, e.g. stuck-at or open line faults
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/22—Detection or location of defective computer hardware by testing during standby operation or during idle time, e.g. start-up testing
- G06F11/2273—Test methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/451—Execution arrangements for user interfaces
- G06F9/452—Remote windowing, e.g. X-Window System, desktop virtualisation
Abstract
The invention provides a method, a device, electronic equipment and a system for detecting delay of a remote desktop, which relate to the technical field of remote desktops, and the method comprises the following steps: in the process of controlling a remote device corresponding to a remote desktop to run a target application program, simultaneously capturing a first time coding image and a second time coding image displayed on a local desktop to obtain a first screen capturing image; decoding and analyzing the first screen capture image to obtain the delay time from the display of the desktop image of the remote desktop to the local desktop when the remote equipment runs the target application program; the first time coding image is generated by coding the remote desktop system time through the remote equipment, and the second time coding image is generated by coding the local desktop system time through the local equipment corresponding to the local desktop. The technical scheme provided by the invention can realize the delay detection of the remote desktop, and the obtained delay time can reflect the delay of the whole link for displaying the desktop image of the remote desktop to the local desktop.
Description
Technical Field
The invention relates to the technical field of remote desktops, in particular to a method, a device, electronic equipment and a system for detecting delay of a remote desktop.
Background
In the technical field of remote desktop, the content displayed on the screen of one computer can be transmitted to another computer by using a remote desktop protocol to be displayed, so that the remote office is realized, and the method is widely applied to the fields of enterprise office and the like.
In the process of transmitting the screen content of the remote desktop, the transmitted screen content needs to be encoded and then transmitted to another computer through a network, and then decoded and displayed on another computer, so that the screen content of the remote desktop is displayed on the local desktop with a delay, and the delay directly determines the fluency of the remote desktop. Therefore, how to detect the delay time of the remote desktop plays an important role in optimizing data transmission and improving the fluency of the remote desktop is a technical problem to be solved in the industry at present.
Disclosure of Invention
The invention provides a method, a device, electronic equipment and a system for delay detection of a remote desktop, which are used for realizing delay detection of the remote desktop.
The invention provides a remote desktop delay detection method, which comprises the following steps:
in the process of controlling the remote equipment corresponding to the remote desktop to run the target application program, simultaneously capturing a first time coding image and a second time coding image displayed on the local desktop to obtain a first screen capturing image; wherein the first time-coded image is generated by the remote device by coding a remote desktop system time; the second time coding image is generated by coding the local desktop system time by the local device corresponding to the local desktop;
and decoding and analyzing the first screen capture image to obtain the delay time of displaying the desktop image of the remote desktop to the local desktop when the remote equipment runs the target application program.
According to a method for detecting a delay of a remote desktop provided by the present invention, the decoding and analyzing the first screen capture image to obtain a delay time for displaying the desktop image of the remote desktop to the local desktop when the remote device runs the target application program, includes:
decoding the first screen capture image to acquire first time corresponding to the first time coding image and second time corresponding to the second time coding image in decoding data;
determining the time difference between the first time and the second time to obtain a first time difference;
determining the delay time for displaying a desktop image of the remote desktop to the local desktop when the target application is run by the remote device based on the first time difference and the inherent time difference; wherein the inherent time difference is an inherent system time difference between the remote desktop and the local desktop.
According to the method for detecting the delay of the remote desktop, before the remote device is controlled to run the target application program, the method further comprises the following steps:
under the condition that the remote equipment does not run an application program, acquiring a screen capturing image for simultaneously capturing the first time code image and the second time code image within a first set time period to obtain a second screen capturing image;
decoding each screen capture image in the second screen capture images, and determining a time difference between the time corresponding to the first time coding image and the time corresponding to the second time coding image in each screen capture image according to decoding data to obtain a second time difference of each screen capture image;
and calculating the average value of the second time differences of all screen capture images in the second screen capture image to obtain the inherent time difference.
According to the method for detecting the delay of the remote desktop, the screen capturing is simultaneously performed on the first time coding image and the second time coding image displayed on the local desktop to obtain the first screen capturing image, and the method comprises the following steps:
responding to a coded image moving instruction, and controlling the first time coded image and the second time coded image to move on the local desktop until the parallel arrangement distance of the first time coded image and the second time coded image is smaller than a set threshold value;
and simultaneously performing screen capture on the moved first time coding image and the second time coding image to obtain a first screen capture image.
The method for detecting the delay of the remote desktop further comprises the following steps:
determining the target application program under the condition that an application scene test instruction is detected;
controlling the remote equipment to run the target application program through a remote desktop window; the remote desktop window is displayed on the local desktop.
According to the method for detecting the delay of the remote desktop provided by the invention, the remote device is controlled to run the target application program through the window of the remote desktop, and the method comprises the following steps:
starting a scene simulation program corresponding to the target application program, wherein the scene simulation program is used for simulating the operation of the target application program;
and controlling the running of the target application program in the remote equipment through the remote desktop window based on the scene simulation program.
According to the method for detecting the delay of the remote desktop, provided by the invention, under the condition that an application scene test instruction is detected, the target application program is determined, and the method comprises the following steps:
under the condition that an application scene test instruction is detected, displaying an application scene selection interface on the local desktop, wherein the application scene selection interface comprises an application program selection control;
and responding to the selection operation facing the application program selection control, and determining the target application program.
The invention also provides a remote desktop delay detection device, comprising:
the screen capture module is used for simultaneously capturing the first time code image and the second time code image displayed on the local desktop to obtain a first screen capture image in the process of controlling the remote equipment corresponding to the remote desktop to run the target application program; wherein the first time-coded image is generated by the remote device by coding a remote desktop system time; the second time coding image is generated by coding the local desktop system time by the local device corresponding to the local desktop;
and the analysis module is used for decoding and analyzing the first screen capturing image to obtain the delay time of displaying the desktop image of the remote desktop to the local desktop when the remote equipment runs the target application program.
The invention also provides an electronic device, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor executes the computer program to realize the remote desktop delay detection method.
The invention also provides a remote desktop delay detection system, which comprises local equipment and remote equipment in communication connection with the local equipment, wherein the local equipment runs a local desktop, and the remote equipment runs a remote desktop;
the remote equipment is used for coding the remote desktop system time, generating a first time coding image and displaying the first time coding image on the remote desktop;
the local device is configured to perform the steps of the remote desktop latency detection method as described in any of the above.
The present invention also provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements a remote desktop latency detection method as described in any of the above.
The invention also provides a computer program product comprising a computer program which, when executed by a processor, implements a method of remote desktop latency detection as described in any one of the above.
According to the method, the device, the electronic equipment and the system for detecting the delay of the remote desktop, provided by the invention, the first time coding image and the second time coding image displayed on the local desktop can be simultaneously captured in the process of controlling the remote equipment corresponding to the remote desktop to run the target application program, so that the first captured image is obtained; the first time coding image can reflect the time when the desktop image of the remote desktop displayed on the local desktop is displayed on the remote desktop, and the second time coding image can reflect the time when the desktop image of the remote desktop is displayed on the local desktop.
Drawings
In order to more clearly illustrate the technical solutions of the present invention or the prior art, the drawings needed for the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.
Fig. 1 is a flowchart illustrating a method for detecting a delay of a remote desktop according to an embodiment of the present invention;
FIG. 2 is a schematic diagram illustrating a method for detecting a delay of a remote desktop according to an embodiment of the present invention;
FIG. 3 is a schematic structural diagram of a remote desktop latency detection system according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of a remote desktop controlled at a local desktop in an embodiment of the present invention;
fig. 5 is a second flowchart illustrating a remote desktop latency detection method according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of a remote desktop delay detection apparatus according to an embodiment of the present invention;
fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the ordinal numbers such as "first", "second", etc., used in the present invention for the names of components or objects are only used for distinguishing the described objects, and do not have any sequential or technical meaning.
The remote desktop latency detection method of the present invention is described below with reference to fig. 1. The remote desktop delay detection method can be applied to electronic equipment such as a server, a mobile phone and a computer, and can also be applied to a remote desktop delay detection device arranged in the electronic equipment such as the server, the mobile phone and the computer, and the remote desktop delay detection device can be realized by software, hardware or the combination of the software and the hardware.
Fig. 1 illustrates one of the flow diagrams of a remote desktop delay detection method provided by an embodiment of the present invention, where the remote desktop delay detection method may be applied to a local device on which a local desktop runs, and the local device may be, for example, an electronic device such as a server, a mobile phone, a computer, and the like. Referring to fig. 1, the remote desktop delay detection method may include the following steps 110 to 120.
Step 110: and in the process of controlling the remote equipment corresponding to the remote desktop to run the target application program, simultaneously capturing the screen of the first time coding image and the second time coding image displayed on the local desktop to obtain a first screen capturing image.
The first time encoded image is generated by the remote device encoding the remote desktop system time. For example, the remote device may obtain the remote desktop system time in real time, encode the remote desktop system time, convert the encoding result into a graphic form, obtain a first time encoded image, and then display the first time encoded image on the remote desktop. When the local device controls the remote desktop, the first time coding image is used as a part of the desktop image of the remote desktop, is transmitted to the local device through the network after being coded with the desktop image of the remote desktop, and is displayed on the local desktop after being decoded by the local device. The remote device may be, for example, an electronic device such as a server, a mobile phone, or a computer.
The second time-encoded image is generated by encoding the local desktop system time by the local device. For example, the local device may obtain the local desktop system time in real time, encode the local desktop system time, convert the encoding result into a graphic form, obtain a second time-encoded image, and then display the second time-encoded image on the local desktop.
The local device may display a remote desktop window on the local desktop through which the remote desktop is controlled, such as through which applications of the remote desktop may be accessed and operated.
In the process that the remote equipment runs the target application program, the local equipment can simultaneously capture the first time coding image and the second time coding image displayed in the local desktop, and the time coding information of the desktop image of the remote desktop when the remote desktop is displayed and the time coding information when the local desktop is displayed can be simultaneously obtained through the capture of the screen. The target application may be an office application in the remote device, and may include, for example and without limitation, at least one of a text editing Word document, a presentation PPT, a web browser, a video player, an audio player, drawing software, and image processing software.
For example, the first time-coded image and the second time-coded image may be displayed in a preset desktop area of the local desktop, which may be, for example, a preset display area on the local desktop, and the local device may capture the first time-coded image and the second time-coded image.
For example, the time-coded image may be identified from a desktop image of the local desktop, and the first time-coded image and the second time-coded image may be captured by identifying and determining a position area where the first time-coded image and the second time-coded image are located, and then capturing the screen of the position area.
For example, the first time-coded image and the second time-coded image may be moved on the local desktop. For example, the first time code image and the second time code image may be moved by a human inspector, the local device controls the first time code image and the second time code image to move on a local desktop in response to the moving operation, and the moved first time code image and second time code image are subjected to screen capture when receiving a screen capture instruction. Alternatively, the first time-coded image and the second time-coded image may be automatically moved on the local desktop until they are close together. For example, after receiving the coded image moving instruction, the local device may control the first time coded image and the second time coded image to move on the local desktop according to the coded image moving instruction, and stop moving until the parallel arrangement distance between the first time coded image and the second time coded image is smaller than a set threshold, and then perform screen capture on the moved first time coded image and the moved second time coded image to obtain a first screen capture image.
For example, the first time-encoded image and the second time-encoded image may be two-dimensional code images.
It will be appreciated that the first time-encoded image is continuously updated with the remote desktop system time and the second time-encoded image is continuously updated with the local desktop system time.
Step 120: and decoding and analyzing the first screen capture image to obtain the delay time from the display of the desktop image of the remote desktop to the local desktop when the remote equipment runs the target application program.
After the first screen capture image is obtained, decoding and analyzing the first screen capture image to obtain time information in a first time coding image and a second time coding image, wherein the first time coding image comprises time information of a desktop image of a remote desktop displayed on a local desktop when the remote desktop is displayed on the remote desktop, and the second time coding image comprises time information of the desktop image of the remote desktop when the local desktop is displayed on the remote desktop.
The remote desktop delay detection method provided by the embodiment of the invention can simultaneously capture the first time coded image and the second time coded image displayed by the local desktop to obtain the first screen capture image in the process of controlling the remote equipment corresponding to the remote desktop to run the target application program; the first time coding image can reflect the time when the desktop image of the remote desktop displayed on the local desktop is displayed on the remote desktop, and the second time coding image can reflect the time when the desktop image of the remote desktop is displayed on the local desktop. Furthermore, delays of different applications can be detected.
Based on the remote desktop delay detection method in the corresponding embodiment of fig. 1, in an example embodiment, decoding and analyzing the first screen capture image to obtain a delay time for displaying the desktop image of the remote desktop to the local desktop when the remote device runs the target application program may include: decoding the first screen capture image to acquire first time corresponding to the first time coding image and second time corresponding to the second time coding image in the decoded data; determining the time difference between the first time and the second time to obtain a first time difference; determining delay time for displaying a desktop image of the remote desktop to the local desktop when the remote device runs the target application program based on the first time difference and the inherent time difference; wherein the inherent time difference is the inherent system time difference between the remote desktop and the local desktop. For example, at the same time, the remote desktop system time is 12, the local desktop system time is 12.
For example, the difference between the first time difference and the inherent time difference may be calculated to obtain a delay time for displaying the desktop image of the remote desktop to the local desktop when the target application program is run on the remote desktop.
For example, the first screen capture image may be a screen capture image captured within a second set time period, each screen capture image in the first screen capture image may be decoded, a first time difference of each screen capture image is calculated according to a first time and a second time corresponding to each screen capture image in decoded data, then an average value of all first time differences obtained within the second set time period is calculated, and a difference value between the average value and a time inherent difference is calculated, so as to obtain a delay time for displaying the desktop image of the remote desktop to the local desktop when the remote desktop runs the target application program.
Based on the remote desktop delay detection method in the embodiment corresponding to fig. 1, in an example embodiment, a step of determining an inherent time difference may be further included before controlling a remote device corresponding to the remote desktop to run the target application program. Specifically, the method may include: under the condition that the remote equipment does not run the application program, acquiring a screen capture image for simultaneously capturing a first time code image and a second time code image within a first set time period to obtain a second screen capture image; decoding each screen capture image in the second screen capture images, and determining the time difference between the time corresponding to the first time coding image and the time corresponding to the second time coding image in each screen capture image according to the decoded data to obtain a second time difference of each screen capture image; and calculating the average value of the second time differences of all screen capture images in the second screen capture image to obtain the inherent time difference.
Based on the foregoing remote desktop delay detection methods of various embodiments, in an example embodiment, the capturing the screen of the first time encoded image and the second time encoded image displayed on the local desktop at the same time to obtain the first screen capture image may include: responding to a coded image moving instruction, and controlling the first time coded image and the second time coded image to move on a local desktop until the parallel arrangement distance of the first time coded image and the second time coded image is smaller than a set threshold value; and simultaneously performing screen capture on the moved first time coding image and the second time coding image to obtain a first screen capture image. Therefore, the first time coding image and the second time coding image are moved to the positions which are arranged in parallel and the distance of which is less than the set threshold value, the first time coding image and the second time coding image can be drawn together, the screen grabbing efficiency is improved, the data volume of the grabbed screen image is small, the efficiency of decoding and analyzing the grabbed screen image is improved, and the efficiency of remote desktop delay detection is improved.
Based on the remote desktop delay detection method of the foregoing embodiments, in an example embodiment, before controlling the remote device to run the target application, a step of determining the target application may be further included. Specifically, the method may include: under the condition that an application scene test instruction is detected, determining a target application program; and controlling the remote equipment to run the target application program through the remote desktop window, wherein the remote desktop window is displayed on the local desktop.
Optionally, controlling the remote device to run the target application through the remote desktop window may include: starting a scene simulation program corresponding to the target application program, wherein the scene simulation program is used for simulating the operation of the target application program; and controlling the running of the target application program in the remote equipment through the remote desktop window based on the scene simulation program. Therefore, the operation executed on the target application program in the remote desktop window can be simulated by utilizing the scene simulation program on the local equipment, the manual operation is not needed, the automation degree is high, and the delay detection efficiency of the remote desktop is improved.
Optionally, in the case that the application scenario test instruction is detected, determining the target application program may include: under the condition that an application scene test instruction is detected, displaying an application scene selection interface on a local desktop, wherein the application scene selection interface comprises an application program selection control; and determining the target application program in response to the selection operation of the application program-oriented selection control.
Based on the methods of the foregoing embodiments, the method for detecting a delay of a remote desktop according to the embodiments of the present invention is further illustrated with reference to fig. 2 to 5.
Fig. 2 is a schematic diagram illustrating a principle of a remote desktop latency detection method according to an embodiment of the present invention, and referring to fig. 2, latency detection of a remote desktop may be implemented by a measurement program. For example, the delay detection of the remote desktop may be implemented by a first measurement program, a second measurement program, a third measurement program, and a fourth measurement program.
The first measuring program runs in the local device, and along with the change of the local desktop system time, the first measuring program can synchronously acquire and encode the local desktop system time in real time, convert the encoded local desktop system time into a graphic form, acquire a second time encoding image and display the second time encoding image on the local desktop, so that the second time encoding image can be captured, decoded and analyzed by the third measuring program.
The second measuring program runs in the remote equipment, and along with the change of the time of the remote desktop system, the second measuring program can synchronously acquire and encode the time of the remote desktop system in real time, and convert the encoded time of the remote desktop system into a graphic form, obtain a first time encoded image and display the first time encoded image on the remote desktop for a third measuring program to capture, decode and analyze.
The third measuring program runs in the local device, can continuously capture the first time coding image and the second time coding image generated by the first measuring program and the second measuring program, and performs decoding analysis on the captured screen capture image, and outputs a delay detection result.
The fourth measurement program runs in the local device, is responsible for simulating an automatic test scenario, is a scenario simulation program for implementing application scenario simulation, and can simulate operations in different application scenarios, where the operations may include at least one of keyboard input, mouse click, mouse movement, and the like, but are not limited thereto. For example, taking the target application program as a Word document as an example, the fourth measurement program may continuously perform text editing for a preset duration, for example, 1 minute, at the same frequency, and change the line after editing the preset number of characters, and the same number of characters may be input into each line, so as to simulate the operation on the Word document. For another example, taking the target application program as a PPT document, the fourth measurement program may perform page turning operations at the same time interval, for example, to simulate operations on the PPT document. For another example, the target application program is a graphic editing software, and the fourth measurement program can control the graphic to move at a constant speed, enlarge, reduce, and the like, so as to simulate the operation of the graphic editing software.
The application scene simulated by the fourth measurement program may include, but is not limited to, at least one of zoom and move of an application window, word document editing, PPT page turning, web browsing, video playing, and layout editing.
For example, the following application scenarios can be simulated by the fourth measurement procedure: (1) window operation: for example, the content of the window is a detailed list and the window is fully paved, the complete content is displayed when the window is dragged and does not exceed the screen, the window is dragged at a constant speed by taking preset time as a period, the dragging track is a diamond shape, and the like; (2) Word document editing: for example, the method can be a full screen mode, no content is initially provided, the text format is default, characters are input at intervals of 0.1s, 30 lines are left and right by 20 characters in each line, and no scrolling occurs during the input period; (3) PPT page turning: for example, the PPT can be opened in a full screen mode and turned at a constant speed; and (4) browsing a webpage: for example, after the webpage is opened, the lower scroll key is continuously triggered for a preset number of times for a preset duration, and then the upper scroll key is triggered in a similar manner, and the steps are alternately circulated; (5) video playing: for example, a video is played in a full screen mode, a still picture is absent in the video playing period, the picture color is rich, and the video is continuously played for a preset time; (6) layout editing: for example, the layout window may be moved by continuously reducing the first preset number of times and then continuously amplifying the second preset number of times at intervals of a set duration.
The above is only an example, and does not limit the present invention, and the application scenario simulated by the fourth measurement program is not limited thereto.
Based on fig. 2, during application scene simulation, a target application program may be selected, such as a selected Word document, and a desktop image of the remote desktop may include a Word document interface and a first time-encoded image generated by the second measurement program, and then the desktop image transmission process of the remote desktop may include: the Word document and the first time coding image on the remote desktop are transmitted to the local equipment through the network after being subjected to graphic coding at the remote desktop end, and are displayed on the local desktop after being decoded by the local equipment.
Based on this, an embodiment of the present invention provides a remote desktop delay detection system, and fig. 3 exemplarily shows a schematic structural diagram of the remote desktop delay detection system, and referring to fig. 3, the remote desktop delay detection system may include a local device 310 and a remote device 320 communicatively connected to the local device 310. Wherein, the local device 310 runs a local desktop, and the remote device 320 runs a remote desktop; the remote device 320 may be configured to encode the remote desktop system time, generate a first time encoded image, and display the first time encoded image on the remote desktop; the local device 310 may be configured to encode the local desktop system time, generate a second time-encoded image, and display the second time-encoded image on the local desktop; the local device 310 may also display a remote desktop window on the local desktop according to the remote desktop protocol through which the remote device 320 is controlled.
For example, fig. 4 illustrates a schematic diagram of controlling a remote desktop on a local desktop, and referring to fig. 4, taking an office application controlling the remote desktop on the local desktop as an example, a remote desktop window 420 may be displayed in the local desktop 410, an interface of the office application in the remote device may be displayed in the remote desktop window 420, and the office application in the remote device may be controlled to run by operating the displayed office application in the remote desktop window 420.
In this embodiment of the present invention, the local device 310 may be configured to, in a process of controlling the remote device 320 to run the target application program, capture a screen of the first time encoded image and the second time encoded image displayed on the local desktop at the same time to obtain a first screen capture image, and then perform decoding analysis on the first screen capture image to obtain a delay time for displaying the desktop image of the remote desktop to the local desktop when the remote device runs the target application program.
With reference to fig. 2 to fig. 4, fig. 5 illustrates a second flowchart of a remote desktop latency detection method according to an embodiment of the present invention, where the remote desktop latency detection method can be applied to the remote desktop latency detection system shown in fig. 5. Referring to fig. 5, the remote desktop delay detection method may include the following steps 501 to 510.
Step 501: the local device generates a first time-coded image and displays the first time-coded image on a local desktop.
The local device may call the first measurement program, encode the local desktop system time through the first measurement program, obtain a first time encoded image, and display the first time encoded image on the local desktop.
Step 502: the remote device generates a second time-coded image and displays it on the remote desktop.
The remote device may invoke a second measurement program, encode the remote desktop system time via the second measurement program to obtain a second time encoded image, and display the second time encoded image on the remote desktop.
Step 503: a remote desktop window is displayed on the local desktop.
According to the remote desktop protocol, the desktop image of the remote desktop is transmitted to the local device through the network after being encoded, and is displayed on the local device after being decoded, and at the moment, the first time encoding image and the second time encoding image can be displayed on the local desktop. For example, the local device may display a remote desktop window at the local desktop, display a desktop image of the remote desktop in real time through the remote desktop window, and may control the remote device through the remote desktop window.
Step 504: and carrying out system time synchronization between the local equipment and the remote equipment to obtain the inherent time difference.
Under the condition that the remote device does not run the application program, namely under the condition that no application program is opened in the remote desktop, the local device can call a third measuring program, screen capture is carried out on a first time coding image and a second time coding image displayed on the local desktop within a first set time period, each screen capture image is decoded, a first time corresponding to the first time coding image and a second time corresponding to the second time coding image are obtained from decoding data of each screen capture image, the time difference between the first time and the second time is calculated, second time differences of all screen capture images within the first set time period can be obtained, then the average value of the second time differences is calculated, and the inherent system time difference between the remote desktop and the local desktop is obtained.
For example, in a first set time period T, screen capture obtains screen capture images A1, A2 and A3, and decoding the screen capture image A1 can obtain a time T corresponding to a first time-coded image 1 Time t corresponding to the second time-coded picture 2 And further obtain a time difference Delta 1 =t 1 -t 2 (ii) a Similarly, the time difference Δ corresponding to the screen capture images A2 and A3 can be obtained 2 And Δ 3 Then calculate Δ 1 、Δ 2 And Δ 3 The average value of (2) is defined as the inherent time difference.
Step 505: a target application is selected.
When the local device detects an application scene test instruction, an application scene selection interface can be displayed on the local desktop, the application scene selection interface can include an application program selection control, and a tester can select a target application program to be tested by operating the application program selection control, namely, the application program scene to be tested can be selected. For example, the application selection control may provide selectable applications in the form of a menu, which may include Word, web browsing, PPT, video playing, and the like, and the tester may select an application scenario that needs to be tested currently from the application scenario, and the local device may determine the target application according to the selection operation of the tester.
Optionally, for the same application program, at least one application scene may be set, different application scenes correspond to different operations, and a user may select one application scene from multiple application scenes through the application program selection control, and determine the application program corresponding to the application scene as the target application program. For example, the application selection control provides a plurality of selectable application scenarios in the form of a menu, such as three application scenarios, application scenario 1, application scenario 2, and application scenario 3. Suppose that an application scenario 1 therein corresponds to a PPT application; the application scenes 2 and 3 correspond to Word application programs, and the two application scenes have different operations on Word documents, such as different Word input frequencies, different Word formats and the like. And if the user selects the application scene 2 through the application program selection control, determining that the target application program is a Word application program. Illustratively, each application scene has a corresponding scene simulator, and when an application scene is selected, the corresponding scene simulator can be selected.
Step 506: the local device controls the remote device to run the target application program.
After the target application program is selected, the local device can call the fourth measurement program, simulate the operation aiming at the target application program through the fourth measurement program, act on the remote desktop window, and control the running of the target application program through the remote desktop window. For example, the selected target application program is a Word document, the local device may start the fourth measurement program, open the Word document in the remote device through the remote desktop window, simulate an operation of editing the Word document through a scene simulator corresponding to the target application program in the fourth measurement program, act on the remote desktop window, and edit the Word document in the remote device through the remote desktop window.
For example, each application program may correspond to a fourth measurement program, and the fourth measurement program may be used as a scene simulation program corresponding to the application scene.
Step 507: and the local equipment simultaneously performs screen capture on the first time coding image and the second time coding image within a second set time period.
The local device may utilize a third measurement program to simultaneously capture the second time-encoded image of the local desktop and the first time-encoded image displayed to the local desktop via the remote desktop window during the process of controlling the remote device to run the target application.
Step 508: and the local equipment decodes the screen capture image obtained by screen capture each time, and calculates a first time difference according to the decoded data.
The local equipment decodes the screen capture images obtained by screen capture each time, acquires first time corresponding to the first time code image and second time corresponding to the second time code image from the decoded data, calculates the time difference between the first time and the second time to obtain the first time difference of the screen capture images, and further can obtain the first time difference corresponding to all the screen capture images in a second set time period.
Step 509: the local device calculates the average value of all the first time differences obtained within the second set time period.
Step 510: and the local equipment calculates the difference value between the average value and the inherent time difference to obtain the delay time.
The delay time obtained at this time is the delay time of displaying the desktop image of the remote desktop to the local desktop when the local device controls the remote device to run the target application program through the remote desktop window, and the delay time can reflect the delay of the whole link from remote encoding, network transmission, local decoding to displaying of the desktop image of the remote desktop to the local desktop, and the delay of the remote desktop is obtained in the scene of running the target application program by the remote device.
If the delay of the remote desktop in the other application program scene needs to be detected, the target application program to be detected can be selected again through the step 505, and the delay detection of the remote desktop in the target application program scene is realized through the steps 506 to 510. Therefore, detection can be performed according to different application program scenes, and delay of the remote desktop under different application program scenes is obtained.
It is understood that, in an alternative embodiment, the inherent time difference may be detected once in the initial stage of the delay detection of the remote desktop without opening the application program in the remote desktop, and the inherent time difference may be used for calculating the delay time in any subsequent application program scenario. In another optional embodiment, the detection of the inherent time difference may be performed once before the target application program is selected, and the inherent time difference obtained by the detection may be used as the inherent time in the target application program scene of the current detection.
On one hand, the remote desktop delay detection method provided by the embodiment of the invention can perform graphic coding on respective desktop system time at the same time at the remote device and the local device, display time coding images obtained by respective coding on the local desktop, then perform screen capture on the time coding images of the local desktop at the local device, decode time information in the screen capture images, calculate delay time based on the time information, and obtain the delay time of the whole link from the coding of the remote device to the display of the desktop images of the remote desktop on the local desktop. On the other hand, delay detection of the remote desktop can be performed aiming at different application programs, delay time under different application program scenes can be obtained, the performance of the remote desktop under a given scene can be represented through the delay time, the pertinence is strong, and the pertinence of data transmission optimization and fluency improvement of the remote desktop is improved. Moreover, the local device can simulate the operation of controlling the application program in the remote device through the remote desktop window under different application program scenes, and does not need the operation of a tester, thereby realizing the automatic detection of the remote desktop delay and improving the efficiency of the remote desktop delay detection.
In the following, the remote desktop delay detection apparatus provided by the present invention is described, and the remote desktop delay detection apparatus described below and the remote desktop delay detection method described above may be referred to in correspondence.
Fig. 6 is a schematic structural diagram illustrating a remote desktop delay detection apparatus according to an embodiment of the present invention, and referring to fig. 6, the remote desktop delay detection apparatus 600 may include: the screen capture module 610 is configured to capture a screen of a first time encoded image and a second time encoded image displayed on a local desktop at the same time in a process of controlling a remote device corresponding to the remote desktop to run a target application program, so as to obtain a first screen capture image; the analysis module 620 is configured to perform decoding analysis on the first screen capture image to obtain a delay time for displaying the desktop image of the remote desktop to the local desktop when the remote device runs the target application program; the first time coding image is generated by coding the remote desktop system time by the remote equipment; the second time coding image is generated by coding the local desktop system time by the local device corresponding to the local desktop.
In an example embodiment, the analysis module 620 may include: the decoding unit is used for decoding the first screen capture image and acquiring first time corresponding to the first time coding image and second time corresponding to the second time coding image in the decoding data; the first determining unit is used for determining the time difference between the first time and the second time to obtain a first time difference; the second determining unit is used for determining the delay time from the display of the desktop image of the remote desktop to the local desktop when the remote equipment runs the target application program based on the first time difference and the inherent time difference; wherein the inherent time difference is the inherent system time difference between the remote desktop and the local desktop.
In an example embodiment, the remote desktop delay detection apparatus 600 may further include: the acquisition module is used for acquiring a screen capture image which simultaneously captures a first time code image and a second time code image within a first set time period under the condition that the remote equipment does not run an application program, so as to obtain a second screen capture image; the processing module is used for decoding each screen capture image in the second screen capture images, determining the time difference between the time corresponding to the first time coding image and the time corresponding to the second time coding image in each screen capture image according to the decoding data, and obtaining the second time difference of each screen capture image; and the calculation module is used for calculating the average value of the second time differences of all screen capturing images in the second screen capturing image to obtain the inherent time difference.
In an example embodiment, the screen capture module 610 may include: the first control unit is used for responding to the coded image moving instruction and controlling the first time coded image and the second time coded image to move on the local desktop until the parallel arrangement distance of the first time coded image and the second time coded image is smaller than a set threshold value; and the screen capturing unit is used for simultaneously capturing the screen of the moved first time coding image and the second time coding image to obtain a first screen capturing image.
In an example embodiment, the remote desktop delay detection apparatus 600 may further include: the determining module is used for determining the target application program under the condition that the application scene test instruction is detected; the control module is used for controlling the remote equipment to run the target application program through the remote desktop window; wherein the remote desktop window is displayed on the local desktop.
In one example embodiment, the control module may include: the starting unit is used for starting a scene simulation program corresponding to the target application program, and the scene simulation program is used for simulating the operation of the target application program; and the second control unit is used for controlling the running of the target application program in the remote equipment through the remote desktop window based on the scene simulation program.
In an example embodiment, the determining module may include: the display unit is used for displaying an application scene selection interface on the local desktop under the condition that the application scene test instruction is detected, wherein the application scene selection interface comprises an application program selection control; and the third determining unit is used for responding to the selection operation of the application program-oriented selection control and determining the target application program.
Fig. 7 illustrates a schematic structural diagram of an electronic device, which may include, as shown in fig. 7: a processor (processor) 710, a Communication Interface (Communication Interface) 720, a memory (memory) 730, and a Communication bus 740, wherein the processor 710, the Communication Interface 720, and the memory 730 may communicate with each other via the Communication bus 740. Processor 710 may invoke logic instructions in memory 730 to perform the remote desktop latency detection methods provided by the above-described method embodiments, which may include, for example: in the process of controlling the remote equipment corresponding to the remote desktop to run the target application program, simultaneously capturing a first time coding image and a second time coding image displayed on the local desktop to obtain a first screen capturing image; decoding and analyzing the first screen capture image to obtain the delay time of displaying the desktop image of the remote desktop to the local desktop when the remote equipment runs the target application program; the first time coding image is generated by coding the remote desktop system time through the remote equipment, and the second time coding image is generated by coding the local desktop system time through the local equipment corresponding to the local desktop.
In addition, the logic instructions in the memory 730 can be implemented in the form of software functional units and stored in a computer readable storage medium when the software functional units are sold or used as independent products. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk, and various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product, where the computer program product includes a computer program, the computer program may be stored on a computer-readable storage medium, and when the computer program is executed by a processor, a computer can execute the remote desktop delay detection method provided by the above-mentioned method embodiments, where the method may include, for example: in the process of controlling the remote equipment corresponding to the remote desktop to run the target application program, simultaneously capturing a first time coding image and a second time coding image displayed on the local desktop to obtain a first screen capturing image; decoding and analyzing the first screen capture image to obtain the delay time from the display of the desktop image of the remote desktop to the local desktop when the remote equipment runs the target application program; the first time coding image is generated by coding the remote desktop system time through the remote equipment, and the second time coding image is generated by coding the local desktop system time through the local equipment corresponding to the local desktop.
In yet another aspect, the present invention further provides a computer-readable storage medium, on which a computer program is stored, the computer program being implemented by a processor to perform the remote desktop delay detection method provided by the above method embodiments, and the method may include: in the process of controlling the remote equipment corresponding to the remote desktop to run the target application program, simultaneously capturing a first time coding image and a second time coding image displayed on the local desktop to obtain a first screen capturing image; decoding and analyzing the first screen capture image to obtain the delay time from the display of the desktop image of the remote desktop to the local desktop when the remote equipment runs the target application program; the first time coding image is generated by coding the remote desktop system time through the remote equipment, and the second time coding image is generated by coding the local desktop system time through the local equipment corresponding to the local desktop.
By way of example, computer-readable storage media may be non-transitory computer-readable storage media.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one position, or may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (11)
1. A remote desktop latency detection method, comprising:
in the process of controlling the remote equipment corresponding to the remote desktop to run the target application program, simultaneously capturing a first time coding image and a second time coding image displayed on the local desktop to obtain a first screen capturing image; the first time coding image is obtained by coding the remote desktop system time by the remote equipment and converting a coding result into a graphic form; the second time coding image is obtained by coding the system time of the local desktop by the local equipment corresponding to the local desktop and converting a coding result into a graphic form;
and decoding and analyzing the first screen capture image to obtain the delay time of displaying the desktop image of the remote desktop to the local desktop when the remote equipment runs the target application program.
2. The method for detecting a delay of a remote desktop according to claim 1, wherein the decoding and analyzing the first screen-grabbed image to obtain a delay time for displaying the desktop image of the remote desktop to the local desktop when the remote device runs the target application program includes:
decoding the first screen capture image to acquire first time corresponding to the first time coding image and second time corresponding to the second time coding image in decoding data;
determining the time difference between the first time and the second time to obtain a first time difference;
determining the delay time for displaying a desktop image of the remote desktop to the local desktop when the target application is run by the remote device based on the first time difference and the inherent time difference; wherein the inherent time difference is an inherent system time difference between the remote desktop and the local desktop.
3. The remote desktop latency detection method of claim 2, prior to controlling the remote device to run the target application, the method further comprising:
under the condition that the remote equipment does not run an application program, acquiring a screen capture image for simultaneously capturing the first time coding image and the second time coding image within a first set time period to obtain a second screen capture image;
decoding each screen capture image in the second screen capture images, and determining a time difference between the time corresponding to the first time code image and the time corresponding to the second time code image in each screen capture image according to decoding data to obtain a second time difference of each screen capture image;
and calculating the average value of the second time differences of all screen capture images in the second screen capture image to obtain the inherent time difference.
4. The method for detecting the delay of the remote desktop according to any one of claims 1 to 3, wherein the simultaneously capturing the first time-coded image and the second time-coded image displayed on the local desktop to obtain the first captured image comprises:
responding to a coded image moving instruction, and controlling the first time coded image and the second time coded image to move on the local desktop until the parallel arrangement distance of the first time coded image and the second time coded image is smaller than a set threshold;
and simultaneously performing screen capture on the moved first time coding image and the second time coding image to obtain a first screen capture image.
5. The remote desktop delay detection method of any of claims 1-3, further comprising:
determining the target application program under the condition that an application scene test instruction is detected;
controlling the remote equipment to run the target application program through a remote desktop window; the remote desktop window is displayed on the local desktop.
6. The remote desktop delay detection method of claim 5, wherein controlling the remote device to run the target application through a remote desktop window comprises:
starting a scene simulation program corresponding to the target application program, wherein the scene simulation program is used for simulating the operation of the target application program;
and controlling the running of the target application program in the remote equipment through the remote desktop window based on the scene simulation program.
7. The method according to claim 5, wherein the determining the target application program in case of detecting the application scenario test instruction comprises:
under the condition that an application scene test instruction is detected, displaying an application scene selection interface on the local desktop, wherein the application scene selection interface comprises an application program selection control;
and responding to the selection operation facing the application program selection control, and determining the target application program.
8. A remote desktop latency detection apparatus, comprising:
the screen capture module is used for simultaneously capturing the first time code image and the second time code image displayed on the local desktop to obtain a first screen capture image in the process of controlling the remote equipment corresponding to the remote desktop to run the target application program; the first time coding image is obtained by coding the remote desktop system time by the remote equipment and converting a coding result into a graphic form; the second time coding image is obtained by coding the system time of the local desktop by the local equipment corresponding to the local desktop and converting a coding result into a graphic form;
and the analysis module is used for decoding and analyzing the first screen capture image to obtain the delay time of displaying the desktop image of the remote desktop to the local desktop when the remote equipment runs the target application program.
9. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the remote desktop latency detection method of any one of claims 1 to 7 when executing the computer program.
10. A remote desktop delay detection system is characterized by comprising a local device and a remote device in communication connection with the local device, wherein a local desktop runs on the local device, and a remote desktop runs on the remote device;
the remote equipment is used for coding the remote desktop system time, generating a first time coding image and displaying the first time coding image on the remote desktop;
the local device is configured to perform the steps of the remote desktop latency detection method of any one of claims 1 to 7.
11. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out a remote desktop latency detection method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211534192.5A CN115543714B (en) | 2022-12-02 | 2022-12-02 | Remote desktop delay detection method and device, electronic equipment and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211534192.5A CN115543714B (en) | 2022-12-02 | 2022-12-02 | Remote desktop delay detection method and device, electronic equipment and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115543714A CN115543714A (en) | 2022-12-30 |
| CN115543714B true CN115543714B (en) | 2023-04-07 |
Family
ID=84722393
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211534192.5A Active CN115543714B (en) | 2022-12-02 | 2022-12-02 | Remote desktop delay detection method and device, electronic equipment and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115543714B (en) |
Family Cites Families (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN202535376U (en) * | 2010-02-10 | 2012-11-14 | 北京视博云科技有限公司 | Acquisition encoding device, decoding device and system used in remote access |
| CN102571735A (en) * | 2011-01-04 | 2012-07-11 | 中国移动通信有限公司 | Method, device and system for carrying out application processing by multiple terminals |
| US10404769B2 (en) * | 2013-12-31 | 2019-09-03 | Google Llc | Remote desktop video streaming alpha-channel |
| CN106598295A (en) * | 2016-12-12 | 2017-04-26 | 广东威创视讯科技股份有限公司 | Method and system for improving operation delay feeling of KVM mouse |
| CN111208960B (en) * | 2019-12-26 | 2023-05-09 | 杭州顺网科技股份有限公司 | Remote display delay reduction method based on frame extraction control and time synchronization algorithm |
| CN112929624B (en) * | 2021-01-21 | 2023-02-17 | 杭州雾联科技有限公司 | Encoding method, encoding device, electronic equipment and computer readable storage medium |
| CN112929741B (en) * | 2021-01-21 | 2023-02-03 | 杭州雾联科技有限公司 | Video frame rendering method and device, electronic equipment and storage medium |
| CN113741744A (en) * | 2021-02-04 | 2021-12-03 | 上海达龙信息科技有限公司 | Cursor display method, device, medium and electronic equipment based on remote desktop |
| CN113849073A (en) * | 2021-08-25 | 2021-12-28 | 中国船舶重工集团公司第七0九研究所 | Remote control-oriented mouse and returned picture synchronization method and system |
| CN115248714A (en) * | 2022-06-27 | 2022-10-28 | 北京和信创天科技股份有限公司 | Method for improving VDI remote desktop performance by using VGPU |
-
2022
- 2022-12-02 CN CN202211534192.5A patent/CN115543714B/en active Active
Non-Patent Citations (1)
| Title |
|---|
| 一种基于分块采集和压缩技术的屏幕共享方案;左强翔;吴洁;;计算机技术与发展(第04期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115543714A (en) | 2022-12-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP6898968B2 (en) | Methods and devices for determining response time | |
| TWI622877B (en) | System and method for recording and replaying mouse behavior, recording media and program products | |
| CN111436005B (en) | Method and apparatus for displaying image | |
| CN110166842B (en) | Video file operation method and device and storage medium | |
| CN111858318B (en) | Response time testing method, device, equipment and computer storage medium | |
| CN113923441A (en) | Video quality evaluation method and device and electronic equipment | |
| CN113452944B (en) | Picture display method of cloud mobile phone | |
| KR102224785B1 (en) | Method And Apparatus for Providing Coding Education Service | |
| KR20160032028A (en) | Execution and display of events in web browsers | |
| CN109271929B (en) | Detection method and device | |
| CN112511818A (en) | Video playing quality detection method and device | |
| CN115396705A (en) | Screen projection operation verification method, platform and system | |
| CN110248235B (en) | Software teaching method, device, terminal equipment and medium | |
| CN110673886B (en) | Method and device for generating thermodynamic diagrams | |
| CN115543714B (en) | Remote desktop delay detection method and device, electronic equipment and system | |
| CN113485605A (en) | Display device, terminal device and screen projection method | |
| CN109871465B (en) | Time axis calculation method and device, electronic equipment and storage medium | |
| KR102106618B1 (en) | System, terminal device, verification apparatus, method and computer-readable medium for testing application performance | |
| CN113496017A (en) | Verification method, device, equipment and storage medium | |
| CN114071190B (en) | Cloud application video stream processing method, related device and computer program product | |
| CN110245068A (en) | Automated testing method, device and the computer equipment of the H5 page | |
| CN114035719B (en) | Remote desktop fluency performance evaluation method, system and medium | |
| CN115022204B (en) | RTC transmission delay detection method, device and equipment | |
| CN111158822A (en) | Display interface control method and device, storage medium and electronic equipment | |
| CN113724225B (en) | Method and device for determining transmission quality of application program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |