CN112685109A - Method and system for dynamically identifying and recognizing remote application window - Google Patents
Method and system for dynamically identifying and recognizing remote application window Download PDFInfo
- Publication number
- CN112685109A CN112685109A CN202011409367.0A CN202011409367A CN112685109A CN 112685109 A CN112685109 A CN 112685109A CN 202011409367 A CN202011409367 A CN 202011409367A CN 112685109 A CN112685109 A CN 112685109A
- Authority
- CN
- China
- Prior art keywords
- window
- cloud application
- application
- cloud
- client
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Abstract
The invention discloses a method and a system for dynamically identifying and recognizing a remote application window. And adding a sub-window with an identity mark to each opened cloud application window in the cloud server, and carrying the application program original data information of the cloud application window on a title bar of the sub-window, so that a complete and accurate original data structure can be transmitted to the client. The original data information of the cloud application window is transmitted by using a cross-process child window technology, so that a client can know which applications each window belongs to, and a user can be supported to prevent repeated opening of an instant messaging program, confirm which applications are running from a client logout window, and count and report additional functions such as which cloud applications the client uses, and the like.
Description
Technical Field
The invention belongs to the technical field of cloud application, and particularly relates to a method for dynamically identifying and recognizing a remote application window.
Background
With the change of cloud computing and cloud desktop technologies, the use scenes of people are further developed from cloud on office desktops to cloud on applications. Through deployment of the cloud application, a user can use existing computer software like a WeChat applet, various redundant software does not need to be installed on the computer of the user, and the traditional Office, 3D design and even software application developed by the user can be opened through direct double-click of an application software store, so that various problems of insufficient space of a personal computer, system compatibility of software, trouble in operation and maintenance of computer failure, too few software licenses, document confidentiality and the like are avoided, and the most obvious advantage brought by the cloud application is achieved.
The cloud application is different from a desktop in that application programs needed by daily work are uniformly deployed in a cloud platform, a local computer where a user is located only needs to install a unique client software package, the application programs needed by the work can be browsed and searched in an application store interface of the cloud platform after being opened and logged in, then the user can directly click and open the application programs for use, downloading and installation on the local computer are not needed, and the experience of using the application programs on the local computer is not different.
The difference between the process of using the cloud application by the user and the cloud desktop is that only the program window running at the cloud end is seen, and the program window is displayed on the local computer desktop of the user just like the running of software installed on a local computer. In fact, the opening, rendering and interaction processes of the cloud application window adopt a window management mechanism of a Windows operating system kernel, when an application program on a cloud host desktop is opened, attribute parameter information of the window object is transmitted to a client through a network channel by capturing a series of events in the life cycle of the window object, and the client constructs the same window object locally by using the data and renders the same window object on the client desktop.
When a user performs window interaction on a cloud application on a local computer, an interaction behavior is sent to the cloud for processing, and a result is fed back, for example: mouse events, keyboard events, touch screen events, other peripheral events, and the like. The window interaction behaviors of the client are sent to the cloud end as required, the appearance, the position and the size of the window are triggered to change or destroy, the process experience is the same as that of the corresponding window on the local computer, and the cloud end and the local are seamlessly integrated.
In the whole process of window interaction of a user on a local computer for cloud applications, the user sees that the user runs a plurality of cloud applications on the local computer, but the application windows are all windows played by the same cloud application client software, and a task manager of the local computer cannot find a process corresponding to the cloud applications. This can be troublesome for some scenarios. For example, a user may want to see what application processes are behind these cloud application windows, and although they can be identified by the naked eye, they cannot be identified by programming means, and they cannot obtain application information accurately only by using information such as window type, window title, window size, window position coordinates, window appearance bitmap, etc.
Disclosure of Invention
The invention aims to provide a method for dynamically identifying and recognizing a remote application window, which solves the technical problems that in the existing cloud application technology, a user cannot check the processes of corresponding cloud applications in a task manager of a local computer and cannot identify what application program process is behind the cloud application windows through a programming means.
In order to solve the technical problems, the invention adopts the technical scheme that:
a method for dynamically identifying and recognizing remote application windows is characterized in that a sub-window with an identity mark is added to each opened cloud application window in a cloud server, and original data information of an application program of the cloud application window is carried on a title bar of the sub-window.
Since the cloud application relates to countless business software for office, teaching and design, and the software itself cannot modify source codes to transmit more information in a cloud application scene, a cloud application daemon agent needs to be added, a hidden and fully transparent child window with an identity is added to each cloud application window, application metadata information of the cloud application window, such as original information of a program name, a process ID and the like, is carried on a title bar of the child window, and thus a complete and accurate original data structure can be transmitted to a client. The original data information of the cloud application window is transmitted by using a cross-process child window technology, so that a client can know which applications each window belongs to, and a user can be supported to prevent repeated opening of an instant messaging program, confirm which applications are running from a client logout window, and count and report additional functions such as which cloud applications the client uses, and the like.
The method for further optimizing, dynamically identifying and recognizing the remote application window comprises the following specific steps:
1) designing a cloud application daemon agent in the cloud server, wherein the agent always runs in the context of each session between a client and the cloud server through a cloud application client;
2) when the agent program detects that the cloud application is opened, a corresponding identification window is established as a child window, the original information of the cloud application is recorded in a title character string of the identification window, and a parent window of the child window is set as a corresponding cloud application window;
3) because the operating system supports the cross-process parent-child window relationship setting behavior, the parent-child relationship information is transmitted to the cloud application client, and a user can analyze the application program background information of the parent window from the character string information recorded in the title of the identification window by detecting the parent window handle of the identification window at the cloud application client, so that all original data of the cloud application are obtained.
Further optimization, in the step 2), the cloud application window monitoring module in the agent program is started along with the login of the user on the cloud application client and is operated all the time. The method comprises the steps that a monitoring module monitors that a certain cloud application is started through a mechanism of capturing window events, a new cloud application window is opened, information of the cloud application window is obtained according to a cloud application window handle, the information is spliced into character strings in a json format and packaged into management metadata, and the character strings are used as title bars to create a corresponding identification window; the size and the position of the identification window can be consistent with those of the corresponding cloud application window and can be adjusted along with the adjustment of the corresponding cloud application window.
Further optimization, in the step 3), when the cloud application client detects that a new window is opened, detecting whether the title bar format of the window is a legal json character string, and if so, analyzing the content of the character string; when the analyzed information conforms to the content of the cloud application identification window, setting the information as an identification window; and then obtaining all the identification information of the cloud application window by obtaining the corresponding father window handle.
Further optimized, the application raw data information of the application window includes, but is not limited to, an application process ID, a path of an application executable file, an application package name, and an application version number.
Further optimization, the creation process of the identification window is completely dynamic, and the identification window is automatically created along with the conventional operations of opening, moving, closing and the like of a new cloud application window in the using process of a user. The appearance of the identification window is a hidden watermark or an explicit watermark.
A system for dynamically identifying and recognizing remote application windows, comprising:
the device terminal is used for accessing the cloud application server through the cloud application client and displaying a cloud application window;
the cloud application client is used for receiving a cloud application request instruction of a user and transmitting the cloud application request instruction to the cloud application server; then receiving the opened cloud application window;
the cloud application server is used for receiving a cloud application request instruction sent by the cloud application client, opening the cloud application, and returning the opened cloud application window to the cloud application client;
the cloud application window monitoring module is used for monitoring the starting of the cloud application and creating a corresponding identification window;
and the cloud application client detection module is used for detecting whether the newly opened window is an identification window or not and then obtaining all identification information of the corresponding parent window.
The technical scheme of the invention has the beneficial effects that:
1. according to the method and the device, the hidden fully transparent sub-window with the identity is added to each cloud application window, and the application program metadata information of the cloud application window, such as program name, process ID and other original information, is carried on the title bar of the sub-window, so that a complete and accurate original data structure can be transmitted to the client.
2. According to the method and the device, on the basis of the original data transmission of the cloud application window image, the identification and recognition technology of the application window is enhanced, so that the client can know which applications each window belongs to, the user can be supported to prevent the instant messaging program from being repeatedly opened, the applications are confirmed to be running from the client logout window, and the additional functions of reporting which cloud applications are used by the client and the like are counted.
Drawings
FIG. 1 is a schematic diagram of a system for dynamically identifying and recognizing remote application windows in accordance with the present invention;
FIG. 2 is a block diagram of dynamically identifying and recognizing remote application windows;
FIG. 3 is a flow chart of a method of dynamically identifying and recognizing remote application windows;
FIG. 4 is a flow diagram of an adjustment of an identification window with an adjustment of a corresponding cloud application window;
fig. 5 is a schematic diagram of opening a plurality of cloud applications and establishing a plurality of corresponding identification windows.
Detailed Description
The technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.
As shown in fig. 1 to 5, a method for dynamically identifying and recognizing a remote application window takes a system for dynamically identifying and recognizing a remote application window as an execution object, and as shown in fig. 1, the system includes an apparatus terminal 1, a cloud application client 2, a cloud application client detection module 21, a cloud application server 3, and a cloud application window monitoring module 31. The device terminal is used for accessing the cloud application server through the cloud application client and displaying a cloud application window; the cloud application client is used for receiving a cloud application request instruction of a user and transmitting the cloud application request instruction to the cloud application server; then receiving the opened cloud application window; the cloud application server is used for receiving a cloud application request instruction sent by the cloud application client, opening the cloud application, and returning the opened cloud application window to the cloud application client; the cloud application window monitoring module is used for monitoring the starting of the cloud application and creating a corresponding identification window; and the client detection module is used for detecting whether the newly opened window is the identification window or not and then obtaining all the identification information of the corresponding parent window.
As shown in fig. 2, in the system, a window event signal capture interface is supported on the bottom layer of a Windows operating system, a new window object can be known to be generated by capturing a window creating and displaying event on a cloud server, information such as a process to which a window belongs, a corresponding cloud application and the like is extracted by using an object handle, and then the information is encapsulated into management metadata and transmitted to a client, so that the cloud application client can use the management metadata to identify the window identity, and more enhanced management behaviors can be realized due to the abundant window identity information.
Specifically, the method for dynamically identifying and recognizing the remote application window, as shown in fig. 3, includes the following steps:
step 1: a cloud application daemon agent program is designed in a cloud server, and a cloud application window monitoring module in the agent program is started along with the fact that a user logs in a cloud application client and always runs in the context of each conversation between the client and the cloud server through the cloud application client.
Step 2: the monitoring module monitors that the cloud application A is started through a mechanism of capturing window events, opens a new cloud application window, acquires information of the cloud application window according to a cloud application window handle, then splices the information into character strings in a json format, packages the character strings into management metadata, and creates a corresponding identification window A by using the character strings as a title bar. The size and position of the identification window and the corresponding cloud application window can be kept consistent, and the size, position and level of the identification window can be adjusted along with the movement, size adjustment, maximization, minimization and level change of the cloud application window, so that the identification window can completely follow the cloud application window, as shown in fig. 4.
And 3, because the operating system supports the setting behavior of the parent-child window relationship of the cross-process, the parent-child relationship information is transmitted to the cloud application client. When the cloud application client detection module detects that a new window is opened, detecting whether the title bar format of the window is a legal json character string, and if so, analyzing the character string content; when the analyzed information conforms to the content of the cloud application identification window, setting the information as an identification window; and then obtaining all the identification information of the cloud application window by obtaining the corresponding father window handle.
The process is completely dynamic, and the new cloud application window is automatically created along with the conventional operations of opening, moving, closing and the like of the new cloud application window in the using process of a user.
In this embodiment, the application raw data information of the application window includes, but is not limited to, an application process ID, a path of an application executable file, an application package name, and an application version number.
In this embodiment, the identification window is a hidden fully transparent sub-window
In other embodiments, the appearance of the identification window may be an explicit watermark.
In other embodiments, when a user opens multiple cloud applications, for example, when a cloud application window monitoring module in a proxy detects that a cloud application a, a cloud application B, and a cloud application C are started, corresponding identification windows a, B, and C are respectively established in sequence according to the start time and are transmitted to a cloud application client, and then the user can obtain all identification information of the three cloud application windows, as shown in fig. 5.
The above examples are merely illustrative for clarity and are not intended to limit the embodiments. Other variations and modifications will be apparent to persons skilled in the art in light of the above description. And are neither required nor exhaustive of all embodiments. And obvious variations or modifications therefrom are within the scope of the invention.
Claims (7)
1. A method for dynamically identifying and recognizing remote application windows is characterized in that a sub-window with an identity mark is added to each opened cloud application window in a cloud server, and original data information of an application program of the cloud application window is carried on a title bar of the sub-window.
2. The method for dynamically identifying and recognizing remote application windows according to claim 1, comprising the following steps:
1) designing a cloud application daemon agent program module in the cloud server, wherein the agent program always runs in the context of each session between a client and the cloud server through a cloud application client;
2) when the agent program detects that the cloud application is opened, a corresponding identification window is established as a child window, the original information of the cloud application is recorded in a title character string of the identification window, and a parent window of the child window is set as a corresponding cloud application window;
3) and the user can analyze the application program background information of the father window from the character string information recorded in the title of the identification window by detecting the father window handle of the identification window at the cloud application client, so that all original data of the cloud application are obtained.
3. The method for dynamically identifying and recognizing a remote application window according to claim 2, wherein in the step 2), a cloud application window monitoring module in an agent program is started and operated all the time as a user logs in a cloud application client, the monitoring module monitors that a certain cloud application is started and opens a new cloud application window through a mechanism for capturing window events, then original data information of the cloud application window is obtained according to a cloud application window handle, the information is spliced into a character string in a json format and packaged into management metadata, and the character string is used as a title bar to create a corresponding identification window; the size and the position of the identification window can be consistent with those of the corresponding cloud application window and can be adjusted along with the adjustment of the corresponding cloud application window.
4. The method for dynamically identifying and recognizing a remote application window according to claim 4, wherein in the step 3), when a cloud application client detects that a new window is opened, it detects whether a title bar format of the window is a legal json character string, and if so, analyzes the character string content; when the analyzed information conforms to the content of the cloud application identification window, setting the information as an identification window; and then obtaining all the identification information of the cloud application window by obtaining the corresponding father window handle.
5. The method for dynamically identifying and tracing a remote application window according to any of claims 1-4, wherein the application window's application raw data information includes but is not limited to application process ID, path of application executable file, application package name, and application version number.
6. The method for dynamically identifying and recognizing a remote application window according to claim 5, wherein the appearance of said sub-window is a hidden watermark or an explicit watermark.
7. A system for dynamically identifying and recognizing remote application windows, comprising:
the device terminal is used for accessing the cloud application server through the cloud application client and displaying a cloud application window;
the cloud application client is used for receiving a cloud application request instruction of a user and transmitting the cloud application request instruction to the cloud application server; then receiving the opened cloud application window;
the cloud application server is used for receiving a cloud application request instruction sent by the cloud application client, opening the cloud application, and returning the opened cloud application window to the cloud application client;
the cloud application window monitoring module is used for monitoring the starting of the cloud application and creating a corresponding identification window;
and the cloud application client detection module is used for detecting whether the newly opened window is an identification window or not and then obtaining all identification information of the corresponding parent window.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011409367.0A CN112685109B (en) | 2020-12-03 | 2020-12-03 | Method and system for dynamically identifying and recognizing remote application window |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011409367.0A CN112685109B (en) | 2020-12-03 | 2020-12-03 | Method and system for dynamically identifying and recognizing remote application window |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112685109A true CN112685109A (en) | 2021-04-20 |
| CN112685109B CN112685109B (en) | 2021-09-21 |
Family
ID=75447361
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011409367.0A Active CN112685109B (en) | 2020-12-03 | 2020-12-03 | Method and system for dynamically identifying and recognizing remote application window |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112685109B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113742026A (en) * | 2021-11-04 | 2021-12-03 | 江苏航天七零六信息科技有限公司 | Method for realizing cloud application exclusive window watermark through grid matrix drawing mechanism |
| CN114327712A (en) * | 2021-12-24 | 2022-04-12 | 阿里巴巴（中国）有限公司 | Cloud application system, window management method and device |
| CN114385046A (en) * | 2022-01-07 | 2022-04-22 | 阿里巴巴(中国)网络技术有限公司 | Method and device for realizing end-side window movement |
| CN114489459A (en) * | 2022-02-16 | 2022-05-13 | 北京蔚领时代科技有限公司 | Cloud input method and device |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2006039222A2 (en) * | 2004-09-30 | 2006-04-13 | Citrix Systems, Inc. | Method and apparatus for virtualizing window information |
| US20070260762A1 (en) * | 2006-02-13 | 2007-11-08 | Hideki Miyamoto | External apparatus function extension application system and firmware download driver system |
| US20110154219A1 (en) * | 2009-12-18 | 2011-06-23 | Livelook, Inc. | Co-browsing Systems and Methods |
| CN102436350A (en) * | 2011-12-06 | 2012-05-02 | 北京航空航天大学 | Remote application window display method, server and user terminal |
| CN102662467A (en) * | 2012-03-28 | 2012-09-12 | 北京奇虎科技有限公司 | Method and device for instruction control in remote assistance process |
| CN103503412A (en) * | 2012-03-23 | 2014-01-08 | 华为技术有限公司 | Method and device for scheduling resources |
| CN104468741A (en) * | 2014-11-21 | 2015-03-25 | 中兴通讯股份有限公司 | Cloud desktop processing method and device |
| CN105589619A (en) * | 2014-11-17 | 2016-05-18 | 深圳市腾讯计算机系统有限公司 | Multi-window display method and device for terminal |
| CN107239679A (en) * | 2017-04-28 | 2017-10-10 | 浙江华途信息安全技术股份有限公司 | program window information protecting method and system |
| CN107332824A (en) * | 2017-06-07 | 2017-11-07 | 北京奇安信科技有限公司 | A kind of recognition methods of cloud application and device |
| CN108108594A (en) * | 2017-12-15 | 2018-06-01 | 北京月新时代科技股份有限公司 | A kind of license management method and device |
-
2020
- 2020-12-03 CN CN202011409367.0A patent/CN112685109B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2006039222A2 (en) * | 2004-09-30 | 2006-04-13 | Citrix Systems, Inc. | Method and apparatus for virtualizing window information |
| US20070260762A1 (en) * | 2006-02-13 | 2007-11-08 | Hideki Miyamoto | External apparatus function extension application system and firmware download driver system |
| US20110154219A1 (en) * | 2009-12-18 | 2011-06-23 | Livelook, Inc. | Co-browsing Systems and Methods |
| CN102436350A (en) * | 2011-12-06 | 2012-05-02 | 北京航空航天大学 | Remote application window display method, server and user terminal |
| CN103503412A (en) * | 2012-03-23 | 2014-01-08 | 华为技术有限公司 | Method and device for scheduling resources |
| CN102662467A (en) * | 2012-03-28 | 2012-09-12 | 北京奇虎科技有限公司 | Method and device for instruction control in remote assistance process |
| CN105589619A (en) * | 2014-11-17 | 2016-05-18 | 深圳市腾讯计算机系统有限公司 | Multi-window display method and device for terminal |
| CN104468741A (en) * | 2014-11-21 | 2015-03-25 | 中兴通讯股份有限公司 | Cloud desktop processing method and device |
| CN107239679A (en) * | 2017-04-28 | 2017-10-10 | 浙江华途信息安全技术股份有限公司 | program window information protecting method and system |
| CN107332824A (en) * | 2017-06-07 | 2017-11-07 | 北京奇安信科技有限公司 | A kind of recognition methods of cloud application and device |
| CN108108594A (en) * | 2017-12-15 | 2018-06-01 | 北京月新时代科技股份有限公司 | A kind of license management method and device |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113742026A (en) * | 2021-11-04 | 2021-12-03 | 江苏航天七零六信息科技有限公司 | Method for realizing cloud application exclusive window watermark through grid matrix drawing mechanism |
| CN113742026B (en) * | 2021-11-04 | 2022-02-11 | 江苏航天七零六信息科技有限公司 | Method for realizing cloud application exclusive window watermark through grid matrix drawing mechanism |
| CN114327712A (en) * | 2021-12-24 | 2022-04-12 | 阿里巴巴（中国）有限公司 | Cloud application system, window management method and device |
| WO2023116184A1 (en) * | 2021-12-24 | 2023-06-29 | 阿里巴巴（中国）有限公司 | Cloud application system, window management method, and device |
| CN114385046A (en) * | 2022-01-07 | 2022-04-22 | 阿里巴巴(中国)网络技术有限公司 | Method and device for realizing end-side window movement |
| CN114489459A (en) * | 2022-02-16 | 2022-05-13 | 北京蔚领时代科技有限公司 | Cloud input method and device |
| CN114489459B (en) * | 2022-02-16 | 2023-11-03 | 北京蔚领时代科技有限公司 | Cloud input method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112685109B (en) | 2021-09-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112685109B (en) | Method and system for dynamically identifying and recognizing remote application window | |
| US20210318866A1 (en) | Auto-generation of api documentation via implementation-neutral analysis of api traffic | |
| EP3335131B1 (en) | Systems and methods for automatic content verification | |
| US11811805B1 (en) | Detecting fraud by correlating user behavior biometrics with other data sources | |
| US8332765B2 (en) | Problem reporting system based on user interface interactions | |
| US7441195B2 (en) | Associating website clicks with links on a web page | |
| WO2015179286A1 (en) | Polymorphic treatment of data entered at clients | |
| US10372958B2 (en) | In-field data acquisition and formatting | |
| CN101751535A (en) | Data loss protection through application data access classification | |
| US11601453B2 (en) | Methods and systems for establishing semantic equivalence in access sequences using sentence embeddings | |
| US8671389B1 (en) | Web application resource manager on the web and localizable components | |
| CN110381026B (en) | Business service packaging and accessing system, method and device for rich client | |
| WO2021129335A1 (en) | Operation monitoring method and apparatus, operation analysis method and apparatus | |
| US20200225927A1 (en) | Methods and systems for automating computer application tasks using application guides, markups and computer vision | |
| CN111796809A (en) | Interface document generation method and device, electronic equipment and medium | |
| CN105637488A (en) | Tracing source code for end user monitoring | |
| CN111651228B (en) | Method and device for optimizing user interface layout and related equipment | |
| CN113704590A (en) | Webpage data acquisition method and device, electronic equipment and storage medium | |
| US20090113017A1 (en) | Arrangements for A 3270 compatible terminal with browser capability | |
| CN116483487A (en) | Robot process automation robot design interface based on browser | |
| CN113515715B (en) | Buried point event code generation method, buried point event code processing method and related equipment | |
| CN113742501A (en) | Information extraction method, device, equipment and medium | |
| US20230394030A1 (en) | Generating event logs from video streams | |
| CN116719986B (en) | Python-based data grabbing method, device, equipment and storage medium | |
| CN115587364B (en) | Firmware vulnerability input point positioning method and device based on front-end and back-end correlation analysis |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |