CN111190564A

CN111190564A - Screen interaction method and system

Info

Publication number: CN111190564A
Application number: CN202010035935.9A
Authority: CN
Inventors: 周清会; 柳荣; 魏贤杰
Original assignee: Shanghai Manheng Digital Technology Co ltd
Current assignee: Shanghai Manheng Digital Technology Co ltd
Priority date: 2020-01-14
Filing date: 2020-01-14
Publication date: 2020-05-22
Anticipated expiration: 2040-01-14
Also published as: CN111190564B

Abstract

The application relates to a screen interaction method and a screen interaction system, and relates to the technical field of internet. The method comprises the following steps: recording the client information into a server queue; a request server queue, which is used for inputting client information into the system through a control terminal; when the client is confirmed to enter the system, screen information is sent to the server through the client; and requesting the server to acquire the IP list of the client, and loading the screen information of the client through the control terminal. According to the screen interaction method and system, the HTTP service is started to be accessed by other terminal equipment, the CMD command is used for naming the firewall rule newly and opening the port, the terminal equipment can be accessed by other terminal equipment in the local area network, the screen of the same terminal equipment can be monitored through multiple screens, the problems of screen sharing delay, blockage, white screen and the like are solved, and user experience is improved comprehensively.

Description

Screen interaction method and system

Technical Field

The application relates to the technical field of internet, in particular to a screen interaction method and system.

Background

With the development of internet application, remote control or sharing realized by a network is used as a new interaction mode, so that different users can interact with each other. The screen interaction is one of important applications of a data sharing technology, and is widely applied to the fields of document display or color operation, cooperative office work or teaching and the like. The screen interaction technology is based on a Graphical User Interface (GUI), and may include various interaction forms, such as monitoring screens of other terminals through one terminal device, sharing a screen of one terminal device through other terminals, and sharing a screen between different types of terminals.

In the Windows self-contained remote monitoring scenario, the monitored screen is closed to the login interface. The monitored end can also operate while the current remote monitoring software realizes monitoring, but only one terminal device is usually monitored. In a scene of monitoring a screen by using a Winform plug-in, a white screen phenomenon occurs when the number of terminal devices is increased, and the screen is easy to jam due to long waiting time.

Therefore, it is desirable to provide a screen interaction method and system, which can be accessed by other terminal devices by starting an HTTP service, utilizing a CMD command to newly name a firewall rule and starting a port, and can be accessed by other terminal devices in a local area network, thereby implementing multi-screen monitoring of the screen of the same terminal device, solving the problems of screen sharing delay, stuck, white screen, and the like, and comprehensively improving user experience.

Disclosure of Invention

According to a first aspect of some embodiments of the present application, there is provided a screen interaction method applied in a terminal (e.g., an electronic device, etc.), the method may include: recording the client information into a server queue; a request server queue, which is used for inputting client information into the system through a control terminal; when the client is confirmed to enter the system, screen information is sent to the server through the client; and requesting the server to acquire the IP list of the client, and loading the screen information of the client through the control terminal.

In some embodiments, the method may further comprise: and executing client monitoring through the control end.

In some embodiments, the method may further comprise: and executing screen sharing of the client to other clients through the control terminal.

In some embodiments, the method may further comprise: selecting a client to be monitored through a control end according to the IP list and sending UDP information; saving screen information and starting HTTP service through the client according to the UDP information; and executing the CMD command through the client to open the accessible port and return the service address to the control end.

In some embodiments, the opening the HTTP service further comprises: according to the UDP information, saving the picture stream of the screen information to the local through the client at a frame rate of 25 frames per second; when the HTTP service is started, the HTTP service points to the local map stream and can be accessed by the client side in the local area network.

In some embodiments, the performing, by the control end, client monitoring further includes: and opening a window through the control terminal according to the service address so as to monitor the screen of the client.

In some embodiments, the performing, by the control terminal, screen sharing of the client to the other clients further includes: sending the service address of the client to other clients through a control end by using UDP (user Datagram protocol); and opening windows through other clients according to the service address to share the client screen.

In some embodiments, said entering client information into a server queue further comprises: sending information to a server through a client according to an HTTP (hyper text transport protocol); and recording the client information into a Redis queue maintained by the server.

In some embodiments, the validating client entry system further comprises: the client sends request information to the system, wherein the request information comprises a request for the system to feed back whether the client enters the system or not; and when the system feeds back that the client enters the system, confirming that the client enters the system.

According to a second aspect of some embodiments of the present application, there is provided a system comprising: a memory configured to store data and instructions; a processor in communication with the memory, wherein the processor, when executing instructions in the memory, is configured to: recording the client information into a server queue; a request server queue, which is used for inputting client information into the system through a control terminal; when the client is confirmed to enter the system, screen information is sent to the server through the client; and requesting the server to acquire the IP list of the client, and loading the screen information of the client through the control terminal.

Therefore, according to the screen interaction method and system provided by some embodiments of the application, the HTTP service is started for other terminal devices to access, the CMD command is used to newly name the firewall rule and start the port, and the terminal devices can access, so that the terminal devices in the local area network can monitor the screen of the same terminal device through multiple screens, the problems of screen sharing delay, screen jamming, white screen and the like are solved, and the user experience is comprehensively improved.

Drawings

For a better understanding and appreciation of some embodiments of the present application, reference will now be made to the description of embodiments taken in conjunction with the accompanying drawings, in which like reference numerals designate corresponding parts in the figures.

FIG. 1 is an exemplary schematic diagram of a screen interaction system provided in accordance with some embodiments of the present application.

FIG. 2 is an exemplary flow diagram of a screen interaction method provided in accordance with some embodiments of the present application.

FIG. 3 is an exemplary flow chart of a particular screen interaction method provided in accordance with some embodiments of the present application.

FIG. 4 is a client-side monitoring exemplary flow diagram of a screen interaction method provided in accordance with some embodiments of the present application.

FIG. 5 is a screen sharing exemplary flow diagram of a screen interaction method provided according to some embodiments of the present application.

Detailed Description

The following description, with reference to the accompanying drawings, is provided to facilitate a comprehensive understanding of various embodiments of the application as defined by the claims and their equivalents. These embodiments include various specific details for ease of understanding, but these are to be considered exemplary only. Accordingly, those skilled in the art will appreciate that various changes and modifications may be made to the various embodiments described herein without departing from the scope and spirit of the present application. In addition, descriptions of well-known functions and constructions will be omitted herein for brevity and clarity.

The terms and phrases used in the following specification and claims are not to be limited to the literal meaning, but are merely for the clear and consistent understanding of the application. Accordingly, it will be appreciated by those skilled in the art that the description of the various embodiments of the present application is provided for illustration only and not for the purpose of limiting the application as defined by the appended claims and their equivalents.

The technical solutions in the embodiments of the present application will be described clearly and completely with reference to the accompanying drawings in some embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

It is to be understood that the terminology used in the embodiments of the present application is for the purpose of describing particular embodiments only, and is not intended to be limiting of the application. As used in the examples of this application and the appended claims, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items. The expressions "first", "second", "the first" and "the second" are used for modifying the corresponding elements without regard to order or importance, and are used only for distinguishing one element from another element without limiting the corresponding elements.

A terminal according to some embodiments of the present application may be an electronic device, which may include one or a combination of several of a personal computer (PC, e.g., a tablet, a desktop, a notebook, a netbook, a PDA), a client device, a virtual reality device (VR), a renderer, a smartphone, a mobile phone, an e-book reader, a Portable Multimedia Player (PMP), an audio/video player (MP 3/MP 4), a camera, a wearable device, and the like. According to some embodiments of the present application, the wearable device may include an accessory type (e.g., watch, ring, bracelet, glasses, or Head Mounted Device (HMD)), an integrated type (e.g., electronic garment), a decorative type (e.g., skin pad, tattoo, or built-in electronic device), and the like, or a combination of several. In some embodiments of the present application, the electronic device may be flexible, not limited to the above devices, or may be a combination of one or more of the above devices. In this application, the term "user" may indicate a person using an electronic device or a device using an electronic device (e.g., an artificial intelligence electronic device).

The embodiment of the application provides a screen interaction method and a screen interaction system. In order to facilitate understanding of the embodiments of the present application, the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

FIG. 1 is an exemplary schematic diagram of a screen interaction system provided in accordance with some embodiments of the present application. As shown in fig. 1, the screen interaction system 100 may include a network 110, a control end 120, a client 130, a server 140, and the like. Specifically, the control end 120 and the client 130 establish communication through a network, for example, the control end 120 and the client 130 may communicate in the same local area network (e.g., a network environment of the same router, etc.). Further, the control end 120 may be connected to the network 110 in a wired (e.g., network cable, etc.) or wireless manner, and the client 130 may establish a communication connection with the network 110 in a wired or wireless (e.g., WIFI, etc.) manner. In some embodiments, the client 130 may send request information to the control end 120, the server 140, and the like. Further, the control end 120 and the server 140 may feed back information to the client 130. Based on the feedback information, the client 130 may determine to enter the system and send the information to the server 140. In some embodiments, the control end 120 may send request information to the server 140 to obtain information sent by the client 130, and the like.

According to some embodiments of the present application, the control end 120 and the client 130 may be the same or different terminal devices, and the like. The terminal device may include, but is not limited to, a smart terminal, a mobile terminal, a computer, a rendering machine, and the like. In the teaching scenario, the control end 120 may comprise a teacher end, and the client 130 may comprise a student end, etc. In a conference scenario, the control peer 120 may comprise a host peer, the client 130 may comprise a report peer, and so on. In some embodiments, server 140 is one type of computer that has the advantages of running faster, being more heavily loaded, etc. than a normal computer, and the corresponding price is higher. In a network environment, a server may provide computing or application services to other clients (e.g., terminals such as PCs, smart phones, ATMs, and large devices such as transportation systems). The server has high-speed CPU computing capability, long-time reliable operation, strong I/O external data throughput capability and better expansibility. The services that the server may provide include, but are not limited to, the ability to undertake responding to service requests, undertake services, secure services, and the like. The server, as an electronic device, has an extremely complex internal structure, including an internal structure similar to that of a general computer, and the like, and the internal structure of the server may include a Central Processing Unit (CPU), a hard disk, a memory, a system bus, and the like, as an example.

In some embodiments of the present application, the screen interaction system 100 may omit one or more elements, or may further include one or more other elements. By way of example, the screen interaction system 100 may include a plurality of clients 130, such as a plurality of client kiosks or the like. As another example, the screen interaction system 100 may include a plurality of control terminals 120. As another example, the screen interaction system 100 may include a plurality of servers 140, and the like. The Network 110 may be any type of communication Network, which may include a computer Network (e.g., a Local Area Network (LAN) or Wide Area Network (WAN)), the internet and/or a telephone Network, etc., or a combination of several. In some embodiments, the network 110 may be other types of wireless communication networks. The wireless communication may include microwave communication and/or satellite communication, among others. The Wireless communication may include cellular communication, such as Global System for Mobile communications (GSM), Code Division Multiple Access (CDMA), third Generation Mobile communication (3G, The 3rd Generation communication), fourth Generation Mobile communication (4G), fifth Generation Mobile communication (5G), sixth Generation Mobile communication (6G), Long Term Evolution (LTE-a), Wideband Code Division Multiple Access (WCDMA, Wideband Code Division Multiple Access), Universal Mobile Telecommunications System (UMTS), Wireless Broadband (Broadband ), and The like, or a combination of several or more. In some embodiments, the client 130 may be other electronic devices with equivalent functional modules, and the electronic devices may include one or a combination of several of a virtual reality device (VR), a rendering machine, a personal computer (PC, such as a tablet computer, a desktop computer, a notebook, a netbook, a PDA), a smart phone, a mobile phone, an e-book reader, a Portable Multimedia Player (PMP), an audio/video player (MP 3/MP 4), a camera, and a wearable device.

In some embodiments, the WIFI may be other types of wireless communication technologies. According to some embodiments of the present application, the Wireless communication may include Wireless local Area Network (WiFi), Bluetooth Low Energy (BLE), ZigBee (ZigBee), Near Field Communication (NFC), magnetic security transmission, radio frequency and Body Area Network (BAN), or the like, or a combination of several. According to some embodiments of the present application, the wired communication may include a Global Navigation Satellite System (Global Navigation Satellite System), a Global Positioning System (GPS), a beidou Navigation Satellite System, galileo (european Global Satellite Navigation System), or the like. The wired communication may include a Universal Serial Bus (USB), a High-Definition multimedia interface (HDMI), a recommended Standard 232 (RS-232, recommended Standard 232), and/or Plain Old Telephone Service (POTS), or the like, or a combination of several.

It should be noted that the above description of the screen interaction system 100 is merely for convenience of description and is not intended to limit the scope of the present application. It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the principles of the system, which may be combined in any manner or combined with other elements to form a subsystem for use in a field of application in which the method and system described above is practiced. Such variations are within the scope of the present application.

FIG. 2 is an exemplary flow diagram of a screen interaction method provided in accordance with some embodiments of the present application. As shown in FIG. 2, the process 200 may be implemented by the screen interaction system 100. In some embodiments, the screen interaction method 200 may be initiated automatically or by command. The instructions may include system instructions, device instructions, user instructions, action instructions, and the like, or a combination of the several. At 201, client information is entered into a server queue. The operation 201 may be implemented by the client 130, the server 140 of the screen interaction system 100. In some embodiments, the client 130 may send information to the server according to the HTTP protocol. Further, the client 130 information is entered into a Redis queue maintained by the server 140. The Redis (remote dictionary Server), which is a remote dictionary service, is a high-performance Key-Value storage system or database. Redis may support the storage of more Value types, which may include String, List, Set, Zset, and Hash.

At 202, a server queue is requested, and client information is entered into the system via the control side. The operation 202 may be implemented by the control end 120 of the screen interaction system 100, the server 140. In some embodiments, the control end 120 may request the server queue and, in turn, enter client 130 information into the system.

At 203, when the client is confirmed to enter the system, screen information is sent to the server through the client. Operation 203 may be implemented by the client 130, server 140 of the screen interaction system 100. In some embodiments, the client 130 may send request information to the system, including requesting the system to feed back whether the client 130 entered the system. When the system feeds back that the client enters the system, it is confirmed that the client 130 enters the system. Further, the client 130 may send screen information to the server 140. The screen information may include a screen shot of the client 130, etc., for example, the screen shot may be a JEPG stream (available as a video picture), etc.

At 204, the server is requested to obtain the client IP list, and the screen information of the client is loaded through the control terminal. Operation 204 may be implemented by the control end 120, the server 140 of the screen interaction system 100. In some embodiments, the control end 120 may request the server 140 to obtain the IP list of the client 130. Further, the control end 120 may load screen information of the client 130, such as a screen shot (JEPG stream), and the like.

According to some embodiments of the present application, the process 200 may further include client monitoring and/or screen sharing, and the like. As shown in fig. 3, fig. 3 is an exemplary flow diagram of a particular screen interaction method provided according to some embodiments of the present application. The process 300 is a specific screen interaction method, and operations 301 to 304 are the same as operations 201 to 204 in the process 200, which are not described herein again.

In some embodiments, operation 304 may further proceed to operation 305 where client monitoring is performed by the control side 305. The operation 305 may be implemented by the control end 120 of the screen interaction system 100, the server 140. By way of example, the control end 120 may perform monitoring and the like on one or more clients 130, which may include viewing screens and the like of the clients 130 in real time. The specific process of performing client monitoring by the control end is shown in fig. 4, and fig. 4 is an exemplary process of client monitoring of a screen interaction method provided according to some embodiments of the present application. The process 400 is a specific process of client monitoring, and can be implemented by the screen interaction system 100.

At 401, according to the IP list, a client to be monitored is selected by the control end and UDP information is sent. In some embodiments, the control end 120 may click on the client 130 in the IP list, select the client 130 to be monitored, and send udp (user data program) information to the client.

At 402, screen information is saved and HTTP services are opened by the client based on the UDP information. In some embodiments, the client 130 may receive the UDP information, save the screen information, and open the HTTP service. The saving the screen information may include saving a picture stream of the screen information to the local at a Frame rate (Frame rate) of 25 frames per second. The frame rate is usually expressed as fps (frames Per second), and may be any integer number of frames Per second, for example, 15 frames Per second, 24 frames Per second, etc. By way of example, the frame rate of a two-dimensional animation is typically 15 frames per second, and the base frame rate of a movie is typically 24 frames per second.

In some embodiments, the client 130 may start an HTTP service, and the client within the lan reads the picture stream local to the shared client. As an example, when the HTTP service is opened, the HTTP service points to a local map stream, accessible to clients within the local area network. The picture stream may include a JEPG stream, which may be a video picture. According to the method and the device, by reading the JEPG stream, the first frame and all the subsequent frames are JEPG pictures, and the problem of white screen caused by the fact that a plug-in cannot read data at the beginning does not exist. According to the method and the device, the HTTP service is very light in weight, the transmission of the picture stream is rapid, and the problem of sharing delay in the screen interaction process is solved.

At 403, a CMD command is executed by the client to open the accessible port and return the service address to the control end. In some embodiments, the client 130 may execute a CMD command to open its own port for access and return a service address to the control end 120. As an example, when the client 130 starts the HTTP service for other clients to access, and when other clients cannot access due to the firewall, the CMD command is used to newly name the firewall rule and open the port, and the local area network authority is set so that the port is exposed to the local area network and can be accessed by other clients in the local area network. The CMD command comprises a Screen Task which is the name of the firewall rule.

At 404, a window is opened by the control end to monitor the client screen according to the service address. In some embodiments, according to the service address returned by the client, the control end 120 may open the embedded browser window to browse the service address, and may view the screen of the client.

In some embodiments, operation 304 may further proceed to operation 306, where screen sharing of the client is performed to other clients through the control end at 306. Operation 306 may be implemented by the control end 120, the server 140 of the screen interaction system 100. By way of example, the control end 120 may share a screen of one client 130 to other clients 130, and the like, which may include multiple clients 130 viewing a screen of a certain client 130 in real time, and the like. The specific process of performing screen sharing of the client to other clients through the control end is shown in fig. 5, and fig. 5 is an exemplary screen sharing process of a screen interaction method provided according to some embodiments of the present application. The process 500 is a specific process of client screen sharing, and can be implemented by the screen interaction system 100, where operations 501 to 503 are consistent with operations 401 to 403 in the process 400, and are not described herein again.

In some embodiments, operation 503 may further proceed to operation 504, and at 504, the service address of the client is sent to other clients through the control end by using UDP. As an example, the control end 120 may send the service address of the client to other clients through UDP.

At 505, windows are opened by other clients to share the client screen according to the service address. According to the service address returned by the client, the other clients 130 can browse the service address by opening the embedded browser window, and can share and view the screen of the client.

It should be noted that, the operation 305 and the operation 306 may be executed simultaneously, or the operation 306 is executed first and then the operation 305 is executed, and the application does not limit this, that is, the operations are not in sequence in the execution timing, and the above description of the operations of the flow 300 is only an exemplary description of the embodiments of the application.

It should be noted that the above descriptions of the

processes

200, 300, 400, and 500 are only for convenience of description, and should not limit the scope of the present application. It will be understood by those skilled in the art that various modifications and changes in form and detail may be made in the functions implementing the above-described processes and operations based on the principles of the present system, in any combination of operations or in combination with other operations constituting sub-processes without departing from the principles. For example, the process 200 may further include operations such as client monitoring and/or screen sharing. Such variations are within the scope of the present application.

In summary, according to the screen interaction method and system provided by the embodiment of the application, the HTTP service is started for other terminal devices to access, the new naming firewall rule and the port are started by the CMD command, and the terminal devices can access, so that the other terminal devices in the local area network can monitor the screen of the same terminal device through multiple screens, the problems of screen sharing delay, screen jamming, white screen and the like are solved, and the user experience is comprehensively improved.

It is to be noted that the above-described embodiments are merely examples, and the present application is not limited to such examples, but various changes may be made.

It should be noted that, in the present specification, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.

Finally, it should be noted that the series of processes described above includes not only processes performed in time series in the order described herein, but also processes performed in parallel or individually, rather than in time series.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware associated with computer program instructions, and the program can be stored in a computer readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.

While the invention has been described with reference to a number of illustrative embodiments, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention.

Claims

1. A screen interaction method, comprising:

recording the client information into a server queue;

a request server queue, which is used for inputting client information into the system through a control terminal;

when the client is confirmed to enter the system, screen information is sent to the server through the client;

and requesting the server to acquire the IP list of the client, and loading the screen information of the client through the control terminal.

2. The method of claim 1, further comprising:

and executing client monitoring through the control end.

3. The method of claim 1, further comprising:

and executing screen sharing of the client to other clients through the control terminal.

4. The method of claim 2 or 3, further comprising:

selecting a client to be monitored through a control end according to the IP list and sending UDP information;

saving screen information and starting HTTP service through the client according to the UDP information;

and executing the CMD command through the client to open the accessible port and return the service address to the control end.

5. The method of claim 4, wherein the opening the HTTP service further comprises:

according to the UDP information, saving the picture stream of the screen information to the local through the client at a frame rate of 25 frames per second;

when the HTTP service is started, the HTTP service points to a local picture stream.

6. The method of claim 5, wherein the performing client monitoring by the control end further comprises:

and opening a window through the control terminal according to the service address so as to monitor the screen of the client.

7. The method of claim 5, wherein the performing screen sharing of the client to other clients through the control end further comprises:

sending the service address of the client to other clients through a control end by using UDP (user Datagram protocol);

and opening windows through other clients according to the service address to share the client screen.

8. The method of claim 1, wherein entering client information into a server queue further comprises:

sending information to a server through a client according to an HTTP (hyper text transport protocol);

and recording the client information into a Redis queue maintained by the server.

9. The method of claim 1, wherein confirming client entry into the system further comprises:

the client sends request information to the system, wherein the request information comprises a request for the system to feed back whether the client enters the system or not;

and when the system feeds back that the client enters the system, confirming that the client enters the system.

10. A system, comprising:

a memory configured to store data and instructions;

a processor in communication with the memory, wherein the processor, when executing instructions in the memory, is configured to:

recording the client information into a server queue;