CN112491973A - Equipment control method and device and nonvolatile storage medium - Google Patents
Equipment control method and device and nonvolatile storage medium Download PDFInfo
- Publication number
- CN112491973A CN112491973A CN202011248526.3A CN202011248526A CN112491973A CN 112491973 A CN112491973 A CN 112491973A CN 202011248526 A CN202011248526 A CN 202011248526A CN 112491973 A CN112491973 A CN 112491973A
- Authority
- CN
- China
- Prior art keywords
- browser
- image
- image data
- source device
- equipment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/02—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
- H04L67/025—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP] for remote control or remote monitoring of applications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
- H04L67/125—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks involving control of end-device applications over a network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/34—Network arrangements or protocols for supporting network services or applications involving the movement of software or configuration parameters
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
Landscapes
- Engineering & Computer Science (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Multimedia (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The application discloses a device control method and device and a nonvolatile storage medium. Wherein, the method comprises the following steps: the browser equipment displays image data from the source equipment through the browser; the method comprises the steps that browser equipment detects an operation event of a target object on image data in a display interface of a browser; when the operation event is detected, forwarding the operation event to an image source device connected with the source end device through the server, wherein the image source device is used for executing an operation corresponding to the operation event, and the image source device is a device for providing image data for the browser. According to the image receiving method and device, the technical problems that in the image receiving process, due to the fact that receiving end software of different versions needs to be developed for a user operating system to decode, a large amount of manpower development cost is consumed, meanwhile, in order to achieve correct decoding, the user also needs to be installed in the receiving end software matched with the operating system, and therefore the user is inconvenient to use are solved.
Description
Technical Field
The present application relates to the field of remote control technologies, and in particular, to a device control method and apparatus, and a non-volatile storage medium.
Background
In order to realize remote image transmission, the current solution is: installing soft S in a source end device (remote computing device) or connecting with hard S, and meanwhile installing soft R in a receiving end device or connecting with hard R; the S terminal (soft S or hard S) is used for collecting a desktop image of the source terminal equipment, compressing and coding the image, and then transmitting an obtained code stream to the R terminal (soft R or hard R) on the receiving terminal equipment in real time through a network; and after receiving the code stream sent by the S end, the R end decodes the code stream and sends the code stream to a display for displaying. In addition, the R terminal also needs to acquire input information of peripheral devices such as a keyboard and a mouse of the user, and send the input information to the source terminal device. If the R-side software is installed in the receiving end to decode the image, when the receiving end adopts different operating systems (Windows, Linux, Mac, Android), etc., different R-side software versions must be developed to match the different operating systems, which increases the workload of developers; meanwhile, for a user, when receiving images on different operating systems, R-side software of a matched version also needs to be installed, which is cumbersome for the user and inconvenient to use.
In view of the above problems, no effective solution has been proposed.
Disclosure of Invention
The embodiment of the application provides a device control method and device and a nonvolatile storage medium, so as to at least solve the technical problem that in the image receiving process, a large amount of manpower development cost is consumed due to the fact that receiving end software of different versions needs to be developed for a user operating system to decode, and meanwhile, in order to achieve correct decoding, the user also needs to install the receiving end software matched with the operating system, so that the user is inconvenient to use.
According to an aspect of an embodiment of the present application, there is provided an apparatus control method including: the method comprises the steps that browser equipment displays image data from source equipment through a browser, wherein the image data are image data from image source equipment and are forwarded by the source equipment; the browser equipment detects an operation event of a target object on the image data in a display interface of the browser; and when the operation event is detected, forwarding the operation event to an image source device connected with the source end device through a server, wherein the image source device is used for executing an operation corresponding to the operation event, and the image source device is a device for providing image data for the browser.
Optionally, the browser device displays the image data from the source device through the browser, and includes: the browser equipment sends an image transmission request to the server, wherein the image transmission request is used for requesting image data displayed in the browser; the browser equipment receives an image source equipment list fed back by the server in response to the image transmission request, and displays the image source equipment list in a display interface of the browser; the browser equipment detects a selection instruction of a target object in a display interface of the browser, and selects target image source equipment from the image source equipment list in response to the selection instruction; the browser device sends the image transmission request to the target image source device and receives image data from the target image source device, wherein the target image source device forwards the image data to the browser device through a source device and the server.
Optionally, the detecting, by the browser device, an operation event of the image data in a display interface of the browser by a target object includes: determining an operation position of the operation event in a display interface of the browser; determining a relative position between the image data displayed in the display interface and the operation position; and generating the operation event at least based on the relative position and the action corresponding to the operation event.
Optionally, forwarding the operation event to an image source device connected to the source device through a server includes: and sending the operation event to the image source device through a special channel among the server, the source device and the image source device, wherein the special channel is different from a channel used for transmitting the image data.
Optionally, before the browser device presents the image data from the source device through the browser, the method further includes: the browser device receives the image data forwarded by the source device through the server, where the server is configured to convert the encoded data of the image data from the source device in the first data format supported by the source device into encoded data of the second data format supported by the browser device.
According to another aspect of the embodiments of the present application, there is also provided another device control method, including: the server receives an operation event detected by browser equipment, wherein the operation event is generated by operating image data from source equipment displayed by a browser by a target object; and the server forwards the operation event to an image source device through the source end device, wherein the image source device is used for executing an operation corresponding to the operation event, and the image source device is a device for providing image data for the browser.
Optionally, before the server receives the operation event detected from the browser device, the method further includes: the server receives an image transmission request sent by the browser equipment, wherein the image transmission request is used for requesting image data displayed in the browser; the server responds to the image transmission request and feeds back an image source device list to the browser device; the server forwards the image transmission request to the target image source device through the source end device, wherein the target image source device is an image source device selected from the image source device list by the browser device in response to a selection instruction detected in a display interface of the browser; and the server receives the image data from the target image source equipment, and sends the image data to the browser equipment after format conversion.
Optionally, before feeding back the list of image source devices to the browser device, the method further comprises: the server determines the authority information of the browser equipment and image source equipment corresponding to the authority information; the server generates the image source device list based on the image source device corresponding to the authority information.
According to another aspect of the embodiments of the present application, there is also provided an apparatus control device, applied to a browser apparatus, including: the display module is used for displaying image data from source equipment through a browser, wherein the image data is image data from image source equipment forwarded by the source equipment; the detection module is used for detecting an operation event of a target object on the image data; and the sending module is used for forwarding the operation event to an image source device connected with the source end device through a server when the operation event is detected, wherein the image source device is used for executing an operation corresponding to the operation event, and the image source device is a device used for providing image data for the browser. According to another aspect of the embodiments of the present application, a non-volatile storage medium is further provided, where the non-volatile storage medium includes a stored program, and when the program runs, the device in which the non-volatile storage medium is located is controlled to execute the device control method.
According to another aspect of the embodiments of the present application, a non-volatile storage medium is further provided, where the non-volatile storage medium includes a stored program, and when the program runs, the device in which the non-volatile storage medium is located is controlled to execute the device control method.
According to another aspect of the embodiments of the present application, there is also provided a processor for executing program instructions stored in a memory, the program instructions when executed perform the apparatus control method described above.
In the embodiment of the application, the browser device is adopted to display the image data from the source end device through the browser, wherein the image data is the image data from the image source device forwarded by the source end device; the method comprises the steps that browser equipment detects an operation event of a target object on image data in a display interface of a browser; and when the operation event is detected, forwarding the operation event to an image source device connected with the source end device through the server, wherein the image source device is used for executing an operation corresponding to the operation event, and the image source device is a device for providing image data for the browser. The method uses the browser device as an image receiving end, the browser device obtains user operation information and forwards the user operation information to the source end device through the server, and then the image source device completes corresponding operation according to the operation information, wherein the image decoding process is completed by the server, and R end software in the related technology is not needed any more, so that the technical problem that in the image receiving process, because receiving end software of different versions needs to be developed for a user operation system for decoding, a large amount of manpower development cost is consumed, and meanwhile, in order to realize correct decoding, a user also needs to install the receiving end software matched with the operation system, so that the user is inconvenient to use is solved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:
fig. 1 is a schematic structural diagram of a remote image transmission system according to an embodiment of the present application;
FIG. 2 is a schematic flow chart diagram of a method for controlling a device according to an embodiment of the present application;
FIG. 3 is a schematic flow chart diagram of another apparatus control method according to an embodiment of the present application;
FIG. 4 is a schematic flow chart diagram of another apparatus control method according to an embodiment of the present application;
fig. 5 is a schematic structural diagram of an apparatus control device according to an embodiment of the present application.
Detailed Description
In order to make the technical solutions better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only partial embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
It should be noted that the terms "first," "second," and the like in the description and claims of this application and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the application described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Example 1
In accordance with an embodiment of the present application, there is provided an embodiment of a device control method, it should be noted that the steps illustrated in the flowchart of the drawings may be performed in a computer system such as a set of computer executable instructions, and that while a logical order is illustrated in the flowchart, in some cases the steps illustrated or described may be performed in an order different than here.
Fig. 1 is a schematic structural diagram of a remote image transmission system according to an embodiment of the present application, as shown in fig. 1, the system at least includes: image source device 18, source device 16, server 14, browser device 12, and browser 120 running on the browser device, wherein:
the browser 120: for presenting image data from an image source device in a display interface; the target object operates on the image data at the browser display interface.
The browser device 12: the method is used for detecting an operation event of the target object on the image data at the browser display interface, and forwarding the operation event to the image source device connected with the source end device through the server.
The server 14: the source terminal equipment is used for receiving and storing image source equipment information from the source terminal equipment; receiving an image transmission request of browser equipment, detecting the authority of the browser equipment, and feeding back an image source equipment list; forwarding the image transmission request to an image source device through a source device; receiving image data, transcoding the image data and sending the transcoded image data to browser equipment; and receiving and forwarding the operation event detected by the browser equipment.
The source device 16: the source device is used for acquiring image data of the image source device, encoding the image data and sending the encoded image data to the server, wherein the source device can be a soft S or a hard S.
Image source device 18: the image source device can be a PC, a virtual machine, a camera and the like.
In the related art, the source device transmits vGTP (Graphics Transfer Protocol) encoded data to the server, and the vGTP code belongs to a private Protocol, and if the browser needs to decode the vGTP encoded data directly, only a private vGTP decoder may be used, so that different versions of browsers need to install different versions of receiving end software.
Therefore, in this embodiment of the present application, a vGTP decoder may be installed in a server, and first the vGTP encoded data sent by a source device is decoded, then secondary encoding (e.g., jpg, png, etc.) is performed on the decoded data by using an encoding manner decodable by a browser device, and finally the encoded data obtained by the secondary encoding is sent to the browser device, and at this time, the browser device may use a built-in decoder to decode and display image data on a browser display interface.
It should be noted that, because the source device and the browser are not located in a network that can be accessed to each other, the server is used as a bridge between the source device and the browser to implement communication between the source device and the browser. In particular implementations, a source device can communicate with one or more browsers via a server.
In the foregoing operating environment, an embodiment of the present application provides a device control method, as shown in fig. 2, the method includes at least steps S202 to S206:
in step S202, the browser device displays image data from the source device through the browser.
The image data is image data from an image source device forwarded by a source device.
In some embodiments of the present application, a browser device presents image data from a source device through a browser, including the following:
the browser equipment sends an image transmission request to a server, wherein the image transmission request is used for requesting image data displayed in the browser, and the connection can be realized by using HTTP (S)/WEBSOCKET;
the browser equipment receives an image source equipment list fed back by the server in response to the image transmission request, and displays the image source equipment list in a display interface of the browser;
the connection can be realized by using HTTP (S)/WEBSOCKET;
the browser device detects a selection instruction of a target object in a display interface of the browser, and selects a target image source device from an image source device list in response to the selection instruction, for example, the target object can select a PC, a camera and the like from the selected target image source device, and in order to ensure real-time performance, the web browser device uses web browser key connection at this time;
the browser device sends an image transmission request to the target image source device and receives image data from the target image source device, wherein the target image source device forwards the image data to the browser device through the source device and the server.
In some embodiments of the present application, before the browser device presents the image data from the source device through the browser, the method further includes: the browser device receives image data forwarded by the source device through the server, wherein the server is used for converting encoded data of the image data from the source device in a first data format supported by the source device into encoded data of a second data format supported by the browser device.
For example, the image data transmitted by the source device to the server is GTP encoded data, and if the browser cannot directly decode and display the image data, a GTP decoder may be installed in the server, and the GTP encoded data transmitted by the source device is decoded first, then the decoded data is secondarily encoded (e.g., jpg, png, etc.) by using a coding method decodable by the browser device, and then the encoded data obtained by the secondary encoding is transmitted to the browser device, and at this time, the browser device may use a built-in decoder to decode and display the image data on a browser display interface.
Step S204, the browser device detects an operation event of the target object on the image data in the display interface of the browser.
In some embodiments of the present application, the browser device detecting an operation event of a target object on image data in a presentation interface of a browser, includes: determining an operation position of an operation event in a display interface of a browser; determining a relative position between the image data displayed in the display interface and the operation position; and generating the operation event at least based on the relative position and the action corresponding to the operation event.
Specifically, since the image window displayed by the browser is not displayed in full screen, at this time, the coordinate position of the user's keyboard and mouse operation directly obtained by the browser is the coordinate on the current display screen and is not the relative coordinate in the image display window of the browser, but for the source device, if the image in the window needs to be correctly operated, it is actually the relative coordinate of the keyboard and mouse operation in the image window, and therefore, after the browser directly obtains the coordinate position information of the keyboard and mouse operation, the obtained coordinate position information needs to be converted, so as to obtain the relative coordinate position information in the image window. The specific conversion scheme is as follows:
generally, the image in the browser is displayed as large as the source end, the program captures the position of the mouse as the coordinates (x1, y1) of the mouse in the visual area of the browser, and then obtains the coordinates (x2, y2) of the upper left corner of the image in the visual area, so the coordinates of the mouse relative to the image are:
x-coordinate: x1-x 2; y-coordinate: y1-y 2.
If the browser display image is zoomed, the source image has width and height respectively being width and height, and the browser display width and height are width1 and height1, then the coordinate calculation formula becomes:
x coordinate (x1-x2) (width/width 1); y coordinates (y1-y2) (height/height 1).
Step S206, when the operation event is detected, forwarding the operation event to the image source device connected to the source device through the server.
The image source device is used for executing operation corresponding to the operation event, and the image source device is a device for providing image data for the browser.
In some embodiments of the present application, forwarding the operation event to an image source device connected to the source device through the server includes: and sending the operation event to the image source device through a special channel among the server, the source end device and the image source device, wherein the special channel and a channel adopted for transmitting image data are different channels.
For example, the transmission image data can be transmitted by a common HTTP protocol, and the transmission operation event information can be transmitted by a Telent protocol, which are not interfered with each other.
According to the above device control method, an embodiment of the present application further provides a reverse control method in remote image transmission, where the method at least includes steps S302-S316:
step S302, the server receives and stores the image source device list from the source device.
In some embodiments of the present application, a source device sends acquired image source device information to a server, and the server records received information of the source device; specifically, the server records the received image source device information acquired by the source device in a local image source device list.
In step S304, the browser sends an image transmission request to the server.
In step S306, the server transmits the image source device list to the browser.
In some embodiments of the application, after the browser sends the image transmission request to the server, the server first selects image source device information available to the current browser locally according to the permission of the current browser, generates an image source device list based on the selected image source device information, sends the generated image source device list to the browser device, and then displays the image source device list on a display interface of the browser. In specific implementation, the connection can be realized by using HTTP (S)/WEBSOCKET.
In step S308, the browser sends the target image source device to the server.
In some embodiments of the present application, a user selects a desired target image source device from a list of image source devices in a browser display interface, for example, the user may select a PC or a camera, and then the server sends an image transmission request for the target image source device to the source end device. To ensure real-time, a webscocket connection is preferably used at this time.
Step S310, the server establishes a connection channel with the corresponding image source device.
In some embodiments of the present application, after receiving a request from a target image source device, a server establishes a connection with a source device corresponding to the target image source device; meanwhile, the server establishes a corresponding number of image transmission connection channels with the current browser according to user setting or system default setting.
In some embodiments of the present application, the image transmission process involves transmission of image, audio, and reverse control information, and a separate transmission channel needs to be established for each information, however, these three channels are not used in every scene, and only one or two channels may be needed in some scenes. Therefore, the number and type of channels opened, such as opening only image and audio channels, or opening all three channels, may be preset in the default settings of the server. In the using process of the user, the number and the type of the opened channels can be set according to the self requirement, wherein the priority set by the user is higher than the priority set by the system. At this time, when the browser initiates a connection request with the server, if the current user performs setting, the corresponding number and type of channels are opened according to the user setting, and if the user does not perform setting, the processing is performed according to the default setting of the system.
In step S312, the source device transcodes the image data of the target image source to the browser.
In some embodiments of the application, a source device acquires image data from an image source device to perform vGTP encoding, and sends the encoded data to a server, and the server decodes the received encoded data, performs secondary encoding on the decoded data in a manner (such as jpg, png, and the like) decodable by a current browser, sends the encoded data obtained by the secondary encoding to the browser device, and displays the encoded data in a display interface of the browser.
In step S314, the browser device detects operation information of the user on the browser display interface.
In some embodiments of the present application, the browser device detects reverse control information of the user on the image data in the browser display interface, and the reverse control information may be keyboard and mouse operation information. Wherein, the mouse operation information comprises: event type information such as left key press, left key lift, right key press, right key lift, roller up, roller down, etc., data information such as coordinates, roller rotation direction, etc.; the keyboard operation information comprises: event type information such as press, lift, etc., data information such as key code, etc.
It should be noted that, due to the browser permission or the operating system permission, there may be a problem in acquiring some combination keys, a simulated combination key may be provided in the browser, for example, a virtual keyboard is displayed in a browser interface, and the combination key of the virtual keyboard is determined in response to a selection instruction of a target object; or, showing a control corresponding to the combined key in the browser interface, and when the control is triggered, determining that the combined key is selected and activated, so as to trigger the corresponding function based on the activated combined key, wherein the combined key comprises but does not correspond to the following combinations: CTRL + ALT + DELETE, CTRL + V, etc.
Step S316, the browser sends the operation information to the image source device through the server and the source device, and the image source device responds to the operation information.
In some embodiments of the application, the source end device receives the reverse control information and sends the reverse control information to the image source device, the image source device injects the keyboard and mouse operation into the local operating system, and the keyboard and mouse operation is restored through the operating system, so that the reverse control of the image source device through the browser can be realized.
In some embodiments of the present application, the source device may transmit the keyboard and mouse operation event to the image source device through USB.
In the embodiment of the application, the browser device is adopted to display the image data from the source device through the browser; detecting an operation event of a target object on image data in a display interface of a browser by browser equipment; when an operation event is detected, forwarding the operation event to an image source device connected with a source end device through a server; the image source device performs an operation corresponding to the operation event. Compared with the prior art, the browser equipment is directly used as the image receiving end in the embodiment, the decoding process of vGTP image data is completed in the server, special software does not need to be installed for decoding, and the user can use the vGTP image data more conveniently; meanwhile, the equipment control method can realize the remote operation of the images in the image source equipment in the browser, and greatly improves the use experience of users.
Example 2
According to an embodiment of the present application, another device control method is provided, as shown in fig. 4, the method at least includes steps S402-S404:
in step S402, the server receives an operation event detected from the browser device.
The operation event is generated by the target object operating the image data from the source device displayed by the browser.
In some embodiments of the present application, before the server receives the operation event detected from the browser device, the method further includes:
the method comprises the steps that a server receives an image transmission request sent by browser equipment, wherein the image transmission request is used for requesting image data displayed in a browser;
the server responds to the image transmission request and feeds back an image source device list to the browser device;
the server forwards the image transmission request to target image source equipment through source end equipment, wherein the target image source equipment is image source equipment selected from an image source equipment list by the browser equipment in response to a selection instruction detected in a display interface of the browser;
the server receives the image data from the target image source device, and sends the image data to the browser device after format conversion.
In some embodiments of the present application, prior to feeding back the list of image source devices to the browser device, the method further comprises: the method comprises the steps that a server determines authority information of browser equipment and image source equipment corresponding to the authority information; the server generates an image source device list based on the image source devices corresponding to the authority information.
Step S404, the server forwards the operation event to the image source device through the source device.
The image source device is used for executing operation corresponding to the operation event, and the image source device is a device for providing image data for the browser.
It should be noted that this embodiment is also implemented in the environment of the remote image transmission system in embodiment 1, and on this basis, the difference from embodiment 1 is that a main body of the device control method in embodiment 1 is a browser device, and a main body of the device control method in this embodiment is a server, and other specific implementation processes are basically the same, and as the description has been already made in embodiment 1, redundant description is not repeated here.
Example 3
According to an embodiment of the present application, there is also provided an apparatus control device, configured to implement the apparatus control method in fig. 3, as shown in fig. 5, the apparatus at least includes a presentation module 50, a detection module 52, and a sending module 54, where:
a display module 50, configured to display, by using a browser, image data from a source device, where the image data is image data from an image source device forwarded by the source device;
a detection module 52, configured to detect an operation event of the target object on the image data; and
and a sending module 54, configured to forward, when an operation event is detected, the operation event to an image source device connected to the source end device through the server, where the image source device is configured to perform an operation corresponding to the operation event, and the image source device is a device configured to provide image data for the browser.
Example 4
According to an embodiment of the present application, a non-volatile storage medium is further provided, where the non-volatile storage medium includes a stored program, and when the program runs, the device in which the non-volatile storage medium is located is controlled to execute the device control method.
Optionally, the apparatus in which the non-volatile storage medium is controlled when the program runs performs the following steps: the method comprises the steps that the browser device displays image data from source-end equipment through a browser, wherein the image data are image data from image source equipment forwarded by the source-end equipment; the method comprises the steps that browser equipment detects an operation event of a target object on image data in a display interface of a browser; and when the operation event is detected, forwarding the operation event to an image source device connected with the source end device through the server, wherein the image source device is used for executing an operation corresponding to the operation event, and the image source device is a device for providing image data for the browser.
Optionally, the apparatus in which the non-volatile storage medium is controlled when the program runs performs the following steps: the method comprises the steps that a server receives an operation event detected by browser equipment, wherein the operation event is generated by operating image data from source equipment displayed by the browser; and the server forwards the operation event to the image source device through the source end device, wherein the image source device is used for executing the operation corresponding to the operation event, and the image source device is a device for providing image data for the browser.
The above-mentioned serial numbers of the embodiments of the present application are merely for description and do not represent the merits of the embodiments.
In the above embodiments of the present application, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
In the embodiments provided in the present application, it should be understood that the disclosed technology can be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units may be a logical division, and in actual implementation, there may be another division, for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, units or modules, and may be in an electrical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be substantially implemented or contributed to by the prior art, or all or part of the technical solution may be embodied in a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes.
The foregoing is only a preferred embodiment of the present application and it should be noted that those skilled in the art can make several improvements and modifications without departing from the principle of the present application, and these improvements and modifications should also be considered as the protection scope of the present application.
Claims (10)
1. An apparatus control method characterized by comprising:
the method comprises the steps that browser equipment displays image data from source equipment through a browser, wherein the image data are image data from image source equipment and are forwarded by the source equipment;
the browser equipment detects an operation event of a target object on the image data in a display interface of the browser;
and when the operation event is detected, forwarding the operation event to an image source device connected with the source end device through a server, wherein the image source device is used for executing an operation corresponding to the operation event, and the image source device is a device for providing image data for the browser.
2. The method of claim 1, wherein the browser device renders the image data from the source device via the browser, comprising:
the browser equipment sends an image transmission request to the server, wherein the image transmission request is used for requesting image data displayed in the browser;
the browser equipment receives an image source equipment list fed back by the server in response to the image transmission request, and displays the image source equipment list in a display interface of the browser;
the browser equipment detects a selection instruction of a target object in a display interface of the browser, and selects target image source equipment from the image source equipment list in response to the selection instruction;
the browser device sends the image transmission request to the target image source device and receives image data from the target image source device, wherein the target image source device forwards the image data to the browser device through a source device and the server.
3. The method according to claim 1, wherein the browser device detects an operation event of a target object on the image data in a presentation interface of the browser, and comprises the following steps:
determining an operation position of the operation event in a display interface of the browser;
determining a relative position between the image data displayed in the display interface and the operation position;
and generating the operation event at least based on the relative position and the action corresponding to the operation event.
4. The method of claim 1, wherein forwarding the operational event to an image source device connected to the source device via a server comprises:
and sending the operation event to the image source device through a special channel among the server, the source device and the image source device, wherein the special channel is different from a channel used for transmitting the image data.
5. The method of claim 1, wherein before the browser device renders the image data from the source device via the browser, the method further comprises:
the browser device receives the image data forwarded by the source device through the server, where the server is configured to convert the encoded data of the image data from the source device in the first data format supported by the source device into encoded data of the second data format supported by the browser device.
6. An apparatus control method characterized by comprising:
the method comprises the steps that a server receives an operation event detected by browser equipment, wherein the operation event is generated by operating image data from source equipment displayed by a browser by a target object;
and the server forwards the operation event to an image source device through the source end device, wherein the image source device is used for executing an operation corresponding to the operation event, and the image source device is a device for providing image data for the browser.
7. The method of claim 6, wherein before the server receives the operation event detected from the browser device, further comprising:
the server receives an image transmission request sent by the browser equipment, wherein the image transmission request is used for requesting image data displayed in the browser;
the server responds to the image transmission request and feeds back an image source device list to the browser device;
the server forwards the image transmission request to target image source equipment through the source end equipment, wherein the target image source equipment is image source equipment selected from the image source equipment list by the browser equipment in response to a selection instruction detected in a display interface of the browser;
and the server receives the image data from the target image source equipment, and sends the image data to the browser equipment after format conversion.
8. The method of claim 7, wherein prior to feeding back a list of image source devices to the browser device, the method further comprises:
the server determines the authority information of the browser equipment and image source equipment corresponding to the authority information;
the server generates the image source device list based on the image source device corresponding to the authority information.
9. An apparatus control device applied to a browser apparatus, comprising:
the display module is used for displaying image data from source equipment through a browser, wherein the image data is image data from image source equipment forwarded by the source equipment;
the detection module is used for detecting an operation event of a target object on the image data; and
and the sending module is used for forwarding the operation event to an image source device connected with the source end device through a server when the operation event is detected, wherein the image source device is used for executing an operation corresponding to the operation event, and the image source device is a device for providing image data for the browser.
10. A non-volatile storage medium, comprising a stored program, wherein a device in which the non-volatile storage medium is located is controlled to execute the device control method according to any one of claims 1 to 8 when the program is executed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011248526.3A CN112491973A (en) | 2020-11-10 | 2020-11-10 | Equipment control method and device and nonvolatile storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011248526.3A CN112491973A (en) | 2020-11-10 | 2020-11-10 | Equipment control method and device and nonvolatile storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112491973A true CN112491973A (en) | 2021-03-12 |
Family
ID=74929781
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011248526.3A Pending CN112491973A (en) | 2020-11-10 | 2020-11-10 | Equipment control method and device and nonvolatile storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112491973A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115002175A (en) * | 2022-05-27 | 2022-09-02 | 中国联合网络通信集团有限公司 | Remote control method, device and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120151369A1 (en) * | 2010-12-10 | 2012-06-14 | Wyse Technology Inc. | Methods and systems for accessing and controlling a remote desktop of a remote machine in real time by a web browser at a client device via http api utilizing a transcoding server |
| CN103460670A (en) * | 2010-12-10 | 2013-12-18 | 韦斯技术有限公司 | Methods and systems for a remote desktop session utilizing a http handler and a remote desktop client common interface |
| US20140168698A1 (en) * | 2012-12-17 | 2014-06-19 | Konica Minolta, Inc. | Remotely-operated apparatus remotely operated by external terminal, remote operation system including the same, and recording medium |
| CN105847307A (en) * | 2015-01-12 | 2016-08-10 | 北京神州泰岳信息安全技术有限公司 | Integrated operation and maintenance method and system |
-
2020
- 2020-11-10 CN CN202011248526.3A patent/CN112491973A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120151369A1 (en) * | 2010-12-10 | 2012-06-14 | Wyse Technology Inc. | Methods and systems for accessing and controlling a remote desktop of a remote machine in real time by a web browser at a client device via http api utilizing a transcoding server |
| CN103460670A (en) * | 2010-12-10 | 2013-12-18 | 韦斯技术有限公司 | Methods and systems for a remote desktop session utilizing a http handler and a remote desktop client common interface |
| CN103582510A (en) * | 2010-12-10 | 2014-02-12 | 韦斯技术有限公司 | Methods and systems for accessing and controlling a remote desktop of a remote machine in real time by a web browser at a client device via HTTP API utilizing a transcoding server |
| US20140168698A1 (en) * | 2012-12-17 | 2014-06-19 | Konica Minolta, Inc. | Remotely-operated apparatus remotely operated by external terminal, remote operation system including the same, and recording medium |
| CN105847307A (en) * | 2015-01-12 | 2016-08-10 | 北京神州泰岳信息安全技术有限公司 | Integrated operation and maintenance method and system |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115002175A (en) * | 2022-05-27 | 2022-09-02 | 中国联合网络通信集团有限公司 | Remote control method, device and storage medium |
| CN115002175B (en) * | 2022-05-27 | 2024-02-06 | 中国联合网络通信集团有限公司 | Remote control method, device and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111629239B (en) | Screen projection processing method, device, equipment and computer readable storage medium | |
| CN107329717B (en) | Processing method, display method and display system of screen display information | |
| CN109002269B (en) | Method, client and system for controlling multiple terminals by single-key mouse | |
| CN111221491A (en) | Interaction control method and device, electronic equipment and storage medium | |
| CN112905289A (en) | Application picture display method, device, terminal, screen projection system and medium | |
| WO2018194803A1 (en) | System and method for wireless screen projection | |
| WO2018107717A1 (en) | Method and system for mitigating feeling of lag of mouse cursor action in kvm system | |
| JP6272331B2 (en) | Method for remote presentation between at least two terminal devices connected via a network | |
| CN110750319A (en) | Conference tablet interaction method and device and conference tablet | |
| CN112422868A (en) | Data processing method, terminal device and server | |
| JP2005049666A (en) | Remote control system and information processor | |
| CN113573135A (en) | Screen projection method of display equipment, display equipment and client | |
| CN112491973A (en) | Equipment control method and device and nonvolatile storage medium | |
| CN112035082A (en) | Image processing method, device and system and electronic equipment | |
| WO2022252977A1 (en) | Screen projection method, storage medium, and communication device | |
| CN113316011B (en) | Control method, system, equipment and storage medium of electronic whiteboard system | |
| CN112714331B (en) | Information prompting method and device, storage medium and electronic equipment | |
| CN112714272A (en) | Inter-device control method, terminal, and computer-readable storage medium | |
| CN111107316B (en) | Image display method, device and system | |
| JP2001197461A (en) | Sharing operation method for multimedia information operation window | |
| CN113329247A (en) | Screen projection method and system | |
| CN112732120A (en) | Display device | |
| CN115129277A (en) | Interaction method, display device and VR device | |
| CN112650418A (en) | Display device | |
| CN112947783A (en) | Display device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |