CN115086725A

CN115086725A - Remote same-screen method, device and equipment based on browser and storage medium

Info

Publication number: CN115086725A
Application number: CN202210585687.4A
Authority: CN
Inventors: 段远超; 文金雷
Original assignee: Ping An Bank Co Ltd
Current assignee: Ping An Bank Co Ltd
Priority date: 2022-05-27
Filing date: 2022-05-27
Publication date: 2022-09-20

Abstract

The invention relates to the technical field of artificial intelligence and discloses a remote same-screen method, device, equipment and storage medium based on a browser. The method comprises the following steps: the method comprises the steps of receiving a same-screen request sent by a terminal and confirming; when the terminal is detected to be in a same-screen sharing state, a remote same-screen interaction interface is established with target equipment; receiving operation tasks sent by a remote data server, and sequencing the operation tasks based on a preset time sequence; during the period that the equipment is on the same screen, acquiring a current user operation video of a user from a preset video link and data association table; and displaying and processing the medical data of the user based on the operation task and the user operation video, and displaying a processing result on a display interface so as to share the same-screen content of the equipment. The invention displays the operation video of the user by receiving the operation task sent by the remote server, realizes the remote same screen between the two devices and improves the working efficiency of customer service.

Description

Remote same-screen method, device and equipment based on browser and storage medium

Technical Field

The invention relates to the technical field of artificial intelligence, in particular to a remote same-screen method, device, equipment and storage medium based on a browser.

Background

With the popularization of multimedia devices such as smart phones, tablet computers, network set-top boxes and smart televisions, the same-screen sharing technology is increasingly used. In the prior art, the same-screen sharing technology is that video data played by a sending end of a multimedia device is sent to a receiving end of the multimedia device in the same local area network, so that videos played on a mobile phone and a tablet personal computer can be conveniently pushed to large-screen extended playing devices such as a smart television and a projector in the same local area network, and the large-screen devices such as the smart television can play videos consistent with the mobile phone while the mobile phone and the tablet personal computer play videos.

However, the internal customer service generally adopts a telephone or video mode for solving the problem for the customer, often needs the customer to dictate the problem, and has the problems of unclear problem description or deviation in customer service understanding, and the like, so that the working efficiency of the customer service is reduced, and the service experience of the customer is not friendly. Therefore, how to realize remote screen sharing between the customer service and the user equipment becomes a technical problem to be solved by the technical personnel in the field.

Disclosure of Invention

The invention mainly aims to display and process the operation video of the user by receiving the operation task sent by the preset remote data server, thereby realizing remote one-screen operation between the customer service and the user equipment and improving the working efficiency of the customer service.

The invention provides a remote on-screen method based on a browser, which comprises the following steps: receiving a same-screen request sent by a terminal, and sending a confirmation request based on the same-screen request; when the terminal is detected to be in the same-screen sharing state, establishing a remote same-screen interaction interface of the terminal and target equipment to be connected with the same-screen receiving equipment; receiving operation tasks sent by a preset remote data server, and sequencing the operation tasks based on a preset time sequence; based on the operation task, acquiring a user operation video associated with the current video link of a user from a video link preset by the remote data server and a preset data association table; and displaying the medical data of the user based on the operation task and the user operation video, and displaying a processing result on a display interface so as to realize the operation synchronization of the terminal and the target equipment.

Optionally, in a first implementation manner of the first aspect of the present invention, before receiving an on-screen request sent by a terminal and sending an acknowledgement request based on the on-screen request, the method further includes: receiving the current load condition of the terminal; and determining target equipment connected with the terminal according to the current load condition of the terminal.

Optionally, in a second implementation manner of the first aspect of the present invention, the determining, according to the current load condition of the terminal, a target device connected to the terminal includes: sending a first request message for requesting the current connection state of a first type of equipment allowed to be accessed in the first on-screen receiving equipment to the remote data server, wherein the current connection number of the first type of equipment is less than the maximum connection number of the first type of equipment; acquiring the current connection status of all the devices in the first type of device based on the first request message; and determining the target equipment connected with the on-screen receiving equipment according to the current connection conditions of all the equipment.

Optionally, in a third implementation manner of the first aspect of the present invention, before the establishing, when it is detected that the terminal is in an on-screen sharing state, a remote on-screen interaction interface between the terminal and a target device to be connected to an on-screen receiving device is established, the method further includes: detecting whether the on-screen sharing service of the terminal is in an on-screen sharing state; and if so, determining that the terminal is in the same-screen sharing state.

Optionally, in a fourth implementation manner of the first aspect of the present invention, the obtaining, based on the operation task, a user operation video associated with the current video link by the user from a video link and preset data association table preset by the remote data server includes: acquiring the network transmission speed of the terminal and the target equipment in real time; performing pause analysis on the remote same-screen interactive interface to obtain a video end with pause phenomenon and a pause time period, and identifying equipment with pause phenomenon in the pause time period of the remote same-screen interactive interface corresponding to the terminal and the target equipment; and acquiring a user operation video associated with the current video link by the user from a video link preset by the remote data server and a preset data association table based on the identification.

Optionally, in a fifth implementation manner of the first aspect of the present invention, the performing a stuck analysis on the remote on-screen interactive interface to obtain a video end and a stuck time period in which a stuck phenomenon occurs, and identifying a device in which the stuck phenomenon occurs in the stuck time period on the remote on-screen interactive interface corresponding to the terminal and the target device includes: respectively acquiring the target equipment and the picture area corresponding to the target equipment from the remote co-screen interaction interface corresponding to the terminal and the target equipment; acquiring a terminal picture image corresponding to a first preset time period from the picture area of the target equipment according to a set acquisition time period, and acquiring a picture image of the target equipment corresponding to the first preset time period from the picture area of the target equipment; numbering the terminal picture images corresponding to the first preset time period according to the sequence of the acquisition time periods, carrying out terminal picture image comparison corresponding to the second preset time period on the terminal picture images corresponding to the first preset time period, and judging whether the terminal picture images corresponding to the second preset time period are the same or not.

Optionally, in a sixth implementation manner of the first aspect of the present invention, after performing image comparison on the terminal image corresponding to the first preset time period and the terminal image corresponding to the second preset time period, and determining whether the terminal image corresponding to the second preset time period is the same, the method further includes: if the time intervals are the same, indicating that the terminal has a pause phenomenon between the second preset time intervals, and recording the second preset time intervals as pause time intervals; numbering the target equipment picture images corresponding to the first preset time period according to the sequence of the acquisition time periods, carrying out picture image comparison on the target equipment picture images corresponding to the first preset time period and the target equipment picture images corresponding to the second preset time period, and judging whether the target equipment picture images corresponding to the second preset time period are the same or not; if the time intervals are the same, indicating that the target device has a pause phenomenon between the second preset time intervals, and recording the second preset time intervals as pause time intervals.

The invention provides a remote same-screen device based on a browser in a second aspect, which comprises: the sending module is used for receiving a same-screen request sent by a terminal and sending a confirmation request based on the same-screen request; the first detection module is used for establishing a remote on-screen interaction interface between the terminal and target equipment to be connected with the on-screen receiving equipment when the terminal is detected to be in an on-screen sharing state; the sequencing module is used for receiving operation tasks sent by a preset remote data server and sequencing the operation tasks based on a preset time sequence; the acquisition module is used for acquiring a user operation video related to the current video link of a user from a video link and preset data association table preset by the remote data server based on the operation task; and the display module is used for displaying and processing the medical data of the user based on the operation task and the user operation video and displaying a processing result on a display interface so as to realize the operation synchronization of the terminal and the target equipment.

Optionally, in a first implementation manner of the second aspect of the present invention, the browser-based remote on-screen device further includes: a receiving module, configured to receive a current load condition of the terminal; and the first determining module is used for determining the target equipment connected with the terminal according to the current load condition of the terminal.

Optionally, in a second implementation manner of the second aspect of the present invention, the first determining module is specifically configured to: sending a first request message for requesting the current connection state of a first type of equipment allowed to be accessed in the first on-screen receiving equipment to the remote data server, wherein the current connection number of the first type of equipment is less than the maximum connection number of the first type of equipment; acquiring the current connection status of all the devices in the first type of device based on the first request message; and determining the target equipment connected with the on-screen receiving equipment according to the current connection conditions of all the equipment.

Optionally, in a third implementation manner of the second aspect of the present invention, the browser-based remote on-screen device further includes: the second detection module is used for detecting whether the on-screen sharing service of the terminal is in an on-screen sharing state; and the second determining module is used for determining that the terminal is in the same-screen sharing state when the same-screen sharing service of the terminal is detected to be in the same-screen sharing state.

Optionally, in a fourth implementation manner of the second aspect of the present invention, the obtaining module includes:

the acquisition unit is used for acquiring the network transmission speeds of the terminal and the target equipment in real time;

the identification unit is used for performing pause analysis on the remote on-screen interactive interface to obtain a video end with a pause phenomenon and a pause time period, and identifying the equipment with the pause phenomenon in the pause time period of the remote on-screen interactive interface corresponding to the terminal and the target equipment;

and the obtaining unit is used for obtaining the user operation video related to the current video link of the user from the video link preset by the remote data server and the preset data association table based on the identification.

Optionally, in a fifth implementation manner of the second aspect of the present invention, the identifying unit is specifically configured to: respectively acquiring the target equipment and the picture area corresponding to the target equipment from the remote co-screen interaction interface corresponding to the terminal and the target equipment; acquiring a terminal picture image corresponding to a first preset time period from the picture area of the target equipment according to a set acquisition time period, and acquiring a picture image of the target equipment corresponding to the first preset time period from the picture area of the target equipment; numbering the terminal picture images corresponding to the first preset time period according to the sequence of the acquisition time periods, carrying out terminal picture image comparison corresponding to the second preset time period on the terminal picture images corresponding to the first preset time period, and judging whether the terminal picture images corresponding to the second preset time period are the same or not.

Optionally, in a sixth implementation manner of the second aspect of the present invention, the identification unit is further specifically configured to: if the time intervals are the same, indicating that the terminal has a pause phenomenon between the second preset time intervals, and recording the second preset time intervals as pause time intervals; numbering the target equipment picture images corresponding to the first preset time period according to the sequence of the acquisition time periods, carrying out picture image comparison on the target equipment picture images corresponding to the first preset time period and the target equipment picture images corresponding to the second preset time period, and judging whether the target equipment picture images corresponding to the second preset time period are the same or not; if the time intervals are the same, indicating that the target device has a pause phenomenon between the second preset time intervals, and recording the second preset time intervals as pause time intervals.

The third aspect of the present invention provides a remote on-screen device based on a browser, comprising: a memory having instructions stored therein and at least one processor, the memory and the at least one processor interconnected by a line;

the at least one processor invokes the instructions in the memory to cause the browser-based remote co-screen device to perform the steps of the browser-based remote co-screen method described above.

A fourth aspect of the present invention provides a computer-readable storage medium having stored therein instructions, which, when run on a computer, cause the computer to perform the steps of the above-described browser-based remote co-screen method.

According to the technical scheme provided by the invention, a same-screen request sent by a terminal is received, and a confirmation request is sent based on the same-screen request; when the terminal is detected to be in a same-screen sharing state, a remote same-screen interaction interface is established with target equipment; receiving operation tasks sent by a remote data server, and sequencing the operation tasks based on a preset time sequence; based on the operation task, acquiring a user operation video associated with the current video link of a user from a video link and data association table preset by a remote data server; and displaying the medical data of the user based on the operation task and the user operation video, and displaying a processing result on a display interface so as to realize the sharing of the content to be displayed on the same screen corresponding to the terminal and the target equipment. According to the invention, through the low-delay operation flow of watching the user and the cooperation of voice or video, remote one-screen interaction between the customer service and the user equipment is realized, the problems that the problem description of the user is unclear or the customer service understanding is deviated and the like are solved, and the technical problem of misjudgment of the working efficiency of the customer service is improved.

Drawings

FIG. 1 is a schematic diagram of a first embodiment of a browser-based remote on-screen method provided by the present invention;

FIG. 2 is a schematic diagram of a second embodiment of the browser-based remote on-screen method provided in the present invention;

FIG. 3 is a schematic diagram of a third embodiment of a remote browser-based on-screen method provided in the present invention;

FIG. 4 is a schematic diagram of a fourth embodiment of the remote browser-based on-screen method provided in the present invention;

FIG. 5 is a schematic diagram of a fifth embodiment of the remote browser-based on-screen method provided in the present invention;

FIG. 6 is a schematic diagram of a first embodiment of a browser-based remote on-screen device provided in the present invention;

FIG. 7 is a schematic diagram of a second embodiment of a browser-based remote on-screen device provided in the present invention;

fig. 8 is a schematic diagram of an embodiment of a remote browser-based on-screen device provided in the present invention.

Detailed Description

According to the remote on-screen method, device, equipment and storage medium based on the browser, the on-screen request sent by the terminal is received, and the confirmation request is sent based on the on-screen request; when the terminal is detected to be in a same-screen sharing state, a remote same-screen interaction interface is established with target equipment; receiving operation tasks sent by a remote data server, and sequencing the operation tasks based on a preset time sequence; based on the operation task, acquiring a user operation video associated with the current video link of a user from a video link and data association table preset by a remote data server; and displaying the medical data of the user based on the operation task and the user operation video, and displaying a processing result on a display interface so as to realize the sharing of the content to be displayed on the same screen corresponding to the terminal and the target equipment. The invention realizes remote one-screen between the customer service and the user equipment by matching the operation flow of watching the user with low time delay and voice or video, solves the problems of unclear description of the user problems or deviation of understanding of the customer service and the like, and improves the working efficiency of the customer service.

The terms "first," "second," "third," "fourth," and the like in the description and in the claims, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It will be appreciated that the data so used may be interchanged under appropriate circumstances such that the embodiments described herein may be practiced otherwise than as specifically illustrated or described herein. Furthermore, the terms "comprises," "comprising," or "having," and any variations thereof, are intended to cover non-exclusive inclusions, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

For convenience of understanding, a specific flow of an embodiment of the present invention is described below, and referring to fig. 1, a first embodiment of a remote browser-based on-screen method in an embodiment of the present invention includes:

101. receiving a same-screen request sent by a terminal, and sending a confirmation request based on the same-screen request;

in this embodiment, the confirmation request sent by the server is a request for confirming whether to perform the same screen with the terminal, that is, whether to confirm that the same screen authorization is performed on the terminal, and authorize to receive and present screen data and/or audio data of the terminal. When receiving a co-screen request sent by a terminal through a browser Web page, a server sends a confirmation request to target equipment corresponding to an equipment identifier according to the equipment identifier in the co-screen request.

Further, in this embodiment, the target device may further receive a device identifier obtaining request, and display a device identifier of the target device according to the device identifier obtaining request. Therefore, the user can provide the displayed device identifier of the target device to the terminal, so that the same-screen request sent by the terminal can carry the device identifier of the target device. The device identifier may be a machine identifier or other identifier that may represent the identity of the target device.

102. When the terminal is detected to be in the same-screen sharing state, establishing a remote same-screen interaction interface of the terminal and target equipment to be connected with the same-screen receiving equipment;

in this embodiment, when a user watches an online video using a multimedia device sending end, such as a smart phone, in order to obtain a better experience effect, the played video is pushed or shared to a multimedia device receiving end, such as a screen of a smart television. When the video is pushed or shared to the smart television from the smart phone, the same-screen sharing service needs to be called, such as one of wireless video transmission services like AirPlay, WiDi, Miracast, DLNA and the like. When one or more enabling of the same-screen sharing services is detected, the multimedia equipment sending end and the multimedia equipment receiving end are considered to be communicated, video sharing is carried out, and the same-screen sharing state of the smart phone is marked as the same-screen sharing state. When the smart phone watches videos on line, if the terminal is detected to be in the same-screen sharing state, an instruction for connecting a remote data server is sent, and communication with the remote data server is established.

103. Receiving operation tasks sent by a preset remote data server, and sequencing the operation tasks based on a preset time sequence;

in this embodiment, the multiple terminals accessing the same remote group may be multiple cross-platform terminals, and may support terminals of multiple platforms, such as an android system, an IOS system, and a Windows system. And the terminals described in the present embodiment may include mobile terminals such as mobile phones, smart phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), navigation devices, etc., and fixed terminals such as digital TVs, desktop computers, etc.

The plurality of terminals can access the same remote group in the following modes: the initiator terminal establishes a long connection with the server, and after the long connection is established, the initiator terminal creates a room and generates the room key, wherein the room is the remote group. The originating terminal may send the room number and the key to the participant terminals of other consultants via a conventional chat channel (e.g., QQ, instant messaging software such as wechat, etc.). The participant terminal establishes a long connection with the server, and after the long connection is established, the participant enters a room created by the initiator terminal through a room number and a secret key, namely, the participant accesses to the remote group created by the initiator terminal.

104. Based on the operation task, acquiring a user operation video associated with the current video link of the user from a video link preset by a remote data server and a preset data association table;

in this embodiment, after the terminal establishes the connection with the target device, the terminal becomes the on-screen receiving device directly or indirectly connected to the on-screen transmitting device, and therefore, the related information stored in the server needs to be updated, and the terminal may transmit the current connection status to the server, so that the server stores the current connection status and updates the current connection status of the target device according to the current connection status.

When the terminal cannot directly acquire the content to be on-screen from the on-screen sending device, the content to be on-screen needs to be acquired through forwarding by the target device, and in order to reduce the loss caused by unnecessary forwarding, the target device should be a device which allows access and has the minimum loss when the content to be on-screen is acquired.

Specifically, the current connection status of the device may include: determining the target device connected to the on-screen receiving device may specifically be: determining the allowable connection coefficients of all the devices in the first class of devices according to the current connection conditions of all the devices in the first class of devices, wherein the allowable connection coefficients are inversely proportional to the connection delay and the connection hop count; and determining one device from the first-class devices as the target device according to the allowed connection coefficients of all the devices in the first-class devices, and acquiring a user operation video associated with the current video link of the user from a video link and data association table preset by the remote data server based on the operation task.

105. And displaying the medical data of the user based on the operation task and the user operation video, and displaying a processing result on a display interface so as to realize the operation synchronization of the terminal and the target equipment.

In this embodiment, the medical data of the user may be medical information stored locally by the terminal, or medical data of the user downloaded by the terminal from the medical system in real time. The terminal and the target device are operated synchronously, namely the terminal and the target device can synchronously display the same medical record content, so that customer service and user personnel using the terminal and the target device can read the same medical record content of a same person at the same time.

Specifically, after receiving operation tasks sent by other terminals, the terminal sorts the operation tasks according to the time sequence of the received operation tasks to obtain an operation task table, performs task deletion preprocessing and task repair preprocessing on the operation task table to obtain a preprocessed operation task table, processes medical data of a user through the operation tasks in the preprocessed operation task table in sequence, and displays an operation result on a display interface to achieve operation synchronization of the terminal and the target device.

In this embodiment, a server determines a target device connected to the server for a terminal in a local area network, where the target device may be a co-screen sending device or a co-screen receiving device directly or indirectly connected to the co-screen sending device, and creates a connection between the terminal and the target device to receive content to be co-screened, where the co-screen sending device is a device storing the content to be co-screened, that is, a source device initiating co-screen sharing to share the content to be co-screened.

In the embodiment of the invention, a same-screen request sent by a terminal is received, and a confirmation request is sent based on the same-screen request; when the terminal is detected to be in a same-screen sharing state, a remote same-screen interaction interface is established with target equipment; receiving operation tasks sent by a remote data server, and sequencing the operation tasks based on a preset time sequence; based on the operation task, acquiring a user operation video associated with the current video link of a user from a video link and data association table preset by a remote data server; and displaying the medical data of the user based on the operation task and the user operation video, and displaying a processing result on a display interface so as to realize the sharing of the content to be displayed on the same screen corresponding to the terminal and the target equipment. The invention realizes remote one-screen between the customer service and the user equipment by matching the operation flow of watching the user with low time delay and voice or video, solves the problems of unclear description of the user problems or deviation of understanding of the customer service and the like, and improves the working efficiency of the customer service.

Referring to fig. 2, a second embodiment of the remote co-screen method based on a browser according to the embodiment of the present invention includes:

201. receiving the current load condition of the terminal;

in this embodiment, a target device connected to a terminal is determined according to a current load condition of the terminal, where the target device is a device capable of acquiring content to be displayed on the same screen, and may be a terminal or a first on-screen receiving device directly or indirectly connected to the terminal, and a connection between the terminal and the target device is created to receive the content to be displayed on the same screen, where the terminal is a device storing the content to be displayed on the same screen, that is, a source device initiating on-screen sharing.

Specifically, the load condition of the terminal indicates that the current connection number of the terminal is less than the maximum connection number thereof, that is, the terminal can still access the device, and therefore, the terminal can be determined as a target device connected with the terminal.

When the terminal allows the access device, the terminal is used as a target device, and the connection between the terminal and the terminal is established, so that the terminal can directly acquire the content to be displayed on the same screen from the terminal, and the content is not acquired by forwarding through other devices, thereby ensuring higher on-screen efficiency. When the terminal does not allow to access the device, one of the same-screen receiving devices directly or indirectly connected with the terminal needs to be selected as a target device, and the terminal cannot directly acquire the content to be displayed on the same screen from the terminal and needs to forward the content to be displayed on the same screen through the target device.

202. Sending a first request message for requesting the current connection state of a first type of equipment allowed to be accessed in a first on-screen receiving device to a remote data server, wherein the current connection number of the first type of equipment is less than the maximum connection number of the first type of equipment;

in this embodiment, there is a server, where a current connection status of the first on-screen receiving device is stored in the server, and when the current load status of the on-screen sending device indicates that the current connection number of the on-screen sending device is equal to the maximum connection number of the on-screen sending device, the determining the target device connected to the on-screen receiving device may include: sending a first request message for requesting the current connection state of a first type of equipment allowed to be accessed in the first on-screen receiving equipment to the server, wherein the current connection number of the first type of equipment is less than the maximum connection number of the first type of equipment; receiving the current connection status of all the devices in the first class sent by the server in response to the first request message; and determining target equipment connected with the on-screen receiving equipment according to the current connection conditions of all the equipment in the first class of equipment.

203. Acquiring the current connection status of all the devices in the first type of device based on the first request message;

in this embodiment, after the creating of the connection between the terminal and the target device, the method may further include: and sending the current connection state of the terminal to the server so that the server stores the current connection state of the terminal and updates the current connection state of the target device which is stored currently.

After the terminal establishes the connection with the target device, the terminal becomes the on-screen receiving device directly or indirectly connected with the on-screen sending device, therefore, the related information stored in the server needs to be updated, and the terminal can send the current connection state of the terminal to the server, so that the server stores the current connection state of the terminal and updates the current connection state of the target device stored currently according to the current connection state of the terminal.

204. Determining target equipment connected with the same-screen receiving equipment according to the current connection conditions of all the equipment;

in this embodiment, the current connection status of the device may include: the determining of the target device connected to the on-screen receiving device may specifically be: determining allowable connection coefficients of all the devices in the first class of devices according to current connection conditions of all the devices in the first class of devices, wherein the allowable connection coefficients are inversely proportional to the connection delay and the connection hop count; and determining one device from the first class devices as the target device according to the allowed connection coefficients of all the devices in the first class devices.

In the above-mentioned multi-device on-screen sharing method, in order to select, as a target device, a device with the minimum loss caused when obtaining content to be on-screen from on-screen receiving devices that are directly or indirectly connected to the on-screen transmitting device and are allowed to access, one of the devices may be selected as the target device according to current connection conditions of the devices, such as connection delay, connection hop count, and the like, where the smaller the connection delay, the smaller the connection hop count, the smaller the loss caused when obtaining content to be on-screen, and the connection hop count of the target device is represented by the number of devices that pass through a link between the target device and the on-screen transmitting device, which represents the distance between the target device and the on-screen transmitting device, and the smaller the connection hop count, which indicates that the target device is closer to the on-screen transmitting device.

205. Receiving a same-screen request sent by a terminal, and sending a confirmation request based on the same-screen request;

206. when the terminal is detected to be in the same-screen sharing state, establishing a remote same-screen interaction interface of the terminal and target equipment to be connected with the same-screen receiving equipment;

207. receiving operation tasks sent by a preset remote data server, and sequencing the operation tasks based on a preset time sequence;

208. based on the operation task, acquiring a user operation video associated with the current video link of a user from a video link preset by a remote data server and a preset data association table;

209. and displaying the medical data of the user based on the operation task and the user operation video, and displaying a processing result on a display interface so as to realize the operation synchronization of the terminal and the target equipment.

Step 205-.

Referring to fig. 3, a third embodiment of the remote on-screen method based on a browser according to the embodiment of the present invention includes:

301. receiving a same-screen request sent by a terminal, and sending a confirmation request based on the same-screen request;

302. detecting whether the on-screen sharing service of the terminal is in an on-screen sharing state;

in this embodiment, when a user watches an online video using a multimedia device sending end, such as a smart phone, in order to obtain a better experience effect, the played video is pushed or shared to a multimedia device receiving end, such as a screen of a smart television. When the video is pushed or shared to the smart television from the smart phone, the same-screen sharing service needs to be called, such as one of wireless video transmission services like AirPlay, WiDi, Miracast, DLNA and the like. When the connection establishing unit 201 detects that one or more of the above on-screen sharing services are enabled, it is determined that the multimedia device sending end and the multimedia device receiving end have established communication and are performing video on-screen sharing, and at this time, the on-screen sharing state of the smart phone is marked as the on-screen sharing state.

303. When the terminal is detected to be in a same-screen sharing state, establishing a remote same-screen interaction interface of the terminal and target equipment to be connected with the same-screen receiving equipment;

304. receiving operation tasks sent by a preset remote data server, and sequencing the operation tasks based on a preset time sequence;

305. based on the operation task, acquiring a user operation video associated with the current video link of a user from a video link preset by a remote data server and a preset data association table;

306. and displaying the medical data of the user based on the operation task and the user operation video, and displaying a processing result on a display interface so as to realize the operation synchronization of the terminal and the target equipment.

The steps 301, 303-306 in this embodiment are similar to the steps 101, 102-105 in the first embodiment, and are not repeated herein.

Referring to fig. 4, a fourth embodiment of the remote on-screen method based on a browser according to the embodiment of the present invention includes:

401. receiving a same-screen request sent by a terminal, and sending a confirmation request based on the same-screen request;

402. when the terminal is detected to be in the same-screen sharing state, establishing a remote same-screen interaction interface of the terminal and target equipment to be connected with the same-screen receiving equipment;

403. receiving operation tasks sent by a preset remote data server, and sequencing the operation tasks based on a preset time sequence;

404. acquiring network transmission speeds of a terminal and target equipment in real time;

in this embodiment, the network transmission speeds of the terminal and the target device are respectively compared with the network transmission speed ranges corresponding to various preset video image quality levels, and the video image quality levels corresponding to the terminal and the target device are screened out from the network transmission speed ranges; s2, matching the screened video quality levels corresponding to the terminal and the target equipment with standard video quality parameters corresponding to various preset video quality levels to obtain standard video quality parameters corresponding to the terminal and the target equipment, wherein the standard video quality parameters comprise a standard bit rate, a standard frame rate and a standard resolution; respectively acquiring current video image quality parameters of a terminal end and a target equipment end corresponding to a remote same-screen interaction interface; and comparing the current video image quality parameters of the acquired terminal end corresponding to the remote same-screen interactive interface with the standard video image quality parameters corresponding to the terminal end, and if the current video image quality parameters are not consistent with the standard video image quality parameters, adjusting the numerical values of the video image quality parameters to be consistent with the standard values corresponding to the video image quality parameters.

Furthermore, comparing the current video image quality parameter of the acquired target equipment end corresponding to the remote on-screen interactive interface with the standard video image quality parameter corresponding to the target equipment end, comparing the current video image quality parameter of the acquired terminal end corresponding to the remote on-screen interactive interface with the standard video image quality parameter corresponding to the terminal end, and if the current video image quality parameter is not consistent with the standard video image quality parameter, adjusting the value of the video image quality parameter to be consistent with the standard value corresponding to the video image quality parameter.

In this embodiment, after the remote on-screen interactive interface is established, the video image quality parameters of the terminal and the target device end corresponding to the remote on-screen interactive interface are adjusted in real time according to the network transmission speed, so that the video image quality parameters of the terminal and the target device end corresponding to the remote on-screen interactive interface can be matched with the network transmission speed in real time, the occurrence of video image blocking is greatly reduced, and the continuity of video images is further improved.

405. Performing pause analysis on the remote same-screen interactive interface to obtain a video end with pause phenomenon and a pause time period, and identifying equipment with pause phenomenon in the pause time period of the remote same-screen interactive interface corresponding to the terminal and the target equipment;

in the embodiment, the picture areas corresponding to the target equipment end and the target equipment end are respectively obtained from the remote same-screen interaction interfaces corresponding to the terminal end and the target equipment end; acquiring terminal side picture images corresponding to the acquisition time periods from a target equipment side picture area according to the set acquisition time periods, and acquiring target equipment side picture images corresponding to the acquisition time periods from the target equipment side picture area; and numbering the terminal side picture images corresponding to each acquisition time period according to the sequence of the acquisition time periods, comparing the terminal side picture images corresponding to each acquisition time period with the terminal side picture images corresponding to the adjacent acquisition time periods according to the numbering sequence, judging whether the terminal side picture images corresponding to the adjacent acquisition time periods are the same, if so, indicating that the terminal side has a pause phenomenon between the adjacent acquisition time periods, and marking the adjacent acquisition time periods as pause time periods.

Further, numbering the target equipment terminal picture images corresponding to each acquisition time period according to the sequence of the acquisition time periods, comparing the target equipment terminal picture images corresponding to adjacent acquisition time periods with the target equipment terminal picture images corresponding to the adjacent acquisition time periods according to the numbering sequence, judging whether the target equipment terminal picture images corresponding to the adjacent acquisition time periods are the same, if so, indicating that the target equipment terminal has a pause phenomenon between the adjacent acquisition time periods, and marking the adjacent acquisition time periods as pause time periods; therefore, the video end with the pause phenomenon in the pause time period of the pause time is identified for the remote same-screen interactive interface corresponding to the terminal end and the target equipment end so as to remind both the same-screen video end of paying attention.

406. Based on the identification, acquiring a user operation video associated with the current video link of the user from a video link preset by the remote data server and a preset data association table;

in this embodiment, in the process of analyzing the stuck phenomenon on the remote same-screen interactive interface corresponding to the terminal and the target device, a mode of comparing the terminal picture image corresponding to each acquisition time period with the terminal picture image corresponding to the adjacent acquisition time period and comparing the target device picture image corresponding to each acquisition time period with the target device picture image corresponding to the adjacent acquisition time period is adopted, because when the stuck phenomenon does not occur, the terminal picture image and the target device picture image in each acquisition time period are rarely identical, and when the stuck phenomenon occurs, the picture images are static and fixed, the terminal picture image or the target device picture image corresponding to each acquisition time period acquired by the picture images are identical, and the analysis result is more accurate and reliable by adopting the stuck phenomenon analysis mode. The adopted identification mode can be character identification and can be voice prompt identification.

407. And displaying the medical data of the user based on the operation task and the user operation video, and displaying a processing result on a display interface so as to realize the operation synchronization of the terminal and the target equipment.

The

steps

401, 403, 407 in the present embodiment are similar to the

steps

101, 103, 105 in the first embodiment, and are not described herein again.

In the embodiment of the invention, the on-screen request sent by the terminal is received, and the confirmation request is sent based on the on-screen request; when the terminal is detected to be in a same-screen sharing state, a remote same-screen interaction interface is established with target equipment; receiving operation tasks sent by a remote data server, and sequencing the operation tasks based on a preset time sequence; based on the operation task, acquiring a user operation video associated with the current video link of a user from a video link and data association table preset by a remote data server; and displaying the medical data of the user based on the operation task and the user operation video, and displaying a processing result on a display interface so as to realize the sharing of the content to be displayed on the same screen corresponding to the terminal and the target equipment. The invention realizes remote one-screen between the customer service and the user equipment by matching the operation flow of watching the user with low time delay and voice or video, solves the problems of unclear description of the user problems or deviation of understanding of the customer service and the like, and improves the working efficiency of the customer service.

Referring to fig. 5, a fifth embodiment of the remote on-screen method based on a browser according to the embodiment of the present invention includes:

501. receiving a same-screen request sent by a terminal, and sending a confirmation request based on the same-screen request;

502. when the terminal is detected to be in the same-screen sharing state, establishing a remote same-screen interaction interface of the terminal and target equipment to be connected with the same-screen receiving equipment;

503. receiving operation tasks sent by a preset remote data server, and sequencing the operation tasks based on a preset time sequence;

504. acquiring the network transmission speed of the terminal and the target equipment in real time;

505. respectively acquiring a target device and a picture area corresponding to the target device from a remote same-screen interaction interface corresponding to the terminal and the target device;

in this embodiment, the target device and the picture area corresponding to the target device are respectively obtained from the terminal and the remote on-screen interaction interface corresponding to the target device.

During the process of analyzing the karton phenomenon of the remote same-screen interaction interface corresponding to the terminal and the target equipment, the terminal picture images corresponding to the acquisition time periods are subjected to picture image comparison corresponding to the adjacent acquisition time periods.

506. Acquiring a terminal picture image corresponding to a first preset time period from a picture area of the target equipment according to a set acquisition time period, and acquiring a picture image of the target equipment corresponding to the first preset time period from the picture area of the target equipment;

in this embodiment, according to a set acquisition time period, a terminal image corresponding to a first preset time period is acquired from the target device image area, and a target device image corresponding to the first preset time period is acquired from the target device image area.

Similarly, in the process of analyzing the karton phenomenon of the remote same-screen interactive interface corresponding to the terminal and the target equipment, the terminal picture images corresponding to the acquisition time periods are subjected to picture image comparison corresponding to the adjacent acquisition time periods.

507. Numbering the terminal picture images corresponding to the first preset time period according to the sequence of the acquisition time periods, carrying out terminal picture image comparison corresponding to the second preset time period on the terminal picture images corresponding to the first preset time period, and judging whether the terminal picture images corresponding to the second preset time period are the same or not;

if the time intervals are the same, indicating that the terminal has a pause phenomenon in a second preset time interval, and recording the second preset time interval as the pause time interval;

in this embodiment, it is determined whether the terminal picture image in the preset time period is the same as the terminal picture image in the preset adjacent time period. Specifically, numbering the terminal picture images corresponding to the first preset time period according to the sequence of the acquisition time periods, performing terminal picture image comparison corresponding to the second preset time period on the terminal picture images corresponding to the first preset time period, and judging whether the terminal picture images corresponding to the second preset time period are the same or not; if the time intervals are the same, the fact that the terminal has a pause phenomenon in the second preset time interval is indicated, and the second preset time interval is marked as a pause time interval.

508. Numbering the target equipment picture images corresponding to the first preset time period according to the sequence of the acquisition time periods, carrying out picture image comparison on the target equipment picture images corresponding to the first preset time period and the second preset time period, and judging whether the target equipment picture images corresponding to the second preset time period are the same or not;

if the time intervals are the same, the target device is indicated to have a pause phenomenon between second preset time intervals, and the second preset time intervals are recorded as pause time intervals.

In this embodiment, it is determined whether the target device side picture image in the preset time period is the same as the target device side picture image in the preset adjacent time period. Specifically, the target device side picture images corresponding to the first preset time period are numbered according to the sequence of the acquisition time periods, the target device side picture images corresponding to the first preset time period are subjected to image comparison of the target device side picture images corresponding to the second preset time period, and whether the target device side picture images corresponding to the second preset time period are the same or not is judged; if the time intervals are the same, the target device end is indicated to have a pause phenomenon between the second preset time intervals, and the second preset time intervals are recorded as pause time intervals.

509. And displaying the medical data of the user based on the operation task and the user operation video, and displaying a processing result on a display interface so as to realize the operation synchronization of the terminal and the target equipment.

The steps 501-503 and 509 in this embodiment are similar to the steps 101-103 and 105 in the first embodiment, and are not described herein again.

In the above description of the remote on-screen method based on the browser in the embodiment of the present invention, referring to fig. 6, a remote on-screen device based on the browser in the embodiment of the present invention is described below, where a first embodiment of the remote on-screen device based on the browser in the embodiment of the present invention includes:

a sending module 601, configured to receive a co-screen request sent by a terminal, and send a confirmation request based on the co-screen request;

a first detection module 602, configured to establish a remote on-screen interaction interface between the terminal and a target device to be connected to an on-screen receiving device when it is detected that the terminal is in an on-screen sharing state;

the sequencing module 603 is configured to receive an operation task sent by a preset remote data server, and sequence the operation task based on a preset time sequence;

an obtaining module 604, configured to obtain, based on the operation task, a user operation video associated with the current video link by the user from a video link and preset data association table preset by the remote data server;

a display module 605, configured to perform display processing on the medical data of the user based on the operation task and the user operation video, and display a processing result on a display interface, so as to implement operation synchronization between the terminal and the target device.

In the embodiment of the invention, a same-screen request sent by a terminal is received, and a confirmation request is sent based on the same-screen request; when the terminal is detected to be in a same-screen sharing state, a remote same-screen interaction interface is established with target equipment; receiving operation tasks sent by a remote data server, and sequencing the operation tasks based on a preset time sequence; based on the operation task, acquiring a user operation video associated with the current video link of a user from a video link and data association table preset by a remote data server; and displaying the medical data of the user based on the operation task and the user operation video, and displaying a processing result on a display interface so as to realize the sharing of the content to be displayed on the same screen corresponding to the terminal and the target equipment. According to the invention, through the low-delay operation flow of watching the user and matching with voice or video, remote one-screen interaction between the customer service and the user equipment is realized, the problems that the user problem description is unclear or the customer service understanding has deviation and the like are solved, and the working efficiency of the customer service is improved.

Referring to fig. 7, a second embodiment of the remote on-screen device based on a browser according to the present invention specifically includes:

the first detection module 602 is configured to, when it is detected that the terminal is in a same-screen sharing state, establish a remote same-screen interaction interface between the terminal and a target device to be connected to a same-screen receiving device;

In this embodiment, the remote on-screen device based on a browser further includes:

a receiving module 606, configured to receive a current load condition of the terminal;

a first determining module 607, configured to determine, according to the current load condition of the terminal, a target device connected to the terminal.

In this embodiment, the first determining module 607 is specifically configured to:

sending a first request message for requesting the current connection state of first-class equipment allowed to be accessed in the first on-screen receiving equipment to the remote data server, wherein the current connection number of the first-class equipment is less than the maximum connection number of the first-class equipment;

acquiring the current connection status of all the devices in the first type of device based on the first request message;

and determining the target equipment connected with the on-screen receiving equipment according to the current connection conditions of all the equipment.

a second detecting module 608, configured to detect whether a same-screen sharing service of the terminal is in a same-screen sharing state;

a second determining module 609, configured to determine that the terminal is in the same-screen sharing state if the terminal is in the same-screen sharing state.

In this embodiment, the obtaining module 604 includes: an acquisition unit 6041 configured to acquire network transmission speeds of the terminal and the target device in real time; an identification unit 6042, configured to perform a stuck analysis on the remote on-screen interaction interface to obtain a video end and a stuck time period in which a stuck phenomenon occurs, and identify a device in which the stuck phenomenon occurs in the stuck time period on the remote on-screen interaction interface corresponding to the terminal and the target device; an obtaining unit 6043, configured to obtain, based on the identifier, a user operation video associated with the current video link by the user from a video link and preset data association table preset by the remote data server.

In this embodiment, the identifying unit 6042 is specifically configured to:

respectively acquiring the target equipment and the picture area corresponding to the target equipment from the remote co-screen interaction interface corresponding to the terminal and the target equipment;

acquiring a terminal picture image corresponding to a first preset time period from the picture area of the target equipment according to a set acquisition time period, and acquiring a picture image of the target equipment corresponding to the first preset time period from the picture area of the target equipment;

numbering the terminal picture images corresponding to the first preset time period according to the sequence of the acquisition time periods, carrying out terminal picture image comparison corresponding to the second preset time period on the terminal picture images corresponding to the first preset time period, and judging whether the terminal picture images corresponding to the second preset time period are the same or not.

In this embodiment, the identifying unit 6042 is further configured to:

if the time intervals are the same, indicating that the terminal has a pause phenomenon between the second preset time intervals, and recording the second preset time intervals as pause time intervals;

numbering the target equipment picture images corresponding to the first preset time period according to the sequence of the acquisition time periods, carrying out picture image comparison on the target equipment picture images corresponding to the first preset time period and the target equipment picture images corresponding to the second preset time period, and judging whether the target equipment picture images corresponding to the second preset time period are the same or not;

if the time intervals are the same, indicating that the target device has a pause phenomenon between the second preset time intervals, and recording the second preset time intervals as pause time intervals.

Fig. 6 and 7 describe the browser-based remote on-screen device in the embodiment of the present invention in detail from the perspective of the modular functional entity, and the browser-based remote on-screen device in the embodiment of the present invention is described in detail from the perspective of hardware processing.

Fig. 8 is a schematic structural diagram of a remote browser-based on-screen device 800 according to an embodiment of the present invention, where the remote browser-based on-screen device 800 may have a relatively large difference due to different configurations or performances, and may include one or more processors (CPUs) 810 (e.g., one or more processors) and a memory 820, one or more storage media 830 (e.g., one or more mass storage devices) storing an application 833 or data 832. Memory 820 and storage medium 830 may be, among other things, transient or persistent storage. The program stored on the storage medium 830 may include one or more modules (not shown), each of which may include a sequence of instruction operations for the browser-based remote on-screen device 800. Further, the processor 810 may be configured to communicate with the storage medium 830, and execute a series of instruction operations in the storage medium 830 on the browser-based remote on-screen device 800 to implement the steps of the browser-based remote on-screen method provided by the above-mentioned method embodiments.

The browser-based remote on-screen device 800 may also include one or more power supplies 840, one or more wired or wireless network interfaces 850, one or more input-output interfaces 860, and/or one or more operating systems 831, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, and the like. Those skilled in the art will appreciate that the browser-based remote co-screen device configuration shown in fig. 8 does not constitute a limitation of the browser-based remote co-screen devices provided herein, and may include more or fewer components than shown, or some components in combination, or a different arrangement of components.

The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium, and may also be a volatile computer-readable storage medium, where instructions are stored, and when the instructions are executed on a computer, the instructions cause the computer to perform the steps of the above-mentioned browser-based remote co-screen method.

It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a read-only memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.

The above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.

Claims

1. A remote screen-sharing method based on a browser is characterized by comprising the following steps:

receiving a same-screen request sent by a terminal, and sending a confirmation request based on the same-screen request;

when the terminal is detected to be in the same-screen sharing state, establishing a remote same-screen interaction interface of the terminal and target equipment to be connected with the same-screen receiving equipment;

receiving an operation task sent by a preset remote data server, and sequencing the operation task based on a preset time sequence, wherein the operation task is a task generated based on the operation of medical data of a user;

based on the operation task, acquiring a user operation video associated with the current video link of a user from a video link preset by the remote data server and a preset data association table;

and displaying the medical data of the user based on the operation task and the user operation video, and displaying a processing result on a display interface so as to realize the operation synchronization of the terminal and the target equipment.

2. The remote browser-based on-screen method according to claim 1, wherein before receiving the on-screen request sent by the terminal and sending an acknowledgement request based on the on-screen request, the method further comprises:

receiving the current load condition of the terminal;

and determining target equipment connected with the terminal according to the current load condition of the terminal.

3. The browser-based remote on-screen method according to claim 2, wherein the determining the target device connected to the terminal according to the current load condition of the terminal comprises:

sending a first request message for requesting the current connection state of a first type of equipment allowed to be accessed in the first on-screen receiving equipment to the remote data server, wherein the current connection number of the first type of equipment is less than the maximum connection number of the first type of equipment;

4. The remote on-screen method based on the browser of claim 1, wherein before the establishing of the remote on-screen interaction interface between the terminal and the target device to be connected to the on-screen receiving device when the terminal is detected to be in the on-screen sharing state, the method further comprises:

detecting whether the on-screen sharing service of the terminal is in an on-screen sharing state;

and if so, determining that the terminal is in the same-screen sharing state.

5. The remote browser-based on-screen method according to claim 1, wherein the obtaining, based on the operation task, a user operation video associated with the current video link by the user from a video link preset by the remote data server and a preset data association table comprises:

acquiring the network transmission speed of the terminal and the target equipment in real time;

performing pause analysis on the remote same-screen interactive interface to obtain a video end with pause phenomenon and a pause time period, and identifying equipment with pause phenomenon in the pause time period of the remote same-screen interactive interface corresponding to the terminal and the target equipment;

and acquiring a user operation video associated with the current video link by the user from a video link preset by the remote data server and a preset data association table based on the identification.

6. The remote on-screen method based on the browser of claim 5, wherein the performing of the katon analysis on the remote on-screen interactive interface to obtain a video end and a katon time period in which a katon phenomenon occurs, and the identifying of the device in which the katon phenomenon occurs in the remote on-screen interactive interface corresponding to the terminal and the target device in the katon time period comprises:

7. The remote on-screen method based on the browser of claim 6, wherein after performing the terminal image comparison corresponding to the second preset time period on the terminal image corresponding to the first preset time period and determining whether the terminal image corresponding to the second preset time period is the same, the method further comprises:

8. A remote browser-based on-screen device, comprising:

the sending module is used for receiving a same-screen request sent by a terminal and sending a confirmation request based on the same-screen request; terminal device

The first detection module is used for establishing a remote on-screen interaction interface between the terminal and target equipment to be connected with the on-screen receiving equipment when the terminal is detected to be in an on-screen sharing state;

the sequencing module is used for receiving operation tasks sent by a preset remote data server and sequencing the operation tasks based on a preset time sequence;

the acquisition module is used for acquiring a user operation video related to the current video link of a user from a video link and preset data association table preset by the remote data server based on the operation task;

and the display module is used for displaying and processing the medical data of the user based on the operation task and the user operation video and displaying a processing result on a display interface so as to realize the operation synchronization of the terminal and the target equipment.

9. A browser-based remote on-screen device, comprising: a memory having instructions stored therein and at least one processor, the memory and the at least one processor interconnected by a line;

the at least one processor invoking the instructions in the memory to cause the browser-based remote co-screen device to perform the steps of the browser-based remote co-screen method of any one of claims 1-7.

10. A computer-readable storage medium, having a computer program stored thereon, wherein the computer program, when executed by a processor, performs the steps of the browser-based remote co-screen method of any one of claims 1-7.