JP6829359B2 - Screen sharing and remote control system - Google Patents

Description

The present invention relates to a screen sharing and remote control system capable of displaying a screen displayed on one terminal on the other terminal and enabling operation between different terminals.

In recent years, many businesses that provide products and services (hereinafter referred to as "operators") publish web pages for the purpose of selling or promoting the products and services that they provide. A user who browses the operator's web page (hereinafter referred to as "user") has a telephone line such as a toll-free number set up by the operator when there is any doubt about the content or operation method on the web page. It is common to make inquiries to the operator through.

However, if the content of the inquiry from the user or the answer to the inquiry from the operator includes visual information such as the operation method on the web page, the operator can only transmit the information by voice only through the telephone line. , There is a possibility that it takes time to grasp the contents of inquiries from users, and that users take time to understand the answers to inquiries from the operator.

As a method for solving the above-mentioned problems, a program for remotely controlling a user's communication terminal (hereinafter referred to as "user terminal") is provided to the user terminal by a method generally called "remote desktop". By installing in advance, the screen displayed on the user terminal is shared with the screen displayed on the operator's communication terminal (hereinafter referred to as "operator terminal"), and then the operator manages the operator terminal. There is a method of remotely controlling the user terminal through. However, in this method, when a user who is unfamiliar with the operation of the communication terminal is assumed, the act of installing the above-mentioned program by the user himself / herself may become a barrier.

In addition, as a method of realizing screen sharing without installing a program on the user terminal in advance, bidirectional communication such as WebRTC (Web Real-Time-Communication) is performed via a program installed on a web page that is a screen sharing source. Using communication technology, the screen displayed on the web browser of the user terminal (hereinafter referred to as "user browser") is converted into video format data, and the data is converted into the web browser of the operator terminal (hereinafter referred to as "operation"). There is also a method of sharing to the screen displayed on the operator browser by sending it to the "person browser". However, with this method, the screen display method shared by the operator browser is in the video format, so if the bandwidth of the user terminal or the network connected to the operator terminal is not stable, the image quality will deteriorate. There is a possibility that it will be delayed or a display delay will occur.

There are other ways to share the screen, but if the content of the inquiry from the user is related to the operation method on the web page, the operator should remotely control the web page displayed on the user browser. Therefore, it is still necessary to respond to inquiries from users by transmitting information only by voice (see, for example, Patent Document 1).

JP-A-2018-197891

The present invention takes into consideration the above-mentioned prior art, can visually promote information transmission between both the operator and the user, and makes it possible to share the screen displayed on the user browser through the operator browser. It is an object of the present invention to provide a screen sharing and remote control system that allows an operator to remotely control a web page displayed on a user browser.

In order to achieve the above object, in the present invention, a user browser for displaying the contents of a web page, a relay server connected to the user browser via a network communication network, and the relay server and the network communication. It is provided with an operator browser connected via a network and a shared processing database for transmitting, receiving and storing HTML structure data possessed by the user browser and the operator browser via the relay server. Provide a featured screen sharing and remote control system.

Preferably, the shared processing database can send, receive, and store user operation data performed by the user browser and operator operation data performed by the operator browser via the relay server.

Preferably, the shared processing database is used when transmitting and receiving the HTML structure data, the user operation data, and the operator operation data between the user browser and the operator browser. This is for surely storing the HTML structure data, the user operation data, and the operator operation data via the relay server and then transmitting the data to the other party's web browser.

Preferably, the relay server is connected to a service database having a setting table in which a portion prohibiting sharing of the information input by the user browser with the operator browser is set in advance.

According to the present invention, for example, information transmission in both the user browser and the operator browser used by the user and the operator can be visually promoted, and the screen displayed on the user browser is shared through the operator browser. It is possible to do. At this time, since HTML structure data is transmitted and received for mutual screen sharing, the bandwidth of the network connected to the user terminal or the operator terminal is not stable by significantly reducing the size of the transmitted data. Even in this case, the possibility of display delay can be reduced without degrading the image quality.

Further, since the operation data in the user browser and the operator browser can be transmitted and received to each other, the user operation in the user browser can be reproduced through the operator browser. Therefore, for example, when the user scrolls. Since the operator browser scrolls to the position where the user scrolls, the operator can accurately browse the display area of the web page displayed on the user browser. In addition, for example, when the user operates the keyboard, the content entered by the user is reflected in the operator browser, so the operator makes an inquiry from the user while checking the content entered by the user. Can be answered. Also, for example, when the user operates the mouse cursor, the position of the mouse cursor moved by the user is displayed on the operator browser, so that the operator tries to operate any part of the web page by the user. You can figure out if you are there.

Further, being able to send and receive this operation data to and from each other means that the operation of the other party can be reproduced on its own screen. That is, since the operator can remotely control the web page displayed on the user browser, even if the content of the inquiry from the user is related to the operation method on the web page, the user browser is actually used. It is possible to respond to inquiries from users while remotely controlling the displayed web page. This is to provide a technology that enables the operator to remotely control the web page displayed on the user browser. In this way, the operator can add remote control to the web page displayed on the user browser. Therefore, for example, the operator can inquire from the user about the operation method on the web page. If there is, you can reply while actually adding remote control to the web page displayed on the user's browser. Therefore, the user is promoted to understand the operation method on the web page.

In addition, instead of directly transmitting the sharing processing data (HTML structure data, user operation data, and operator operation data) to the user browser or the operator browser using the sharing processing database, the sharing process is once performed. After recording (storing) the data for sharing processing in the database, it is sent to the user browser or the operator browser (the other party's web browser). As a result, for example, when a large processing load is applied to the relay server, the processing load can be distributed by increasing the number of relay servers. That is, the shared processing database is for temporarily recording (storing) the shared processing data. The configuration requirement of a database for shared processing will be effective by having such a technical idea.

In addition, since the service database has a setting table, the operator can change the design of the UI expansion part displayed on the user browser, and in consideration of the user's security, the secret by the user. It is possible to reflect the setting that prohibits sharing of highly likely data input operations.

It is the schematic of the screen sharing and remote control system which concerns on this invention. It is a figure which shows an example of the functional structure of a program executed by a user browser. It is a figure which shows an example of the functional structure of a relay server. It is a figure which shows an example of the functional structure of the program executed by the operator browser. It is a figure which shows an example of the data item of the service database. It is a figure which shows an example of the data item of the database for shared processing. It is a sequence diagram which shows an example of the processing procedure performed by the screen sharing and remote control system which concerns on this invention. It is a sequence diagram which shows an example of the processing procedure performed by the screen sharing and remote control system which concerns on this invention. It is a sequence diagram which shows an example of the processing procedure performed by the screen sharing and remote control system which concerns on this invention. It is a sequence diagram which shows an example of the processing procedure performed by the screen sharing and remote control system which concerns on this invention. It is the schematic which shows an example of the screen in the user browser in S05 of FIG. It is the schematic which shows an example of the screen in the operator browser in S06 of FIG. It is the schematic which shows an example of the screen in the operator browser in S11 of FIG. It is the schematic which shows an example of the screen in the user browser in S12 of FIG. It is the schematic which shows an example of the screen in the operator browser in S21 of FIG. It is the schematic which shows an example of the screen in the operator browser in S26 of FIG. It is the schematic which shows an example of the screen in the user browser in S31 of FIG.

Preferred embodiments of the present invention will be described with reference to the accompanying drawings. In each figure, those having the same reference numerals have the same or similar configurations.
The screen sharing and remote control system described below is a screen displayed on the user browser used by the user on a web page that allows a simple calculation of the payment amount of the life insurance service provided to general consumers. Will be described on the premise that screen sharing and remote control are performed through the operator browser used by the operator, but the present embodiment is not intended to be limited to this. The screen sharing and remote control system according to the present embodiment can be used in any form as long as the screen displayed on the web browser of the communication terminal is shared and remotely controlled through the web browser of another communication terminal. It is possible to apply.

<System configuration>
As shown in FIG. 1, this system includes a user browser 10, a relay server 20, an operator browser 30, a service database 40, and a shared processing database 50. Between the user browser 10 and the relay server 20, between the relay server 20 and the operator browser 30, between the relay server 20 and the service database 40, and between the relay server 20 and the shared processing database 50. Are connected by a network communication network that is wireless or wired, or a combination of wireless and wired, respectively.

The user browser 10 is an application for performing web browsing that operates on the user terminal. The user browser 10 is an application installed or installed in a communication terminal such as a personal computer, a tablet, or a smartphone. The user browser 10 is a user for transmitting / receiving data to / from the relay server 20 via a program installed in advance on the screen sharing source web page in a state where the screen sharing source web page is developed. It has a function to develop an interface (hereinafter referred to as "UI"), a function to execute various processes such as data acquisition on a web page serving as a screen sharing source, and a function to send and receive data to and from the relay server 20. The program installed in advance on the web page that is the screen sharing source includes a unique identifier (hereinafter, referred to as "operator ID") for identifying the operator.

The relay server 20 has a function of transmitting and receiving data with the user browser 10 and a function of transmitting and receiving data with the operator browser 30.

The operator browser 30 is an application for performing web browsing that operates on the operator terminal. The operator browser 30 is an application installed or installed in a communication terminal such as a personal computer, a tablet, or a smartphone. The operator browser 30 is a UI for transmitting and receiving data to and from the relay server 20 via a program installed in advance on the screen sharing destination web page in a state where the screen sharing destination web page is developed. It has a function of deploying data, a function of executing various processes such as data acquisition on a web page to be screen-shared, and a function of transmitting and receiving data to and from the relay server 20.

The service database 40 includes information for managing operators, information for managing settings for each operator in providing screen sharing and remote control services (hereinafter referred to as "operator setting data"). , And a database that stores information for managing the connection between the user browser 10 and the operator browser 30.

The sharing processing database 50 is a database that stores information for processing screen sharing and remote control between the user browser 10 and the operator browser 30.

<Functional configuration>
(User browser)
As shown in FIG. 2, the user browser 10 includes a UI expansion unit 101, a processing unit 102, and a communication unit 103. These functions are executed via a program installed in advance on the web page that is the screen sharing source.

The UI expansion unit 101 is a UI that is expanded on the screen sharing source web page displayed by the user browser 10. The UI expansion unit 101 is displayed as a UI for transmitting / receiving data to / from the relay server 20 via the communication unit 103.

The processing unit 102 has a function of executing various processes such as data acquisition on a web page that is a screen sharing source. The processing unit 102 includes an operator ID acquisition unit 1021, an operator setting data processing unit 1022, an HTML structure data acquisition unit 1023, a user operation data acquisition unit 1024, and an operator operation data processing unit 1025.

The communication unit 103 has a function of transmitting and receiving data to and from the relay server 20. The communication unit 103 includes an operator ID transmission unit 1031, an operator setting data reception unit 1032, a connection ID transmission unit 1033, an HTML structure data transmission unit 1034, a user operation data transmission unit 1035, and an operator operation data. Includes receiver 1036.

The operator ID acquisition unit 1021 has a function of acquiring an operator ID from a program installed in advance on a web page that is a screen sharing source and handing over the operator ID to the operator ID transmission unit 1031.

The operator setting data processing unit 1022 changes the design (shape, color arrangement, position, size, etc.) of the UI development unit 101 based on the operator setting data handed over from the operator setting data receiving unit 1032. User operation data acquisition of settings that prohibit sharing of input operations by users (for example, input operations for login passwords and credit card numbers) (for example, selectors for identifying input items that correspond to confidential information) It has a function to be reflected in the unit 1024.

The HTML structure data acquisition unit 1023 continuously monitors the HTML structure of the web page that is the screen sharing source (hereinafter, referred to as "HTML structure", and this data is referred to as "HTML structure data"), and the HTML structure data acquisition unit 1023 performs continuous monitoring of the HTML structure. Whenever a change is detected, it has a function of passing the entire HTML structure data or the change difference to the HTML structure data transmission unit 1034.

The user operation data acquisition unit 1024 continuously monitors user operations (hereinafter referred to as "user operations" and this data is referred to as "user operation data") on the web page that is the screen sharing source. It has a function of handing over the user operation data to the user operation data transmission unit 1035 each time when the user operation is detected. When the user operation data acquisition unit 1024 receives the sharing prohibition setting of the input operation by the user from the operator setting data processing unit 1022, the user operation data acquisition unit 1024 does not transmit the user operation data subject to the sharing prohibition. The operation referred to here includes all operations that can be realized through a web browser, such as mouse or touch screen operations (click, touch, hover, swipe, scroll, cursor movement, etc.) and keyboard operations (key input, etc.).

Based on the operator operation data handed over from the operator operation data receiving unit 1036, the operator operation data processing unit 1025 operates the operator on the screen sharing destination web page on the screen sharing source web page ( Hereinafter, it is referred to as "operator operation", and this data is referred to as "operator operation data") and has a function of reproducing the same operation event.

The operator ID transmission unit 1031 has a function of transmitting the operator ID handed over from the operator ID acquisition unit 1021 to the relay server 20.

The operator setting data receiving unit 1032 has a function of receiving the operator setting data from the relay server 20 and delivering the operator setting data to the operator setting data processing unit 1022.

When the user makes an inquiry to the operator by telephone or other means and requests screen sharing and remote control support, an arbitrary identifier for identifying the connection between the user browser 10 and the operator browser 30 (hereinafter referred to as an arbitrary identifier). After confirming with the operator (referred to as "connection ID"), it is necessary to input the connection ID through the UI expansion unit 101. The connection ID transmission unit 1033 has a function of transmitting the connection ID input by the user through the UI expansion unit 101 to the relay server 20.

The HTML structure data transmission unit 1034 has a function of transmitting the HTML structure data delivered from the HTML structure data acquisition unit 1023 to the relay server 20.

The user operation data transmission unit 1035 has a function of transmitting the user operation data handed over from the user operation data acquisition unit 1024 to the relay server 20.

The operator operation data receiving unit 1036 has a function of receiving the operator operation data from the relay server 20 and delivering the operator operation data to the operator operation data processing unit 1025.

(Relay server)
As shown in FIG. 3, the relay server 20 includes a user browser communication unit 201 and an operator browser communication unit 202.

The user browser communication unit 201 has a function of transmitting and receiving data to and from the user browser 10. The user browser communication unit 201 operates the operator ID relay unit 2011, the operator setting data relay unit 2012, the connection ID relay unit 2013, the HTML structure data reception unit 2014, and the user operation data reception unit 2015. The user operation data transmission unit 2016 and the web socket communication disconnection unit 2017 are included.

The operator browser communication unit 202 has a function of transmitting and receiving data to and from the operator browser 30. The operator browser communication unit 202 includes a connection ID relay unit 2021, an HTML structure data transmission unit 2022, a user operation data transmission unit 2023, an operator operation data reception unit 2024, and a web socket communication disconnection unit 2025. ..

The operator ID relay unit 2011 has a function of receiving the operator ID from the user browser 10 and transmitting the operator ID to the service database 40. As a result, it becomes possible to acquire the operator setting data associated with the operator ID in the setting table 402 of the service database 40.

The operator setting data relay unit 2012 has a function of receiving the operator setting data from the service database 40 and transmitting the operator setting data to the user browser 10.

The connection ID relay unit 2013 has a function of receiving the connection ID from the user browser 10 and transmitting the connection ID to the service database 40. As a result, in the connection table 403 of the service database 40, it is confirmed whether the connection ID is issued by the instruction of the operator browser 30, and if the connection ID is issued by the instruction of the operator browser 30, the connection ID is used. After recording an identifier (for example, Cookie or the like) for identifying the user browser 10 in the associated record, websocket communication between the user browser 10 and the relay server 20 is started.

The HTML structure data receiving unit 2014 has a function of receiving the HTML structure data from the user browser 10 and delivering the HTML structure data to the HTML structure data transmitting unit 2022.

The user operation data receiving unit 2015 has a function of receiving the user operation data from the user browser 10 and delivering the user operation data to the user operation data transmitting unit 2023.

The operator operation data transmission unit 2016 has a function of transmitting the operator operation data handed over from the operator operation data reception unit 2024 to the user browser 10.

The websocket communication disconnection unit 2017 has a function of disconnecting the websocket communication between the user browser 10 and the relay server 20.

The connection ID relay unit 2021 has a function of receiving a connection ID issuance instruction from the operator browser 30, issuing a connection ID combined with a random 6-digit number, and transmitting the connection ID to the service database 40. .. As a result, it is confirmed whether the connection ID is unique in the connection table 403 of the service database 40, and if the connection ID is unique, the operator ID, the connection ID, and the operator browser are displayed as new records in the connection table 403. After recording an identifier (for example, Cookie etc.) for identifying 30 and transmitting the connection ID to the operator browser 20, websocket communication between the relay server 20 and the operator browser 30 is started, but the connection ID If is not unique, the connection ID relay unit 2021 is instructed to issue the connection ID again.

The HTML structure data transmission unit 2022 has a function of transmitting the HTML structure data delivered from the HTML structure data reception unit 2014 to the operator browser 30.

The user operation data transmission unit 2023 has a function of transmitting the user operation data handed over from the user operation data reception unit 2015 to the operator browser 30.

The operator operation data receiving unit 2024 has a function of receiving the operator operation data from the operator browser 30 and delivering the operator operation data to the operator operation data transmitting unit 2016.

The websocket communication disconnection unit 2025 has a function of disconnecting the websocket communication between the relay server 20 and the operator browser 30.

(Operator browser)
As shown in FIG. 4, the operator browser 30 includes a UI expansion unit 301, a processing unit 302, and a communication unit 303. These functions are executed via a program installed in advance on the web page to which the screen is shared.

The UI expansion unit 301 is a UI that is expanded on the screen sharing destination web page displayed by the operator browser 30. The UI expansion unit 301 is displayed as a UI for transmitting / receiving data to / from the relay server 20 via the communication unit 303.

The processing unit 302 has a function of executing various processes such as data acquisition on the web page to be the screen sharing destination. The processing unit 302 includes an HTML structure data processing unit 3021, a user operation data processing unit 3022, and an operator operation data acquisition unit 3023.

The communication unit 303 has a function of transmitting and receiving data to and from the relay server 20. The communication unit 303 includes a connection ID issuing unit 3031, a connection ID receiving unit 3032, an HTML structure data receiving unit 3033, a user operation data receiving unit 3034, and an operator operation data transmitting unit 3035.

Based on the HTML structure data delivered from the HTML structure data receiving unit 3033, the HTML structure data processing unit 3021 displays the same display content as the HTML structure of the screen sharing source web page on the screen sharing destination web page. It has a function to draw.

The user operation data processing unit 3022 has a function of reproducing an operation event similar to the user operation on the web page to be the screen sharing destination based on the user operation data delivered from the user operation data receiving unit 3034. Have.

The operator operation data acquisition unit 3023 has a function of continuously monitoring the operator operation and handing over the operator operation data to the operator operation data transmission unit 3035 each time when the operator operation is detected. The operation referred to here includes all operations that can be realized through a web browser, such as mouse or touch screen operations (click, touch, hover, swipe, scroll, cursor movement, etc.), keyboard operations (key input, etc.).

The connection ID issuing unit 3031 has a function of instructing the relay server 20 to issue a connection ID via the UI expansion unit 301.

The connection ID receiving unit 3032 has a function of receiving the connection ID from the relay server 20 and handing over the connection ID to the UI expansion unit 301.

The HTML structure data receiving unit 3033 has a function of receiving the HTML structure data from the relay server 20 and delivering the HTML structure data to the HTML structure data processing unit 3021.

The user operation data receiving unit 3034 has a function of receiving the user operation data from the relay server 20 and delivering the user operation data to the user operation data processing unit 3022.

The operator operation data transmission unit 3035 has a function of transmitting the operator operation data handed over from the operator operation data acquisition unit 3023 to the relay server 20.

<Database>
(Service database)
In FIG. 5, the data items are shown as ER (Something-Relationship) diagrams, and the relationships between the tables are shown as “1: 1” or “1: n”. The service database 40 includes an operator table 401, a setting table 402, and a connection table 403.

The operator table 401 is a table for storing information about the operator. The operator table 401 stores the name of the operator.

The setting table 402 is a child table (1: 1) of the operator table 401, and is a table that stores settings for each operator in providing screen sharing and remote control services. The setting table 402 includes an operator ID, settings related to the UI design (shape, color scheme, position, size, etc.) developed on the user browser 10 (hereinafter referred to as "UI settings"), and input by the user. Store the setting that prohibits sharing of operations (hereinafter referred to as "sharing prohibition setting").

The connection table 403 is a child table (1: n) of the operator table 401, and is a table for managing the connection between the user browser 10 and the operator browser 30. The connection table 403 includes an operator ID, a connection ID for specifying the connection between the user browser 10 and the operator browser 30, and an identifier for specifying each of the user browser 10 and the operator browser 30 (for example,). Cookie etc.) is memorized.

(Database 50 for shared processing)
As shown in FIG. 6, the shared processing database 50 includes a shared processing table 501. Since the shared processing database 50 needs to store all the contents of processing related to screen sharing and remote control between the user browser 10 and the operator browser 30, an in-memory database that reads and writes data at high speed is adopted. It is desirable to do.

The shared processing table 501 stores the contents of processing (HTML structure data, user operation data, and operator operation data) related to screen sharing and remote control between the user browser 10 and the operator browser 30.

<Processing procedure>
The processing procedure using the present invention will be described with reference to FIGS. 7, 8, 9, and 10.

The processing procedure of this system will be described on the premise that the user browser 10 develops a web page that allows a simple calculation of the payment amount of the life insurance service provided for general consumers. First, in the user browser 10, at the same time that the program installed in advance on the web page that is the screen sharing source is completely read, the operator ID is acquired from the program through the operator ID acquisition unit 1021, and the operator ID transmission unit is used. The operator ID is handed over to 1031 and the operator ID is transmitted to the relay server 20 through the operator ID transmission unit 1031 (S01).

Subsequently, in the relay server 20, the operator ID is received from the user browser 10 through the operator ID relay unit 2011, and the operator ID is transmitted to the service database 40, so that the setting table 402 of the service database 40 is obtained. In (S02), the acquisition request of the operator setting data associated with the operator ID is made.

Subsequently, in the service database 40, when there is operator setting data associated with the operator ID received from the relay server 20 in the setting table 402, the operator associated with the operator ID is associated with the relay server 20. The setting data is transmitted (S03).

Subsequently, the relay server 20 receives the operator setting data from the service database 40 through the operator setting data relay unit 2012, and transmits the operator setting data to the user browser 10 (S04).

Subsequently, the user browser 10 receives the operator setting data from the relay server 20 through the operator setting data receiving unit 1032, delivers the operator setting data to the operator setting data processing unit 1022, and processes the operator setting data. Through section 1022, for example, as shown in FIG. 11, a button (S05) for expanding the UI is displayed based on the UI setting, and if there is a sharing prohibition setting, it is shared with the user operation data acquisition section 1024. Reflect the prohibition setting.

Subsequently, when the user makes an inquiry to the operator by means such as a telephone and requests screen sharing and remote control support, for example, as shown in FIG. 12, the operator deploys the UI of the operator browser 30. By clicking or touching the connection start button (S06) displayed through the unit 301, the connection ID issuing unit 3031 instructs the relay server 20 to issue a connection ID (S07).

Subsequently, the relay server 20 issues a connection ID combined with a random 6-digit number through the connection ID relay unit 2021 and transmits the connection ID to the service database 40 to obtain the service database 40. In the connection table 403, a record request for the connection ID is made (S08).

Subsequently, in the service database 40, if the connection ID received from the relay server 20 is unique in the connection table 403, the operator ID, the connection ID, and the operator browser 30 are specified as new records in the connection table 403. (For example, Cookie) is recorded and the connection ID is transmitted to the relay server 20 (S09), but if the connection ID received from the relay server 20 is not unique, S08 is sent to the relay server 20. Send an instruction to return to the processing procedure of.

Subsequently, the relay server 20 receives the connection ID from the service database 40 through the connection ID relay unit 2021 and transmits the connection ID to the operator browser 30 (S10). At this point, the relay server 20 establishes a session by websocket communication with the carrier browser 30.

Subsequently, the operator browser 30 receives the connection ID from the relay server 20 through the connection ID receiving unit 3032, delivers the connection ID to the UI expansion unit 301, and passes the connection ID through the UI expansion unit 301, for example, as shown in FIG. , The connection ID is displayed (S11). Subsequently, the operator transmits the connection ID displayed on the UI expansion unit 301 to the user by means such as a telephone.

Subsequently, the user inputs the connection ID transmitted from the operator through the UI expansion unit 101 of the user browser 10, for example, as shown in FIG. 14 (S12).

Subsequently, in the user browser 10, the connection ID is handed over from the UI expansion unit 101 to the connection ID transmission unit 1033, and the connection ID is transmitted to the relay server 20 through the connection ID transmission unit 1033 (S13).

Subsequently, the relay server 20 receives the connection ID from the user browser 10 through the connection ID relay unit 2013 and transmits the connection ID to the service database 40, thereby using the connection ID in the connection table 403 of the service database 40. An authentication request is made to see if the connection ID received from the user browser 10 is the connection ID issued by the instruction of the operator browser 30 (S14).

Subsequently, in the service database 40, if the connection ID received from the relay server 20 is the connection ID issued by the instruction of the operator browser 30 in the connection table 403, the user browser 10 is attached to the record associated with the connection ID. An identifier (for example, Cookie, etc.) for identifying the above is recorded, and an authentication response is sent to the relay server 20 (S15).

Subsequently, the relay server 20 establishes a session by websocket communication with the user browser 10. At this point, websocket communication between the user browser 10 and the operator browser 30 is started via the relay server 20 (S16).

Subsequently, the user browser 10 continuously monitors the HTML structure of the web page that is the screen sharing source through the HTML structure data acquisition unit 1023, and when a change in the HTML structure is detected, the HTML structure data is obtained each time. The entire HTML structure data or the change difference is passed to the transmission unit 1034, and the HTML structure data is transmitted to the relay server 20 through the HTML structure data transmission unit 1034 (S17).

Subsequently, the relay server 20 receives the HTML structure data from the user browser 10 through the HTML structure data receiving unit 2014, delivers the HTML structure data to the HTML structure data transmitting unit 2022, and shares the HTML structure data through the HTML structure data transmitting unit 2022. By transmitting the HTML structure data to the processing database 50, a recording request for the HTML structure data is made in the shared processing table 501 of the shared processing database 50 (S18).

Subsequently, in the shared processing database 50, the HTML structure data received from the relay server 20 is recorded in the shared processing table 501, and the HTML structure data is transmitted to the relay server 20 (S19).

Subsequently, the relay server 20 receives the HTML structure data from the shared processing database 50 through the HTML structure data reception unit 2014, delivers the HTML structure data to the HTML structure data transmission unit 2022, and passes the HTML structure data to the HTML structure data transmission unit 2022 through the HTML structure data transmission unit 2022. HTML structure data is transmitted to the operator browser 30 (S20).

Subsequently, the operator browser 30 receives the HTML structure data from the relay server 20 through the HTML structure data receiving unit 3033, delivers the HTML structure data to the HTML structure data processing unit 3021, and passes the HTML structure data to the HTML structure data processing unit 3021, for example, through the HTML structure data processing unit 3021. , As shown in FIG. 15, in the web page to be the screen sharing destination, the same display content as the HTML structure of the web page to be the screen sharing source is drawn based on the HTML structure data (S21).

Here, the HTML structure data is data obtained by converting the structure information and the character string information of the HTML tag for drawing on the web page into a character string. The HTML tag refers to a structural language for describing document information expressed on a web page in a tag format, and is composed of HTML elements and attributes (attribute name and attribute value). Compared to video format data that is generally used for screen sharing, HTML structure data has a lighter data size, which reduces the processing load on this system and stabilizes the processing speed of this system. There are advantages such as.

Subsequently, the user browser 10 continuously monitors the user operation through the user operation data acquisition unit 1024, and when the user operation is detected, the user operates the user operation data transmission unit 1035 each time. The operation data is delivered, and the user operation data is transmitted to the relay server 20 through the user operation data transmission unit 1035 (S22).

Subsequently, the relay server 20 receives the user operation data from the user browser 10 through the user operation data receiving unit 2015, delivers the user operation data to the user operation data transmitting unit 2023, and transmits the user operation data. By transmitting the user operation data to the shared processing database 50 through the unit 2023, a recording request of the user operation data is made in the shared processing table 501 of the shared processing database 50 (S23).

Subsequently, in the shared processing database 50, the user operation data received from the relay server 20 is recorded in the shared processing table 501, and the user operation data is transmitted to the relay server 20 (S24).

Subsequently, the relay server 20 receives the user operation data from the shared processing database 50 through the user operation data receiving unit 2015, delivers the user operation data to the user operation data transmitting unit 2023, and delivers the user operation data to the user operation data transmitting unit 2023. User operation data is transmitted to the operator browser 30 through the transmission unit 2023 (S25).

Subsequently, the operator browser 30 receives the user operation data from the relay server 20 through the user operation data receiving unit 3034, delivers the user operation data to the user operation data processing unit 3022, and processes the user operation data. Through section 3022, for example, as shown in FIG. 16, in a web page to be shared with a screen, a selection operation of a selection item (“gender” item) and an input operation of an input item (“date of birth” item) are performed. Based on such user operation data, the same operation event as the user operation is reproduced (S26).

Subsequently, the operator browser 30 continuously monitors the operator operation through the operator operation data acquisition unit 3023, and when the operator operation is detected, the operator operates the operator operation data transmission unit 3035 each time. The data is delivered, and the operator operation data is transmitted to the relay server 20 through the operator operation data transmission unit 3035 (S27).

Subsequently, the relay server 20 receives the operator operation data from the operator browser 30 through the operator operation data receiving unit 2024, delivers the operator operation data to the operator operation data transmitting unit 2016, and transmits the operator operation data. By transmitting the operator operation data to the shared processing database 50 through the unit 2016, a recording request for the operator operation data is made in the shared processing table 501 of the shared processing database 50 (S28).

Subsequently, in the shared processing database 50, the operator operation data received from the relay server 20 is recorded in the shared processing table 501, and the operator operation data is transmitted to the relay server 20 (S29).

Subsequently, the relay server 20 receives the operator operation data from the shared processing database 50 through the operator operation data receiving unit 2024, delivers the operator operation data to the operator operation data transmitting unit 2016, and delivers the operator operation data to the operator operation data transmitting unit 2016. The operator operation data is transmitted to the user browser 10 through the transmission unit 2016 (S30).

Subsequently, the user browser 10 receives the operator operation data from the relay server 20 through the operator operation data receiving unit 1036, delivers the operator operation data to the operator operation data processing unit 1025, and processes the operator operation data. Through section 1025, for example, as shown in FIG. 17, on the web page that is the screen sharing source, the operator operation is performed based on the operator operation data such as the selection operation of the selection item (item of "insurance type"). A similar operation event is reproduced (S31).

Here, the user operation data and the operator operation data, that is, the operation data are operations by the mouse or touch screen (click, touch, hover, swipe, scroll, cursor movement, etc.), keyboard operations (key input, etc.), etc. , The information of the operation event that can be realized through the web browser is converted into a character string. The operation data is composed of the name of the operation event (for example, "click" in the case of a click) and the content of the operation event (for example, in the case of a click, "screenX" or "screenY" indicating the screen coordinates where the click occurred). Will be done. At this time, there are cases where the HTML structure is rewritten as a result of the user performing an operation such as clicking. In that case, in this system, both the user operation data and the HTML structure data (difference from before the operation) are sent to the other party. Send.

The processing procedures in S17 to S31 are executed in parallel and continuously until the screen sharing and remote control support are terminated.

Then, when terminating the screen sharing and remote control support, the operator clicks or touches the connection end button (S32) displayed through the UI expansion unit 301 of the operator browser 30, for example, as shown in FIG. As a result, the relay server 20 is instructed to disconnect the websocket communication with the user browser 10 (S33).

Subsequently, the relay server 20 disconnects the websocket communication with the user browser 10 through the websocket communication disconnection unit 2017 (S34). Further, the relay server 20 disconnects the websocket communication with the operator browser 30 through the websocket communication disconnection unit 2025 (S35).

According to the processing procedure described above, the screen displayed on the user browser 10 can be shared through the operator browser 30, and the operator can remotely control the web page displayed on the user browser 10. It is possible to add. As a result, even if the content of the inquiry from the user or the response to the inquiry from the operator includes visual information such as the operation method on the web page, the operator can display the content of the inquiry from the user. It can be grasped accurately, and the user can promote understanding of the inquiry and answer from the operator.

Further, in the present embodiment, screen sharing and remote control support can be provided via programs installed in advance on the web pages developed by the user browser 10 and the operator browser 30, so that the user terminal and the operator terminal can be provided. It is not necessary to pre-install a program for remote control.

Further, in the present embodiment, the size of the transmitted data is significantly reduced by sharing the screen displayed on the user browser 10 with the operator browser 30 as HTML structure data instead of as moving image format data. Even when the band of the network to which the user terminal or the operator terminal is connected is not stable, it is possible to reduce the possibility of display delay without deteriorating the image quality. In addition, since the operator can remotely control the web page displayed on the user browser 10, even if the content of the inquiry from the user is related to the operation method on the web page, the user browser is actually used. It is possible to answer inquiries from users while remotely controlling the web page displayed in 10.

Further, in the present embodiment, the relay server 20 does not directly transmit the shared processing data (HTML structure data, user operation data, and operator operation data) to the user browser 10 or the operator browser 30. After recording the shared processing data in the shared processing database once, it is transmitted to the user browser 10 or the operator browser 30. As a result, for example, when a large processing load is applied to the relay server 20, the processing load can be distributed by increasing the number of relay servers 20. That is, the shared processing database 50 is for temporarily recording (storing) the shared processing data.

Further, in the present embodiment, by adding the setting table 402 to the service database 40, the operator can change the design of the UI expansion unit 101 displayed on the user browser 10 or consider the security of the user. As a result, it is possible to reflect the setting that prohibits the sharing of highly confidential data input operations by the user. However, the data items of the setting table 402 described above are merely examples, and are for facilitating the understanding of the present invention. Therefore, by adding settings other than the data items of the setting table 402, new settings can be made. It is also possible to reflect various settings.

10: User browser, 20: Relay server, 30: Operator browser, 40: Service database, 50: Shared processing database

Claims (4)

The user browser that displays the contents of the web page and
A relay server connected to the user browser via a network communication network,
An operator browser connected to the relay server via the network communication network, and
It is provided with a shared processing database for transmitting, receiving and storing HTML structure data possessed by the user browser and the operator browser via the relay server .
The screen sharing and remote control system , wherein the HTML structure data is data obtained by converting the structure information and the character string information of the HTML tag for drawing on a web page into a character string . The shared processing database can send, receive, and store user operation data performed by the user browser and operator operation data performed by the operator browser via the relay server. Item 2. The screen sharing and remote control system according to item 1. The shared processing database is used in the relay server when exchanging the HTML structure data, the user operation data, and the operator operation data between the user browser and the operator browser. The screen according to claim 2, wherein the HTML structure data, the user operation data, and the operator operation data are always stored and then transmitted to the other party's web browser. Shared and remote control system. The relay server is characterized in that a portion for prohibiting sharing of information input by the user browser with the operator browser is connected to a service database having a preset setting table. The screen sharing and remote control system according to any one of claims 1 to 3.