KR102624835B1 - Mobile web-based consultation system and method of using the same - Google Patents

Abstract

The present invention is a consultation system that obtains a web page of a customer terminal using the mobile web and performs consultation. When a counselor requests to share the current web page of the customer terminal, a web page sharing request event is transmitted to the customer terminal. Web page sharing module; When acceptance information of the web page sharing request event is received from the customer terminal, an iframe module that checks the existence and coordinate information of an iframe (inline frame, iframe) for the current web page; A capture processing module that captures the current web page through HTML2Canvas, and includes an iframe capture processing unit that performs capture when an iframe exists in the current web page; And it is linked with the iframe capture processing unit, and generates a composite frame capture image by uploading the iframe capture image for the iframe on the parent frame to a location corresponding to the coordinate information of the iframe, and captures the composite frame. An image processing module that transmits a URL (uniform resource locator) for the image to the counselor's terminal; The iframe capture processing unit transmits a post message from the parent frame to the iframe, and captures the iframe capture image using HTML2Canvas on the iframe. You can.

Description

Mobile web-based consultation system and method of using the same {Mobile web-based consultation system and method of using the same}

The present invention relates to a mobile web-based counseling system and a method of using the same.

As the use of the Internet becomes closely related to daily life, commercial transactions that were performed offline are rapidly changing to online.

E-commerce refers to commerce conducted electronically in an online space. In particular, in recent years, e-commerce on the Internet has been actively developed due to the rapid spread of the Internet. The emergence of the World Wide Web (sometimes referred to as the 'Web') is promoting more active commercial use of the Internet, and the hyper-media functions provided by the Web provide simple and effective products. We provide an environment that makes commerce more convenient by enabling search and ordering.

For this reason, e-commerce is considered one of the most popular network application services. And in the near future, it is expected that commerce will be carried out not only through electronic transactions in which electronic data is exchanged between computers, but also through global communication networks such as the Internet, linked to the physical transaction environment.

Online shopping malls are a representative example of e-commerce through the Internet, but in addition to general products, e-commerce systems related to commerce for intangible products or ideas such as banking, insurance, and securities have already reached a significant level.

This e-commerce system is a system composed of software and processing procedures to provide various functions, and various research and attempts are being made to create newer and more advanced processing in various fields.

Most of these propose solutions for safe payments through open networks such as the Internet, and are comprised of a variety of approaches, ranging from software methods using encryption technology to hardware methods such as smart cards. In particular, things that were previously carried out offline only due to the importance of identity verification, such as insurance contracts, are now using e-commerce systems in modern times due to the development of the Internet.

We are entering a fierce competition for customer service. In addition, in order to provide more convenient services to customers and manage resources accurately, various business operation methods and computer systems that support them are being introduced.

In the midst of this, the call center is the highest priority contact route with customers. Work at a counseling center is largely divided into inbound work and outbound work. Inbound work is where a call center counselor receives a request from a customer via phone, fax, or e-mail. This is a method of conducting consultation.

Currently, the biggest problem with such inbound work is that communication between agents and customers is not smooth. In particular, it is technically difficult to share mobile web screens currently displayed to customers in real time. Even if the data provided by the counselor is not properly shared on the customer's mobile phone screen due to network conditions or various error factors during the consultation, there is no way for the counselor to know what the customer's consultation screen is like. I'm having a hard time. In the case of inbound requests where a customer applies for consultation on a specific page of a mobile page, there is a problem that the web page screen on the mobile phone cannot be seen even if connected to a counselor.

The purpose of the present invention to solve the above problems is to enable more accurate and faster consultation by sharing the mobile web page that the customer is currently viewing with the counselor during a consultation between a customer and a counselor using a mobile phone. I would like to propose a system.

The technical challenges to be achieved in the various embodiments are not limited to the matters mentioned above, and other technical challenges not mentioned may be considered by those skilled in the art from the various embodiments described below. It can be.

In order to achieve the above object, the present invention is a consultation system that obtains a web page of a customer terminal using the mobile web and performs consultation. When a counselor requests to share the current web page of the customer terminal, a web page sharing request event is sent. A web page sharing module transmitted to the customer terminal; When acceptance information of the web page sharing request event is received from the customer terminal, an iframe module that checks the existence and coordinate information of an iframe (inline frame, iframe) for the current web page; A capture processing module that captures the current web page through HTML2Canvas, and includes an iframe capture processing unit that performs capture when an iframe exists in the current web page; And it is linked with the iframe capture processing unit, and generates a composite frame capture image by uploading the iframe capture image for the iframe on the parent frame to a location corresponding to the coordinate information of the iframe, and captures the composite frame. An image processing module that transmits a URL (uniform resource locator) for the image to the counselor's terminal; The iframe capture processing unit transmits a post message from the parent frame to the iframe, and captures the iframe capture image using HTML2Canvas on the iframe. You can.

The capture processing module includes an HTML2Canvas library unit that converts HTML elements on the current web page into a capture image and captures document.body for the current web page of the customer terminal; It further includes, wherein the iframe capture processing unit can capture document.body of the iframe and transmit it to the parent frame through the HTML2Canvas library unit.

When the first to nth iframes are confirmed in the current web page, the iframe module tags the first to nth coordinate information, which is outline coordinate information for each, and each of the coordinate information is a plurality of coordinates. It can be configured as, and for the first to nth iframes, iframe capture images can be independently generated and uploaded to the parent frame.

The counseling system includes a display processing module that displays a counselor on a web page of the customer terminal so that the customer can select one of the first to nth iframes of the customer terminal; It further includes, wherein the display processing module displays and processes the iframe selected by the counselor among the first to nth iframes in a preset manner, so that the customer can identify the displayed iframe. can be set.

The counselor web page output to the counselor terminal and the composite frame captured image are compared in a preset manner, and a preprocessing process is performed to correct the composite frame captured image to the same ratio as the counselor web page and output it to the counselor terminal. processing module; It further includes, wherein the comparison processing module renders a URL for the composite frame captured image, and based on the counselor web page, corrects pixel information of the composite frame captured image to have the same ratio as the counselor web page. can be formed.

The comparison processing module includes: a remote control unit that requests remote control authorization from the customer terminal when it is determined that the counselor web page and the composite frame captured image match; It includes, wherein the remote control unit transmits an event so that identification information about the customer is input to the customer terminal in a preset manner, and when the identification information is accurately entered from the customer terminal, remote control rights to the customer terminal This can be granted to the counselor.

In addition, the present invention is a method of using the above-described consultation system, comprising: (a) transmitting a current web page sharing request event of a customer terminal from a counselor terminal to the customer terminal; (b) confirming the existence and coordinate information of an iframe for the current web page through an iframe module; (c) In step (b), if there is no iframe in the current web page, document.body for the current web page of the customer terminal is captured through the HTML2Canvas library unit, and the current web page capture image is displayed. is created, and in step (b), if an iframe is confirmed in the current web page, the iframe capture image is captured using HTML2Canvas on the iframe through the iframe capture processing unit, and then transferred to the parent frame. Uploading to create a composite frame capture image; (d) in step (c), transmitting the URL for the current web page capture image or composite frame capture image to the counselor terminal in a preset manner; and (e) in step (d), the URL is rendered and output from the counselor terminal; A method including a may be provided.

Step (c) includes, when an iframe is confirmed in the current web page in step (b), (c1) sending an event from the parent frame to the iframe using a post message; (c2) receiving the event from the iframe, capturing document.body for the iframe using HTML2Canvas, and transmitting it to the parent frame; and (c3) generating the composite frame capture image by uploading the iframe capture image to a location corresponding to the coordinate information of the iframe among the parent frames. may include.

According to the present invention, during consultation between a customer and a counselor using a mobile phone, the mobile web page currently being viewed by the customer is shared with the counselor, thereby enabling more accurate and faster consultation. In particular, for customers who are not familiar with video consultations, guidance on which iframe to select can be provided on the customer terminal, thereby increasing customer convenience.

The effects that can be obtained from various embodiments are not limited to the effects mentioned above, and other effects not mentioned are clearly derived to those skilled in the art based on the detailed description below. It can be understood.

The above-described and other aspects, features and benefits of certain preferred embodiments of the present invention will become more apparent from the following description taken in conjunction with the accompanying drawings.
1 is a conceptual diagram of a counseling system according to an embodiment of the present invention.
Figure 2 is an overall configuration diagram of a counseling system according to an embodiment of the present invention.
Figure 3 is a schematic diagram schematically illustrating the current web page of a customer terminal to which a consultation system according to an embodiment of the present invention is applied.
Figure 4 is a schematic diagram schematically showing a customer terminal and a counselor terminal of a counseling system according to an embodiment of the present invention.
Figure 5 is a flowchart of a method using a counseling system according to an embodiment of the present invention.
FIG. 6 is a flow chart illustrating step S130 of FIG. 5 in more detail.
It should be noted that throughout the drawings, like reference numerals are used to illustrate identical or similar elements, features and structures.

Since the present invention can make various changes and have various embodiments, specific embodiments will be illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, and should be understood to include all changes, equivalents, and substitutes included in the spirit and technical scope of the present invention. While describing each drawing, similar reference numerals are used for similar components.

Terms such as first, second, A, B, etc. may be used to describe various components, but the components should not be limited by the terms. Terms are used only to distinguish one component from another. For example, a first component may be named a second component, and similarly, the second component may also be named a first component without departing from the scope of the present invention. The term “and/or” includes any of a plurality of related stated items or a combination of a plurality of related stated items.

When a component is said to be "connected" or "connected" to another component, it is understood that it may be directly connected to or connected to the other component, but that other components may exist in between. It should be. On the other hand, when it is mentioned that a component is “directly connected” or “directly connected” to another component, it should be understood that there are no other components in between.

The terms used in this application are only used to describe specific embodiments and are not intended to limit the invention. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, terms such as “comprise” or “have” are intended to designate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, but are not intended to indicate the presence of one or more other features. It should be understood that this does not exclude in advance the possibility of the existence or addition of elements, numbers, steps, operations, components, parts, or combinations thereof.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person of ordinary skill in the technical field to which the present invention pertains. Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related technology, and unless explicitly defined in the present application, should not be interpreted in an ideal or excessively formal sense. No.

Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the attached drawings.

1 is a conceptual diagram of a counseling system according to an embodiment of the present invention. Referring to FIG. 1, when iframes exist in the current web page of the customer terminal 20, more efficient inbound consultation can be conducted by sharing the screen with the counselor in real time.

This hospital assumes screen sharing or screen capture based on a mobile web page, and excludes cases where the customer terminal 20 uses a consultation-only application. This is because the consultation-only application can set its own function to enable screen capture in real time.

As shown in FIG. 1, the counselor receives the web page currently being displayed on the customer terminal 20 and can guide the customer more clearly based on this. For example, if there are currently multiple iframes that can be selected in the customer terminal 20, it is easy to explain which iframe the customer should select among them. Additionally, since the process can be followed in the same way as the customer selects (or clicks) the iframe, customer understanding can be improved.

Figure 2 is an overall configuration diagram of a counseling system according to an embodiment of the present invention. Referring to Figure 2, the overall configuration of the present inventor's consultation system will be described.

The consultation system 100 includes a web page sharing module 110, an iframe module 120, a capture processing module 130, an image processing module 140, a display processing module 150, and a comparison processing module 160. do.

The web page sharing module 110 is configured to manage the transmission of a web page sharing request event to the customer terminal when a counselor requests to share the current web page of the customer terminal 20. The counselor terminal 10 may include an icon called 'View customer screen' (see FIG. 4), and when the counselor selects the icon, a web page sharing request event is transmitted to the customer terminal 20. If the customer who received the event accepts it in a preset manner, web page information of the customer terminal 20 may be transmitted to the counselor terminal 10.

When acceptance information of a web page sharing request event is received from the customer terminal 20, the iframe module 120 is configured to check the existence and coordinate information of an iframe (inline frame, iframe) for the current web page.

First to nth iframes may be included in the current web page. As an example, an iframe can be understood as an icon that can be selected by a customer. If the customer selects a specific iframe, the browser can retrieve an external page from the specified URL and embed it in the current page when it encounters the iframe element. Embedded pages are displayed according to the size and settings of the iframe element, and when the user selects an iframe, the URL corresponding to the iframe can be rendered.

For reasons such as security policy, it is not possible to take a screenshot or capture an iframe from another domain, so the iframe module 120 is used to individually process iframes within the parent frame. After forming each as an iframe capture image, each of them can be uploaded to the initial location on the parent frame.

Figure 3 is a schematic diagram schematically illustrating the current web page of a customer terminal to which a consultation system according to an embodiment of the present invention is applied. The description will be made with reference to Figure 3.

The capture processing module 130 includes an iframe capture processing unit 131 and an HTML2Canvas library unit 132. The iframe capture processing unit 131 is configured to capture an iframe when it is identified in the current web page. It is configured to capture document.body of the iframe and transmit it to the parent frame through the HTML2Canvas library unit 132.

More specifically, the iframe capture processing unit 131 can select the corresponding iframe using JavaScript and capture the iframe using HTML2Canvas. iframe.contentDocument.body can represent the document inside the iframe, so use it. To capture this, you can draw an image using the 'canvas' element of HTML5, or save and use it in the required format.

The HTML2Canvas library unit 132 is configured to convert HTML elements on the current web page into a capture image and capture document.body for the current web page of the customer terminal. That is, the iframe capture processing unit 131 is configured to render the corresponding iframe, and capturing in this process can be performed through the HTML2Canvas library unit 132.

The image processing module 140 is linked with the iframe capture processing unit 131 and generates a composite frame capture image by uploading the iframe capture image for the iframe on the parent frame to a location corresponding to the coordinate information of the iframe. It is configured to do so.

That is, a composite frame capture image for a specific viewpoint of the customer terminal 20 is generated by the image processing module 140. Here, the composite frame may mean a state in which the parent frame and the first to nth iframes are composited. The image processing module 140 transmits a URL (uniform resource locator) for the composite frame captured image to the counselor terminal, so that the composite frame captured image can be output to the counselor terminal 10.

The counseling system 100 according to the present invention may include an integrated server (not shown), and the integrated server may be provided between the customer terminal 20 and the counselor terminal 10.

The display processing module 150 is configured by the counselor to display one of the first to nth iframes of the customer terminal 20 on the web page of the customer terminal so that the customer can select one. As an example, a customer may not be sure which item (meaning an iframe) to select in order to apply for the service he or she wants, so the counselor helps the customer apply for the correct service during the video consultation. I can guide you. To this end, the counselor is configured to select any one of the first to nth iframes, and the selected iframe may be displayed on the customer terminal 20 to be identifiable.

Figure 4 is a schematic diagram schematically showing a customer terminal and a counselor terminal of a counseling system according to an embodiment of the present invention. The description will be made with reference to FIG. 4 .

The comparison processing module 160 is configured to compare the customer's current web page and the counselor's current web page, and is configured to automatically perform the comparison for more accurate and faster consultation. By providing the comparison processing module 160, the present invention allows the counselor to receive the current problem situation without having to find the cause directly, allowing rapid counseling.

That is, the comparison processing module 160 is configured to receive the above-described composite frame capture image and perform comparative analysis on it with the current counselor web page. Since the composite frame captured image is a mobile-based web page, the output screen and ratio of the counselor terminal 10 that generally uses a PC may be different. In this case, since it is difficult to automatically perform comparative analysis, the composite frame capture image is configured to be corrected based on the counselor's web page.

The comparison processing module 160 includes a ratio correction unit 161, a comparison determination unit 162, and a remote control unit 163.

The ratio correction unit 161 is configured to render the URL for the composite frame captured image and correct the pixel information of the composite frame captured image based on the counselor web page to form the same ratio as the counselor web page. In other words, a comparison can be made with the current counselor webpage image. As an example, while the counselor web page is output from the desktop, the composite frame capture image output from the customer terminal may be output from the customer's mobile phone (or smart device), and the ratios may be different. This means correcting the same. By correcting the ratio to be the same, quick counseling can be performed by automatically providing the counselor with the differences between the two images without the counselor having to make a judgment himself.

Meanwhile, the ratio correction unit 161 may be configured to not only correct the ratio but also delete noise images in the composite frame captured image. As an example, content related to personal information, such as time information, battery level information, personal message information, etc. located at the top of the composite frame capture image may be treated as noise and filtered so that it is not provided to the counselor.

Before comparative analysis is performed through the comparison determination unit 162, it is preferably configured to pass through the ratio correction unit 161.

The comparison determination unit 162 is configured to load the current counselor web page image and the composite frame capture image and extract meaningful features for each. For example, algorithms such as SIFT (Scale-Invariant Feature Transform) or ORB (Oriented FAST and Rotated BRIFF) can be used. It is configured to perform matching between two images based on the extracted features and then filter the matching results. It is configured to filter out matches that do not meet certain criteria among the matched features and measure the similarity between them.

The remote control unit 163 may be activated when the customer's error state is serious, communication with the customer is not smooth, or when remote control rights are granted to the counselor after the customer's consent. On that premise, if it is determined that the counselor's web page and the composite frame captured image match, it can be configured to request remote control authorization from the customer terminal 20. Since the customer terminal 20 can be operated through remote control, matching conditions between the counselor's web page and the composite frame captured image may be required. Here, 'match conditions' may not mean 100% match, and may require a degree of match that the counselor can control.

The remote control unit 163 transmits an event so that identification information about the customer is input to the customer terminal 20 in a preset manner, and when the identification information is accurately entered from the customer terminal 20, the remote control unit 163 sends an event to the customer terminal 20. Remote control may be granted to the counselor. In other words, this requires customer consent.

Figure 5 is a flowchart of a method using a counseling system according to an embodiment of the present invention. Referring to FIG. 5, the method includes steps S110 to S150.

Step (S110) is a step in which the current web page sharing request event of the customer terminal 20 is transmitted from the counselor terminal 10 to the customer terminal 20. When the counselor clicks the customer screen view icon, an event is transmitted from the counselor terminal 10 to the customer terminal 20.

Step S120 is a step in which the existence and coordinate information of an iframe are confirmed for the current web page through the iframe module.

In step S130, if an iframe does not exist in the current web page in step S120, document.body for the current web page of the customer terminal is captured through the HTML2Canvas library unit to generate a capture image of the current web page. In step S120, if an iframe is confirmed in the current web page, the iframe capture image is captured using HTML2Canvas on the iframe through the iframe capture processing unit, and then uploaded to the parent frame and synthesized. This is the stage where a frame capture image is created. That is, if it is confirmed that there is no iframe, document.body is captured using the HTML2Canvas library unit in the customer terminal 20, and HTML2Canvas renders the dom to be captured on a virtual canvas and returns the canvas. It can work.

FIG. 6 is a flow chart illustrating step S130 of FIG. 5 in more detail. With reference to FIG. 6, step S130 will be described in more detail, especially based on the case where an iframe is present. Step S130 includes steps S131 to S133.

Step S131 is a step of sending an event from the parent frame to the iframe using a post message. If there are multiple iframes, a post message may be sent for each.

Step S132 is a step of receiving the event from the iframe, capturing the document.body of the iframe itself using HTML2Canvas, and then transmitting it to the parent frame. In this process, it is necessary to automatically obtain coordinate information (especially outline) of each individual iframe.

Step S133 is a step of generating the composite frame capture image by uploading the iframe capture image to a location corresponding to the coordinate information of the iframe among the parent frames.

Step S140 is a step in which the URL for the current web page captured image or composite frame captured image is transmitted to the counselor terminal 10 in a preset manner in step S130. The composite frame capture image uploads a data URL (blob) from the canvas to the server, and after uploading, the URL of the uploaded image can be sent to the counselor through a socket.

Step S150 is a step in which the URL is rendered and output from the counselor terminal 10 in step S140. Figure 4 shows an example output to the counselor terminal 10. (A-1 screen) displays a web page operated by the counselor, and (A-2 screen) displays a composite frame capture image of the customer terminal.

Counselor terminals that can communicate include desktop computers, laptop computers, laptops, smart phones, tablet PCs, mobile phones, and smart watches. ), smart glass, e-book reader, PMP (portable multimedia player), portable game console, navigation device, digital camera, DMB (digital multimedia broadcasting) player, digital voice recorder (digital) It may include an audio recorder, a digital audio player, a digital video recorder, a digital video player, a PDA (Personal Digital Assistant), etc.

The preferred embodiments of the present invention described above may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. Computer-readable media may include program instructions, data files, data structures, etc., singly or in combination. Program instructions recorded on a computer-readable medium may be specially designed and constructed for the present invention or may be known and usable by those skilled in the computer software art.

Examples of computer-readable media may include hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, etc. Examples of program instructions may include machine language code such as that created by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The above-described hardware device may be configured to operate with at least one software module to perform the operations of the present invention, and vice versa.

Additionally, the above-described method or device may be implemented by combining all or part of its components or functions, or may be implemented separately.

Although the present invention has been described above with reference to preferred embodiments, those skilled in the art may make various modifications and changes to the present invention without departing from the spirit and scope of the present invention as set forth in the claims below. You will understand that you can do it.

110: Web page sharing module
120: iframe module 130: capture processing module
140: Image processing module 150: Display processing module
160: Comparison processing module

Claims (5)

A consultation system that performs consultation by obtaining a web page of a customer terminal using the mobile web,
A web page sharing module that transmits a web page sharing request event to the customer terminal when a counselor requests to share the current web page of the customer terminal;
When acceptance information of the web page sharing request event is received from the customer terminal, an iframe module that checks the existence and coordinate information of an iframe (inline frame, iframe) for the current web page;
A capture processing module that captures the current web page through HTML2Canvas, and includes an iframe capture processing unit that performs capture when an iframe exists in the current web page; and
It is linked with the iframe capture processing unit, and uploads the iframe capture image for the iframe on the parent frame to a location corresponding to the coordinate information of the iframe, thereby generating a composite frame capture image, and generating the composite frame capture image. An image processing module that transmits a uniform resource locator (URL) to the counselor terminal; Including,
The iframe capture processing unit,
Sending a postMessage from the parent frame to the iframe, capturing the iframe capture image using HTML2Canvas on the iframe,
The counselor web page output to the counselor terminal and the composite frame captured image are compared in a preset manner, and a preprocessing process is performed to correct the composite frame captured image to the same ratio as the counselor web page and output it to the counselor terminal. processing module; It further includes,
The comparison processing module is,
A counseling system that renders a URL for the composite frame captured image, but corrects pixel information of the composite frame captured image based on the counselor web page to form the same ratio as the counselor web page. According to claim 1,
The capture processing module,
An HTML2Canvas library unit that converts HTML elements on the current web page into a capture image and captures document.body for the current web page of the customer terminal; It further includes,
The iframe capture processing unit,
A consultation system that captures document.body of the iframe and transmits it to the parent frame through the HTML2Canvas library unit. According to clause 2,
The iframe module is,
When the first to nth iframes are confirmed in the current web page, the first to nth coordinate information, which is outline coordinate information for each, is tagged, and each of the coordinate information can be composed of a plurality of coordinates,
The first to nth iframes are,
A consultation system in which each iframe capture image is created independently and uploaded to the parent frame. According to clause 3,
The consultation system is,
a display processing module that displays a counselor on a web page of the customer terminal so that the customer can select one of the first to nth iframes of the customer terminal; It further includes,
The display processing module is,
A counseling system set to allow the customer to identify the displayed iframe by displaying and processing the iframe selected by the counselor in a preset manner among the first to nth iframes. delete

Images