KR102359988B1 - System for providing non face-to-face interactive visual collaboration service for city gas customer service - Google Patents

Abstract

The present invention relates to a system for providing a non-face-to-face video service for a city gas customer service, which includes: a user terminal receiving a notification message, inputting a query message with respect to the notification message as a voice utterance after sending a call, accessing by selecting a link in a received text message, and then sharing a captured screen; an agent terminal call-connected by the user terminal sending the call, generating the link for screen sharing, transmitting the text message to the user terminal, and receiving the sharing of a screen captured by a camera of the user terminal in a case where the link in the text message is selected at the user terminal; and a video service providing server including a connection unit connecting to the agent terminal in a case where a call is sent from the user terminal after a notification message is sent from a city gas server to the user terminal, a generating unit generating a channel for user terminal-agent terminal screen sharing in a case where a text message including a link is transmitted to the user terminal from the agent terminal and then the link is selected at the user terminal, a sharing unit for sharing streaming a screen captured at the user terminal to the agent terminal in real time, and a transmission unit mapping and storing query and response messages and performing real-time transmission to the city gas server in a case where the response message corresponding to the query message of the user terminal is input to the agent terminal as a voice utterance.

Description

Non-face-to-face video service provision system for city gas customer consultation

The present invention relates to a non-face-to-face video service providing system for city gas customer consultation, and provides a two-way real-time remote video consultation platform that shares a screen shot with a call connection with an agent and also shares the agent's pen input on the screen to the user do.

With the development of technology, it is very difficult to differentiate products in terms of function or performance. Accordingly, companies are strengthening their corporate competitiveness by providing excellent service quality to their customers by strengthening their service competitiveness. Services can be broadly divided into face-to-face and non-face-to-face services according to customer response methods. A representative service of non-face-to-face service can be called a call center. Call centers can be further divided into inbound and outbound services according to business methods. It is operated mainly on the service work that responds to customer requests, and refers to a center where counselors perform various requests and complaints handling tasks using non-face-to-face phone calls with customers.

At this time, a method of conducting a consultation while sharing a screen and a method of overlay processing a pen input on the screen were researched and developed. and Korean Patent Application Laid-Open No. 2015-0037805 (published on April 08, 2015), a service consultation request from a customer terminal is received through a call center system to implement an overlay writing function on images and video screens during video consultation. In this case, the system sends a message including the access address of the recall meeting to the customer terminal, connects to the access address from the customer terminal and delivers the customer image to the system, the system connects the agent terminal and video consultation, and the In order to provide a configuration for providing a writing function while transmitting a guide video to a customer terminal, and to provide a real-time screen sharing chat, when real-time screen sharing is requested during chatting between a plurality of terminals, the screen of any one terminal in real time Each configuration is disclosed so that chat contents are displayed overlaid on the shared video while sharing with other terminals.

However, in the case of non-face-to-face self-safety check-ups or consultations related to city gas inspection, you have to ask and answer questions while looking at related devices. That is, in the former case, after the customer terminal downloads the video, only the pen function is added thereon, not the configuration in which the screen itself captured by the customer terminal is shared by the counselor terminal. Even in the latter case, screen sharing does not mean sharing the screen being filmed by any one terminal, but sharing the window screen itself including the wallpaper, not the sharing of the camera screen being photographed. Accordingly, research and development of a remote video consulting platform that shares the screen that the user is shooting while maintaining the call connection is required when a second civil complaint related to city gas inspection is received.

In one embodiment of the present invention, when a call is sent from a user terminal that has received a notification message from a city gas server to a customer service center, the user terminal is connected between the customer service center's counselor terminal and the user terminal. The screen is shared with the agent terminal in real time, and when an input is made using an I/O interface such as a pen on the screen taken from the agent terminal, it can be shared in real time so that it is overlaid on the screen shot of the user terminal, The queries and responses between the user terminal and the agent terminal are converted into text and video data and accumulated as a dataset to learn the deep learning algorithm. It is possible to provide a method of providing a non-face-to-face video service for city gas customer consultation that can be modeled so as to be able to do so. However, the technical task to be achieved by the present embodiment is not limited to the technical task as described above, and other technical tasks may exist.

As a technical means for achieving the above-described technical problem, an embodiment of the present invention transmits at least one type of notification message related to city gas business processing, and transmits a response message to a query message corresponding to the notification message in real time Receive city gas server and notification message, input the inquiry message for notification message through voice communication after sending a call, select a link in the received text message to connect, and then share the captured screen A user terminal that calls a user terminal, a call is connected by outgoing a call from the user terminal, a link for screen sharing is created and a text message is transmitted to the user terminal, and a screen captured by the camera of the user terminal is selected when a link in the text message is selected from the user terminal After a notification message is delivered to the user terminal from the agent terminal and city gas server sharing When a link is selected in the user terminal after transmission, a generation unit that creates a channel for screen sharing between the user terminal and the agent terminal, a sharing unit that shares the screen shot from the user terminal to receive streaming in real time from the agent terminal, and the user terminal When a response message corresponding to the inquiry message is input by voice from the counselor's terminal, the image service providing server includes a transmission unit that maps and stores the inquiry message and the response message and transmits the response message to the city gas server in real time.

According to any one of the above-described problem solving means of the present invention, when a call is sent from the user terminal receiving the notification message from the city gas server to the customer service center, the connection between the customer service center counselor terminal and the user terminal while sharing the screen shot from the user terminal with the agent terminal in real time, and overlaying it on the screen shot of the user terminal when an input is made using an I/O interface such as a pen on the screen photographed from the agent terminal in real time It can be shared, and the queries and responses between the user terminal and the agent terminal are converted into text and video data, accumulated as a dataset to learn the deep learning algorithm, and then the artificial intelligence algorithm is used when the same query is input It can be modeled so that an automatic response can be made.

1 is a view for explaining a non-face-to-face video service providing system for city gas customer consultation according to an embodiment of the present invention.
FIG. 2 is a block diagram illustrating an image service providing server included in the system of FIG. 1 .
3 and 4 are diagrams for explaining an embodiment in which a non-face-to-face video service for city gas customer consultation is implemented according to an embodiment of the present invention.
5 is an operation flowchart illustrating a method of providing a non-face-to-face video service for city gas customer consultation according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art can easily implement them. However, the present invention may be embodied in several different forms and is not limited to the embodiments described herein. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Throughout the specification, when a part is "connected" with another part, this includes not only the case of being "directly connected" but also the case of being "electrically connected" with another element interposed therebetween. . In addition, when a part "includes" a certain component, it means that other components may be further included, rather than excluding other components, unless otherwise stated, and one or more other features However, it is to be understood that the existence or addition of numbers, steps, operations, components, parts, or combinations thereof is not precluded in advance.

The terms "about", "substantially", etc. to the extent used throughout the specification are used in or close to the numerical value when manufacturing and material tolerances inherent in the stated meaning are presented, and are intended to enhance the understanding of the present invention. To help, precise or absolute figures are used to prevent unfair use by unconscionable infringers of the stated disclosure. As used throughout the specification of the present invention, the term "step of (to)" or "step of" does not mean "step for".

In this specification, a "part" includes a unit realized by hardware, a unit realized by software, and a unit realized using both. In addition, one unit may be implemented using two or more hardware, and two or more units may be implemented by one hardware. Meanwhile, '~ unit' is not limited to software or hardware, and '~ unit' may be configured to be in an addressable storage medium or to reproduce one or more processors. Thus, as an example, '~' denotes components such as software components, object-oriented software components, class components, and task components, and processes, functions, properties, and procedures. , subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays and variables. The functions provided in the components and '~ units' may be combined into a smaller number of components and '~ units' or further separated into additional components and '~ units'. In addition, components and '~ units' may be implemented to play one or more CPUs in a device or secure multimedia card.

Some of the operations or functions described as being performed by the terminal, apparatus, or device in the present specification may be performed instead of by a server connected to the terminal, apparatus, or device. Similarly, some of the operations or functions described as being performed by the server may also be performed in a terminal, apparatus, or device connected to the server.

In this specification, some of the operations or functions described as mapping or matching with the terminal means mapping or matching the terminal's unique number or personal identification information, which is the identification data of the terminal. can be interpreted as

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram for explaining a non-face-to-face video service providing system for city gas customer consultation according to an embodiment of the present invention. Referring to FIG. 1 , the non-face-to-face video service providing system 1 for city gas customer consultation includes at least one user terminal 100 , an image service providing server 300 , at least one city gas server 400 , It may include at least one agent terminal 500 and a field technician terminal 600 . However, since the non-face-to-face video service providing system 1 for city gas customer consultation of FIG. 1 is only an embodiment of the present invention, the present invention is not limitedly interpreted through FIG. 1 .

At this time, each component of FIG. 1 is generally connected through a network (Network, 200). For example, as shown in FIG. 1 , at least one user terminal 100 may be connected to the image service providing server 300 through the network 200 . In addition, the video service providing server 300 includes at least one user terminal 100 , at least one city gas server 400 , at least one counselor terminal 500 , and a field engineer terminal 600 through the network 200 . ) can be associated with In addition, at least one city gas server 400 may be connected to the image service providing server 300 through the network 200 . In addition, the at least one agent terminal 500 may be connected to the at least one user terminal 100 , the image service providing server 300 , and the at least one city gas server 400 through the network 200 . Finally, the field technician terminal 600 may be connected to the user terminal 100 , the city gas server 400 , the image service providing server 300 , and at least one agent terminal 500 through the network 200 . .

Here, the network refers to a connection structure in which information exchange is possible between each node, such as a plurality of terminals and servers, and an example of such a network includes a local area network (LAN), a wide area network (WAN: Wide Area Network), the Internet (WWW: World Wide Web), wired and wireless data communication networks, telephone networks, wired and wireless television networks, and the like. Examples of wireless data communication networks include 3G, 4G, 5G, 3rd Generation Partnership Project (3GPP), 5th Generation Partnership Project (5GPP), Long Term Evolution (LTE), World Interoperability for Microwave Access (WIMAX), Wi-Fi (Wi-Fi) , Internet, LAN (Local Area Network), Wireless LAN (Wireless Local Area Network), WAN (Wide Area Network), PAN (Personal Area Network), RF (Radio Frequency), Bluetooth (Bluetooth) network, NFC ( Near-Field Communication) networks, satellite broadcast networks, analog broadcast networks, Digital Multimedia Broadcasting (DMB) networks, and the like are included, but are not limited thereto.

In the following, the term at least one is defined as a term including the singular and the plural, and even if the at least one term does not exist, each element may exist in the singular or plural, and may mean the singular or plural. it will be self-evident In addition, that each component is provided in singular or plural may be changed according to embodiments.

At least one user terminal 100 sends a notification message (notification talk) from the city gas server 400 without using or using a non-face-to-face video service related web page, app page, program or application for city gas customer consultation. It may be a receiving terminal. In addition, at least one user terminal 100 sends a call to the city gas customer service center in order to make a second civil complaint with an inquiry about the notification message, and a message including a link after the call is connected to the counselor terminal 500 In addition, when a link is selected, the user terminal 100 may be a terminal that is connected to a link address and shares a screen with the counselor terminal 500 .

Here, the at least one user terminal 100 may be implemented as a computer that can access a remote server or terminal through a network. Here, the computer may include, for example, navigation, a laptop equipped with a web browser, a desktop, and a laptop. In this case, the at least one user terminal 100 may be implemented as a terminal capable of accessing a remote server or terminal through a network. At least one user terminal 100 is, for example, as a wireless communication device that guarantees portability and mobility, navigation, PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS(Personal Handyphone System), PDA(Personal Digital Assistant), IMT(International Mobile Telecommunication)-2000, CDMA(Code Division Multiple Access)-2000, W-CDMA(W-Code Division Multiple Access), Wibro(Wireless Broadband Internet) ) terminal, a smart phone, a smart pad, a tablet PC, etc. may include all kinds of handheld-based wireless communication devices.

The image service providing server 300 may be a server that provides a non-face-to-face image service web page, an app page, a program, or an application for city gas customer consultation. In addition, the image service providing server 300 may be a server configured to register at least one counselor terminal 500 and a field engineer terminal 600 , and to be linked with a customer database of the city gas server 400 . In addition, the video service providing server 300 connects the user terminal 100, which makes a call after receiving a notification message (reminder talk) from the city gas server 400, to the counselor terminal 500, and the counselor terminal It may be a server that allows real-time sharing of a shooting screen or pen input between the 500 and the user terminal 100 . If the counselor terminal 500 cannot provide counseling, it may be a server that allows multiple connections between the field technician terminal 600 and three parties. Of course, it is not limited to three people.

Here, the image service providing server 300 may be implemented as a computer capable of accessing a remote server or terminal through a network. Here, the computer may include, for example, navigation, a laptop equipped with a web browser, a desktop, and a laptop.

At least one city gas server 400 is a server that transmits a notification message to the user terminal 100 with or without using a non-face-to-face video service related web page, app page, program or application for city gas customer consultation can In addition, the city gas server 400 provides information on the user terminal 100 when requested from the image service providing server 300 , and provides information and queries on the user terminal 100 from the image service providing server 300 . And when the response of the agent terminal 400 is shared in real time, it may be a server that stores it in a customer database. Here, the at least one city gas server 400 may be implemented as a computer capable of accessing a remote server or terminal through a network. Here, the computer may include, for example, navigation, a laptop equipped with a web browser, a desktop, and a laptop.

At least one agent terminal 500 is a non-face-to-face video service related web page for city gas customer consultation, an application page, a program or an application of an agent who shares a screen while making a call connection with the user terminal 100 It may be a terminal. In this case, the counselor terminal 500 displays the screen on the shared screen using a pen, and the pen input event may also be a shareable terminal to the user terminal 100 via the image service providing server 300 .

Here, the at least one counselor terminal 500 may be implemented as a computer capable of accessing a remote server or terminal through a network. Here, the computer may include, for example, navigation, a laptop equipped with a web browser, a desktop, and a laptop. In this case, the at least one agent terminal 500 may be implemented as a terminal capable of accessing a remote server or terminal through a network. At least one agent terminal 500, for example, as a wireless communication device that guarantees portability and mobility, navigation, PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS(Personal Handyphone System), PDA(Personal Digital Assistant), IMT(International Mobile Telecommunication)-2000, CDMA(Code Division Multiple Access)-2000, W-CDMA(W-Code Division Multiple Access), Wibro(Wireless Broadband Internet) ) terminal, a smart phone, a smart pad, a tablet PC, etc. may include all kinds of handheld-based wireless communication devices.

The field engineer terminal 600 is, when there is a consultation request from the agent terminal 500 using a non-face-to-face video service related web page, app page, program or application for city gas customer consultation, the user terminal 100 and the agent It may be a terminal of a field technician that is multi-connected to the terminal 500 . In this case, the field technician terminal 600 may also be a terminal capable of inputting a pen on the shooting screen shared by the user terminal 100 like the counselor terminal 500 .

Here, the field technician terminal 600 may be implemented as a computer that can connect to a remote server or terminal through a network. Here, the computer may include, for example, navigation, a laptop equipped with a web browser, a desktop, and a laptop. In this case, the field technician terminal 600 may be implemented as a terminal capable of accessing a remote server or terminal through a network. Field engineer terminal 600 is, for example, as a wireless communication device that guarantees portability and mobility, navigation, PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS ( Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), Wibro (Wireless Broadband Internet) terminal , may include all types of handheld-based wireless communication devices such as smartphones, smartpads, and tablet PCs.

2 is a block diagram for explaining the image service providing server included in the system of FIG. 1, and FIGS. 3 and 4 are non-face-to-face image services for city gas customer consultation according to an embodiment of the present invention. It is a diagram for explaining an embodiment.

Referring to FIG. 2 , the image service providing server 300 includes a connection unit 310 , a generation unit 320 , a sharing unit 330 , a transmission unit 340 , an input unit 350 , a multiplexing unit 360 , and a database. It may include a flower unit 370 .

At least one user terminal 100 , at least one city gas server 400 , and at least one counselor may include the image service providing server 300 according to an embodiment of the present invention or another server (not shown) operating in conjunction with each other. When transmitting a non-face-to-face video service application, program, app page, web page, etc. for city gas customer consultation to the terminal 500, at least one user terminal 100, at least one city gas server 400, and at least One counselor terminal 500 may install or open a non-face-to-face video service application, program, app page, web page, etc. for city gas customer consultation. In addition, the service program may be driven in at least one user terminal 100 , at least one city gas server 400 , and at least one counselor terminal 500 by using a script executed in a web browser. Here, the web browser is a program that enables the use of a web (WWW: World Wide Web) service, and refers to a program that receives and displays hypertext written in HTML (Hyper Text Mark-up Language), for example, Netscape. , Explorer, Chrome, and the like. In addition, the application means an application on the terminal, for example, includes an app (App) executed in a mobile terminal (smartphone).

Referring to FIG. 2 , the connection unit 310 is, after a notification message is transmitted from the city gas server 400 to the user terminal 100 , when a call is sent from the user terminal 100 , the counselor terminal 500 ) can be connected. The city gas server 400 may transmit at least one type of notification message related to city gas business processing and receive a response message to a query message corresponding to the notification message in real time. The user terminal 100 may receive the notification message, and may input an inquiry message for the notification message through voice utterance after sending a call. The agent terminal 500 may be connected to a call through the call origination of the user terminal 100 . At this time, although questions and responses are made by voice utterance after call connection, the reason for expressing this as a message is that the result of voice recognition through STT (Speech to Text) can be converted into text message (textualization). This is later used when mapping and storing [question-answer-video or photo] in the deep learning algorithm, and it lays the foundation for automatically building a dataset without hiring a labeling man. This is because, when only keywords are extracted from voice utterances, data can be formed in the form of [question-question keyword-response keyword-response], and based on the keyword, it is possible to find and respond to response keywords thereafter. In addition, as will be described later, a specific part of the screen shared by the agent terminal 500 may be displayed with a pen, etc., which is designated as an ROI (Region of Interest) and may be mapped and stored with each keyword, which is also deep learning By specifying the ROI in the algorithm, it is possible to form a feature vector or start learning immediately without a separate labeling task when learning.

The generator 320 transmits a text message including a link from the agent terminal 500 to the user terminal 100 and then selects a link from the user terminal 100, the user terminal 100 and the agent terminal 500 You can create a channel for screen sharing between users. The user terminal 100 may select a link in the received text message to be connected, and then share a photographed screen.

Accordingly, in an embodiment of the present invention, traffic smoothing may be further performed. The basic condition for traffic smoothing is that the server end knows the encoding curve. In a single link, knowing one's own encoding curve means knowing the decoding curve, so there was no problem in traffic smoothing. However, in order to perform optimized traffic smoothing in multi-link, the encoding curve of the transcoder must be considered together when data is transmitted on the first link, so the entire curve cannot be known in the absence of transcoding. Therefore, when performing traffic smoothing in multi-link, it is necessary to adjust reference delay values within a stable range. , δndown) respectively.

Here, when the maximum delays (δnup, δndown) in the uplink and downlink are determined as in Equation 2 below, if the amount of data generated in the uplink is large, the transmission time transmitted to the uplink is increased, If there is a large amount of data in the downlink, the transmission time is increased in the downlink so that the level of data amount is properly divided.

On the other hand, the traffic smoothing method in a single link is applied to each of the multi-links. First, in the traffic smoothing process in a single link, a frame in which a lot of data is generated is given as much weight as the size of the frames in the GOP (Group of Pictures). It is a process in which a large amount of transmission time is given to a frame, and a small transmission time is given to a frame in which there is little data. In this way, the overall amount of data generated becomes constant, and the transmission load of the entire network can be uniformed, thereby improving stability. The second is the process of determining the actual transmission rate for each allocated transmission time for each frame. In this process, a constraint that the transmission rate must be set to prevent decoding underflow of the decoder in units of slices within a frame is required. Through this process, traffic smoothing can be achieved on a single link.

The sharing unit 330 may share the screen captured by the user terminal 100 to be streamed in real time from the counselor terminal 500 . The agent terminal 500 generates a link for screen sharing and transmits a text message to the user terminal 100 , and when a link in the text message is selected in the user terminal 100 , the image is captured by the camera of the user terminal 100 . You can share your screen. When a screen is shared between the user terminal 100 and the counselor terminal 500 , the screen may be shared by connecting to a non-installed web service. For example, a video in a web page may be designed to receive real-time streaming based on HTML5. Here, HTML5 is the latest standard of HTML (Hyoer Text Markup Language), which is a web document standard, and includes a function of executing pictures, videos, and music in the center of document creation.

HTML5 eliminates the need to separately install programs such as plug-in-based Active-X and Flash, and most of the problems caused by this can be solved. At this time, the biggest difference between the conventional HTML standard and HTML5 is that semantic markup is supported and an additional API is provided. Semantic markup is a new element that can semantically structure web documents. For example, if a document is created with elements such as <head>, <section>, and <footer> so that the contents of the document can be divided into header, body, and footer, the contents of the document can be clearly understood structurally. In addition, HTML5 provides various APIs. The API provided by HTML5 allows you to add various functions to web applications, such as supporting 2D graphics or checking the geographic location of a device.

In addition, HTML5 is a markup language used to create web pages, and it can express and provide HTML, which used to be displayed as hyperlinks and texts, to various applications such as multimedia. HTML5 can be implemented including HTML, a markup language for web document structure design, CSS for design expression, JavaScript for expressing interactive action, etc., and new tags are added to existing HTML tags. More can be added, and the structure to contain contents can be improved to even perform the role of a platform. Also, although HTML5 is not a standard for the mobile web, it provides many functions necessary for the implementation of the mobile web among the various functions described above. Two-dimensional graphics, audio, and various web forms can enrich and diversify the UI (User Interface) of the mobile web. That is, it is possible to provide multimedia information or support a user's dynamic interaction even in a mobile web environment. In the mobile web environment, there is a limitation in transmitting large amounts of data due to the nature of the mobile environment, and there is often an environment in which the Internet is not connected. Alternatively, you can store the DB locally and query it with an SQL statement.

Alternatively, you can use SaaS (Software as a Servcie), which can be easily used anywhere on the Internet and can be accessed anywhere with a web browser. At this time, BaaS (Backend as a Service) may be further added according to the requirements of institutions, users, and instructors. Of course, for users, institutions, and instructors who do not have the infrastructure or platform itself, IaaS (Infrastructure as a Service) can provide various infrastructure resources such as server resources, IP, network, storage, and power required for server operation, flexibly and quickly. have. IaaS makes it easy to add or remove resources and provides an operating system so a friendly environment can be given. In addition, the service can be provided as a PaaS (Platform as a Service), which provides a stable environment and an API for developing applications using the environment. PaaS can easily build and deploy source code, and it is easy to scale out according to usage. BaaS is a service specialized for mobile applications and can provide server-side applications such as member management, push, and file processing that are difficult to process in mobile applications. The platform of PaaS can be used and the provided backend module can also be configured so that it can be used directly through REST API or library CALL.

Alternatively, it can be run within an on-premises platform. On-premises means that physical servers are installed and operated directly, and there is no proper term that exactly corresponds to the Korean language, so the English translation is hereinafter referred to as 'on-premises'. An on-premises platform refers to a platform that directly installs and operates one or more physical servers. Since it is a concept that literally manages servers directly, it is most important to secure a physical server above all else. Server equipment cost is also the biggest reason for on-premises platform cost. If the necessary servers are secured, space is needed to physically place them. If there are one or two servers, you can put them in one corner of the lab, but with ten or more servers, the situation is different. First of all, it is almost impossible to stay in the same room with researchers because of the noise and heat emitted by the equipment. Stable power must be supplied to servers that operate 24 hours a day, and an uninterruptible power supply called a UPS (Uninterruptible Power Supply) is also required in case of a power outage in the case of servers performing important tasks that should not be lost. Connection to an external network such as the Internet is important, but in some cases, an internal network connection between servers that enables fast communication between servers is also important. You may need a rack equipment to efficiently place many servers and cables, and a thermo-hygrostat to keep the temperature and humidity in the server room constant.

After completing this hardware preparation, the next step is to prepare the necessary software. First, you can think of the operating system (OS) of the server, and if you use an open source such as Linux, there may be no cost. In addition, various types of software are required depending on the purpose of the server, and license fees may occur for commercial software. After completing the software preparation, finally, the manpower to perform all these tasks should be prepared. You will need both a manpower to perform the hardware work and a manpower to perform the software work, but in the case of small and simple platforms, it is often the case that the software manpower also performs the hardware work.

Manpower for the one-time task of installing all these things at the beginning is needed, but also operation manpower to perform maintenance after normal operation is required. In particular, in the case of a platform used by many users, service personnel may be required to directly respond to users' inquiries and requests at the front end. In order to build and operate an on-premises platform like this, hardware equipment, a software environment, and manpower to perform it are required. Depending on the embodiment, a cloud platform or an on-premises platform may be used, but it is not limited to any one platform.

Describes the cloud platform in terms of comparison with on-premises. This is not a physical platform installed and operated directly like on-premises, but it means a platform that exists somewhere in the cloud and performs the same role as on-premises. Therefore, the biggest difference compared to on-premises is that there is no need to directly secure and manage server equipment. You don't know, and you don't need to know, where the equipment actually resides physically. Users simply need to use the computing resources and storage space provided by the cloud service.

Not only the server equipment, but also all hardware equipment such as the aforementioned server space, power, and network disappear from the target to be managed by the user for the same reason. It is also an advantage that gives it a clear comparative advantage compared to on-premises. With software, the situation is almost identical to on-premises. The necessary software must be installed by the user. Nevertheless, in many cases, the operating system or major software is already installed and provided by default, so the effort required for installing the software can be significantly reduced. In the case of manpower, the effort for the initial installation and subsequent operation of hardware equipment disappears, so you only need to focus on the installation and operation of the software. In conclusion, when operating a cloud platform, consideration for hardware equipment disappears completely, leaving only consideration for core software.

When a response message corresponding to the query message of the user terminal 100 is input by voice utterance from the counselor terminal 500, the transmission unit 340 maps and stores the query message and the response message, and stores the query message in the city gas server 400 . can be transmitted in real time.

The input unit 350 receives an input event corresponding to any one or a combination of at least one text input using a remote pointer, a live pen, a whiteboard, and a keyboard from the agent terminal 500 . When generated, the input event may be shared with the user terminal 100 in real time. For example, a whiteboard sharing function can be provided by extending the collaboration service of C3ware, a collaboration middleware. The extended collaboration service can manage various types of resources generated by the shared whiteboard and support the shared whiteboard workspace using c3ware's group workspace. In addition, it is possible to provide a concurrency control service for maintaining resource consistency with respect to asynchronous requests occurring in real time in a shared whiteboard work environment.

C3ware is a cloud storage-based collaboration middleware that provides various collaboration services. C3ware supports stable resource management using cloud storage such as Dropbox, Google Drive, and OpenStack's Swift. In addition, it provides a systematic collaboration environment using various types of workspaces such as individual workspaces, group workspaces, and public workspaces. C3ware's collaboration service consists of user service, group service, and workspace service. The user service provides functions related to user creation and authentication, and the group service supports functions such as group creation and management and group information confirmation. The workspace service may provide functions such as creation of various types of workspaces and management of resources within the workspace.

This is a collaboration tool in which users communicate using various drawing tools. Group users can collaborate by adding photos or drawings to the same shared screen. In general, a shared whiteboard application is often used as a tool for virtual meetings, presentations, and education. Recently, it has been developed together with a messenger function so that users can use it easily and conveniently at any time. It may be provided to operate on various platforms by providing an API in HTML format that can be used in the above-mentioned website.

C3ware's collaboration service does not provide specialized management functions for various types of collaboration resources that can occur in various collaboration situations. Accordingly, in an embodiment of the present invention, a collaboration service that manages various types of resources generated in a shared whiteboard collaboration environment and provides a shared whiteboard workspace can be defined and implemented as a web service of C3ware. The shared whiteboard is a collaboration application widely used in various collaborative situations such as virtual meetings, and communicates between users by using various types of drawing tools.

The shared whiteboard web service according to an embodiment of the present invention may provide a function of managing a workspace of a shared whiteboard or resources generated in a shared whiteboard operation. The shared whiteboard workspace can be implemented by extending C3ware's group workspace, and objects created by the user using the drawing function can be saved in a predefined structure in the group workspace. The undo and redo services can provide the function of canceling and restoring the work by managing the history information of the shared whiteboard work. C3ware can create Drawinglist and Redolist to manage job history information, and each queue stores the jobs requested by users to restore or restore jobs as needed. The object generated in the shared whiteboard may include information such as the type and color of the object, and coordinates drawn on the screen. C3ware defines and uses the class of the whiteboard object to manage the shared whiteboard object.

In the shared whiteboard work environment, asynchronous requests for drawn objects occur, so a concurrency control service is needed to maintain object consistency. The getPermission service provides a function to obtain permission by using locking on an object in such a real-time work situation. C3ware manages the locking table to manage the use right of the object, and the modification or deletion of the drawn object can be performed after the user acquires the use right for the object.

The multi-unit 360, after being connected between the user terminal 100 and the agent terminal 500, when the agent terminal 500 requests multi-remote support linkage, the user terminal 100 and the field technician terminal 600 A link to be connected to the agent terminal 500 may be transmitted, and screen sharing may be performed between the user terminal 100 , the agent terminal 500 , and the field technician terminal 600 .

The database conversion unit 370 converts video data corresponding to a screen shared between the user terminal 100 and the counselor terminal 500 and voice utterances input from the user terminal 100 and the counselor terminal 500 to STT (Speech to Text). ) can be converted into text data and accumulated as a dataset for learning and testing at least one artificial intelligence algorithm. The dataset may be accumulated on the above-described cloud platform or on-premises platform.

Although various methods exist for deep learning, an embodiment of the present invention is based on an image-based, that is, a video taken in the user terminal 100 and based on a specific frame or picture in the video, deep learning CNN, which is an artificial intelligence algorithm. (Convolutional Neural Network) and YOLO (You Only Look Once) developed based on this will be used.

Recently, as a method of recognizing objects in an image, a method using artificial intelligence is frequently used. Among them, the most widely used method is the R-CNN series algorithm. In a method using region-based convolution, several neural networks are continuously connected, and weight values corresponding to each neural network exist. This weight value is determined by learning, and the determined values are used for object recognition, and there are Fast R-CNN, Faster R-CNN, Mask R-CNN, and the like.

The first Fast R-CNN can secure a backbone network through single-step learning. Among the three methods, only Xavier initial value can be used without using pre-learned weights. It has the advantage of simply securing an artificial intelligence model because it can perform single-step learning, but has a disadvantage in that it takes time to generate multiple Region of Interest (ROI) Boundary Box candidates. This is mainly due to performing a selective search algorithm with a long computation time.

In order to improve this point, Faster R-CNN proposed a new method, in particular, it uses a new deep neural network to generate the region-of-interest bounding box candidates. In other words, the existing algorithm mainly uses CPU resources and takes a relatively large amount of computation time. Such a deep neural network is called a region proposal network because it functions to provide a candidate group for the region of interest, and has relatively few neural network layers compared to the backbone network. Through this neural network, most of the hardware resources required for object recognition can be moved from the CPU to the GPU. However, in order to create a model completed with such a structure, it is necessary to perform learning through several steps. Faster R-CNN is composed of a backbone network, a region proposal network, and a classifier network. To learn these three deep neural networks, several steps are required, so it takes time to obtain a completed model with a multi-step learning technique. may be extended.

The biggest feature of Mask R-CNN, which is the last of the R-CNN series, is that segmentation of object recognition is possible. Until then, object recognition estimated the position of an object through a rectangular box in the image and classified its type, but Mask R-CNN designs a deep neural network to classify the position of an object in units of pixels in object recognition technology. The role of this deep neural network is to distinguish whether each pixel corresponds to an object or not, and is called a binary mask. By putting a binary mask on the feature map obtained from the backbone network, the division is made in units of pixels. However, the size of the general feature map is also small, from 1/4 to 1/16 the size of the original image. Even if the object area is determined in units of pixels in the reduced feature map, the error is large in the original image. To reduce such an error, a technique called ROI alignment is used, which is a method of reducing the decimal point error occurring in the ROI pool area by the second-order linear interpolation method.

Among YOLOs, YOLO-v3 is an algorithm in which one convolutional neural network predicts multiple bounding boxes at the same time and predicts class probability for each bounding box. At this time, the convolutional neural network used is used for generating a feature map, and it is called a backbone network because it is in charge of the backbone of the algorithm. This feature map consists of several grid cells, and it is an algorithm that simultaneously determines the type and location of a target object by applying a score method to each cell. This algorithm is configured to recognize a total of 20 types, including vehicles, people, airplanes, and animals, for every image frame.

Hereinafter, an operation process according to the configuration of the image service providing server of FIG. 2 will be described in detail with reference to FIGS. 3 and 4 as an example. However, it will be apparent that the embodiment is only one of various embodiments of the present invention and is not limited thereto.

Referring to FIG. 3A , to solve the inconvenience of city gas users and the inefficiency of city gas officials in responding to civil complaints as a secondary customer requirement that occurs due to notification messages such as information and publicity delivered from city gas companies to customers. In order to do so, an embodiment of the present invention provides a non-face-to-face video service. Referring to FIG. 3B , the city gas video interactive service allows you to directly view the customer's situation in real time as if you were next to the customer in the interactive service for secondary civil complaints caused by one-sided publicity and guidance, and the service environment is shown in FIG. 3C same as

As a non-installation type web service as shown in FIG. 4A , a user (customer) can be connected to a city gas company counselor by using the customer's device anywhere without a separate installation process by simply clicking the invitation text. At this time, since the call connection is based on the CS network and the data connection uses the PS network, which is LTE or VoLTE, there is no problem that both are connected. One-stop connection is possible even during a voice call. In addition, as shown in FIG. 4B , as an interactive live service, call centers and customer support teams can use it to strengthen services and increase work efficiency, and support by sharing the situation with not only customers but also experts in each field. Real-time visualization is also possible, as shown in Fig. 4c, so that the agent can directly see what the customer is seeing and enable smoother communication through various functions, such as a remote pointer, live pen, whiteboard, or chatting function that can accurately point out the customer's desired point. do. In Figure 4d, field technicians can also be invited. If it is difficult to solve the problem due to the counselor's judgment, an expert can be invited through multi-remote support linkage during consultation with the customer to connect the customer's screen, and through this, the problem can be solved more smoothly You can make sure that you are prepared in case you need to pay A/S and go to A/S.

4E, the present invention is a non-installation type web service that enables remote video consultation without customer installation, and interactively live video or photo or screen itself can be shared through customer's smart phone. , real-time visualization enables more accurate and quick work through various functions, and the on-site technician invitation function allows multiple remote support to be linked for smoother problem solving during one-on-one consultations. The platform according to an embodiment of the present invention may have the same effects and features as in FIGS. 4F and 4G.

The matters that have not been described with respect to the non-face-to-face video service providing method for city gas customer consultation in FIGS. 2 to 4 are described above with respect to the non-face-to-face video service providing method for city gas customer consultation through FIG. 1 and Since it can be easily inferred from the same or described content, the following description will be omitted.

FIG. 5 is a diagram illustrating a process in which data is transmitted/received between components included in the non-face-to-face image service providing system for city gas customer consultation of FIG. 1 according to an embodiment of the present invention. Hereinafter, an example of a process in which data is transmitted and received between each component will be described with reference to FIG. 5, but the present application is not limited to such an embodiment, and the example shown in FIG. 5 according to various embodiments described above will be described. It is apparent to those skilled in the art that the data transmission/reception process may be changed.

Referring to FIG. 5 , after a notification message is transmitted from the city gas server to the user terminal, the video service providing server connects to the counselor terminal when a call is sent from the user terminal ( S5100 ).

In addition, the video service providing server creates a channel for screen sharing between the user terminal and the counselor terminal when the user terminal selects the link after transmitting a text message including the link from the agent terminal to the user terminal (S5200), and the user The screen captured by the terminal is shared so that it is streamed in real time from the agent terminal (S5300).

Finally, when a response message corresponding to the query message of the user terminal is input by voice in the counselor terminal, the video service providing server maps and stores the query message and the response message and transmits it to the city gas server in real time (S5400) ).

The order between the above-described steps ( S5100 to S45400 ) is merely an example and is not limited thereto. That is, the order between the above-described steps S5100 to S45400 may be mutually changed, and some of these steps may be simultaneously executed or deleted.

The matters that have not been described with respect to the non-face-to-face video service providing method for city gas customer consultation in FIG. 5 as described above for the non-face-to-face video service providing method for city gas customer consultation through FIGS. Since it can be easily inferred from the same or described content, the following description will be omitted.

The non-face-to-face video service providing method for city gas customer consultation according to an embodiment described with reference to FIG. 5 is in the form of a recording medium including instructions executable by a computer, such as an application or program module executed by a computer. can also be implemented. Computer-readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. Also, computer-readable media may include all computer storage media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.

The non-face-to-face video service providing method for city gas customer consultation according to an embodiment of the present invention described above may include an application basically installed in a terminal (which may include a program included in a platform or operating system basically installed in the terminal) ), and may be executed by an application (ie, a program) installed directly on the master terminal by a user through an application providing server such as an application store server, an application, or a web server related to the corresponding service. In this sense, the non-face-to-face video service providing method for city gas customer consultation according to an embodiment of the present invention described above is implemented as an application (that is, a program) installed basically in a terminal or directly installed by a user, and is installed in the terminal, etc. It can be recorded on a computer-readable recording medium of

The above description of the present invention is for illustration, and those of ordinary skill in the art to which the present invention pertains can understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. For example, each component described as a single type may be implemented in a dispersed form, and likewise components described as distributed may be implemented in a combined form.

The scope of the present invention is indicated by the following claims rather than the above detailed description, and all changes or modifications derived from the meaning and scope of the claims and their equivalents should be interpreted as being included in the scope of the present invention. do.

Claims (6)

delete delete a city gas server that transmits at least one type of notification message related to city gas business processing and receives a response message to a query message corresponding to the notification message in real time;
a user terminal for receiving the notification message, inputting an inquiry message for the notification message through voice utterance after sending a call, selecting a link in the received text message to connect, and then sharing a photographed screen;
A call is connected through the call origination of the user terminal, a link for screen sharing is created and the text message is transmitted to the user terminal, and when a link in the text message is selected in the user terminal, the camera of the user terminal takes a picture A counselor terminal receiving a screen to be shared; and
After the notification message is transmitted from the city gas server to the user terminal, when a call is sent from the user terminal, a connection unit that connects to the agent terminal, and the agent terminal sends a text message including the link to the user terminal a generation unit that creates a channel for screen sharing between the user terminal and the agent terminal when the link is selected from the user terminal after transmission to the user terminal; and a transmission unit for mapping and storing the inquiry message and the response message when a response message corresponding to the inquiry message of the user terminal is input by voice from the counselor terminal, and transmitting it to the city gas server in real time and an image service providing server to
The image service providing server,
At least one artificial intelligence algorithm by converting video data corresponding to the screen shared between the user terminal and the agent terminal, and text data converted from speech to text input from the user terminal and the agent terminal to STT (Speech to Text) Further comprising a database conversion unit for accumulating as a dataset (DataSet) for learning and testing, the dataset is accumulated on a cloud platform or an on-premises platform,
The image service providing server,
When an input event corresponding to any one or a combination of at least one of text input using a remote pointer, a live pen, a whiteboard, and a keyboard occurs in the agent terminal, to the user terminal It further includes; an input unit for sharing the input event in real time;
The image service providing server,
After being connected between the user terminal and the agent terminal, when the agent terminal requests a multi-remote support linkage, a link for connecting the user terminal and the agent terminal is transmitted to the field technician terminal, and the user terminal and the agent terminal and a multi-unit that allows screen sharing between the field technician terminals;
Non-face-to-face video service providing system for city gas customer consultation, characterized in that it further comprises.
delete delete delete

Images