KR102127909B1 - Chatting service providing system, apparatus and method thereof - Google Patents

Abstract

The present invention relates to a chat service providing system that provides a chat service using one or more of video, voice, and text, and an apparatus and method therefor. When providing a chat service, a user interface screen is divided into multiple areas, The divided areas are allocated for each type of information, and at least one of video information, audio information, and text information is separated from chat data transmitted and received through the communication unit, and the separated video information, audio information, and text information are allocated through the assigned area. It is implemented to visualize and output each.

Description

Chat service providing system, apparatus and method therefor {Chatting service providing system, apparatus and method thereof}

The present invention relates to a chat service using one or more of video, audio, and text, and more specifically, a user interface screen is divided into a plurality of areas, and video information and audio information transmitted and received through the chat service, And a chat service providing system for separating and outputting text information through different areas, and an apparatus and method therefor.

The contents described in this section merely provide background information for this embodiment, and do not constitute a prior art.

2. Description of the Related Art Recently, with the development of information and communication technology and semiconductor technology, the spread and use of portable terminals are rapidly increasing. In particular, recent portable terminals have reached the mobile convergence stage through performance enhancement, not only in their traditional unique areas but also in the areas of other terminals.

Typically, a mobile communication terminal has a TV viewing function (for example, mobile broadcasting such as Digital Multimedia Broadcasting (DMB) or Digital Video Broadcasting), music playback function (eg, MP3 (eg, MP3)) Various functions such as MPEG Audio Layer-3), picture taking function, and Internet access function are being added.

In addition, recent mobile communication terminals have developed various types of user functions based on the open OS in the form of smart phones that users can install arbitrarily, and provide various conveniences and entertainments. It supports application to be easily obtained, and it can process many functions processed by PC with high performance.

As a service commonly used in a mobile communication terminal, there is a communication or community service for exchanging various types of information between users. As a service available in such a mobile communication terminal, a short message service (SMS) and a multimedia message service (MMS) are basically provided, and according to the appearance of a smart phone equipped with an open OS, an instant messenger or a messenger service such as a mobile messenger , And even e-mail services. Such a service is very expensive because it is cheaper than a call service and can transmit its intention or message to the other user more simply than a direct call.

In particular, in the case of recent messenger service, convergence with a voice/video call function has been attempted to exchange text and audio in real time in addition to basically exchanging text.

On the other hand, since both the text chat and the call can be used through the mobile communication terminal, the user convenience is improved. However, when making a video or voice call during text chat, there is an inconvenience in that the chat must be stopped.

Published Korean Patent No. 10-2013-0005174, published on January 15, 2013 (name: portable electronic device and control method thereof)

Accordingly, the present invention has been proposed to solve a problem in which a chat function using video or voice and a text chat can be performed simultaneously in a chat service using at least one of video, voice, and text. A chat service providing system that divides into a plurality of areas, allocates at least one of video, voice, and text to each area, separates video, voice, and text transmitted and received through the chat service and outputs them through the assigned area, It is intended to provide an apparatus and method for this.

As one means of solving the above-described problems of the present invention, a communication unit for transmitting and receiving data through a communication network; An input unit for inputting a user request; An output unit for outputting a user interface screen; And, when providing a chat service using at least one of video, voice, and text according to a user request, divides the user interface screen into multiple areas, allocates the divided areas for each type of information, and transmits and receives chat data through a communication unit. It provides a terminal device comprising a control unit for separating at least one of the video information, the audio information and the text information, and visualizing and outputting the separated video information, the audio information and the text information through the allocated area, respectively. .

In the terminal device according to the present invention, the control unit may convert speech information into text and output it, or extract the wavelength of the speech information and visualize and output the extracted wavelength.

In the terminal device according to the present invention, the control unit may display the text converted from the voice information of the users participating in the chat service in chronological order in the same area.

In addition, the controller of the terminal device according to the present invention may further divide the area for each user participating in the chat service, and output at least one of the separated video information, voice information, and text information for each user.

At this time, the controller may arrange a plurality of areas divided for each type of user and information on the user interface screen to be mapped based on the user.

In addition, the control unit of the terminal device according to the present invention may select a target to visualize voice information among users participating in a chat service through the input unit, and visualize and output only the voice information of the selected user.

In addition, the present invention is another solution of the above-described problems, a communication unit for communicating with two or more terminal devices through a communication network, for transmitting and receiving data for a chat service; A user interface that relays chat data including at least one of video information, audio information, and text information in real time between two or more terminal devices accessed through a communication unit, but is divided into two or more areas and each area is assigned to each type of information A region that provides a screen and separates at least one of video information, audio information, and text information from chat data transmitted and received between two or more terminal devices, and assigns at least one of the separated video information, audio information, and text information to the information It provides a service device for providing a chat service, characterized in that it comprises a service providing unit to visualize and output through.

In the service device according to the present invention, the service providing unit may convert speech information into text and output it, or extract the wavelength of the speech information and visualize and output the extracted wavelength.

In addition, in the service device according to the present invention, the control unit may select a visualization target of voice information among users participating in the chat service from each terminal device, and visualize and provide only the voice information of the selected user.

In addition, the present invention is another solution for the above-described problems, and when providing a chat service using at least one of video, audio, and text, the user interface screen is divided into two or more regions, and each of at least one of video, audio, and text. Assigned to one, separating at least one of video information, audio information, and text information of the chat data transmitted and received through the chat service, and visualizing at least one of the separated video information, audio information, and text information through the assigned area A plurality of terminal devices to output; And it provides a chat service providing system comprising a service device for relaying chat data between a plurality of terminal devices.

In addition, the present invention is another solution of the above-described problem, the apparatus for providing a chat service, when the chat service is started, dividing the user interface screen into two or more areas; Assigning one of video, audio, and text to two or more areas, respectively; Separating at least one of video information, audio information, and text information from chat data transmitted and received through a chat service; And visualizing and outputting at least one of the separated video information, audio information, and text information through the allocated area.

In the method of providing a chat service according to the present invention, the outputting step includes: converting speech information separated from chat data into text, and outputting the converted text in an area allocated to speech; And extracting the wavelength of the voice information separated from the chat data, and visualizing the extracted wavelength to output it to a region allocated to the voice.

In addition, in the method for providing a chat service according to the present invention, the step of dividing determines the number of regions to be divided according to the type of information included in the chat data, and in addition, in the step of dividing, the user participating in the chat service In the step of dividing and allocating regions according to the number of regions, regions may be allocated according to types of users and information.

In addition, the present invention is another solution of the above-described problem, the apparatus for providing a chat service, when the chat service is started, receiving a selection of a visualization target of voice information among users participating in the chat service; Dividing the user interface screen into two or more regions according to the number of visualization targets of the selected voice information and the type of information included in the chat data; Allocating two or more divided areas according to types of users and information; Separating at least one of video information, audio information, and text information from chat data transmitted and received through a chat service; And visualizing and outputting at least one of the separated video information, audio information, and text information through the allocated area.

In addition, the above-described method for providing a chat service may be implemented as a program and recorded in a computer-readable recording medium.

The present invention divides a user interface screen into a plurality of areas while providing a chat service using one or more of video, audio, and text, and provides video information, audio information, and text information transmitted and received through the chat service. It is possible to clearly communicate the intentions of participating users by visualizing and outputting them separately through different areas.

In particular, the present invention, while performing a video chat or voice chat, while outputting the user's voice information audibly, by converting the voice information to text and output through the user interface screen, each user during the chat uttered Try to understand the content more clearly or make it possible to check it later.

In addition, the present invention can perform a video chat or a voice chat, extract the wavelength of the voice information of users, visualize and output the wavelength of the voice information, and provide a variety of new types of chat service .

1 is a diagram showing the configuration of a chat service system according to embodiments of the present invention.
2 is a diagram showing the configuration of a terminal device in a chat service system according to embodiments of the present invention.
3 is a diagram illustrating the configuration of a service device in a chat service system according to embodiments of the present invention.
4 to 7 are flowcharts illustrating a method for providing a chat service according to embodiments of the present invention.
8 is a diagram illustrating an example screen of a chat service according to an embodiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, in the following description and accompanying drawings, detailed descriptions of well-known functions or configurations that may obscure the subject matter of the present invention are omitted. In addition, it should be noted that the same components throughout the drawings are represented by the same reference numerals as much as possible.

The terms or words used in the present specification and claims described below should not be construed as being limited to ordinary or dictionary meanings, and the inventor is appropriate as a concept of terms for explaining his or her invention in the best way. Based on the principle that it can be defined, it should be interpreted as a meaning and a concept consistent with the technical idea of the present invention. Therefore, the embodiments shown in the embodiments and the drawings described in this specification are only the most preferred embodiments of the present invention, and do not represent all of the technical spirit of the present invention, and can replace them at the time of this application. It should be understood that there may be equivalents and variations.

In addition, when referring to a component being "connected" or "connected" to another component, it means that it can be connected or connected logically or physically. In other words, it may be understood that a component may be directly connected to or connected to other components, but other components may exist in the middle and may be connected or connected indirectly.

In addition, the terms used in this specification are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In addition, terms such as "comprises" or "have" described herein are intended to indicate that there are features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, or one or more thereof. It should be understood that the above other features or numbers, steps, actions, components, parts or combinations thereof are not excluded in advance.

Prior to the description, the chat service according to embodiments of the present invention means a service that enables real-time conversation between two or more users by exchanging video information and audio information of each user in real time between two or more users based on a communication network. The concept is to include an instant messenger service including a call function, an instant messenger service interworking with a call service application, and a VoIP and mVoIP-based call service.

In addition, the video information and audio information provided by the chat service system according to the present invention is a data packet that converts a user's video and audio into digital signals based on a mobile voice over Internet Protocol (mVoIP). It includes information.

First, with reference to FIG. 1, the overall configuration of a chat service system according to the present invention will be briefly described.

1 is a view showing the configuration of a chat service system according to an embodiment of the present invention.

Referring to FIG. 1, a chat service system may include one or more terminal devices 100 and service devices 200 connected through a communication network 10.

The terminal device 100 refers to a terminal capable of transmitting and receiving various data via the communication network 10 according to a user's key operation, and includes a tablet PC, a laptop, and a personal computer (PC). ), a smart phone, a personal digital assistant (PDA), a smart TV, and a mobile communication terminal.

In particular, the terminal device 100 is a terminal that performs data communication including video and audio using the communication network 10, a browser, programs and protocols for communicating with the service device 200 via the communication network 10 Refers to a terminal having a memory for storing, a microprocessor for executing and controlling various programs, and the like. That is, the terminal device 100 is capable of server-client communication with the service device 200, and any terminal can be used if a chat service can be executed, and all communication computing devices such as notebook computers, mobile communication terminals, PDAs, and the like. It is a broad concept to include. Meanwhile, the terminal device 100 is preferably manufactured in a form having a touch screen, but is not limited thereto.

In particular, the terminal device 100 according to the present invention performs a chat service using one or more of video, voice, and text. Specifically, while providing a chat service, the terminal device 100 receives at least one of a user's video, voice, and text and transmits it to one or more other terminal devices through the service device 200, while the other terminal device By outputting at least one of the video, audio, and text of the other party received from the user to the user, real-time conversation using at least one of video, audio, and text between users is enabled. Generally, the video and text included in the chat service are output as a visual signal through a display means, and the audio is output as a hearing signal through a speaker or the like.

At this time, the terminal device 100 according to the present invention divides the user interface screen displayed while providing the chat service into a plurality of areas, and then separates the video information, voice information, and text information transmitted and received among the chat services, respectively. Visualize it in another area and output it. At this time, the visualization of the voice information may be achieved by converting the voice information into text or extracting and visualizing the wavelength of the voice information.

For example, in the case of a voice chat, while outputting the voices of users participating in the chat service as an audio signal, the voice information of each user can be converted into text, or the wavelength of the voice information can be visualized and output through a user interface screen. .

In addition, in the case of a video chat, while outputting video of users participating in the chat service, the user's voice information may be changed to text or the wavelength of the voice information may be visualized and output to a different area from the video information.

Here, the chat service may perform a text chat together with a video chat or a voice chat at the same time, wherein the text information is also output through a different area from the video information or the voice information.

At this time, video information, audio information, and text information output through different areas are mapped and output based on users participating in the chat service, so that the relevance of video information, audio information, and text information can be intuitively recognized. Can.

The service device 200 provides a chat service by interworking with one or more terminal devices 100 through the communication network 10. At this time, the service device 200 may provide a client program or application for interworking with a chat service to the one or more terminal devices 100, and through this, interwork with the one or more terminal devices 100. Specifically, the service device 200 registers two or more terminal devices 100 as a chat group according to requests and responses from the one or more terminal devices 100, and two or more terminal devices 100 registered as chat groups. ) Relays chat data including at least one of video information, audio information, and text information of each user.

In an embodiment of the present invention, the service device 200 provides a user interface screen divided into two or more areas to the terminal device 100 according to a request from the terminal device 100, and the terminal device 100 ), video information, audio information, and text information included in chat data transmitted and received can be controlled to be output through different areas.

The terminal device 100 and the service device 200 are interlocked through the communication network 10, and the communication network 10 is an Internet protocol using various wired/wireless communication technologies such as an Internet network, an intranet network, a mobile communication network, and a satellite communication network. A network that can transmit and receive data. The communication network 10 is a closed area network such as a local area network (LAN), a wide area network (WAN), and an open network such as the Internet, as well as code division multiple access (CDMA) and wideband code division (WCDMA). It is a concept that collectively refers to networks such as Multiple Access (GSM), Global System for Mobile Communications (GSM), Long Term Evolution (LTE), Evolved Packet Core (EPC), and next-generation networks and computing networks to be implemented in the future.

The configuration of the terminal device according to the embodiments of the present invention in the chat service system configured as described above will be described in detail with reference to the accompanying drawings.

2 is a diagram illustrating a configuration of a terminal device 100 in a chat service system according to embodiments of the present invention.

Referring to FIG. 2, the terminal device 100 may include a control unit 110, an input unit 120, a communication unit 130, a storage unit 140, an output unit 150, and an audio processing unit 160. have. Of course, the terminal device 100 may further include other components, such as a sensor and a GPS receiver, in addition to the above-described configuration, but these are not directly related to the present invention, and thus illustration and description are omitted.

The function of each component provided in the terminal device 100 according to the present invention and interworking between components will be described in detail.

First, the control unit 110 is a configuration for performing overall control of the terminal device 100, and may be implemented by including one or more processors. This processor may be a single-threaded processor, and in other implementations, the processor may be a multithreaded processor. Furthermore, the control unit 110 may operate by processing commands stored on a memory or a storage device through the one or more processors. In this case, the commands may include interpreted commands such as script commands such as JavaScript or ECMAScript commands, executable code, or other commands stored in a computer-readable medium.

When the power is turned on, the control unit 110 executes an operating system (OS) program stored in the storage unit 140 to build an execution environment in which other application programs can be executed, and based on this, one or more other application programs are installed. Execute to perform a predetermined function.

In particular, the control unit 110 may execute an application program related to the chat service and execute the chat service according to the present invention. To this end, the control unit 110 may include a chat processing module 111. Here, the module (Module), which is implemented as software, or hardware or a combination of software and hardware, means a component that performs a certain function, and the chat processing module 111 according to the present invention includes a processor for a chat-related application program. It is implemented by performing the functions described below.

Specifically, the control unit 100 activates by executing the chat processing module 111 according to a user's request for a chat service, and requests the creation of two or more chat groups through the activated chat processing module 111, Requests to participate in a specific chat group, and then, through a connected chat session, processes and outputs the chat data transmitted and received and the chat data of users participating in the chat group.

At this time, the chat data may include video information, audio information, and text information. Here, the video information may be an image captured by the user, the audio information may be an audio signal uttered by the user, and the text information may be text input by the corresponding user.

The chat processing module 111 configures a user interface screen divided into two or more regions for outputting such chat data. Then, any one of video, audio, and text is assigned to the two or more regions, respectively. Then, the chat processing module 111 separates at least one of video information, audio information, and text information from chat data transmitted and received through the chat service, and visualizes the separated video information, audio information, and text information, Output to the allocated area.

In this case, the chat processing module 111 may convert the voice information into text and output it, or extract the wavelength of the voice information and visualize and output the extracted wavelength.

In this case, the chat processing module 111 may further divide the area for each user participating in the chat service, and output at least one of voice information, video information, and text information for each user. For example, when chatting, video information is divided and displayed for each user, while voice information and text information of a plurality of users participating in the chat can be displayed by arranging in the order of occurrence time in one area without distinction by user. . In another example, video information, audio information, and text information for each user may be separately output. At this time, by arranging each area allocated to the video information, audio information, and text information on the user interface screen so as to be mapped based on users participating in the chat service, intuitively separated and output video information, audio information, and text information To be connected. For example, for each user, regions in which voice information and text information are output are sequentially arranged under a region in which image information is output, so that it is possible to easily recognize which user's intention each voice information and text information is. .

In addition, the chat processing module 111 may receive at least one selected from among two or more users participating in a chat service through the input unit 130 and visualize and output at least one of video, text, and voice of the selected user. have.

For example, during video chat and voice chat, only the voice information of the selected user may be visualized, for example, the voice information may be texted or the wavelength of the voice information may be visualized and output. At this time, voice information of all users participating in the chat is output as an audio signal through the audio processing unit 160. Therefore, the user can visually check only the voice information of the desired user during the voice chat and separately check through the user interface screen.

Next, among the configuration of the terminal device 100 according to an embodiment of the present invention, the communication unit 120 may communicate with the service device 200 through the communication network 10 to transmit and receive data related to a chat service. . Specifically, the communication unit 120 transmits or receives a request or response related to a chat service, such as a request to create a chat group or a request to join a specific chat group, to the service device 200. In addition, the communication unit 120 connects a chat session under the control of the service device 200, and through the connected chat session, chat data including at least one of video information, voice information, and text information of users participating in the chat. Send and receive.

The communication unit 120 may transmit and receive data through various communication methods as well as wired and wireless methods. In addition, the communication unit 120 may transmit and receive data using one or more communication methods, and for this purpose, the communication unit 120 may include a plurality of communication modules that transmit and receive data according to different communication methods.

In addition, the input unit 130 is a configuration for the operation of the terminal device 100, specifically, according to the user's operation, the user input corresponding to the user request or information for controlling the function or operation of the terminal device 100 The signal is generated and transmitted to the control unit 110. The input unit 130 may be implemented with various input means that are currently commercialized or commercially available in the future. For example, as well as general input devices such as a keyboard, mouse, joystick, touch screen, and touch pad, the user's motion It may include gesture input means for detecting and generating a specific input signal, and voice recognition means for recognizing a user's voice. In particular, the input unit 130 may be used to create a chat group, select a participant, and select whether to participate.

Subsequently, the storage unit 140 is configured to store data or programs executed or processed by the control unit 110. Basically, the storage unit 140 supports an operating system (OS) for booting the terminal device 100 and operating each of the above-described components, various user functions, for example, a call function of the terminal device 100. User functions for playback, MP3 user functions for reproducing other sound sources, image output functions for reproducing images such as photos, and application programs for supporting video replay functions, etc. can be stored. In particular, the storage 140 may store an application program for a chat service. The storage unit 140 includes optical media such as magnetic media such as hard disks, floppy disks, and magnetic tapes, compact disk read only memory (CD-ROM), and digital video disks (DVD). , Magnetic-Optical Media, such as a Floppy Disk, and ROM, Random Access Memory (RAM), and Flash memory.

In addition, when the output unit 150 is configured as a touch screen, the storage unit 140 may store a key map or a menu map for operating the touch screen. Here, the key map and the menu map may each have various forms. That is, the key map may be a keyboard map, a 3*4 key map, a QWERTY key map, etc., or a control key map for operating control of the currently active application program. In addition, the menu map may be a menu map for controlling application operation that is currently active.

The output unit 150 is a configuration that provides a user to recognize an operation result or state of the terminal device 100, and in particular, outputs information visually using display means. For example, the output unit 150 includes a discharge light display (ELD), a vacuum fluorescent display (VFD), a liquid crystal display (LCD), a light emitting diode display, a plasma display panel (PDP), and an organic light emitting diode. It may include various two-dimensional display means such as (OLED), as well as three-dimensional display means such as holography or laser beam.

Particularly, in the present invention, the output unit 150 outputs a user interface screen divided into a plurality of areas configured by the chat processing module 111 under the control of the control unit 110, and the user interface screen Visualize and output video information, audio information, and text information to each area of. For reference, the user interface screen may further include a user selectable GUI element in addition to an area in which the video information, audio information, and text information are output.

Finally, the audio processing unit 160 in the terminal device 100, as a configuration for the recognition of voice information and output of voice information, in conjunction with a microphone (MIC), the voice information recognized through the microphone (MIC) Is digitally converted and transmitted to the control unit 110, and outputs voice information transmitted from the control unit 110 as a hearing signal through a speaker SPK. At this time, the audio processing unit 160 further performs signal processing such as adjusting the size (volume) of the output voice information and removing noise.

Next, the configuration and operation of the service device 200 according to the present invention in the chat service system will be described in detail with reference to FIG. 3.

3 is a diagram illustrating a configuration of a service device 200 in a chat service system according to embodiments of the present invention.

3 is a diagram illustrating the configuration of a service device in a chat service system according to embodiments of the present invention.

Referring to FIG. 3, the service device 200 may include a communication unit 210, a service providing unit 220, and a storage unit 230.

The communication unit 210 may transmit and receive data for a chat service with one or more terminal devices 100 through the communication network 10. The communication unit 210 may transmit and receive data through various communication methods as well as wired and wireless methods. In addition, the communication unit 210 may transmit and receive data using one or more communication methods, and for this purpose, the communication unit 210 may include a plurality of communication modules that transmit and receive data according to different communication methods. Specifically, the communication unit 210 transmits and receives a request for the creation, participation, etc. of one or more terminal devices 100 and a chat group, connects a chat session with the one or more terminal devices 100, and establishes a connected chat session. Through this, chat data including one or more of video information, text information, and audio information is transmitted and received.

The service providing unit 220 according to embodiments of the present invention is a configuration for controlling a chat service by interworking with two or more terminal devices 100 through the communication network 10, and may be implemented by including one or more processors. At this time, the processor mounted on the service device 200 according to the present invention may process a program command for executing the method according to the present invention to perform a predetermined function described below. In one implementation, this processor may be a single-threaded processor, and in other implementations, the processor may be a multithreaded processor. Specifically, the service providing unit 220 is functionally interlocked with the communication unit 210, and includes one or more of video information, audio information, and text information between two or more terminal devices 100 set as a chat group. To relay chat data.

In addition, the service providing unit 220 may further perform user interface control according to the present invention while providing a chat service. Specifically, the service providing unit 220 relays at least one of video information, voice, and text in real time between two or more terminal devices 100 registered as a chat group, and the two or more terminal devices may transmit to at least two areas. Each terminal device provides a divided user interface screen, separates video, audio, and text of chat data transmitted and received between the two or more terminal devices, and allocates and outputs the separated video, audio, and text to different areas, respectively. (100) is controlled.

Particularly, while outputting video information of a plurality of users participating in a chat, the service providing unit 220 may control to select and output the voice information and text information according to a user's selection. That is, each terminal device 100 may visualize and output only voice information or text information of a specific user desired by the corresponding user.

In this case, even if the terminal devices 100 registered as the same chat group, the chat data of the user visualized and output according to each request may be different.

In addition, the storage unit 230 of the service device 200 is configured to store data and programs necessary to provide a chat service. Specifically, the storage 230 may store information of the currently generated chat group and terminal devices 100 registered in the chat group, and may store chat data for each chat group for a predetermined time.

The storage unit 230 includes an optical media such as a hard disk, a floppy disk, and a magnetic tape (Magnetic Media), a CD-ROM (Compact Disk Read Only Memory), and a DVD (Digital Video Disk). , Magnetic-Optical Media, such as a Floppy Disk, and ROM, Random Access Memory (RAM), and Flash memory.

Next, a method for providing a chat service based on the chat service system configured as described above will be described with reference to FIGS. 4 to 8.

4 to 7 are flowcharts illustrating a method of providing a chat service according to embodiments of the present invention, and FIG. 8 is a view showing an example screen of a chat service according to an embodiment of the present invention.

For reference, each step of the chat service providing method described below may be performed through any one of the terminal device 100 and the service device 200, hereinafter, it is described based on the terminal device 100. I will do it.

First, referring to FIG. 4, the terminal device 100 according to the present invention performs a chat service supporting real-time conversation by exchanging at least one of video information, voice information, and text information between two or more users according to a user request The terminal device 100 according to the present invention confirms whether such a chat service is started (S105). Specifically, the step S105 is performed according to the response of other terminal devices through the service device 200 after sending a message requesting the creation of a chat group to the service device 200 or the service device 200 If a response to accept participation is transmitted to a request for creating a chat group of another terminal device transmitted through ), it may be determined that the chat service has been started.

When the chat service is started as described above, the terminal device 100 divides a user interface screen for outputting chat-related information into two or more areas (S110).

Then, the terminal device 100 allocates at least one of video, audio, and text to each of the divided two or more regions (S115). At this time, the step S115 may be performed according to a default setting value or may be performed according to a user selection. In addition, by arranging regions allocated to the same user so that they can be mapped to each other, the user can intuitively recognize the association between information that is visualized and output.

Then, through the communication unit 120 of the terminal device 100, through a chat session connected to the corresponding chat service, chat data including at least one of video information, voice information, and text information of users participating in the chat is transmitted and received. When starting, the control unit 110 separates video information, text information, and voice information from chat data transmitted and received through the chat processing module 111, respectively (S120).

Then, the chat processing module 111 of the terminal device 100 visualizes and outputs one of the separated video information, text information, and voice information to a corresponding area of the assigned user interface screen (S125). At this time, when the chat data includes voice information, the terminal device 100 allocates the user interface screen while outputting the voice information as an audio signal through the audio processing unit 160, similar to a normal processing method. It can be visualized and output through the area.

At this time, visualization of the voice information may be performed in the same manner as in FIG. 5 or FIG. 6.

According to an embodiment of the present invention, in step S125, the chat processing module 111 of the terminal device 100, as shown in Figure 5, when the chat data includes voice information, TTS (Text To Speech) technology may be used to convert the separated speech information into text using various speech recognition technologies (S205). Then, the converted voice information may be output to the allocated area of the user interface screen (S210). As a result, the voice content of the user participating in the chat can be confirmed by text.

According to another embodiment of the present invention, in step S125, the chat processing module 111 of the terminal device 100 extracts the wavelength of the audio information, as shown in FIG. 6 (S305), and extracts the extracted wavelength. After visualizing (S310), the visualized wavelength may be output to an area allocated to voice information of the user interface screen (S315). As a result, it is possible to visually recognize a change in the volume or tone of the voice of the user participating in the chat.

In addition, according to another embodiment of the present invention, in visualizing and outputting video information, voice information, and text information of users participating in a chat through different areas, the voice information is selectively visualized only for a user selected among the users. Can print

7 is a flowchart illustrating a method for providing a chat service according to another embodiment of the present invention.

Referring to FIG. 7, the terminal device 100 according to the present invention checks whether a chat service supporting real-time conversation is started by exchanging at least one of video information, voice information, and text information between three or more users according to a user request (S405). As described above, the step S205 is performed according to the response of other terminal devices through the service device 200 after transmitting a message requesting the creation of a chat group to the service device 200 or the service device When a response for accepting participation is transmitted in response to a request for creating a chat group of another terminal device transmitted through (200), it may be determined that the chat service is started. In addition, in step S205, it may be further confirmed whether there are three or more participants in the chat service.

When a chat service in which three or more users participate is started, the terminal device 100 selects a target to visualize voice information among users participating in the chat service (S410). The terminal device 100 may output a participant list to the output unit 150, and then select at least one of the users included in the participant list through the input unit 130.

Thereafter, the terminal device 100 divides the user interface screen for outputting chat-related information into two or more areas, depending on the number of users participating in the chat service and the type of information exchanged in the chat service. The region is divided (S415). For example, in the case of a video chat, the terminal device 100 outputs as many video information output areas as the number corresponding to the number of users participating in the chat, and outputs voice information as many as the number of users selected as the audio information visualization targets. The area can be divided.

On the other hand, in the case of a voice chat, the terminal device 100 may divide an area for outputting voice information by the number of users selected as the object for visualizing the voice information.

Then, the terminal device 100 allocates a user and a kind of information for each divided area (S420). At this time, the areas allocated to the same user are arranged to be mapped to each other, so that the user can intuitively recognize the association between the information that is visualized and output.

Then, through a chat session connected to the corresponding chat service, when the chat data including at least one of video information, voice information, and text information of users participating in the chat starts to be transmitted and received, the terminal device 1000, chat processing At least one of video information, text information, and audio information is separated from the chat data transmitted and received through the module 111 (S425). At this time, the video information, text information, and audio information may be classified for each corresponding user.

Then, the chat processing module 111 of the terminal device 100 visualizes and outputs one of the separated video information, text information, and voice information to a corresponding area of the assigned user interface screen (S125). Here, the visualization of the voice information may be made as shown in FIGS. 5 and 6 described above.

In addition, the voice information included in the chat data is also output as an audio signal through the audio processing unit 160 of the terminal device 100.

Therefore, the user of the terminal device 100 can check separately by visualizing only the voice information of a specific user while performing a chat as in the past.

Through this, the user can more accurately check the chat content of a specific user during the chat. For example, when a specific user's voice cannot be easily heard due to noise during chat, the content of the user's voice can be visualized and output, thereby accurately recognizing the content of the chat.

8 is a diagram illustrating a chat service screen according to the present invention. In FIG. 8, it is a diagram illustrating a case where two users chat using video, voice, and text.

Referring to FIG. 8, the user interface screen 80 output to the output unit 150 of the terminal device 100 according to the present invention includes a first area 81 allocated to video information of the other party and voice of the other party A second area 82 allocated to information and a third area 83 allocated to text information of the other party may be included. In the first area 81, the video of the other party participating in the chat service is displayed, and in the second area 82, the voice of the other party participating in the chatting service is visualized and displayed, and in the third area 83 Text transmitted by the other party participating in the service may be displayed. In this case, the second region 82 may display text corresponding to the other party's voice or visualize the wavelength of the other party's voice. According to this, the user can accurately recognize the utterance of the other party or the voice tone or state of the other party by referring to the second area of the user interface screen 80 during the chat.

The method for providing a chat service according to the present invention may be implemented in a form of software readable through various computer means and recorded in a computer-readable recording medium. Here, the recording medium may include program instructions, data files, data structures, or the like alone or in combination. The program instructions recorded on the recording medium may be specially designed and configured for the present invention, or may be known and available to those skilled in computer software. For example, the recording medium includes magnetic media such as hard disks, floppy disks and magnetic tapes (Magnetic Media), compact disk read only memory (CD-ROM), optical media such as DVD (Digital Video Disk), and optical media. Includes magneto-optical media, such as a floppy disk, and hardware devices specifically configured to store and execute program instructions such as ROM, random access memory (RAM), flash memory, etc. do. Examples of program instructions may include high-level language code that can be executed by a computer using an interpreter or the like, as well as machine language codes produced by a compiler. Such a hardware device can be configured to operate as one or more software modules to perform the operation of the present invention, and vice versa.

Although the specification and drawings describe exemplary device configurations, the functional operations and subject implementations described herein may be implemented with other types of digital electronic circuits, or the structures disclosed herein and their structural equivalents. It may be implemented in the included computer software, firmware or hardware, or a combination of one or more of them. Implementations of the subject matter described herein are one or more computer program products, that is, one for computer program instructions encoded on a tangible program storage medium to control or thereby execute the operation of a device according to the invention. It can be implemented as the above modules. The computer-readable medium may be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of materials affecting a machine-readable propagated signal, or a combination of one or more of these.

The terms "system" and "apparatus" used herein encompass all apparatus, devices and machines for processing data, including, for example, programmable processors, computers or multiple processors or computers. In particular, the system may include, in addition to hardware, code that forms an execution environment for the computer program upon request, such as code constituting the processor firmware, protocol stack, database management system, operating system, or a combination of one or more of these. have.

A computer program (also known as a program, software, software application, script or code) mounted on a device according to the present invention and executing the method according to the present invention is a programming comprising a compiled or interpreted language or a priori or procedural language It can be written in any form of language, and can be deployed in any form, including standalone programs or modules, components, subroutines, or other units suitable for use in a computer environment. Computer programs do not necessarily correspond to files in the file system. The program is in a single file provided to the requested program, or in multiple interactive files (e.g., one or more modules, files storing subprograms or parts of code), or part of a file holding other programs or data (Eg, one or more scripts stored in a markup language document). The computer program may be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a communication network.

Implementations of the subject matter described herein include, for example, a back-end component such as a data server, for example, a middleware component such as an application server, or, for example, a web browser or graphic user that allows a user to interact with the implementation of the subject matter described herein. It can be implemented in a front-end component, such as a client computer with an interface, or in a computing system that includes all one or more combinations of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication, such as a communication network.

This specification includes details of many specific implementations, but these should not be understood as limiting on the scope of any invention or claim, but rather as a description of features that may be specific to a particular embodiment of the particular invention. It should be understood. Certain features that are described in this specification in the context of separate embodiments may be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented in multiple embodiments individually or in any suitable subcombination. Further, although features may operate in a particular combination and may initially be depicted as so claimed, one or more features from the claimed combination may be excluded from the combination in some cases, and the claimed combination subcombined. Or sub-combinations.

Likewise, although the operations are depicted in the drawings in a particular order, it should not be understood that such operations should be performed in the particular order shown or in sequential order, or that all shown actions should be performed in order to obtain desirable results. In certain cases, multitasking and parallel processing may be advantageous. Also, the separation of various system components of the above-described embodiments should not be understood as requiring such separation in all embodiments, and the described program components and systems are generally integrated together into a single software product or packaged in multiple software products. You should understand that you can.

Specific embodiments of the subject matter described herein have been described. Other embodiments are within the scope of the following claims. For example, the operations recited in the claims may be performed in different orders while still achieving desirable results. As an example, the process illustrated in the accompanying drawings does not necessarily require that particular illustrated order or sequential order to obtain desirable results. In certain implementations, multitasking and parallel processing may be advantageous.

The described description presents the best mode of the present invention, and provides examples for explaining the present invention and for those skilled in the art to make and use the present invention. This written specification is not intended to limit the invention to the specific terms presented. Therefore, although the present invention has been described in detail with reference to the above-described examples, those skilled in the art can make modifications, alterations, and modifications to the examples without departing from the scope of the present invention.

Therefore, the scope of the present invention should not be determined by the described embodiments, but should be determined by the claims.

The present invention can be applied to a chat service using one or more of video, audio, and text. The user interface screen is divided into a plurality of areas, and video information, audio information, and text transmitted and received through the chat service By visualizing and outputting information through different areas, it is possible to clearly communicate the intentions of participating users.

In particular, the present invention, while performing a video chat or voice chat, while outputting the user's voice information audibly, by converting the voice information to text and output through the user interface screen, each user during the chat uttered Try to understand the content more clearly or make it possible to check it later.

In addition, the present invention can perform a video chat or a voice chat, extract the wavelength of the voice information of users, visualize and output the wavelength of the voice information, and provide a variety of new types of chat service .

10: communication network 100: terminal device 110: control unit
111: chat processing module 120: communication unit 130: input unit
140: storage unit 150: output unit 160: audio processing unit
200: service device 210: communication unit 220: service providing unit
230: storage

Claims (18)

Communication unit for transmitting and receiving data through a communication network;
An input unit for inputting a user request;
An output unit for outputting a user interface screen; And
According to the user request, when providing a chat service using at least one of video, voice, and text, the user interface screen is divided into multiple areas, the divided areas are allocated for each type of information, and transmitted and received through the communication unit. A control unit separating at least one of video information, audio information, and text information from the chat data, and visualizing and outputting at least one of the separated video information, audio information, and text information through an allocated region;
It includes,
The control unit,
The video information is displayed by dividing the area for each user, and text information of multiple users participating in the chat is displayed in the order of occurrence time in one area without distinction by user.
In the case of a video chat, it is divided into an area for outputting video information corresponding to the number of users participating in the chat, and an area for outputting audio information as many as the number of users selected as targets for visualizing audio information,
In the case of a voice chat, a terminal device characterized in that it is divided into regions for outputting voice information as many as the number of users selected as the object for visualizing the voice information. The method of claim 1, wherein the control unit
A terminal device characterized in that the voice information is converted into text and output. The method of claim 2, wherein the control unit
A terminal device characterized in that the text converted from the voice information of users participating in the chat service is displayed in the same area in chronological order. The method of claim 1, wherein the control unit
A terminal device characterized in that the wavelength of the voice information is extracted and the extracted wavelength is visualized and output. The method of claim 1, wherein the control unit
The terminal device, characterized in that further dividing the area for each user participating in the chat service, and outputting at least one of the separated video information, audio information, and text information for each user. The method of claim 5, wherein the control unit
A terminal device characterized by arranging a plurality of areas divided for each type of user and information on the user interface screen so as to be mapped based on the user. The method of claim 1, wherein the control unit
A terminal device characterized in that a target for visualizing voice information is selected from users participating in the chat service through the input unit, and only the voice information of the selected user is visualized and output. delete delete delete delete delete A device that provides chat services,
When the chat service is started, dividing the user interface screen into two or more areas;
Assigning one of video, audio, and text to the two or more regions, respectively;
Separating at least one of video information, audio information, and text information from chat data transmitted and received through the chat service;
And visualizing and outputting at least one of the separated video information, audio information, and text information through the allocated area,
The step of outputting,
The video information is displayed by dividing the area for each user, and text information of multiple users participating in the chat is displayed in the order of occurrence time in one area without distinction by user.
The dividing step,
In the case of a video chat, it is divided into an area for outputting video information corresponding to the number of users participating in the chat, and an area for outputting audio information as many as the number of users selected as audio information visualization targets,
In the case of a voice chat, a method for providing a chat service, characterized in that it is divided into regions for outputting voice information as many as the number of users selected as the object for visualizing the voice information. The method of claim 13, wherein the step of outputting
Converting voice information separated from the chat data into text, and outputting the converted text in an area allocated to voice; And
And extracting a wavelength of speech information separated from the chat data, and visualizing the extracted wavelength to output it to a region allocated to speech. The method of claim 13, wherein the dividing step
A method of providing a chat service, characterized in that the number of areas to be divided is determined according to the type of information included in the chat data. The method of claim 13,
The step of dividing
Further dividing the area according to the number of users participating in the chat service,
The allocating step is
Method of providing a chat service, characterized in that an area is allocated for each type of user and information. delete A computer-readable recording medium recording a program implemented to perform a method for providing a chat service according to any one of claims 13 to 16.

Images