KR20150108098A - Chatting service providing system, apparatus and method thereof - Google Patents

Abstract

The present invention relates to a chatting service providing system, which provides a chatting service performed by using one or more from images, voice, and texts, an apparatus and a method for the same. When the chatting service is provided, a user interface screen is divided into multiple regions; the divided regions are assigned based on kinds of the information; at least one from image information, voice information, and text information is separated from chatting data transmitted and received through a communication unit; and the image information, the voice information, and the text information which have been separated are visualized through the assigned regions and output.

Description

[Technical Field] The present invention relates to a chat service providing system, a chat service providing system,

The present invention relates to a chat service using at least one of video, audio, and text. More particularly, the present invention relates to a chat service using at least one of video, audio, and text, And a chat service providing system that separates character information and outputs the chat service through different areas, and an apparatus and method for the same.

The contents described in this section merely provide background information on the present embodiment and do not constitute the prior art.

Recently, due to the remarkable development of information communication technology and semiconductor technology, the spread and use of mobile terminals are rapidly increasing. In particular, recent mobile terminals have achieved a mobile convergence step by upgrading their performance to the area of other terminals without staying in their traditional unique areas.

Typically, the mobile communication terminal may be provided with a TV viewing function (for example, a mobile broadcasting such as DMB (Digital Multimedia Broadcasting) or DVB (Digital Video Broadcasting)), a music playback function (e.g., MP3 MPEG Audio Layer-3), a picture taking function, and an Internet access function.

In addition, recent mobile communication terminals have developed various types of convenience and enjoyment by developing various types of user functions based on an open OS, such as a smart phone capable of arbitrarily installing the user functions. The open market enables various functions It is easy to get applications, and it can handle many functions that are processed by PC with high performance.

As a service universally used in a mobile communication terminal, there is a communication or community service for exchanging various types of information among users. As a service available in such a mobile communication terminal, a short message service (SMS) and a multimedia message service (MMS) have been basically provided. In the emergence of a smart phone equipped with an open OS, a messenger service such as instant messenger or mobile messenger , And e-mail services. Such services are cheaper than the call service, and their use is very high because they can transmit their intention or message to the other users simply as compared with the direct call.

Especially, in recent messenger service, in addition to the exchange of characters, a fusion of voice and video communication functions is being attempted so that video and voice can be exchanged in real time.

On the other hand, since the user can use both the chat and the call using the character via the mobile communication terminal, the convenience of the user is improved. However, when the user makes a video or voice call while chatting, the chat has to be interrupted.

Korean Patent Publication No. 10-2013-0005174, January 15, 2013 (name: portable electronic device and control method thereof)

Accordingly, the present invention provides a chat service using at least one of video, voice, and text to solve a problem that a chat function using a video or voice and a chat using a character can be performed at the same time. A chat service providing system for assigning at least one of video, voice, and characters to each of a plurality of areas, separating video, voice, and characters transmitted and received through the chat service and outputting the separated video through an allocated area, And an apparatus and method for this purpose.

As a solving means of the above-described problem of the present invention, there is provided a communication apparatus comprising: a communication unit for transmitting and receiving data through a communication network; An input for inputting a user request; An output unit for outputting a user interface screen; And a controller for dividing the user interface screen into a plurality of areas, allocating the divided areas according to the types of information, and transmitting the chat data transmitted and received through the communication part when the chat service using at least one of video, And a control unit for separating at least one of video information, audio information and character information from the video information, visualizing the separated video information, audio information and character information through the allocated area, and outputting the visualized video information, respectively .

In the terminal apparatus according to the present invention, the control unit may convert the voice information into text and output it, extract the wavelength of the voice information, visualize the extracted wavelength, and output it.

In the terminal device according to the present invention, the controller can display the converted text from the voice information of the users participating in the chat service in the same area in chronological order.

In addition, the control unit of the terminal device according to the present invention may further divide the area for each user participating in the chat service, and may output at least one of the separated image information, voice information, and character information for each user.

At this time, the control unit may arrange a plurality of areas divided according to the type of user and information on the user interface screen so as to be mapped based on the user.

In addition, the control unit of the terminal device according to the present invention may select an object to visualize voice information among users participating in the chat service through the input unit, and visualize and output only voice information of the selected user.

According to another aspect of the present invention, there is provided a communication system comprising: a communication unit that communicates with two or more terminal devices through a communication network to transmit and receive data for a chat service; A chat system for relaying chat data including at least one of video information, voice information, and character information in real time between two or more terminal apparatuses connected through a communication unit, wherein the chat data is divided into two or more areas, And separates at least one of video information, audio information, and character information from chat data transmitted and received between two or more terminal devices, and separates at least one of the separated video information, audio information, and character information into an area And a service providing unit for visualizing the chat service and outputting the service.

In the service device according to the present invention, the service provider may convert the voice information into text and output it, extract the wavelength of the voice information, and visualize and output the extracted wavelength.

Also, in the service apparatus according to the present invention, the control unit may select a visualization target of the voice information among the users participating in the chat service from each terminal apparatus, and visualize only the voice information of the selected user.

In addition, the present invention provides, as another solution to the above problem, to provide a chat service using at least one of video, audio, and text by dividing the user interface screen into two or more areas, At least one of video information, voice information, and character information of chat data transmitted and received through a chat service is separated and at least one of separated video information, voice information, and character information is visualized through an allocated area A plurality of terminal devices for outputting; And a service device for relaying chat data between a plurality of terminal devices.

Further, the present invention provides, as another solution to the above-mentioned problem, a method for providing a chat service, comprising: dividing a user interface screen into two or more areas when a chat service is started; Allocating one of video, audio, and character to two or more areas; Separating at least one of video information, audio information, and character information from chat data transmitted and received through a chat service; And visualizing at least one of the separated image information, the voice information, and the character information through the allocated area, and outputting the visualized voice information.

In the chat service providing method according to the present invention, the step of outputting may include converting voice information separated from the chat data into text, and outputting the converted text to an area assigned to the voice; Extracting the wavelength of the voice information separated from the chat data, visualizing the extracted wavelength, and outputting the visualized voice to the area assigned to the voice.

In the chat service providing method according to the present invention, the dividing step may include determining the number of areas to be divided according to the type of the information included in the chat data, and in the dividing step, It is possible to allocate an area according to the type of user and information in the step of dividing and allocating the area according to the number of users.

In addition, another object of the present invention is to provide an apparatus for providing a chat service, when a chat service is started, selecting a visualization target of voice information among users participating in a chat service; Dividing the UI screen into two or more areas according to the number of visualization targets of the selected audio information and the type of information included in the chat data; Allocating two or more divided areas by user and information type; Separating at least one of video information, audio information, and character information from chat data transmitted and received through a chat service; And visualizing at least one of the separated image information, the voice information, and the character information through the allocated area, and outputting the visualized voice information.

In addition, the chat service providing method described above can be implemented as a program and recorded on a computer-readable recording medium.

The present invention divides a user interface screen into a plurality of areas while providing a chat service using one or more of video, audio, and text, and displays video information, audio information, and character information transmitted and received through the chat service By visualizing and outputting them separately through different areas, it is possible to clearly communicate the opinions of the participating users.

Particularly, in the present invention, during the video chatting or the voice chatting, the voice information of the users is audibly output, the voice information is converted into text and outputted through the user interface screen, Make the contents more clearly understandable or to be confirmed later.

In addition, while performing a video chat or a voice chat, the present invention can extract wavelengths of voice information of users, visualize and output the wavelength of the voice information, and provide various new types of chat services .

1 is a diagram illustrating a configuration of a chat service system according to an embodiment of the present invention.
2 is a diagram illustrating a configuration of a terminal device in a chat service system according to embodiments of the present invention.
3 is a diagram illustrating a configuration of a service apparatus in a chat service system according to embodiments of the present invention.
4 to 7 are flowcharts illustrating a method of providing a chat service according to embodiments of the present invention.
8 is a diagram illustrating an exemplary screen of a chat service according to an embodiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following description and the accompanying drawings, detailed description of well-known functions or constructions that may obscure the subject matter of the present invention will be omitted. It should be noted that the same constituent elements are denoted by the same reference numerals as possible throughout the drawings.

The terms and words used in the present specification and claims should not be construed to be limited to ordinary or dictionary meanings and the inventor is not limited to the concept of terminology for describing his or her invention in the best way. It should be interpreted as meaning and concept consistent with the technical idea of the present invention. Therefore, the embodiments described in the present specification and the configurations shown in the drawings are merely the most preferred embodiments of the present invention, and not all of the technical ideas of the present invention are described. Therefore, It is to be understood that equivalents and modifications are possible.

In addition, when referring to an element as being "connected" or "connected" to another element, it means that it can be connected or connected logically or physically. In other words, it is to be understood that although an element may be directly connected or connected to another element, there may be other elements in between, or indirectly connected or connected.

Also, the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular expressions include plural expressions unless the context clearly dictates otherwise. It is also to be understood that the terms such as " comprising "or" having ", as used herein, are intended to specify the presence of stated features, integers, It should be understood that the foregoing does not preclude the presence or addition of other features, numbers, steps, operations, elements, parts, or combinations thereof.

Prior to the description, the chat service according to embodiments of the present invention refers to a service that enables real-time conversation between two or more users by exchanging video information and voice information of each user between two or more users based on a communication network in real time An instant messenger service including a call function, an instant messenger service interworking with a call service application, and a call service based on VoIP and mVoIP.

In addition, the video information and the voice information provided in the chat service system according to the present invention are transmitted to a data packet in which a video and a voice of a user are converted into a digital signal based on a Mobile Voice over Internet Protocol (mVoIP) .

First, with reference to FIG. 1, a general configuration of a chat service system according to the present invention will be schematically described.

1 is a diagram illustrating a configuration of a chat service system according to an embodiment of the present invention.

Referring to FIG. 1, a chat service system may include one or more terminal devices 100 and a service device 200 connected through a communication network 10.

The terminal device 100 is a terminal capable of transmitting and receiving various data via the communication network 10 in accordance with a key operation of a user. The terminal device 100 includes a tablet PC, a laptop, a personal computer ), A smart phone, a personal digital assistant (PDA), a smart TV, and a mobile communication terminal.

In particular, the terminal device 100 is a terminal for performing data communication including video and audio using the communication network 10, and is a browser, a program and a protocol for communicating with the service device 200 via the communication network 10 A memory for storing various programs, a microprocessor for executing and controlling various programs, and the like. That is, if the terminal apparatus 100 is capable of server-client communication with the service apparatus 200 and is capable of executing a chat service, any terminal can be used, and all communication computing apparatuses such as a notebook computer, a mobile communication terminal, It is a broad concept that includes. Meanwhile, the terminal device 100 is preferably manufactured with a touch screen, but is not limited thereto.

In particular, the terminal device 100 according to the present invention performs a chat service using at least one of video, audio, and text. Specifically, while providing the chat service, the terminal device 100 may receive at least one of video, audio, and characters of the user and transmit the received video data to at least one other terminal device through the service device 200, Voice, and characters received from the other party and provides the user with at least one of the video, audio, and characters received from the user, thereby enabling real-time conversation using at least one of video, audio, and characters among users. Generally, images and characters included in a chat service are output as visual signals through a display means, and audio is output as auditory signals through a speaker or the like.

At this time, the terminal device 100 according to the present invention divides the user interface screen displayed while providing the chat service into a plurality of areas, divides the video information, the voice information, and the character information transmitted and received in the chat service, Visualize it in another area and output it. At this time, the visualization of the audio information may be performed by converting the audio information into text, or by extracting the wavelength of the audio information and visualizing it.

For example, in the case of voice chat, the voice of the users participating in the chat service may be outputted as auditory signals, the voice information of each user may be converted into text, or the wavelength of the voice information may be visualized and outputted through the user interface screen .

In addition, in the case of a video chat, the video of the users participating in the chat service may be output, the voice information of the users may be changed to text, or the wavelength of the voice information may be visualized and output to a region different from the video information.

Here, the chat service may simultaneously perform a text chat or a voice chat and simultaneously perform a text chat. At this time, the character information is also output through an area different from the image information or the voice information.

At this time, the image information, the voice information, and the character information output through the different areas are mapped based on the users participating in the chat service and outputted, so that the relevance of the video information, the voice information, and the character information can be intuitively recognized .

The service device 200 provides a chat service by interworking with one or more terminal devices 100 through a communication network 10. At this time, the service device 200 may provide a client program or application for interworking with a chat service to the one or more terminal devices 100, and may interwork with the one or more terminal devices 100 through the same. Specifically, the service apparatus 200 registers two or more terminal apparatuses 100 as a chat group according to a request and a response from the one or more terminal apparatuses 100, and registers two or more terminal apparatuses 100 The chat information including at least one of image information, voice information, and character information of each user.

In an embodiment of the present invention, the service apparatus 200 provides a user interface screen divided into two or more areas to the terminal apparatus 100 according to a request from the terminal apparatus 100, and the terminal apparatus 100 The voice information, and the character information included in the chat data transmitted and received by the mobile terminal 100 are output through different areas.

The terminal device 100 and the service device 200 are interworked through the communication network 10. The communication network 10 can be connected to the Internet using the various wired / wireless communication technologies such as the Internet network, the intranet network, the mobile communication network, Which can send and receive data. The communication network 10 may be a closed network such as a LAN (Local Area Network) or a WAN (Wide Area Network), an open network such as the Internet, a CDMA (Code Division Multiple Access), a WCDMA The concept is called the next generation network and computing network that will be implemented next, such as Multiple Access, Global System for Mobile Communications (GSM), Long Term Evolution (LTE) and Evolved Packet Core (EPC)

Hereinafter, the configuration of a terminal device according to embodiments of the present invention in a chat service system configured as above will be described in detail with reference to the accompanying drawings.

2 is a diagram illustrating a configuration of a terminal device 100 in a chat service system according to embodiments of the present invention.

2, the terminal device 100 includes a control unit 110, an input unit 120, a communication unit 130, a storage unit 140, an output unit 150, and an audio processing unit 160 have. Of course, the terminal device 100 may further include other components such as a sensor, a GPS receiver, etc., in addition to the above-described configuration, but these are not directly related to the present invention, and the description and the description are omitted.

Interworking between the functions and configurations of the respective constituent elements provided in the terminal device 100 according to the present invention will be described concretely.

First, the control unit 110 is configured to perform overall control of the terminal device 100, and may include one or more processors. The processor may be a single-threaded processor, and in other embodiments, the processor may be a multithreaded processor. Further, the control unit 110 can operate by processing instructions stored on the memory or storage device via the one or more processors. The instructions may include interpreted instructions or executable code such as, for example, JavaScript or ECMAScript commands, or other instructions stored in a computer-readable medium.

When the power is turned on, the control unit 110 executes an OS (Operating System) program stored in the storage unit 140 to establish an execution environment in which another application program can be executed, And can perform a predetermined function.

In particular, the controller 110 may execute an application program related to the chat service to execute the chat service according to the present invention. To this end, the control unit 110 may be configured to include a chat processing module 111. Here, the module refers to a component that is implemented by software or hardware or a combination of software and hardware to perform a certain function, and the chat processing module 111 according to the present invention is a module in which a processor executes a chat- To perform the functions described below.

Specifically, the controller 100 activates and activates the chat processing module 111 in response to a chat service request from the user, requests to create two or more chat groups through the activated chat processing module 111, Requests participation in a specific chat group, and processes output of chat data transmitted / received and transmitted / received by users participating in the chat group through a connected chat session.

At this time, the chat data may include video information, audio information, and character information. Here, the image information may be an image of a user, the voice information may be a voice signal uttered by the user, and the character information may be a character input by the user.

The chat processing module 111 forms a user interface screen divided into two or more areas for outputting the chat data. Then, one of video, audio, and text is assigned to each of the two or more areas. The chat processing module 111 separates at least one of video information, voice information, and character information from chat data transmitted and received through the chat service, visualizes separated video information, voice information, and character information, And outputs it to the allocated area.

At this time, the chat processing module 111 may convert the voice information into text and output it, extract the wavelength of the voice information, and visualize and output the extracted wavelength.

At this time, the chat processing module 111 may further divide the area for each user participating in the chat service, and output at least one of voice information, image information, and character information for each user. For example, while chatting, the video information is divided and displayed for each user, and the voice information and the character information of a plurality of users participating in the chat can be displayed in one area in the order of occurrence time, . In another example, video information, audio information, and character information may be separately output for each user. At this time, each area allocated to the video information, voice information, and text information is arranged on the user interface screen so as to be mapped on the basis of users participating in the chat service, so that intuitively and separately outputted image information, To be connected. For example, by arranging areas for outputting audio information and character information in the lower part of the area where video information is output for each user, it is possible to easily recognize which user's voice each of the audio information and the character information is .

The chat processing module 111 may select at least one of two or more users participating in the chat service through the input unit 130 and visualize at least one of video, have.

For example, during the video chatting and the voice chatting, only the voice information of the selected user can be visualized, for example, the voice information can be textized or the wavelength of the voice information can be visualized and output. At this time, the voice information of all the users participating in the chat is output as an auditory signal through the audio processing unit 160. [ Accordingly, the user can visualize only the voice information of the desired user during the voice chat and separately check through the user interface screen.

Next, among the configurations of the terminal device 100 according to the embodiment of the present invention, the communication unit 120 can communicate with the service device 200 through the communication network 10 to transmit and receive data related to the chat service . Specifically, the communication unit 120 transmits or receives a request and a response related to a chat service, such as a request to create a chat group or a request to participate in a specific chat group, to the service apparatus 200. In addition, the communication unit 120 connects a chat session under the control of the service device 200, and transmits chat data including at least one of video information, voice information, and character information of users participating in the chat through a connected chat session Send and receive.

The communication unit 120 can transmit and receive data through various communication methods as well as wire and wireless methods. In addition, the communication unit 120 may transmit and receive data using one or more communication methods. For this purpose, the communication unit 120 may include a plurality of communication modules that transmit and receive data according to different communication methods, respectively.

The input unit 130 is a configuration for operating the terminal device 100. Specifically, the input unit 130 may include a user input for controlling a function or an operation of the terminal device 100, And transmits the generated signal to the control unit 110. The input unit 130 may be implemented by a variety of input means that are currently commercialized or may be commercialized in the future. For example, the input unit 130 may include a general input device such as a keyboard, a mouse, a joystick, a touch screen, Gesture input means for sensing and generating a specific input signal, and voice recognition means for recognizing the user's voice. In particular, the input unit 130 may be used for creating a chat group, selecting a participant, and selecting participation.

The storage unit 140 is a structure for storing data or programs executed or processed by the control unit 110. [ Basically, the storage unit 140 may include an operating system (OS) for operating each of the above-described configurations of the terminal 100, various user functions, for example, A user function for playing a sound source, an MP3 user function for playing a sound source, an image outputting function for reproducing an image such as a photograph, and an application program for supporting a moving image playback function, respectively. In particular, the storage unit 140 may store an application program for a chat service. The storage unit 140 may be an optical storage medium such as a magnetic medium such as a hard disk, a floppy disk and a magnetic tape, a CD-ROM (Compact Disc Read Only Memory), a DVD (Digital Video Disk) A magneto-optical medium such as a floppy disk and a ROM, a random access memory (RAM), and a flash memory.

In addition, the storage unit 140 may store a key map or a menu map for the touch screen operation when the output unit 150 is configured as a touch screen. Here, the key map and the menu map may be various types. That is, the key map may be a keyboard map, a 3 * 4 key map, a QWERTY key map, or the like, and may be a control key map for controlling the operation of the currently activated application program. In addition, the menu map may be a menu map for currently operating application program control.

The output unit 150 provides a user with an operation result or status of the terminal device 100. In particular, the output unit 150 visually outputs information using the display means. For example, the output unit 150 may be an organic light emitting diode (OLED), a vacuum fluorescent display (VFD), a liquid crystal display (LCD), a light emitting diode display, a plasma display panel (OLED), etc., as well as three-dimensional display means such as holography or laser beam.

In particular, in the present invention, the output unit 150 outputs a user interface screen divided into a plurality of areas configured by the chat processing module 111 under the control of the controller 110, Visual information, voice information, and character information are visualized and output to the respective regions of the display device. For example, the user interface screen may further include a user-selectable GUI element in addition to a region in which the image information, audio information, and character information are output.

The audio processing unit 160 is a unit for recognizing audio information and outputting audio information in the terminal device 100. The audio processing unit 160 is connected to a microphone (MIC) And transmits the converted voice information to the controller 110. The voice information transmitted from the controller 110 is output as an audible signal through a speaker SPK. At this time, the audio processor 160 further performs signal processing such as volume adjustment and noise removal of the audio information output from the audio processor 160.

Next, the configuration and operation of the service apparatus 200 according to the present invention in a chat service system will be described in detail with reference to FIG.

3 is a diagram illustrating the configuration of a service apparatus 200 in a chat service system according to an embodiment of the present invention.

3 is a diagram illustrating a configuration of a service apparatus in a chat service system according to embodiments of the present invention.

3, the service apparatus 200 may include a communication unit 210, a service providing unit 220, and a storage unit 230.

The communication unit 210 can transmit and receive data for chat service with one or more terminal devices 100 through the communication network 10. [ The communication unit 210 can transmit and receive data through various communication methods as well as wire and wireless methods. In addition, the communication unit 210 may transmit and receive data using one or more communication methods. For this purpose, the communication unit 210 may include a plurality of communication modules that transmit and receive data according to different communication methods, respectively. Specifically, the communication unit 210 transmits and receives a request for creation and participation of a chat group with one or more terminal devices 100, connects a chat session with the one or more terminal devices 100, And transmits and receives chat data including at least one of video information, character information, and voice information.

The service providing unit 220 according to the embodiments of the present invention may be implemented by including one or more processors in a configuration for controlling a chat service in cooperation with two or more terminal devices 100 through a communication network 10. At this time, the processor mounted on the service apparatus 200 according to the present invention can perform a predetermined function described below by processing a program command for executing the method according to the present invention. In one implementation, the processor may be a single-threaded processor, and in other embodiments, the processor may be a multithreaded processor. Specifically, the service providing unit 220 includes at least one of video information, audio information, and text information between two or more terminal devices 100 set as a chat group in a functional interlock with the communication unit 210 And relays the chat data.

In addition, the service providing unit 220 may further perform the user interface control according to the present invention during the provision of the chat service. Specifically, the service providing unit 220 relays at least one of video information, voice, and characters between two or more terminal apparatuses 100 registered in a chat group in real time, A divided user interface screen is provided and the video, audio, and characters of the chat data transmitted and received between the two or more terminal devices are separated to allocate the separated video, voice, and character to different areas, (100).

Particularly, the service providing unit 220 may control to output the video information of a plurality of users participating in the chat while selectively outputting the voice information and the character information according to a selection of a corresponding user. That is, each terminal device 100 can visualize and output only voice information or character information of a specific user desired by the user.

In this case, even for the terminal device 100 registered in the same chat group, the chat data of the user visualized and outputted according to each request may be different.

The storage unit 230 of the service device 200 is a configuration for storing data and programs necessary for providing a chat service. Specifically, the storage unit 230 stores the information of the terminal devices 100 registered in the currently generated chat group and the chat group, and may store the chat data for each chat group for a predetermined time.

The storage unit 230 may be an optical recording medium such as a magnetic medium such as a hard disk, a floppy disk and a magnetic tape, a CD-ROM (Compact Disk Read Only Memory), a DVD (Digital Video Disk) A magneto-optical medium such as a floppy disk and a ROM, a random access memory (RAM), and a flash memory.

Next, a chat service providing method based on the chat service system configured as described above will be described with reference to FIG. 4 to FIG.

4 to 7 are flowcharts illustrating a method of providing a chat service according to an embodiment of the present invention, and FIG. 8 is a diagram illustrating an exemplary screen of a chat service according to an exemplary embodiment of the present invention.

Each step of the chat service providing method described below can be performed through any one of the terminal device 100 and the service device 200. Hereinafter, description will be made on the basis of the terminal device 100 .

4, the terminal device 100 according to the present invention exchanges at least one of image information, voice information, and character information between two or more users in response to a user request to perform a chat service for supporting real-time conversation And the terminal device 100 according to the present invention checks whether the chat service is started (S105). Specifically, the step S105 may be performed according to a response of other terminal devices through the service device 200 after transmitting a message requesting the creation of a chat group to the service device 200, ), It can be determined that the chat service is started when a response to accept the participation is transmitted to the chat group creation request of the other terminal device transmitted through the terminal.

When the chat service is started, the terminal device 100 divides the user interface screen for outputting chat related information into two or more areas (S110).

Then, the terminal device 100 allocates at least one of video, audio, and characters to the divided two or more areas (S115). In this case, the step S115 may be performed according to a default setting value or may be performed according to a user selection. In addition, by arranging the areas allocated to the same user so that they can be mapped to each other, it is possible to intuitively recognize the association between information that the user visualizes and outputs.

Through the communication unit 120 of the terminal device 100, chat data including at least one of video information, voice information, and character information of users participating in a chat is transmitted and received through a chat session connected by the chat service At step S120, the control unit 110 separates the video information, the character information, and the audio information from the chat data transmitted and received through the chat processing module 111, respectively.

The chat processing module 111 of the terminal device 100 visualizes one of the separated video information, text information, and audio information as a corresponding area of the assigned user interface screen and outputs the visualized information (S125). If voice data is included in the chat data, the terminal device 100 outputs the voice information as an auditory signal through the audio processor 160 in the same manner as the normal processing method, Visualized through the region of interest and output.

At this time, visualization of the audio information may be performed in the same manner as in FIG. 5 or 6.

According to an embodiment of the present invention, in step S125, when the chat data includes voice information, the chat processing module 111 of the terminal device 100 transmits a TTS (Text To Speech technology) to convert the separated voice information into text (S205). Then, the converted voice information can be output to the allocated area of the user interface screen (S210). As a result, the voice contents of the user who has participated in the chat can be confirmed as text.

According to another embodiment of the present invention, in step S125, the chat processing module 111 of the terminal device 100 extracts the wavelength of the voice information as shown in FIG. 6 (S305) After visualization (S310), the visualized wavelength may be output as the area allocated to the audio information of the user interface screen (S315). Thus, it is possible to visually recognize the change in the size or tone of the voice of the user participating in the chat.

According to another embodiment of the present invention, the visual information, the voice information, and the character information of the users participating in the chat are visualized through different areas, and the voice information is selectively visualized only for the selected user Can be output.

7 is a flowchart illustrating a method of providing a chat service according to another embodiment of the present invention.

Referring to FIG. 7, the terminal device 100 according to the present invention exchanges at least one of video information, voice information, and character information among three or more users in response to a user request to check whether a chat service supporting real- (S405). As described above, the step S205 may be performed according to a response of other terminal devices via the service device 200 after transmitting a message requesting the creation of a chat group to the service device 200, The mobile terminal 200 may determine that the chat service has been started when transmitting a response to the chat group creation request of another terminal device transmitted through the mobile terminal 200, In addition, in step S205, it is further confirmed whether there are three or more participants in the chat service.

When a chat service in which three or more users participate is started, the terminal device 100 selects an object to visualize voice information among users participating in the chat service (S410). The terminal 100 may output a list of participants to the output unit 150 and may receive at least one of the users included in the participant list through the input unit 130. [

Thereafter, the terminal device 100 divides a user interface screen for outputting chat-related information into two or more areas, and according to the number of users participating in the chat service and the type of information exchanged in the chat service, Area is divided (S415). For example, in the case of video chatting, the terminal device 100 can display video information for a number of users corresponding to the number of users participating in chatting and audio information output for the number of users selected as the audio information visualization target The area can be divided.

On the other hand, in the case of voice chat, the terminal device 100 can divide the audio information output area for the number of users selected as the audio information visualization object.

Then, the terminal device 100 allocates the types of users and information for each divided area (S420). At this time, the regions allocated to the same user can be mapped to each other, so that it is possible to intuitively recognize the association between information visualized and output by the user.

When chat data including at least one of video information, voice information, and character information of users participating in a chat starts to be transmitted and received through a connected chat session in the corresponding chat service, the terminal device 1000 performs chat processing At least one of video information, text information, and audio information is separated from the chat data transmitted / received through the module 111 (S425). At this time, the image information, character information, and audio information may be classified into corresponding users.

The chat processing module 111 of the terminal device 100 visualizes one of the separated video information, text information, and audio information as a corresponding area of the assigned user interface screen and outputs the visualized information (S125). Here, visualization of the audio information can be performed as shown in Figs. 5 and 6 described above.

The voice information included in the chat data is also output as an auditory signal through the audio processing unit 160 of the terminal device 100. [

Accordingly, the user of the terminal device 100 can visualize only the voice information of a specific user during chatting as in the conventional method.

Through this, the user can more accurately check the chat contents of a specific user during chatting. For example, when the voice of a specific user can not be heard due to noise during chatting, the voice information of the user is visualized and outputted, so that the chat contents can be accurately recognized.

8 is a diagram illustrating a chat service screen according to the present invention. FIG. 8 is a diagram illustrating a case where two users chat using video, voice, and text.

8, the user interface screen 80 output to the output unit 150 of the terminal device 100 according to the present invention includes a first area 81 allocated to the image information of the other party, A second area 82 assigned to the information, and a third area 83 assigned to the character information of the other party. In the first area 81, the image of the other party participating in the chat service is displayed. In the second area 82, the voice of the other party participating in the chat service is visualized and displayed. In the third area 83, Characters transmitted by the party participating in the service may be displayed. At this time, text corresponding to the voice of the other party may be outputted, or the wavelength of the voice of the other party may be visualized and displayed in the second area 82. Accordingly, the user can accurately recognize the voice of the other party or recognize the voice tone or state of the other party by referring to the second area of the user interface screen 80 during chatting.

 The chat service providing method according to the present invention may be implemented in a form of software readable by various computer means and recorded in a computer-readable recording medium. Here, the recording medium may include program commands, data files, data structures, and the like, alone or in combination. Program instructions to be recorded on a recording medium may be those specially designed and constructed for the present invention or may be available to those skilled in the art of computer software. For example, the recording medium may be an optical recording medium such as a magnetic medium such as a hard disk, a floppy disk and a magnetic tape, a compact disk read only memory (CD-ROM), a digital video disk (DVD) Includes a hardware device that is specially configured to store and execute program instructions such as a magneto-optical medium such as a floppy disk and a ROM, a random access memory (RAM), a flash memory, do. Examples of program instructions may include machine language code such as those generated by a compiler, as well as high-level language code that may be executed by a computer using an interpreter or the like. Such a hardware device may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

Although the present specification and drawings describe exemplary device configurations, the functional operations and subject matter implementations described herein may be embodied in other types of digital electronic circuitry, or alternatively, of the structures disclosed herein and their structural equivalents May be embodied in computer software, firmware, or hardware, including, or in combination with, one or more of the foregoing. Implementations of the subject matter described herein may be embodied in one or more computer program products, i. E. One for computer program instructions encoded on a program storage medium of the type for < RTI ID = 0.0 & And can be implemented as a module as described above. The computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of matter that affects the machine readable propagation type signal, or a combination of one or more of the foregoing.

The terms "system" and "device" as used herein encompass all apparatuses, devices, and machines for processing data, including, for example, a programmable processor, a computer, or a multiprocessor or computer. In particular, the system may include, in addition to the hardware, code that forms an execution environment for a computer program upon request, such as code comprising a processor firmware, a protocol stack, a database management system, an operating system, have.

A computer program (also known as a program, software, software application, script, or code) that is loaded on an apparatus according to the present invention and which implements the method according to the present invention may be a compiled or interpreted language or a program containing a priori or procedural language Language, and may be deployed in any form including standalone programs or modules, components, subroutines, or other units suitable for use in a computer environment. A computer program does not necessarily correspond to a file in the file system. The program may be stored in a single file provided to the requested program, or in multiple interactive files (e.g., a file storing one or more modules, subprograms, or portions of code) (E.g., one or more scripts stored in a markup language document). A computer program may be deployed to run on multiple computers or on one computer, located on a single site or distributed across multiple sites and interconnected by a communications network.

Implementations of the subject matter described herein may include, for example, a back-end component such as a data server, or may include a middleware component, such as an application server, or may be a web browser or a graphical user, for example a user, who may interact with an implementation of the subject- Front-end components such as client computers with interfaces, or any combination of one or more of such back-end, middleware, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication, such as, for example, a communications network.

While the specification contains a number of specific implementation details, it should be understood that they are not to be construed as limitations on the scope of any invention or claim, but rather on the description of features that may be specific to a particular embodiment of a particular invention Should be understood. Certain features described herein in the context of separate embodiments may be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented in multiple embodiments, either individually or in any suitable subcombination. Further, although the features may operate in a particular combination and may be initially described as so claimed, one or more features from the claimed combination may in some cases be excluded from the combination, Or a variant of a subcombination.

Likewise, although the operations are depicted in the drawings in a particular order, it should be understood that such operations must be performed in that particular order or sequential order shown to achieve the desired result, or that all illustrated operations should be performed. In certain cases, multitasking and parallel processing may be advantageous. Also, the separation of the various system components of the above-described embodiments should not be understood as requiring such separation in all embodiments, and the described program components and systems will generally be integrated together into a single software product or packaged into multiple software products It should be understood.

Certain embodiments of the subject matter described herein have been described. Other embodiments are within the scope of the following claims. For example, the operations recited in the claims may be performed in a different order and still achieve desirable results. By way of example, the process illustrated in the accompanying drawings does not necessarily require that particular illustrated or sequential order to obtain the desired results. In certain implementations, multitasking and parallel processing may be advantageous.

The description sets forth the best mode of the invention, and is provided to illustrate the invention and to enable those skilled in the art to make and use the invention. The written description is not intended to limit the invention to the specific terminology presented. Thus, while the present invention has been described in detail with reference to the above examples, those skilled in the art will be able to make adaptations, modifications, and variations on these examples without departing from the scope of the present invention.

Therefore, the scope of the present invention should not be limited by the described embodiments but should be defined by the claims.

The present invention can be applied to a chat service using one or more of video, audio, and text. The user interface screen is divided into a plurality of areas, and video information, voice information, and text By separating the information through different areas and visualizing and outputting them, it is possible to more clearly communicate the opinions of the participating users.

Particularly, in the present invention, during the video chatting or the voice chatting, the voice information of the users is audibly output, the voice information is converted into text and outputted through the user interface screen, Make the contents more clearly understandable or to be confirmed later.

In addition, while performing a video chat or a voice chat, the present invention can extract wavelengths of voice information of users, visualize and output the wavelength of the voice information, and provide various new types of chat services .

10: communication network 100: terminal device 110:
111: chat processing module 120: communication unit 130: input unit
140: storage unit 150: output unit 160: audio processing unit
200: service apparatus 210: communication unit 220:
230:

Claims (18)

A communication unit for transmitting and receiving data through a communication network;
An input for inputting a user request;
An output unit for outputting a user interface screen; And
Wherein the user interface screen is divided into a plurality of areas, a divided area is allocated to each type of information when the chat service using at least one of video, audio, and text is provided according to the user request, A controller for separating at least one of video information, audio information and character information from the chat data and visualizing the separated video information, audio information, and character information through an assigned area, respectively, and outputting the visual information;
And a terminal device. The apparatus of claim 1, wherein the control unit
Converts the voice information into text and outputs the converted voice. 3. The apparatus of claim 2, wherein the control unit
And displays the text converted from the voice information of users participating in the chat service in the same area in chronological order. The apparatus of claim 1, wherein the control unit
Extracts a wavelength of the audio information, visualizes the extracted wavelength, and outputs the visualized information. The apparatus of claim 1, wherein the control unit
Further dividing the area for each user participating in the chat service, and separating and outputting at least one of the separated image information, voice information, and character information for each user. 6. The apparatus of claim 5, wherein the control unit
Wherein the plurality of areas divided by the type of the user and the information are arranged on the user interface screen so as to be mapped on the basis of the user. The apparatus of claim 1, wherein the control unit
Wherein the user selects an object to visualize voice information among users participating in the chat service through the input unit, and visualizes only the voice information of the selected user and outputs the visual information. A communication unit that communicates with two or more terminal devices through a communication network to transmit and receive data for a chat service;
A chat server for relaying chat data including at least one of video information, voice information, and character information between two or more terminal devices connected through the communication unit in real time, wherein the chat information is divided into two or more areas, An interface screen, separating at least one of video information, audio information, and character information from chat data transmitted and received between the two or more terminal devices, and separating at least one of the separated video information, Visualizing and outputting through the allocated area;
And a service provider for providing the chat service. The method of claim 8, wherein the service providing unit
Wherein the voice information is converted into text and provided. The method of claim 8, wherein the service providing unit
Extracts a wavelength of the voice information, and visualizes the extracted wavelength to provide the extracted voice information. 9. The apparatus of claim 8, wherein the control unit
Wherein the terminal device selects a visualization object of voice information among users participating in the chat service from each terminal device, and visualizes only the voice information of the selected user. A user interface screen is divided into two or more areas and allocated to at least one of video, voice, and characters, and a chat service using at least one of video, audio, A plurality of terminal devices for separating at least one of video information, audio information, and character information and visualizing at least one of the separated video information, audio information, and character information through an allocated area and outputting the visual information; And
And a service device for relaying the chat data among the plurality of terminal devices. A device for providing a chat service,
When the chat service is started, dividing the user interface screen into two or more areas;
Assigning one of video, audio, and text to each of the two or more areas;
Separating at least one of video information, audio information, and character information from chat data transmitted and received through the chat service;
Visualizing at least one of the separated image information, the voice information, and the character information through an allocated area, and outputting the visualized voice information. 14. The method of claim 13, wherein the outputting comprises:
Converting the voice information separated from the chat data into text and outputting the converted text to an area allocated to voice; And
Extracting a wavelength of the voice information separated from the chat data, visualizing the extracted wavelength, and outputting the visualized voice to an area allocated to the voice. 14. The method of claim 13,
And determines the number of areas to be divided according to the type of information included in the chat data. 14. The method of claim 13,
The dividing step
Further dividing the region according to the number of users participating in the chat service,
The allocating step
And allocating an area for each type of user and information. A device for providing a chat service,
Selecting a visualization target of the voice information among the users participating in the chat service when the chat service is started;
Dividing the UI screen into two or more areas according to the number of visualization targets of the selected audio information and the type of information included in the chat data;
Allocating the divided two or more areas for each type of user and information;
Separating at least one of video information, audio information, and character information from chat data transmitted and received through the chat service;
Visualizing at least one of the separated image information, the voice information, and the character information through an allocated area, and outputting the visualized voice information. A computer-readable recording medium recording a program embodied to perform the chat service providing method recited in any one of claims 13 to 17.

Images