KR20120090455A - Method for providing chatting service in multimedia device and multimedia device thereof - Google Patents

Abstract

PURPOSE: A chatting service providing method of a multimedia apparatus is provided to improve user convenience by enabling a user to input a chatting message with various images. CONSTITUTION: A multimedia apparatus displays a chatting message and a user image of a user who attends to a chatting service. The multimedia apparatus captures the user image(S101). The multimedia apparatus captures the photographed image based on the pre-et capturing conditions(S103,S104). The multimedia apparatus updates the displayed user image as the captured image(S105).

Description

METHOD FOR PROVIDING CHATTING SERVICE IN MULTIMEDIA DEVICE AND MULTIMEDIA DEVICE THEREOF}

The present invention relates to a method for operating a multimedia device and a multimedia device, and more particularly, to a method of operating a multimedia device that can facilitate smooth communication between users when the user uses a chat service through the multimedia device. Relates to a multimedia device.

The multimedia apparatus may include, for example, an image display device having a function of receiving and processing a broadcast image that can be viewed by a user. For example, the multimedia apparatus displays a broadcast selected by a user on a display among broadcast signals transmitted from a broadcasting station. Currently, broadcasting is shifting from analog broadcasting to digital broadcasting worldwide.

Digital broadcasting refers to broadcasting for transmitting digital video and audio signals. Digital broadcasting is more resistant to external noise than analog broadcasting, so it has less data loss, is advantageous for error correction, has a higher resolution, and provides a clearer picture. In addition, unlike analog broadcasting, digital broadcasting is capable of bidirectional services.

On the other hand, smart TVs, which combine the functions of video display devices, have recently been in the spotlight.

The smart TV is an intelligent terminal that adds computer support functions such as internet communication and information retrieval to a conventional television, and has a large memory and a high-performance central processing unit (CPU) compared to the existing TV, and executes various applications and performs voice / data. An operating system (OS) for supporting communication and PC (Personal Computer) interworking is mounted. When using such a smart TV, unlike the existing multimedia device, the use of services through the Internet is free. In particular, the user can download and install the desired application from the outside. In addition, a chat message that can communicate with other users connected to the network through the application can be entered and sent to other users.

However, when using the chat service through the multimedia device of the prior art, when inputting a chat message through text or voice only, there is a problem that it is difficult to check the facial expression change and gesture of the other party, and when the chat message is input through a video. There has been a problem that a large amount of data transmission is required, causing a delay in the chat process.

Therefore, by solving the problems of the multimedia device of the prior art, when using a chat service in the multimedia device, by inputting any video with a text or voice together to increase the accuracy in the communication between users, clear communication to other users There is an urgent need to develop a chat service providing method and a multimedia device accordingly.

An object of the present invention is to provide a multimedia device and a method of operating the same, when a user is chatting with another user connected to a network through a multimedia device, the user's ease of use in a communication process via chat. have.

In addition, another object of the present invention is to provide a multimedia device and a method of operating the same that can provide various user interfaces in the delivery of chat messages.

According to another aspect of the present invention, there is provided a chat service providing method of a multimedia device, the method including: displaying a chat message and a user image of a user participating in the chat service; Initiating photographing of a user image; Capturing the captured image based on a preset capture condition; And

Updating the displayed user image with the captured image.

In addition, a multimedia apparatus capable of providing a chat service according to an embodiment of the present invention for achieving the above object includes a network interface unit for transmitting and receiving data through a network; A camera unit for photographing a user image; A storage unit which stores a capture condition for the user image; A display unit configured to display a chat message and a user image of a user participating in the chat service; And loading the capture condition in the storage unit, capturing a user image photographed by the camera unit based on the loaded capture condition, and controlling to update the displayed user image with the captured image. It includes a control unit.

According to an embodiment of the present invention, when chatting with another user connected to a network through a multimedia device, a chat message including various images may be input, thereby delivering an accurate message to another user.

In addition, according to an embodiment of the present invention, in the network chat process through the multimedia device, it is possible to prevent a delay due to a decrease in transmission speed.

On the other hand, according to another embodiment of the present invention, it is possible to provide a variety of user interface in the multimedia device, it can be increased user convenience.

1 is a diagram schematically showing an example of an entire broadcasting system including a multimedia apparatus according to an embodiment of the present invention.
2 is a diagram schematically showing another example of an entire broadcasting system including a multimedia apparatus according to another embodiment of the present invention.
3 is a diagram illustrating another example of the multimedia apparatus illustrated in FIG. 1 or 2 in more detail.
4 is an internal block diagram of the controller illustrated in FIG. 6.
5 is a diagram illustrating a user and a multimedia apparatus according to an exemplary embodiment.
6 is a diagram illustrating a display screen including a menu for acquiring a user image according to an exemplary embodiment.
7 is an internal block diagram of a remote control apparatus for controlling any one of the multimedia apparatus according to an embodiment of the present invention.
8 is a diagram illustrating a first embodiment of a user interface provided by any one of multimedia devices according to an embodiment of the present invention.
9 is a diagram illustrating a second embodiment of a user interface provided by any one of multimedia devices according to an embodiment of the present invention.
10 is a diagram illustrating a display screen for providing a chat service in a multimedia apparatus according to the prior art.
11 is a flowchart illustrating a chat message input method including a user image according to an exemplary embodiment of the present invention.
12 is a flowchart illustrating a chat message input method including a keyword search result according to an exemplary embodiment of the present invention.
13 is a diagram illustrating a chat message input method according to an embodiment of the present invention.
14 is a diagram illustrating a display screen including a chat mode setting menu according to an exemplary embodiment of the present invention.
15 is a diagram illustrating a display screen including a chat window in which a user image is displayed according to an exemplary embodiment of the present invention.
FIG. 16 illustrates a display screen including a chat window in which a keyword search result is displayed according to an exemplary embodiment of the present invention.
17 is a diagram illustrating a database for storing capture conditions of a multimedia device according to an embodiment of the present invention.

Hereinafter, the present invention will be described in more detail with reference to the drawings.

The suffix "module" and " part "for components used in the following description are given merely for ease of description, and the" module "and" part "

Meanwhile, the multimedia apparatus described in the present specification is, for example, an intelligent multimedia apparatus in which a computer support function is added to a broadcast reception function. The multimedia device is faithful to the broadcast reception function and has an Internet function added thereto. It can have a more convenient interface than a remote control. In addition, it is connected to the Internet and a computer with the support of a wired or wireless Internet function, and can perform functions such as e-mail, web browsing, banking or gaming. A standardized general-purpose OS can be used for these various functions.

Therefore, in the multimedia apparatus described in the present invention, various applications can be freely added or deleted, for example, on a general-purpose OS kernel, so that various user-friendly functions can be performed. More specifically, the multimedia apparatus may be, for example, a network TV, an HBBTV, a smart TV, or the like, and may be applicable to a smartphone in some cases.

Furthermore, although an embodiment of the present invention will be described in detail with reference to the accompanying drawings and the contents described in the accompanying drawings, the present invention is not limited or restricted by the embodiments.

As used herein, terms used in the present invention are selected from general terms that are widely used in the present invention while taking into account the functions of the present invention, but these may vary depending on the intention or custom of a person skilled in the art or the emergence of new technologies. In addition, in certain cases, there may be a term arbitrarily selected by the applicant, in which case the meaning thereof will be described in the description of the corresponding invention. Therefore, it is intended that the terminology used herein should be interpreted based on the meaning of the term rather than on the name of the term, and on the entire contents of the specification.

 1 is a diagram schematically showing an example of an entire broadcasting system including a multimedia apparatus according to an embodiment of the present invention.

As shown in FIG. 1, an entire broadcasting system including a multimedia apparatus according to an embodiment of the present invention includes a content provider (CP) 10, a service provider (SP) 20, and a network. It may be divided into a network provider (NP) 30 and an HNED 40. The HNED 40 corresponds to, for example, a client 100 that is a multimedia device according to an embodiment of the present invention. The client 100 corresponds to a multimedia device according to an embodiment of the present invention, and the multimedia device is, for example, a network TV, a smart TV, an IPTV, or the like.

The content provider 10 produces and provides various contents. As shown in FIG. 1, the content provider 10 includes a terrestrial broadcaster, a cable system operator or a multiple system operator, a satellite broadcaster, and an internet broadcaster. An internet broadcaster and the like can be exemplified.

In addition to the broadcast content, the content provider 10 may provide various applications. In this regard, it will be described later in more detail.

The service provider 20 may provide a service package of contents provided by the content provider 10. For example, the service provider 20 of FIG. 1 may package and provide a first terrestrial broadcast, a second terrestrial broadcast, a cable MSO, satellite broadcast, various internet broadcasts, applications, and the like to a user.

Meanwhile, the service provider 20 may provide a service to the client 100 using a unicast or multicast scheme. The unicast method is a method of transmitting data 1: 1 between one sender and one receiver. For example, in the unicast method, when a receiver requests data from a server, the server may transmit data to the receiver according to the request. Multicasting is a method of transmitting data to a plurality of recipients of a specific group. For example, the server can send data to multiple pre-registered receivers at once. For such multicast registration, the Internet Group Management Protocol (IGMP) protocol may be used.

The network provider 30 may provide a network for providing a service to the client 100. The client 100 may establish a home network end user (HNED) to receive a service.

As a means for protecting content transmitted from the multimedia device system as described above, conditional access or content protection may be used. As one example for such a restriction reception or content protection, a scheme such as a cable card or a downloadable conditional access system (DCAS) may be used.

On the other hand, the client 100 can also provide content through the network. In this case, unlike the above, the client 100 may be a content provider, and the content provider 10 may receive content from the client 100. In the case of designing as described above, an interactive content service or a data service is possible.

2 is a diagram schematically showing another example of an entire broadcasting system including a multimedia apparatus according to another embodiment of the present invention.

As shown in FIG. 2, the multimedia apparatus 100 according to another embodiment of the present invention is connected to a broadcast network and an internet network. The multimedia apparatus 100 is, for example, a network TV, a smart TV, an HBBTV, or the like.

In addition, the multimedia apparatus 100 may include, for example, a broadcast interface 101, a section filter 102, an AIT filter 103, an application data processor 104, a broadcast data processor 111, and a media player 106. , An internet protocol processor 107, an internet interface 108, and a runtime module 109.

Through the broadcast interface 101, application information table (AIT) data, real-time broadcast content, application data, and stream events are received. Meanwhile, the real-time broadcast content may be referred to as linear ave content (Linear A / V Content).

The section filter 102 performs section filtering on four pieces of data received through the broadcast interface 101 to transmit AIT data to the AIT filter 103, and the linear AV content is broadcast data processing unit 111. The stream event and the application data are transmitted to the application data processing unit 104.

Meanwhile, non-linear AV content and application data are received through the Internet interface 108. The non-linear AV content may be, for example, a Content On Demand (COD) application.

The non-linear AV content is sent to media player 106 and the application data is sent to runtime module 109.

Further, the runtime module 109 includes, for example, an application manager and a browser as shown in FIG. 2. The application manager, for example, uses AIT data to control the life cycle for the interactive application. The browser, for example, performs a function of displaying and processing the interactive application.

3 is a diagram illustrating another example of the multimedia apparatus illustrated in FIG. 1 or 2 in more detail.

Referring to FIG. 3, the multimedia apparatus 100 according to an embodiment of the present invention may include a broadcast receiver 105, an external device interface 135, a storage 140, a user input interface 150, and a controller. The display unit 180 may include a display unit 180, an audio output unit 185, and a camera unit 190. The broadcast receiver 105 may include a tuner 110, a demodulator 120, and a network interface unit 130. Of course, if necessary, the tuner 110 and the demodulator 120 may be provided so as not to include the network interface unit 130. On the contrary, the tuner 110 and the network interface unit 130 may be provided. The demodulator 120 may be designed so as not to be included.

The tuner 110 selects an RF broadcast signal corresponding to a channel selected by a user or all pre-stored channels among RF (Radio Frequency) broadcast signals received through an antenna. Also, the selected RF broadcast signal is converted into an intermediate frequency signal, a baseband image, or a voice signal.

In addition, the tuner 110 may receive an RF broadcast signal of a single carrier according to an Advanced Television System Committee (ATSC) scheme or an RF broadcast signal of a plurality of carriers according to a digital video broadcasting (DVB) scheme.

The demodulator 120 receives the digital IF signal DIF converted by the tuner 110 and performs a demodulation operation.

For example, when the digital IF signal output from the tuner 110 is an ATSC scheme, the demodulator 120 performs 8-VSB (8-Vestigal Side Band) demodulation, for example. Also, the demodulation unit 120 may perform channel decoding. To this end, the demodulator 120 includes a trellis decoder, a de-interleaver, and a reed solomon decoder to perform trellis decoding, deinterleaving, Solomon decoding can be performed.

For example, when the digital IF signal output from the tuner 110 is a DVB scheme, the demodulator 120 performs coded orthogonal frequency division modulation (COFDMA) demodulation, for example. Also, the demodulation unit 120 may perform channel decoding. For this, the demodulator 120 may include a convolution decoder, a deinterleaver, and a reed-solomon decoder to perform convolutional decoding, deinterleaving, and reed solomon decoding.

The demodulator 120 may output a stream signal TS after performing demodulation and channel decoding. In this case, the stream signal may be a signal multiplexed with a video signal, an audio signal, or a data signal. For example, the stream signal may be an MPEG-2 Transport Stream (TS) multiplexed with an MPEG-2 standard video signal, a Dolby AC-3 standard audio signal, and the like. Specifically, the MPEG-2 TS may include a header of 4 bytes and a payload of 184 bytes.

The stream signal output from the demodulator 120 may be input to the controller 170. After performing demultiplexing, image / audio signal processing, and the like, the controller 170 outputs an image to the display unit 180 and outputs an audio to the audio output unit 185.

The external device interface unit 135 may connect the external device and the multimedia device 100. To this end, the external device interface unit 135 may include an A / V input / output unit (not shown) or a wireless communication unit (not shown).

The external device interface unit 135 may be connected to an external device such as a digital versatile disk (DVD), a Blu-ray, a game device, a camera, a camcorder, a computer (laptop), or the like by wire / wireless. The external device interface unit 135 transmits an externally input image, audio or data signal to the controller 170 of the multimedia apparatus 100 through a connected external device. In addition, the controller 170 may output an image, audio, or data signal processed by the controller 170 to a connected external device. To this end, the external device interface unit 135 may include an A / V input / output unit (not shown) or a wireless communication unit (not shown).

The A / V input / output unit includes a USB terminal, a Composite Video Banking Sync (CVBS) terminal, a component terminal, an S-video terminal (analog), and a DVI (USB) terminal to input video and audio signals from an external device to the multimedia apparatus 100. Digital Visual Interface (HDMI) terminal, High Definition Multimedia Interface (HDMI) terminal, RGB terminal, D-SUB terminal and the like.

The wireless communication unit can perform short-range wireless communication with other electronic devices. The multimedia apparatus 100 may include, for example, Bluetooth, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, and Digital Living Network Alliance (DLNA). It can be connected to other electronic devices and networks according to the communication standards.

In addition, the external device interface unit 135 may be connected through at least one of the various set top boxes and the various terminals described above, and perform input / output operations with the set top box.

Meanwhile, the external device interface unit 135 may receive an application or a list of applications in a neighboring external device and transmit the received application or list to the controller 170 or the storage 140.

The network interface unit 130 provides an interface for connecting the multimedia apparatus 100 to a wired / wireless network including an internet network. The network interface unit 130 may include, for example, an Ethernet terminal for connection with a wired network, and for example, for connection with a wireless network, for example, a wireless LAN (WLAN) (Wi-). Fi, Wibro (Wireless broadband), Wimax (World Interoperability for Microwave Access), HSDPA (High Speed Downlink Packet Access) communication standards, and the like can be used.

The network interface unit 130 may transmit or receive data with another user or another electronic device through a connected network or another network linked to the connected network. In particular, some content data stored in the multimedia apparatus 100 may be transmitted to another user or a selected electronic apparatus selected from other electronic apparatuses or other electronic apparatuses registered in the multimedia apparatus 100.

Meanwhile, the network interface unit 130 may access a predetermined web page through a connected network or another network linked to the connected network. That is, by accessing a predetermined web page through the network, it is possible to send or receive data with the server. In addition, content or data provided by a content provider or a network operator may be received. That is, it can receive contents such as a movie, an advertisement, a game, a VOD, a broadcast signal, and related information provided from a content provider or a network provider through a network. In addition, the update information and the update file of the firmware provided by the network operator can be received. It may also transmit data to the Internet or content provider or network operator.

In addition, when the user uses a chat service through a network, the network interface unit 130 may transmit / receive data such as a chat message or a user image for using the chat service through a network with a predetermined server.

The storage 140 may store a program for processing and controlling each signal in the controller 170, or may store a signal processed video, audio, or data signal.

In addition, the storage 140 may perform a function for temporarily storing an image, audio, or data signal input from the external device interface 135 or the network interface 130. In addition, the storage 140 may store information on a predetermined broadcast channel through a channel storage function.

In addition, when the user uses a chat service through the multimedia apparatus, the storage 140 may store a capture condition for capturing an image captured by the camera 190 as a database 141. The database will be described in detail later with reference to FIG. 17.

The storage unit 140 may be, for example, a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (for example, SD or XD). Memory, etc.), RAM, ROM (e.g., EEPROM, etc.) at least one type of storage medium. The multimedia apparatus 100 may reproduce and provide a content file (video file, still image file, music file, document file, application file, etc.) stored in the storage 140 to a user.

3 illustrates an embodiment in which the storage unit 140 is provided separately from the control unit 170, but the scope of the present invention is not limited thereto. The storage unit 140 may be included in the control unit 170.

The user input interface unit 150 transmits a signal input by the user to the control unit 170 or a signal from the control unit 170 to the user.

For example, the user input interface unit 150 may be powered on / off, channel selection, and screen from the remote controller 200 according to various communication methods such as a radio frequency (RF) communication method and an infrared (IR) communication method. A control signal such as a setting may be received and processed, or a control signal from the controller 170 may be transmitted to the remote controller 200.

In addition, for example, the user input interface unit 150 may transmit a control signal input from a local key (not shown) such as a power key, a channel key, a volume key, and a set value to the controller 170.

Also, for example, the user input interface unit 150 may transmit a control signal input from a sensing unit (not shown) that senses a user's gesture to the controller 170 or sense a signal from the controller 170. It can transmit to a part (not shown). Here, the sensing unit (not shown) may include a touch sensor, an audio sensor, a position sensor, an operation sensor, and the like.

The control unit 170 demultiplexes the input stream or processes the demultiplexed signals through the tuner 110, the demodulator 120, or the external device interface unit 135, and outputs a video or audio signal. You can create and output.

The image signal processed by the controller 170 may be input to the display unit 180 and displayed as an image corresponding to the image signal. The video signal processed by the controller 170 may be input to the external output device through the external device interface 135. [

The audio signal processed by the control unit 170 may be output to the audio output unit 185 through audio. In addition, the voice signal processed by the controller 170 may be input to the external output device through the external device interface unit 135.

Although not shown in FIG. 3, the controller 170 may include a demultiplexer, an image processor, and the like. This will be described later with reference to FIG. 4.

In addition, the controller 170 may control overall operations of the multimedia apparatus 100. For example, the controller 170 may control the tuner 110 to control the tuner 110 to select an RF broadcast corresponding to a channel selected by a user or a pre-stored channel.

In addition, the controller 170 may control the multimedia apparatus 100 by a user command or an internal program input through the user input interface unit 150. In particular, it is possible to connect to the network so that the user can download the desired application or application list into the multimedia apparatus 100.

For example, the controller 170 controls the tuner 110 to input a signal of a selected channel according to a predetermined channel selection command received through the user input interface unit 150. And processes video, audio or data signals of the selected channel. The controller 170 allows the channel information selected by the user to be output through the display unit 180 or the audio output unit 185 together with the processed image or audio signal.

As another example, the controller 170 may receive an external device image playback command received through the user input interface unit 150 from an external device, for example, a camera or a camcorder, input through the external device interface unit 135. The video signal or the audio signal may be output through the display unit 180 or the audio output unit 185.

The controller 170 may control the display 180 to display an image. For example, a broadcast image input through the tuner 110, an external input image input through the external device interface unit 135, an image input through a network interface unit, or an image stored in the storage unit 140 may be stored. The display unit 180 may control the display to be displayed on the display unit 180. In this case, the image displayed on the display unit 180 may be a still image or a video, and may be a 2D image or a 3D image.

In addition, the controller 170 may control to reproduce the content. In this case, the content may be content stored in the multimedia apparatus 100, received broadcast content, or external input content input from the outside. The content may be at least one of a broadcast image, an external input image, an audio file, a still image, a connected web screen, and a document file.

The controller 170 may control to install and run an application downloaded from an external network along with various user interfaces. In addition, by selecting a user, an image related to an application to be executed may be controlled to be displayed on the display unit 180.

In particular, in relation to an embodiment of the present invention, when a user executes an application for using a chat service, a chat message and a user image of a user participating in a chat service for using the chat service are displayed on the display unit 180. ) Can be displayed. This will be described in detail later with reference to FIG. 15.

Meanwhile, in accordance with an embodiment of the present disclosure, the controller 170 may load a capture condition from the database 141 stored in the storage 140, and then may configure the camera unit 190 based on the loaded capture condition. ) To capture a user image captured by the user, and to update the user image included in the chat window displayed on the display unit 180 with the captured image.

In addition, the controller 170 obtains coordinate information of each part of the user's body in the user image photographed by the camera unit 190, and tracks the movement of the obtained coordinates to track the coordinates of the tracked coordinates. When the movement corresponds to a preset capture condition, the controller may control to capture the captured image.

In addition, the controller 170, in the user image photographed by the camera unit 190, obtains coordinate information of each part of the eye area in the user's face area, and tracks the movement of the obtained coordinates, When the movement of the tracked coordinates corresponds to a preset capture condition, the controller may control to capture the captured image.

Accordingly, for the above tracking operations, the camera unit 190 of the multimedia apparatus may include an image sensor capable of acquiring two different information. The image sensor and an image acquired through the image sensor will be described below with reference to FIG. 6.

The controller 170 may capture an image captured by the camera unit 190 when a state corresponding to the capture condition is not detected for a predetermined time or more. In other words, even if a capture condition does not occur, the user image can be updated at a predetermined time interval. The information about the predetermined time may be stored in the storage 140 by the user.

In addition, the controller 170 extracts a keyword from a chat message displayed on the display 190, and transmits a search request using the extracted keyword as a search word to the search server through the network interface unit 130. The search result may be received from the search server, and the search result may be displayed on the display unit 180.

In addition, the controller 170 extracts a keyword from a chat message displayed on the display 190, searches for the content or service corresponding to the extracted keyword, and stores the search result in the storage unit. It may be controlled to display on 180.

In addition, the controller 170 may control to crop the captured image from the image captured by the camera unit 190. That is, since not all parts of the captured image are necessary as the user image, an image cropped in a specific region may be used as the user image. The information about the specific area may be stored in the database 141 together with the capture condition.

The display unit 180 converts an image signal, a data signal, an OSD signal processed by the controller 170 or an image signal, data signal, etc. received from the external device interface unit 135 into R, G, and B signals, respectively. Generate a drive signal.

The display unit 180 may be a PDP, an LCD, an OLED, a flexible display, a 3D display, or the like.

The display unit 180 may be configured as a touch screen and used as an input device in addition to the output device.

The audio output unit 185 receives a signal processed by the controller 170, for example, a stereo signal, a 3.1 channel signal, or a 5.1 channel signal, and outputs a voice signal. The voice output unit 185 may be implemented by various types of speakers.

Meanwhile, in order to detect a gesture of a user, as described above, a sensing unit (not shown) including at least one of a touch sensor, a voice sensor, a position sensor, and a motion sensor may be further provided in the multimedia apparatus 100. . The signal detected by the sensing unit (not shown) may be transmitted to the controller 170 through the user input interface unit 150.

On the other hand, the camera unit 190 for photographing the user may be further provided. Image information photographed by the camera unit 190 may be input to the controller 170.

In addition, the camera unit 190 includes a depth image sensor for acquiring distance information of the user as an image and an RGB image sensor for acquiring color information of the user as an image, and acquires different information from each other through the two image sensors. To obtain an accurate user image.

The controller 170 may detect a gesture of a user by combining or combining an image captured by a photographing unit (not shown) or a detected signal from a sensing unit (not shown).

The remote control apparatus 200 transmits the user input to the user input interface unit 150. To this end, the remote control apparatus 200 can use Bluetooth, RF (radio frequency) communication, infrared (IR) communication, UWB (Ultra Wideband), ZigBee, or the like.

In addition, the remote control apparatus 200 may receive an image, an audio or a data signal output from the user input interface unit 150, display it on the remote control apparatus 200 or output an audio or vibration.

The above-described multimedia apparatus 100 is a fixed type of ATSC (8-VSB) digital broadcasting, DVB-T (COFDM) digital broadcasting, ISDB-T (BST-OFDM) digital broadcasting, and the like. It may be a digital broadcast receiver capable of receiving at least one.

Meanwhile, a block diagram of the multimedia apparatus 100 illustrated in FIG. 3 is a block diagram for an embodiment of the present invention. Each component of the block diagram may be integrated, added, or omitted according to the specification of the multimedia apparatus 100 that is actually implemented. That is, two or more constituent elements may be combined into one constituent element, or one constituent element may be constituted by two or more constituent elements, if necessary. In addition, the functions performed in each block are intended to illustrate the embodiments of the present invention, and the specific operations and apparatuses do not limit the scope of the present invention.

Meanwhile, unlike the example illustrated in FIG. 6, the multimedia apparatus 100 does not include the tuner 110 and the demodulator 120 illustrated in FIG. 3, but the network interface 130 or the external device interface 135. Through, the video content may be received and played back.

Meanwhile, the multimedia apparatus 100 is an example of an image signal processing apparatus that performs signal processing of an image stored in an apparatus or an input image. As another example of the image signal processing apparatus, the display unit 180 illustrated in FIG. 3 is used. And a set top box in which the audio output unit 185 is excluded, the above-described DVD player, Blu-ray player, game device, computer, etc. may be further illustrated.

4 is an internal block diagram of the controller illustrated in FIG. 3. A description with reference to FIGS. 4 and 3 is as follows.

According to an embodiment of the present invention, the controller 170 may include a demultiplexer 310, an image processor 320, an OSD generator 340, a mixer 350, a frame rate converter 355, and a formatter ( 360). In addition, the apparatus may further include a voice processor (not shown) and a data processor (not shown).

The demultiplexer 310 demultiplexes an input stream. For example, when an MPEG-2 TS is input, it may be demultiplexed and separated into video, audio, and data signals, respectively. Here, the stream signal input to the demultiplexer 310 may be a stream signal output from the tuner 110, the demodulator 120, or the external device interface unit 135.

The image processor 320 may perform image processing of the demultiplexed image signal. To this end, the image processor 320 may include an image decoder 325 and a scaler 335.

The image decoder 325 decodes the demultiplexed image signal, and the scaler 335 performs scaling to output the resolution of the decoded image signal on the display unit 180.

The video decoder 325 may include decoders of various standards.

For example, when the demultiplexed video signal is an encoded video signal of MPEG-2 standard, it may be decoded by an MPEG-2 decoder.

Also, for example, when the demultiplexed video signal is a digital video broadcasting (DMB) scheme or an encoded video signal of H.264 standard according to DVB-H, it may be decoded by an H.264 decoder.

On the other hand, the video signal decoded by the image processor 320 is input to the mixer 350.

The OSD generator 340 generates an OSD signal according to a user input or itself. For example, based on a control signal from the user input interface unit 150, a signal for displaying various types of information on a screen of the display unit 180 as a graphic or text may be generated. The generated OSD signal may include various data such as a user interface screen, various menu screens, widgets, and icons of the multimedia apparatus 100.

For example, the OSD generator 340 may generate a signal for displaying broadcast information based on subtitles or EPGs of a broadcast image.

The mixer 350 may mix the OSD signal generated by the OSD generator 340 and the decoded image signal processed by the image processor 220. The mixed signal is provided to the formatter 360. Since the decoded broadcast video signal or the external input signal and the OSD signal are mixed, the OSD can be overlaid on the broadcast image or the external input image.

The frame rate converter 512 may convert the frame rate of the input video. For example, a 60Hz frame rate is converted to 120Hz or 240Hz. When converting a frame rate of 60 Hz to 120 Hz, it is possible to insert the same first frame or insert a third frame predicted from the first frame and the second frame between the first frame and the second frame. When converting a frame rate of 60 Hz to 240 Hz, it is possible to insert three more identical frames or three predicted frames. Meanwhile, it is also possible to maintain the input frame rate without additional conversion.

The formatter 360 receives the output signal of the frame rate converter 355 and changes the format of the signal to be suitable for the display unit 180 and outputs the signal. For example, the R, G, and B data signals may be output, and the R, G, and B data signals may be output as low voltage differential signaling (LVDS) or mini-LVDS.

The voice processing unit (not shown) in the controller 170 may perform voice processing of the demultiplexed voice signal. To this end, the voice processing unit (not shown) may include various decoders.

The speech processing unit (not shown) in the controller 170 may decode the demultiplexed speech signal if it is an encoded speech signal. For example, the demultiplexed speech signal may be decoded by an MPEG-2 decoder, or an MPEG 4 decoder, or an AAC decoder, or an AC-3 decoder.

Also, the voice processing unit (not shown) in the controller 170 may process a base, a treble, a volume control, and the like.

The data processor (not shown) in the controller 170 may perform data processing of the demultiplexed data signal. For example, when the demultiplexed data signal is an encoded data signal, it may be decoded. The encoded data signal may be EPG (Electronic Program Guide) information including broadcast information such as a start time and an end time of a broadcast program broadcasted in each channel. For example, the EPG information may be ATSC-PSIP (ATSC-Program and System Information Protocol) information in the case of the ATSC scheme, and may include DVB-Service Information (DVB-SI) in the case of the DVB scheme. .

The ATSC-PSIP information or the DVB-SI information may be information included in the aforementioned stream, that is, header (4 bytes) of the MPEG-2 TS.

On the other hand, since the block diagram of the controller 170 shown in Figure 4 is a block diagram for an embodiment of the present invention, it is also possible to add another module or omit some of the illustrated module according to the needs of those skilled in the art.

5 is a diagram illustrating a user and a multimedia apparatus according to an exemplary embodiment.

According to an embodiment, the multimedia apparatus 1900 acquires a user image through the camera unit 1901 of the multimedia apparatus, in order to capture an image of the user 1903.

In addition, the camera unit may be composed of two image sensor modules for obtaining different information from each other in order to accurately recognize the distance information of the user. That is, according to an embodiment, the camera unit may include a depth image sensor and an RGB image sensor.

In addition, as shown, the camera unit 1901 of the multimedia device 1900 is located at the lower end of the multimedia device, so that it is easy to detect the center of the user's body and is relatively less affected in the illuminance environment. Bar gestures can be recognized and acquisition of user images can be performed smoothly.

In addition, the multimedia apparatus 1900 may include a voice input unit 1902 to receive a user voice.

The voice input unit 1902 may receive a predetermined voice from the user. That is, the voice input unit 1902 may include a microphone. In addition, the input voice may be transmitted to the controller of the multimedia apparatus to perform an operation for voice recognition.

Through the above configuration, the multimedia apparatus may obtain information about the image and the voice of the user and perform a control operation of the multimedia apparatus.

6 is a diagram illustrating a display screen including a menu for acquiring a user image according to an exemplary embodiment.

According to an embodiment of the present disclosure, the multimedia apparatus may identify an image 2401 of a depth image sensor for recognizing skeleton information of a user and a portion of a user that may recognize a user based on the image of the depth image sensor, according to an exemplary embodiment. A magnified image 2405 magnified through the RGB image sensor may be displayed.

Referring to FIG. 6, the multimedia apparatus may obtain coordinate information of each part of the user's body through the image 2401 captured by the depth image sensor.

That is, for example, coordinate information about only the right elbow 2402 of the user's body may be obtained. Also, when the user can determine the user through the chest 2403 of the body, the user acquires coordinate information of the body part and enlarges the area where the body part is located through the RGB image sensor. ) To obtain a more detailed magnified image 2405. In addition, when the user moves a body to perform a specific action, by detecting the positional movement of the body skeletal information, it can recognize what kind of action.

7 is an internal block diagram of a remote control apparatus for controlling any one of the multimedia apparatus according to embodiments of the present invention.

Referring to the drawings, the remote control apparatus 200 includes a wireless communication unit 225, a user input unit 235, a sensor unit 240, an output unit 250, a power supply unit 260, a storage unit 270, The controller 280 may be included.

The wireless communication unit 225 transmits and receives a signal with any one of the multimedia apparatus according to the embodiments of the present invention described above. Among the multimedia devices according to embodiments of the present invention, one multimedia device 100 will be described as an example.

In the present embodiment, the remote control apparatus 200 may include an RF module 221 capable of transmitting and receiving a signal with the multimedia apparatus 100 according to the RF communication standard. In addition, the remote control apparatus 200 may include an IR module 223 capable of transmitting and receiving a signal with the multimedia apparatus 100 according to the IR communication standard.

In the present embodiment, the remote control apparatus 200 transmits a signal containing information on the movement of the remote control apparatus 200 to the multimedia apparatus 100 through the RF module 221.

In addition, the remote control apparatus 200 may receive a signal transmitted from the multimedia apparatus 100 through the RF module 221. In addition, the remote control apparatus 200 may transmit a command regarding power on / off, channel change, volume change, etc. to the multimedia apparatus 100 through the IR module 223 as necessary.

The user input unit 235 may be configured as a keypad, a button, a touch pad, or a touch screen. The user may input a command related to the multimedia apparatus 100 to the remote control apparatus 200 by manipulating the user input unit 235. When the user input unit 235 includes a hard key button, the user may input a command related to the multimedia apparatus 100 to the remote control apparatus 200 through a push operation of the hard key button. When the user input unit 235 includes a touch screen, a user may input a command related to the multimedia apparatus 100 to the remote control apparatus 200 by touching a soft key of the touch screen. In addition, the user input unit 235 may include various kinds of input means that a user can operate, such as a scroll key or a jog key, and the embodiment does not limit the scope of the present invention.

The sensor unit 240 may include a gyro sensor 241 or an acceleration sensor 243.

The gyro sensor 241 may sense information about the movement of the remote controller 200.

For example, the gyro sensor 241 may sense information about an operation of the remote control apparatus 200 based on the x, y, and z axes. The acceleration sensor 243 may sense information about a moving speed of the remote controller 200. On the other hand, the distance measuring sensor may be further provided, whereby the distance to the display unit 180 can be sensed.

The output unit 250 may output a video or audio signal corresponding to an operation of the user input unit 235 or a signal transmitted from the multimedia apparatus 100. The user may recognize whether the user input unit 235 is manipulated or whether the multimedia apparatus 100 is controlled through the output unit 250.

For example, the output unit 250 may be a LED module 251 that is turned on when the user input unit 235 is manipulated or a signal is transmitted and received with the multimedia apparatus 100 through the wireless communication unit 225, and the vibration module 253 generating vibration. ), A sound output module 255 for outputting sound, or a display module 257 for outputting an image.

The power supply unit 260 supplies power to the remote control device 200. The power supply unit 260 may reduce power waste by stopping the power supply when the remote controller 200 does not move for a predetermined time. The power supply unit 260 may resume power supply when a predetermined key provided in the remote control apparatus 200 is operated.

The storage unit 270 may store various types of programs, application data, and the like necessary for controlling or operating the remote control apparatus 200. If the remote control apparatus 200 transmits and receives a signal wirelessly through the multimedia apparatus 100 and the RF module 221, the remote control politics 200 and the multimedia apparatus 100 transmit and receive a signal through a predetermined frequency band. . The controller 280 of the remote controller 200 stores and stores information on a frequency band, etc., capable of wirelessly transmitting and receiving signals with the multimedia apparatus 100 paired with the remote controller politics 200 in the storage 270. can do.

The control unit 280 controls various matters related to the control of the remote control device 200. [ The controller 280 transmits a signal corresponding to a predetermined key manipulation of the user input unit 235 or a signal corresponding to the movement of the remote controller 200 sensed by the sensor unit 240 through the wireless communication unit 225. 100).

In addition, the remote control device 200 may include a voice input unit 236.

The voice input unit 236 may receive a predetermined voice from the user. That is, the voice input unit 236 may include a microphone. In addition, the input voice may be transmitted to the multimedia apparatus 100 so that the multimedia apparatus may perform an operation for speech recognition.

FIG. 8 is a diagram illustrating a first embodiment of a user interface provided by any one of multimedia devices according to embodiments of the present disclosure.

8 illustrates that a web screen is displayed on a display unit of a multimedia apparatus.

First, FIG. 8A illustrates that a searchable predetermined web screen 710 having a search window 720 is displayed on the display unit 180. The user may use a text box (not shown) provided in a keypad (not shown) or a local key (not shown) displayed on the screen, or a text key (not shown) provided in a remote control device. In 720, a predetermined character can be input.

Next, FIG. 8B illustrates that the search result screen 730 is displayed on the display unit 180 according to the search word input to the search box. The multimedia device according to an embodiment of the present invention can be fully browsed when the web screen is displayed, so that the user can easily grasp the web screen.

That is, the multimedia apparatus may obtain a search result related to a search word input by the user through the display screen as shown in FIG. 8.

In addition, in relation to an embodiment of the present disclosure, the search word may be a predetermined keyword included in a chat message input by a user.

9 is a diagram illustrating a second embodiment of a user interface provided by any one of multimedia devices according to embodiments of the present disclosure.

9 illustrates that a web screen is displayed on a display unit of a multimedia apparatus.

First, FIG. 9A illustrates that a mail service screen 810 including an ID input window 820 and a password input window 825 is displayed on the display unit 180. The user inputs an ID using a keypad (not shown) displayed on the screen, a text key (not shown) provided in a local key (not shown), or a text key (not shown) provided in a remote control device. In 820 and the password input window 825, predetermined characters can be input. This will allow you to log in to the mail service.

Next, FIG. 9B illustrates that the screen 830 after logging in to the mail service is displayed on the display unit 180. For example, a 'read a letter' item, a 'write a letter' item, a 'sent out' item, a 'inbox' item, a 'trash box' item, and the like may be exemplified. In addition, mails may be sorted and displayed as 'sent' and 'title' items in the 'inbox' item.

The multimedia apparatus according to an embodiment of the present invention enables full browsing when displaying a mail service screen, so that a user can easily use a mail service.

That is, the multimedia apparatus may use various services such as a mail service through the display screen shown in FIG. 9.

10 is a diagram illustrating a display screen for providing a chat service in a multimedia apparatus according to the prior art.

According to the prior art, the multimedia device transmits and receives data to and from other multimedia devices connected to a network, and provides a chat service to a user, and a video chat mode for transmitting a real time user image through a video, or text and a preset user stop. A chat service may be provided through a still video chat mode in which a chat message is transmitted through a video.

Referring to FIG. 10 (a), the multimedia apparatus displays a video chat window 900 and displays a video chat window 900 on the video chat window 900 through a real-time video 901 and a multimedia device of the other party having a conversation. The video 902 of the user who is using the chat service may be included and displayed.

That is, the multimedia apparatus may provide a user with a real-time image of each user participating in the chat service by continuously photographing the user image through the camera unit of the multimedia apparatus and transmitting the captured image to another multimedia apparatus through a network. .

However, when providing a chat service using the video chat window 900 as shown in FIG. 10 (a), the above chat service is provided in an area where a network having a high data transmission speed is not installed. Otherwise, a delay may occur in the user image 901 displayed on the video chat window 900 and a disconnection may occur, thereby preventing the user from using a smooth chat service. In addition, when the data transmission amount of the network provided with the chat service is increased, the transmission speed may be lowered and the above problems may occur.

In addition, referring to FIG. 10 (b), the multimedia apparatus displays a text chat window 910, and in the text chat window 910, still images 911 and 912 of the other party who are talking via a chat service, and It can be displayed including a chat message.

That is, when a user uses a chat service, the multimedia apparatus may capture a still image of the user through the camera unit of the multimedia apparatus, or search for a user still image previously stored in the multimedia apparatus, and display the user image through another network. By transmitting to the user, it is possible to provide the user with a user image that can identify each user participating in the chat service.

However, when providing a chat service through the text chat window 910 as shown in Figure 10 (b), there is a problem that it is difficult to determine the exact user intention through only the user image and text that is still, effective communication between users The problem of not doing this may occur.

11 is a flowchart illustrating a chat message input method including a user image according to an exemplary embodiment of the present invention.

According to an embodiment, the multimedia apparatus may display a chat window including a chat message and a user image, and update the user image included in the chat window when a specific capture condition occurs. Chat service can be provided.

To this end, first, the multimedia apparatus starts capturing the user image (S101).

When the chat service starts and the chat window is displayed, the multimedia apparatus captures a user image through a camera unit provided in the multimedia apparatus.

The camera unit may include a depth image sensor capable of acquiring distance information of the user and an RGB image sensor capable of acquiring color information of the user.

Therefore, the multimedia apparatus may acquire coordinate information of each part of the user's body or coordinate information of each part of the user's face through the camera unit.

In addition, the multimedia apparatus may display a selection menu for selecting a mode of a chat service as shown in FIG. 14, in which the user image is displayed according to a specific condition according to an embodiment of the present invention. Only when a menu for capturing and updating a user image is selected, photographing of the user image through the camera unit may be started.

Next, the multimedia device detects a user state (S102).

According to an embodiment of the present disclosure, the multimedia apparatus may detect what operation or expression is performed by a user photographed through the camera unit.

For example, when the coordinates or skeleton information of each part of the user's body is obtained through the camera unit, the multimedia apparatus tracks the movement of the coordinates so as to detect what action the user photographed through the camera unit performs. Can be.

Also, for example, when the coordinate information of each part of the user's face of the user's face is obtained through the camera unit, the multimedia apparatus tracks the movement of the coordinates, so that the user photographed through the camera unit may make a certain expression. Can be detected.

Next, the multimedia device detects whether a capture condition previously stored in the multimedia device occurs (S103).

The multimedia apparatus may store a capture condition for the user image photographed through the camera unit.

The capture condition may include a specific gesture or a specific facial expression of the user.

That is, the multimedia apparatus stores information on what kind of facial expressions the user makes or how to capture the user's image when the user makes an action, and through the information detected in the user state sensing step S102, the user It is possible to search whether the operation corresponds to the capture condition or the expression corresponding to the capture condition.

Next, when the capture condition occurs, the multimedia apparatus captures a still image of the user image photographed through the camera unit of the multimedia apparatus (S104).

That is, when a capture condition occurs, the multimedia apparatus may capture an image captured by the camera unit and store the captured image as a capture image.

On the other hand, if the capture condition does not occur, the multimedia device continues to sense the user state.

Next, the multimedia apparatus updates the user image included in the chat window displayed on the display unit with the capture image (S105).

According to an embodiment, the multimedia apparatus may obtain a capture image including a user expression corresponding to a capture condition or a capture image including a user action corresponding to the capture condition from a user image photographed through the camera unit. The captured image may be transmitted to other users participating in the chat service through the network, thereby updating the user image.

Through the above steps, since the multimedia apparatus photographs and transmits a user image only when a specific capture condition occurs, the multimedia device can reduce a load on data transmission and reception in a network and facilitate communication between users.

12 is a flowchart illustrating a chat message input method including a keyword search result according to an exemplary embodiment of the present invention.

According to an embodiment of the present disclosure, when displaying a chat window including a chat message, the multimedia apparatus may display a search result using a keyword included in the chat message as a search word and provide the same to a user.

To this end, first, the multimedia apparatus receives text to be displayed as a chat message from the user (S201).

The multimedia apparatus may receive text through the user interface. In addition, as a user input device (UID) for transmitting a predetermined text input signal to the user interface, a touch screen input through a remote keyboard, a remote controller, a spatial remote controller, an OSD (On Screen Display) keyboard, Various input means may be used. In addition, the voice recognition unit of the multimedia apparatus may receive text by converting the user's voice into text.

In addition, the multimedia apparatus may display the input text as a chat message on the display unit and transmit the text to other chat service participants through a network.

Next, a keyword included in the chat message is extracted (S202).

That is, when a chat message is input, the multimedia apparatus may extract texts included in the chat message, determine the importance of each extracted text, and extract text having a high importance as a keyword.

To this end, the multimedia device may store a database storing a list of important texts.

Next, search for information related to the extracted keyword (S203).

According to an embodiment, the multimedia apparatus may extract a keyword and perform a search process using the extracted keyword as a keyword.

For example, the multimedia apparatus may store a list of contents or services available in the multimedia apparatus, and when the keywords are extracted, the multimedia apparatus may search for contents or services corresponding to the extracted keywords.

In another example, when the keyword is extracted, the multimedia apparatus may transmit a search request using the extracted keyword as a search word to a search server through a network and receive a search result from the search server.

Next, a search result according to the search is displayed (S204).

According to an embodiment, the multimedia apparatus may display the search result along with a chat window for providing a chat service.

In addition, the search result may be displayed in a semi-transparent color or minimized in a predetermined area where the chat message on the display screen is not displayed so as not to disturb the chat message of the ongoing chat service.

Therefore, the user can simultaneously check the search results related to the chat message through the above steps, thereby increasing the user's convenience in the process of using the chat service.

13 is a diagram illustrating a chat message input method according to an embodiment of the present invention.

According to an embodiment, the multimedia apparatus may execute a command included in a user image and update an image information window displayed as a user image.

The above embodiment will be described in detail as follows.

First, the multimedia apparatus recognizes a user image through a camera unit of the multimedia apparatus (S305).

According to an embodiment, the camera unit may include a plurality of image sensors capable of acquiring different information. That is, for example, a depth image sensor and an RGB image sensor may be included to determine distance information and color information of the user in the user image.

Therefore, the multimedia apparatus may acquire various information by photographing a user image.

Next, the multimedia apparatus determines whether a text included in the chat message or a command for executing a specific function of the multimedia apparatus is included in the user image recognized in the step S301 (S302).

According to an embodiment, the multimedia apparatus may execute a specific function of the multimedia apparatus through a text input or a user gesture input.

Therefore, the multimedia apparatus may determine whether a chat message input for using the chat service or a user image captured by the camera unit includes text or user gestures in which mapping information related to execution of a specific function is stored. have.

Next, when the text or the user image recognized in the step S301 includes a command for executing a specific function of the multimedia device, the multimedia device performs an operation corresponding to the included command (S303).

Accordingly, even when a multimedia device receives a chat message for using a chat service or captures a user image, if a text or user gesture for executing a specific command is detected, the multimedia apparatus switches the display screen to execute the specific command, or In particular, the specific command may be executed using multi-tasking.

On the other hand, if the text or the user image recognized in the step S302 does not include a command for executing a specific function of the multimedia device, corresponding to the capture condition to the user image recognized in the step S301 It is determined whether the user's motion or facial expression is included (S304).

That is, as described with reference to FIG. 11, the multimedia apparatus may set a specific capture condition for the user image, and thus, the multimedia apparatus may determine whether a state corresponding to the preset capture condition is detected in the captured user image. Can be.

Next, when the user image includes the user's motion or facial expression corresponding to the capture condition, the multimedia apparatus captures the user image and updates the user image information window (S306).

That is, according to an embodiment, the multimedia apparatus captures a user image whenever a capture condition occurs, thereby acquiring a new user image, and transmitting the acquired image to another user participating in a chat service through a network. The included user image information window can be updated.

On the other hand, if the user's image does not include a user's motion or facial expression corresponding to a capture condition, it may be determined whether a predetermined time has elapsed after the image of the user image information window included in the chat window is updated. In operation S308, when a predetermined time has elapsed, the user image is captured to update the user image information window in operation S306.

That is, even if a state corresponding to a capture condition does not occur, the multimedia apparatus may provide a new user image by updating the user image after a predetermined time elapses.

The predetermined time may be a time set at the time of manufacture of the multimedia apparatus or a time changed according to a user setting.

On the other hand, if a predetermined time has not elapsed after the image of the user image information window included in the chat window is updated, the multimedia apparatus continues to recognize the user image.

14 is a diagram illustrating a display screen including a chat mode setting menu according to an exemplary embodiment of the present invention.

According to an embodiment of the present disclosure, when a chat service through a network is provided to a user, the multimedia apparatus may provide different display screens to the user according to a mode of the chat service.

That is, the multimedia apparatus may include a first mode 1002 for providing a user image through a still image, a second mode 1003 for providing a user image updated according to a predetermined condition, and a second image for providing a user image through a video. The chat service may be provided using the three modes 1004.

Accordingly, the multimedia apparatus may display the mode selection menu 1001 including the predetermined mode selection items 1002, 1003, and 1004.

In addition, when the user selects the first mode 1002 through the predetermined pointer 205, the multimedia apparatus uses the still image chat service to the user through the same display screen as the chat window shown in FIG. If the third mode 1004 is selected, the multimedia apparatus may provide a chat service using a video to the user through the same display screen as the chat window shown in FIG.

In addition, when the user selects the second mode 1003 through the pointer 205, a chat service may be provided using the user image updated according to a predetermined condition according to FIGS. 11 to 13. An example of the display screen of the chat window provided in the second mode may be described with reference to FIGS. 15 and 16.

15 is a diagram illustrating a display screen including a chat window in which a user image is displayed according to an exemplary embodiment of the present invention.

According to an embodiment, when providing a chat service through a network to a user, the multimedia apparatus may display user images 1101, 1102, 1103, and 1104 corresponding to a chat message input by the user.

Accordingly, the multimedia apparatus tracks a change in an operation or facial expression of a user who is using a chat service through the multimedia apparatus through a camera unit of the multimedia apparatus, and corresponds to a capture condition preset in the multimedia apparatus. When the motion or facial expression is detected, the multimedia apparatus may capture the image of the user and display the chat message 110 together with the chat message input from the user.

That is, referring to FIG. 15, the multimedia apparatus detects a user's motion or facial expression through the camera unit of the multimedia apparatus while displaying a chat message by displaying a chat window 1100 for providing a chat service to the user. do. In this case, the multimedia apparatus may acquire coordinates of each part of the user's body or coordinates of each part of the face, and detect a movement type of the coordinates to determine what the user is doing or what facial expression is made. .

Accordingly, when the shaking of the first user participating in the chatting service is detected while the chat service is in progress, the multimedia apparatus captures an image including the shaking of the first user. The capture image 1101 may be displayed along with the chat message 1105.

In addition, the second user participating in the chat service displays the basic image 1102 and the chat message 1106 of the second user when no motion or facial expression corresponding to the capture condition is detected.

In addition, when the motion of raising the fist of the third user participating in the chat service is detected, the multimedia apparatus captures an image including the motion of the fist of the third user, and the third user's chat message 1107. The capture image 1103 may be displayed together with the image.

In addition, when a winking facial expression of the fourth user participating in the chat service is detected, the multimedia apparatus captures an image including the winking facial expression of the fourth user, and together with the chat message 1108 of the fourth user. Capture image 1108 may be displayed.

In addition, according to an embodiment, the user images 1101, 1102, 1103, 1104 and chat messages 1105, 1106, 1107, and 1108 may include a top portion of a display screen when a new chat message is input or a new user image is captured. Scrolls to a new chat message or a new user image, or replaces only the displayed chat message or the user image while fixing the position of the user image and the chat message.

FIG. 16 illustrates a display screen including a chat window 1200 in which a keyword search result is displayed according to an exemplary embodiment of the present invention.

According to an embodiment of the present disclosure, when providing a chat service through a network to a user, the multimedia apparatus may display a search result using the keyword 1201 included in the user's chat message as a search word.

The multimedia apparatus displays each chat message of the users who are using the chat service, and detects whether a keyword is included in the chat message.

Therefore, the word included in each chat message is analyzed, and a word corresponding to the keyword among the analyzed words is extracted based on the keyword information stored in the multimedia apparatus. The keyword information may include, for example, statistical data on words frequently entered in an internet search engine.

In addition, when a keyword 1201 is detected in a chat message of a specific user among users who are using a chat service, the detected keyword 1201 may be underlined or displayed in bold font.

That is, as shown in Fig. 16, let's watch a movie. If the word "minority report" is detected as a keyword in the chat message "minority report", the multimedia apparatus may display the "minority report" underlined.

In addition, the multimedia apparatus may perform a search process using the detected keyword as a search word. That is, the multimedia apparatus may search for contents or services corresponding to the detected keyword in the multimedia apparatus or search for contents or services connected through a network.

Referring to FIG. 16, the multimedia apparatus may display a search result window 1203 including a result obtained through a search engine connected to a network, using 'minority report' as a search word. In addition, the user image 1202 included in the chat window 1200 may be changed to an image corresponding to the search word.

The search result window 1203 may include shortcut items 1204 of a service or content available through a network.

That is, since the multimedia apparatus may obtain link information of the available service or content through a search engine connected to the network, the shortcut device 1204 is included in the search result window 1203 using the link information. Can be displayed.

Accordingly, as shown in FIG. 16, the user may view a 'minority report wallpaper' related to the 'minority report' keyword, go to a service page for 'minority report booking', or go to a service page for 'minority report download' Or a shortcut item for moving to a service page for 'minority report rental' or a web page for 'buying a minority report DVD'. Thus, the user can immediately use the selected service or content by selecting any one of the shortcut items.

In addition, according to an embodiment, the display screen according to the selected service or content may be minimized or displayed in a predetermined area or may be displayed in a translucent color so as not to disturb the display screen of the chat message.

In addition, unlike FIG. 16, the multimedia apparatus may search for content or services stored in the multimedia apparatus and display a search result. In addition, the user may select the content or service included in the search result and immediately use the selected content or service.

17 is a diagram illustrating a database for storing capture conditions of a multimedia device according to an embodiment of the present invention.

According to an embodiment of the present disclosure, when a user uses a chat service through a multimedia device, the multimedia device tracks a facial expression or an operation of the user through a camera unit of the multimedia device, and when a specific facial expression or motion is detected, the multimedia device is predetermined. The user image of may be captured and provided to another user. Accordingly, the multimedia apparatus may store a database 1300 including mapping information on the capture condition 1301 for capturing a user image.

That is, the database 1300 may include mapping information about the capture area 1302 of the user image according to the capture condition 1301.

According to an embodiment, the database 1300 may detect a facial expression of a user who uses a chat service (1303) or recognize a motion (1304) to detect a capture condition.

For example, the multimedia apparatus recognizes the user's expression, and when the user makes a smiley face, makes a crying face, or makes an angry face, the multimedia apparatus displays an image of the face of the user through the camera unit of the multimedia device. I can capture it.

That is, for this purpose, the camera unit of the multimedia apparatus acquires position coordinates of a user's eyes, nose, mouth, etc., tracks whether the acquired position coordinates are moved, and data on an expression matching the movement of the position coordinates. By searching for, the user may recognize what facial expressions the user makes, and the multimedia apparatus includes a face part of the user who makes the facial expression to another user who is using a chat service when the user makes the facial expression. You can capture and provide a video to say.

Also, for example, the multimedia apparatus recognizes a user's motion, and when the user shakes his hand or lifts a fist, captures an image including the upper body of the user, and the user stands up or When the two hands are lifted, an image including the whole body of the user may be captured.

That is, for this purpose, the camera unit of the multimedia apparatus acquires position coordinates of an arm, a leg, a shoulder, etc. of a user, and tracks whether the acquired position coordinates are moved and data about an operation matching the movement of the position coordinates. By searching for, the user may recognize what action the user takes, and the multimedia apparatus may provide a capture image, which may include the user's action, to another user who is using the chat service.

In addition, for example, the multimedia apparatus may capture the user image when a predetermined time elapses while the operation or the expression corresponding to the capture conditions is not searched. For example, as shown in FIG. 17, when 5 minutes or more have elapsed while the user has not performed any facial expressions or actions, a video including the face of the user is captured to chat the most recent user video. It can be provided to other users who are using.

In addition, the mapping data included in the database illustrated in FIG. 17 is only an example, and the mapping data may be stored differently according to a multimedia manufacturer, and capture conditions or capture regions may be edited by a user.

The multimedia device and its operation method according to the present invention are not limited to the configuration and method of the embodiments described as described above, but the embodiments may be modified in whole or in part to enable various modifications. It may alternatively be configured in combination.

On the other hand, the operating method of the multimedia apparatus of the present invention can be implemented as a processor-readable code on a processor-readable recording medium provided in the multimedia apparatus. The processor-readable recording medium includes all kinds of recording apparatuses in which data that can be read by the processor is stored. Examples of the recording medium that can be read by the processor include a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like, and may also be implemented in the form of a carrier wave such as transmission over the Internet . The processor-readable recording medium can also be distributed over network coupled computer systems so that the processor-readable code is stored and executed in a distributed fashion.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, It will be understood by those skilled in the art that various changes and modifications may be made without departing from the spirit and scope of the present invention.

100: multimedia device
105: broadcast receiving unit
110: tuner
120: demodulator
130: network interface unit
135: external device interface
140: storage unit
150: user interface unit
170:
180: display unit
185: audio output
190: power supply
200: remote control unit

Claims (16)

In the chat service providing method of the multimedia device,
Displaying a chat message and a user image of a user participating in the chat service;
Initiating photographing of a user image;
Capturing the captured image based on a preset capture condition; And
Updating the displayed user image with the captured image
Chat service providing method of a multimedia device comprising a. The method of claim 1,
Capturing the captured image,
Acquiring coordinate information of each part of a user's body from the user image;
Tracking the movement of the obtained coordinates;
Capturing the captured image when the movement of the tracked coordinates corresponds to a preset capture condition
Chat service providing method of a multimedia device comprising a. The method of claim 1,
Capturing the captured image,
Acquiring coordinate information of each part of the eye area in the face area of the user from the user image;
Tracking the movement of the obtained coordinates;
Capturing the captured image when the movement of the tracked coordinates corresponds to a preset capture condition
Chat service providing method of a multimedia device comprising a. The method of claim 1,
Displaying the chat message and the user image,
Extracting keywords from the chat message;
Transmitting a search request using the extracted keyword as a search word to a search server;
Receiving a search result from the search server; And
Displaying the search results
Chat service providing method of a multimedia device comprising a. The method of claim 1,
Displaying the chat message and the user image,
Extracting keywords from the chat message;
Searching for the content or service corresponding to the extracted keyword in the multimedia apparatus; And
Displaying the search results
Chat service providing method of a multimedia device comprising a. The method of claim 1,
Capturing the captured image,
Capturing the captured image when the state corresponding to the capture condition is not detected for a preset time in the captured image
Chat service providing method of a multimedia device comprising a. The method of claim 1,
Capturing the captured image,
Cropping the captured image based on the capture condition
Chat service providing method of the multimedia device further comprising. The method of claim 1,
Initiating the recording of the user image,
Acquiring user distance information through a first image sensor of the multimedia apparatus; And
Obtaining image information based on the distance information through a second image sensor of the multimedia apparatus;
Chat service providing method of a multimedia device comprising a. In the multimedia device that can provide a chat service,
A network interface unit for transmitting and receiving data through a network;
A camera unit for photographing a user image;
A storage unit which stores a capture condition for the user image;
A display unit configured to display a chat message and a user image of a user participating in the chat service; And
Load the capture condition from the storage unit, capture a user image photographed by the camera unit based on the loaded capture condition, and update the displayed user image with the captured image; Control unit
Multimedia device capable of providing a chat service comprising a. 10. The method of claim 9,
The control unit,
In the user image, coordinate information of each part of the user's body is obtained and the movement of the acquired coordinates is tracked so that the movement of the tracked coordinates corresponds to a preset capture condition. To capture the
Multimedia device that can provide chat service. 10. The method of claim 9,
The control unit,
If the movement of the tracked coordinates corresponds to a preset capture condition by acquiring coordinate information of each part of the eye area in the face region of the user, tracking the movement of the acquired coordinates in the user image, To capture the captured image
Multimedia device that can provide chat service. 10. The method of claim 9,
The control unit,
Extracting a keyword from the displayed chat message, transmitting a search request using the extracted keyword as a search word to the search server through the network interface unit, receiving a search result from the search server, and receiving the received search result. To display on the display
Multimedia device that can provide chat service. 10. The method of claim 9,
The storage unit,
Store data on content or services available in the multimedia device;
The control unit,
Extracting a keyword from the displayed chat message, searching the content or service corresponding to the extracted keyword in the storage unit, and controlling to display the search result on the display unit;
Multimedia device that can provide chat service. 10. The method of claim 9,
The control unit,
If a state corresponding to the capture condition is not detected for a preset time in the user image photographed by the camera unit, controlling to capture the captured user image.
Multimedia device that can provide chat service. 10. The method of claim 9,
The control unit,
Based on the capture condition, controlling to crop the captured image
Multimedia device that can provide chat service. 10. The method of claim 9,
The camera unit,
A depth image sensor obtaining distance information of the user as an image; And an RGB image sensor which acquires color information of the user as an image.
Multimedia device that can provide chat service.

Images