KR101804913B1 - Method for providing chatting service in multimedia device and multimedia device thereof - Google Patents

Abstract

The present invention relates to a method of operating a multimedia device and a multimedia device using the same, and more particularly, to a multimedia device capable of smooth communication between users when a user uses a chat service through a multimedia device. The multimedia device includes a chat message and a user image Capturing the user image, capturing the captured image based on a predetermined capture condition, and updating the displayed user image with the captured image. Accordingly, when chatting with another user connected to the network through the multimedia device, a chat message including various images can be input, so that a user using the chat service can transmit the correct message to another user.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a chat service providing method for a multimedia device,

The present invention relates to a method of operating a multimedia device and a multimedia device therefor, and more particularly, to a method of operating a multimedia device capable of smooth communication between users when a user uses a chat service through a multimedia device, To a multimedia device.

The multimedia device may include, for example, an image display device having a function of receiving and processing a broadcast image that a user can view. The multimedia device displays, for example, a broadcast selected by a user among broadcast signals transmitted from the broadcast station on the display. Currently, broadcasting is changing from analog broadcasting to digital broadcasting around the world.

Digital broadcasting refers to broadcasting in which digital video and audio signals are transmitted. Compared to analog broadcasting, digital broadcasting is strong against external noise and has a small data loss, is advantageous for error correction, has a high resolution, and provides a clear screen. Also, unlike analog broadcasting, digital broadcasting is capable of bidirectional service.

Meanwhile, in recent years, a smart TV (Smart TV), which combines the functions of video display devices, is in the spotlight.

This smart TV is an intelligent terminal that adds computer support functions such as internet communication and information retrieval to existing television. It has a large memory and high performance CPU (Central Processing Unit) compared to existing TV, And an operating system (OS) for supporting communications and PC (personal computer) interworking. Using this smart TV, unlike existing multimedia devices, it is free to use services through the Internet. In particular, it is characterized in that an application desired by the user can be downloaded and installed from the outside. In addition, a chat message capable of communicating with other users connected to the network through the application can be input and transmitted to other users.

However, in the case of using the chat service through the multimedia device of the related art, there is a problem that it is difficult to check the facial expression change and the gesture of the other party when the chat message is inputted only through text or voice. There has been a problem in that a high-capacity data transmission is required and delays occur in the chatting process.

Therefore, in the case of using the chat service in the multimedia device, it is possible to input an arbitrary image together with text or voice to increase the accuracy in communication between users, and to communicate clearly with other users It is necessary to develop a chat service providing method and a multimedia device therefor.

It is an object of the present invention to provide a multimedia device and a method of operating the same in a communication process through a chat when a user is chatting with another user connected to the network through a multimedia device have.

It is another object of the present invention to provide a multimedia device capable of providing various user interfaces in the transmission of a chat message and an operation method thereof.

According to another aspect of the present invention, there is provided a method of providing a chat service in a multimedia device, the method comprising: displaying a chat message and a user image of a user participating in the chat service; Starting shooting of a user image; Capturing the captured image based on a predetermined capture condition; And

And updating the displayed user image with the captured image.

According to another aspect of the present invention, there is provided a multimedia device capable of providing a chat service, including: a network interface unit for transmitting and receiving data through a network; A camera unit for photographing a user image; A storage unit for storing a capture condition for the user image; A display unit for displaying a chat message and a user image of a user participating in the chat service; And loading the capture condition in the storage unit, capturing a user image captured by the camera unit based on the loaded capture condition, and controlling the displayed user image to be updated with the captured image And a control unit.

According to the embodiment of the present invention, when chatting with another user connected to the network through the multimedia device, a chat message including various images can be input and the correct message can be delivered to other users.

In addition, according to the embodiment of the present invention, in the network chatting process through the multimedia device, it is possible to prevent delays due to a decrease in the transmission rate.

Meanwhile, according to another embodiment of the present invention, various user interfaces can be provided in the multimedia device, and the user's convenience of use can be increased.

1 is a diagram schematically showing an example of an entire broadcasting system including a multimedia device according to an embodiment of the present invention.
2 is a diagram schematically showing another example of the entire broadcasting system including a multimedia device according to another embodiment of the present invention.
3 is a view showing in further detail another example of the multimedia device shown in Fig. 1 or Fig.
4 is an internal block diagram of the control unit shown in FIG.
5 illustrates a user and multimedia device in accordance with an embodiment of the present invention.
6 is a view illustrating a display screen including a menu for acquiring a user image according to an embodiment of the present invention.
7 is an internal block diagram of a remote controller for controlling any one of multimedia devices according to an embodiment of the present invention.
FIG. 8 is a diagram illustrating a first embodiment of a user interface provided by any one of the multimedia devices according to the embodiment of the present invention.
9 is a diagram illustrating a second embodiment of a user interface provided by any one of the multimedia devices according to the embodiment of the present invention.
10 is a diagram illustrating a display screen for providing a chat service in a multimedia device according to the related art.
11 is a flowchart illustrating a method of inputting a chat message including a user image according to an embodiment of the present invention.
12 is a flowchart illustrating a method of inputting a chat message including a keyword search result according to an embodiment of the present invention.
13 is a diagram illustrating a chat message input method according to an embodiment of the present invention.
14 is a view illustrating a display screen including a chat mode setting menu according to an embodiment of the present invention.
15 is a view illustrating a display screen including a chat window in which a user image is displayed according to an embodiment of the present invention.
FIG. 16 is a view showing a display screen including a chat window in which a keyword search result according to an embodiment of the present invention is displayed.
17 is a diagram illustrating a database for storing capture conditions of a multimedia device according to an embodiment of the present invention.

Hereinafter, the present invention will be described in more detail with reference to the drawings.

The suffix "module" and " part "for components used in the following description are given merely for ease of description, and the" module "and" part "

Meanwhile, the multimedia device described in this specification is an intelligent multimedia device that adds a computer support function to a broadcast reception function, for example, and is equipped with an Internet function while being faithful to a broadcast reception function. The multimedia device includes a handwriting input device, A remote controller and the like can be provided. With the support of wired or wireless internet function, it can be connected to internet and computer and can perform functions such as e-mail, web browsing, banking or game. A standardized general-purpose OS can be used for these various functions.

Accordingly, the multimedia device described in the present invention can perform various user-friendly functions because various applications can be freely added or deleted on a general-purpose OS kernel, for example. More specifically, the multimedia device may be, for example, a network TV, an HBBTV, a smart TV, or the like, and may be applied to a smart phone in some cases.

BRIEF DESCRIPTION OF THE DRAWINGS The above and other features and advantages of the present invention will be more apparent from the following detailed description taken in conjunction with the accompanying drawings, in which: FIG.

As used herein, terms used in the present invention are selected from general terms that are widely used in the present invention while taking into account the functions of the present invention, but these may vary depending on the intention or custom of a person skilled in the art or the emergence of new technologies. In addition, in certain cases, there may be a term arbitrarily selected by the applicant, in which case the meaning thereof will be described in the description of the corresponding invention. Therefore, it is intended that the terminology used herein should be interpreted based on the meaning of the term rather than on the name of the term, and on the entire contents of the specification.

 1 is a diagram schematically showing an example of an entire broadcasting system including a multimedia device according to an embodiment of the present invention.

1, an overall broadcasting system including a multimedia device according to an embodiment of the present invention includes a content provider (CP) 10, a service provider (SP) 20, A network provider (NP) 30 and an HNED 40. The HNED 40 corresponds to, for example, a client 100 that is a multimedia device according to an embodiment of the present invention. The client 100 corresponds to a multimedia device according to an embodiment of the present invention, and the multimedia device is, for example, a network TV, a smart TV, an IPTV, or the like.

The content provider 10 produces and provides various contents. 1, a terrestrial broadcaster, a cable SO (System Operator), a MSO (Multiple System Operator), a satellite broadcaster, Internet broadcaster and the like can be exemplified.

In addition, the content provider 10 may provide various applications in addition to the broadcast content. In this regard, it will be described in detail later.

The service provider 20 can provide the contents provided by the contents provider 10 in a service package. For example, the service provider 20 of FIG. 1 can package and provide the first terrestrial broadcast, second terrestrial broadcast, cable MSO, satellite broadcast, various Internet broadcasts, applications, etc. to the user.

Meanwhile, the service provider 20 can provide a service to the client 100 using a unicast or multicast method. The unicast method is a method of transmitting data 1: 1 between one sender and one receiver. For example, in a unicast method, when a receiver requests data from a server, the server can transmit data to the receiver upon request. The multicast method is a method of transmitting data to a plurality of receivers of a specific group. For example, the server can transmit data to a plurality of pre-registered receivers at once. An Internet Group Management Protocol (IGMP) protocol may be used for multicast registration.

The network provider 30 may provide a network for providing services to the client 100. The client 100 may be provided with a service by building a home network end user (HNED).

As a means for protecting contents transmitted in the multimedia device system, conditional access or content protection may be used. As an example of such restriction reception or content protection, a scheme such as a cable card (CableCARD) and a DCAS (Downloadable Conditional Access System) may be used.

On the other hand, the client 100 can also provide the content through the network. In this case, conversely, conversely, conversely, the client 100 may be a content provider, and the content provider 10 may receive content from the client 100. [ In this case, there is an advantage that an interactive content service or a data service is possible.

2 is a diagram schematically showing another example of the entire broadcasting system including a multimedia device according to another embodiment of the present invention.

As shown in FIG. 2, the multimedia device 100 according to another embodiment of the present invention is connected to a broadcast network and an Internet network. The multimedia device 100 may be, for example, a network TV, a smart TV, an HBBTV, or the like.

The multimedia device 100 includes a broadcast interface 101, a section filter 102, an AIT filter 103, an application data processing unit 104, a broadcast data processing unit 111, a media player 106, An Internet protocol processing unit 107, an Internet interface 108, and a runtime module 109. [

Application information table (AIT) data, real-time broadcast content, application data, and stream events are received through the broadcast interface 101. Meanwhile, the real-time broadcast content may be referred to as Linear A / V content.

The section filter 102 performs section filtering on four pieces of data received through the broadcast interface 101 to transmit the AIT data to the AIT filter 103. The linear AV contents are transmitted to the broadcast data processing unit 111, And transmits the stream event and application data to the application data processing unit 104. [

Non-Linear A / V content and application data are received through the Internet interface 108. [ The non-linear AVE content may be, for example, a COD (Content On Demand) application.

The non-Linear Above content is transmitted to the media player 106, and the application data is transmitted to the runtime module 109.

Further, the runtime module 109 includes, for example, an application manager and a browser as shown in FIG. The application manager controls the life cycle for the interactive application using, for example, AIT data. The browser performs, for example, the function of displaying and processing the interactive application.

3 is a view showing in further detail another example of the multimedia device shown in Fig. 1 or Fig.

3, a multimedia device 100 according to an exemplary embodiment of the present invention includes a broadcast receiver 105, an external device interface 135, a storage 140, a user input interface 150, A display unit 180, an audio output unit 185, and a camera unit 190. The broadcast receiving unit 105 may include a tuner 110, a demodulator 120, and a network interface unit 130. Of course, the tuner 110 and the demodulator 120 may be provided, but the network interface 130 may not be included. Alternatively, the tuner 110 and the demodulator 120 may be included, It is also possible to design the demodulation unit 120 not to include it.

The tuner 110 selects an RF broadcast signal corresponding to a channel selected by the user or all previously stored channels among RF (Radio Frequency) broadcast signals received through the antenna. Also, the selected RF broadcast signal is converted into an intermediate frequency signal, a baseband image, or a voice signal.

Also, the tuner 110 can receive RF carrier signals of a single carrier according to an Advanced Television System Committee (ATSC) scheme or RF carriers of a plurality of carriers according to a DVB (Digital Video Broadcasting) scheme.

The demodulator 120 receives the digital IF signal DIF converted by the tuner 110 and performs a demodulation operation.

For example, when the digital IF signal output from the tuner 110 is the ATSC scheme, the demodulator 120 performs 8-VSB (8-Vestigal Side Band) demodulation, for example. Also, the demodulation unit 120 may perform channel decoding. To this end, the demodulator 120 includes a trellis decoder, a de-interleaver, and a reed solomon decoder to perform trellis decoding, deinterleaving, Solomon decoding can be performed.

For example, when the digital IF signal output from the tuner 110 is a DVB scheme, the demodulator 120 performs CODDMA (Coded Orthogonal Frequency Division Modulation) demodulation, for example. Also, the demodulation unit 120 may perform channel decoding. For this, the demodulator 120 may include a convolution decoder, a deinterleaver, and a reed-solomon decoder to perform convolutional decoding, deinterleaving, and reed solomon decoding.

The demodulation unit 120 may perform demodulation and channel decoding, and then output a stream signal TS. At this time, the stream signal may be a signal in which a video signal, a voice signal, or a data signal is multiplexed. For example, the stream signal may be an MPEG-2 TS (Transport Stream) multiplexed with an MPEG-2 standard video signal, a Dolby AC-3 standard audio signal, or the like. Specifically, the MPEG-2 TS may include a header of 4 bytes and a payload of 184 bytes.

The stream signal output from the demodulation unit 120 may be input to the controller 170. The control unit 170 performs demultiplexing, video / audio signal processing, and the like, and then outputs the video to the display unit 180 and audio to the audio output unit 185.

The external device interface unit 135 can connect the multimedia device 100 with an external device. To this end, the external device interface unit 135 may include an A / V input / output unit (not shown) or a wireless communication unit (not shown).

The external device interface unit 135 can be connected to an external device such as a DVD (Digital Versatile Disk), a Blu ray, a game device, a camera, a camcorder, a computer (notebook) The external device interface unit 135 transmits external video, audio, or data signals to the controller 170 of the multimedia device 100 through a connected external device. Also, the control unit 170 can output the processed video, audio, or data signal to the connected external device. To this end, the external device interface unit 135 may include an A / V input / output unit (not shown) or a wireless communication unit (not shown).

The A / V input / output unit includes a USB terminal, a CVBS (Composite Video Banking Sync) terminal, a component terminal, an S-video terminal (analog), a DVI (analog) Digital Visual Interface) terminal, an HDMI (High Definition Multimedia Interface) terminal, an RGB terminal, a D-SUB terminal, and the like.

The wireless communication unit can perform short-range wireless communication with other electronic devices. For example, the multimedia device 100 may include Bluetooth, Radio Frequency Identification (RFID), infrared data association (IrDA), Ultra Wideband (UWB), ZigBee, Digital Living Network Alliance (DLNA) And the like.

Also, the external device interface unit 135 may be connected to the various set-top boxes via at least one of the various terminals described above to perform input / output operations with the set-top box.

Meanwhile, the external device interface unit 135 may receive the application or application list in the adjacent external device, and may transmit the received application or application list to the control unit 170 or the storage unit 140.

The network interface unit 130 provides an interface for connecting the multimedia device 100 to a wired / wireless network including the Internet network. The network interface unit 130 may include an Ethernet terminal or the like for connection to a wired network and may be a WLAN (Wireless LAN) (Wi- Fi wireless broadband (Wibro), World Interoperability for Microwave Access (Wimax), and HSDPA (High Speed Downlink Packet Access) communication standards.

The network interface unit 130 can transmit or receive data with other users or other electronic devices via a network connected thereto or another network linked to the connected network. Particularly, it is possible to transmit some content data stored in the multimedia device 100 to another user registered in the multimedia device 100 or a selected user or selected one of the other electronic devices.

On the other hand, the network interface unit 130 can access a predetermined web page through the connected network or another network linked to the connected network. That is, it is possible to access a predetermined web page through a network and transmit or receive data with the server. In addition, content or data provided by a content provider or a network operator can be received. That is, it can receive contents such as a movie, an advertisement, a game, a VOD, a broadcast signal, and related information provided from a content provider or a network provider through a network. In addition, the update information and the update file of the firmware provided by the network operator can be received. It may also transmit data to the Internet or a content provider or network operator.

In addition, when the user uses the chat service through the network, the network interface unit 130 can transmit and receive data such as a chat message or a user image for using the chat service with a predetermined server via a network.

The storage unit 140 may store a program for each signal processing and control in the control unit 170 or may store the processed video, audio, or data signals.

The storage unit 140 may also function to temporarily store video, audio, or data signals input from the external device interface unit 135 or the network interface unit 130. In addition, the storage unit 140 may store information on a predetermined broadcast channel through the channel memory function.

The storage unit 140 may store the capture condition for capturing an image shot through the camera unit 190 in the database 141 when the user uses the chat service through the multimedia device. The database will be described later in detail with reference to FIG.

The storage unit 140 may be a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (for example, SD or XD Memory, etc.), RAM, ROM (EEPROM, etc.), and the like. The multimedia device 100 may reproduce and provide a content file (a moving image file, a still image file, a music file, a document file, an application file, etc.) stored in the storage unit 140 to a user.

FIG. 3 shows an embodiment in which the storage unit 140 is provided separately from the control unit 170, but the scope of the present invention is not limited thereto. The storage unit 140 may be included in the controller 170.

The user input interface unit 150 transmits a signal input by the user to the control unit 170 or a signal from the control unit 170 to the user.

For example, the user input interface unit 150 may be configured to turn on / off the power, select the channel, and display the screen from the remote control device 200 according to various communication methods such as a radio frequency (RF) communication method and an infrared Or a control signal from the control section 170 to the remote control apparatus 200. The remote control apparatus 200 may be configured to receive the control signal from the control section 170,

Also, for example, the user input interface unit 150 can transmit a control signal input from a local key (not shown) such as a power key, a channel key, a volume key, and a set value to the controller 170.

For example, the user input interface unit 150 may transmit a control signal input from a sensing unit (not shown) that senses a user's gesture to the control unit 170, or may sense a signal from the control unit 170 (Not shown). Here, the sensing unit (not shown) may include a touch sensor, an audio sensor, a position sensor, an operation sensor, and the like.

The control unit 170 demultiplexes the input stream or processes the demultiplexed signals through the tuner 110 or the demodulation unit 120 or the external device interface unit 135 to generate a signal for video or audio output Can be generated and output.

The video signal processed by the controller 170 may be input to the display unit 180 and displayed as an image corresponding to the video signal. The video signal processed by the controller 170 may be input to the external output device through the external device interface 135. [

The audio signal processed by the control unit 170 may be output to the audio output unit 185 through audio. The voice signal processed by the control unit 170 may be input to the external output device through the external device interface unit 135.

Although not shown in FIG. 3, the controller 170 may include a demultiplexer, an image processor, and the like. This will be described later with reference to FIG.

In addition, the control unit 170 can control the overall operation in the multimedia device 100. [ For example, the control unit 170 may control the tuner 110 to control the tuner 110 to select an RF broadcast corresponding to a channel selected by the user or a previously stored channel.

In addition, the controller 170 may control the multimedia device 100 by a user command or an internal program input through the user input interface unit 150. [ In particular, the user can access a network and download a list of applications or applications desired by the user to the multimedia device 100.

For example, the controller 170 controls the tuner 110 to input a signal of a selected channel according to a predetermined channel selection command received through the user input interface unit 150. And processes video, audio or data signals of the selected channel. The control unit 170 allows the display unit 180 or the audio output unit 185 to output the video or audio signal processed by the user through the channel information or the like selected by the user.

For example, the control unit 170 may control the operation of the external device through the external device interface unit 135, for example, a camera or a camcorder, according to an external device video playback command received through the user input interface unit 150 So that the video signal or the audio signal of the audio signal output through the display unit 180 or the audio output unit 185 can be outputted.

Meanwhile, the control unit 170 may control the display unit 180 to display an image. For example, a broadcast image input through the tuner 110, an external input image input through the external device interface unit 135, an image input through the network interface unit, or an image stored in the storage unit 140 , And display on the display unit 180 can be controlled. At this time, the image displayed on the display unit 180 may be a still image or a moving image, and may be a 2D image or a 3D image.

Also, the control unit 170 can control to reproduce the content. The content at this time may be content stored in the multimedia device 100, received broadcast content, or external input content input from the outside. The content may be at least one of a broadcast image, an external input image, an audio file, a still image, a connected web screen, and a document file.

The control unit 170 can control, in addition to various user interfaces, to install and operate an application downloaded from the external network. Further, it is possible to control so that an image related to the executed application is displayed on the display unit 180 by the user's selection.

Particularly, in connection with the embodiment of the present invention, when a user executes an application for using a chat service, a chat message and a user image of a user participating in the chat service for using the chat service are displayed on the display 180 As shown in FIG. This will be described in detail below with reference to FIG. 15 and the following.

The control unit 170 loads the capture condition in the database 141 stored in the storage unit 140 and controls the camera unit 190 based on the loaded capture condition And controls to update the user image included in the chat window, which is displayed on the display unit 180, with the captured image.

The control unit 170 acquires coordinate information of each part of the user's body from the user image captured by the camera unit 190 and tracks the movement of the obtained coordinates, If the movement corresponds to a preset capture condition, the control unit can control to capture the captured image.

In addition, the control unit 170 acquires coordinate information of each part of the eye contour in the user's face area in the user image photographed by the camera unit 190, tracks the movement of the obtained coordinates, If the movement of the tracked coordinates corresponds to a predetermined capture condition, the control unit can control to capture the captured image.

Thus, for such tracking operations, the camera portion 190 of the multimedia device may include an image sensor capable of acquiring two different pieces of information. Referring to FIG. 6, the image obtained through the image sensor and the image sensor will be referred to.

The controller 170 can capture an image captured by the camera unit 190 when the state corresponding to the capturing condition is not detected for a predetermined period of time. That is, the user image can be updated even if the capture condition does not occur, with a predetermined period of time. The information on the predetermined time may be stored in the storage unit 140 by the user.

The control unit 170 extracts a keyword from the chat message displayed on the display unit 190 and transmits a search request using the extracted keyword as a search word to the search server through the network interface unit 130, And may receive the search result from the search server and control the display unit 180 to display the search result.

The control unit 170 extracts a keyword from the chat message displayed on the display unit 190, searches the content or service corresponding to the extracted keyword in the storage unit, (180).

In addition, the control unit 170 may control the cropping of the captured image in the captured image through the camera unit 190. That is, since not all portions of the captured image are required as a user image, an image obtained by cropping a specific region can be used as a user image. The information about the specific area may be stored in the database 141 together with the capture condition.

The display unit 180 converts an image signal, a data signal, an OSD signal processed by the control unit 170 or a video signal and a data signal received from the external device interface unit 135 into R, G, and B signals, respectively Thereby generating a driving signal.

The display unit 180 may be a PDP, an LCD, an OLED, a flexible display, a 3D display, or the like.

Meanwhile, the display unit 180 may be configured as a touch screen and used as an input device in addition to the output device.

The audio output unit 185 receives a signal processed by the control unit 170, for example, a stereo signal, a 3.1 channel signal, or a 5.1 channel signal, and outputs it as a voice. The voice output unit 185 may be implemented by various types of speakers.

In order to detect a user's gesture, a sensing unit (not shown) having at least one of a touch sensor, a voice sensor, a position sensor, and an operation sensor may be further provided in the multimedia device 100 . A signal sensed by a sensing unit (not shown) may be transmitted to the controller 170 through the user input interface unit 150.

Meanwhile, a camera unit 190 for capturing a user may be further provided. The image information photographed by the camera unit 190 may be input to the control unit 170. [

The camera unit 190 may include a depth image sensor for acquiring distance information of the user as an image and an RGB image sensor for acquiring color information of the user as an image to acquire different information through the two image sensors So that an accurate user image can be obtained.

The control unit 170 may sense the gesture of the user by combining the images photographed by the photographing unit (not shown) or the signals sensed by the sensing unit (not shown).

The remote control apparatus 200 transmits the user input to the user input interface unit 150. [ To this end, the remote control apparatus 200 can use Bluetooth, RF (radio frequency) communication, infrared (IR) communication, UWB (Ultra Wideband), ZigBee, or the like.

Also, the remote control apparatus 200 can receive the video, audio or data signal output from the user input interface unit 150 and display it on the remote control apparatus 200 or output sound or vibration.

The above-described multimedia device 100 may be a fixed-type digital broadcast, an ATSC-based (8-VSB-based) digital broadcast, a DVB-T-based (COFDM-based) digital broadcast, or an ISDB-T (BST- And may be a digital broadcast receiver capable of receiving at least one.

Meanwhile, the block diagram of the multimedia device 100 shown in FIG. 3 is a block diagram for an embodiment of the present invention. Each component of the block diagram can be integrated, added, or omitted according to the specifications of the multimedia device 100 actually implemented. That is, two or more constituent elements may be combined into one constituent element, or one constituent element may be constituted by two or more constituent elements, if necessary. In addition, the functions performed in each block are intended to illustrate the embodiments of the present invention, and the specific operations and apparatuses do not limit the scope of the present invention.

6, the multimedia device 100 does not include the tuner 110 and the demodulation unit 120 shown in FIG. 3, and the network interface unit 130 or the external device interface unit 135, It is possible to receive and reproduce the image content.

The multimedia device 100 is an example of an image signal processing apparatus that performs signal processing of an image stored in a device or an input image. The display unit 180 shown in FIG. 3 is another example of the image signal processing apparatus. A set-top box excluding the audio output unit 185, a DVD player, a Blu-ray player, a game machine, a computer, and the like may be further exemplified.

4 is an internal block diagram of the control unit shown in FIG. Referring to FIGS. 4 and 3, the following will be described.

The controller 170 according to an exemplary embodiment of the present invention includes a demultiplexing unit 310, an image processing unit 320, an OSD generating unit 340, a mixer 350, a frame rate converting unit 355, 360). A voice processing unit (not shown), and a data processing unit (not shown).

The demultiplexer 310 demultiplexes the input stream. For example, when an MPEG-2 TS is input, it can be demultiplexed into video, audio, and data signals, respectively. The stream signal input to the demultiplexer 310 may be a stream signal output from the tuner 110 or the demodulator 120 or the external device interface 135.

The image processing unit 320 may perform image processing of the demultiplexed image signal. For this, the image processing unit 320 may include a video decoder 325 and a scaler 335.

The video decoder 325 decodes the demultiplexed video signal and the scaler 335 performs scaling so that the resolution of the decoded video signal can be output from the display unit 180.

The video decoder 325 can include a decoder of various standards.

For example, if the demultiplexed video signal is an MPEG-2 standard encoded video signal, it can be decoded by an MPEG-2 decoder.

Also, for example, if the demultiplexed video signal is an encoded video signal of the H.264 standard according to the DMB (Digital Multimedia Broadcasting) scheme or DVB-H, it can be decoded by the H.264 decoder.

Meanwhile, the video signal decoded by the video processor 320 is input to the mixer 350.

The OSD generation unit 340 generates an OSD signal according to a user input or by itself. For example, based on a control signal from the user input interface unit 150, a signal for displaying various kinds of information in a graphic or text format can be generated on the screen of the display unit 180. The generated OSD signal may include various data such as a user interface screen of the multimedia device 100, various menu screens, a widget, and an icon.

For example, the OSD generating unit 340 may generate a signal for displaying broadcast information based on a caption of a broadcast image or an EPG.

The mixer 350 may mix the OSD signal generated by the OSD generator 340 and the decoded video signal processed by the image processor 220. The mixed signal is provided to formatter 360. Since the decoded broadcast video signal or the external input signal and the OSD signal are mixed, the OSD can be overlaid on the broadcast image or the external input image.

A frame rate converter (FRC) 355 can convert a frame rate of an input image. For example, a frame rate of 60 Hz is converted to 120 Hz or 240 Hz. When converting the frame rate of 60 Hz to 120 Hz, it is possible to insert the same first frame between the first frame and the second frame, or insert the third frame predicted from the first frame and the second frame. When converting a frame rate of 60 Hz to 240 Hz, it is possible to insert three more identical frames or insert three predicted frames. On the other hand, it is also possible to maintain the input frame rate without performing another conversion.

The formatter 360 receives the output signal of the frame rate conversion unit 355 and changes the format of the signal to be suitable for the display unit 180 and outputs the signal. For example, the R, G, and B data signals may be output as low voltage differential signaling (LVDS) signals or mini-LVDS signals.

On the other hand, the audio processing unit (not shown) in the control unit 170 can perform the audio processing of the demultiplexed audio signal. To this end, the voice processing unit (not shown) may include various decoders.

When a demultiplexed speech signal is a coded speech signal, the speech processing unit (not shown) in the control unit 170 can decode the demultiplexed speech signal. For example, the demultiplexed speech signal may be decoded by an MPEG-2 decoder, an MPEG 4 decoder, an AAC decoder, or an AC-3 decoder.

In addition, the audio processing unit (not shown) in the control unit 170 can process a base, a treble, a volume control, and the like.

The data processing unit (not shown) in the control unit 170 can perform data processing of the demultiplexed data signal. For example, if the demultiplexed data signal is a coded data signal, it can be decoded. The encoded data signal may be EPG (Electronic Program Guide) information including broadcast information such as a start time and an end time of a broadcast program broadcasted on each channel. For example, the EPG information may be ATSC-PSIP (ATSC-Program and System Information Protocol) information in the case of the ATSC scheme and may include DVB-SI information in the DVB scheme .

The ATSC-PSIP information or the DVB-SI information may be information included in the above-described stream, that is, the header (4 bytes) of the MPEG-2 TS.

Meanwhile, since the block diagram of the controller 170 shown in FIG. 4 is a block diagram for one embodiment of the present invention, it is also possible to add other modules according to needs of those skilled in the art, or omit some of the modules shown.

5 illustrates a user and multimedia device in accordance with an embodiment of the present invention.

According to the embodiment, the multimedia device 1900 acquires the user image through the camera unit 1901 of the multimedia device to capture the image of the user 1903.

In addition, the camera unit may include two image sensor modules for acquiring different information from each other in order to accurately recognize distance information of a user. That is, according to an embodiment, the camera unit may include a depth image sensor, and an RGB image sensor.

As shown in the drawing, the camera unit 1901 of the multimedia device 1900 is located at the lower end of the multimedia device, and is easy to detect the user's body center, and is relatively less influenced in the illumination environment. Bar, smooth gesture recognition, and user image acquisition.

In addition, the multimedia device 1900 may include a voice input unit 1902 for receiving user voice.

The voice input unit 1902 can receive a predetermined voice from the user. That is, the voice input unit 1902 may include a microphone. In addition, the input voice may be transmitted to the controller of the multimedia device to perform an operation for voice recognition.

Through the above-described configuration, the multimedia device can acquire information on the video and audio of the user and perform the control operation of the multimedia device.

6 is a view illustrating a display screen including a menu for acquiring a user image according to an embodiment of the present invention.

According to an embodiment, the multimedia device may include an image 2401 of a depth image sensor that recognizes skeleton information of a user according to an embodiment of the present invention, and a portion that can recognize the user based on the image of the depth image sensor An enlarged image 2405 obtained by zooming through the RGB image sensor can be displayed.

Referring to FIG. 6, the multimedia device can acquire coordinate information for each part of the user's body through the image 2401 captured through the depth image sensor.

That is, for example, coordinate information for only the right elbow 2402 of the user's body can be obtained. In addition, when the user can identify the user through the chest 2403 of the body, it is possible to acquire the coordinate information of the body part and enlarge the area where the body part is located through the RGB image sensor To obtain a more detailed enlarged image 2405. In addition, when the user moves the body and performs a specific action, it can detect the movement of the body skeleton information and recognize what kind of action it takes.

7 is an internal block diagram of a remote controller for controlling any one of multimedia devices according to embodiments of the present invention.

The remote control device 200 includes a wireless communication unit 225, a user input unit 235, a sensor unit 240, an output unit 250, a power supply unit 260, a storage unit 270, And may include a control unit 280.

The wireless communication unit 225 transmits / receives a signal to / from any one of the multimedia devices according to the embodiments of the present invention described above. Among the multimedia devices according to the embodiments of the present invention, one multimedia device 100 will be described as an example.

In this embodiment, the remote control apparatus 200 may include an RF module 221 capable of transmitting and receiving signals with the multimedia device 100 according to the RF communication standard. In addition, the remote control apparatus 200 may include an IR module 223 capable of transmitting and receiving signals with the multimedia device 100 according to the IR communication standard.

In the present embodiment, the remote control device 200 transmits a signal containing information on the motion of the remote control device 200 to the multimedia device 100 through the RF module 221.

Also, the remote control device 200 can receive the signal transmitted by the multimedia device 100 through the RF module 221. [ The remote control device 200 can transmit a command regarding power on / off, channel change, volume change, and the like to the multimedia device 100 through the IR module 223 as needed.

The user input unit 235 may include a keypad, a button, a touch pad, or a touch screen. The user can input a command related to the multimedia device 100 to the remote control device 200 by operating the user input unit 235. [ When the user input unit 235 has a hard key button, the user can input a command related to the multimedia device 100 to the remote control device 200 through the push operation of the hard key button. When the user input unit 235 has a touch screen, the user can touch a soft key of the touch screen to input a command related to the multimedia device 100 to the remote control device 200. [ In addition, the user input unit 235 may include various types of input means such as a scroll key, a jog key, and the like that can be operated by the user, and the present invention does not limit the scope of the present invention.

The sensor unit 240 may include a gyro sensor 241 or an acceleration sensor 243.

The gyro sensor 241 can sense information on the motion of the remote control device 200. [

For example, the gyro sensor 241 can sense information about the operation of the remote control device 200 based on the x, y, and z axes. The acceleration sensor 243 can sense information on the moving speed of the remote control device 200 and the like. Meanwhile, a distance measuring sensor may be additionally provided, so that the distance to the display unit 180 can be sensed.

The output unit 250 may output an image or voice signal corresponding to the operation of the user input unit 235 or corresponding to the signal transmitted from the multimedia device 100. The user can recognize whether the user input unit 235 is operated or whether the multimedia device 100 is controlled through the output unit 250.

For example, the output unit 250 may include an LED module 251 that is turned on when the user input unit 235 is operated or a signal is transmitted to or received from the multimedia device 100 through the wireless communication unit 225, a vibration module 253 An audio output module 255 for outputting sound, or a display module 257 for outputting an image.

The power supply unit 260 supplies power to the remote control device 200. The power supply unit 260 can reduce the power consumption by stopping the power supply when the remote controller 200 is not moving for a predetermined time. The power supply unit 260 may resume power supply when a predetermined key provided in the remote control device 200 is operated.

The storage unit 270 may store various types of programs, application data, and the like necessary for the control or operation of the remote control apparatus 200. [ If the remote control device 200 wirelessly transmits / receives a signal through the multimedia device 100 and the RF module 221, the remote control device 200 and the multimedia device 100 transmit / receive signals through a predetermined frequency band . The control unit 280 of the remote control device 200 stores information on the frequency band and the like capable of wirelessly transmitting and receiving signals with the multimedia device 100 paired with the remote control device 200 in the storage unit 270, can do.

The control unit 280 controls various matters related to the control of the remote control device 200. [ The control unit 280 transmits a signal corresponding to a predetermined key operation of the user input unit 235 or a signal corresponding to the motion of the remote control device 200 sensed by the sensor unit 240 to the multimedia device 100).

Further, the remote control device 200 may include an audio input unit 236. [

The voice input unit 236 can receive a predetermined voice from the user. That is, the voice input unit 236 may include a microphone. In addition, the input voice may be transmitted to the multimedia device 100 to perform an operation for voice recognition in the multimedia device.

FIG. 8 is a diagram illustrating a first embodiment of a user interface provided by any one of the multimedia devices according to the embodiments of the present invention.

8 illustrates that a web screen is displayed on the display unit of the multimedia device.

8 (a) illustrates that a searchable predetermined web screen 710 including a search window 720 is displayed on the display unit 180. In FIG. The user can use a character key (not shown) provided on a keypad (not shown) or a local key (not shown) displayed on the screen or a character key (not shown) 720, it is possible to input a predetermined character.

Next, FIG. 8B illustrates that the search result screen 730 according to the search word input in the search window is displayed on the display unit 180. [0156] FIG. Since the multimedia device according to the embodiment of the present invention can perform full browsing in displaying a web screen, the user can easily grasp the web screen.

That is, the multimedia device can obtain a search result related to a search term input by a user through a display screen as shown in FIG.

Also, in connection with the embodiment of the present invention, the search term may be a predetermined keyword included in the chat message input by the user.

9 is a diagram illustrating a second embodiment of a user interface provided by any one of the multimedia devices according to the embodiments of the present invention.

9 illustrates that the web screen is displayed on the display unit of the multimedia device.

9A illustrates that a mail service screen 810 having an ID input window 820 and a password input window 825 is displayed on the display unit 180. [ The user can input an ID input window (not shown) using a character key (not shown) provided on a keypad (not shown) or a local key (not shown) displayed on the screen or a character key It is possible to input a predetermined character into the password input window 820 and the password input window 825. As a result, the user can log in to the corresponding mail service.

Next, FIG. 9B illustrates that a screen 830 after login to the mail service is displayed on the display unit 180. FIG. For example, items such as 'Read a letter', 'Write a letter', 'Sent', 'Inbox', and 'Trash' items can be illustrated. In addition, the 'Inbox' item is classified into 'From' item and 'Title' item, and the mail can be sorted and displayed.

The multimedia device according to the embodiment of the present invention can perform full browsing when displaying the mail service screen, so that the user can easily use the mail service.

That is, the multimedia device can use various services such as a mail service through a display screen as shown in Fig.

10 is a diagram illustrating a display screen for providing a chat service in a multimedia device according to the related art.

According to the related art, the multimedia device transmits and receives data to and from a multimedia device connected to the network and provides a chatting service to the user. The multimedia device includes a video chat mode for transmitting a real-time user video through a video, A chat service can be provided through a still image chat mode in which a chat message is transmitted through a video.

10A, the multimedia device displays a video chat window 900 and displays the real time image 901 of the other party in conversation via the chat service and the multimedia device through the video chat window 900 It is possible to display the user's own image 902 including the chat service.

That is, the multimedia device continuously captures the user image through the camera unit of the multimedia device, and transmits the captured image to another multimedia device through the network, thereby providing a real-time image of each user participating in the chat service to the user .

However, in the case of providing a chat service using the video chat window 900 as shown in FIG. 10 (a), in a region where a network having a high data transmission rate is not installed, A delay may occur in the user image 901 displayed in the video chat window 900 and the user may not be able to use the smooth chat service due to the occurrence of a hang-up phenomenon. In addition, when the data transmission amount of the network network provided with the chat service is increased, the transmission speed is lowered, and the above-described problems may occur.

10 (b), the multimedia device displays a text chat window 910, and displays still images 911 and 912 of the other party in conversation via the chat service in the text chat window 910, Including a chat message.

That is, when the user uses the chat service, the multimedia device may photograph the still image of the user through the camera unit of the multimedia device, or may search the user still image pre-stored in the multimedia device, and transmit the user image to another multimedia device Thereby providing a user with a user image that can identify each user participating in the chat service.

However, when providing a chat service through the text chat window 910 as shown in FIG. 10 (b), it is difficult to grasp the accurate user intention through only the stopped user image and text, A problem may occur that it can not be performed.

11 is a flowchart illustrating a method of inputting a chat message including a user image according to an embodiment of the present invention.

According to an embodiment, the multimedia device may display a chat window containing a chat message and a user image, update the user image contained in the chat window in response to a capture condition, A chat service can be provided.

To this end, the multimedia device starts shooting the user's image (S101).

When a chat service is started and a chat window is displayed, the multimedia device captures a user image through a camera unit provided in the multimedia device.

In addition, the camera unit may include a depth image sensor capable of acquiring distance information of a user, and an RGB image sensor capable of acquiring color information of a user.

Accordingly, the multimedia device can obtain coordinate information of each part of the user's body, or coordinate information of each part of the user's face through the camera unit.

In addition, the multimedia device may display a selection menu for selecting a mode of a chat service as shown in FIG. 14, and may display a user image according to a specific condition according to an embodiment of the present invention Only when the menu for capturing and updating the user image is selected, the user can start shooting the image through the camera unit.

Next, the multimedia device detects the user status (S102).

According to the embodiment, the multimedia apparatus can sense what kind of action the user who is photographed through the camera unit performs or what kind of facial expression it makes.

For example, when the coordinate or skeleton information of each part of the user's body is acquired through the camera unit, the multimedia device tracks the movement of the coordinates and detects what action the user takes through the camera unit .

In addition, for example, when the coordinate information of the eyeball portion of the face of the user is obtained through the camera unit, the multimedia device tracks the movement of the coordinates and displays the face of the user photographed through the camera unit Can be detected.

Next, the multimedia device detects whether a capture condition pre-stored in the multimedia device occurs (S103).

The multimedia device may store capture conditions for a user image captured through the camera unit.

The capture condition may include a user ' s specific action, or a particular facial expression.

That is, the multimedia device stores information on whether a user performs a certain facial expression or when a user performs an operation, and the user senses the user through the information detected in the user state sensing step (S102) It is possible to search whether the operation corresponding to the capture condition is performed or the facial expression corresponding to the capture condition is formed.

Next, when the capture condition occurs, the multimedia device captures a still image of the user image captured through the camera unit of the multimedia device (S104).

That is, when a capture condition occurs, the multimedia device can capture an image photographed through the camera unit and store the captured image as a captured image.

On the other hand, the multimedia device continuously detects the user status when the capture condition does not occur.

Next, the multimedia device updates the user image included in the chat window displayed on the display unit with the captured image (S105).

According to an embodiment, the multimedia device may acquire, from a user image captured through a camera portion, a captured image including a user expression corresponding to a capture condition, or a user image corresponding to a capture condition, And transmit the acquired captured image to other users participating in the chat service through the network, thereby updating the user image.

Through the above steps, the multimedia device captures and transmits the user image only when a specific capture condition occurs, thereby reducing the load on data transmission / reception in the network, and facilitating smooth communication among users.

12 is a flowchart illustrating a method of inputting a chat message including a keyword search result according to an embodiment of the present invention.

According to an embodiment of the present invention, when displaying a chat window including a chat message, the multimedia device may display a search result using a keyword included in the chat message as a search word, and provide the result to a user.

To this end, the multimedia device receives a text to be displayed as a chat message from a user (S201).

The multimedia device can receive text through the user interface unit. Also, as a user input device (UID) for transmitting a predetermined text input signal to the user interface, a touch screen input such as a remote keyboard, a remote controller, a space remote controller, an OSD (On Screen Display) Various input means can be used. Also, text can be received by converting the user's voice into text through the voice recognition unit of the multimedia device.

In addition, the multimedia device displays the input text as a chat message on the display unit, and transmits the input text to other chat service participants through the network.

Next, keywords included in the chat message are extracted (S202).

That is, when a chat message is input, the multimedia device extracts texts included in the chat message, determines importance of each extracted text, and extracts a text having a high degree of importance as a keyword.

To this end, the multimedia device may store a database that stores a list of important texts.

Next, information related to the extracted keyword is retrieved (S203).

According to an embodiment, the multimedia device may extract a keyword and perform a search process using the extracted keyword as a search word.

For example, the multimedia device stores a list of contents or services available in the multimedia device, and, when the keyword is extracted, searches the content or service corresponding to the extracted keyword.

In another example, when the keyword is extracted, the multimedia device may transmit a search request using the extracted keyword as a search word to the search server through the network, and receive the search result from the search server.

Next, the search result according to the search is displayed (S204).

According to an embodiment, the multimedia device may display a search result along with a chat window for providing a chat service.

In addition, the search result may be displayed in a semi-transparent color, or may be minimized and displayed in a predetermined area where the chat message of the display screen is not displayed, so as not to interfere with the chat message of the chat service in progress.

Accordingly, the user can simultaneously check the search result related to the chat message through the steps described above, thereby enhancing user convenience in the process of using the chat service.

13 is a diagram illustrating a chat message input method according to an embodiment of the present invention.

According to an embodiment, the multimedia device may execute commands contained in the user image and update the image information window displayed as a user image.

The above embodiment will be described in detail as follows.

First, the multimedia apparatus recognizes the user image through the camera unit of the multimedia apparatus (S305).

According to an embodiment, the camera unit may include a plurality of image sensors capable of acquiring different information. That is, for example, a depth image sensor and an RGB image sensor may be used to obtain distance information and color information for a user in a user image.

Accordingly, the multimedia device can acquire various information by photographing the user image.

Next, the multimedia device determines whether the text included in the chat message or the user image recognized in step S301 includes a command for executing a specific function of the multimedia device (S302).

According to an embodiment, the multimedia device may execute certain functions of the multimedia device through text input or user gesture input.

Accordingly, the multimedia device can determine whether the text message or the user gesture storing the mapping information related to the execution of the specific function is included in the chat message input in order to use the chat service or the user image captured in the camera unit have.

If the text or the user image recognized in step S301 includes an instruction to execute a specific function of the multimedia device, the multimedia device performs an operation corresponding to the included instruction (S303).

Accordingly, when the text message or the user gesture for executing a specific command is detected even when a chat message for using a chat service is input or a user image is being shot, the multimedia apparatus switches the display screen to execute the specific command , And can execute the specific command using multi-tasking.

On the other hand, if the text or the instruction for executing the specific function of the multimedia device is not included in the user image recognized in the step S302, the user image recognized in the step S301 may be associated with the capture condition It is determined whether the user's action or facial expression is included (S304).

That is, as described with reference to FIG. 11, the multimedia device can set a specific capture condition for the user image, so that the multimedia device determines whether a state corresponding to the predetermined capture condition is detected in the captured user image .

Next, if the user image includes a user's action or facial expression corresponding to the capture condition, the multimedia device captures the user image and updates the user image information window (S306).

That is, according to the embodiment, the multimedia device captures a user image every time a capture condition occurs, acquires a new user image, and transmits the obtained image to another user participating in the chat service through the network, You can update the embedded user image information window.

On the other hand, if the user image does not include the user's action or facial expression corresponding to the capture condition, it can be determined whether a predetermined time has passed after the image of the user image information window displayed in the chat window is updated (S308). If the predetermined time has elapsed, the user image is captured and the user image information window is updated (S306).

That is, even when the state corresponding to the capture condition does not occur, the multimedia device can update the user image when a predetermined time elapses, thereby providing a new user image.

The predetermined time may be a time set at the time of manufacturing the multimedia device or a time changed according to user setting.

On the other hand, if the predetermined time has not elapsed since the image of the user image information window displayed in the chat window is updated, the multimedia device continuously recognizes the user image.

14 is a view illustrating a display screen including a chat mode setting menu according to an embodiment of the present invention.

According to the embodiment, when the multimedia service can provide the chat service through the network to the user, the multimedia device can provide different display screens to the user according to the mode of the chat service.

That is, the multimedia device includes a first mode 1002 for providing a user image through a still image, a second mode 1003 for providing a user image to be updated according to a predetermined condition, 3 mode 1004 to provide a chat service.

Accordingly, the multimedia device may display a mode selection menu 1001 including predetermined mode options 1002, 1003, and 1004. [

When the user selects the first mode 1002 through the predetermined pointer 205, the multimedia device displays a chat service using a still image to the user through the same display screen as the chat window shown in FIG. 10B When the user selects the third mode 1004, the multimedia device can provide a chat service using the moving picture to the user through the same display screen as the chatting window shown in FIG. 10 (a).

Also, when the user selects the second mode 1003 through the pointer 205, the chat service can be provided using the user image updated according to the predetermined condition according to the above FIG. 11 to FIG. One embodiment of the display screen of the chat window provided according to the second mode may be described with reference to FIGS. 15 and 16. FIG.

15 is a view illustrating a display screen including a chat window in which a user image is displayed according to an embodiment of the present invention.

According to an embodiment, when providing a chat service through a network to a user, the multimedia device may display user images 1101, 1102, 1103, and 1104 corresponding to chat messages input by the user.

Accordingly, the multimedia device may track changes in the operation or facial expression of the user using the chat service through the multimedia device through the camera unit of the multimedia device, The multimedia device can capture the user's image and display it in the chat window 110 together with the chat message input from the user.

15, the multimedia device displays a chat window 1100 for providing a chat service to a user, detects a user's action or facial expression through a camera unit of the multimedia device in a state of displaying a chat message, do. In this case, the multimedia device can acquire coordinate information of each part of the user's body or each part of the face of the user, detects the movement form of the coordinates, and can grasp what action the user performs or what kind of facial expression .

Accordingly, when the operation of shaking the hand of the first user participating in the chat service is detected during the chat service, the multimedia device captures an image including a shaking motion of the first user, And display the captured image 1101 together with a chat message 1105. [

In addition, the second user participating in the chat service displays the base image 1102 of the second user and the chat message 1106 when no action or facial expression corresponding to the capture condition is detected.

In addition, when an operation of raising the fist of the third user participating in the chat service is detected, the multimedia device captures an image including the operation of raising the third user's fist, and the chat message 1107 of the third user, The capture image 1103 can be displayed together with the captured image 1103.

In addition, if a winking facial expression of a fourth user participating in the chat service is detected, the multimedia device captures an image including a winking expression of the fourth user, and, together with the chat message 1108 of the fourth user, A captured image 1108 can be displayed.

Also, according to the embodiment, when the new chat message is input or a new user image is captured, the user images 1101, 1102, 1103, and 1104 and the chat messages 1105, 1106, 1107, The user can replace the new chat message or the new user image or only the chat message or the user image displayed while the position of the user image and the chat message is fixed.

FIG. 16 is a diagram illustrating a display screen including a chat window 1200 in which a keyword search result according to an embodiment of the present invention is displayed.

According to the embodiment, when the multimedia device provides the chat service through the network to the user, the multimedia device can display the search result using the keyword 1201 included in the user's chat message as the search term.

The multimedia device displays each chat message of users using the chat service, and detects whether or not the keyword is included in the chat message.

Accordingly, words included in each chat message are analyzed, and words corresponding to the keywords of the analyzed words are extracted based on the keyword information stored in the multimedia device. The keyword information may include, for example, statistical data on words frequently input to the Internet search engine.

Also, when the keyword 1201 is detected in the chat message of a specific user among the users using the chat service, the detected keyword 1201 may be underlined or displayed in bold.

That is, as shown in FIG. Quot; Minority Report " is detected as a keyword, the multimedia device can display the 'Minority Report' underlined and displayed.

In addition, the multimedia device may perform a search process using the detected keyword as a search word. That is, the multimedia device can search contents or services corresponding to the detected keyword in the multimedia device, or search for contents or services connected through the network.

Referring to FIG. 16, the multimedia device may display a search result window 1203 including a result obtained through a search engine connected to a network, using a 'minority report' as a search term. In addition, the user image 1202 included in the chat window 1200 can be changed to an image corresponding to the search word.

The search results window 1203 may include shortcut items 1204 of services or content available through the network.

That is, since the multimedia device can acquire link information of an available service or contents through a search engine connected to the network, shortcut items 1204 are included in the search result window 1203 using the link information .

Therefore, as shown in FIG. 16, the user can browse the 'Minority Report Wallpaper' associated with the keyword 'Minority Report', move to the service page for 'Minority Report Pre-purchase' or go to the service page for 'Minority Report Down' Or a shortcut page for moving to a service page for renting a 'minority report' or moving to a web page for purchasing a 'Minority Report DVD'. Accordingly, the user can immediately use the selected service or content by selecting any one of the shortcut items.

In addition, according to the embodiment, the display screen according to the selected service or content can be minimized and displayed in a certain area or displayed in a translucent color in order not to interfere with the display screen of the chat message.

16, the multimedia device can search contents or services stored in the multimedia device, and display the search results. In addition, the user can select the content or service included in the search result, and immediately use the selected content or service.

17 is a diagram illustrating a database for storing capture conditions of a multimedia device according to an embodiment of the present invention.

According to the embodiment, when the user uses the chat service through the multimedia device, the multimedia device tracks the user's facial expression or operation through the camera unit of the multimedia device, and when the specific facial expression or operation is detected, And can provide the captured image to other users. Thus, the multimedia device may store a database 1300 that includes mapping information for capture conditions 1301 for capturing user images.

That is, the database 1300 may include mapping information for the capture region 1302 of the user image according to the capture condition 1301.

According to an embodiment, the database 1300 may recognize (1303) a facial expression of a user using a chat service, and recognize (1304) an operation to detect a capture condition.

For example, the multimedia device recognizes the user's facial expression, and when the user makes a smiling face, makes a crying expression, or makes an angry expression, the multimedia device displays an image of the user's face through the camera unit of the multimedia device You can capture.

That is, for this purpose, the camera unit of the multimedia device acquires position coordinates of the user's eyes, nose, mouth, and the like, tracks whether the obtained coordinates are moved, The multimedia device may include a face portion of a user who builds each of the facial expressions to another user who is using the chat service when the user constructs the facial expressions Can be captured and provided.

Further, for example, the multimedia device recognizes the motion of the user, captures an image including the upper half of the user when the user takes a motion of waving a hand or a motion of fist, When taking an operation of lifting both hands, an image including the whole body of the user can be captured.

That is, for this purpose, the camera unit of the multimedia device acquires position coordinates of the user's arms, legs, shoulders, etc., and tracks movement of the obtained position coordinates, So that the multimedia device can provide the captured image to other users who are using the chat service, which may include the operation of the user.

Also, for example, the multimedia device may capture a user image when a predetermined time or more has elapsed without an operation or a facial expression corresponding to the capture conditions being retrieved. For example, as shown in FIG. 17, when the user has not faced or operated for more than 5 minutes, the image including the face portion of the user is captured and the latest user image is transmitted to the chat service To other users in use.

Also, the mapping data included in the database shown in FIG. 17 is only an example, and the mapping data may be stored differently according to the multimedia manufacturer, and the capture condition or capture areas may be edited by the user.

The multimedia device and its operation method according to the present invention are not limited to the configuration and method of the embodiments described above but the embodiments may be modified so that all or part of each embodiment Or may be selectively combined.

Meanwhile, the method of operating the multimedia device of the present invention can be implemented as a code that can be read by a processor on a recording medium readable by a processor included in the multimedia device. The processor-readable recording medium includes all kinds of recording apparatuses in which data that can be read by the processor is stored. Examples of the recording medium that can be read by the processor include a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like, and may also be implemented in the form of a carrier wave such as transmission over the Internet . In addition, the processor-readable recording medium may be distributed over network-connected computer systems so that code readable by the processor in a distributed fashion can be stored and executed.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, It will be understood by those skilled in the art that various changes and modifications may be made without departing from the spirit and scope of the present invention.

100: Multimedia device
105: Broadcast receiver
110: tuner
120: demodulator
130: Network interface unit
135: External device interface unit
140:
150:
170:
180:
185: Audio output section
190: Power supply
200: remote control device

Claims (16)

delete delete delete delete delete delete delete delete A multimedia device capable of providing a chat service,
A network interface unit for transmitting and receiving data through a network;
A camera unit for photographing a user image;
A storage unit for storing a capture condition for the user image;
A display unit for displaying a chat message and a user image of a user participating in the chat service; And
Loading the capture condition in the storage unit, capturing a user image captured by the camera unit based on the loaded capture condition, updating the displayed user image with the captured image And a controller for scrolling the updated user image to the upper portion of the display screen,
Wherein,
Obtaining coordinates information of each part of the user's body part and face part in the face area in the user image, tracking the movement of the obtained coordinates, and displaying the coordinates corresponding to the specific operation or the specific expression And to capture the captured image when the tracked coordinate information coincides with at least one coordinate information stored in the storage unit. delete delete 10. The method of claim 9,
Wherein,
Extracting a keyword from the displayed chat message, transmitting a search request using the extracted keyword as a search word to the search server through the network interface unit, receiving the search result from the search server, To be displayed on the display unit
Multimedia device capable of providing chat service. 10. The method of claim 9,
Wherein,
Storing data on contents or services available in the multimedia device,
Wherein,
Extracting a keyword from the displayed chat message, searching the content or service corresponding to the extracted keyword in the storage unit, and controlling the display unit to display the search result
Multimedia device capable of providing chat service. 10. The method of claim 9,
Wherein,
When the state corresponding to the capturing condition is not sensed for a preset time in the user image captured by the camera unit, control is performed to capture the captured user image
Multimedia device capable of providing chat service. 10. The method of claim 9,
Wherein,
And control to crop the captured image based on the capture condition
Multimedia device capable of providing chat service. 10. The method of claim 9,
The camera unit includes:
A depth image sensor for acquiring distance information of a user as an image; And an RGB image sensor for acquiring color information of the user as an image
Multimedia device capable of providing chat service.

Images