KR102224484B1 - Multimedia deivice and method for controlling the same - Google Patents

Abstract

Disclosed are a multimedia device and a method for controlling the same.
A multimedia device according to an embodiment of the present invention includes a wireless communication unit wirelessly connected to a remote controller including a first microphone and a second microphone, a display unit outputting video data of a specific content, and audio of the specific content. An audio output unit for outputting data, a camera, and a control unit, wherein the control unit controls the first microphone to receive an external sound signal and convert it into electrical first audio data, and an external device from the remote controller When controlling the wireless communication unit to receive second audio data corresponding to the sound signal of, and controlling the camera to acquire an image, and when the value of the first audio data and the value of the second audio data are different, the Controls a preset specific function based on the acquired image.

Description

Multimedia device and its control method {MULTIMEDIA DEIVICE AND METHOD FOR CONTROLLING THE SAME}

The present invention relates to a multimedia device and a control method thereof.

Recently, TVs are playing a role as a multimedia device to which Internet functions and smart functions have been added beyond the broadcasting reception function. Accordingly, a voice recognition function is implemented on the TV so that the user can control each function of the TV by uttering a command with a voice, and a face recognition function is implemented on the TV to recognize a specific user and create a user environment customized for a specific user. It is also possible to provide. In addition, when a voice recognition function is implemented in a TV, in order to increase the voice recognition rate and emphasize user convenience, most of the remote controllers for controlling the TV have a built-in microphone. Accordingly, when the value of the audio data acquired through the microphone built in the TV and the audio data acquired through the microphone built in the remote controller are different, it is a question of which value to control the function of the TV based on.

On the other hand, the TV is used by several users due to its characteristics. In this case, a question is given to which user is given priority and how to control the function of the TV according to the intention of the plurality of users.

It is an object of the present invention to solve the above and other problems.

In addition, according to an embodiment of the present invention, when a value of audio data acquired through a microphone built in the multimedia device and a value of audio data acquired through a microphone built in a remote controller are different, the multimedia device may be It aims to provide a solution as to whether or not to control the function.

In addition, an embodiment of the present invention provides a solution for which user priority is given when multiple users use the multimedia device and how to control the function of the multimedia device according to the intention of multiple users. It is aimed at.

The technical problems to be achieved in the present invention are not limited to the technical problems mentioned above, and other technical problems that are not mentioned will be clearly understood by those of ordinary skill in the technical field to which the present invention belongs from the following description. I will be able to.

In order to achieve the above or other objects, according to an aspect of the present invention, a multimedia device includes a wireless communication unit wirelessly connected to a remote controller including a first microphone and a second microphone, and a display that outputs video data of a specific content. A unit, an audio output unit for outputting audio data of the specific content, a camera, and a control unit, wherein the control unit receives an external sound signal and converts the first microphone into electrical first audio data. And controlling the wireless communication unit to receive second audio data corresponding to an external sound signal from the remote controller, controlling the camera to acquire an image, and controlling the value of the first audio data and the second audio When the data values are different, a preset specific function is controlled based on the acquired image.

In order to achieve the above or other objects, according to an aspect of the present invention, a method of controlling a multimedia device includes outputting video data and audio data of a specific content, and receiving an external sound signal to provide electrical first audio data. Converting to, receiving second audio data corresponding to an external sound signal from a remote controller, acquiring an image from a camera, a value of the first audio data and a value of the second audio data And adjusting the volume of audio data of the specific content to be output based on the obtained image when the sizes of are different.

The effects of the multimedia device and its control method according to the present invention will be described as follows.

According to at least one of the embodiments of the present invention, when a value of audio data acquired through a microphone embedded in a multimedia device and a value of audio data acquired through a microphone embedded in a remote controller are different, the multimedia It can provide a solution as to whether to control the function of the device.

In addition, according to at least one of the embodiments of the present invention, when a multimedia device is used by multiple users, which user is given priority and how to control the function of the multimedia device according to the intention of the plurality of users. Solutions can be provided.

Further scope of the applicability of the present invention will become apparent from the detailed description below. However, since various changes and modifications within the spirit and scope of the present invention can be clearly understood by those skilled in the art, specific embodiments such as the detailed description and preferred embodiments of the present invention should be understood as being given by way of example only.

1 is a block diagram illustrating an example of a multimedia device according to an embodiment of the present invention.
2 is a block diagram illustrating a detailed configuration of a control unit of FIG. 1 according to an embodiment of the present invention.
3 is a diagram showing examples of a remote controller for controlling a multimedia device according to an embodiment of the present invention.
4 is a block diagram showing configuration modules of a remote controller according to an embodiment of the present invention.
5 is a flowchart illustrating an example of a method of controlling a multimedia device according to an embodiment of the present invention.
6 is a diagram illustrating an example of a method of controlling a multimedia device according to an embodiment of the present invention.
7 is a diagram illustrating another example of a method of controlling a multimedia device according to an embodiment of the present invention.
8 is a diagram for explaining another example of a method of controlling a multimedia device according to an embodiment of the present invention.
9 is a diagram illustrating an example of a method of controlling a multimedia device according to an embodiment of the present invention.
10 is a diagram illustrating an example of a method of controlling a multimedia device according to an embodiment of the present invention.
11 is a diagram illustrating an example of a method of controlling a multimedia device according to an embodiment of the present invention.
12 is a diagram illustrating an example of a method of controlling a multimedia device according to an embodiment of the present invention.
13 is a diagram illustrating an example of a method of controlling a multimedia device according to an embodiment of the present invention.
14 is a diagram illustrating an example of a method of controlling a multimedia device according to an embodiment of the present invention.
15 is a diagram illustrating an example of a method of controlling a multimedia device according to an embodiment of the present invention.
16 is a diagram illustrating an example of a method of controlling a multimedia device according to an embodiment of the present invention.
17 is a diagram illustrating an example of a method of controlling a multimedia device according to an embodiment of the present invention.
18 is a diagram illustrating another example of a method of controlling a multimedia device according to an embodiment of the present invention.
19 is a diagram for explaining another example of a method of controlling a multimedia device according to an embodiment of the present invention.
20 is a diagram illustrating a process of registering a user in a multimedia device according to an embodiment of the present invention.
21 is a diagram illustrating a process of registering another user by a master user in a multimedia device according to an embodiment of the present invention.
22 is a diagram for explaining a difference in authority between a user registered in a multimedia device and an unregistered user according to an embodiment of the present invention.
23 is a diagram illustrating a process of registering another user by a master user in a multimedia device according to an embodiment of the present invention.
FIG. 24 is a diagram for describing an example in which an automatic volume control function and an automatic brightness control function for a specific user are set in a multimedia device according to an embodiment of the present invention.

Hereinafter, exemplary embodiments disclosed in the present specification will be described in detail with reference to the accompanying drawings, but identical or similar elements are denoted by the same reference numerals regardless of reference numerals, and redundant descriptions thereof will be omitted. The suffixes "module" and "unit" for constituent elements used in the following description are given or used interchangeably in consideration of only the ease of preparation of the specification, and do not have meanings or roles that are distinguished from each other by themselves. In addition, in describing the embodiments disclosed in the present specification, when it is determined that a detailed description of related known technologies may obscure the subject matter of the embodiments disclosed in the present specification, the detailed description thereof will be omitted. In addition, the accompanying drawings are for easy understanding of the embodiments disclosed in the present specification, and the technical idea disclosed in the present specification is not limited by the accompanying drawings, and all changes included in the spirit and scope of the present invention It should be understood to include equivalents or substitutes.

Terms including ordinal numbers such as first and second may be used to describe various elements, but the elements are not limited by the terms. The above terms are used only for the purpose of distinguishing one component from another component.

When a component is referred to as being "connected" or "connected" to another component, it is understood that it may be directly connected or connected to the other component, but other components may exist in the middle. It should be. On the other hand, when a component is referred to as being "directly connected" or "directly connected" to another component, it should be understood that there is no other component in the middle.

Singular expressions include plural expressions unless the context clearly indicates otherwise.

In this application, terms such as "comprises" or "have" are intended to designate the presence of features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, but one or more other features. It is to be understood that the presence or addition of elements or numbers, steps, actions, components, parts, or combinations thereof does not preclude in advance.

1 is a block diagram illustrating an example of a multimedia device according to an embodiment of the present invention.

The multimedia device 100 according to an embodiment of the present invention is, for example, an intelligent network TV in which a computer support function is added to a broadcast reception function, and is a device that is faithful to a broadcast reception function and an Internet function is added. For example, the multimedia device 100 may correspond to network TV, smart TV, Hybrid Brodacast Broadband Television (HBBTV), Internet TV, web TV, Internet Protocol Television (IPTV), and the like.

The multimedia device 100 includes a receiving unit 110, an external device interface unit 120, an input unit 130, a memory 140, a control unit 150, a user interface unit 160, a display unit 170, and an audio output. It may include a unit 180, a power supply unit 190, and the like.

The reception unit 110 may include at least one tuner 111, a demodulation unit 112, and a network interface unit 113. However, in some cases, the receiver 110 may include the tuner 111 and the demodulation unit 112, but may not include the network interface unit 113, and vice versa. In addition, although not shown, the receiving unit 110 includes a multiplexer to multiplex the signal demodulated by the demodulation unit 112 through the tuner 111 and the signal received through the network interface unit 113. You may. In addition, although not shown, the receiving unit 110 may also include a demultiplexer to demultiplex the multiplexed signal or demultiplex the demodulated signal or a signal that has passed through the network interface unit 113. .

The tuner 111 receives an RF broadcast signal by tuning a channel selected by a user or all previously stored channels among radio frequency (RF) broadcast signals received through an antenna. In addition, the tuner 111 converts the received RF broadcast signal into an intermediate frequency (IF) signal or a baseband signal.

For example, if the received RF broadcast signal is a digital broadcast signal, it is converted to a digital IF signal (DIF), and if the received RF broadcast signal is an analog broadcast signal, it is converted to an analog baseband video or audio signal (CVBS/SIF). That is, the tuner 111 may process both digital broadcast signals or analog broadcast signals. The analog baseband video or audio signal (CVBS/SIF) output from the tuner 111 may be directly input to the controller 150.

In addition, the tuner 111 may receive an RF broadcast signal of a single carrier or multiple carriers. On the other hand, the tuner 111 sequentially tunes and receives RF broadcast signals of all broadcast channels stored through a channel memory function among RF broadcast signals received through an antenna, and receives them, and receives them as an intermediate frequency signal or a baseband signal (DIF: Digital Intermediate Signal). Frequency or baseband signal).

The demodulator 112 may receive and demodulate the digital IF signal DIF converted by the tuner 111, and may perform channel decoding or the like. To this end, the demodulation unit 112 includes a Trellis decoder, a de-interleaver, a Reed-Solomon decoder, or the like, or a convolution decoder, a deinterleaver, and a read. -A soloman decoder, etc. can be provided.

The demodulator 112 may output a stream signal TS after performing demodulation and channel decoding. In this case, the stream signal may be a signal in which a video signal, an audio signal, or a data signal is multiplexed. For example, the stream signal may be an MPEG-2 Transport Stream (TS) in which an MPEG-2 standard video signal and a Dolby AC-3 standard audio signal are multiplexed.

The stream signal output from the demodulation unit 112 may be input to the control unit 150. The controller 150 may control demultiplexing, image/audio signal processing, and the like, and may control output of an image through the display unit 170 and an audio output through the audio output unit 180.

The external device interface unit 120 provides an interfacing environment between the multimedia device 100 and various external devices. To this end, the external device interface unit 120 may include an A/V input/output unit (not shown) or a wireless communication unit (not shown).

The external device interface unit 120 includes a digital versatile disk (DVD), a Blu-ray, a game device, a camera, a camcorder, a computer (laptop), a tablet PC, a smart phone, and a Bluetooth device. device) and external devices such as the cloud, etc. may be connected by wire or wirelessly. The external device interface unit 120 transmits a signal including data, such as an image, a video, and an audio input through a connected external device, to the control unit 150 of the multimedia device 100. The controller 150 may control the processed image, video, audio, etc. to output a data signal to a connected external device. To this end, the external device interface unit 120 may further include an A/V input/output unit (not shown) or a wireless communication unit (not shown).

The A/V input/output unit includes a USB terminal, a CVBS (Composite Video Banking Sync) terminal, a component terminal, an S-video terminal (analog), so that video and audio signals from an external device can be input to the multimedia device 100. It may include a Digital Visual Interface (DVI) terminal, a High Definition Multimedia Interface (HDMI) terminal, an RGB terminal, and a D-SUB terminal.

The wireless communication unit may perform wireless communication with other digital devices. The multimedia device 100 includes, for example, Wireless LAN (WLAN), Wireless-Fidelity (Wi-Fi), Wireless Fidelity (Wi-Fi) Direct, Bluetooth, Radio Frequency Identification (RFID), and infrared communication (IrDA). , infrared Data Association), UWB (Ultra Wideband), ZigBee, DLNA (Digital Living Network Alliance), and other digital devices according to communication protocols.

In addition, the external device interface unit 120 may be connected through at least one of the set-top-box STB and the above-described various terminals to perform input/output operations with the set-top-box STB.

Meanwhile, the external device interface unit 120 may receive an application or an application list in an adjacent external device and transmit it to the controller 150 or the memory 140.

The network interface unit 113 provides an interface for connecting the multimedia device 100 to a wired/wireless network including an Internet network. The network interface unit 113 may be provided with, for example, an Ethernet terminal for connection with a wired network, and for connection with a wireless network, for example, WLAN (Wireless LAN) (Wi- Fi), Wireless broadband (Wibro), World Interoperability for Microwave Access (Wimax), and High Speed Downlink Packet Access (HSDPA) communication standards can be used.

The network interface unit 113 may transmit or receive data with other users or other digital devices through a connected network or another network linked to the connected network. Particularly, some content data stored in the multimedia device 100 may be transmitted to another user registered in advance with the multimedia device 100 or to a selected user or a selected digital device among other digital devices.

Meanwhile, the network interface unit 113 may access a predetermined web page through a connected network or another network linked to the connected network. That is, by accessing a predetermined web page through a network, the server and data can be transmitted or received. In addition, content or data provided by a content provider or network operator may be received. That is, content such as movies, advertisements, games, VODs, and broadcast signals provided from a content provider or a network provider may be received through a network, and information related thereto. In addition, it is possible to receive update information and an update file of firmware provided by a network operator. It can also transmit data to the Internet or content provider or network operator.

In addition, the network interface unit 113 may select and receive a desired application from among applications opened through a network.

The input unit 130 is for inputting image information (or signal), audio information (or signal), data, etc., and includes a camera 131 for inputting an image signal and a microphone 132 for inputting an audio signal. Can include.

The camera 131 processes an image frame such as a still image or a video obtained by an image sensor in a video call mode or a photographing mode. The processed image frame may be displayed on the display unit 170 or stored in the memory 140.

The microphone 132 processes an external sound signal into electrical voice data. The processed voice data may be utilized in various ways according to a function (or an application program being executed) being executed by the mobile terminal 100. Meanwhile, the microphone 132 may be implemented with various noise removal algorithms for removing noise generated in the process of receiving an external sound signal.

The memory 140 may store a program for processing and controlling each signal in the controller 150, or may store a signal-processed video, audio, or data signal.

In addition, the memory 140 may perform a function for temporary storage of a video, audio, or data signal input from the external device interface unit 120 or the network interface unit 113. The memory 140 may store information on a predetermined broadcast channel through a channel memory function.

The memory 140 may store an application or an application list input from the external device interface unit 120 or the network interface unit 113.

In addition, the memory 140 may store various platforms, which will be described later.

The memory 140 is, for example, a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (eg, SD or XD memory). Etc.), RAM, and ROM (EEPROM, etc.). The multimedia device 100 may reproduce and provide a content file (movie file, still image file, music file, document file, application file, etc.) stored in the memory 140 to a user. Depending on the embodiment, the memory 140 may be included in the controller 150 and implemented.

The user interface unit 160 transmits a signal input by a user to the controller 150 or transmits a signal from the controller 150 to the user.

For example, the user interface unit 160 receives control signals such as power on/off, channel selection, and screen setting from the remote controller 200 according to various communication methods such as an RF communication method and an infrared (IR) communication method. It may be received and processed, or may be processed to transmit a control signal of the control unit 150 to the remote controller 200.

In addition, the user interface unit 160 may transmit a control signal input from a local key (not shown) such as a power key, a channel key, a volume key, and a set value to the control unit 150.

The user interface unit 160 transmits a control signal input from a sensing unit (not shown) that senses a user's gesture to the controller 150 or transmits a signal from the controller 150 to the sensing unit ( Not shown). Here, the sensing unit (not shown) may include a touch sensor, a voice sensor, a position sensor, a motion sensor, and the like.

The control unit 150 demultiplexes the stream input through the tuner 111, the demodulation unit 112, or the external device interface unit 120 or processes the demultiplexed signals to generate a signal for video or audio output. And can be output.

The image signal processed by the controller 150 may be input to the display unit 170 and displayed as an image corresponding to the corresponding image signal. In addition, the image signal processed by the controller 150 may be input to an external output device through the external device interface unit 120.

The control unit 150 may include a demultiplexer, an image processing unit, and the like.

The controller 150 may control the overall operation of the multimedia device 100. For example, the controller 150 may control the tuner 111 to tune an RF broadcast corresponding to a channel selected by a user or a previously stored channel.

The control unit 150 may control the multimedia device 100 according to a user command or an internal program input through the user interface unit 160. In particular, it is possible to connect to a network to download an application or an application list desired by a user into the multimedia device 100.

For example, the control unit 150 controls the tuner 111 to input a signal of a selected channel according to a predetermined channel selection command received through the user interface unit 160. And it processes video, audio, or data signals of the selected channel. The controller 150 enables channel information selected by the user to be output through the display unit 170 or the audio output unit 180 together with the processed image or audio signal.

As another example, the control unit 150 may be input from an external device input through the external device interface unit 120, for example, from a camera or camcorder, according to an external device image playback command received through the user interface unit 160. , An image signal or an audio signal may be output through the display unit 170 or the audio output unit 180.

Meanwhile, the control unit 150 may control the display unit 170 to display an image. For example, a broadcast image input through the tuner 111, an external input image input through the external device interface unit 120, an image input through the network interface unit 113, or the memory 140 The stored image may be controlled to be displayed on the display unit 170. In this case, the image displayed on the display unit 170 may be a still image or a moving image, and may be a 2D image or a 3D image.

In addition, the controller 150 may control to play content. The content at this time may be content stored in the multimedia device 100, received broadcast content, or external input content input from the outside. The content may be at least one of a broadcast image, an external input image, an audio file, a still image, a connected web screen, and a document file.

Meanwhile, when entering the application view item, the controller 150 may control to display an application or a list of applications that can be downloaded from the multimedia device 100 or from an external network.

The controller 150 may control to install and run an application downloaded from an external network in addition to various user interfaces. In addition, according to the user's selection, it is possible to control the display unit 170 to display an image related to an executed application.

Meanwhile, although not shown in the drawing, a channel browsing processing unit for generating a thumbnail image corresponding to a channel signal or an external input signal may be further provided.

The channel browsing processing unit receives the stream signal TS output from the demodulation unit 112 or the stream signal output from the external device interface unit 120 and extracts an image from the input stream signal to generate a thumbnail image. I can. The generated thumbnail image may be input to the controller 150 as it is or after being encoded. In addition, the generated thumbnail image may be encoded in a stream form and input to the controller 150. The controller 150 may display a thumbnail list including a plurality of thumbnail images on the display unit 170 by using the input thumbnail images. Meanwhile, the thumbnail images in the thumbnail list may be updated sequentially or simultaneously. Accordingly, the user can easily grasp the contents of a plurality of broadcast channels.

Meanwhile, the control unit 150 includes a voice recognition unit 151 and a face recognition unit 152.

The voice recognition unit 151 may receive an external sound signal from the microphone 132, analyze audio data processed as electrical data, and convert it into a predetermined command. For example, the speech recognition unit 151 receives audio data corresponding to a command uttered by a user from the microphone 132 and performs a speech to text (STT) operation using a speech recognition algorithm stored in the memory 140. You can do it. In addition, the controller 180 may generate a control signal corresponding to a predetermined command converted by the STT operation, and control each function of the multimedia device 100 according to the generated control signal. The memory 140 may store specific commands and mapping data for each function of the multimedia device 100. In addition, the voice recognition unit 151 may recognize and distinguish each user's voice. The memory 140 stores audio data corresponding to each registered user, and may store an algorithm required for voice distinction in advance. The voice recognition unit 151 analyzes the audio data received from the microphone 122, determines whether the audio data includes audio data corresponding to the voices of multiple users, and is stored in the memory 140 in the audio data. Whether or not audio data corresponding to the registered user's voice is included can be recognized.

The face recognition unit 152 may recognize a face included in an image acquired by the camera 131. The memory 140 stores image data corresponding to a face of each registered user, and may store an algorithm required for face recognition in advance. The face recognition unit 152 may recognize whether a registered user's face stored in the memory 140 is included in the image acquired by the camera 131.

The display unit 170 converts an image signal, a data signal, an OSD signal processed by the control unit 150, or an image signal and a data signal received from the external device interface unit 120 into R, G, and B signals, respectively. Generate a drive signal.

The display unit 170 may be a PDP, LCD, OLED, flexible display, 3D display, or the like.

Meanwhile, the display unit 170 may be configured as a touch screen and used as an input device in addition to an output device.

The audio output unit 180 receives a signal processed by the control unit 150, for example, a stereo signal, a 3.1 channel signal, or a 5.1 channel signal, and outputs it as audio. The audio output unit 180 may be implemented with various types of speakers.

Meanwhile, in order to detect a user's gesture, as described above, a sensing unit (not shown) including at least one of a touch sensor, a voice sensor, a position sensor, and a motion sensor may be further provided in the multimedia device 100. . The signal sensed by the sensing unit (not shown) may be transmitted to the control unit 150 through the user interface unit 160.

Meanwhile, a photographing unit (not shown) for photographing a user may be further provided. Image information captured by the photographing unit (not shown) may be input to the control unit 150.

The controller 150 may detect a user's gesture by combining or combining an image captured from a photographing unit (not shown) or a signal sensed from a sensing unit (not shown).

The power supply unit 190 supplies corresponding power throughout the multimedia device 100.

In particular, power is supplied to the control unit 150 that can be implemented in the form of a System on Chip (SoC), the display unit 170 for displaying an image, and the audio output unit 180 for outputting audio. I can.

To this end, the power supply unit 190 may include a converter (not shown) that converts AC power into DC power. Meanwhile, for example, when the display unit 170 is implemented as a liquid crystal panel having a plurality of backlight lamps, an inverter capable of operating a Pulse Width Modulation (PWM) for luminance variable or dimming driving (inverter) (not shown) may be further provided.

The remote controller 200 transmits a user input to the user interface unit 160. To this end, the remote controller 200 may use Bluetooth, Radio Frequency (RF) communication, infrared (IR) communication, Ultra Wideband (UWB), ZigBee, or the like.

In addition, the remote controller 200 may receive an image, audio, or data signal output from the user interface unit 160 and display it on the remote controller 200 or output a voice or vibration.

The above-described multimedia device 100 may be a digital broadcast receiver capable of processing a digital broadcast signal of a fixed or mobile ATSC method or a DVB method.

In addition, the display device according to the present invention may omit some of the illustrated configurations as necessary, or may further include a configuration not illustrated. On the other hand, the display device does not have a tuner and a demodulation unit, and may receive and play content through a network interface unit or an external device interface unit, unlike the above.

2 is a block diagram illustrating a detailed configuration of a control unit of FIG. 1 according to an embodiment of the present invention.

The control unit 150 of the multimedia device 100 according to an embodiment of the present invention includes a demultiplexer 153, an image processing unit 154, an OSD generator 155, a mixer 156, and a frame rate. A frame rate converter (FRC) 157 and a formatter 158 may be included. In addition, although not shown, the control unit 150 may further include a voice processing unit and a data processing unit.

The demultiplexer 153 demultiplexes an input stream. For example, the demultiplexer 153 may demultiplex input MPEG-2 TS video, audio, and data signals. Here, the stream signal input to the demultiplexer 153 may be a stream signal output from a tuner or demodulator or an external device interface unit.

The image processing unit 154 performs image processing on the demultiplexed image signal. To this end, the image processing unit 154 may include an image decoder 154a and a scaler 154b.

The image decoder 154a decodes the demultiplexed image signal, and the scaler 272b scales the resolution of the decoded image signal to be output from the display unit.

The video decoder 154a may support various standards. For example, the video decoder 154a functions as an MPEG-2 decoder when the video signal is encoded in the MPEG-2 standard, and the video signal is encoded in the Digital Multimedia Broadcasting (DMB) method or the H.264 standard. In this case, it can perform the function of an H.264 decoder.

Meanwhile, the image signal decoded by the image processing unit 154 is input to the mixer 156.

The OSD generator 155 generates OSD data by itself or according to a user input. For example, the OSD generation unit 155 generates data for displaying various types of data on the screen of the display unit 170 in the form of a graphic or text based on the control signal of the user interface unit 160. Generate. The generated OSD data includes various data such as a user interface screen (eg, GUI) of the multimedia device 100, various menu screens, widgets, icons, and viewing rate information. do. The OSD generator 155 may generate data for displaying a caption of a broadcast image or broadcast information based on EPG.

The mixer 156 mixes the OSD data generated by the OSD generation unit 155 and the image signal processed by the image processing unit and provides them to the formatter 158. As the decoded video signal and OSD data are mixed, the OSD is displayed as an overlay on a broadcast video or an external input video.

The frame rate conversion unit (FRC) 157 converts a frame rate of an input video. For example, the frame rate converter 157 may convert the frame rate of an input 60 Hz image to have a frame rate of, for example, 120 Hz or 240 Hz according to an output frequency of the display unit. As described above, there may be various methods for converting the frame rate. For example, when converting the frame rate from 60Hz to 120Hz, the frame rate converter 157 inserts the same first frame between the first frame and the second frame, or It can be converted by inserting 3 frames. As another example, when converting the frame rate from 60Hz to 240Hz, the frame rate converter 157 may convert by inserting three more identical frames or predicted frames between existing frames. Meanwhile, when a separate frame conversion is not performed, the frame rate conversion unit 157 may be bypassed.

The formatter 158 changes the output of the input frame rate conversion unit 157 according to the output format of the display unit. For example, the formatter 158 may output R, G, B data signals, and these R, G, B data signals are output as a low voltage differential signal (LVDS) or mini-LVDS. Can be. In addition, when the output of the input frame rate conversion unit 157 is a 3D video signal, the formatter 158 configures and outputs a 3D format according to the output format of the display unit, thereby providing a 3D service through the display unit 170. You can also apply.

Meanwhile, the voice processing unit (not shown) in the control unit 150 may perform voice processing of the demultiplexed voice signal. Such a voice processing unit (not shown) may support processing various audio formats. For example, even when a voice signal is encoded in a format such as MPEG-2, MPEG-4, AAC, HE-AAC, AC-3, BSAC, a decoder corresponding thereto may be provided and processed.

In addition, a voice processing unit (not shown) in the control unit 150 may process a base, a treble, and a volume control.

A data processing unit (not shown) in the control unit 150 may perform data processing of the demultiplexed data signal. For example, the data processing unit may decode the demultiplexed data signal even when it is encoded. Here, the encoded data signal may be EPG information including broadcast information such as a start time and an end time of a broadcast program aired on each channel.

Meanwhile, the multimedia device 100 described above is an example according to the present invention, and each component may be integrated, added, or omitted according to the specifications of a display device that is actually implemented. That is, if necessary, two or more components may be combined into one component or one component may be subdivided into two or more components. In addition, functions performed by each block are for explaining an embodiment of the present invention, and specific operations or devices thereof do not limit the scope of the present invention.

3 is a diagram showing examples of a remote controller for controlling a multimedia device according to an embodiment of the present invention.

In order to control the multimedia device 100, a front panel (not shown) provided on the multimedia device 100 may be used or a remote controller 200 may be used.

The remote controller 200 is a user interface device (UID) capable of wired and wireless communication, and is a remote controller 203, a keyboard 202, and a pointing device 201 mainly implemented for the control purpose of the multimedia device 100. , A touch-pad, etc. may be included. In addition, a mobile device such as a smart phone or a tablet PC capable of controlling the multimedia device 100 through mode switching, etc., which is not intended to control the multimedia device 100 may be included in the remote controller 200.

The remote controller 200 includes Bluetooth, Radio Frequency Identification (RFID), infrared data association (IrDA), Ultra Wideband (UWB), ZigBee, Digital Living Network Alliance (DLNA), RS, etc. It is possible to communicate with the multimedia device 100 by employing at least one or more communication protocols as needed.

The remote control 203 refers to a conventional input means provided with various key buttons necessary for controlling the multimedia device 100.

The pointing device 201 implements a pointer corresponding to the screen of the multimedia device 100 based on the user's movement, pressure, rotation, etc. by mounting a gyro sensor, etc., and the multimedia device 100 ) To transmit a predetermined control command. The pointing device 203 may be named by various names such as a magic remote control and a magic controller.

The keyboard 202 is an intelligent integrated digital device that goes beyond the conventional broadcasting only and provides various services such as web browsers, applications, and SNS (Social Network Service), so that the conventional remote control 203 can be used alone to control the keyboard 202. It was not easy, and it was implemented to improve the convenience of inputting text, etc. by implementing it similar to the keyboard of a PC.

On the other hand, the remote controller 200 such as the remote control 203, the pointing device 201, and the keyboard 202 is provided with a touch pad as necessary, so that it is more convenient to input text, move a pointer, and enlarge/reduce photos or videos. And can be used for various control purposes.

4 is a block diagram showing configuration modules of a remote controller according to an embodiment of the present invention. However, FIG. 4 is an embodiment for convenience of explanation, and a form in which other constituent modules other than the constituent modules shown in FIG. 4 are included or some constituent modules are omitted also falls within the scope of the present invention.

4, the remote controller 200 according to an embodiment of the present invention includes a power supply unit 211, a memory 212, a control unit 213, a wireless communication unit 214, a user input unit 215, It may include an output unit 216, a sensor unit 217, a microphone 218, and the like.

The wireless communication unit 214 is designed to be able to communicate with any external device.

The wireless communication unit 214 includes an RF module 214a capable of transmitting and receiving signals with the multimedia device 100 according to RF communication standards, and an IR module capable of transmitting and receiving signals with the multimedia device 100 according to IR communication standards ( 214b).

The remote controller 200 may transmit a signal containing information about the movement of the remote controller 200 to the multimedia device 100 through the RF module 211. In addition, the remote controller 200 may receive a signal transmitted by the multimedia device 100 through the RF module 211.

In addition, the remote controller 200 may transmit a command regarding power on/off, channel change, volume change, etc. to the multimedia device 100 through the IR module 214b as needed.

The user input unit 215 may be composed of a keypad, a button, a touch pad, or a touch screen. A user may input a command related to the multimedia device 100 by manipulating the user input unit 215.

The sensor unit 217 may include a gyro sensor 217a and an acceleration sensor 217b.

For example, the gyro sensor 217a may sense information on the operation of the remote controller 200 based on the x, y, and z axes. The acceleration sensor 217b may sense information about a moving speed of the remote controller 200 and the like. Meanwhile, a distance measurement sensor may be further provided, thereby sensing a distance to the multimedia device 100.

The output unit 216 may output an image or audio signal corresponding to an operation of the user input unit 215 or corresponding to a signal transmitted from the multimedia device 100. Through the output unit 216, the user can recognize whether the user input unit 215 is operated or whether the multimedia device 100 is controlled.

For example, the output unit 216 is an LED module 216a that lights up when a user input unit 215 is manipulated or a signal is transmitted and received with the multimedia device 100 through the wireless communication unit 214, and a vibration module 216b that generates vibration. ), a sound output module 216c for outputting sound, or a display module 216d for outputting an image.

The power supply 211 supplies power to the remote controller 200. The power supply unit 211 may reduce power waste by stopping power supply when the remote controller 200 does not move for a predetermined period of time. The power supply unit 211 may resume power supply when a predetermined key provided in the remote controller 200 is operated.

The memory 212 may store various types of programs and application data necessary for controlling or operating the remote controller 200.

The control unit 213 controls all matters related to the control of the remote controller 200. The control unit 213 transmits a signal corresponding to a predetermined key operation of the user input unit 215 or a signal corresponding to the movement of the remote controller 200 sensed by the sensor unit 217 through the wireless communication unit 214. ).

The remote controller 200 may include a microphone 218. The microphone 218 processes an external sound signal into electrical voice data. Depending on the embodiment, the microphone 218 may always be maintained in an active state, or may be activated only when a specific button in the user input unit 215 is selected. The controller 213 may transmit audio data received from the microphone 218 to the multimedia device 100 through the wireless communication unit 214.

Hereinafter, embodiments of the present invention will be described in detail with reference to FIGS. 5 to 24.

5 is a flowchart illustrating an example of a method of controlling a multimedia device according to an embodiment of the present invention.

The controller 150 of the multimedia device 100 controls the display unit 170 to output video data of a specific content and controls the audio output unit 180 to output audio data of the specific content. The specific content may be included in a live broadcast signal received through the tuner 111, received from an external server through the network interface unit 113, or received from an external device through the external device interface unit 120. It can also be content.

The controller 150 controls the microphone 132 to receive an external sound signal and convert it into electrical first audio data (S302). In order to facilitate the distinction from the microphone 218 in the remote controller 200 to be described later, the microphone 132 may be referred to as a first microphone 132 and the microphone 218 may be referred to as a second microphone 218. Depending on the embodiment, the first microphone 132 may be kept in an active state at all times, may be activated for a predetermined time at preset time intervals, and multiple faces are recognized through images acquired by the camera 131 If so, it may be activated.

The control unit 150 controls the wireless communication unit to receive second audio data corresponding to an external sound signal from the remote controller 200 (S303). The second audio data is obtained by converting an external sound signal received through the second microphone 218 in the remote controller 200 into electrical data. Depending on the embodiment, the wireless communication unit may be implemented in the user interface unit 160 or in the external device interface unit 120. The second audio data may be received from the remote controller 200 according to an RF communication standard.

The controller 150 controls the camera 131 to acquire an image (S304). Depending on the embodiment, the step (S304) may be performed by temporally overlapping with at least some of the above-described steps (S301 to S303).

When the value of the first audio data and the value of the second audio data are different from each other, the controller 150 controls a predetermined specific function based on the acquired image (S305). The preset specific function may be a volume control function of audio data output through the audio output unit 180, a function related to reproduction of the specific content, and the like. Here, the value of the first audio data means, according to an embodiment, the strength of a signal corresponding to the audio data obtained through the first microphone 132, and the audio output unit from the audio data obtained through the first microphone 132 The strength of the noise signal excluding the audio data output through 180, the audio data output through the audio output unit 180 from the audio data obtained through the first microphone 132 is extracted, and the extracted audio data is It may be any one of the strength of the corresponding signal. The same is true for the value of the second audio data.

The controller 150 may refer to the acquired image to control a preset specific function. At this time, the control unit 150, through the obtained image, the user, the user's gaze, the distance from the user, the user's location, the location of the remote controller 200, whether the user holds the remote controller 200 and the user At least one of the movements of the mouth of the person may be detected. For example, the controller 150 may recognize faces of users included in the acquired image through the face recognition unit 151, and whether there is a registered user stored in the memory 140 among the recognized faces. Whether or not it can be detected. In addition, the controller 150 may detect a user's gaze based on a direction in which the recognized face is facing and features included in the recognized face. In addition, the controller 150 may detect a distance between the multimedia device 100 and the user by comparing the size of the user included in the acquired image with the reference size of the person stored in the memory 140. In addition, the controller 150 may detect the locations of users included in the acquired image. In addition, the controller 150 may detect the location of the remote controller 200 based on whether the remote controller 200 is included in the acquired image and the size of the remote controller 200. In addition, the control unit 150 determines whether a user included in the acquired image is holding the remote controller 200 and whether any of the users included in the acquired image is adjacent to the remote controller 200. Can be detected. In addition, the controller 150 may detect whether the user's mouth is moving or not, based on the recognized facial features.

6 is a diagram illustrating an example of a method of controlling a multimedia device according to an embodiment of the present invention.

In FIG. 6, the multimedia device 100 is turned on and several users are in the vicinity of the multimedia device 100, but no one is watching the multimedia device 100 or only a small number of users are watching the multimedia device 100. Suppose you are watching 100). In addition, it is assumed that the remote controller 200 is located adjacent to users rather than the multimedia device 200.

The controller 150 controls the display unit 170 to output video data of a specific content, and controls the audio output unit 180 to output audio data of the specific content.

The controller 150 controls the first microphone 132 to receive an external sound signal and convert it into electrical first audio data, and the second audio data corresponding to the external sound signal from the remote controller 200 Control the wireless communication unit to receive.

The controller 150 compares the value of the first audio data and the value of the second audio data. Here, the value of the first audio data means the strength of a signal corresponding to the audio data acquired through the first microphone 132 or the audio data acquired through the first microphone 132 and output through the audio output unit 180 It may be the strength of the noise signal excluding the audio data. In addition, the value of the second audio data means that when the value of the first audio data is the strength of a signal corresponding to the audio data acquired through the first microphone 132, the second audio data is received from the remote controller 200. It may be the strength of a signal corresponding to the audio data, and the value of the first audio data is of the noise signal excluding the audio data output through the audio output unit 180 from the audio data obtained through the first microphone 132. In the case of the intensity, it may be the intensity of a noise signal excluding audio data output through the audio output unit 180 from the second audio data received from the remote controller 200.

When the value of the first audio data and the value of the second audio data are different from each other, the controller 150 may control a specific function of the multimedia device 100 based on an image acquired by the camera 131. The controller 150 may control the camera 131 to acquire an image every preset time interval, and when the value of the first audio data and the value of the second audio data are different, the image previously acquired by the camera 131 May be used, or when the value of the first audio data and the value of the second audio data are different, the camera 131 may be controlled to acquire an image, and the acquired image may be used.

The control unit 150, through the acquired image, the user, the user's gaze, the distance from the user, the user's location, the location of the remote controller 200, whether the user holds the remote controller 200, and the user's mouth. At least one of the movements of may be detected.

The controller 150 has a larger value of the second audio data than the value of the first audio data, and based on the detected user's gaze, the controller 150 is configured with a display unit ( If there are more users who are not facing 170), the volume of audio data output through the audio output unit 180 may be reduced by a preset degree. If there are multiple users in front of the multimedia device 100 but no one is watching the multimedia device 100 or only a few users are watching the multimedia device 100, reducing the volume is consistent with the intentions of many users. Because you can. According to an embodiment, the control unit 150 has a larger value of the second audio data than the value of the first audio data, and the gaze is directed toward the display unit 170 based on the detected gaze of the user. When there are more users who are not facing the display unit 170 than users, and the users included in the acquired image do not hold the remote controller 200, the audio data output through the audio output unit 180 is You can also decrease the volume by a preset amount.

According to an embodiment, the control unit 150 has a larger value of the second audio data than the value of the first audio data, and the gaze is directed toward the display unit 170 based on the detected gaze of the user. When there is no user, playback of the specific content may be stopped or paused. Here, when the specific content is a live broadcast signal, playback may be paused and a time-shift function may be activated.

7 is a diagram illustrating another example of a method of controlling a multimedia device according to an embodiment of the present invention.

In FIG. 7, although the multimedia device 100 is turned on and several users are in the vicinity of the multimedia device 100, no one is watching the multimedia device 100 or only a small number of users are watching the multimedia device 100. Suppose you are watching 100). In addition, it is assumed that the remote controller 200 is located adjacent to users rather than the multimedia device 200.

The controller 150 controls the display unit 170 to output video data of a specific content, and controls the audio output unit 180 to output audio data of the specific content.

The controller 150 controls the first microphone 132 to receive an external sound signal and convert it into electrical first audio data, and the second audio data corresponding to the external sound signal from the remote controller 200 Control the wireless communication unit to receive.

The controller 150 compares the value of the first audio data and the value of the second audio data. And, when the value of the first audio data and the value of the second audio data are different from each other, the controller 150 may control a specific function of the multimedia device 100 based on the image acquired by the camera 131. .

The controller 150 has a larger value of the second audio data than the value of the first audio data, and based on the detected user's gaze, the controller 150 is configured with a display unit ( When there are more users who are not facing the display unit 170, but the user whose gaze is facing the display unit 170 is holding the remote controller 200, the volume of audio data output through the audio output unit 180 is fixed. It can be maintained or increased by a preset degree. Even if there are a large number of users who are not watching the multimedia device 100, this is to give priority to a user holding the remote controller 200.

Alternatively, the control unit 150 may display a value of the second audio data greater than the value of the first audio data, and a gaze of a user who is facing the display unit 170 based on the detected gaze of the user. When there are more users who are not facing the unit 170, but the user whose gaze is facing the display unit 170 is a registered user stored in the memory 140, the volume of audio data output through the audio output unit 180 Can be kept constant or can be increased by a preset degree.

8 is a diagram for explaining another example of a method of controlling a multimedia device according to an embodiment of the present invention.

In FIG. 8, the multimedia device 100 is turned on and several users are in the vicinity of the multimedia device 100, and a plurality of users are watching the multimedia device 100, and a few users are the multimedia devices. Suppose you are watching (100). In addition, the remote controller 200 may be located adjacent to users rather than the multimedia device 200.

The controller 150 controls the display unit 170 to output video data of a specific content, and controls the audio output unit 180 to output audio data of the specific content.

The controller 150 controls the first microphone 132 to receive an external sound signal and convert it into electrical first audio data, and the second audio data corresponding to the external sound signal from the remote controller 200 Control the wireless communication unit to receive.

The controller 150 compares the value of the first audio data and the value of the second audio data. And, when the value of the first audio data and the value of the second audio data are different from each other, the controller 150 may control a specific function of the multimedia device 100 based on the image acquired by the camera 131. .

The control unit 150 determines that the user whose first audio data value is greater than the second audio data value, and whose gaze is toward the display unit 170 based on the detected gaze of the user When there are more users than users who are not facing 170), the volume of audio data output through the audio output unit 180 may be increased by a preset degree. This is because increasing the volume can suit the intentions of multiple users.

9 is a diagram illustrating an example of a method of controlling a multimedia device according to an embodiment of the present invention.

The controller 150 of the multimedia device 100 controls the display unit 170 to output video data of a specific content and controls the audio output unit 180 to output audio data of the specific content.

The controller 150 controls the camera 131 to acquire an image every preset time interval.

When the controller 150 detects a gesture of a user putting the phone to his ear or a movement of the user's mouth, which is included in the acquired image, through the acquired image, the audio output unit 180 The volume of audio data that is output through may be reduced by a preset degree.

According to an embodiment, when the controller 150 detects a gesture of a user putting a phone to his ear or a movement of the user's mouth, included in the acquired image, through the acquired image, the You can also stop or pause playback of specific content. Here, when the specific content is a live broadcast signal, playback may be paused and a time-shift function may be activated.

10 is a diagram illustrating an example of a method of controlling a multimedia device according to an embodiment of the present invention.

The controller 150 of the multimedia device 100 controls the display unit 170 to output video data of a specific content and controls the audio output unit 180 to output audio data of the specific content.

The controller 150 controls the first microphone 132 to receive an external sound signal and convert it into electrical first audio data, and the second audio data corresponding to the external sound signal from the remote controller 200 Control the wireless communication unit to receive.

The controller 150 may adjust the volume of audio data output through the audio output unit 180 so that the value of the second audio data is kept constant. The value of the second audio data may be the strength of a signal corresponding to the extracted audio data after extracting the audio data output through the audio output unit 180 from the second audio data received from the remote controller 200 have. The control unit 150, from the second audio data received from the remote controller 200, removes the ambient lock sound excluding the audio data of the specific content output through the audio output unit 180, and removes the ambient noise. The volume of audio data output through the audio output unit 180 may be adjusted so that the value of the second audio data is kept constant. When the user holding the remote controller 200 moves away from or closes to the multimedia device 100, the size of the second audio data changes accordingly, and the user can enjoy the content at a constant volume. This is to adjust the volume of audio data of the specific content.

According to an embodiment, the controller 150 adjusts the volume of audio data output through the audio output unit 180 so that the value of the second audio data is kept constant based on the image acquired by the camera 131. You can also adjust it. Here, it is assumed that the memory 140 stores the registered user's face in advance. Among the registered users stored in the memory 140, a user registered as a master user may also exist. If the registered user stored in the memory 140 is holding the remote controller 200 based on the acquired image, or the user registered as a master user among the registered users stored in the memory 140 Only when the remote controller 200 is held, the volume of audio data output through the audio output unit 180 may be adjusted so that the value of the second audio data is kept constant.

Alternatively, according to an embodiment, the value of the second audio data remains constant only when the user holding the remote controller 200 matches the user who has preset the automatic volume control function based on the acquired image. The volume of audio data output through the audio output unit 180 may be adjusted to be maintained.

Alternatively, according to an embodiment, the control unit 150 may provide the user who does not include the remote controller 200 or is holding the remote controller 200 in the acquired image. In the case of a distance, the change in the value of the second audio data may be ignored and the volume of the audio data output through the audio output unit 180 may be kept constant.

11 is a diagram illustrating an example of a method of controlling a multimedia device according to an embodiment of the present invention.

In FIG. 11, it is assumed that a user approaches the multimedia device 100 and utters a specific command. In addition, it is assumed that the remote controller 200 is somewhat far apart from the user.

The control unit 150 of the multimedia device 100 controls the first microphone 132 to receive an external sound signal and convert it into electrical first audio data, and from the remote controller 200 to the external sound signal. Controls the wireless communication unit to receive the corresponding second audio data. It is assumed that the first audio data and the second audio data include audio data corresponding to a specific command uttered by a user.

When the value of the audio data corresponding to the specific command included in the first audio data is greater than a value of the audio data corresponding to the specific command included in the second audio data by a predetermined degree or more , It is possible to ignore the specific command without executing a function corresponding to the specific command. Here, the value of the audio data corresponding to the specific command refers to the specific command separated by removing the audio data output through the audio output unit 180 and ambient noise from the first audio data or the second audio data. It may mean the strength of a signal corresponding to the corresponding audio data. This is to give the initiative to a user who is holding the remote controller 200 or is in proximity to the remote controller 200.

According to an embodiment, the control unit 150 is configured such that a value of audio data corresponding to the specific command included in the first audio data is greater than a value of audio data corresponding to the specific command included in the second audio data. When it is larger than the set degree, the display unit 170 and/or the audio output unit 180 may be controlled to output a guide message to approach the remote controller 200 and utter a command.

12 is a diagram illustrating an example of a method of controlling a multimedia device according to an embodiment of the present invention.

In FIG. 12, it is assumed that the user is closer to the remote controller 200 than the multimedia device 100 and utters a specific command.

The control unit 150 of the multimedia device 100 controls the first microphone 132 to receive an external sound signal and convert it into electrical first audio data, and from the remote controller 200 to the external sound signal. Controls the wireless communication unit to receive the corresponding second audio data. It is assumed that the first audio data and the second audio data include audio data corresponding to a specific command uttered by a user.

When the value of the audio data corresponding to the specific command included in the second audio data is greater than a value of the audio data corresponding to the specific command included in the first audio data by a predetermined degree or more , It is possible to execute a function corresponding to the specific command. This is to give the initiative to a user who is holding the remote controller 200 or is in proximity to the remote controller 200.

According to an embodiment, when the user utters a specific command without pressing a specific button corresponding to the voice recognition function provided in the remote controller 200, the controller 150 may increase the voice recognition rate, the remote controller 200 Controlling the display unit 170 and/or the audio output unit 180 to output a guide message to utter a specific command while pressing a specific button corresponding to the voice recognition function provided in the device, or predetermined to the remote controller 200 The user interface unit 160 may be controlled to transmit a control signal of. The control unit 213 of the remote controller 200 outputs a specific light through the LED module 216a according to the control signal, or generates a specific pattern of vibration through the vibration module 216c, or a sound output module By outputting a specific audio signal through 216c or outputting a predetermined guide message through the display module 216d, the user may recognize that a specific button corresponding to the voice recognition function has not been pressed.

13 is a diagram illustrating an example of a method of controlling a multimedia device according to an embodiment of the present invention.

In FIG. 13, the user is closer to the remote controller 200 than the multimedia device 100 and utters a specific command, but the remote controller 200 is farther away from the multimedia device 100 or different from the multimedia device 100. Assume a situation located in space.

The control unit 150 of the multimedia device 100 controls the first microphone 132 to receive an external sound signal and convert it into electrical first audio data, and from the remote controller 200 to the external sound signal. Controls the wireless communication unit to receive the corresponding second audio data. It is assumed that the second audio data includes audio data corresponding to a specific command uttered by the user. In the first audio data, audio data corresponding to the specific command may not be included, or even if included, the intensity may be less than a preset level.

When the first audio data does not include audio data corresponding to the specific command, the controller 150 may ignore the specific command without executing a function corresponding to the specific command. This is because when the remote controller 200 exists in a space different from that of the multimedia device 100, it is not necessary to give the user adjacent to the remote controller 200 the initiative.

According to an embodiment, the control unit 150 is configured such that a value of audio data corresponding to the specific command included in the first audio data is greater than a value of audio data corresponding to the specific command included in the second audio data. When the remote controller 200 is not included in the acquired image based on the image acquired by the camera 131 or the movement of the user's mouth included in the acquired image is not detected. , It is possible to ignore the specific command without executing a function corresponding to the specific command.

14 is a diagram illustrating an example of a method of controlling a multimedia device according to an embodiment of the present invention.

In FIG. 14, it is assumed that there are several users in front of the multimedia device 100 and a user holding the remote controller 200 utters a specific command.

The control unit 150 of the multimedia device 100 controls the first microphone 132 to receive an external sound signal and convert it into electrical first audio data, and from the remote controller 200 to the external sound signal. Controls the wireless communication unit to receive the corresponding second audio data. It is assumed that the first audio data and the second audio data include audio data corresponding to a specific command uttered by a user.

When the value of the audio data corresponding to the specific command included in the second audio data is greater than a value of the audio data corresponding to the specific command included in the first audio data by a predetermined degree or more , Based on the movement of the user's mouth detected through the image acquired by the camera 131, a specific user who uttered the specific command among users included in the acquired image may be detected. Further, the controller 150 may execute a specific function corresponding to the specific command only when the specific user holds the remote controller 200 based on the acquired image. This is to give the initiative to a user holding the remote controller 200 among several users watching the multimedia device 100.

15 is a diagram illustrating an example of a method of controlling a multimedia device according to an embodiment of the present invention.

In FIG. 15, it is assumed that there are several users in front of the multimedia device 100 and a user registered as a master user among them utters a specific command.

The control unit 150 of the multimedia device 100 controls the first microphone 132 to receive an external sound signal and convert it into electrical first audio data, and from the remote controller 200 to the external sound signal. Controls the wireless communication unit to receive the corresponding second audio data. It is assumed that the first audio data and the second audio data include audio data corresponding to a specific command uttered by a user. A value of audio data corresponding to the specific command included in the second audio data may be greater than a value of audio data corresponding to the specific command included in the first audio data by a predetermined degree or more.

The memory 140 stores faces and voices of registered users, and among registered users stored in the memory 140, a user registered as a master user may exist.

When it is recognized that the audio data corresponding to the specific command included in the first audio data or the second audio data corresponds to the voice of the master user, the controller 150 performs a function corresponding to the specific command. You can do it. The control unit 150 controls the remote controller 200 based on the image acquired by the camera 131 to a user other than the master user (for example, an unregistered user (Guest) or a user registered as a child)). Even if held, when it is recognized that the audio data corresponding to the specific command corresponds to the voice of the master user, a function corresponding to the specific command can be executed. In the case of the master user, it is to give him the initiative regardless of whether the remote controller 200 is held.

According to an embodiment, when the first audio data or the second audio data includes audio data corresponding to a command uttered by two or more users, the controller 150 The function can also be executed according to a command of audio data corresponding to the voice.

16 is a diagram illustrating an example of a method of controlling a multimedia device according to an embodiment of the present invention.

In FIG. 16, it is assumed that there are several users in front of the multimedia device 100 and a user watching the multimedia device 100 utters a specific command.

The control unit 150 of the multimedia device 100 controls the first microphone 132 to receive an external sound signal and convert it into electrical first audio data, and from the remote controller 200 to the external sound signal. Controls the wireless communication unit to receive the corresponding second audio data. It is assumed that the first audio data and the second audio data include audio data corresponding to a predetermined command uttered by two or more users. A value of audio data corresponding to the specific command included in the second audio data may be greater than a value of audio data corresponding to a specific command included in the first audio data by a predetermined degree or more.

The controller 150 detects a specific user who uttered the specific command among users included in the acquired image based on the movement of the user's mouth detected through the image acquired by the camera 131 can do. Further, the controller 150 may execute a specific function corresponding to the specific command only when the specific user's gaze is directed toward the display unit 170 based on the acquired image. This is to give the initiative to a user who is actually watching the multimedia device 100.

17 is a diagram illustrating an example of a method of controlling a multimedia device according to an embodiment of the present invention.

At least one of the multimedia device 100 and the remote controller 200 may include an illuminance sensor. The illuminance sensor detects the intensity of external light, and the controller 150 of the multimedia device 100 may adjust the brightness of the display unit 170 based on the sensed intensity of light. When the illuminance sensor is included in the remote controller 200, the controller 213 of the remote controller 200 transmits a signal including the intensity of light sensed by the illuminance sensor to the multimedia device 100. 214), and the controller 150 of the multimedia device 100 may receive the signal through a wireless communication unit and adjust the brightness of the display unit 170 based on the intensity of light included in the signal. .

The controller 150 of the multimedia device 100 increases the brightness of the display unit 170 when the external light intensity is greater than a preset level, and when the external light intensity is less than a preset level, the display unit 170 ) Brightness can be reduced.

The control unit 150 of the multimedia device 100, when the value of the intensity of external light detected by the built-in illuminance sensor and the value of the intensity of external light received from the remote controller 200 are different, the built-in illuminance The brightness of the display unit 170 may be adjusted based on the intensity of external light detected by the sensor.

18 is a diagram illustrating another example of a method of controlling a multimedia device according to an embodiment of the present invention.

The multimedia device 100 may have a built-in clock. The controller 150, based on the current time acquired from the built-in clock, when the current time is within a preset range (for example, when the current time is in the middle of the night or an early color wall), the brightness of the display unit 170 The volume of audio data output through the audio output unit 180 may be lowered to less than or equal to a preset level, and the volume of audio data output through the audio output unit 180 may be lowered to or less than a preset level.

19 is a diagram for explaining another example of a method of controlling a multimedia device according to an embodiment of the present invention.

Referring to (a) of FIG. 19, a specific user sets the volume level of the audio output unit 180 to 25, the brightness level of the display unit 170 to 50, and selects an automatic adjustment function of volume and brightness. Suppose you did. Then, the user has switched channels as shown in (b) of FIG. 19.

As an example, when a user switches a channel to a first channel (eg, a news channel), the control unit 150 uses the first channel to deliver accurate information based on service information included in the broadcast signal. It can detect that it is a news channel. In addition, the controller 150 may maintain the volume and brightness currently set by the user without applying an automatic adjustment function to the volume of the audio output unit 180 and the brightness of the display 170.

As another example, when a user switches to a second channel (eg, a movie channel), the controller 150 may detect that the second channel is a movie channel based on service information included in a broadcast signal. . In the case of movie channels, since the set volume is relatively high and the scene change is not fast, the controller 150 can reduce the volume than the volume preset in the audio output unit 180 by the user when the automatic adjustment function is applied. In addition, the brightness may be reduced compared to the brightness set in the display unit 170 by the user.

As another example, when the user switches to a third channel (eg, a sports channel), the controller 150 may detect that the third channel is a sports channel based on service information included in a broadcast signal. have. In the case of a sports channel, since the set volume is relatively low and the scene change is usually fast, the controller 150 can increase the volume than the volume preset in the audio output unit 180 by the user when the automatic adjustment function is applied. , It is possible to increase the brightness than the brightness set in the display unit 170 by the user.

Hereinafter, a process of registering a user in the multimedia device 100, functions customized to a registered user, and rights of a registered user will be described with reference to FIGS. 20 to 24.

20 is a diagram illustrating a process of registering a user in a multimedia device according to an embodiment of the present invention.

When a user executes a specific application installed in the multimedia device 100, a screen 310 for user registration may be displayed on the display unit 170. The user may execute the specific application using the remote controller 200.

The screen 310 includes a first menu 311 for capturing an image of a user to be registered, a second menu 312 for recording the voice of the user to be registered, and a function-related menu that can be set by the user to be registered ( 313) and the like may be included. Although not shown, the user may also set whether to register himself/herself as a master user through the screen 310.

When the user selects the first menu 311, the controller 150 controls the camera 131 to acquire the user's image, and displays the acquired image to the area of the first menu 311. The unit 170 can be controlled.

When the user selects the second menu 312, the controller 150 may control the microphone 132 to record the user's voice.

The third menu 313 includes an automatic volume adjustment menu for automatically adjusting the volume of the audio output unit 180 as set by the user, and automatically adjusting the brightness of the display unit 170 as set by the user. It may include a brightness automatic adjustment menu to make it possible.

When the user selects the registration menu 314 in the screen 310, the user's name, the face in the acquired image, the recorded voice, and the user's status (for example, the master user) are mapped to each other. And stored in the memory 140.

When registered as a master user, all functions of the multimedia device 100 can be used, and as described above, when multiple users are using the multimedia device 100, they can take the initiative in controlling the multimedia device 100. .

21 is a diagram illustrating a process of registering another user by a master user in a multimedia device according to an embodiment of the present invention. Contents overlapping with those described above with respect to FIG. 20 will not be described again, and the following will focus on differences.

A user registered as a master user may register another user (eg, child) with the multimedia device 100. Another user may register himself/herself in the multimedia device 100, but may register another user with the multimedia device 100 with the authority of the master user and set the authority of the other user.

Since the process of registering the image or voice of another user is the same as described above with respect to FIG. 20, a description thereof will be omitted.

The master user may set the status of the other user as a child and restrict the authority of the other user. For example, the master user may limit the use time of the multimedia device 100 of the other user, limit the channel switching function, or limit the Internet function. The control unit 150 includes the other user in the image acquired by the camera 131 during use of the multimedia device 100 or the voice of the other user in the audio data acquired through the microphone 132 If present, functions restricted by the master user may not be executed.

22 is a diagram for explaining a difference in authority between a user registered in a multimedia device and an unregistered user according to an embodiment of the present invention.

Referring to FIG. 22, there are a plurality of users in front of the multimedia device 100, and among the plurality of users, a user registered as a master user, a user registered as a child, and an unregistered user (Guest ).

A user registered as an child and a non-registered user may use only some functions of the multimedia device 100, for example, only a volume up/down function or a power on/off function.

As an example, when the audio data acquired through the microphone 132 or the audio data received from the remote controller 200 includes audio data corresponding to a channel up command, the controller 150 responds to the channel up command. The channel up function corresponding to the channel up command can be executed only when the corresponding audio data corresponds to the voice of the master user stored in the memory 140.

As another example, the control unit 150, based on the image acquired by the camera 131, only when the master user stored in the memory 140 is holding the remote controller 200, from the remote controller 200 Functions corresponding to various received control signals can be executed.

23 is a diagram illustrating a process of registering another user by a master user in a multimedia device according to an embodiment of the present invention. Contents overlapping with those described above with respect to FIG. 20 will not be described again, and the following will focus on differences.

A user registered as a master user may register another user (eg, an elderly person) with the multimedia device 100.

Since the process of registering the image or voice of another user is the same as described above with respect to FIG. 20, a description thereof will be omitted.

The master user can set customized functions for the other users. For example, the master user may set an automatic volume control function and an automatic brightness control function through the third menu 313 in light of the other user being the elderly.

FIG. 24 is a diagram for describing an example in which an automatic volume control function and an automatic brightness control function for a specific user are set in a multimedia device according to an embodiment of the present invention. It is assumed that the example of FIG. 24 is based on the contents described above with respect to FIG. 23.

The control unit 150 of the multimedia device 100 includes a pre-registered user (for example, an elderly user in FIG. 23) in the image acquired by the camera 131 or audio acquired through the microphone 132. When the data includes the voice of the user that is already registered, the master user sets the volume of the audio output unit 180 to the volume previously set for the other user, and the master user sets the volume of the audio output unit 180 for the other user. The brightness of the display unit 170 may be set as the brightness. Considering that the registered user is an elderly person, this is to prevent adverse effects on the body due to loud noise or glare. According to an embodiment, when a master user has registered a favorite channel for the registered user, the controller 150 includes the user already registered in the image acquired by the camera 131 or the microphone 132 When the previously registered user's voice is included in the audio data acquired through ), the tuner 111 may be controlled to automatically change the channel to the registered channel.

According to the above-described embodiments of the present invention, when a value of audio data acquired through a microphone embedded in the multimedia device and a value of audio data acquired through a microphone embedded in the remote controller are different, the multimedia device It can provide a solution as to whether to control the function of the device.

In addition, according to the above-described embodiments of the present invention, when a multimedia device is used by multiple users, a solution for which user is given priority and how to control the function of the multimedia device according to the intention of the plurality of users. Can provide.

The present invention described above can be implemented as a computer-readable code on a medium on which a program is recorded. The computer-readable medium includes all types of recording devices that store data that can be read by a computer system. Examples of computer-readable media include hard disk drives (HDDs), solid state disks (SSDs), silicon disk drives (SDDs), ROMs, RAM, CD-ROMs, magnetic tapes, floppy disks, optical data storage devices, etc. There is also a carrier wave (for example, transmission over the Internet) also includes the implementation of the form. In addition, the computer may include the control unit 150 of the terminal. Therefore, the detailed description above should not be construed as restrictive in all respects and should be considered as illustrative. The scope of the present invention should be determined by reasonable interpretation of the appended claims, and all changes within the equivalent scope of the present invention are included in the scope of the present invention.

100: multimedia device
200: remote controller

Claims (20)

A first microphone;
A wireless communication unit wirelessly connected to a remote controller including a second microphone;
A display unit that outputs video data of specific content;
An audio output unit outputting audio data of the specific content;
camera; And
Includes;
The control unit,
Controlling the first microphone to receive an external sound signal and convert it into electrical first audio data,
Controlling the wireless communication unit to receive second audio data corresponding to an external sound signal from the remote controller,
Controlling the camera to acquire an image,
When the value of the first audio data and the value of the second audio data are different, a predetermined specific function is performed based on a comparison result of the value of the first audio data and the value of the second audio data and the obtained image. Controlling, multimedia device. The method of claim 1,
The control unit detects at least one of the user, the user's gaze, the distance from the user, the user's position, the remote controller position, whether the user holds the remote controller, and the movement of the user's mouth through the acquired image. A multimedia device. The method of claim 2,
The control unit,
When the value of the second audio data is larger than the value of the first audio data, and based on the detected user's gaze, there are more users who are not facing the display unit than the users whose gaze is facing the display unit, A multimedia device for reducing a volume of audio data output through the audio output unit by a preset degree. The method of claim 2,
When the value of the first audio data is greater than the value of the second audio data, and based on the detected user's gaze, a user whose gaze is facing the display unit is greater than a user who is not facing the display unit, A multimedia device that increases the volume of audio data output through the audio output unit by a preset degree. The method of claim 3,
The control unit,
When a user whose gaze is facing the display unit based on the acquired image holds the remote controller, the volume of audio data output through the audio output unit is kept constant or increased by a predetermined amount. The method of claim 2,
The control unit,
Controlling the wireless communication unit to receive second audio data from the remote controller every preset time interval,
From the received second audio data, ambient noise other than the audio data of the specific content output through the audio output unit is removed,
A multimedia device for adjusting a volume of audio data output through the audio output unit so that the value of the second audio data from which the ambient noise has been removed is kept constant. The method of claim 6,
A memory for storing data for recognizing a face and a voice of a user registered as a master user; further includes,
The control unit, based on the acquired image, is output through the audio output unit so that the value of the second audio data from which the ambient noise has been removed remains constant only when the master user holds the remote controller. A multimedia device that adjusts the volume of audio data. The method of claim 6,
The control unit,
When the remote controller is not included in the acquired image or the user holding the remote controller is farthest from the multimedia device than other users, the second audio data value is ignored and the audio A multimedia device that maintains a constant volume of audio data output through the output unit. The method of claim 2,
The control unit includes a voice recognition unit,
When a specific user utters a specific command, the control unit,
When a value of audio data corresponding to the specific command included in the first audio data is greater than a value of audio data corresponding to the specific command included in the second audio data by a predetermined degree or more, the specific command is ignored. (ignore) a multimedia device. The method of claim 9,
The control unit controls the display unit to output a guide message to approach the remote controller and utter a command. The method of claim 2,
The control unit includes a voice recognition unit,
When a specific user utters a specific command, the control unit,
When the value of the audio data corresponding to the specific command included in the second audio data is greater than the value of the audio data corresponding to the specific command included in the first audio data, a specific function corresponding to the specific command Running, multimedia device. The method of claim 11,
The control unit includes a voice recognition unit,
When a specific user utters a specific command, the control unit,
If audio data corresponding to the specific command is included only in the second audio data among the first audio data and the second audio data, the multimedia device ignores the specific command. The method of claim 11,
The control unit,
Detecting a specific user who uttered the specific command among users included in the acquired image based on the movement of the user's mouth detected through the acquired image,
The multimedia device, which executes a specific function corresponding to the specific command only when the specific user holds the remote controller based on the acquired image. The method of claim 13,
A memory for storing data for recognizing a face and a voice of a user registered as a master user; further includes,
The control unit, based on the data stored in the memory, when it is recognized that the audio data corresponding to the specific command corresponds to the voice of the master user, to execute a specific function corresponding to the specific command, multimedia device. The method of claim 3,
A memory for storing data for recognizing a face and a voice of a user registered as a master user; further includes,
When the master user is included among users whose gaze is facing the display unit based on the acquired image, the volume of audio data output through the audio output unit is kept constant or increased by a predetermined amount. The method of claim 2,
The control unit includes a voice recognition unit,
When a specific user utters a command, the control unit executes a specific function corresponding to the specific command only when the gaze of the specific user is facing the display unit based on the acquired image. The method of claim 3,
The control unit includes a user whose gaze is not facing the display unit than a user whose gaze is facing the display unit based on the detected gaze of the user, and the value of the second audio data is larger than the value of the first audio data. In more cases, stopping or pausing the playback of the specific content. Outputting video data and audio data of specific content;
Receiving an external sound signal and converting it into electrical first audio data;
Receiving second audio data corresponding to an external sound signal from the remote controller;
Obtaining an image from the camera; And
If the size of the value of the first audio data and the value of the second audio data are different from each other, adjusting the volume of the audio data of the output specific content based on the obtained image; Control method. The method of claim 18,
Adjusting the volume of the audio data of the output specific content,
Detecting at least one of a user, a user's gaze, a distance from the user, a user's position, a remote controller position, whether the user holds the remote controller, and a movement of the user's mouth through the acquired image; And
And adjusting the volume of the audio data of the specific content to be output based on the detection result. The method of claim 19,
Adjusting the volume of the audio data of the output specific content,
When the value of the second audio data is larger than the value of the first audio data, and based on the detected user's gaze, when there are more users who are not facing the display unit than the users whose gaze is facing the display unit, the A method of controlling a multimedia device, comprising reducing a volume of audio data of a specific content to be output by a predetermined degree.

Images