KR101799381B1 - Display apparatus and control method thereof - Google Patents

Abstract

A display device according to an embodiment of the present invention includes a display unit; A user input unit for instructing execution of a graphic user interface (GUI); Wherein when the instruction to execute the GUI is received from the user input unit during the display of the first image, the display unit displays the first image including the at least one object on the display unit, And at least one processor for processing the GUI so that the GUI is displayed on the display unit.

Description

[0001] DISPLAY APPARATUS AND CONTROL METHOD THEREOF [0002]

The present invention relates to a display device capable of displaying images of various image contents including broadcast programs and images of predetermined various supplementary services to a user and a control method thereof, To a display device and a control method therefor which are improved in the display method of the screen so as to reduce the visual fatigue of the display device.

The image processing apparatus processes image signal / image data received from the outside according to various image processing processes. The image processing apparatus can display the processed image data on the display panel of the self-contained display panel or output the processed image signal to the display device so that the image data is displayed on the other display device having the panel. That is, the image processing apparatus includes both a case including a panel capable of displaying an image and a case including no panel, as long as the apparatus can process image data. The former case is referred to as a display device in particular, and an example thereof is a TV. In the latter case, there is a set-top box.

A display device implemented with a TV, a tablet, a mobile phone or the like is provided not only to display image contents, but also to provide various additional services in accordance with development of technology and demand of users. As an example of such an additional service, the display device may be provided with various functions such as execution of a game, display of a web page, notification of a time or weather based on an installation area, notification of a current date, display of still images such as photographs, Such as the activation and use of the hardware installed in the display device, and the setting of the display device usage environment. This additional service can also be provided as a network-based service by connecting the display device in a wired or wireless communication with the external network, or as a client-based service regardless of the connection to the external network.

The user can instruct the display device to enter the additional service from the image contents by generating a specific trigger event in a state where the image contents are displayed on the display device. By the way, when the display device enters the supplementary service according to the trigger event, the image of the image content is rapidly changed from the image of the supplementary service to the image of the supplementary service. Such a sudden change causes a visual fatigue to the user, and the fatigue may become larger when the motion of the image content is large and the images to be switched are full-screen. Therefore, a structure or a method for reducing the visual fatigue that occurs when switching from the image content to the additional service needs to be applied to the display device.

A display device according to an embodiment of the present invention includes a display unit; A user input unit for instructing execution of a graphic user interface (GUI); Wherein when the instruction to execute the GUI is received from the user input unit during the display of the first image, the display unit displays the first image including the at least one object on the display unit, And at least one processor for processing the GUI so that the GUI is displayed on the display unit. The at least one processor may process the predetermined image information acquired from the first image to generate a second image corresponding to the outline of the at least one object and to convert the second image into a background image of the GUI It can be processed to be displayed. Thus, even when the first image of the image content is not displayed due to the display of the GUI, the visual feedback of the first image to the user is not interrupted, thereby preventing the user's viewing experience of the first image from being cut off as much as possible . In addition, it is possible to prevent a sudden change in the screen contents when switching from the first image to the GUI, thereby reducing the visual fatigue of the user.

Here, the at least one processor may acquire the video information changing in accordance with the playback of the first video in real time, and process the second video in accordance with the video information acquired in real time. Thus, even when the first image is not displayed, it is possible to provide the visual feedback of the first image through the change of the second image so as not to be disconnected.

The at least one processor adjusts the second image to be generated first based on a gray scale and gradually adjusts the RGB value of the second image so that the second image changes based on a predetermined color . Thus, when the first image is switched to the second image implemented based on a specific color, it is possible to prevent the user from giving visual fatigue due to a sudden change in color.

In addition, the image information may include RGB values of pixels included in one image frame of the first image. Here, the image frame is divided into a plurality of regions, a brightness value and a transparency derived corresponding to the RGB values of the pixels included in each of the divided regions are applied to the respective divided regions, The second image can be generated by adjusting the horizontal width of each divided region and applying blur filtering to each of the divided regions. Here, the at least one processor adjusts the reduction level of the horizontal width of the divided area relatively higher than the luminance value, and sets the reduction level of the horizontal width of the divided area having the relatively lower luminance value as a relative . In addition, the at least one processor may apply the blur filtering to left and right edges of upper, lower, right, and left four-directional edges of the divided area. Thereby, a smooth wave effect can be expressed so that the visual fatigue of the user is less when switching from the first image to the second image.

In addition, the GUI may include at least one for each of a plurality of predetermined services, and one or more GUIs provided for each of the services may be switched between the four directions of upward, downward, leftward, Lt; / RTI > Wherein the at least one processor is configured to switch between the GUIs of the different services in response to an instruction to move up and down from the user input unit and to switch between the GUIs in the same service in response to a left- You can switch between GUIs. Accordingly, since the service switching and the display of the image can be performed only by the intuitive movement instruction of the user in four directions of up, down, left, and right directions, the additional service can be used more easily.

According to another aspect of the present invention, there is provided a method of controlling a display device, including: displaying a first image including at least one object; Receiving a GUI execution instruction from a user during display of the first image; And displaying the GUI corresponding to an outline of the at least one object. The step of displaying the GUI may include processing a predetermined image information acquired from the first image to generate a second image corresponding to an outline of the at least one object, And displaying the second image as a background image of the GUI. Thus, even when the first image of the image content is not displayed due to the display of the GUI, the visual feedback of the first image to the user is not interrupted, thereby preventing the user's viewing experience of the first image from being cut off as much as possible . In addition, it is possible to prevent a sudden change in the screen contents when switching from the first image to the GUI, thereby reducing the visual fatigue of the user.

Here, the step of displaying the GUI may include a step of acquiring the video information changing in accordance with the reproduction of the first video in real time and processing the second video to be changed in accordance with the video information acquired in real time . Thus, even when the first image is not displayed, it is possible to provide the visual feedback of the first image through the change of the second image so as not to be disconnected.

Here, the displaying the GUI may include adjusting the second image to be generated first based on a gray scale, and adjusting an RGB value of the second image so that the second image changes based on a predetermined color, And may include steps of progressively changing. Thus, when the first image is switched to the second image implemented based on a specific color, it is possible to prevent the user from giving visual fatigue due to a sudden change in color.

In addition, the image information may include RGB values of pixels included in one image frame of the first image. The generating of the second image may include: dividing the image frame into a plurality of regions; Applying a luminance value and a transparency derived corresponding to RGB values of the pixels included in each of the divided regions to each of the divided regions; Adjusting widths of the respective divided regions corresponding to the luminance values; And applying blur filtering to each of the divided areas. Here, the step of adjusting the width of each of the divided areas corresponding to the brightness value may include: adjusting a reduction level of the width of the divided area relatively high, wherein the brightness value is relatively high; And adjusting the reduction level of the horizontal width of the divided area relatively low in the luminance value. The step of applying blur filtering to each of the divided areas may include applying the blur filtering to left and right edges of the upper, lower, left, and right four-directional edges of the divided area. Thereby, a smooth wave effect can be expressed so that the visual fatigue of the user is less when switching from the first image to the second image.

The GUI may include at least one of a plurality of predetermined services, and the at least one GUI provided for each of the services may be a dictionary Can be set. Switching between the GUIs of the services different from each other in response to a vertical movement instruction from the user; And switching between the GUIs in the same service in response to a leftward and rightward movement instruction from the user. Accordingly, since the service switching and the display of the image can be performed only by the intuitive movement instruction of the user in four directions of up, down, left, and right directions, the additional service can be used more easily.

1 is an exemplary view of a display device according to a first embodiment of the present invention,
FIG. 2 is an exemplary view showing a state in which a service image is switched from a content image to a service image when a service switching event occurs in the display device of FIG. 1;
3 is a block diagram of a display device according to a second embodiment of the present invention.
4 is a block diagram showing the configuration of a signal processing unit in the display device of FIG. 3;
5 to 13 are views illustrating a process of generating a background image of an image of an additional service by the display device of FIG. 3;
FIG. 14 is a flowchart showing a process of displaying a service image by the display device of FIG. 3;
FIG. 15 is a flowchart showing a process of generating a background image of a service image by the display device of FIG. 3;
16 is an exemplary view of a user input unit according to the third embodiment of the present invention,
17 is an exemplary view of a user input unit according to a fourth embodiment of the present invention,
18 is an exemplary view showing a content mapping principle of an additional service image according to the fifth embodiment of the present invention;
19 is an exemplary view showing a principle of switching from a title screen to a content screen in one category of an additional service in a display device according to the fifth embodiment of the present invention,
20 is an exemplary view showing a principle in which a service image is switched and displayed between two categories of an additional service in a display device according to a fifth embodiment of the present invention;
21 is a flowchart showing a process of displaying a service image of an additional service by the display apparatus according to the fifth embodiment of the present invention;
22 is an exemplary view showing a UI menu overlaid on a content image in a display device according to a sixth embodiment of the present invention;
23 is a flowchart showing a process of displaying the UI menu by the display device of FIG. 22;
24 is an exemplary view showing a service image displayed on a content image in a PIP format in a display apparatus according to a seventh embodiment of the present invention,
25 is a flowchart showing a process of displaying a service image by the display device of FIG. 24,
26 is an exemplary view of a screen displayed in response to user input by the display device according to the eighth embodiment of the present invention,
27 is an exemplary view of a UI displayed by a display device according to an eighth embodiment of the present invention in response to execution of a TV icon;
28 is an exemplary view of a UI displayed by a display device according to an eighth embodiment of the present invention in response to execution of an app icon;
29 is an exemplary view of a UI displayed by a display device according to an eighth embodiment of the present invention in response to execution of a speaker icon;
30 is an exemplary view showing a UI showing an output state of a speaker according to an eighth embodiment of the present invention;
31 is an exemplary view of a UI displayed by a display device according to an eighth embodiment of the present invention in response to execution of a photo icon;
32 is an exemplary view showing an image displayed by a display device according to an eighth embodiment of the present invention,
33 is an exemplary view of a UI displayed by a display device according to an eighth embodiment of the present invention in response to execution of a clock icon;
34 is an exemplary view of a UI displayed by a display device according to an eighth embodiment of the present invention in response to execution of a background setting icon;
35 is an exemplary view of a UI provided by a display device according to a ninth embodiment of the present invention for function adjustment;

Hereinafter, embodiments according to the present invention will be described in detail with reference to the accompanying drawings. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to like elements throughout.

If the term includes an ordinal such as a first component, a second component, or the like in the embodiment, such term is used to describe various components, and the term is used to distinguish one component from another And these components are not limited in meaning by their terms. The terms used in the embodiments are applied to explain the embodiments, and do not limit the spirit of the present invention.

In addition, in the embodiment, only the configurations directly related to the concept of the present invention will be described, and the other configurations will not be described. However, it is to be understood that, in the implementation of the apparatus or system to which the spirit of the present invention is applied, it is not meant that the configuration omitted from the description is unnecessary. It should be understood that terms such as " comprise "or" have "in the embodiments are intended to specify that there is a feature, number, step, operation, , ≪ / RTI > an operation, an element, or a combination thereof.

1 is an exemplary view of a display device 100 according to a first embodiment of the present invention.

As shown in FIG. 1, the display device 100 according to the first embodiment is implemented as a TV, but can be implemented as a general electronic device capable of displaying images such as a tablet, a mobile phone, a multimedia player, an electronic frame, have. In addition, the display device 100 may be in a form of being installed in a place, as well as in a mobile form that the user freely carries and uses. Also, the idea of the present invention can be applied not only to the case where the apparatus itself displays an image, but also to an image processing apparatus having a separate monitor for displaying an image, such as the display apparatus 100.

The display device 100 processes data of image contents received from the outside to display an image. The video contents are transmitted from the broadcasting apparatus 10 of the broadcasting station to the RF broadcasting signal, transmitted from the server 20 via the network as data packets, or reproduced from the multimedia player 30 locally connected to the display apparatus 100 And transmitted. Alternatively, the image contents may be stored in the display device 100 as digital data.

The display apparatus 100 has a user input unit 130 for the user to select image contents from any one of the image sources 10, 20 and 30, for example, A remote controller may be implemented as the user input unit 130. [ When the user selects one of the video sources 10, 20 and 30 through the user input unit 130, the display device 100 processes the video contents received from the selected video sources 10, 20 and 30, And displays an image of the content. For example, when the user selects a specific broadcast channel through the user input unit 130, the display apparatus 100 displays the broadcast image by tuning the broadcast signal received from the broadcast equipment 10 of the broadcast station to the broadcast channel .

The functions provided by the display device 100 may not only display image contents received from the image sources 10, 20 and 30 as images, but may also provide various additional services to the user. Here, the supplementary service refers to various services provided by the display device 100 in addition to basic services for processing image contents and displaying them as images, and is not limited to a specific service type. Further, the supplementary service may be provided on a network basis or on a client basis.

Examples of the additional service include a service for setting a device environment or a user environment of the display device 100, a service for notifying date, time, weather, etc. based on an installation area or a usage area of the display device 100, A service for displaying an image such as a photograph and a picture which are locally stored or a service for displaying a specific web page through a web browser, and the like.

While the image of the image content is displayed, the user can instruct the user through the user input unit 130 to display an image of an additional service. Accordingly, the display apparatus 100 operates so that an image of an additional service is displayed instead of the image content.

FIG. 2 is an exemplary view showing a state in which the service image 220 is switched from the content image 210 when a service switching event occurs in the display device 100. FIG.

As shown in FIG. 2, the display device 100 processes the data of the image content received from the outside to display the content image 210. When the content image 210 is displayed, the display device 100 detects an event instructing to switch to the additional service. As an example of such an event, it is possible to input an instruction to request the user to provide an additional service for notifying the time zone of the area where the display device 100 is currently located.

The display device 100 switches from the content image 210 to the service image 220 according to an event and displays the service image. The service image 220 represents the time zone corresponding to the current position. In the present embodiment, the content image 210 and the service image 220 are displayed in full screen on the entire display area where the display device 100 can display an image. The user can receive a desired service through the service image 220 displayed on the display device 100.

Here, when the user switches from the content image 210 to the service image 220, the display device 100 performs the conversion without any visual effect between the content image 210 and the service image 220 . However, there are a few things to be perceived by the user in such a case.

When the user directly switches from the content image 210 to the service image 220, the user can feel the visual fatigue due to the rapid change of the screen. Particularly, when the display screen of the display device 100 is large, when the content image 210 and the service image 220 are displayed in full screen, when the amount of motion of the content image 210 is large, Is increased.

In addition, since the content image 210 is not displayed while the service image 220 is being displayed, the viewing experience of the content image 210 is disconnected from the viewpoint of the user. In other words, since the visual feedback of the content image 210 is disconnected from the user's point of view, the user may be a factor of reluctant to use the supplementary service.

A second embodiment for overcoming the problem of the first embodiment is proposed, and a more detailed description of the second embodiment will be described later.

3 is a configuration block diagram of a display device 100 according to a second embodiment of the present invention. The display device 100 according to the second embodiment has substantially the same basic structure as the first embodiment described above.

3, the display apparatus 100 includes a signal receiving unit 110 for receiving a video signal from the outside, a display unit 120 for displaying a video signal received by the signal receiving unit 110 as an image, A storage unit 140 for storing data and information, a display unit 120 for processing a video signal so that an image is displayed on the display unit 120, And a signal processing unit 150 for controlling the signal processing unit.

The signal receiving unit 110 receives image signals transmitted from various image sources 10, 20 and 30 (see FIG. 1). The signal receiving unit 110 can not only receive an image signal transmitted from the outside, but can also perform bidirectional communication by transmitting a signal to the outside. The signal receiving unit 110 is implemented by a communication port or an aggregate of communication modules corresponding to a plurality of communication standards, and the supportable protocol and communication connection objects are not limited to any one type or format. For example, the signal receiving unit 110 may include an RFIC (not shown) for receiving an RF signal, a WiFi communication module (not shown) for wireless network communication, an Ethernet module (not shown) for wired network communication, And a USB port (not shown) for local connection, etc.

The display unit 120 displays a video signal processed by the signal processing unit 150 as an image. For example, the display unit 120 displays a tuned broadcast signal output from the signal processing unit 150 as a broadcast image. The method of implementing the display unit 120 is not limited, and examples thereof include a liquid crystal, a plasma, a light-emitting diode, an organic light-emitting diode, surface-conduction electron-emitter, carbon nano-tube, nano-crystal, and the like.

The display unit 120 may additionally include an additional configuration depending on the implementation method of the panel, in addition to the panel structure for displaying an image. For example, in the liquid crystal type, the display unit 120 includes a liquid crystal display panel (not shown), a backlight unit (not shown) for supplying light to a liquid crystal display panel (not shown), a liquid crystal display panel (Not shown) for driving the panel driving board (not shown).

The user input unit 130 transmits various preset control commands or information to the signal processing unit 150 according to a user's operation or input. The user input unit 130 informs various events generated by the user's operation according to the user's intention and transmits the information to the signal processing unit 150. The user input unit 130 may be realized in various forms according to the information input method. For example, the user input unit 130 may include a key / button installed outside the display device 100, a separate remote controller A touch screen integrated with the display unit 120, or an input device provided to communicate with the display device 100, or the like.

The storage unit 140 stores various data according to the processing and control of the signal processing unit 150. The storage unit 140 is accessed by the signal processor 150 to read, write, edit, delete, and update data. The storage unit 140 may be implemented as a nonvolatile memory such as a flash memory or a hard disk drive so as to store data regardless of whether the system power of the display apparatus 100 is provided or not. do.

The signal processing unit 150 performs various processes on the data / signal received by the signal receiving unit 110. When a video signal is received by the signal receiving unit 110, the signal processing unit 150 performs an image processing process on the video signal and outputs the broadcast signal on which the process is performed to the display unit 120, Is displayed.

The type of the image processing process performed by the signal processing unit 150 is not limited. For example, the image processing unit 150 may perform de-multiplexing to divide an input stream into respective lower streams of video, audio, and additional data, De-interlacing to convert a video stream of an interlaced format into a progressive format, scaling to adjust a video stream to a preset resolution, image quality enhancement, For example, noise reduction, detail enhancement, frame refresh rate conversion, and the like.

Since the signal processor 150 can perform various processes according to the types and characteristics of signals or data, it is not possible to limit the processes that can be performed by the signal processor 150 to the image processing processes, Data can not be limited to being received by the signal receiving unit 110 only. For example, when the user's utterance is input to the display device 100, the signal processor 150 can process the utterance according to a predetermined voice processing process. The signal processor 150 may be implemented as a system-on-a-chip (SOC) in which various functions are integrated or individual chip-sets capable of independently performing each of these processes are mounted on a printed circuit board Image processing board (not shown).

The hardware configuration of the display device 100 may vary in detail depending on the form in which the display device 100 is implemented and the functions supported by the display device 100. For example, a configuration for tuning a broadcast signal to a specific frequency may be required if the display device 100 is a TV, but may be excluded if the display device 100 is a tablet.

Hereinafter, a specific configuration of the signal processing unit 150 when the display device 100 is a TV will be described.

4 is a block diagram of the configuration of the signal processing unit 150. As shown in FIG. The signal processing unit 150 in this figure shows only the basic configuration actually implemented in the product, and further includes configurations other than those described in this embodiment in actual products.

As shown in FIG. 4, the signal receiving unit 110 includes a tuner 111 for tuning a received broadcast signal to a specific frequency. The signal processing unit 150 includes a demux 151 for demultiplexing a broadcast signal transmitted from the tuning unit 111 into a plurality of sub signals and outputting the sub signals, A scaler 153 for scaling the video signal among the decoded sub-signals and outputting the scaled video signal to the display unit 120, A central processing unit (CPU) 154 for performing control and a buffer 155 for temporarily storing signals or data while being processed in the signal processing unit 150.

The tuning unit 111 tunes a broadcast signal at a frequency of a specified channel and converts the broadcast signal into a transport stream when a broadcast signal is received in an RF antenna (not shown). The tuning unit 111 converts a high-frequency carrier wave transmitted from an antenna (not shown) into an intermediate frequency band and converts it into a digital signal to generate a transport stream. To this end, the tuning unit 111 may have an A / D converter (not shown). Depending on the design, an A / D converter (not shown) may be connected to a demodulator (not shown) . ≪ / RTI >

The demux 151 or demultiplexer basically acts in opposition to the multiplexer (not shown). That is, the demux 151 connects one input terminal to a plurality of output terminals, and plays a role of distributing a stream input to the input terminal to each output terminal in accordance with the selection signal. For example, if there are four output stages for one input stage, the demux 151 can select each of the four output stages by combining the states of the two selection signals having states of 0 or 1.

When the demux 151 is applied to the display device 100, the demux 151 divides the transport stream delivered from the tuning unit 111 into sub-signals of a video stream, a voice stream, and an additional data stream, and outputs the sub-signals to the respective output terminals.

For example, the demultiplexer 151 demultiplexes the PID (packet), which is an identifier assigned to each packet in the transport stream, identifies the transport stream as each sub-signal. Sub-signals for each channel in the transport stream are independently compressed and packetized, and packets corresponding to one channel are assigned to the same PID, thereby being distinguished from packets of other channels. The demux 151 classifies packets by PID in the transport stream and extracts sub-signals having the same PID.

The decoder 152 decodes the sub signals output from the demux 151, respectively. In this figure, the decoder 152 is represented as one, but the decoder 152 may be provided with a plurality of configurations to individually decode the respective subsignals. That is, the decoder 152 may include a video decoder for decoding the video signal, a voice decoder for decoding the audio signal, and an additional data decoder for decoding the additional data.

Since the sub signal transmitted to the decoder 152 is encoded in a specific format, the decoder 152 performs a decoding process for restoring the pre-encoding state of the corresponding sub signal by performing the encoding process for the sub signal inversely do. Therefore, if the sub-signal output from the demultiplexer 151 is uncompressed and not encoded, the sub-signal is not processed by the decoder 152 and is transmitted to the scaler 153 or the corresponding sub- Decoder 152 may be bypassed and delivered to the scaler 153. [

The scaler 153 scales the video signal decoded by the decoder 152 according to the resolution of the display unit 120 or a separately specified resolution. The scaled image signal is displayed on the display unit 120 as an image.

The CPU 154 performs a central operation for operating all the components in the signal processor 150, and basically plays a central role in interpreting and calculating data. The CPU 154 internally includes a processor register (not shown) in which instructions to be processed are stored, an arithmetic logic unit (ALU) (not shown) responsible for comparison, determination, and operation, A control unit (not shown) for internally controlling the CPU 154 for proper execution, an internal bus BUS (not shown), a cache (not shown), and the like.

The CPU 154 performs operations necessary for operations of the respective components of the signal processor 150 such as the demultiplexer 151, the decoder 152 and the scaler 153. Depending on the design method of the signal processing unit 150, the signal processing unit 150 may be configured to operate without any data operation by the CPU 154 or by a separate microcontroller (not shown).

The buffer 155 temporarily stores data to be processed by the respective components of the signal processing unit 150 when the signal processing unit 150 processes the broadcast signal. The buffer 155 is a nonvolatile memory unlike the storage unit 140 (see FIG. 4) in which data is held even when the system power is off, since the data is temporarily loaded at the time of signal processing. Although the input / output of the buffer 155 is described as being performed through the CPU 154 in the figure, each configuration of the signal processing unit 150 may be provided to connect to the buffer 155 without passing through the CPU 154. [

Under this structure, the display device 100 processes the received image content to display the image of the image content. The display apparatus 100 displays an image of the supplementary service instead of the image of the image content when the execution command of the supplementary service is received from the user input unit 130 while the image of the image content is displayed on the display unit 120. [

Hereinafter, a method of generating the background image of the image of the additional service by the display apparatus 100 according to the present embodiment will be described.

FIGS. 5 to 13 illustrate a process in which the display device 100 according to the second embodiment generates a background image of an image of an additional service. In the following embodiments, the image displayed by the image signal of the image content is referred to as a content image, and the image of the additional service is referred to as a service image.

While the service image including the generated background screen is displayed according to the embodiment, the display device 100 does not display the content image, but continues to process the image content. The reason for processing the data of the video that is not displayed will be described later.

As shown in FIG. 5, the display device 100 captures a one-image frame 310 of a content image when an event instructing execution of the supplementary service occurs while displaying the content image. Although this process is basically computed and executed by the CPU 154 (see FIG. 4), a separate configuration for executing the process may be provided in the display device 100. FIG.

In addition, the data in the following processing including the captured image frame 310 is stored in the buffer 155 (see FIG. 4).

As shown in FIG. 6, the display apparatus 100 divides an image frame 310 captured from a content image into a grid 320 in the form of a matrix of M * N. Here, M and N are preset values, and various values may be applied according to the resolution of the image frame 310. [

The grid 320 divides the image frame 310 into a plurality of rectangles 330 and each rectangle 330 includes a plurality of pixels.

As shown in FIG. 7, the display device 100 extracts RGB values of each of a plurality of pixels included in each quadrangle 350 and stores the extracted RGB values. Then, the display device 100 converts all the image frames 340 to gray scale.

As shown in FIG. 8, the display apparatus 100 calculates luminance values and transparency based on RGB values extracted and stored for each quadrangle 350 (see FIG. 7). The display device 100 reflects the calculated luminance value and transparency on the corresponding quadrangle 360. FIG. 8 shows only a part of the area shown in FIG. 7 for the sake of brevity.

Here, the following equation is used to calculate the luminance value Br and the transparency alpha of the corresponding quadrangle 360 based on the RGB values of the pixels of each quadrangle 360 as follows.

[Equation 1]

^{Br = sqrt (r 2 * 0.241} + g 2 * 0.691 + b 2 * 0.068); (0 < Br < 255)

alpha = 1-Br / 255; (0 < alpha < 1)

In the above equation, the variables r, g, and b are an average value of the R values of the pixels in the quadrangle 360, an average value of the G values, and an average value of the B values. sqrt is a function that returns the square root of the positive of a given numeric value.

As shown in FIG. 9, the display device 100 adjusts the width of each of the rectangles 361 and 362 in correspondence with the luminance value calculated for each of the rectangles 361 and 362. At this time, the display device 100 does not adjust the height of each of the rectangles 361 and 362.

The display apparatus 100 determines the width w of each of the rectangles 361 and 362 according to the following equation.

&Quot; (2) &quot;

w = 1-Br / 255

Assuming that the maximum value of w is 1, the range of w is 0 < w < 1. According to this equation, the widths of the rectangles 361 and 362 decrease as the luminance value increases, and the widths of the rectangles 361 and 362 increase as the luminance value decreases.

For comparison, an example of a first rectangle 361 having a relatively low luminance value and a second rectangle 362 having a relatively high luminance value will be exemplified. Before the adjustment of the width, the width w1 of the first rectangle 361 and the width w2 of the second rectangle 362 show the same value.

If the width of the first rectangle 361 is adjusted from w1 to w1 'and the width of the second rectangle 362 is adjusted from w2 to w2' based on the above equation, the width w1 'is compared with the width w2' It represents a relatively large value. That is, as the luminance value increases, the amount of reduction in the width increases.

As shown in FIG. 10, when the above-described adjustment process is applied to all of the squares in the image frame 370, each of the squares of the image frame 370 has the same height and appears to have a different width according to the luminance value. Specifically, a rectangle having a relatively low luminance value has a relatively wide width, and a rectangle having a relatively high luminance value has a relatively narrow width.

As shown in FIG. 11, the display device 100 applies blur processing only to the vertical lines of each rectangle in the image frame 380. The blur processing only for the vertical lines of the rectangle means that the upper and lower edges of the rectangle are subjected to the blur processing only on the left and right edges of the rectangle without blur processing.

Blurring, or blurring, is a filtering technique that blurs or softens an image by removing fine detail of the image. The detailed part of the image referred to here refers to a part where a change of an image occurs rapidly in a pixel unit, that is, an edge part of an object in the image. When the image is viewed in the frequency dimension, the edge portion of the object appears as a high-frequency component, and blurring filters the high-frequency component, resulting in blurring of the edge portion of the object in the image. That is, blurring is essentially low-pass filtering.

The blurring method can be applied to various forms in actual implementation steps, and a detailed description thereof will be omitted in the present embodiment. Various techniques may be applied to the blurring applied in the present embodiment, for example, a Gaussian blur may be applied to the image frame 380. Gaussian blur is a form of image blurring filter using Gaussian function. The Gaussian function in one dimension is expressed by the following equation.

&Quot; (3) &quot;

The Gaussian function in two dimensions is expressed by the following equation.

&Quot; (4) &quot;

x is the distance from the origin on the horizontal axis, y is the distance from the origin on the vertical axis, and σ is the standard deviation of the Gaussian distribution.

The display device 100 repeats the above process while displaying the service image, and performs the process for the content image reproduced in the background for a preset number of times per second in real time. That is, the display device 100 captures an image frame in real time from a content image that is reproduced but not displayed, and displays the background image of the service image by executing the above-described process for the captured image frame.

The number of times the display apparatus 100 executes the process per second may be various values in the designing stage, for example, 15 fps, or 15 times per second.

As shown in FIG. 12, the display device 100 sequentially changes the RGB values of each quadrangle in a process unit so that the background screen 390 becomes a predetermined color in the process of repeating the process.

Here is an example of changing this method of changing one black rectangle to orange. The RGB value of the black square is (0, 0, 0) and the RGB values of the orange color are (152, 83, 44) If there is a rectangle, the RGB values are increased by 1 each to the target value until the RGB value of the rectangle becomes (152, 83, 44).

That is, every time the process is repeated, the RGB values of the rectangle are (0,0,0) -> (1,1,1) -> (2,2,2) -> ... -> (44,44 , 44) -> ... -> (83,83,44) -> ...-> (152,83,44). When the display device 100 firstly reaches a specific value among the RGB values, the display device 100 maintains the value and changes the remaining values in units of processes.

On the other hand, with respect to the rectangle having the RGB values (255, 255, 255), the display device 100 outputs RGB values to the target values one by one each time the process proceeds until the RGB values of the corresponding rectangle become (152, To a close proximity.

Then, the display apparatus 100 increases the blur value to be reflected in the corresponding image frame by 1 every time the process is repeated from the first one to a predetermined value, for example, 20. The higher the blur value, the greater the blur effect, and the display device 100 increases the blurring weight each time the process is repeated.

As shown in FIG. 13, when the process of the image frame unit is repeated, the boundaries between vertically adjacent rectangles in the image frame disappear and appear as one line. Accordingly, the effect that the content image gradually changes abstractly becomes the background image 400 based on the specific color of the service image.

The background screen 400 changes in real time corresponding to the reproduction of the content image while the service image is being displayed, and the internal lines continuously change and display in the form of a waving wave. Since the real-time change of the background screen 400 corresponds to the change of the image frame of the content image, the visual feedback of the content image can be continuously provided to the user even though the service image is displayed and the content image is not displayed . Further, when switching from a content image to a service image or switching from a service image to a content image, the visual fatigue due to a sudden change in the image can be reduced.

Hereinafter, a process of displaying the service image by the display apparatus 100 according to the present embodiment will be described.

14 is a flowchart showing a process of displaying the service image by the display device 100. [

As shown in FIG. 14, in step S110, the display device 100 receives an image content signal. In step S120, the display device 100 processes the image content signal to reproduce and display the content image.

In step S130, the display device 100 determines whether an additional service execution instruction has been issued.

If it is determined that an additional service execution instruction has been issued, the display device 100 acquires the image frame at the current time from the content image to be reproduced in step S140. In step S150, the display device 100 acquires preset image information from the image frame. In step S160, the display device 100 modifies the image information according to a predetermined algorithm to generate a background image. In step S170, the display device displays a service image including a background screen.

In step S180, the display device 100 determines whether a predetermined time has elapsed after the service image is displayed.

If it is determined that the service image is displayed and the predetermined time has elapsed, the display apparatus 100 proceeds to step S140 and repeats the background screen creation process.

Accordingly, the background image of the service image changes in accordance with the image information of the content image to be reproduced with the lapse of time.

Hereinafter, a process of generating a background image of a service image by the display apparatus 100 according to the present embodiment will be described.

15 is a flowchart showing a process of the display device 100 generating a background image of a service image.

As shown in FIG. 15, in step S210, the display device 100 captures an image frame from the content image.

In step S220, the display device 100 divides the image frame into a grid including a predetermined number of squares.

In step S230, the display apparatus 100 calculates the luminance value and the transparency based on the RGB values of the pixels included in the respective quadrangles.

In step S240, the display device 100 applies the calculated luminance value and transparency to the corresponding quadrangle.

In step S250, the display device 100 adjusts the horizontal width of each rectangle corresponding to the luminance value.

In step S260, the display device 100 performs blurring on the vertical lines of each rectangle.

The display device 100 performs the above process in real time while the service image is displayed. As the process repeats, the display device 100 changes the RGB values of each quadrangle so that it approaches RGB values of a predetermined color, and also raises the blur value to a preset value. Accordingly, the display apparatus 100 can display the background image of the service image that changes in accordance with the image information of the content image to be reproduced.

That is, if the content image includes one or more objects, the background image of the service image is expressed by a part of the image information of the object, for example, a screen corresponding to the outline of the object.

Hereinafter, an implementation example of an additional service using the generated background screen according to the above embodiment will be described.

The user can move the cursor displayed on the display device 100 or instruct the UI displayed on the display device 100 to move in a specific direction by operating the user input unit 130 (see Fig. 3). In this case, the user can easily perceive a specific direction with the easiest and intuitive way by judging from the four directions of up, down, left and right. When the cursor advances in either direction with respect to the origin, the direction can be displayed two-dimensionally, so that it can be expressed on the horizontal axis and the vertical axis. Therefore, when the horizontal direction corresponding to the horizontal axis and the vertical direction corresponding to the vertical axis are taken as a reference, the user can easily recognize the two-dimensional movement direction.

In consideration of this point, the user input unit 130 (see FIG. 4) provides an input environment in which the user can designate four directions of up, down, left, and right directions.

16 is an exemplary diagram of a user input unit 410 according to a third embodiment of the present invention.

As shown in FIG. 16, the user input unit 410 is implemented as a remote controller so that the user can easily carry it. The user input unit 410 includes an upward direction key 411, a downward direction key 412, a left direction key 413, and a right direction key 414 corresponding to the four directions. Each of the direction keys 411, 412, 413, and 414 is a button that is physically or mechanically mutually separated. When the user presses any one of the buttons, a move command in the direction corresponding to the button is generated from the user input unit 410.

The command generated from the user input unit 410 is transmitted to the display device.

17 is an exemplary diagram of a user input unit 420 according to a fourth embodiment of the present invention.

As shown in FIG. 17, the user input unit 420 includes a touch pad 421 that the user inputs by touching with a finger or a stylus pen (not shown). Of course, the user can drag and input in various directions on two dimensions through the touch pad 421, but the direction in which the user can most accurately input is the most intuitive four directions.

Alternatively, the user input unit 420 may include a motion sensor (not shown) for sensing its own motion, thereby generating a command corresponding to the direction in which the user shakes the user input unit 420.

As described above, the user can recognize and input the four directions in the most intuitive manner. Therefore, it is preferable from the viewpoint of ease of use that the category-specific mappings of additional services displayed on the display device correspond to the four directions.

FIG. 18 is an exemplary view showing a content mapping principle of an additional service image according to the fifth embodiment of the present invention.

18, the additional service 500 may include, for example, a CLOCK representing a current time, a WEATHER representing a current local weather, a CALENDAR indicating a current date, a PHOTOS displaying an image such as a photograph or a picture, And a SPEAKER that sets the speaker to be activated. This category enumerates one of various implementations to explain the concept of the present invention, and the category does not limit the idea of the present invention.

Each of the squares shown in the figure are service images displayed on the display device. The horizontal line indicates the category of the additional service, and the vertical line indicates the function of the service image.

Service images 510 and 515 of the first horizontal line are start categories indicating the first entry of the additional service and service images 520, 521 and 525 of the second horizontal line are CLOCK category, service images 530 and 531 of the third horizontal line 532 and 535 of the fifth horizontal line, the CATALEND category of the service images 540, 541, 542 and 545 of the fourth horizontal line, the service images 550 and 551 of the fifth horizontal line, the PHOTOS category, 560, 561) represents the SPEAKER category.

Settings are display screens 515, 525, 535 and 545 for setting display environments of categories, title pages are title screens 510, 520, 530, 540, 550 and 560 indicating service entry of a category, Are screens 521, 531, 532, 541, 542, 551, and 561 that contain actual information for providing a category service.

The background image as described in the above embodiment is applied to these service images. Of course, all service images may include a background image to which the idea of the present invention is applied, or only certain screens such as title screens 510, 520, 530, 540, 550, and 560 in the category And may include a background screen. However, the backdrop for each category is based on a different color for the distinction between the categories.

The arrows around the service image indicate which service image is switched and displayed when the user instructs the user to move in one of the four directions of up, down, left, and right while the service image is displayed on the display device. 'Loop to First' means that the service image is switched to the first service image of the corresponding line, 'Loop to Last' means that the service image is switched to the last service image of the corresponding line, and 'Loop to Settings' The screen is switched to the setting screen of the corresponding horizontal line.

For example, when the user first enters the supplementary service, the start title screen 510 is switched from the content image. In this state, when the user instructs movement in the left direction or the right direction, a setting screen 515 for adjusting settings applied to the entire additional service is displayed. On the other hand, when the user instructs movement in the downward direction, Screen 520 is displayed.

When the user instructs movement in the right direction in the state that the title screen 520 of the CLOCK category is displayed, the content screen 521 of the CLOCK category indicating the current time is displayed. On the other hand, when the user instructs movement in the downward direction, The title screen 530 is displayed.

When an instruction to execute the supplementary service is generated while the content image is being displayed on the display device, the display device may initially display the supplementary service start screen 510 and then switch the service image according to the direction instruction of the user.

Alternatively, when an execution instruction of a certain category is generated in a state in which an execution instruction is previously provided for each category of the additional service, the display device may display the title screen 520, 530, 540, 550, 560 of the category have.

19 is an exemplary diagram showing a principle of switching from the title screen 530 of one category of the additional service to the content screen 531a.

As shown in FIG. 19, for example, if the user enters the title screen 530 of the WEATHER category according to the user's instruction, the display device switches to the weather information image 531a according to one of the two methods.

One method is to display the weather information image 531a automatically after a predetermined time from the display time of the title screen 530 even if the user does not have any user input while the title screen 530 is displayed. For example, the display device displays the title screen 530 and displays the weather information image 531a after one second elapses.

Another method is that the display device displays the weather information image 531a in accordance with the user's rightward direction indication. As described in the map in FIG. 18, the weather information image 531a is mapped in the right direction of the title screen 530, and the display device displays the service image based on the map.

20 is an exemplary diagram showing a principle in which service images are switched and displayed between two categories of supplementary services.

As shown in Fig. 20, the display device switches or moves between categories in the additional service in accordance with the user's direction of upward and downward movement. Further, the display device switches or moves between the title screen 520 and the information screens 521a and 521b including the information provided by the category service in one category in accordance with the user's left / right direction movement instruction.

Switching between categories is possible between title screens 520 and 530 of the category. Since it is difficult for the user to recognize the mapping structure of the service image as shown in FIG. 18, it is difficult to determine which category of information screen is displayed This is because it is not easy for the user to predict.

However, if the information screens 521a, 521b, 531a, and 531b among the categories have a relation, categories may be switched between the information screens 521a, 521b, 531a, and 531b.

For example, the first information screen 521a of the CLOCK category includes the time information of the area 'City 1', the first information screen 531a of the WEATHER category includes the weather information of the area of 'City 1' do. As described above, the first information screen 521a and the first information screen 531a each include service information of different categories, but each information is all about the same area. As described above, since there is a relationship between the two screens 521a and 531a, the display device switches to the first information screen 531a if the first information screen 521a is displayed and there is an instruction to move the user downward .

Similarly, the second information screen 521b of the CLOCK category includes time information of the area 'City 2', and the second information screen 531b of the WEATHER category includes weather information of the area of 'City 2' . The second information screen 521b and the second information screen 531b are related to the same area 'City 2'. In the display device, when the second information screen 521b is displayed and the user moves downward The second information screen 531b is displayed.

As described above, the display device according to the present embodiment switches the service image of the supplementary service in response to the user's instruction to move in the up, down, left, and right directions, and displays the service image so that the user can intuitively use the supplementary service.

Hereinafter, a process in which the display apparatus displays the service image of the additional service according to the fifth embodiment will be described.

21 is a flowchart showing a process of the display device displaying a service image of an additional service.

As shown in FIG. 21, in step S310, the display device receives an additional service progress instruction from the user.

In step S320, the display device calls up a map of the service image corresponding to the moving direction. This map means data defining the correspondence relationship between the service images according to the four directions of up, down, left, and right as shown in FIG.

In step S330, the display device generates a background screen based on the image information of the content image. The method of generating the background screen is as described in the foregoing embodiment.

In step S340, the display device displays a service image including a background screen. The displayed service image may be a start screen informing the entry of the additional service or a title screen of a specific service category.

In step S350, the display device determines whether one of four directions is received from the user.

If it is determined in step S360 that the movement instruction in one of the four directions is received from the user, the display device selects a service image corresponding to the corresponding direction on the map of the service image.

In step S370, the display device switches to the selected service image.

According to the embodiments described above, the display device can provide an intuitive environment so that the user can easily use additional services.

On the other hand, in the above-described embodiment, since the content image and the service image are all displayed in full screen on the display device, the content image is not displayed when the service image is displayed. In addition, in the above-described embodiment, the image generated based on the image information of the content image is used as the background image of the service image. However, the spirit of the present invention is not limited to these embodiments, and can be modified and applied in various forms.

22 is an exemplary view showing a UI menu 670 overlaid on the content image 660 in the display device 600 according to the sixth embodiment of the present invention.

As shown in FIG. 22, the display apparatus 600 according to the sixth embodiment displays a content image 660 on the display unit 620. The display device 600 displays the UI menu 670 on the content image 660 when a display instruction of the UI menu 670 is received from the user input unit 630 while the content image 660 is displayed on the display unit 620. [ Overlay.

Here, if the content image 660 is displayed on the display unit 620 in a full screen, the UI menu 670 is displayed in a form in which a part of the content image 660 is covered.

In displaying the UI menu 670, the display device 600 generates the background of the UI menu 670 by acquiring image information of the content image 660 and transforming the image information according to a predetermined algorithm. Since these algorithms can be applied to the above embodiments, a detailed description will be omitted.

Here, the image information of the content image 660 is image information of the pixel region overlaid by the UI menu 670 among the whole pixel regions of the content image 660.

According to this embodiment, the UI menu 670 is displayed together while the content image 660 is continuously displayed, and the background of the UI menu 670 changes in real time corresponding to the change of the content image 660. Accordingly, the user can perceive the movement of the content image 660 blocked by the UI menu 670 through a change in the background of the UI menu 670 in real time.

In the present embodiment, a screen generated based on the image information of the content image 660 is used as the background of the UI menu 670, but it may be used as an icon of the UI menu, a selection item, or the like depending on the design method.

23 is a flowchart showing a process of displaying the UI menu 670 by the display apparatus 600 according to the sixth embodiment.

As shown in FIG. 23, in step S410, the display device 600 displays a content image.

In step S420, the display device 600 determines whether a display instruction of the UI menu has been received.

If it is determined that the display instruction of the UI menu 670 has been received, the display device 600 specifies one area of the content image to be overlaid on the UI menu in step S430.

In step S440, the display device 600 generates a background screen based on the image information of the specified area.

In step S450, the display device 600 displays the UI menu including the generated background image overlaid on the content image.

24 is an exemplary view showing a service image 770 displayed on a content image 760 in a PIP format in a display device 700 according to a seventh embodiment of the present invention.

24, when an additional service execution instruction is received from the user input unit 730 while the content image 760 is displayed on the display unit 720, the display device 700 displays the service image 770 ).

In the case of the above embodiment, since the content image and the service image are displayed in full screen, the display device displays the service image by switching from the content image to the service image.

In contrast, in the present embodiment, the display apparatus 700 displays the service image 770 in the content image 760 in a picture-in-picture (PIP) manner. That is, the display apparatus 700 displays the content image 760 as the main image of the PIP and the service image 770 as the sub-image of the PIP.

At this time, the display device 700 determines an area corresponding to the display position of the service image 770 in the entire area of the content image 760, and acquires the image information of the determined area. The display device 700 generates a background image of the service image 770 based on the acquired image information.

25 is a flowchart showing a process of the display device 700 displaying a service image.

As shown in FIG. 25, in step S510, the display device 700 displays a content image. In this embodiment, the content image is displayed in full screen.

In step S520, the display device 700 determines whether an instruction to execute the supplementary service is received.

If it is determined that the execution instruction of the supplementary service is received, the display device 700 acquires the image information of the content image corresponding to the sub video position of the PIP in step S530.

In step S540, the display device 700 generates a background screen based on the acquired image information. The method of generating a background screen can be applied to the above embodiment, and a detailed description thereof will be omitted.

In step S550, the display apparatus 700 displays the service image including the background image as a sub-image of the PIP, and displays the content image as the main image of the PIP.

Hereinafter, an implementation example of the UI including the background screen as described in the foregoing embodiment will be further described.

26 is an exemplary diagram of a screen 800 displayed by a display device according to an eighth embodiment of the present invention in response to user input.

26, the display device displays a main screen 800 of the UI when predetermined user input such as a user pressing a specific button of the remote controller occurs or when the system power of the display device is turned on .

The main screen 800 includes a background screen 810 which changes in real time in response to the reproduction of the content image as a whole, and in which the lines of the main screen 800 continuously change in the form of waves. The background screen 810 has been described in detail in the foregoing embodiment. Also, the background screen 810 can be applied to UIs other than the main screen 800 in the embodiment described below.

If it is determined that there is no input from the user for a predetermined period of time while the main screen 800 is being displayed, the display device can switch only the background screen 810 from the main screen 800 to be displayed. If the user input is performed while only the background screen 810 is displayed, the display device displays the main screen 800 again.

The main screen 800 includes a plurality of icons 820, 830, 840, 850, 860, 870 corresponding to each service in order to execute a predetermined service. The icons 820, 830, 840, 850, 860 and 870 may be arranged on the lower side in the main screen 800, but the icons 820, 830, 840, But is not limited thereto.

Each of the icons 820, 830, 840, 850, 860, 870 may include a label of a title that briefly summarizes the corresponding service. For example, in the present embodiment, a TV icon 820, an app icon 830, a speaker icon 840, a photo icon 850, a clock icon 860, and a wallpaper setting icon 870 are included. The TV icon 820, the app icon 830, the speaker icon 840, the photo icon 850 and the clock icon 860 are arranged in a line on the right side of the main screen 800 along the lower edge, A background screen setting icon 870 which is different in nature from the service and the service is arranged on the left and lower sides of the main screen 800.

830, 840, 850, 860 and 870 corresponding to each service and the arrangement form of the icons 820, 830, 840, 850, 860 and 870 in the present embodiment However, this is only one example of various forms of the main screen 800 that can be displayed on the display device, and does not limit the spirit and implementation of the present invention to one.

The TV icon 820, the app icon 830, the speaker icon 840, the photo icon 850 and the clock icon 860 are displayed on a plurality of lines extending vertically from the lower side of the main screen 800 831, 841, 851, and 861, respectively.

For example, the line includes a specific color, and the effect of being animated on the background screen 810 as it goes from the lower side to the upper side of the main screen 800 may be reflected.

Each of the areas 821, 831, 841, 851 and 861 can be highlighted to indicate that the icons 820, 830, 840, 850, 860 in the corresponding areas 821, 831, 841, 851, have. That is, as the user presses the direction key of the remote controller, a highlight indicating that the currently selected state can be moved. For example, when the TV icon 820 is selected, the TV icon 820 is enlarged in size compared to other icons, while the area 821 including the TV icon 820 is highlighted and displayed do. Thus, the user can easily recognize that the TV icon 820 is currently selected.

In this state, when the user moves the highlight to the app icon 830, the TV icon 820 returns to its original size, and the highlight of the area 821 including the TV icon 820 is released. Then, the size of the app icon 830 is enlarged, while the area 831 including the app icon 830 is highlighted and displayed.

In the state where one of the icons 820, 830, 840, 850, 860 and 870 is selected, the user executes the icons 820, 830, 840, 850, 860 and 870 by pressing the enter key of the remote controller, . The display device executes a service corresponding to the icons 820, 830, 840, 850, 860, 870 in response to the execution instruction of the icons 820, 830, 840, 850, 860, 870.

Hereinafter, a corresponding service of each of the icons 820, 830, 840, 850, 860, 870 will be briefly described.

In response to the execution of the TV icon 820, the display device displays the image by processing the currently received image signal. The display device can receive a video signal from various video sources. For example, the display device can display a broadcast image of a specific channel by tuning a broadcast signal received from an output device of a broadcast station, The image of the video signal received from the apparatus can be displayed.

In response to the execution of the app icon 830, the display device displays the items of the various applications currently installed on the display device in an executable manner.

In response to the execution of the speaker icon 840, the display device displays a setting screen of a speaker for outputting a sound or a screen showing a sound output state of the speaker.

In response to the execution of the photo icon 850, the display device displays a photo image that the display device can access.

In response to the execution of the clock icon 860, the display device displays a clock representing the current time.

In response to the execution of the background setting icon 870, the display device displays a UI for changing the setting of the background screen 810 currently displayed.

Hereinafter, examples of the UI displayed by the display device for each of the icons 820, 830, 840, 850, 860, 870 when the icons 820, 830, 840, 850, 860, 870 are selected and executed do. When any one of the TV icon 820, the app icon 830, the speaker icon 840, the photo icon 850 and the clock icon 860 displayed on the right side of the main screen 800 is executed, 820, 830, 840, 850, and 860 are provided. These services can be displayed in various forms. For example, the main screen 800 can be switched to a service providing screen for each of the icons 820, 830, 840, 850, and 860. Or, the areas 821, 831, 841, 851, and 861 including the icons 820, 830, 840, 850, and 860 extend to the left side of the main screen 800, UIs corresponding to the corresponding icons 820, 830, 840, 850, and 860 may be displayed within the UIs 821, 831, 841, 851, and 861.

FIG. 27 is an exemplary view of a UI displayed by a display device according to an eighth embodiment of the present invention in response to execution of a TV icon. FIG.

As shown in FIG. 27, in response to the user selecting and executing the TV icon 820 (see FIG. 26), the display device processes the currently received content video signal to display the content video 910. The content image 910 may vary depending on what image signal is currently selected to be processed among the image signals received by the display device. The display device determines the setting state of the video signal in displaying the content video 910.

For example, the display device acquires the setting state information of the video signal immediately before the main screen 800 (see Fig. 26) is displayed. The setting state information includes processing information for displaying the content image 910. For example, among the various video signals that can be received by the display device, identification information of the video signal processed by the display device, Identification information of the corresponding channel, and other reproduction information such as the resolution of the image and the volume of the luminance or voice when the plurality of image channels are included. The display device switches from the main screen 800 (see FIG. 26) to the content image 910 based on the setting state information, and displays it.

28 is an exemplary view of a UI displayed by a display device according to an eighth embodiment of the present invention in response to execution of an app icon.

28, the display device expands the area 831 including the app icon 830 in the main screen 800 in response to the execution of the app icon 830. [ For example, the display device moves the TV icon 820 and the app icon 830, which are arranged in the right half of the main screen 800 to the left in the main screen 800, and displays the speaker icons 840, The area 831 can be extended in such a manner that the position of the photo icon and the clock icon 860 is maintained.

The display device displays an execution icon 832 of one or more applications that are stored or installed in the display device in the extended area 831. Each execution icon 832 includes a thumbnail image such as a logo that the user can intuitively recognize the application, and may further include title information or brief description information of the application. In response to the user selecting and executing an execution icon 832, the display device executes an application corresponding to the execution icon 832 corresponding thereto.

Here, the display device may highlight the corresponding execution icon 832 or display the corresponding execution icon 832 so that the user can easily distinguish the currently selected execution icon 832 among the plurality of execution icons 832. [ The size can be adjusted to be relatively larger than the other execution icons 832.

The display device can further display the app add icon 833 along with the execute icon 832 in the extended area 831. [ In response to the user executing the app add icon 833, the display device may display a UI for adding an application running on the display device, and the execution icon 832 of the added application may be displayed in the extended area (831).

29 is an exemplary diagram of a UI displayed by a display device according to an eighth embodiment of the present invention in response to execution of a speaker icon.

29, the display device expands the area 841 containing the speaker icon 840 in response to the execution of the speaker icon 840. [ The extended form of the area 841 can be applied to the case of the preceding app icon 830.

The display device determines whether or not an external device is connected. Here, the external device is a device in which audio data for output through a speaker provided in the display device is stored. If the display device determines that the external device is not connected, it displays a message 842 requesting the user to connect the external device on the extended area 841. [

While the message 842 is being displayed, the display device determines whether or not the external device is connected. When it is judged that the external device is connected, the display device receives and processes the voice data from the external device, and outputs it through the speaker. The display device displays a UI indicating the output state of the audio data while the audio data is outputted through the speaker.

30 is an exemplary diagram showing a display unit according to an eighth embodiment of the present invention displaying a UI showing an output state of a speaker.

As shown in FIG. 30, the display device displays a UI 920 including output status information of voice data while the speaker outputs voice data. The output state information in the UI 920 includes, for example, an equalizer visualization 921 indicating the frequency-dependent state of the audio data, title information 922 of the audio data, artist, performer or artist information 923, A time bar 924 in which an indicator indicating the reproduction time point is moved in real time, and the like. The UI 920 may also include various kinds of audio data related information.

While the audio data is being reproduced, the equalizer visualization 921 of the UI 920 and the indicator of the time bar 924 change in real time according to the reproduction state. If playback of audio data is temporarily suspended, the equalizer visualization 921 of the UI 920 and the indicator of the time bar 924 remain stationary. In addition, the display device may additionally display a message indicating the stop state on the UI 920 when playback of sex data is temporarily stopped.

31 is an exemplary view of a UI displayed by a display device according to an eighth embodiment of the present invention in response to execution of a photo icon.

As shown in FIG. 31, in response to the execution of the photo icon 850, the display device expands the area 851 including the photo icon 850. The expanded form of the area 851 can be applied to the case of the preceding app icon 830.

The display device determines whether the external device or the external memory is connected to the display device. Here, the external device or the external memory is a device in which images for display are stored in the display device. The display device displays a message 852 to guide the user to connect the external memory in which the image is stored in the extended area 851 to the display device if the display device determines that the external memory in which the images are stored is not connected.

If the display device determines that the external memory is connected while the message 852 is displayed, the display device receives the image from the external memory and displays a UI for displaying the received image.

If the display device determines that there is no image in the connected external memory, a message indicating that there is no image in the external memory in the extended area 851 can be displayed. Here, the determination of the presence or absence of the image may be performed depending on whether or not the display apparatus has data in a displayable image format.

32 is an exemplary view showing an image displayed by a display device according to an eighth embodiment of the present invention.

As shown in FIG. 32, the display apparatus buffers the images 931, 932, and 933 received from the external memory. Here, the number or the data amount of the images 931, 932, and 933 to be buffered may be changed according to the device environment or the user setting of the display device.

For example, the display device may display the images 931, 932, and 933 in a slide-show manner by displaying the images 931, 932, and 933 buffered by the display device. can do. The display device displays the slide show UI 930 and moves each image 931, 932, 933 within the UI 930 at a predetermined rate. According to the slide show method, the user automatically scrolls a plurality of images 931, 932, and 933 on the screen without a separate input for switching the displayed images 931, 932, and 933. Accordingly, the display device can display the plurality of images 931, 932, and 933 sequentially.

The display device can move the images 931, 932, and 933 at a uniform speed without stopping in displaying the plurality of images 931, 932, and 933 while sliding them. Alternatively, the display device may stop the movement of the images 931, 932, and 933 for a preset time at the time when the image 932 is moved to the center and the image 932 is located at the center, And then moves the images 931, 932, and 933 again. Since the movement of the images 931, 932, and 933 is stopped for a predetermined time at the time when the images 931, 932, and 933 are positioned at the center, the user can easily view the images 931, 932, and 933.

33 is an exemplary view of a UI displayed by a display device according to an eighth embodiment of the present invention in response to execution of a clock icon.

As shown in Fig. 33, in response to the execution of the clock icon 860 (see Fig. 26), the display device displays the UI 940 representing time. The UI 940 may include various types of clocks, such as an analog clock or a digital clock, as long as it includes information indicating the current time.

At least a part of the UI 940 can be flicker displayed so that the user can easily recognize that the display device normally determines the current time and displays the UI 940. [ For example, the display device displays a symbol ":" between a number representing a time and a number representing a minute by flickering in seconds.

FIG. 34 is an exemplary view of a UI displayed by the display device according to the eighth embodiment of the present invention in response to execution of a background setting icon. FIG.

As shown in Fig. 34, in response to the execution of the background setting icon 870 (see Fig. 26), the display device displays the UI 950 providing an option to change the background setting. The UI 950 is configured to change three items: Color, Grid, and Auto Start. The option that can be changed in the UI 950 in this embodiment is only an example, and does not limit the actual product implementation realizing the idea of the present invention.

The Color item is provided to specify the main color of the wallpaper of the curtain effect that represents the silhouette of the content image. In addition to the default value, the Color item can be selected from among red, green, blue, and gray. The display device displays the background image in the UI based on the color designated in the Color item.

The Grid item is designed to specify the size of the grid that splits the screen when creating the background image.

The Auto Start item determines whether or not the initial screen displayed when the system power of the display apparatus is turned on is the main screen 800 (see Fig. 26). When the Auto Start item is set to On, the display device sets the main screen as the initial screen. On the other hand, if the Auto Start item is set to Off, the display device does not set the main screen as the initial screen. For example, when the system power is turned on, the content image can be displayed first.

As described above, the display device can display various types of UIs having a background screen to which a curtain effect is applied.

Hereinafter, an example of the UI provided by the display device for adjusting the function of the display device will be described.

35 is an exemplary view of a UI provided by a display device according to a ninth embodiment of the present invention for function adjustment;

As shown in Fig. 35, the display device displays a UI 960, which provides for adjusting various functions of the display device, in response to a predetermined user input, for example, a button input of the remote controller, on the content image 970 do.

In displaying the UI 960 overlaid on the content image 970, the display device is not limited at which position it is displayed. The display device can slide the UI 960 from the right side of the content image 970 and display the UI 960 from the left side or the upper side or the lower side of the content image 970 .

The UI 960 is provided such that the content image 970 behind the UI 960 can be viewed through the UI 960 by applying transparency of a predetermined value. Here, the UI 960 is divided into a first region 961 and a second region 962, and the transparency of the first region 961 and the second region 962 are different from each other, .

On the first area 961, changeable function items of the display device are selectively arranged. On the second area 962, setting values for the selected specific item among the plurality of items of the first area 961 are arranged.

For example, the display device displays items such as a picture mode for selecting a picture mode, a sound mode for selecting a voice mode, and a sleep timer for selecting an automatic ending time on the first area 961. By pushing the direction button of the remote controller by the user, the cursor can move between items on the first area 961. [

The display device displays a plurality of setting values provided for a specific item on the currently selected first area 961 on the second area 962. [ As the user moves between items on the first area 961, the selected item changes, and the contents of the set value displayed on the second area 962 corresponding to the item change also change.

For example, when an item of Picture Mode is selected from the items on the first area 961, the display device displays Standard, Dynamic, Movie, Natural, and the like, which are set values of Picture Mode, on the second area 962. When the user selects and determines any one of these set values, the display device can adjust the function by reflecting the determined selected value.

The methods according to exemplary embodiments of the present invention may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer readable medium. Such computer readable media may include program instructions, data files, data structures, etc., alone or in combination. For example, the computer-readable medium may be a volatile or non-volatile storage device, such as a storage device such as a ROM, or a memory such as a RAM, a memory chip, a device, or an integrated circuit, whether removable or rewritable. , Or a storage medium readable by a machine (e.g., a computer), such as a CD, a DVD, a magnetic disk, or a magnetic tape, as well as being optically or magnetically recordable. It will be appreciated that the memory that may be included in the mobile terminal is an example of a machine-readable storage medium suitable for storing programs or programs containing instructions for implementing the embodiments of the present invention. The program instructions recorded on the storage medium may be those specially designed and constructed for the present invention or may be those known to those skilled in the art of computer software.

The above-described embodiments are merely illustrative, and various modifications and equivalents may be made by those skilled in the art. Accordingly, the true scope of protection of the present invention should be determined by the technical idea of the invention described in the following claims.

100, 600, 700: Display device
110:
111: Tuning unit
120, 620, 720:
130, 410, 420, 630, 730:
140:
150: Signal processor
151: Dip Mills
152: decoder
153: Scaler
154: CPU
155: buffer

Claims (20)

In the display device,
A display unit;
A user input section;
And at least one processor for processing the content image to be displayed on the display unit,
Wherein the at least one processor generates a background image of the GUI corresponding to an outline of at least one object in the content image upon receiving an instruction to execute a graphic user interface from the user input unit during display of the content image And processes the GUI to be displayed on the display unit together with the background image. The method according to claim 1,
Wherein the at least one processor processes the predetermined image information acquired from the content image to generate the background image corresponding to the outline of the at least one object. 3. The method of claim 2,
Wherein the at least one processor is configured to acquire the video information changing in accordance with the reproduction of the content video in real time and to process the background image in accordance with the video information acquired in real time. The method of claim 3,
Wherein the at least one processor adjusts the background image to be initially generated on a gray scale basis and gradually changes the RGB value of the background image so that the background image changes on a predetermined color basis with the passage of time . 3. The method of claim 2,
Wherein the image information includes an RGB value of each pixel included in one frame of the content image. 6. The method of claim 5,
The image frame is divided into a plurality of regions, a luminance value and a transparency derived corresponding to RGB values of the pixels included in each of the divided regions are applied to the respective divided regions, And the background image is generated by adjusting the horizontal width of the area and applying blur filtering to each of the divided areas. The method according to claim 6,
Wherein the at least one processor relatively adjusts the reduction level of the horizontal width of the divided area with the relatively high luminance value so that the reduction level of the horizontal width of the divided area having the relatively low luminance value is relatively small And a display device for displaying the image. The method according to claim 6,
Wherein the at least one processor applies the blur filtering to the left and right edges of the upper, lower, right, and left four-directional edges of the divided area. The method according to claim 1,
Wherein the GUI is configured to include at least one for each of a plurality of predetermined services,
Wherein one or more of the GUIs provided for each of the services are preset to be switched in correspondence with four-directional movement in the up, down, left, and right directions indicated by the user input unit. 10. The method of claim 9,
Wherein the at least one processor is configured to switch between the GUIs of the services different from each other in response to an instruction to move up and down from the user input unit and to switch between the GUIs in the same service in the same service in response to a left- Is switched. A method of controlling a display device,
Displaying a content image;
Receiving a GUI execution instruction from a user during display of the content image;
Generating a background image of the GUI corresponding to an outline of at least one object in the content image, and displaying the GUI together with the background image. 12. The method of claim 11,
Wherein the displaying the GUI comprises:
And processing the predetermined image information acquired from the content image to generate the background image corresponding to the outline of the at least one object. 13. The method of claim 12,
Wherein the displaying the GUI comprises:
Acquiring the video information changing in accordance with the reproduction of the content video in real time and processing the background image to be changed by the video information acquired in real time. 14. The method of claim 13,
Wherein the displaying the GUI comprises:
Adjusting the background image initially generated on a gray scale basis and progressively changing the RGB value of the background image so that the background image changes based on a predetermined color as time elapses A method of controlling a device. 13. The method of claim 12,
Wherein the image information includes RGB values of pixels included in one frame of the content image. 16. The method of claim 15,
Wherein the generating the background image comprises:
Dividing the image frame into a plurality of regions;
Applying a luminance value and a transparency derived corresponding to RGB values of the pixels included in each of the divided regions to each of the divided regions;
Adjusting widths of the respective divided regions corresponding to the luminance values;
And applying blur filtering to each of the divided areas. 17. The method of claim 16,
And adjusting the width of each of the divided regions corresponding to the luminance value,
Adjusting a reduction level of a horizontal width of the divided area relatively higher than the luminance value;
And adjusting the reduction level of the horizontal width of the divided area to be relatively small so that the brightness value is relatively low. 17. The method of claim 16,
Wherein applying the blur filtering to each of the divided regions comprises applying the blur filtering to the left and right edges of the upper, lower, right, and left four corners of the divided region. 12. The method of claim 11,
Wherein the GUI is configured to include at least one for each of a plurality of predetermined services,
Wherein the at least one GUI provided for each of the services is preset to be switched between the four directions indicated by the user in four directions. 20. The method of claim 19,
Switching between the GUIs of the services different from each other in response to an up / down movement instruction from the user;
Further comprising the step of switching between the GUIs in the same service in response to a leftward / rightward movement instruction from the user.

Images