KR102400640B1 - Display device and operating method thereof - Google Patents

Abstract

A method of operating a display device connected to a terminal according to an embodiment of the present invention includes the steps of: receiving, by the display device, a screen displayed on the terminal, and displaying the received screen on a display unit; Recognizing the user's status using a first command to display the content of the notification generated in the terminal on the screen based on the recognition result, and not to display the content of the notification on the screen It may include transmitting any one of the second commands to the terminal.

Description

Display device and its operating method {DISPLAY DEVICE AND OPERATING METHOD THEREOF}

An embodiment according to the concept of the present invention relates to a display apparatus, and in particular, a display apparatus for controlling a notification displayed on a screen of a terminal connected to the display apparatus based on a recognition result of a user state such as a user's gaze or presence; It relates to a method of its operation.

Mirroring (or screen mirroring) refers to displaying the screen of a terminal such as a smartphone or tablet PC on another display device (eg, TV, monitor, projector, etc.) as it is. In general, since the size of the display screen of the terminal is limited, the user can use the contents provided from the terminal through a device having a large screen such as a TV by using mirroring.

When the screen of the terminal is mirrored and displayed on the TV, the user may control the operation of the terminal by manipulating the TV, that is, the screen of the terminal displayed on the TV. In this case, various methods for conveniently controlling the screen of the terminal through the TV are required.

An object of the present invention is to provide a display device capable of controlling a notification displayed on a screen of a terminal based on a result of recognition of a user's state, such as a user's gaze or presence or the like, and an operating method thereof.

A method of operating a display device connected to a terminal according to an embodiment of the present invention includes the steps of: receiving, by the display device, a screen displayed on the terminal, and displaying the received screen on a display unit; Recognizing the user's status using a first command to display the content of the notification generated in the terminal on the screen based on the recognition result, and not to display the content of the notification on the screen It may include transmitting any one of the second commands to the terminal.

According to an embodiment, the displaying of the received screen on the display may include, by the display apparatus, receiving the screen displayed on the terminal in real time, and displaying the received screen on the display in real time.

According to another embodiment, the displaying of the received screen on the display unit may include displaying the received screen on a predetermined area of the display unit and displaying an image of content received from the outside in the remaining area of the display unit. there is.

According to an embodiment, the step of recognizing the state of the user may include recognizing the gaze of the user using the camera, and transmitting the one command to the terminal may include, as a result of the recognition, that the gaze of the user is the If the user faces the predetermined area, the first command may be transmitted to the terminal, and if the user's gaze does not face the predetermined area, the second command may be transmitted to the terminal.

According to another embodiment, the step of recognizing the state of the user may include recognizing the presence of the user using the camera, and transmitting the one command to the terminal may include, as a result of recognition, the presence of the user. , the first command may be transmitted to the terminal, and if the user does not exist, the second command may be transmitted to the terminal.

According to another embodiment, the step of recognizing the user's state may include recognizing the existence of another person other than the user using the camera, and transmitting the one command to the terminal may include: When the other person does not exist, the first command can be transmitted to the terminal, and when the other person exists, the second command can be transmitted to the terminal.

The method includes, when the remaining battery level of the terminal is lower than a reference remaining amount, capturing a screen displaying the contents of the notification received from the terminal, and displaying the contents of the captured notification based on the recognition result The method may further include displaying a screen on the display unit.

According to an embodiment, the transmitting of the one command to the terminal may include, when the screen including the contents of the notification is displayed on the display unit as the first command is transmitted, based on the contents of the notification. The method may further include receiving a request, and transmitting a control command corresponding to the received request to the terminal.

A display device connected to a terminal according to an embodiment of the present invention includes a display unit, a camera, an external device interface unit for receiving a screen displayed on the terminal, and a control to display the received screen on the display unit, and the camera a first command for recognizing the user's status using It may include a control unit for transmitting any one command to the terminal.

The display device may include any one of a TV, a monitor, and a projector.

The display apparatus according to an embodiment of the present invention has an effect of recognizing the user's state using a camera module, and controlling whether or not a notification window is included in the terminal screen displayed on the display apparatus based on the recognition result.

In addition, since the display device can capture and store the screen including the notification window of the terminal, even if the connection with the terminal is disconnected or the power of the terminal is turned off, it is possible to notify the user of the notification generated in the terminal.

1 is a block diagram showing the configuration of a display device according to an embodiment of the present invention.
2 is a block diagram of a remote control device according to an embodiment of the present invention.
3 shows an actual configuration example of a remote control device according to an embodiment of the present invention.
4 shows an example of using a remote control device according to an embodiment of the present invention.
5 is a ladder diagram for explaining a method of operating a display device according to an embodiment of the present invention.
6 is an exemplary diagram illustrating an operation in which a display apparatus displays a screen of the terminal transmitted from a terminal connected to the display apparatus according to an embodiment of the present invention.
FIG. 7 is a ladder diagram for explaining an embodiment of a case in which the user's state shown in FIG. 5 is the user's gaze.
8A to 8D are exemplary views illustrating an operation of the display device according to the embodiment shown in FIG. 7 .
FIG. 9 is a ladder diagram for explaining a case in which the state of the user shown in FIG. 5 is the presence or absence of the user.
10A to 10D are exemplary views illustrating an operation of the display device according to the embodiment shown in FIG. 9 .
FIG. 11 is a ladder diagram for explaining a case in which the state of the user shown in FIG. 5 is the presence or absence of another person.
12A to 12D are exemplary views illustrating an operation of the display device according to the embodiment shown in FIG. 11 .
13A to 13C are exemplary views illustrating an operation of a display device according to another embodiment of the present invention.
14 is an exemplary view for a case in which the display device shown in FIG. 1 is a desk lighting device including a projector.

Hereinafter, the embodiments disclosed in the present specification will be described in detail with reference to the accompanying drawings, but the same or similar components are assigned the same reference numbers regardless of reference numerals, and redundant description thereof will be omitted. The suffixes "module" and "part" for components used in the following description are given or mixed in consideration of only the ease of writing the specification, and do not have distinct meanings or roles by themselves. In addition, in describing the embodiments disclosed in the present specification, if it is determined that detailed descriptions of related known technologies may obscure the gist of the embodiments disclosed in this specification, the detailed description thereof will be omitted. In addition, the accompanying drawings are only for easy understanding of the embodiments disclosed in the present specification, and the technical idea disclosed herein is not limited by the accompanying drawings, and all changes included in the spirit and scope of the present invention , should be understood to include equivalents or substitutes.

Terms including an ordinal number such as 1st, 2nd, etc. may be used to describe various elements, but the elements are not limited by the terms. The above terms are used only for the purpose of distinguishing one component from another.

When an element is referred to as being “connected” or “connected” to another element, it is understood that it may be directly connected or connected to the other element, but other elements may exist in between. it should be On the other hand, when it is said that a certain element is "directly connected" or "directly connected" to another element, it should be understood that the other element does not exist in the middle.

The singular expression includes the plural expression unless the context clearly dictates otherwise.

In the present application, terms such as “comprises” or “have” are intended to designate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, but one or more other features It should be understood that this does not preclude the existence or addition of numbers, steps, operations, components, parts, or combinations thereof.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram showing the configuration of a display device according to an embodiment of the present invention.

Referring to FIG. 1 , the display device 100 may be implemented as a TV, a monitor, a projector, or a device including the same, a laptop computer, a tablet PC, a smart phone, or the like.

The display device 100 includes a broadcast receiving unit 130 , an external device interface unit 135 , a storage unit 140 , a user input interface unit 150 , a camera 160 , a control unit 170 , a display unit 180 , It may include an audio output unit 185 and a power supply unit 190 .

The broadcast receiving unit 130 may include a tuner 131 , a demodulator 132 , and a network interface unit 133 .

The tuner 131 may tune into a specific broadcasting channel according to a channel selection command. The tuner 131 may receive a broadcast signal for a specific selected broadcast channel.

The demodulator 132 may divide the received broadcast signal into a video signal, an audio signal, and a data signal related to a broadcast program, and may restore the separated video signal, audio signal, and data signal to an outputable form.

The network interface unit 133 may provide an interface for connecting the display apparatus 100 to a wired/wireless network including an Internet network. The network interface unit 133 may transmit or receive data to or from another user or other electronic device through the connected network or another network linked to the connected network.

In addition, some content data stored in the display apparatus 100 may be transmitted to another user registered in advance in the display apparatus 100 , or to a user selected from among other electronic devices or to a selected electronic device.

The network interface unit 133 may access a predetermined web page through the connected network or another network linked to the connected network. That is, by accessing a predetermined web page through a network, it is possible to transmit or receive data with a corresponding server.

In addition, the network interface unit 133 may receive content or data provided by a content provider or a network operator. That is, the network interface unit 133 may receive contents such as movies, advertisements, games, VODs, broadcast signals, etc. and related information provided from the contents provider or the network provider through the network.

Also, the network interface unit 133 may receive firmware update information and an update file provided by a network operator, and transmit data to the Internet or a content provider or network operator.

The network interface unit 133 may select and receive a desired application from among applications open to the public through a network.

The external device interface unit 135 may receive an application or an application list in an adjacent external device and transmit it to the control unit 170 or the storage unit 140 .

The external device interface unit 135 may provide a connection path between the display device 100 and an external device. The display device 100 may be connected to an external device by wire or wirelessly through the external device interface unit 135 .

The external device interface unit 135 may receive one or more of an image and audio output from an external device, and transmit it to the controller 170 . An external device connectable to the external device interface unit 135 may be any one of a set-top box, a Blu-ray player, a DVD player, a game machine, a sound bar, a smart phone, a PC, a USB memory, and a home theater.

The storage unit 140 may store a program for each signal processing and control in the control unit 170 , and may store a signal-processed image, audio, or data signal.

In addition, the storage unit 140 may perform a function for temporarily storing the video, audio, or data signal input from the external device interface unit 135, the network interface unit 133, or the camera 160, Information on a predetermined image may be stored through the channel storage function.

The storage unit 140 may store an application or an application list input from the external device interface unit 135 or the network interface unit 133 .

The display apparatus 100 may reproduce content files (movie files, still image files, music files, document files, application files, etc.) stored in the storage unit 140 and provide them to the user.

The user input interface unit 150 may transmit a signal input by the user to the control unit 170 or may transmit a signal from the control unit 170 to the user. For example, the user input interface unit 150 is Bluetooth (Bluetooth), UWB (Ultra Wideband), ZigBee (ZigBee) method, according to various communication methods such as RF (Radio Frequency) communication method or infrared (IR) communication method, such as, Control signals such as power on/off, channel selection, and screen setting may be received and processed from the remote control device 200 , or a control signal from the control unit 170 may be transmitted to the remote control device 200 .

Also, the user input interface unit 150 may transmit a control signal input from a local key (not shown) such as a power key, a channel key, a volume key, and a setting value to the control unit 170 .

The image signal processed by the controller 170 may be input to the display unit 180 and displayed as an image corresponding to the image signal. Also, the image signal processed by the control unit 170 may be input to an external output device through the external device interface unit 135 .

The audio signal processed by the control unit 170 may be audio output to the audio output unit 185 . In addition, the audio signal processed by the control unit 170 may be input to an external output device through the external device interface unit 135 .

In addition, the controller 170 may control overall operations within the display apparatus 100 .

In addition, the control unit 170 may control the display apparatus 100 according to a user command input through the user input interface unit 150 or an internal program, and access the network to display an application or a list of applications desired by the user. You can make it available for download in (100).

The control unit 170 allows the user-selected channel information to be output through the display unit 180 or the audio output unit 185 together with the processed image or audio signal.

In addition, the control unit 170 according to the external device image reproduction command received through the user input interface unit 150, from an external device input through the external device interface unit 135, for example, a camera or camcorder, An image signal or an audio signal may be output through the display unit 180 or the audio output unit 185 .

Meanwhile, the controller 170 may control the display unit 180 to display an image, for example, a broadcast image input through the tuner 131 or an external input input through the external device interface unit 135 . An image, an image input through the network interface unit, or an image stored in the storage unit 140 may be controlled to be displayed on the display unit 180 . In this case, the image displayed on the display unit 180 may be a still image or a moving image, and may be a 2D image or a 3D image.

In addition, the control unit 170 may control the content stored in the display apparatus 100, received broadcast content, or external input content input from the outside to be reproduced, and the content may include a broadcast image, an external input image, an audio file, It may be in various forms, such as a still image, a connected web screen, and a document file.

The camera 160 processes an image frame such as a still image or a moving image obtained by an image sensor. The image frame processed by the camera 160 may be stored in the storage unit 140 .

In particular, the camera 160 according to an embodiment of the present invention may be used to recognize the user's state, such as the user's gaze, the user's presence, or the presence of another person. This will be described later with reference to FIGS. 7 to 13C .

The display unit 180 converts the image signal, the data signal, the OSD signal processed by the controller 170, or the image signal and the data signal received from the external device interface unit 135 into R, G, and B signals, respectively, and drives them. signal can be generated.

Meanwhile, since the display device 100 shown in FIG. 1 is only an embodiment of the present invention. Some of the illustrated components may be integrated, added, or omitted according to the specifications of the actually implemented display apparatus 100 .

That is, two or more components may be combined into one component, or one component may be subdivided into two or more components as needed. In addition, the function performed by each block is for explaining the embodiment of the present invention, and the specific operation or device does not limit the scope of the present invention.

According to another embodiment of the present invention, the display apparatus 100 does not include a tuner 131 and a demodulator 132, and does not include a network interface unit 133 or an external device interface unit ( 135), the image may be received and reproduced.

For example, the display apparatus 100 is divided into an image processing apparatus such as a set-top box for receiving contents according to a broadcast signal or various network services, and a content reproduction apparatus which reproduces content input from the image processing apparatus. can be implemented.

In this case, the operating method of the display apparatus according to an embodiment of the present invention, which will be described below, includes not only the display apparatus 100 as described with reference to FIG. 1 , but also an image processing apparatus such as the separated set-top box or a display unit ( 180) and the audio output unit 185 may be performed by any one of the content reproducing apparatus.

Next, a remote control apparatus according to an embodiment of the present invention will be described with reference to FIGS. 2 to 3 .

2 is a block diagram of a remote control apparatus according to an embodiment of the present invention, and FIG. 3 shows an actual configuration example of the remote control apparatus according to an embodiment of the present invention.

First, referring to FIG. 2 , the remote control device 200 includes a fingerprint recognition unit 210 , a wireless communication unit 220 , a user input unit 230 , a sensor unit 240 , an output unit 250 , and a power supply unit 260 . ), a storage unit 270 , a control unit 280 , and a voice acquisition unit 290 .

Referring to FIG. 2 , the wireless communication unit 220 transmits and receives signals to and from the display device according to the embodiment of the present invention described above.

The remote control device 200 includes an RF module 221 capable of transmitting and receiving signals to and from the display device 100 according to the RF communication standard, and capable of transmitting and receiving signals to and from the display device 100 according to the IR communication standard. An IR module 223 may be provided. In addition, the remote control device 200 may include a Bluetooth module 225 capable of transmitting and receiving signals to and from the display device 100 according to the Bluetooth communication standard. In addition, the remote control device 200 includes an NFC module 227 capable of transmitting and receiving signals to and from the display device 100 according to a near field communication (NFC) communication standard, and a display according to a wireless LAN (WLAN) communication standard. A WLAN module 229 capable of transmitting and receiving signals to and from the device 100 may be provided.

In addition, the remote control device 200 transmits a signal containing information about the movement of the remote control device 200 to the display device 100 through the wireless communication unit 220 .

Meanwhile, the remote control device 200 may receive a signal transmitted by the display device 100 through the RF module 221 , and if necessary, power on/off the display device 100 through the IR module 223 . It can transmit commands about off, channel change, volume change, and the like.

The user input unit 230 may include a keypad, a button, a touch pad, or a touch screen. The user may input a command related to the display apparatus 100 to the remote control apparatus 200 by manipulating the user input unit 230 . When the user input unit 230 includes a hard key button, the user may input a command related to the display apparatus 100 to the remote control apparatus 200 through a push operation of the hard key button. This will be described with reference to FIG. 3 .

Referring to FIG. 3 , the remote control device 200 may include a plurality of buttons. The plurality of buttons include a fingerprint recognition button 212, a power button 231, a home button 232, a live button 233, an external input button 234, a volume control button 235, a voice recognition button 236, It may include a channel change button 237 , a confirmation button 238 , and a back button 239 .

The fingerprint recognition button 212 may be a button for recognizing a user's fingerprint. In an embodiment, the fingerprint recognition button 212 may perform a push operation, and thus may receive a push operation and a fingerprint recognition operation. The power button 231 may be a button for turning on/off the power of the display apparatus 100 . The home button 232 may be a button for moving to the home screen of the display apparatus 100 . The live button 233 may be a button for displaying a real-time broadcast program. The external input button 234 may be a button for receiving an external input connected to the display apparatus 100 . The volume control button 235 may be a button for adjusting the level of the volume output by the display apparatus 100 . The voice recognition button 236 may be a button for receiving a user's voice and recognizing the received voice. The channel change button 237 may be a button for receiving a broadcast signal of a specific broadcast channel. The confirmation button 238 may be a button for selecting a specific function, and the back button 239 may be a button for returning to a previous screen.

Fig. 2 will be described again.

When the user input unit 230 includes a touch screen, the user may input a command related to the display apparatus 100 to the remote control apparatus 200 by touching a soft key of the touch screen. In addition, the user input unit 230 may include various types of input means that the user can operate, such as a scroll key or a jog key, and this embodiment does not limit the scope of the present invention.

The sensor unit 240 may include a gyro sensor 241 or an acceleration sensor 243 , and the gyro sensor 241 may sense information regarding the movement of the remote control device 200 .

For example, the gyro sensor 241 may sense information about the operation of the remote control device 200 based on the x, y, and z axes, and the acceleration sensor 243 is the moving speed of the remote control device 200 . It is possible to sense information about the Meanwhile, the remote control device 200 may further include a distance measuring sensor, so that the distance to the display unit 180 of the display device 100 may be sensed.

The output unit 250 may output an image or audio signal corresponding to a manipulation of the user input unit 235 or a signal transmitted from the display apparatus 100 . Through the output unit 250 , the user may recognize whether the user input unit 235 is manipulated or whether the display apparatus 100 is controlled.

For example, the output unit 250 includes an LED module 251 that is turned on when the user input unit 235 is manipulated or a signal is transmitted and received with the display device 100 through the wireless communication unit 225, and a vibration module that generates vibration ( 253), a sound output module 255 for outputting a sound, or a display module 257 for outputting an image may be provided.

In addition, the power supply unit 260 supplies power to the remote control device 200, and when the remote control device 200 does not move for a predetermined period of time, stops the power supply, thereby reducing power wastage. The power supply unit 260 may resume power supply when a predetermined key provided in the remote control device 200 is operated.

The storage unit 270 may store various types of programs and application data required for control or operation of the remote control device 200 . If the remote control device 200 wirelessly transmits and receives signals through the display device 100 and the RF module 221 , the remote control device 200 and the display device 100 transmit and receive signals through a predetermined frequency band. .

The control unit 280 of the remote control device 200 stores information about a frequency band in which a signal can be wirelessly transmitted and received with the display device 100 paired with the remote control device 200 in the storage unit 270 and is referenced. can do.

The controller 280 controls all matters related to the control of the remote control device 200 . The control unit 280 transmits a signal corresponding to a predetermined key operation of the user input unit 235 or a signal corresponding to the movement of the remote control device 200 sensed by the sensor unit 240 to the display device ( 100) can be transmitted.

Also, the voice acquisition unit 290 of the remote control device 200 may acquire a voice.

The voice acquisition unit 290 may include at least one microphone, and may acquire a voice through the microphone.

Next, FIG. 4 will be described.

4 shows an example of using a remote control device according to an embodiment of the present invention.

4A illustrates that a pointer 205 corresponding to the remote control device 200 is displayed on the display unit 180 .

The user may move or rotate the remote control device 200 up, down, left and right. The pointer 205 displayed on the display unit 180 of the display apparatus 100 corresponds to the movement of the remote control apparatus 200 . As shown in the drawing, the remote control apparatus 200 may be called a space remote controller because the corresponding pointer 205 is moved and displayed according to movement in 3D space.

4B illustrates that when the user moves the remote control apparatus 200 to the left, the pointer 205 displayed on the display unit 180 of the display apparatus 100 also moves to the left correspondingly.

Information on the movement of the remote control device 200 sensed through the sensor of the remote control device 200 is transmitted to the display device 100 . The display apparatus 100 may calculate the coordinates of the pointer 205 from information about the movement of the remote control apparatus 200 . The display apparatus 100 may display the pointer 205 to correspond to the calculated coordinates.

4C illustrates a case in which the user moves the remote control device 200 away from the display unit 180 while pressing a specific button in the remote control device 200 . Accordingly, the selected area in the display unit 180 corresponding to the pointer 205 may be zoomed in and displayed.

Conversely, when the user moves the remote control apparatus 200 closer to the display unit 180 , the selected area in the display unit 180 corresponding to the pointer 205 may be zoomed out and displayed.

Meanwhile, when the remote control apparatus 200 moves away from the display unit 180 , the selection area is zoomed out, and when the remote control apparatus 200 approaches the display unit 180 , the selection area may be zoomed in.

In addition, while a specific button in the remote control device 200 is pressed, recognition of vertical and horizontal movements may be excluded. That is, when the remote control device 200 moves away from or close to the display unit 180 , up, down, left, and right movements are not recognized, but only forward and backward movements may be recognized. In a state in which a specific button in the remote control device 200 is not pressed, only the pointer 205 moves according to the up, down, left, and right movements of the remote control device 200 .

Meanwhile, the moving speed or moving direction of the pointer 205 may correspond to the moving speed or moving direction of the remote control device 200 .

Meanwhile, in the present specification, a pointer refers to an object displayed on the display unit 180 in response to an operation of the remote control device 200 . Accordingly, objects of various shapes other than the arrow shape shown in the drawing are possible with the pointer 205 . For example, it may be a concept including a dot, a cursor, a prompt, a thick outline, and the like. In addition, the pointer 205 may be displayed corresponding to any one point among a horizontal axis and a vertical axis on the display unit 180, and may also be displayed corresponding to a plurality of points such as a line and a surface. Do.

Next, a method of operating a display apparatus according to an embodiment of the present invention will be described.

5 is a ladder diagram for explaining a method of operating a display device according to an embodiment of the present invention.

Referring to FIG. 5 , the display apparatus 100 may be connected to the terminal 200 ( S100 ).

Specifically, the display apparatus 100 may be connected to the terminal 200 by wire and/or wirelessly through the external apparatus interface unit 135 . For example, the display device 100 and the terminal 200 may be connected by wire through a universal serial bus (USB) interface, a high definition multimedia interface (HDMI), etc. or wirelessly through Wi-Fi or Wi-Fi Direct. However, the type of the connection method is not limited thereto.

The control unit 170 of the display apparatus 100 may receive a screen displayed on the display unit of the terminal 200 in real time (S200), and display the received screen on a specific area of the display unit 180 ( S300).

For example, the controller 170 may mirror a screen displayed on the display unit of the terminal 200 and display it on the display unit 180 of the display apparatus 100 . Mirroring refers to streaming a screen displayed on one display device (or terminal) to display the same on the display unit of another display device (or terminal), and display the other display device (or terminal) on the display unit It may refer to a technology capable of controlling any one of the display devices (or terminals) by using the displayed screen.

Although in FIG. 5, the screen displayed on the terminal 200 is transmitted to the display device 100 in real time only in steps S200 and S300, and the control unit 170 displays the screen received in real time on a specific area of the display unit 180. Although shown as shown in the drawings, the operations of the display apparatus 100 and the terminal 200 according to steps S200 and S300 may be always being performed at all time points. That is, when the screen displayed on the terminal 200 is changed by other steps to be described later, the changed screen may be transmitted to the display apparatus 100 and displayed on the display unit 180 .

An embodiment of step S300 will be described with reference to FIG. 6 .

6 is an exemplary diagram illustrating an operation in which a display apparatus displays a screen of the terminal transmitted from a terminal connected to the display apparatus according to an embodiment of the present invention.

Referring to (a) and (b) of FIG. 6 , the controller 170 of the display device 100 is located in a specific area of the display unit 180 in real time from the terminal 200 through the external device interface unit 135 . The terminal screen 220 or 221 that is received (eg, streamed) may be displayed, and a screen for content received from the broadcast receiver 130 may be displayed on the remaining area of the display unit 180 .

In FIG. 6 , only an embodiment in which the controller 170 displays the terminal screen 220 or 221 in a specific region of the lower right portion of the display unit 180 is illustrated, but the position at which the terminal screen 220 or 221 is displayed, and The size of the terminal screen 220 or 221 displayed on the display unit 180 is not limited thereto. Accordingly, the position and/or size at which the terminal screen 220 or 221 is displayed may be changed according to a request received from the remote control device 200 .

When the screen displayed on the terminal 200 is a vertical screen, the controller 170 displays the terminal screen 220 in a vertical direction on a predetermined area of the display unit 180, and the screen displayed on the terminal 200 is horizontally displayed. In the case of a screen, the controller 170 may display the terminal screen 221 in a horizontal direction on a predetermined area of the display unit 180 .

Accordingly, the user can view the content received from the broadcast receiver 130 or the like through the display unit 180 of the display device 100 and simultaneously check the terminal screen 220 or 221 .

Fig. 5 will be described again.

When a notification occurs in the terminal 200 (S400), the display apparatus 100 may recognize the user's state (S500).

Specifically, notifications (eg, call reception, message reception, notifications related to terminal status, etc.) may be generated from various applications installed and executed in the terminal 200 . In general, the terminal 200 may display a notification icon corresponding to the generated notification on the status bar (or notification bar) or display a notification window including the contents of the notification.

When a notification occurs in the terminal 200 , the control unit 170 of the display apparatus 100 may recognize the user's state by using the camera 160 . For example, the controller 170 may receive a signal (or flag) indicating that a notification has occurred from the terminal 200 , and may recognize the user's state through the camera 160 in response to the received signal. Alternatively, the controller 170 determines whether a notification icon and/or a notification window is included in the screen received from the terminal 200, and if the determination result includes a notification icon and/or a notification window, the camera 160 It is also possible to recognize the user's status through

The user's state may mean a state related to whether the user of the terminal 200 can recognize the terminal screen displayed on the display unit 180 (or whether a notification included in the terminal screen can be recognized). In the present specification, a case in which the user's state is the user's gaze, the user's presence, and the presence of another person will be described as an example.

In the present specification, although it is illustrated that the notification is generated in the terminal 200 at a time corresponding to step S400, the time at which the notification is generated may vary according to embodiments. For example, the notification may have already been generated before the display apparatus 100 and the terminal 200 are connected.

The control unit 170 of the display apparatus 100 may transmit a command to the terminal 200 based on the recognition result of the user's state (S600), and the terminal 200 may transmit the content of the notification based on the received command may or may not be displayed on the screen (S700).

Steps S500 to S700 will be described in detail using various embodiments of the user's state with reference to FIGS. 7 to 12D .

7 is a ladder diagram for explaining an embodiment when the user's state shown in FIG. 5 is the user's gaze, and FIGS. 8A to 8D are diagrams illustrating the operation of the display device according to the embodiment shown in FIG. 7 . It is an example diagram.

Referring to FIG. 7 , when a notification is generated in the terminal 200 according to step S400 of FIG. 5 , the controller 170 may recognize the user's gaze using the camera 160 ( S501 ).

As shown in FIG. 8A , when the first notification and the second notification occur in the terminal 200, the terminal 200 displays the first notification window 241 including the contents of the first notification and the second notification. A corresponding second notification icon 232 may be displayed on the screen. In response to this, the control unit 170 of the display apparatus 100 displays the terminal screen 220 including the first notification window 241 and the second notification icon 232 in a specific area of the display unit 180 . can According to an embodiment, the terminal 200 may display the first notification icon on the screen instead of the first notification window 241 .

The controller 170 may recognize the gaze of the user of the terminal 200 using the camera 160 . Specifically, the controller 170 may recognize, from an image obtained by using the camera 160 , the direction in which the user's gaze is directed, included in the image.

Referring back to FIG. 7 , when the user's gaze does not face the area where the terminal screen is displayed as a result of the recognition according to step S501 (NO in S502), the controller 170 may determine that the user does not check the terminal screen. .

When it is determined that the user does not check the terminal screen, the control unit 170 may transmit a notification non-display command to the terminal 200 so as not to display a notification window including the contents of the notification on the terminal screen (S601). The terminal 200 may not display the notification window including the contents of the notification on the screen in response to the received notification non-display command (S701). In this case, the terminal 200 may display only the notification icon on the screen instead of the notification window.

In this regard, referring to FIG. 8B , when recognizing that the user's gaze is directed toward the center of the display unit 180 , the controller 170 may determine that the user does not check the terminal screen 220 . Accordingly, the control unit 170 transmits a notification non-display command for not displaying the first notification window 241 on the terminal screen 220 to the terminal 200, and the terminal 200 responds to the received notification non-display command The first notification window 241 may not be displayed on the screen. According to an embodiment, the terminal 200 may display the first notification icon 231 on the screen instead of displaying the first notification window 241 on the screen. The terminal screen 220 may include a first notification icon 231 and a second notification icon 232 .

Although not shown, when it is determined that the user's gaze is not directed toward the terminal screen 220 , the control unit 170 displays the terminal screen 220 itself on the display unit instead of transmitting a not-notification command to the terminal 200 . It may not be displayed on 180, or the terminal screen 220 may be displayed in a semi-transparent form.

Referring back to FIG. 7 , as a result of the recognition according to step S501 , if the user's gaze is directed toward the area where the terminal screen is displayed (YES in S502 ), the controller 170 may determine that the user checks the terminal screen.

When it is determined that the user checks the terminal screen, the control unit 170 may transmit a notification display command to display a notification window including the contents of the notification on the terminal screen to the terminal 200 (S602). The terminal 200 may display a notification window including the contents of the notification on the screen in response to the received notification display command (S702).

In this regard, referring to FIG. 8C , when the control unit 170 recognizes that the user's gaze is directed toward the area where the terminal screen 220 of the display unit 180 is displayed, the control unit 170 determines that the user checks the terminal screen 220 . can judge Accordingly, the control unit 170 transmits a notification display command to display a notification window (eg, the first notification window 241 ) on the terminal screen 220 to the terminal 200 , and the terminal 200 transmits the received notification The first notification window 241 may be displayed on the screen in response to the display command.

Although not shown, when it is determined that the user's gaze is directed toward the terminal screen 220 , the control unit 170 is not displayed on the display unit 180 or is translucent instead of transmitting a notification display command to the terminal 200 . The terminal screen 220 that was displayed in the form may be displayed on the display unit 180 as it is.

Referring back to FIG. 7 , when receiving a request based on a notification (notification window including the contents of the notification) displayed on the terminal screen 220 on the terminal screen 220 ( S800 ), the control unit 170 sends a control command corresponding to the received request to the terminal It can be transmitted to (200) (S810). The terminal 200 may perform an operation based on the received control command (S820).

In this regard, referring to FIG. 8D , the control unit 170 may receive a request for the terminal screen 220 through the user input interface unit 150 or the like. For example, when the received request is a request to switch the first notification window 241 displayed on the terminal screen 220 to the second notification window 242 in order to check the contents of the second notification, the control unit 170 A command to switch the first notification window 241 to the second notification window 242 and display it may be transmitted to the terminal 200 .

The terminal 200 converts and displays the first notification window 241 to the second notification window 242 in response to the received command, and transmits the screen on which the second notification window 242 is displayed to the display device 100 . can The control unit 170 may display the terminal screen 220 on which the second notification window 242 is displayed on a specific area of the display unit 180 .

9 is a ladder diagram for explaining a case in which the state of the user shown in FIG. 5 is the presence or absence of the user, and FIGS. 10A to 10C are the operation of the display device according to the embodiment shown in FIG. It is an example diagram showing.

Referring to FIG. 9 , when a notification occurs in the terminal 200 according to step S400 of FIG. 5 , the controller 170 may recognize the presence of the user using the camera 160 ( S503 ).

Since the terminal screen 220 displayed on the display unit 180 of the display apparatus 100 when a notification occurs in the terminal 200 is substantially the same as that described above with reference to FIG. 8A , a description thereof will be omitted.

The controller 170 may recognize the presence of the user of the terminal 200 by using the camera 160 . Specifically, the controller 170 may recognize, from an image obtained using the camera 160 , whether the user is included in the image.

If the user does not exist as a result of the recognition according to step S503 (NO in S504), the controller 170 may determine that the user does not check the terminal screen.

When it is determined that the user does not check the terminal screen, the control unit 170 may transmit a notification non-display command to the terminal 200 so as not to display a notification window including the contents of the notification on the terminal screen (S601). The terminal 200 may not display the notification window including the contents of the notification on the screen in response to the received notification non-display command (S701). In this case, the terminal 200 may display only the notification icon on the screen instead of the notification window.

In this regard, referring to FIG. 10A , when it is recognized that the user does not exist in the image acquired through the camera 160 , the controller 170 may determine that the user does not check the terminal screen 220 . there is. Accordingly, the control unit 170 transmits a notification non-display command for not displaying the first notification window 241 on the terminal screen 220 to the terminal 200, and the terminal 200 responds to the received notification non-display command The first notification window 241 may not be displayed on the screen. According to an embodiment, the terminal 200 may display the first notification icon 231 on the screen instead of displaying the first notification window 241 on the screen. The terminal screen 220 may include a first notification icon 231 and a second notification icon 232 .

Referring back to FIG. 9 , if the user exists as a result of the recognition according to step S503 (YES in S504), the controller 170 may determine that the user checks the terminal screen.

When it is determined that the user checks the terminal screen, the control unit 170 may transmit a notification display command to display a notification window including the contents of the notification on the terminal screen to the terminal 200 (S602). The terminal 200 may display a notification window including the contents of the notification on the screen in response to the received notification display command (S702).

In this regard, referring to FIG. 10B , the controller 170 controls the user 300 to check the terminal screen 220 when it is recognized that the user 300 is present in the image acquired through the camera 160 . can be judged as Accordingly, the control unit 170 transmits a notification display command to display a notification window (eg, the first notification window 241 ) on the terminal screen 220 to the terminal 200 , and the terminal 200 transmits the received notification The first notification window 241 may be displayed on the screen in response to the display command.

Referring back to FIG. 9 , when receiving a request based on a notification (notification window including the contents of the notification) displayed on the terminal screen 220 on the terminal screen 220 ( S800 ), the control unit 170 sends a control command corresponding to the received request to the terminal It can be transmitted to (200) (S810). The terminal 200 may perform an operation based on the received control command (S820).

Referring to FIG. 10C in this regard, for example, when the first notification is a message reception notification and the received request is a response request to a received message, the control unit 170 sends a command corresponding to a response request to the received message. It can be transmitted to the terminal 200 .

The terminal 200 may display a screen including the message writing window 251 in response to the received command, and transmit the screen on which the message writing window 251 is displayed to the display apparatus 100 . The control unit 170 may display the terminal screen 220 on which the message writing window 251 is displayed on a specific area of the display unit 180 .

11 is a ladder diagram for explaining a case in which the user's state shown in FIG. 5 is the presence of another person, and FIGS. 12A to 12C show the operation of the display device according to the embodiment shown in FIG. It is an example diagram showing.

Referring to FIG. 11 , when a notification occurs in the terminal 200 according to step S400 of FIG. 5 , the controller 170 uses the camera 160 to recognize whether there is another person other than the user of the terminal 200 . It can be done (S505).

Since the terminal screen 220 displayed on the display unit 180 of the display apparatus 100 when a notification occurs in the terminal 200 is substantially the same as that described above with reference to FIG. 8A , a description thereof will be omitted.

The controller 170 may use the camera 160 to recognize whether there is another person other than the user of the terminal 200 . Specifically, the control unit 170 recognizes, from the image obtained using the camera 160 , whether a person other than the user of the terminal 200 is included in the image, and displays the content of the notification on the terminal screen 220 . You can decide whether to do it or not. This is to prevent the content of the notification from being shown to others when there is another person other than the user of the terminal 200 .

As a result of the recognition according to step S505, if there is another person (YES in S506), the control unit 170 may transmit to the terminal 200 a non-notification command not to display a notification window including the contents of the notification on the terminal screen. (S601). The terminal 200 may not display the notification window including the contents of the notification on the screen in response to the received notification non-display command (S701). In this case, the terminal 200 may display only the notification icon on the screen instead of the notification window.

In this regard, referring to FIG. 12A , when it is recognized that the other person 301 is present in the image acquired through the camera 160 , the controller 170 displays a first notification window 241 on the terminal screen 220 . may be transmitted to the terminal 200 , and the terminal 200 may not display the first notification window 241 on the screen in response to the received notification non-display command. According to an embodiment, the terminal 200 may notify the user that a notification has occurred by displaying the first notification icon 231 on the screen instead of displaying the first notification window 241 on the screen.

Referring back to FIG. 11 , if there is no other person other than the user as a result of the recognition according to step S505 (NO in S506 ), the control unit 170 displays a notification display command to display a notification window including the contents of the notification on the terminal screen can be transmitted to the terminal 200 (S602). The terminal 200 may display a notification window including the contents of the notification on the screen in response to the received notification display command (S702).

In this regard, referring to FIG. 12B , when it is recognized that there is no other person other than the user 300 in the image acquired through the camera 160, the controller 170 displays a notification window (eg, , transmits a notification display command to display the first notification window 241) to the terminal 200, and the terminal 200 displays the first notification window 241 on the screen in response to the received notification display command. can

Referring back to FIG. 11 , when receiving a request based on a notification (notification window including the contents of the notification) displayed on the terminal screen 220 on the terminal screen 220 ( S800 ), the control unit 170 sends a control command corresponding to the received request to the terminal It can be transmitted to (200) (S810). The terminal 200 may perform an operation based on the received control command (S820).

In this regard, referring to FIG. 12C , for example, when the first notification is a message reception notification and the received request is a call origination request to the counterpart of the message, the controller 170 responds to the call origination request to the counterpart of the received message. A corresponding command may be transmitted to the terminal 200 .

The terminal 200 may display a screen including the call origination window 252 in response to the received command, and transmit the screen on which the call origination window 252 is displayed to the display apparatus 100 . The control unit 170 may display the terminal screen 220 on which the call origination window 252 is displayed on a specific area of the display unit 180 .

13A to 13C are exemplary views illustrating an operation of a display device according to another embodiment of the present invention.

The controller 170 may capture a notification window for a notification generated in the terminal 200 (or capture a terminal screen including the notification window).

As shown in FIG. 13A , for example, the user does not check the notification (when the user's gaze does not face the area where the terminal screen 220 is displayed, the user does not exist, or there is another person other than the user), the terminal ( 200), when the first notification, the second notification, and the third notification occur in the terminal 200 when the remaining battery level is lower than the reference remaining amount, the controller 170 controls the first notification window including the contents of the first notification At 241 , each of the terminal screens displaying a second notification window including the contents of the second notification and a third notification window including the contents of the third notification may be captured.

Referring to FIGS. 13B and 13C , when the user can check the notification through the display unit 180 (eg, the gaze is directed toward the area where the terminal screen is displayed, the user is present, or there is no other person other than the user) case), the control unit 170 displays the terminal screen 220 including the low battery window 245 , and displays at least one of the captured first notification window to the third notification window instead of the screen transmitted from the terminal 200 . can be displayed The displayed notification window may be switched according to a request received through the remote control device 200 or the like.

According to an embodiment, after the first notification window to the third notification window are captured, when the battery of the terminal 200 is exhausted and the power of the terminal 200 is turned off, the controller 170 controls the user to select the terminal. Upon checking the screen, after displaying a screen informing that the power of the terminal 200 is turned off, at least one of the captured first notification window to third notification window may be displayed.

That is, according to the embodiment shown in FIGS. 13A to 13C , the display apparatus 100 displays a pre-captured terminal screen even when the battery of the terminal 200 is low or the power of the terminal 200 is turned off. can display a notification to

14 is a perspective view of a device including a projector module as an embodiment of the display device shown in FIG. 1 .

Referring to FIG. 14 , as an embodiment of a device including a projector module, a desk lighting device 1000 equipped with a projector module 1200 is shown, but the embodiment of the present invention is limited to the desk lighting device 1000 only. It is not, and can be applied to various devices including a projector module.

The desk lighting device 1000 may include a light source unit 1010 for emitting light, and a light diffusion unit 1020 for diffusing the emitted light. Although not shown, the table-top lighting device 1000 may further include various components of known table-top lighting devices.

In particular, the desk lighting device 1000 according to an embodiment of the present invention may include a camera module 1100 and a projector module 1200, and controls the operation of each of the camera module 1100 and the projector module 1200. It may include a control unit.

Since each of the camera module 1100 and the control unit is substantially the same as each of the camera 160 and the control unit 170 illustrated in FIG. 1 , a description thereof will be omitted.

The projector module 1200 may project light corresponding to an image through a lens and display it on a screen or wall spaced apart from the projector module 1200 .

According to an embodiment, the camera module 1100 , the projector module 1200 , and the control unit may be implemented as a module or device separate from the tabletop lighting device 1000 , and may be used while being attached to the tabletop lighting device 1000 .

The controller may detect the user's terminal 200 using the camera module 1100 and connect to the detected terminal 200 . To this end, the control unit, the module including the control unit, or the desktop lighting device 1000 may include a wireless communication unit.

The controller may detect a user of the terminal 200 using the camera module 1100 and perform user authentication for the terminal 200 according to the detection result.

The controller may display the screen of the terminal 200 using the projector module 1200 . For example, the control unit may mirror the screen of the terminal 200 and display it through the projector module 1200, and when a notification occurs in the terminal 200, it receives information about the notification and displays the information about the received notification. It may be displayed through the projector module 1200 .

Similar to the embodiments described above with reference to FIGS. 5 to 12D , the control unit of the tabletop lighting device 1000 shown in FIG. 14 uses the camera module 1100 to determine whether a user exists, or whether there is another person in addition to the user. can recognize

As a result of the recognition, when the user exists or there is no other person other than the user, the control unit displays a notification window 241 including the contents of the notification generated on the terminal screen 220 or the terminal 200 through the projector module 1200 . can do.

As a result of the recognition, when the user does not exist or someone other than the user exists, the control unit transmits a command to cause the terminal 200 not to display a notification window on the screen, or the projector module 1200 does not display any screen. can be controlled not to.

In addition, similar to the embodiment described above with reference to FIGS. 13A to 13C , the control unit of the desk lighting apparatus 1000 generates in the terminal 200 when the user does not exist and the remaining battery level of the terminal 200 is lower than the reference remaining amount. Information on the notification may be stored (eg, an image captured by the screen of the terminal displaying the notification window, etc.). Then, when the user returns, the controller may display information on the stored notification through the projector module 1200 .

Accordingly, even if the battery of the terminal 200 is exhausted and the power of the terminal 200 is turned off, the controller may notify the user of the stored notification information through the projector module 1200 . According to an embodiment, when the user checks information about the stored notification, the controller may delete the information about the stored notification.

The present invention described above can be implemented as computer-readable codes on a medium in which a program is recorded. The computer-readable medium includes all types of recording devices in which data readable by a computer system is stored. Examples of computer-readable media include Hard Disk Drive (HDD), Solid State Disk (SSD), Silicon Disk Drive (SDD), ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc. There is also a carrier wave (eg, transmission over the Internet) that is implemented in the form of. In addition, the computer may include the control unit 180 of the terminal. Accordingly, the above detailed description should not be construed as restrictive in all respects but as exemplary. The scope of the present invention should be determined by a reasonable interpretation of the appended claims, and all modifications within the equivalent scope of the present invention are included in the scope of the present invention.

Claims (17)

In the display device connected to the terminal,
display unit;
camera;
an external device interface unit for receiving a screen displayed on the terminal; and
A first command for controlling the display of the received screen to be displayed on the display unit, recognizing the user's status using the camera, and displaying the contents of a notification generated in the terminal based on the recognition result on the screen; And a control unit for transmitting any one of the second commands for not displaying the content of the notification on the screen to the terminal,
the control unit
A display device for receiving a signal indicating that the notification has occurred in the terminal from the terminal, and recognizing the state of the user through the camera in response to the received signal. According to claim 1,
The user's state is the user's gaze,
The external device interface unit receives the screen displayed on the terminal in real time,
The control unit is
Display the screen displayed on the terminal received in real time on a predetermined area of the display unit,
When the user's gaze is directed toward the predetermined area as a result of recognition through the camera, the first command is transmitted to the terminal;
When the user's gaze does not face the predetermined area, the display device transmits the second command to the terminal. According to claim 1,
The state of the user is whether the user exists,
The control unit is
If the user is present as a result of recognition, the first command is transmitted to the terminal,
When the user does not exist, the display device transmits the second command to the terminal. According to claim 1,
The state of the user is whether there is a person other than the user,
The control unit is
As a result of the recognition, when the other person does not exist, the first command is transmitted to the terminal,
The display device transmits the second command to the terminal when the other person exists. According to claim 1,
The control unit is
When the remaining battery level of the terminal is lower than the reference remaining amount, capturing a screen displaying the contents of the notification received from the terminal,
A display device for controlling to display a screen on which the content of the captured notification is displayed on the display unit based on the recognition result. delete delete delete delete delete delete delete delete delete delete delete delete

Images