KR20210073921A - Display device, method for controlling the same and terminal device - Google Patents

Abstract

Disclosed are a display device, a control method thereof, and a terminal device. The terminal device transmits aspect ratio information of the terminal device to the display device, and the display device overlays a screen displayed on the terminal device on a screen displayed on the display device based on the received aspect ratio information and displays the screens together. Accordingly, it is possible to minimize a time delay occurring during a mirroring (screen casting) service.

Description

Display device, control method thereof, and terminal device {DISPLAY DEVICE, METHOD FOR CONTROLLING THE SAME AND TERMINAL DEVICE}

The present invention relates to a display device capable of intuitively minimizing a time delay occurring during a mirroring (screen casting) service, a control method thereof, and a terminal device performing a mirroring service with the display device.

A display device is a device having a function of providing an image that a user can watch. A user may watch a broadcast through the display device. The display device provides a broadcast selected by a user from among broadcast signals transmitted from a broadcast station, and the broadcast image is displayed on the display. A TV is a typical example of a display device.

Meanwhile, with the development of a wired/wireless communication network, digital devices that display a screen and output data that a user can visually recognize may be interconnected through a wired/wireless communication network.

In particular, a terminal device such as a smart phone may perform a mirroring (screen casting) service with a display device such as a TV. The mirroring service is a technology in which a screen of a terminal device is transmitted to a display device, and the display device displays the screen with the terminal device.

It is an object of the present invention to provide a display device capable of minimizing a time delay occurring during a mirroring (screen casting) service, a control method thereof, and a terminal device performing a mirroring service with the display device.

Another object of the present invention is to provide a display device capable of providing a mirroring service without time delay even in a terminal device having a novel aspect ratio, a control method thereof, and a terminal device performing a mirroring service with the display device.

The objects of the present invention are not limited to the above-mentioned objects, and other objects and advantages of the present invention not mentioned may be understood by the following description, and will be more clearly understood by the examples of the present invention. Moreover, it will be readily apparent that the objects and advantages of the present invention may be realized by the means and combinations thereof indicated in the claims.

According to an embodiment of the present invention, the terminal device transmits aspect ratio information of the terminal device to the display device, and the display device displays the screen displayed on the terminal device on the screen displayed on the display device based on the received aspect ratio information. Overlay and display together.

A terminal device according to an embodiment of the present invention is a terminal device that performs a mirroring service with a display device, and controls a communication unit, a display unit, and a communication unit connected to the display device in communication with the display device based on a first protocol, and the communication unit and the display unit and a processor, wherein the processor controls the communication unit to transmit aspect ratio information of the display to the display device.

A display apparatus according to an embodiment of the present invention is a display apparatus that performs a mirroring service with a terminal apparatus, and includes a communication unit and a display connected to the terminal apparatus based on a first protocol, and controlling the communication unit and the display unit. a processor, wherein the communication unit receives aspect ratio information of the display unit from the terminal device, and the processor overlays the screen of the terminal device on the screen displayed on the display unit based on the aspect ratio information and displays the screen together to control the display unit.

A method of controlling a display apparatus according to an embodiment of the present invention is a method of controlling a display apparatus performing a mirroring service with a terminal apparatus, the method comprising: a communication unit receiving aspect ratio information of the display unit from the terminal apparatus; , controlling to overlay and display a screen of the terminal device on the screen displayed on the display unit based on the aspect ratio information.

According to the present invention, it is possible to minimize the time delay occurring during the mirroring (screen casting) service.

In addition, according to the present invention, it is possible to provide a mirroring service without time delay even in a terminal device having a new aspect ratio.

In addition to the above-described effects, the specific effects of the present invention will be described together while describing specific details for carrying out the invention below.

1 is a diagram illustrating a schematic configuration of a digital device according to an embodiment of the present invention.
2 is a diagram illustrating a schematic configuration of a network system according to an embodiment of the present invention.
3 is a diagram illustrating a flowchart of capability negotiation of WFD using RTSP.
4 is a diagram illustrating a schematic configuration of a display device according to an embodiment of the present invention.
5 is a diagram illustrating a schematic configuration of a terminal device according to an embodiment of the present invention.
6 is a diagram illustrating an operation flowchart of a network system according to an embodiment of the present invention.
7 and 8 are diagrams illustrating an example of an overlay mirroring service according to an embodiment of the present invention.

The above-described objects, features and advantages will be described below in detail with reference to the accompanying drawings, and accordingly, those of ordinary skill in the art to which the present invention pertains will be able to easily implement the technical idea of the present invention. In describing the present invention, if it is determined that a detailed description of a known technology related to the present invention may unnecessarily obscure the gist of the present invention, the detailed description will be omitted. Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the accompanying drawings. In the drawings, the same reference numerals are used to indicate the same or similar components.

Although the first, second, etc. are used to describe various elements, these elements are not limited by these terms, of course. These terms are only used to distinguish one component from other components, and unless otherwise stated, the first component may be the second component, of course.

Also, when it is described that a component is "connected", "coupled" or "connected" to another component, the components may be directly connected or connected to each other, but other components are "interposed" between each component. It is to be understood that “or, each component may be “connected,” “coupled,” or “connected” through another component.

Throughout the specification, unless otherwise stated, each element may be singular or plural.

As used herein, the singular expression includes the plural expression unless the context clearly dictates otherwise. In this specification, terms such as "consisting of" or "comprising" should not be construed as necessarily including all of the various components or various steps described in the specification, some of which components or some steps are It should be construed that it may not include, or may further include additional components or steps.

Throughout the specification, when “A and/or B” is used, it means A, B or A and B, unless specifically stated to the contrary, and when “C to D” is used, it means that there is no specific contrary description. Unless otherwise specified, it means C or more and D or less.

On the other hand, the display device described herein is, for example, an intelligent display device in which a computer support function is added to a broadcast reception function, and is faithful to a broadcast reception function and an Internet function is added, so that a handwritten input device, a touch screen, or a space A more user-friendly interface such as a remote control can be provided. In addition, it is connected to the Internet and a computer with the support of a wired or wireless Internet function, and functions such as e-mail, web browsing, banking, or games can also be performed. A standardized general-purpose OS may be used for these various functions.

In addition, the mobile terminal device described in this specification is a mobile phone, a smart phone (smart phone), a laptop computer (laptop computer), a digital broadcasting terminal, PDA (personal digital assistants), PMP (portable multimedia player), navigation, slate PC ( slate PC), tablet PC (tablet PC), ultrabook (ultrabook), wearable device (e.g., watch-type terminal (smartwatch), glass-type terminal (smart glass), HMD (head mounted display)), etc. may be included.

However, it is easy for those skilled in the art that the configuration according to the embodiments described in this specification may be applied to fixed terminal devices such as digital display devices, desktop computers, digital signage, etc., except when applicable only to mobile terminal devices. you will know

Hereinafter, a display apparatus and a control method thereof according to some embodiments of the present invention will be described.

1 is a diagram illustrating a schematic configuration of a digital device according to an embodiment of the present invention.

Referring to FIG. 1 , the digital device 100 includes a broadcast receiving unit 110 , an external device interface unit 120 , a storage unit 130 , a user interface unit 140 , a display unit 150 , and an audio output unit ( 160 ), a power supply unit 170 , and a control unit 180 .

Here, the broadcast receiving unit 110 may include a tuner 111 , a demodulator 112 , and a network interface unit 113 . However, in some cases, the broadcast receiving unit 110 may include the tuner 111 and the demodulator 112 , but may not include the network interface unit 113 , and vice versa.

Also, although not shown, the broadcast receiving unit 110 may include a multiplexer. In this case, the multiplexer may multiplex the signal demodulated by the demodulator 112 and the signal received through the network interface unit 113 . In addition, although not shown, the broadcast receiving unit 110 may include a demultiplexer. The demultiplexer may demultiplex the multiplexed signal or demultiplex a signal demodulated by the demodulator 112 or a signal received through the network interface unit 113 .

The tuner 111 tunes a specific radio frequency (RF) broadcast signal. A specific RF broadcast signal corresponds to a channel selected by a user or all channels previously stored. Also, the tuner 111 converts the RF broadcast signal into an intermediate frequency (IF) signal or a baseband signal.

For example, the tuner 111 converts an RF broadcast signal that is a digital broadcast signal into a digital IF signal (DIF), and converts an RF broadcast signal that is an analog broadcast signal into an analog baseband image or audio signal (CVBS/SIF). . That is, the tuner 111 may process both a digital broadcast signal and an analog broadcast signal. An analog baseband video or audio signal (CVBS/SIF) output from the tuner 111 may be directly input to the controller 180 .

Also, the tuner 111 may receive an RF broadcast signal of a single carrier or a multiple carrier. Meanwhile, the tuner 111 may sequentially tune and receive RF broadcast signals of all broadcast channels stored through the channel storage function and convert them into an intermediate frequency signal or a baseband signal.

The demodulator 112 may receive and demodulate the digital IF signal converted by the tuner 111 , and may perform channel decoding and the like. To this end, the demodulator 112 includes a trellis decoder, a de-interleaver, a Reed-Solomon decoder, or the like, or a convolution decoder, a deinterleaver and A Reed-Soloman decoder or the like may be provided.

The demodulator 112 may output a stream signal TS after performing demodulation and channel decoding. In this case, the stream signal may be a multiplexed image signal, an audio signal, or a data signal. For example, the stream signal may be an MPEG-2 Transport Stream (TS) in which an MPEG-2 standard video signal, a Dolby AC-3 standard audio signal, and the like are multiplexed.

The stream signal output from the demodulator 112 may be input to the controller 180 . The controller 180 may control demultiplexing, image/audio signal processing, and the like, and may control output of an image through the display unit 150 and audio through the audio output unit 160 .

The external device interface unit 120 provides an interface environment between the digital device 100 and various external devices.

External device interface unit 120, DVD (Digital Versatile Disk), Blu-ray (Blu-ray), game device, camera, camcorder (Camcorder), computer (laptop), tablet PC, smart phone, Bluetooth device (Bluetooth) device) and an external device such as a cloud, and the like, may be connected via wired/wireless. The external device interface unit 120 transmits a signal including data such as an image, video, and voice input through a connected external device to the controller 180 . The controller 180 may control the processed image, video, audio, etc. to output a data signal to a connected external device. To this end, the external device interface unit 120 may further include an A/V input/output unit (not shown) or a wireless communication unit (not shown).

The A/V input/output unit provides a USB terminal, a CVBS (Composite Video Banking Sync) terminal, a component terminal, an S-video terminal (analog), and a DVI terminal to input video and audio signals of an external device to the digital device 100. It may include a (Digital Visual Interface) terminal, a High Definition Multimedia Interface (HDMI) terminal, an RGB terminal, a D-SUB terminal, and the like.

The wireless communication unit may perform short-range wireless communication with another digital device. The digital device 100 includes, for example, Bluetooth, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, and Digital Living Network Alliance (DLNA). It may be network-connected with other digital devices according to a communication protocol such as .

In addition, the external device interface unit 120 may be connected to the set-top-box (STB) and at least one of the above-described various terminals, and may perform input/output operations with the set-top-box (STB).

Meanwhile, the external device interface unit 120 may receive an application or an application list in an adjacent external device and transmit it to the controller 180 or the storage unit 130 .

The network interface unit 113 provides an interface for connecting the digital device 100 to a wired/wireless network. The network interface unit 113 may include an Ethernet terminal for connection with a wired network, and the like, and for connection with a wireless network, WLAN (Wireless LAN), Wi-Fi, Wibro (Wireless broadband), Wimax Communication standards such as (World Interoperability for Microwave Access) and HSDPA (High Speed Downlink Packet Access) may be used.

The network interface unit 113 may transmit/receive data to/from another digital device through the connected network or another network linked to the connected network. In particular, some content data stored in the digital device 100 may be transmitted to a digital device selected from among other previously registered digital devices.

Meanwhile, the network interface unit 113 may access a predetermined web page through the connected network or another network linked to the connected network. That is, it is possible to access a predetermined web page through a network, and transmit or receive data with the corresponding server. In addition, content or data provided by a content provider or network operator may be received. That is, content such as movies, advertisements, games, VODs, broadcast signals, etc. and information related thereto may be received from the content provider or the network provider through the network. Also, it is possible to receive firmware update information and an update file provided by a network operator. It may also transmit data to the Internet or to a content provider or network operator.

The network interface unit 113 may select and receive a desired application from among applications opened through a network.

The storage unit 130 may store a program for each signal processing and control in the control unit 180, and may store a signal-processed image, audio, or data signal.

The storage unit 130 may temporarily store an image, audio, or data signal input from the external device interface unit 120 or the network interface unit 113 . The storage unit 130 may store information about a predetermined broadcast channel through a channel storage function.

The storage unit 130 may store an application or an application list input from the external device interface unit 120 or the network interface unit 330 .

The storage unit 130 may store various platforms to be described later.

The storage unit 130 is, for example, a flash memory type, a hard disk type, a multimedia card micro type, or a card type memory (eg, SD or XD). memory, etc.), a RAM (RAM), and a ROM (EEPROM, etc.) may include at least one type of storage medium. The digital apparatus 100 may reproduce content files (movie files, still image files, music files, document files, application files, etc.) stored in the storage unit 130 and provide them to the user.

Meanwhile, the storage unit 130 may be included in the control unit 180 , and may be provided separately from the control unit 180 .

The user input interface unit 140 transmits a signal input by the user to the control unit 180 or transmits a signal of the control unit 180 to the user.

For example, the user input interface unit 140 is connected to the remote control device 190 according to various communication methods such as an RF communication method, an infrared (IR) communication method, and the power on/off transmitted from the remote control device 190 . Control signals such as off, channel selection, and screen setting may be received and processed, or a control signal of the controller 180 may be transmitted to the remote control device 500 . Also, the user input interface unit 140 may transmit a control signal input from a local key (not shown) such as a power key, a channel key, a volume key, and a setting value to the controller 180 .

The user input interface unit 140 transmits a control signal input from a sensing unit (not shown) that senses a user's gesture to the control unit 180 or transmits a signal of the control unit 180 to a sensing unit (not shown) can be sent to Here, the sensing unit (not shown) may include a touch sensor, a voice sensor, a position sensor, a motion sensor, and the like.

The controller 180 demultiplexes a stream input through the tuner 111 , the demodulator 112 , or the external device interface unit 120 , or processes the demultiplexed signals to generate a signal for video or audio output. and output.

The image signal processed by the controller 180 may be input to the display unit 150 and displayed as an image corresponding to the image signal. Also, the image signal processed by the controller 180 may be input to an external output device through the external device interface unit 120 .

The audio signal processed by the controller 180 may be audio output to the audio output unit 160 . Also, the voice signal processed by the controller 180 may be input to an external output device through the external device interface unit 120 .

Meanwhile, the controller 180 may include a demultiplexer, an image processor, and the like.

The controller 180 may control the overall operation of the digital device 100 . For example, the controller 180 may control the tuner 111 to tune an RF broadcast corresponding to a channel selected by the user or a previously stored channel.

The controller 180 may control the digital device 100 according to a user command input through the user input interface unit 140 or an internal program. In particular, by accessing a network, an application or an application list desired by the user may be downloaded into the digital device 100 .

For example, the controller 180 controls the tuner 111 to input a signal of a selected channel according to a predetermined channel selection command received through the user input interface 140 . And it processes the video, audio or data signal of the selected channel. The controller 180 controls the channel information selected by the user to be output through the display unit 150 or the audio output unit 160 together with the processed image or audio signal.

As another example, the controller 180 transmits an image signal or an audio signal from the external device to the display unit 150 or the audio output unit 160 according to an external device image reproduction command received through the user input interface unit 140 . control so that it can be output through

Meanwhile, the controller 180 may control the display 150 to display an image. For example, a broadcast image input through the tuner 111 or an external input image input through the external device interface unit 120 or an image input through the network interface unit or an image stored in the storage unit 130 is displayed on the display unit ( 150) can be controlled to be displayed. In this case, the image displayed on the display unit 150 may be a still image or a moving image, and may be a 2D image or a 3D image.

Also, the controller 180 may control the display 150 to reproduce content. In this case, the content may be content stored in the digital device 100 , received broadcast content, or external input content input from the outside. The content may be at least one of a broadcast image, an external input image, an audio file, a still image, a connected web screen, and a document file.

Meanwhile, when entering the application view item, the controller 180 may control to display an application or an application list downloadable from an internal or external network of the digital device 100 .

The controller 180 may control to install and drive an application downloaded from an external network together with various user interfaces. Also, the controller 180 may control an image related to an application executed by a user's selection to be displayed on the display unit 150 .

Meanwhile, although not shown in the drawings, a channel browsing processing unit for generating a thumbnail image corresponding to a channel signal or an external input signal may be further provided.

The channel browsing processing unit receives the stream signal TS output from the demodulator 320 or the stream signal output from the external device interface unit 120, and extracts an image from the input stream signal to generate a thumbnail image. have. The generated thumbnail image may be input to the controller 180 as it is or after being encoded. Also, the generated thumbnail image may be encoded in a stream form and input to the controller 180 . The controller 180 may display a thumbnail list including a plurality of thumbnail images on the display 150 by using the input thumbnail images. Meanwhile, the thumbnail images in the thumbnail list may be updated sequentially or simultaneously. Accordingly, the user can easily grasp the contents of a plurality of broadcast channels.

The display unit 150 converts the image signal, the data signal, the OSD signal processed by the controller 180, or the image signal and the data signal received from the external device interface unit 120 into R, G, and B signals, respectively, and drives them. generate a signal

The display unit 150 may be a PDP display unit, an LCD display unit, an OLED display unit, a flexible display unit, a three-dimensional display unit (3D display), or the like.

On the other hand, the display unit 150 is configured as a touch screen, it is also possible to be used as an input device in addition to the output device.

The audio output unit 160 receives a signal processed by the control unit 180, for example, a stereo signal, a 3.1 channel signal, or a 5.1 channel signal, and outputs the audio signal. The audio output unit 160 may be implemented with various types of speakers.

Meanwhile, in order to detect a user's gesture, as described above, a sensing unit (not shown) including at least one of a touch sensor, a voice sensor, a position sensor, and a motion sensor may be further provided in the digital apparatus 100 . . The signal sensed by the sensing unit (not shown) may be transmitted to the control unit 3180 through the user input interface unit 140 .

On the other hand, a photographing unit (not shown) for photographing the user may be further provided. Image information captured by the photographing unit (not shown) may be input to the controller 180 .

The controller 180 may detect a user's gesture by individually or in combination with an image captured by a photographing unit (not shown) or a signal sensed from a sensing unit (not shown).

The power supply unit 170 supplies power to the components of the digital device 100 . In particular, to supply power to the control unit 180 that can be implemented in the form of a system on chip (SoC), the display unit 150 for displaying an image, and the audio output unit 160 for outputting audio. can

To this end, the power supply unit 170 may include a converter (not shown) that converts AC power into DC power. On the other hand, when the display unit 150 is implemented as a liquid crystal panel including a plurality of backlight lamps, for luminance variable or dimming driving, PWM (Pulse Width Modulation) operable inverter (inverter) ( Not shown) may be further provided.

The digital apparatus 100 may be a digital broadcast receiver capable of processing a digital broadcast signal of a fixed or mobile ATSC method or a DVB method.

In addition, the digital apparatus 100 may omit some components from among the illustrated components, or, conversely, further include components not shown. Meanwhile, unlike the above, the digital device 100 may receive and reproduce content through a network interface unit or an external device interface unit without a tuner and a demodulator.

Meanwhile, among the digital devices 100 , a terminal device such as a smart phone and a display device such as a TV may perform a mirroring (screen casting) service through interworking. In particular, the mirroring service may include an overlay mirroring service in which a screen displayed on the display device and a screen displayed on the terminal device are overlaid together on the display device.

When performing the conventional overlay mirroring service, the display device transmits the screen displayed on the terminal device as it is to the display device, and the display device transmits the received screen ratio (that is, the length of the screen in the horizontal direction to the length in the vertical direction of the terminal device) ratio), and the screen of the terminal device may be displayed together with the screen of the display device based on the calculated ratio of the screen. The process of calculating the aspect ratio of the screen is performed through a background task. For example, the display apparatus calculates the screen ratio by calculating the horizontal/vertical length of a black area (ie, an empty area in which image data does not exist) existing on the screen of the terminal device.

However, in the conventional overlay mirroring service, a time delay may occur while the display device calculates the screen ratio. Also, when a new terminal device whose screen ratio is unknown performs an overlay mirroring service, there is no definition of a new screen ratio, and thus a blank may occur on the screen of the terminal device during the overlay process.

Hereinafter, a display apparatus and a control method thereof according to an embodiment of the present invention will be described with reference to the above description.

2 is a diagram illustrating a schematic configuration of a network system according to an embodiment of the present invention.

Referring to FIG. 2 , the network system may be a small network system installed at home, and includes a display device 210 and a terminal device 220 .

The display device 210 may be, for example, a TV, and displays a video to the user. For example, a video received from an external server is displayed through a communication connection with an external server through the Internet.

The terminal device 220 may be, for example, a smart phone, which is a mobile terminal device, and is communicatively connected to the display device 210 . A software program may be installed in the terminal device 220 , the installed software program is executed, and an execution screen of the software program is displayed to the user.

Meanwhile, the display device 210 and the terminal device 220 may perform a mirroring service. That is, the screen displayed by the terminal device 220 is transmitted to the display device 210 , and the display device 210 may display the received screen of the terminal device 220 to the user. In particular, the display device 210 may display the screen of the terminal device 220 together with the screen to be provided (overlay mirroring).

Meanwhile, a mirroring service may be performed based on the first protocol, and the display device 210 and the terminal device 220 may be communicatively connected through WiFi. In particular, the first protocol may be a Real-Time Streaming Protocol (RTSP).

RTSP is a communication protocol (protocol) for transmitting and receiving voice or video in real time. RTSP is a network control protocol designed for use in communication systems to control streaming media and servers. RTSP is used to establish a media session between end points and to control the established session. The clients of the media server facilitate real-time media streaming control by issuing VHS style commands such as play, record, pause, etc.

Compared to the HTTP protocol, the RTSP protocol is similar in syntax and operation, but the RTSP is different in that it defines a control sequence useful for controlling multimedia playback. Another difference is that HTTP is stateless while RTSP is stateful.

3 is a diagram illustrating a flowchart of capability negotiation of WFD using RTSP.

The terminal device 220 (ie, WFD Source or Primary Sink) may transmit an RTSP M1 request message for starting RSTP procedure and WFD capability negotiation to the display device 210 (ie, WFD Sink or Second Sink) (step S302).

The RTSP M1 request message may include an RTSP OPTIONS request for determining a set of RTSP methods supported by the display device 210 (ie, WFD Sink).

Upon receiving the RTSP M1 request message, the display device 210 may transmit an RTSP M1 response message listing RTSP methods supported by the display device 210 to the terminal device 220 (step S304).

Subsequently, the display device 210 may transmit an RTSP M2 request message for determining the RTSP method set supported by the terminal device 220 (step S306).

When the RTSP M2 request message is received, the terminal device 220 may transmit an RTSP M2 response message listing the RTSP methods supported by the terminal device 220 to the display device 210 (step S308).

Then, the terminal device 220 may transmit an RTSP M3 request message (RTSP GET_PARAMETER request message) specifying a list of WFD capabilities to know to the display device 210 (step S310).

When the RTSP M3 request message is received, the display device 210 may transmit an RTSP M3 response message (RTSP GET_PARAMETER response message) to the terminal device 220 (step S312).

Thereafter, the terminal device 220 determines an optimal parameter set to be used during the WFD session based on the RTSP M3 response message, and displays an RTSP M4 request message (RTSP SET_PARAMETER request message) including the determined parameter set to the display device 210 . can be transmitted to (step S314).

Upon receiving the RTSP M4 request message, the display device 210 may transmit an RTSP M4 response message (RTSP SET_PARAMETER response message) to the terminal device 220 (step S316).

Hereinafter, the display device 210 and the mobile terminal device 220 will be described in more detail.

4 is a diagram illustrating a schematic configuration of a display device 210 according to an embodiment of the present invention.

Referring to FIG. 4 , the display device 210 includes a first communication unit 211 , a first display unit 212 , a first memory 213 , and a first processor 214 .

Hereinafter, the function of each component will be described in detail.

The first communication unit 211 is communicatively connected to a plurality of external servers and the terminal device 220 . In this case, the communication unit 211 may include a mobile communication module for communication with a plurality of external servers and a short-range communication module for communication with the terminal device 220 .

The mobile communication module may perform communication using technical standards or communication methods for mobile communication. Here, the technical standards or communication methods are Global System for Mobile communication (GSM), Code Division Multi Access (CDMA), Code Division Multi Access 2000 (CDMA2000), Enhanced Voice-Data Optimized or Enhanced Voice-Data Only (EV-DO). ), Wideband CDMA (WCDMA), High Speed Downlink Packet Access (HSDPA), High Speed Uplink Packet Access (HSUPA), Long Term Evolution (LTE), Long Term Evolution-Advanced (LTE-A), 5G networks, etc. can

The short-range communication module is for short range communication, and includes Bluetooth, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, and Near Field Communication (NFC). ), Wireless-Fidelity (Wi-Fi), Wi-Fi Direct, and Wireless Universal Serial Bus (USB) technology.

The first display unit 212 may include a liquid crystal display unit (LCD), a light emitting diode display unit (LED), an organic light emitting diode display unit (OLED), etc., and performs a function of displaying an image or a moving image to the user. .

The first memory 213 may be a volatile and/or non-volatile memory, and stores commands or data related to at least one other component of the display device 210 . The first memory 213 may store a software program for performing a mirroring service.

The first processor 214 may include one or more of a central processing unit, an application processor, or a communication processor. The first processor 214 may execute an operation or data processing related to control and/or communication of at least one other component of the display device 210 , and may execute the command. In particular, the first processor 214 may execute the above-described software program.

5 is a diagram illustrating a schematic configuration of a terminal device 220 according to an embodiment of the present invention.

Referring to FIG. 5 , the terminal device 220 includes a second communication unit 221 , a second display unit 222 , a sensing unit 223 , a second memory 224 , and a second processor 225 .

Hereinafter, the function of each component will be described in detail.

The second communication unit 221 is communicatively connected to the display device 210 . In this case, the communication unit 211 may include a short-range communication module for communication with the display device 210 . Of course, the second communication unit 221 may be communicatively connected with other digital devices, and may include a mobile communication module for communication with other digital devices.

The second display unit 222 may include a liquid crystal display unit (LCD), a light emitting diode display unit (LED), an organic light emitting diode display unit (OLED), and the like. The second display unit 222 performs a function of displaying an image or a moving image to the user. Also, the second display unit 222 may be a touch display unit capable of receiving a touch signal.

The sensing unit 223 acquires internal information and surrounding environment information of the terminal device 220 using various sensors.

For example, the sensing unit 223 may include a proximity sensor, a touch sensor, an acceleration sensor, a magnetic sensor, a gravity sensor (G-sensor), and a gyroscope. sensor), and the like.

The second memory 224 may be a volatile and/or non-volatile memory, and stores instructions or data related to at least one other component of the terminal device 220 . The second memory 224 may store a software program for performing a mirroring service.

Stores data associated with a plurality of software programs.

The second processor 225 may include one or more of a central processing unit, an application processor, or a communication processor. The second processor 225 may execute an operation or data processing related to control and/or communication of at least one other component of the terminal device 220 , and may execute the command.

Hereinafter, an operation in which the terminal device 220 controls the display device 210 will be described in more detail with reference to the drawings below.

6 is a diagram illustrating an operation flowchart of a network system according to an embodiment of the present invention.

The network system includes the display device 210 and the terminal device 220 as described above. In this case, the steps shown on the left correspond to the control method of the terminal device 220 , and the steps shown on the right correspond to the control method of the display device 210 .

Hereinafter, the process performed for each step will be described in more detail.

In step S602, a mirroring service is requested from the user.

In this case, the mirroring service may be performed based on RTSP.

In step S604 , the second communication unit 221 attempts to connect according to the RTSP. In this case, the first protocol may be RTSP.

In step S606 , the terminal device 220 and the display device 210 perform WFD capability negotiation. This is the same as the content shown in FIG. 3 .

Meanwhile, within the WFD capability negotiation, the second communication unit 221 may transmit ratio information of the screen displayed on the second display unit 222 to the first communication unit 212 .

Here, the screen ratio information may mean a ratio of length information in a horizontal direction (first direction) to length information in a vertical direction (second direction) of the screen displayed on at least a part of the second display unit 220 .

That is, the second display unit 220 has a physical size, and the screen may be displayed on all of the second display unit 220 or a part of the second display unit 220 . The ratio of the screen may be determined according to a user's setting or based on a default setting of a software program implementing the screen. Accordingly, the second processor 225 may calculate the aspect ratio of the second display unit 220 according to the set situation.

For example, the aspect ratio of the screen may be 4:3, 18:9, 16:9, or the like.

Meanwhile, the second processor 225 knows in advance screen information of the software program displayed on the second display unit 221 . Accordingly, the terminal device 220 may transmit screen ratio information to the display device 210 without a separate additional procedure.

According to an embodiment of the present invention, the second communication unit 221 may transmit screen ratio information to the first communication unit 211 using the M4 request message of the RTSP.

The M4 request message of the RTSP may be a message used to transmit additional information on the RTSP. Accordingly, the terminal device 220 may transmit screen ratio information to the display device 210 through the M4 request message of the RTSP.

In step S608, the terminal device 220 and the display device 210 are connected to the communication for the mirroring service is completed.

In step S610, the first processor 214 may overlay the screen displayed on the second display unit 222 on the screen displayed on the first display unit 212 based on the received aspect ratio information and display it together. have.

In summary, in the conventional overlay mirroring service, a time delay may occur in the process in which the display device calculates the screen ratio, and when a new terminal device whose screen ratio is unknown performs the overlay mirroring service, the screen is blank during the overlay process. This can happen.

However, in the present invention, the terminal device 220 transmits the screen ratio information known in advance to the display device 210 , and the display device 210 overlays and displays the screens using the received screen ratio information. can do.

Accordingly, since the display apparatus 210 does not need to calculate the screen ratio, a time delay does not occur. Also, even if the new terminal device 220 is overlaid and displayed on the display device 210 , a blank does not occur on the screen of the overlaid terminal device 220 .

Meanwhile, although not shown in FIG. 6 , a case in which a ratio of a screen displayed on the terminal device 220 is changed may occur. Even in this case, the terminal device 220 may transmit the changed screen ratio to the display device 210 . That is, the second processor 225 may control the second communication unit 221 to transmit screen ratio information to the display device 210 whenever the screen ratio is changed. The display device 210 may overlay and display the screen displayed on the second display unit 222 on the screen displayed on the first display unit 212 based on the changed screen ratio information.

In this case, the change in the aspect ratio may be detected through the acceleration sensor included in the sensing unit 223 . That is, the second processor 225 determines whether the terminal device 220 is positioned in a horizontal direction or a vertical direction based on the information sensed by the acceleration sensor, and the terminal device 220 moves from the horizontal direction to the vertical direction. When the position is changed to , or the terminal device 220 changes the position from the vertical direction to the horizontal direction, it may be determined that the screen ratio is changed.

7 is a diagram for explaining the above-described situation. In this case, it is assumed that the ratio of the physical screen of the second display unit 222 is 18:9 based on the horizontal mode.

Referring to FIG. 7A , when the terminal device 220 is positioned in the portrait mode and the screen of the second display unit 222 is displayed on all of the second display unit 222 at a first time point, the terminal The device 220 may transmit information of a ratio of 9:18 to the display device 210 .

And, referring to (b) of FIG. 7 , at a second time point after the first time point, the terminal device 220 is positioned in the landscape mode and the screen of the second display unit 222 is all of the second display unit 222 . is displayed, the terminal device 220 transmits 18:9 ratio information to the display device 210 . In this case, the display apparatus 210 may change the direction of the screen of the second display unit 222 displayed on the first display unit 212 based on the changed screen ratio information.

Meanwhile, an image may be displayed on the second display device 222 of the terminal device 220 . In this case, it is a concept including video and video. In this case, the terminal device 220 may further transmit image ratio information to the display device 210 . Also, the terminal device 220 may further transmit image ratio information to the display device 210 whenever the image ratio is changed. In this case, the ratio information of the image may mean a ratio of length information in a horizontal direction (first direction) to length information in a vertical direction (second direction) of the image.

8 is a diagram for explaining the above-described situation. In this case, it is assumed that the ratio of the physical screen of the second display unit 222 is 18:9 based on the horizontal mode.

Referring to (a) of FIG. 8 , at a first viewpoint, the terminal device 220 is positioned in a portrait mode, an image is displayed in the second display unit 222 in 16:9, and an upper region and a lower portion of the image are displayed. Regions are indicated as black regions. In this case, the terminal device 220 transmits, to the display device 210 , data related to the image except for the black region along with ratio information of the displayed image, and the display device 210 overlays only the image displayed on the terminal device 220 . can be displayed as

Referring to (b) of FIG. 8 , at the second time point, the terminal device 220 is positioned in the landscape mode, the image is displayed in the second display unit 222 in 16:9, and the left region and the right region of the image are displayed. The loan area is indicated by a black area. Even in this case, the terminal device 220 transmits data related to the image except for the black region along with the ratio information of the displayed image to the display device 210 , and the display device 210 transmits only the image displayed on the terminal device 220 . It can be displayed as an overlay.

In other words, since the terminal device 200 provides the aspect ratio information to the display device 210 , a time delay occurring during the mirroring service can be minimized. In addition, since the terminal device 200 provides the image ratio information together with the screen ratio information to the display device 210 , a mirroring service can be provided without time delay even in a terminal device having a new screen ratio.

In addition, even if all the components constituting the embodiment of the present invention are described as being combined or operated as one, the present invention is not necessarily limited to this embodiment, and all components within the scope of the present invention are One or more may be selectively combined to operate. In addition, all of the components may be implemented as one independent hardware, but some or all of the components are selectively combined to perform some or all functions of the combined components in one or a plurality of hardware program modules It may be implemented as a computer program having Codes and code segments constituting the computer program can be easily deduced by those skilled in the art of the present invention. Such a computer program is stored in a computer readable storage medium (Computer Readable Media), read and executed by the computer, thereby implementing the embodiment of the present invention. The storage medium of the computer program includes a magnetic recording medium, an optical recording medium, and a storage medium including a semiconductor recording device. In addition, the computer program implementing the embodiment of the present invention includes a program module that is transmitted in real time through an external device.

As described above, the present invention has been described with reference to the illustrated drawings, but the present invention is not limited by the embodiments and drawings disclosed in this specification, and various methods can be obtained by those skilled in the art within the scope of the technical spirit of the present invention. It is obvious that variations can be made. In addition, although the effects according to the configuration of the present invention are not explicitly described and described while describing the embodiments of the present invention, it is natural that the effects predictable by the configuration should also be recognized.

Claims (14)

A terminal device performing a mirroring service with a display device, the terminal device comprising:
a communication unit communicatively connected to the display device based on a first protocol;
display unit; and
A processor for controlling the communication unit and the display unit; including,
The processor controls the communication unit to transmit the aspect ratio information of the display unit to the display device, the terminal device.
According to claim 1,
The first protocol is a real-time streaming protocol (RTSP),
The aspect ratio information is transmitted to the display device through an M4 request message of the RTSP.
According to claim 1,
The display device overlays a screen of the display unit on a screen displayed on the display device based on the aspect ratio information and displays the screen together.
4. The method of claim 3,
The processor controls the communication unit to transmit information of the aspect ratio to the display device whenever the aspect ratio is changed.
5. The method of claim 4,
Accelerometer; further comprising
The processor is
determining whether the terminal device is positioned in a horizontal direction or a vertical direction based on the information sensed by the acceleration sensor;
determining that the aspect ratio is changed when the terminal device changes a position from a horizontal direction to a vertical direction or when the terminal device changes a position from a vertical direction to a horizontal direction.
4. The method of claim 3,
The processor may control the communication unit to further transmit the image ratio information to the display apparatus when the image is displayed on the display unit.
7. The method of claim 6,
The processor controls the communication unit to transmit information on the image ratio to the display device whenever the image ratio is changed.
7. The method of claim 6,
The display unit displays an image and a black area located adjacent to the image together,
The processor controls the communication unit to transmit data related to the image except for the black region to the display device.
9. The method of claim 8,
wherein the display device overlays and displays only the image with a screen displayed on the display device based on the image ratio information.
A display device performing a mirroring service with a terminal device, the display device comprising:
a communication unit communicatively connected to the terminal device based on a first protocol;
display unit; and
A processor for controlling the communication unit and the display unit; including,
The communication unit receives the aspect ratio information of the display from the terminal device,
and the processor controls the display unit to overlay and display a screen of the terminal device on the screen displayed on the display unit based on the aspect ratio information.
11. The method of claim 10,
The first protocol is a real-time streaming protocol (RTSP),
The aspect ratio information is received from the terminal device through the M4 request message of the RTSP.
11. The method of claim 10,
The communication unit further receives ratio information of the image displayed on the terminal device,
and the processor controls the display unit to overlay and display a screen of the terminal device on the screen displayed on the display unit further based on the calculated ratio information of the image.
14. The method of claim 13,
the terminal device displays the image and a black region adjacent to the image together, and transmits data related to the image excluding the black region to the communication unit;
The processor controls the display unit to display only the image by overlaying the screen displayed on the display unit on the basis of the image ratio information.
A method for controlling a display device performing a mirroring service with a terminal device, the method comprising:
receiving, by a communication unit, aspect ratio information of the display from the terminal device;
Controlling, by the processor, to overlay and display the screen of the terminal device on the screen displayed on the display unit based on the aspect ratio information.

Images