KR20220160817A - A display device and operating method thereof - Google Patents

Abstract

A display device according to an embodiment of the present disclosure includes: a control part for extracting a sample and decoding information from content when a content playback command is received and decoding the sample according to the decoding information; and a display for outputting the content through the decoded sample. The control part may collect the sample based on a reproduction state of the content or a state of a buffer in which the decoded sample is stored before being output, and decode collected samples together.

Description

A display device and its operating method {A display device and operating method thereof}

The present disclosure relates to a display device and an operating method thereof, and more particularly, to content decoding of a display device.

To protect A/V content of a streaming-based OTT (Over The Top) service or ATSC 3.0 broadcasting, a content protection technology such as Digital Rights Management (DRM) or Conditional Access System (CAS) is used.

Due to the server-client nature of technologies such as DRM and CAS, the display device corresponding to the client must also be equipped with DRM and CAS technologies. In addition, a display device equipped with technologies such as DRM and CAS needs to support a secure execution environment such as Trusted Execution Environment (TEE) to satisfy the level of security required by the corresponding technologies.

1 is an exemplary diagram of a system supporting TEE. In a system that supports TEE, it is divided into two modes: secure world, a secure execution environment, and normal world, a non-secure execution environment, and various applications, services and embedded OS are executed in the normal world. In addition, key management, storage, and decryption processing of encrypted A/V contents, such as DRM and CAS, are safely executed within the Secure World, which cannot be accessed from the Normal World.

In particular, functions such as key management, storage, and content decryption are performed in the TA (Trusted Application) of FIG. . In the case of a display device supporting multi-tasking, there are TAs that need to be concurrently processed at the same time, and as the number of TAs increases and diversifies, the number of TAs that need to be simultaneously operated also increases.

Simultaneous and parallel processing of these TAs may cause processing delays depending on computational performance, and accordingly, various problems such as buffering, playback interruption, and scene skipping may occur in applications operating in the normal world, thereby reducing quality. It is lowered.

In addition, the frame rate of contents encoded in OTT VOD, live streaming, and broadcasting tends to increase from 24 or 25 fps to 60 or 120 fps.

Accordingly, there is a demand for a content decoding method for processing and reproducing 60 fps and 120 fps based content without problems even in an environment in which multiple TAs are concurrently processed.

An object of the present disclosure is to provide a display device and a method of operating the same that minimize processing delay when decoding content to be reproduced through a secure execution environment.

A display device according to an embodiment of the present disclosure extracts a sample and decoding information from the content when receiving a content playback command, and a control unit for decoding the sample according to the decoding information, and outputting the content through the decoded sample. A display may be included, and the controller may collect the samples based on a playback state of the content or a state of a buffer in which the decoded samples are stored before being output, and decode the collected samples together.

The controller may collect the samples and decoding information when the content is in a playback state, and request decoding of a plurality of pre-collected samples when the content is not in a playback state.

The control unit may determine that the content is not in a playback state in the case of a state within a predetermined time after the start of reproduction of the content, a search state, and a pause state.

If the state of the buffer is sufficient, the controller collects the samples and decoding information, and if the state of the buffer is not sufficient, it can request decoding of a plurality of pre-collected samples.

The controller may obtain whether the state of the buffer is sufficient based on the current time stamp of the content and the time stamp of the samples collected in the buffer.

The controller may request decoding of a plurality of pre-collected samples if there are enough samples collected in the buffer, and collect the samples and decoding information if there are not enough samples collected in the buffer.

The controller may count the number of samples collected in the buffer and obtain whether there are enough samples collected in the buffer based on the counted number.

The control unit may manage a chunk list by including a plurality of samples to be decoded together in one chunk.

The control unit may determine a chunk including a sample according to whether a key rotation in which an encryption key is changed occurs.

When the key rotation occurs, the control unit may manage a chunk list such that samples having different encryption keys are included in different chunks.

According to an embodiment of the present disclosure, since a plurality of samples are decrypted together, there is an advantage in minimizing a processing delay problem by reducing the number of times of communication for requesting and obtaining an encryption key.

According to the present disclosure, since the number of samples to be decoded together is flexibly adjusted according to the playback state or buffer state of content, the performance of decoding operation is improved while minimizing the occurrence of buffering and screen interruption.

1 is an exemplary diagram of a system supporting TEE.
2 is a block diagram illustrating the configuration of a display device according to an embodiment of the present invention.
3 is a block diagram of a remote control device according to an embodiment of the present invention.
Figure 4 shows an example of the actual configuration of the remote control device according to an embodiment of the present invention.
5 shows an example of utilizing a remote control device according to an embodiment of the present invention.
6 is an exemplary diagram of a TEE-based content protection technology system in a display device.
7 is a flowchart illustrating a method of decoding in a conventional content protection technology system.
8 is a block diagram illustrating a configuration for performing decoding by collecting a plurality of samples according to an embodiment of the present disclosure.
9 is a flowchart illustrating a method of operating a display device according to an embodiment of the present disclosure.
10 is a flowchart illustrating a method of managing a plurality of samples as chunks by a display device according to an embodiment of the present disclosure.
11 is a flowchart illustrating a method of decoding a plurality of samples in chunk units by a display device according to an embodiment of the present disclosure.

Hereinafter, embodiments related to the present invention will be described in more detail with reference to the drawings. The suffixes “module” and “unit” for the components used in the following description are given or used interchangeably in consideration of only the ease of writing the specification, and do not have meanings or roles that are distinct from each other by themselves.

2 is a block diagram illustrating the configuration of a display device according to an embodiment of the present invention.

Referring to FIG. 2 , the display device 100 includes a broadcast reception unit 130, an external device interface unit 135, a storage unit 140, a user input interface unit 150, a control unit 170, and a wireless communication unit 173. , a voice acquisition unit 175, a display unit 180, an audio output unit 185, and a power supply unit 190.

The broadcast receiving unit 130 may include a tuner 131, a demodulation unit 132 and a network interface unit 133.

The tuner 131 may select a specific broadcasting channel according to a channel selection command. The tuner 131 may receive a broadcast signal for a selected specific broadcast channel.

The demodulator 132 may separate the received broadcast signal into a video signal, an audio signal, and a data signal related to the broadcast program, and restore the separated video signal, audio signal, and data signal into a form capable of being output.

The network interface unit 133 may provide an interface for connecting the display device 100 to a wired/wireless network including the Internet network. The network interface unit 133 may transmit or receive data with other users or other electronic devices through a connected network or another network linked to the connected network.

The network interface unit 133 may access a predetermined web page through a connected network or another network linked to the connected network. That is, by accessing a predetermined web page through a network, data can be transmitted or received with a corresponding server.

Also, the network interface unit 133 may receive content or data provided by a content provider or network operator. That is, the network interface unit 133 may receive content and related information, such as movies, advertisements, games, VOD, and broadcast signals, provided from content providers or network providers through a network.

In addition, the network interface unit 133 may receive firmware update information and an update file provided by a network operator, and may transmit data to the Internet or a content provider or network operator.

The network interface unit 133 may select and receive a desired application among applications open to the public through a network.

The external device interface unit 135 may receive an application or an application list in an adjacent external device and transfer the received application to the controller 170 or the storage unit 140 .

The external device interface unit 135 may provide a connection path between the display device 100 and an external device. The external device interface unit 135 may receive at least one of video and audio output from an external device connected to the display device 100 by wire or wirelessly, and transmit the received image to the controller 170 . The external device interface unit 135 may include a plurality of external input terminals. The plurality of external input terminals may include an RGB terminal, one or more High Definition Multimedia Interface (HDMI) terminals, and component terminals.

An image signal of an external device input through the external device interface unit 135 may be output through the display unit 180 . The audio signal of the external device input through the external device interface unit 135 may be output through the audio output unit 185 .

An external device connectable to the external device interface unit 135 may be any one of a set-top box, a Blu-ray player, a DVD player, a game machine, a sound bar, a smartphone, a PC, a USB memory, and a home theater, but this is only an example. .

In addition, some content data stored in the display apparatus 100 may be transmitted to another user pre-registered in the display apparatus 100 or to a user selected from among other electronic devices or to a selected electronic device.

The storage unit 140 may store programs for processing and controlling each signal in the control unit 170 and may store signal-processed video, audio, or data signals.

In addition, the storage unit 140 may perform a function for temporarily storing video, audio, or data signals input from the external device interface unit 135 or the network interface unit 133, and stores a predetermined value through a channel storage function. It can also store information about the image.

The storage unit 140 may store an application input from the external device interface unit 135 or the network interface unit 133 or an application list.

The display device 100 may reproduce and provide content files (video files, still image files, music files, document files, application files, etc.) stored in the storage unit 140 to the user.

The user input interface unit 150 may transmit a signal input by the user to the controller 170 or may transmit a signal from the controller 170 to the user. For example, the user input interface unit 150 uses various communication methods such as Bluetooth, Ultra Wideband (WB), ZigBee, Radio Frequency (RF) communication, or IR (Infrared) communication. Control signals such as power on/off, channel selection, and screen setting may be received and processed from the remote control device 200, or a control signal from the controller 170 may be transmitted to the remote control device 200.

In addition, the user input interface unit 150 may transmit a control signal input from a local key (not shown) such as a power key, a channel key, a volume key, and a set value to the control unit 170 .

The image signal processed by the controller 170 may be input to the display unit 180 and displayed as an image corresponding to the corresponding image signal. In addition, the image signal processed by the control unit 170 may be input to an external output device through the external device interface unit 135 .

The audio signal processed by the controller 170 may be output as audio to the audio output unit 185 . Also, the voice signal processed by the control unit 170 may be input to an external output device through the external device interface unit 135 .

In addition, the controller 170 may control overall operations within the display device 100 .

In addition, the controller 170 may control the display device 100 according to a user command input through the user input interface unit 150 or an internal program, and access the network to display an application or application list desired by the user. It can be downloaded within (100).

The control unit 170 allows the channel information selected by the user to be output through the display unit 180 or the audio output unit 185 together with the processed video or audio signal.

In addition, the control unit 170 receives an external device video playback command received through the user input interface unit 150, from an external device input through the external device interface unit 135, for example, a camera or a camcorder, A video signal or audio signal can be output through the display unit 180 or the audio output unit 185.

Meanwhile, the controller 170 may control the display unit 180 to display an image, for example, a broadcast image input through the tuner 131 or an external input input through the external device interface unit 135 An image, an image input through the network interface unit, or an image stored in the storage unit 140 may be controlled to be displayed on the display unit 180 . In this case, the image displayed on the display unit 180 may be a still image or a moving image, and may be a 2D image or a 3D image.

In addition, the controller 170 may control content stored in the display device 100, received broadcast content, or external input content input from the outside to be reproduced, and the content includes a broadcast image, an external input image, an audio file, and a still image. It can be in various forms, such as an image, a connected web screen, and a document file.

The wireless communication unit 173 may communicate with an external device through wired or wireless communication. The wireless communication unit 173 may perform short range communication with an external device. To this end, the wireless communication unit 173 uses Bluetooth™, Bluetooth Low Energy (BLE), Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, and Near NFC (Near Field Communication), Wi-Fi (Wireless-Fidelity), Wi-Fi Direct, and wireless USB (Wireless Universal Serial Bus) technology may be used to support short-distance communication. The wireless communication unit 173 may be used between the display device 100 and a wireless communication system, between the display device 100 and other display devices 100, or between the display device 100 and the display device 100 through wireless local area networks. Wireless communication between networks in which the display device 100 (or an external server) is located may be supported. The local area network may be a local area wireless personal area network (Wireless Personal Area Networks).

Here, the other display device 100 is a wearable device capable of (or interlocking) exchanging data with the display device 100 according to the present invention (for example, a smart watch), smart glasses (smart glass), head mounted display (HMD), and a mobile terminal such as a smart phone. The wireless communication unit 173 may detect (or recognize) a communicable wearable device around the display apparatus 100 . Furthermore, when the detected wearable device is an authenticated device to communicate with the display apparatus 100 according to the present invention, the controller 170 transmits at least a portion of data processed by the display apparatus 100 to a wireless communication unit ( 173) to the wearable device. Accordingly, a user of the wearable device may use data processed by the display apparatus 100 through the wearable device.

The voice acquisition unit 175 may acquire audio. The voice acquisition unit 175 may include at least one microphone (not shown), and may acquire audio around the display device 100 through the microphone (not shown).

The display unit 180 converts the video signal, data signal, OSD signal processed by the control unit 170 or the video signal or data signal received from the external device interface unit 135 into R, G, and B signals, respectively, and drives the display unit 180. signal can be generated.

On the other hand, since the display device 100 shown in FIG. 2 is only one embodiment of the present invention. Some of the illustrated components may be integrated, added, or omitted according to specifications of the display device 100 that is actually implemented.

That is, if necessary, two or more components may be combined into one component, or one component may be subdivided into two or more components. In addition, functions performed in each block are for explaining an embodiment of the present invention, and the specific operation or device does not limit the scope of the present invention.

According to another embodiment of the present invention, the display device 100 does not include a tuner 131 and a demodulation unit 132, but a network interface unit 133 or an external device interface unit (unlike shown in FIG. 2). 135), the video may be received and reproduced.

For example, the display device 100 is implemented separately as an image processing device such as a set-top box for receiving broadcast signals or content according to various network services, and a content reproducing device that reproduces content input from the image processing device. It can be.

In this case, a method of operating a display device according to an embodiment of the present invention to be described below includes not only the display device 100 as described with reference to FIG. 2 , but also an image processing device or display unit 180 such as a separate set-top box ) and a content playback device having an audio output unit 185.

The audio output unit 185 receives the audio-processed signal from the control unit 170 and outputs it as audio.

The power supply 190 supplies corresponding power throughout the display device 100 . In particular, power is supplied to the control unit 170, which can be implemented in the form of a system on chip (SOC), the display unit 180 for displaying images, and the audio output unit 185 for outputting audio. can

Specifically, the power supply unit 190 may include a converter that converts AC power to DC power and a dc/dc converter that converts the level of the DC power.

Next, a remote control device according to an embodiment of the present invention will be described with reference to FIGS. 3 and 4 .

Figure 3 is a block diagram of a remote control device according to an embodiment of the present invention, Figure 4 shows an example of the actual configuration of the remote control device according to an embodiment of the present invention.

First, referring to FIG. 3, the remote control device 200 includes a fingerprint recognition unit 210, a wireless communication unit 220, a user input unit 230, a sensor unit 240, an output unit 250, and a power supply unit 260. ), a storage unit 270, a control unit 280, and a voice acquisition unit 290 may be included.

Referring to FIG. 3 , the wireless communication unit 220 transmits and receives signals with any one of the display devices according to the above-described embodiments of the present invention.

The remote control device 200 includes an RF module 221 capable of transmitting and receiving signals to and from the display device 100 according to RF communication standards, and capable of transmitting and receiving signals to and from the display device 100 according to IR communication standards. An IR module 223 may be provided. In addition, the remote control device 200 may include a Bluetooth module 225 capable of transmitting and receiving signals to and from the display device 100 according to Bluetooth communication standards. In addition, the remote control device 200 includes an NFC module 227 capable of transmitting and receiving signals to and from the display device 100 according to the NFC (Near Field Communication) communication standard, and displays the display according to the WLAN (Wireless LAN) communication standard. A WLAN module 229 capable of transmitting and receiving signals to and from the device 100 may be provided.

In addition, the remote control device 200 transmits a signal containing information about the movement of the remote control device 200 to the display device 100 through the wireless communication unit 220 .

On the other hand, the remote control device 200 can receive the signal transmitted by the display device 100 through the RF module 221, and powers on/off the display device 100 through the IR module 223 as necessary. Commands for off, channel change, volume change, etc. can be transmitted.

The user input unit 230 may include a keypad, buttons, a touch pad, or a touch screen. A user may input a command related to the display device 100 to the remote control device 200 by manipulating the user input unit 230 . When the user input unit 230 includes a hard key button, the user may input a command related to the display device 100 to the remote control device 200 through a push operation of the hard key button. This will be described with reference to FIG. 4 .

Referring to FIG. 4 , the remote control device 200 may include a plurality of buttons. The plurality of buttons include a fingerprint recognition button 212, a power button 231, a home button 232, a live button 233, an external input button 234, a volume control button 235, a voice recognition button 236, A channel change button 237, an OK button 238, and a back button 239 may be included.

The fingerprint recognition button 212 may be a button for recognizing a user's fingerprint. As an example, the fingerprint recognition button 212 may perform a push operation, and thus may receive a push operation and a fingerprint recognition operation. The power button 231 may be a button for turning on/off the power of the display device 100. The home button 232 may be a button for moving to a home screen of the display device 100 . The live button 233 may be a button for displaying a real-time broadcasting program. The external input button 234 may be a button for receiving an external input connected to the display device 100 . The volume control button 235 may be a button for adjusting the volume output from the display device 100 . The voice recognition button 236 may be a button for receiving a user's voice and recognizing the received voice. The channel change button 237 may be a button for receiving a broadcast signal of a specific broadcast channel. The confirmation button 238 may be a button for selecting a specific function, and the back button 239 may be a button for returning to a previous screen.

Figure 3 will be described again.

When the user input unit 230 includes a touch screen, the user may input a command related to the display device 100 to the remote control device 200 by touching a soft key on the touch screen. In addition, the user input unit 230 may include various types of input means that the user can manipulate, such as a scroll key or a jog key, and the present embodiment does not limit the scope of the present invention.

The sensor unit 240 may include a gyro sensor 241 or an acceleration sensor 243 , and the gyro sensor 241 may sense information about movement of the remote control device 200 .

For example, the gyro sensor 241 may sense information about the operation of the remote control device 200 based on x, y, and z axes, and the acceleration sensor 243 may measure the moving speed of the remote control device 200. etc. can be sensed. On the other hand, the remote control device 200 may further include a distance measuring sensor, so that the distance to the display unit 180 of the display device 100 may be sensed.

The output unit 250 may output a video or audio signal corresponding to manipulation of the user input unit 230 or a signal transmitted from the display device 100 . Through the output unit 250, the user can recognize whether the user input unit 230 has been manipulated or whether the display device 100 has been controlled.

For example, the output unit 250 includes an LED module 251 that lights up when the user input unit 230 is manipulated or a signal is transmitted and received with the display device 100 through the wireless communication unit 220, and a vibration module that generates vibration ( 253), a sound output module 255 that outputs sound, or a display module 257 that outputs images.

In addition, the power supply unit 260 supplies power to the remote control device 200, and when the remote control device 200 does not move for a predetermined time, the power supply is stopped to reduce power waste. The power supply unit 260 may resume power supply when a predetermined key provided in the remote control device 200 is manipulated.

The storage unit 270 may store various types of programs and application data necessary for controlling or operating the remote control device 200 . If the remote control device 200 transmits and receives signals wirelessly through the display device 100 and the RF module 221, the remote control device 200 and the display device 100 transmit and receive signals through a predetermined frequency band. .

The control unit 280 of the remote control device 200 stores information about a frequency band that can wirelessly transmit/receive signals with the display device 100 paired with the remote control device 200 in the storage unit 270 and refers thereto. can do.

The control unit 280 controls all matters related to the control of the remote control device 200. The control unit 280 transmits a signal corresponding to a predetermined key manipulation of the user input unit 230 or a signal corresponding to the movement of the remote control device 200 sensed by the sensor unit 240 through the wireless communication unit 220 to the display device ( 100) can be transmitted.

Also, the voice acquisition unit 290 of the remote control device 200 may acquire voice.

The voice acquisition unit 290 may include one or more microphones 291 and may acquire voice through the microphone 291 .

Next, Fig. 5 will be described.

5 shows an example of utilizing a remote control device according to an embodiment of the present invention.

5(a) illustrates that the pointer 205 corresponding to the remote control device 200 is displayed on the display unit 180.

A user may move or rotate the remote control device 200 up and down, left and right. A pointer 205 displayed on the display unit 180 of the display device 100 corresponds to the movement of the remote control device 200 . As shown in the drawing, such a remote control device 200 may be named a space remote controller because a corresponding pointer 205 is moved and displayed according to movement in 3D space.

(b) of FIG. 5 illustrates that when the user moves the remote control device 200 to the left, the pointer 205 displayed on the display unit 180 of the display device 100 also moves to the left correspondingly.

Information on the movement of the remote control device 200 detected through the sensor of the remote control device 200 is transmitted to the display device 100 . The display device 100 may calculate the coordinates of the pointer 205 from information about the movement of the remote control device 200 . The display device 100 may display a pointer 205 to correspond to the calculated coordinates.

(c) of FIG. 5 illustrates a case where the user moves the remote control device 200 away from the display unit 180 while pressing a specific button in the remote control device 200 . Accordingly, a selection area within the display unit 180 corresponding to the pointer 205 may be zoomed in and displayed.

Conversely, when the user moves the remote control device 200 closer to the display unit 180, a selection area within the display unit 180 corresponding to the pointer 205 may be zoomed out and reduced.

Meanwhile, when the remote control device 200 moves away from the display unit 180, the selected area may be zoomed out, and when the remote control device 200 moves closer to the display unit 180, the selected area may be zoomed in.

In addition, when a specific button in the remote control device 200 is pressed, recognition of vertical and horizontal movement may be excluded. That is, when the remote control device 200 moves away from or approaches the display unit 180, up, down, left, and right movements are not recognized, and only forward and backward movements may be recognized. In a state in which a specific button in the remote control device 200 is not pressed, only the pointer 205 moves as the remote control device 200 moves up, down, left, or right.

Meanwhile, the moving speed or moving direction of the pointer 205 may correspond to the moving speed or moving direction of the remote control device 200 .

Meanwhile, a pointer in this specification refers to an object displayed on the display unit 180 corresponding to an operation of the remote control device 200 . Therefore, objects of various shapes other than the arrow shape shown in the drawing are possible as the pointer 205 . For example, it may be a concept including a point, a cursor, a prompt, and a thick outline. In addition, the pointer 205 may be displayed in correspondence with any one point of the horizontal axis and the vertical axis on the display unit 180, as well as displayed in correspondence with a plurality of points such as a line and a surface. do.

6 is an exemplary diagram of a TEE-based content protection technology system in a display device.

The content protection technology may be Digital Rights Management (DRM) or Conditional Access System (CAS).

The REE of FIG. 6 corresponds to the Normal World of FIG. 1 as a Reach Execution Environment, and the TEE corresponds to the Secure World of FIG. 1 as a Trusted Execution Environment.

The content protection technology system can obtain a key by receiving information extracted from an application (App) on REE, and each subsystem running on TEE can perform storage and management. In addition, decoding may be performed by linking the A/V contents of units of applications given by the application on the TEE with the TA of each subsystem through bidirectional communication technology (eg, RPC/SMC).

7 is a flowchart illustrating a method of decoding in a conventional content protection technology system.

The controller 170 can start playing AV content (S11).

That is, the controller 170 can play AV content upon receiving a content play command. Specifically, upon receiving an AV content playback command, the controller 170 may receive the AV content through the broadcast receiving unit 130, the external device interface unit 135, or the wireless communication unit 173.

The control unit 170 may extract samples and decoding information (S13).

The controller 170 can extract samples and decoding information from the received AV content.

Here, a sample may mean one frame. AV content is divided into sample units through a demuxer provided by an app or platform.

Meanwhile, encryption of AV content may be performed in units of samples or subsamples. Here, the subsample may mean that one sample is divided into a plurality of samples. Since AV content is received in an encrypted state for security, decryption information for decrypting it may be extracted in units of samples or subsamples.

Extraction of decryption information may include a process of requesting and obtaining a key for decryption.

The controller 170 may determine whether an encryption unit of the AV content is a subsample (S15).

If the encryption unit of the AV content is a subsample, the control unit 170 may request subsample[i] decoding (S21) and decrypt subsample[i] (S23).

That is, if the encryption unit of the AV content is a subsample, the controller 170 may request decryption as many as the number of subsamples and perform decryption. For example, when one sample is divided into n subsamples, the control unit 170 may perform a decoding request and decoding process on the n subsamples. Accordingly, the control unit 170 can complete decoding processing for one sample.

Meanwhile, if the encryption unit of the AV content is not a subsample, that is, if the encryption unit of the AV content is a sample, the controller 170 may request sample decoding (S17) and perform sample decoding (S19).

Meanwhile, as shown in FIG. 7 , steps S11, S13, S15, S17, S21, S27, S29, and S31 may be performed in REE, and steps S19, S23, and S25 may be performed in TEE.

When decoding is performed, the control unit 170 may store the decoded data in a decoder buffer (S25).

The controller 170 may output AV contents through decoded data stored in the decoder buffer (S27).

The controller 170 may output AV content and obtain whether or not there is a type of stream (S29).

When the stream ends, the controller 170 ends reproduction of the AV content (S31), and if the stream does not end, the control unit 170 can continuously extract samples and decoding information to output the AV content.

In this way, according to the prior art, the display device 180 performs decoding processing in units of single samples. And, the display device 180 performed the decoding process regardless of the playback state (eg, Play/Pause/Seek/Resume). Therefore, according to the prior art, the number of REE/TEE bi-directional communication for decoding consecutive samples while content is reproduced increases in proportion to the number of samples and subsamples. Therefore, especially in the case of high-definition content, the sample decoding processing speed is highly likely to be delayed in an environment where computational performance is low or multiple TAs are processed simultaneously, and the decoding processing speed delay is a playback problem (for example, buffer underrun). ), a state in which data is allocated at a rate lower than the speed at which data is read) may be caused.

Accordingly, the present disclosure intends to reduce the number of REE/TEE bi-directional communication in the content decoding process. Specifically, general encryption/decryption technologies have performance differences depending on the size of data required for one-time decryption. That is, there is a significant difference between the performance of decoding small-sized data several times and the performance of decoding large-sized data at once.

Therefore, the present disclosure aims to improve the performance of the decoding processing speed by minimizing the possibility of delay in the decoding processing speed by reducing the number of REE/TEE bidirectional communication and increasing the data size for one decoding. In addition, the present disclosure intends to determine whether or not adaptively input samples can be collected by monitoring the playback state and the buffer state of the decoder, and perform decoding by combining a plurality of collected samples.

8 is a block diagram illustrating a configuration for performing decoding by collecting a plurality of samples according to an embodiment of the present disclosure.

The controller 170 includes the content receiver 1001 and 1003, the DRM CDM 1005, the player 1007, the sample collector 1009 and 1017, the decryptor 1011 and 1019, the parser 1013, and the audio It may include at least some or all of the decoder 1015, the DRM TA 102, the buffer 1023, and the video decoder 1025. Meanwhile, the above configurations are merely examples for convenience of explanation, and some may be omitted.

The content receiving units 1001 and 1003 may receive content. The content receiving units 1001 and 1003 may be web apps or browsers as shown in FIG. 8 .

The DRM CDM (1005) is a Digital Rights Management Content Decryption Module, and can provide information required for decryption of content conforming to the DRM mechanism. For example, when content is provided from the browser 1003, the DRM CDM 1005 may provide information necessary for decryption of the content provided from the browser to the decryptors 1011 and 1019.

When content is input, the player 1007 may form an audio pipeline and a video pipeline to decode audio and video, respectively.

The sample collection unit 1009 may collect samples related to audio, and the sample collection unit 1017 may collect samples related to video. That is, although samples have not been separately collected in the prior art, the display device 100 according to an embodiment of the present disclosure may further include sample collection units 1009 and 1017 for collecting samples.

The decryptor 1011 may decode audio related samples. In particular, the decryptor 1011 may decode a plurality of samples related to audio collected in the sample collecting unit 1009 .

Specifically, the decryptor 1011 may transmit a plurality of samples to the DRM TA 1021, receive a plurality of decrypted samples from the DRM TA 1021 again, and transmit them to the audio parser 1013. The above-described operation may be an in-place mode. The audio parser 1013 may decompose a plurality of decoded samples and pass them to the audio decoder 1015 . The audio decoder 1015 may convert a signal into a form in which the plurality of samples received from the audio parser 1013 can be output from the audio output unit 185 .

The decryptor 1019 may decode video related samples. In particular, the decryptor 1011 may decode a plurality of video-related samples collected in the sample collection unit 1017.

Specifically, the decryptor 1019 may transmit a plurality of samples to the DRM TA 1021, and the DRM TA 1021 may transfer the decrypted plurality of samples to the buffer 1023 protected by the TEE. That is, a plurality of video related samples may be accumulated and stored in the buffer 1023 so as not to be exposed as decoded REE. The decryptor 1019 may manage the buffer 1023 so that a plurality of samples related to video are stored in the buffer 1023. The above-described operation may be a pass-through mode. The buffer 1023 transfers the stored plurality of samples to the video decoder 1025, and the video decoder 1025 may convert a signal into a form in which the plurality of samples can be output on the display 180.

The DRM TA 1021 may be a Digital Rights Management Trusted Application. The DRM TA 1021 may decrypt a plurality of encrypted samples. In addition, the DRM TA 1021 may transmit the decoded plurality of samples, in particular video related samples, to the video decoder 1025 through the buffer 1023 so that they are not exposed to REE.

Summarizing the description of FIG. 8 , the display apparatus 100 may decrypt a plurality of collected samples together after dozens of samples without decrypting each sample, which is an encryption unit of content.

According to an embodiment, the controller 170 may collect a predetermined number of samples and then decode the collected samples. That is, the number of samples to be decoded together may be fixed.

According to another embodiment, the controller 170 may adjust the number of collected samples according to a playback state, a buffer state, and the like. That is, the number of samples to be decoded together may be flexible according to a playback state, a buffer state, and the like. For example, if the control unit 170 receives a content playback command, extracts samples and decoding information from the content, decodes the sample according to the decoding information, and displays the content through the decoded sample, the display 180 displays the content. The controller 170 may collect samples based on a playback state of content or a state of the buffer 1023 in which decoded samples are stored before being output, and may decode a plurality of collected samples together. Hereinafter, with reference to FIGS. 9 to 11 , a method of decoding a sample in consideration of a playback state, a buffer state, and the like in a display device according to an embodiment of the present disclosure will be described.

9 is a flowchart illustrating a method of operating a display device according to an embodiment of the present disclosure.

The controller 170 can start playing AV content (S101).

That is, the controller 170 can play AV content upon receiving a content play command. Specifically, upon receiving an AV content playback command, the controller 170 may receive the AV content through the broadcast receiving unit 130, the external device interface unit 135, or the wireless communication unit 173.

The control unit 170 may extract samples and decoding information (S103).

The controller 170 can extract samples and decoding information from the received AV content.

The controller 170 may extract a sample representing one frame and decoding information from AV content. Decoding information may be in units of samples or subsamples. Since the detailed description is the same as that described in FIG. 7 , redundant description will be omitted.

The controller 170 may monitor the playback state (S105).

For example, the controller 170 may determine whether the AV content is in a playback state or an event has occurred by monitoring the playback state. Here, the event occurrence state may include a state within a predetermined time after the playback starts, a search state, and a pause state, but since this is only an example for convenience of explanation, it is appropriate that it is not limited thereto. Also, the playback state may refer to a state in which AV content is being reproduced except for an event occurrence state.

The controller 170 may determine whether AV content is being reproduced for a predetermined period of time or longer (S107).

That is, the controller 170 can determine whether AV content is in a playback state. For example, the control unit 170 determines that the content is not in the playback state when the content has been reproduced within a predetermined time, in the search state, and in the pause state, and in other than the above cases, the AV content is displayed. It can be determined that it is in a regeneration state.

When the AV content is not in a playback state, the controller 170 may request sample decoding (S119).

That is, if the AV content is not in a playback state, the controller 170 may stop collecting samples and request decoding of the samples collected so far.

Meanwhile, if the AV content is in a playback state, the controller 170 may collect samples and decoding information (S110).

That is, the controller 170 may continue to collect samples and decoding information when the AV content is in a playback state.

In summary, the controller 170 can collect samples and decoding information when the AV content is in a playback state, and request decoding of a plurality of pre-collected samples when the AV content is not in a playback state.

And, the controller 170 may monitor the buffer status (S113).

The buffer state may refer to a remaining space state of the buffer 1023 described in FIG. 8 .

The controller 170 may determine whether the buffer state is sufficient (S115).

According to an embodiment, the controller 170 may obtain whether the state of the buffer is sufficient based on the current time stamp of the content and the time stamp of the sample collected in the buffer.

Specifically, the controller 170 compares the current PTS (Present Time Stamp) of AV content being output with the time stamp of samples collected in the buffer 1023 to determine whether the buffer state is sufficient. . For example, the control unit 170 controls the control unit 170 if the difference between the current PTS (Present Time Stamp) of the AV content being output and the time stamp of the most recently collected sample in the buffer 1023 is within a preset standard, It can be determined that the buffer state is not sufficient.

When the buffer state is not sufficient, the control unit 170 may request sample decoding (S119).

That is, when a sample recently collected in the buffer 1023 is a sample to be output soon, the control unit 170 may determine that the buffer state is insufficient and request sample decoding.

In summary, the controller 170 may collect samples and decoding information if the buffer state is sufficient, and may request decoding of a plurality of pre-collected samples if the buffer state is not sufficient.

Meanwhile, if the buffer state is sufficient, the control unit 170 may determine whether the number of samples collected in the buffer 1023 is sufficient (S117).

According to an embodiment, the controller 170 may count the number of samples collected in the buffer 1023 and determine whether the number of samples collected in the buffer 1023 is sufficient based on the counted number of samples. For example, the controller 170 determines that the number of samples collected in the buffer 1023 is sufficient when the number of counted samples is equal to or greater than the preset reference number, and if the number of counted samples is less than the preset reference number, the buffer 1023 ) can be judged to be insufficient.

When the number of samples collected in the buffer 1023 is sufficient, the control unit 170 may request sample decoding (S119).

That is, the display apparatus 100 may request sample decoding if enough samples are collected in the buffer 1023 even if the buffer state is sufficient.

In summary, the control unit 170 requests decoding of a plurality of pre-collected samples if there are enough samples collected in the buffer 1023, and collects samples and decoding information if there are not enough samples collected in the buffer 1023. can do.

Meanwhile, the control unit 170 may perform the operation of step S103 again when not enough samples are collected in the buffer 1023.

Steps before sample decoding is requested, that is, steps up to S119 may be performed on REE. Steps after requesting sample decoding may be performed on the TEE.

The control unit 170 may process sample decoding according to the sample decoding request (S120).

That is, the control unit 170 may perform sample decoding through the decryptor 1019 according to the sample decoding request and store the decoded samples in the decoder buffer 1023 (S123).

The controller 170 may output AV content using the decoded samples stored in the decoder buffer 1023 (S125).

The controller 170 may output content to the display 180 through decoded samples stored in the decoder buffer 1023 .

The controller 170 may determine whether the stream has ended while AV content is being output (S127).

A stream may refer to a flow of data that is continuously received to output AV content being played.

When the stream ends, the controller 170 ends playback of the AV content (S129), and when the stream does not end, the operation of step S103 can be performed again.

As described above, according to the present disclosure, the display apparatus 100 may flexibly adjust the number of collected samples based on at least one of a reproduction state or a buffer state of AV content. Accordingly, when an event occurs, collection of samples is minimized and decoding is performed immediately, so there is an advantage in that loading time is minimized when output of AV content is resumed. In addition, since samples are collected or decoding is performed according to the buffer state, problems such as buffering or screen interruption are minimized.

Meanwhile, when collecting a plurality of samples, the display device 100 according to an embodiment of the present disclosure may manage a plurality of samples as one chunk. A chunk is a unit of bundling a plurality of samples, and the plurality of samples may be decoded in units of chunks. That is, a plurality of samples included in one chunk can be decoded together.

Next, referring to FIG. 10, a method of managing a plurality of samples in chunks will be described in detail.

10 is a flowchart illustrating a method of managing a plurality of samples as chunks by a display device according to an embodiment of the present disclosure. 10 may be a flowchart in which step S110 of FIG. 9 is embodied.

When the sample collection starts, the control unit 170 may determine whether the sample is an encrypted sample (S1101).

If the sample is not an encrypted sample, the control unit 170 may add the sample to the last chunk (S1105).

Depending on the content, unencrypted samples may be included to minimize delay due to decoding in a specific period (eg, a predetermined number of frames at the beginning of playback). Accordingly, the control unit 170 may add the sample to the last chunk if the sample is not encrypted.

In the case of an encrypted sample, the control unit 170 may determine whether key rotation has occurred (S1103).

Key rotation means that an encryption key is changed from a certain point in time (for example, a program change in Live Streaming or real-time broadcasting) for a higher level of security.

Accordingly, when key rotation occurs, the controller 170 manages a chunk list so that chunks are distinguished (S1107), and can add a sample to the last chunk (S1105).

The control unit 170 may manage a chunk list by including a plurality of samples to be decoded together in one chunk.

For example, the controller 170 may determine a chunk including a sample according to whether a key rotation in which an encryption key is changed occurs. The controller 170 may manage the chunk list so that samples having different encryption keys are included in different chunks according to key rotation. That is, when key rotation occurs, the control unit 170 can create a new chunk and manage the chunk list.

For example, the control unit 170 includes samples having an encryption key as a first key in a first chunk, and when key rotation occurs while collecting samples, samples having an encryption key as a second key are included in a second chunk as a new chunk. Samples can be collected in an inclusive manner. The chunk list may include at least a first chunk and a second chunk.

A chunk that is a combination of an encrypted sample and multiple unencrypted samples can be processed in one decryption operation, but if the decryption key is changed by applying key rotation, the sample cannot be included in the existing chunk, so it must be managed as a new chunk. . Accordingly, the control unit 170 may collect multiple samples while managing the chunk list by distinguishing cases such as key rotation in the logic for collecting multiple samples.

And, as described above, when a plurality of samples are managed in chunks, the controller 170 may request decoding in units of chunks. Next, referring to FIG. 11, a method of decoding a plurality of samples in chunk units will be described.

11 is a flowchart illustrating a method of decoding a plurality of samples in chunk units by a display device according to an embodiment of the present disclosure. 11 may be a flowchart in which step S120 is embodied.

When sample decoding is started, the control unit 170 may sequentially request decoding of a plurality of samples included in each chunk from the first chunk to the n-th chunk (where n is the number of chunks) (S1201), and perform decoding (S1201). S1203).

That is, the control unit 170 may decode all collected samples while looping as many as the number of chunks included in the chunk list.

The present disclosure described above can be implemented as computer readable codes in a medium on which a program is recorded. The computer-readable medium includes all types of recording devices in which data that can be read by a computer system is stored. Examples of computer-readable media include Hard Disk Drive (HDD), Solid State Disk (SSD), Silicon Disk Drive (SDD), ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc. there is Accordingly, the above detailed description should not be construed as limiting in all respects and should be considered as illustrative. The scope of this specification should be determined by reasonable interpretation of the appended claims, and all changes within the equivalent scope of this specification are included in the scope of this specification.

The above description is merely an example of the technical idea of the present invention, and various modifications and variations can be made to those skilled in the art without departing from the essential characteristics of the present invention.

Therefore, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention, but to explain, and the scope of the technical idea of the present invention is not limited by these embodiments.

The protection scope of the present invention should be construed according to the claims below, and all technical ideas within the equivalent range should be construed as being included in the scope of the present invention.

Claims (10)

When a content reproduction command is received, a controller extracts samples and decoding information from the content and decodes the samples according to the decoding information; and
And a display for outputting content through the decoded sample,
The control unit
Collecting the sample based on the playback state of the content or the state of the buffer in which the decoded sample is stored before output, and decoding the collected plurality of samples together
display device.
The method of claim 1,
The control unit
Collecting the sample and decoding information when the content is in a playback state;
Requesting decoding of a plurality of pre-collected samples if the content is not in a playback state
display device.
The method of claim 1,
The control unit
Determining that the content is not in a playback state in the case of a state within a predetermined time after the start of playback of the content, a search state, and a pause state
display device.
The method of claim 1,
The control unit
If the state of the buffer is sufficient, collect the sample and decoding information;
If the state of the buffer is not sufficient, requesting decoding of a plurality of pre-collected samples
display device.
The method of claim 4,
The control unit
Obtaining whether the state of the buffer is sufficient based on the current time stamp of the content and the time stamp of the sample collected in the buffer
display device.
The method of claim 1,
The control unit
If there are enough samples collected in the buffer, request decoding of a plurality of pre-collected samples;
Collecting the samples and decoding information if there are not enough samples collected in the buffer
display device.
The method of claim 6,
The control unit
Counting the number of samples collected in the buffer and obtaining whether or not there are enough samples collected in the buffer based on the counted number
display device.
The method of claim 1,
The control unit
Manages a chunk list by including a plurality of samples that are decoded together in one chunk
display device.
The method of claim 8,
The control unit
Determines which chunks a sample is included in depending on whether key rotation occurs, in which the encryption key changes.
display device.
The method of claim 9,
The control unit
Managing the chunk list so that samples with different encryption keys are included in different chunks when the key rotation occurs
display device.

Images