KR102545468B1 - IPTV set-top box capable of providing cloud game service and method of providing cloud game service in IPTV set-top box - Google Patents

Abstract

An IPTV set-top box capable of providing cloud game service and a method for providing cloud game service in the IPTV set-top box are disclosed. In the IPTV set-top box capable of providing a cloud game service according to an embodiment of the present invention, the input interface unit for receiving an input signal; a network communication unit communicating with an external server in response to the input signal; a video processing unit processing video data among contents received from the external server; an audio processing unit that processes audio data among the contents; and generating a control signal that controls at least one of the processing methods of the input signal of the input interface unit, the video data of the video processor, and the audio data of the audio processor to be different in the broadcast mode in the game mode. It includes; a game mode controller that does.

Description

IPTV set-top box capable of providing cloud game service and method of providing cloud game service in IPTV set-top box {IPTV set-top box capable of providing cloud game service and method of providing cloud game service in IPTV set-top box}

The following disclosure relates to an IPTV set-top box capable of providing a cloud game service by applying a low-latency media processing technology and a method for providing a cloud game service in the IPTV set-top box.

While various streaming services are popular, cloud games in which game usage is immediately streamed from a user terminal while the game is stored in a server as a kind of cloud service are also rapidly building a market. Furthermore, there is an increasing demand to play cloud games through a large TV screen.

However, existing IPTV set-top boxes (Internet Protocol Television Set-top Boxes) cannot perform high-performance processing, and thus have limitations in providing cloud games requiring fast screen switching. Accordingly, when cloud games are to be played through a large screen of an IPTV, there is a problem in that a separate expensive high-performance game machine must be provided.

Accordingly, we intend to provide an IPTV set-top box capable of providing cloud game service and a method for providing cloud game service in the IPTV set-top box.

In the IPTV set-top box capable of providing a cloud game service according to an embodiment of the present invention for solving the above technical problem, an input interface unit for receiving an input signal; a network communication unit communicating with an external server in response to the input signal; a video processing unit processing video data among contents received from the external server; an audio processing unit that processes audio data among the contents; and generating a control signal that controls at least one of the processing methods of the input signal of the input interface unit, the video data of the video processor, and the audio data of the audio processor to be different in the broadcast mode in the game mode. It includes; a game mode controller that does.

In a game mode, the game application may further include a game application that executes the contents by communicating with a game server among the external servers through the network communication unit in response to the input signal.

The input interface unit may process the input signal in a first cycle in a broadcasting mode and process the input signal in real time in a game mode in response to the control signal.

The video processing unit processes the first frame of the video data after receiving the second frame in the broadcasting mode and processes the first frame of the video data before receiving the second frame in the game mode in response to the control signal. can do.

The audio processor may process the audio data at a first frequency in a broadcast mode and process the audio data at a second frequency shorter than the first frequency in a game mode in response to the control signal.

The IPTV set-top box may operate with an Android operating system.

The input interface unit processes the input signal according to a VSync (Vertical Synchronization) signal in a broadcasting mode and, in response to the control signal, transmits the input signal to the IPTV set-top box in real time regardless of the VSync signal in a game mode. It may include an Android framework that transmits to a game application.

The video processing unit may include: a video decoder for decoding the video data; a video renderer that renders the decoded video data; and a surface flinger that synthesizes and outputs rendered video data.

The video processing unit may not perform a buffering operation when there is no B frame in the video data received from the game application of the IPTV set-top box.

The audio processing unit, among a first mixer mode operating at a first frequency and a second mixer mode operating at a second frequency faster than the first frequency, the audio received from the game application of the IPTV set-top box For data, an AudioFlinger operating in the second mixer mode may be included.

The audio processor may set an audio buffering time and a buffering check time in the game mode to be shorter than the audio buffering time and buffering check time in the broadcast mode, respectively.

In the method for providing a cloud game service in an IPTV set-top box according to an embodiment of the present invention for achieving the above technical problem, receiving an input signal; communicating with an external server in response to the input signal; processing video data among contents received from the external server; processing audio data among the contents; and controlling, in a game mode, different processing methods of at least one of the input signal of the input interface unit, the video data of the video processor, and the audio data of the audio processor in a broadcast mode. do.

The receiving of the input signal may include processing the input signal in a first cycle in a broadcasting mode; and processing the input signal in real time in a game mode.

The processing of the video data may include processing a first frame of the video data after receiving a second frame in a broadcast mode; and processing the first frame of the video data before receiving the second frame in the game mode.

The processing of the audio data may include processing the audio data at a first frequency in a broadcasting mode; and processing the audio data at a second frequency shorter than the first frequency in a game mode in response to the control signal. can include

According to an IPTV set-top box capable of providing a cloud game service and a method for providing a cloud game service in the IPTV set-top box according to an embodiment of the present invention, the IPTV set-top box in which many buffering operations are performed with emphasis on continuous provision and stability of broadcasting By providing cloud games where speed is important, you can enjoy games on a large TV without having to have a separate cloud game terminal.

1 is a diagram showing an IPTV set-top box according to an embodiment of the present invention.
2 is a diagram showing a cloud game service providing method in an IPTV set-top box according to an embodiment of the present invention.
3 is a diagram showing an IPTV set-top box according to an embodiment of the present invention.
4 and 5 are views for explaining an effect of solving an input processing delay according to an embodiment of the present invention.
6 is a diagram showing a video processing unit according to an embodiment of the present invention in more detail.
7 is a diagram for explaining the operation of a video processing unit according to an embodiment of the present invention.
8 is a diagram for explaining an effect of solving a video processing delay according to an embodiment of the present invention.
9 is a diagram showing an audio processing unit according to an embodiment of the present invention in more detail.
10 is a diagram for explaining an effect of solving an audio processing delay according to an embodiment of the present invention.

Hereinafter, specific embodiments of the present invention will be described with reference to the drawings. The detailed descriptions that follow are provided to provide a comprehensive understanding of the methods, devices and/or systems described herein. However, this is only an example and the present invention is not limited thereto.

In describing the embodiments of the present invention, if it is determined that the detailed description of the known technology related to the present invention may unnecessarily obscure the subject matter of the present invention, the detailed description will be omitted. In addition, terms to be described later are terms defined in consideration of functions in the present invention, which may vary according to the intention or custom of a user or operator. Therefore, the definition should be made based on the contents throughout this specification. Terminology used in the detailed description is only for describing the embodiments of the present invention and should in no way be limiting. Unless expressly used otherwise, singular forms of expression include plural forms. In this description, expressions such as "comprising" or "comprising" are intended to indicate any characteristic, number, step, operation, element, portion or combination thereof, one or more other than those described. It should not be construed to exclude the existence or possibility of any other feature, number, step, operation, element, part or combination thereof.

Directional terms such as top, bottom, one side, the other, etc. are used in reference to the orientation of the disclosed figures. Since components of embodiments of the present invention may be positioned in a variety of orientations, directional terms are used for purposes of illustration and not limitation.

Terms such as first and second may be used to describe various components, but the components should not be limited by the terms. The terms may be used for the purpose of distinguishing one component from another. For example, a first element may be termed a second element, and similarly, a second element may be termed a first element, without departing from the scope of the present invention.

1 is a diagram showing an IPTV set-top box according to an embodiment of the present invention, and FIG. 2 is a diagram showing a cloud game service providing method in an IPTV set-top box according to an embodiment of the present invention.

Referring to FIGS. 1 and 2 , the IPTV set-top box 100 according to the embodiment of the present invention can operate as the cloud game service providing method 200 in the IPTV set-top box according to the embodiment of the present invention. In addition, the cloud game service providing method 200 in the IPTV set-top box according to an embodiment of the present invention can be executed in the IPTV set-top box 100 according to the embodiment of the present invention. However, it is not limited thereto. The IPTV set-top box 100 according to the embodiment of the present invention may operate in a method different from the method 200 for providing cloud game service in the IPTV set-top box according to the embodiment of the present invention. In addition, the cloud game service providing method 200 in the IPTV set-top box according to an embodiment of the present invention may be executed in a system different from the IPTV set-top box 100 according to the embodiment of the present invention. However, in the following, for convenience of explanation, the IPTV set-top box 100 according to the embodiment of the present invention operates as the cloud game service providing method 200 in the IPTV set-top box according to the embodiment of the present invention, The method 200 for providing a cloud game service in an IPTV set-top box according to an embodiment of will be described only for an example in which the method 200 is executed in the IPTV set-top box 100 according to an embodiment of the present invention.

The IPTV set-top box 100 according to an embodiment of the present invention includes an input interface unit 110, a network communication unit 120, a video processing unit 130, an audio processing unit 140, and a game mode control unit 150, In the IPTV set-top box 100 where a lot of buffering operations are performed by placing importance on the continuous provision and stability of , it is possible to provide a cloud game where response speed is emphasized.

The input interface unit 110 receives the input signal KIN (S210). For example, the input signal KIN may be a key input transmitted from a remote controller or a game controller. Alternatively, it may be a touch input transmitted from a smartphone, tablet PC, or the like. The input signal KIN may be received by the input interface unit 110 through wireless communication such as Bluetooth or a USB cable.

The network communication unit 120 communicates with an external server in response to the input signal KIN (S220). The external server may be a server that provides content such as broadcasting and games. The network communication unit 120 according to an embodiment of the present invention may communicate with at least two or more heterogeneous servers. The content CNT transmitted from an external server to the network communication unit 120 in response to the input signal KIN may be streaming data.

In order to efficiently use network traffic, since it is being transmitted in multicast, the IPTV set-top box 100 must perform multiple buffering in broadcasting mode, taking into account the case where the order of packets is changed and arrives at the network communication unit 120. and can operate at a slow response speed accordingly. In the game mode, as described below, such a delay can be reduced.

The video processing unit 130 processes video data (VDTA) among the contents (CNT) received through the network communication unit 120 (S130), and the audio processing unit 140 processes the content (CNT) received through the network communication unit 120. ) of which audio data (ADTA) is processed (S140). The video data VDTA and audio data ADTA processed by the video processor 130 and the audio processor 140 may be transmitted to an external TV as output video data VDTAo and output audio data ADTAo. However, it is not limited thereto, and the output video data VDTAo and the output audio data ADTAo may be transferred to another external display device (eg, a monitor).

The game mode control unit 150 controls the input signal (KIN) of the input interface unit 110, video data (VDTA) from the video processor 130, and audio data (ADTA) from the audio processor 140 in a broadcasting mode. A control signal (XCTL) for controlling to change at least one of the processing methods is generated (S250). The control signal XCTL may be applied to the input interface unit 110 , the video processing unit 130 and the audio processing unit 140 . The game mode control unit 150 may be included in the central processing unit module (not shown) of the IPTV set-top box 100 or may be provided as a separate module from it.

In this case, the game mode may indicate a mode in which a cloud game is executed in the IPTV set-top box 100 . However, it is not limited thereto, and may be applicable even when various games, such as cloud games, in which a response speed is required to be faster than that of the IPTV set-top box 100 in a broadcasting mode are executed.

The game mode control unit 150 directly determines whether the game mode is in game mode based on the input signal (KIN), whether or not the communication session between the network communication unit 120 and the game server is connected, and generates a control signal (XCTL), or IPTV set-top box (100 In another configuration of ), the control signal XCTL may be generated by receiving information determined as a game mode.

A specific control operation in the game mode will be described later.

3 is a diagram showing an IPTV set-top box according to an embodiment of the present invention.

Referring to FIG. 3, the IPTV set-top box 100 of FIG. 3 responds to an input signal (KIN) in a game mode, through a network communication unit 120, and communicates with a game server among external servers to execute a game. It may further include an application 160 . As shown in FIG. 3, when the game application 160 is included in the IPTV set-top box 100, the video data VDTA and audio data ADTA are transmitted to the video processor 130 and It can be delivered to the audio processing unit 140. Hereinafter, for convenience of description, only the case of including the game application 160 like the IPTV set-top box 100 of FIG. 3 will be described.

The IPTV set-top box 100 according to an embodiment of the present invention can operate with Linux OS (Operating System), Android OS, and AOSP (Android Open Source Project). However, in the following, for convenience of description, only an example operating in the Android OS will be described.

When the IPTV set-top box 100 according to an embodiment of the present invention operates as an Android OS, the game application 160 may be installed in the IPTV set-top box 100 through the Play Store or the like. In addition, when the IPTV set-top box 100 according to an embodiment of the present invention operates as an Android OS, the input interface unit 110 may include an Android framework related to input processing.

4 and 5 are views for explaining an effect of solving an input processing delay according to an embodiment of the present invention.

First, referring to FIGS. 3 and 4 , the input interface unit 110 according to the embodiment of the present invention may process the input signal KIN in real time in the game mode in response to the control signal XCTL. That is, when the B button key input (B) is received as the input signal (KIN), it can be directly transmitted to the game application 160 regardless of a vertical synchronization (VSync) signal. In this case, the game application 160 may stop the previously executed A operation corresponding to the A button key input (A) and immediately perform the B operation corresponding to the B button key input (B).

Similarly, after B button key input (B) is input, C button key input (C), A button key input (A), and B button key input (B) are sequentially received as input signals (KIN), VSync and Regardless, it is transmitted to the game application 160, and the game application 160 may perform an operation corresponding to the input signal KIN in real time.

On the other hand, in the broadcasting mode, the input interface unit 110 according to the embodiment of the present invention may process the input signal in a first cycle (eg, a VSync cycle).

3 and 5, the input interface unit 110 according to the embodiment of the present invention synchronizes and processes the input signal KIN input at the same time and sequence as in FIG. can For example, B button key input (B) received before VSync 1 is processed in synchronization with VSync 1. Similarly, C button keystrokes (C) received between VSync 1 and VSync 2 are processed in synchronization with VSync 2, A button keystrokes (A) received in VSync 2 are processed in sync with VSync 3, and VSync 3 And the B button key input (B) received between VSync 4 is processed in synchronization with VSync 4.

When the game application 160 is executed with the processing method of the input signal (KIN) in such a broadcasting mode, the game application 160 executing operation A before VSync 1 synchronizes with VSync 1 and transmits the B button key input ( B) has no choice but to run on VSync 2. Accordingly, a delay occurs for the B button key input (B). In addition, the game application 160 fails to execute the C button key input (C) transmitted by the input interface unit 110 from VSync 2 due to the B operation, and then the A button key input (A) is transmitted from VSync 3 to A As operation A for button key input (A) is performed, operation C for button key input (C) may not be performed.

The input interface unit 110 according to an embodiment of the present invention operates in real time as shown in FIG. 4 in a game mode, thereby solving problems such as delay.

6 is a diagram showing a video processing unit according to an embodiment of the present invention in more detail, and FIG. 7 is a diagram for explaining the operation of the video processing unit according to an embodiment of the present invention.

Referring to FIGS. 6 and 7 , the video processing unit 130 according to an embodiment of the present invention processes the first frame of video data VDTA after receiving the second frame in the broadcasting mode, and generates a control signal XCTL In response to, in the game mode, the first frame of the video data (VDTA) may be processed before receiving the second frame. Accordingly, in the game mode, the game application 160 can implement a response speed capable of performing a cloud game.

When the IPTV set-top box 100 according to the embodiment of the present invention operates as an Android OS, the video processing unit 130 includes a video decoder 132, a video renderer 134, and a surface flinger. Flinger, 136). The video decoder 132 decodes video data (VDTA) compressed with a codec (Codec), etc., the video renderer 134 performs rendering on the decoded video data (VDTA), and the surface flinger 136 The rendered video data is synthesized and output as output video data (VDTAo).

At this time, in decoding the compressed video data (VDTA), in the case of a B frame among I frames (Intra Frame), B frames (Bi-predictive Interframe) and P frames (Predictive Frame), frames received after the corresponding frame Since it must be referenced, an essential buffering operation is required. On the other hand, in the case of a cloud game executed in real time, since it does not include a B frame, buffering may not be performed.

At this time, the video processing unit 130 may check whether the B frame is included in the video data VDTA through a parameter of the video data VDTA. For example, for video data (VDTA) compressed according to the H.264 standard, the video processing unit 130, when the “num_reorder_frames” value of the video usability information (VUI) is “0”, the video data ( It can be determined that the B frame is not included in VDTA). The video processing unit 130 transmits this to the game mode controller 150 of FIG. 1 and the like, and the game mode controller 150 may generate a control signal XCTL in response thereto.

8 is a diagram for explaining an effect of solving a video processing delay according to an embodiment of the present invention.

3, 6 and 8, by operating the input interface unit 110 and the video processing unit 130 as described above in the game mode, the IPTV set-top box 100 according to the embodiment of the present invention is shown in FIG. As shown in Table 1 shown in , end-to-end (E2E) delay criteria of video processing for execution of cloud game execution may be satisfied.

In Table 1, the input delay is the time required for the input signal (KIN) from the game controller to be received by the input interface unit 110 and transmitted to the game application 160, and the game server/network delay is the input of the game application 160. This is the time required for the video data VDTA of the content CNT for the request corresponding to the signal KIN to be transferred from the game server to the game application 160 . In addition, the decoder delay is the time taken until the video data (VDTA) is transferred from the game application 160 to the video decoder 132 and the decoding operation by the video decoder 132 is completed, and the renderer delay is the decoded data is the time required until is output from the IPTV set-top box 100 as output video data (VDTAo). At this time, the video data (VDTA') decoded by the video decoder 132 may be transmitted to the game application 160 and then applied to the video renderer 134 again. In this case, the renderer delay is the time including the corresponding time. can The TV delay is the time taken until the output video data (VDTAO) is displayed on the TV.

9 is a diagram showing an audio processing unit according to an embodiment of the present invention in more detail.

Referring to FIG. 9 , the audio processing unit 140 may process audio data ADTA at a first frequency in broadcasting mode, but process audio data ADTA at a second frequency shorter than the first frequency in game mode. there is. For example, when the IPTV set-top box 100 according to an embodiment of the present invention operates with an Android OS, the audio processing unit 140 uses an audio API (Application Programming Interface, 142), a native framework (144) And it may include an audio flinger (Audio Flinger, 146).

The audio API 142, the native framework 144, and the audio flinger 146 correspond to the video decoder 132, the video renderer 134, and the surface flinger 136 of FIG. 6, respectively. That is, the audio API 142 decodes the audio data ADTA, the native framework 144 renders the audio data ADTA, and the audio flinger 146 synthesizes the audio data ADTA.

At this time, the audio flinger 146 operates in a first mixer mode, that is, a normal mixer mode, operating at a first frequency in the broadcasting mode, and in a second mixer mode, that is, Fast Mixer, operating at a second frequency in the game mode. Among the modes, the game mode may operate in the second mixer mode.

In addition, the audio processing unit 140 processes the audio data accumulated for the buffering time at once and checks the audio data accumulated in the corresponding buffer at every buffering check time (cycle). and the buffering check time may be set shorter than the audio buffering time and the buffering check time in the broadcasting mode, respectively. For example, the audio processing unit 140 may operate by setting an audio buffering time of 60 ms in broadcast mode to 25 ms in game mode. For example, a buffering check time of 10 ms in broadcast mode may be set to 3 ms in game mode.

10 is a diagram for explaining an effect of solving an audio processing delay according to an embodiment of the present invention.

3, 9 and 10, by operating the input interface unit 110 and the audio processing unit 140 as described above in the game mode, the IPTV set-top box 100 according to the embodiment of the present invention is shown in FIG. As shown in Table 2 shown in , end-to-end (E2E) delay criteria for audio processing for execution of cloud game execution may be satisfied.

In Table 2, the input delay is the time required for the input signal (KIN) from the game controller to be received by the input interface unit 110 and transmitted to the game application 160, and the game server/network delay is the input of the game application 160. This is the time required for the audio data ADTA of the content CNT for the request corresponding to the signal KIN to be transferred from the game server to the game application 160 . Audio/TV delay is when the audio data (ADTA) is transmitted from the game application 160 to the audio processing unit 140, processed, outputted as output audio data (ADTAo) from the IPTV set-top box 100, and then played on the TV. is the time it takes to

As described above, according to the IPTV set-top box 100 and the method 200 for providing a cloud game service in the IPTV set-top box according to an embodiment of the present invention, data or events are not processed in real time in broadcasting mode, and resource There is an allocation waiting time, and while a lot of buffering can be performed for input/output processing to provide stable IPTV service, high-performance cloud games can be executed by minimizing E2E (End-to-End) delay in game mode.

At this time, the game application 160 according to an embodiment of the present invention executes a cloud game stored in a domestic data center, so that the response speed in the IPTV set-top box 100 can be further improved.

Although representative embodiments of the present invention have been described in detail above, those skilled in the art will understand that various modifications are possible to the above-described embodiments without departing from the scope of the present invention. . Therefore, the scope of the present invention should not be limited to the described embodiments and should not be defined, and should be defined by not only the claims to be described later, but also those equivalent to these claims.

100: IPTV set-top box
110: input interface unit
120: network communication unit
130: video processing unit
140: audio processing unit
150: game mode control unit
KIN: input signal
VDTA: video data
ODAT: audio data
XCTL: control signal
200: Method for providing cloud game service in IPTV set-top box

Claims (15)

In an IPTV set-top box capable of providing cloud game service,
an input interface unit that receives an input signal;
a network communication unit communicating with an external server in response to the input signal;
a video processing unit processing video data among contents received from the external server;
an audio processing unit that processes audio data among the contents; and
A game that generates a control signal that controls at least one of the processing methods of the input signal of the input interface unit, the video data of the video processor, and the audio data of the audio processor to be different in the game mode. Including; mode control unit;
The video processing unit,
In the broadcast mode, processing a first frame of the video data after receiving the second frame of the video data;
in the game mode, processing a first frame of video data before receiving a second frame of video data;
IPTV set-top box. According to claim 1,
In the game mode, in response to the input signal, a game application that executes the content by communicating with a game server among the external servers through the network communication unit;
IPTV set-top box. According to claim 1,
The input interface unit,
processing the input signal in a first cycle in the broadcasting mode;
In response to the control signal, real-time processing of the input signal in the game mode,
IPTV set-top box. delete According to claim 1,
The audio processing unit,
processing the audio data at a first frequency in the broadcasting mode;
processing the audio data at a second frequency shorter than the first frequency in the game mode;
IPTV set-top box. According to claim 1,
Running on the Android operating system,
IPTV set-top box. According to claim 1,
The input interface unit,
Processing the input signal according to a VSync (Vertical Synchronization) signal in the broadcasting mode;
Transmitting the input signal to the game application of the IPTV set-top box in real time regardless of the VSync signal in the game mode,
IPTV set-top box. According to claim 6,
The video processing unit,
a video decoder to decode the video data;
a video renderer that renders the decoded video data; and
A surface flinger that synthesizes and outputs rendered video data;
IPTV set-top box. According to claim 8,
The video processing unit,
When there is no B frame in the video data received from the game application of the IPTV set-top box, buffering operation is not performed.
IPTV set-top box. According to claim 6,
The audio processing unit,
Among the first mixer mode operating at a first frequency and the second mixer mode operating at a second frequency faster than the first frequency, for the audio data received from the game application of the IPTV set-top box, the Including an AudioFlinger operating in the second mixer mode,
IPTV set-top box. According to claim 6,
The audio processing unit,
The audio buffering time and buffering check time in game mode are set shorter than the audio buffering time and buffering check time in broadcast mode, respectively.
IPTV set-top box. In the method of providing cloud game service in IPTV set-top box,
receiving an input signal;
communicating with an external server in response to the input signal;
processing video data among contents received from the external server;
processing audio data among the contents; and
Controlling at least one of processing methods of the input signal, the video data, and the audio data in a broadcast mode to be different in a game mode;
Processing the video data,
processing the first frame of the video data after receiving the second frame of the video data in a broadcast mode; and
processing the first frame of video data before receiving the second frame of video data in a game mode;
How to provide cloud game service in IPTV set-top box. According to claim 12,
Receiving the input signal,
processing the input signal in a first cycle in the broadcasting mode; and
Including, processing the input signal in real time in the game mode;
How to provide cloud game service in IPTV set-top box. delete According to claim 12,
Processing the audio data,
processing the audio data at a first frequency in the broadcasting mode; and
processing the audio data at a second frequency shorter than the first frequency in the game mode;
How to provide cloud game service in IPTV set-top box.

Images