KR20180064597A - Method and apparatus for random access of multiple contents streaming service - Google Patents

Abstract

The present disclosure discloses a contents random access method and apparatus. The contents random access method based on scheduling according to an embodiment of the present disclosure includes the steps of: determining a branch point to an inter frame based on scheduling information; performing preprocessing on an intra frame associated with the inter frame corresponding to the determined branch point; and performing decoding on the inter frame based on the preprocessed intra frame information. Accordingly, the present invention can support the random access in a streaming service of contents.

Description

METHOD AND APPARATUS FOR RANDOM ACCESS OF MULTIPLE CONTENTS STREAMING SERVICE FIELD OF THE INVENTION [0001]

The present disclosure is directed to multimedia content streaming, and more particularly, to a random access method and apparatus for streaming service of multimedia content.

As information and communication services have evolved, multimedia contents have evolved in various ways, and various multimedia standards and formats have been used to support them. Tools for editing such various multimedia contents are also evolving. For example, the development of a scheduler for reproducing various multimedia contents in combination is also being accelerated.

An existing multimedia content editing tool supports reproduction of a complete program of a combination of various multimedia contents. In this case, when a completed program is reproduced at the time of reproduction, it is possible to reproduce one multimedia content There is no problem at the same time.

On the other hand, in the case of the scheduler, it is required to branch to another multimedia content at a certain time during real-time playback (or streaming) of a plurality of multimedia contents to perform real-time playback.

However, in the schedulers so far, there has been a problem that actual random access can not be supported because arbitrary playback start points of certain multimedia contents are not correctly reproduced or can not be reproduced when they are inter frames. Therefore, it is necessary to provide a method of supporting true random access in a content streaming service.

SUMMARY OF THE INVENTION The present invention provides a method and apparatus for supporting random access in a content streaming service.

It is another object of the present disclosure to provide a method and apparatus for preprocessing a specific frame for random access in a content streaming service.

The technical objects to be achieved by the present disclosure are not limited to the above-mentioned technical subjects, and other technical subjects which are not mentioned are to be clearly understood from the following description to those skilled in the art It will be possible.

According to an aspect of the disclosure, a scheduling based content random access method is disclosed. The method includes determining a branch point to an inter-frame based on scheduling information; Performing preliminary processing on an intra frame associated with the interframe corresponding to the determined branch point; And decoding the interframe based on the preprocessed intra frame information.

The features briefly summarized above for this disclosure are only exemplary aspects of the detailed description of the disclosure which follow, and are not intended to limit the scope of the disclosure.

According to the present disclosure, a method and apparatus for supporting random access in a streaming service of content can be provided.

According to the present disclosure, a method and apparatus for preprocessing a specific frame to support random access in a streaming service of content can be provided.

According to the present disclosure, a method and apparatus for relieving the limitation of random access in a streaming service of contents can be provided.

According to the present disclosure, a method and apparatus for supporting random access even when an arbitrary point in time of each program in the streaming service of contents is an inter frame can be provided.

The effects obtainable from the present disclosure are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art from the description below will be.

1 is a diagram for explaining multimedia content random access according to the present disclosure;
2 is a flow diagram illustrating a random access aiding operation in accordance with the present disclosure;
3 is a block diagram illustrating a random access aiding device according to the present disclosure;
4 is a flow chart for explaining a random access method according to the present disclosure.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings, which will be easily understood by those skilled in the art. However, the present disclosure may be embodied in many different forms and is not limited to the embodiments described herein.

In the following description of the embodiments of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present disclosure rather unclear. Parts not related to the description of the present disclosure in the drawings are omitted, and like parts are denoted by similar reference numerals.

In the present disclosure, when an element is referred to as being "connected", "coupled", or "connected" to another element, it is understood that not only a direct connection relationship but also an indirect connection relationship May also be included. Also, when an element is referred to as " comprising "or" having "another element, it is meant to include not only excluding another element but also another element .

In the present disclosure, the terms first, second, etc. are used only for the purpose of distinguishing one element from another, and do not limit the order or importance of elements, etc. unless specifically stated otherwise. Thus, within the scope of this disclosure, a first component in one embodiment may be referred to as a second component in another embodiment, and similarly a second component in one embodiment may be referred to as a first component .

In the present disclosure, the components that are distinguished from each other are intended to clearly illustrate each feature and do not necessarily mean that components are separate. That is, a plurality of components may be integrated into one hardware or software unit, or a single component may be distributed into a plurality of hardware or software units. Thus, unless otherwise noted, such integrated or distributed embodiments are also included within the scope of this disclosure.

In the present disclosure, the components described in the various embodiments are not necessarily essential components, and some may be optional components. Thus, embodiments consisting of a subset of the components described in one embodiment are also included within the scope of the present disclosure. Also, embodiments that include other elements in addition to the elements described in the various embodiments are also included in the scope of the present disclosure.

The definitions of the terms used in the present disclosure are as follows.

- Multimedia content: Information including text, video, video, sound, voice, etc. (E.g., generated, stored, reproduced, delivered, distributed) through an information processing device (e.g., a TV, a computer, a mobile communication device, etc.).

- Scheduler: A tool that plays any combination of one or more multimedia content. May be composed of software, hardware, or firmware, or may be composed of a combination of two or more of software, hardware, or firmware.

- Program: Each multimedia content to be combined by the scheduler. For example, the scheduler may branch from a first program to a second program at a particular point in time.

- Random access: The process of deciphering or reproducing from an arbitrary point in the middle rather than the initial starting point of the multimedia contents.

- Intra frame: A frame that is encoded or decoded using information of the frame itself without referring to another frame.

- Inter frame: A frame that is encoded or decoded with reference to another frame. The first interframe may be encoded or decoded with reference to the second interframe, but at least one interframe is encoded or decoded with reference to the intra frame.

The scope of the present disclosure in the definition of the above terms is not limited by the expression scheduler. For example, in this disclosure, the scheduler may include any software, hardware, firmware, etc. that supports operations to decode or play back from any point in time of one or more multimedia content.

Further, the scope of the present disclosure is not limited to a random access scheme in a streaming service of a plurality of multimedia contents, but includes a random access scheme for one multimedia contents.

The present disclosure can also be applied to a method of arbitrarily combining or editing various multimedia contents due to the evolution of multimedia standards and formats such as HD (High Definition), UHD (Ultra HD), panorama, multi-view video and 3D video. In this case, random access to certain multimedia contents is required. However, there has been a problem in that when a frame corresponding to a certain time point of the multimedia contents is not an intra frame, random access is not supported. This is because an interframe can be reproduced only when referring to an intra frame. That is, in the case of an interframe of a frame corresponding to an arbitrary time point of the multimedia contents, the interframe can not be decoded unless an intra frame referred to by the interframe exists before the interframe.

In the case of local reproduction, since an intra frame referred to by an inter frame exists locally, the corresponding intra frame can be referred to, and therefore random access to any inter frame is possible.

However, in the multimedia contents scheduled by the scheduler, there is no information about an intra frame referenced by an inter frame, so that random access to the inter frame is impossible.

For example, in the case of branching to streaming for any point in the middle of the second multimedia content during streaming for the first multimedia content, if any point in the second multimedia content corresponds to an interframe, The decoding of the interframe is attempted in a state in which there is no information on the intraframe to be decoded, so that the interframe may not be successfully decoded.

To address this problem, the present disclosure provides a method and apparatus for supporting random access to any point in time of any multimedia content by examining and pre-processing when any starting point in each of one or more multimedia content is an interframe .

Accordingly, in the program scheduler capable of continuously reproducing a plurality of multimedia contents, it is possible to support random access to any program.

Here, the preprocessing may include decoding, storing, and managing intra frames referenced by the inter frames in advance.

Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings.

1 is a diagram for explaining multimedia content random access according to the present disclosure;

Figure 1 shows the scheduling (100) of a plurality of programs. In Fig. 1, the horizontal axis indicates time.

In Figure 1, for example, a first program (program # 1) including news, a second program (program # 2) including drama, a third program (concert # 3) And a fourth program (program # 4) to be reproduced.

Furthermore, the description with reference to Fig. 1 is merely an example, and the scope of the present disclosure is not limited by the number of such programs, the category of contents, the order or the timing of branching.

In the example of Fig. 1, there is no problem when the reproduction from the first frame of the program # 2 is started immediately after completing the reproduction up to the last frame of the program # 1 (A of Fig. 1). That is, since decoding and reproduction are started from the first intra frame 110 of the program # 2, information necessary for decoding the inter frames thereafter is secured, so that the subsequent program # 2 can be reproduced.

In the example of FIG. 1, a problem may arise when, after playing back to a certain frame in the middle of the program # 2 (B in FIG. 1), the reproduction of a certain frame in the middle of the program # 3 continuously starts. For example, a problem may arise particularly when the time at which the reproduction of the program # 3 starts corresponds to the interframe 140.

Concretely, in order to decode the interframe 140 of the program # 3, not the intra frame 130 of the program # 3 (refer to (2) in FIG. 1) (1 in Fig. 1), an error occurs in decoding the interframe 140 of the program # 3, so that the user may be provided with a heterogeneous image or may be unable to decode the intra frame 140. [

Next, in the example of FIG. 1, a problem may arise even when playback of a certain frame in the middle of the program # 3 is started (C in FIG. 1), and playback of a certain frame in the middle of the program # 4 is continuously started .

Even in this case, since the time at which playback of the program # 4 starts is the interframe 160, the intra frame 150 of the program # 4 can not be referred to, and the intra frame of the program # 3 can not be referred to. The intra frame 160 of the program # 4 is referred to because it refers to the intra frame 120 of the program # 2 which is the decoded intra frame, so that the decoding of the inter frame 160 of the program # 4 may be erroneous or impossible.

In order to prevent a decoding error that may occur in a random access to an interframe as described with reference to FIG. 1, a random access supporting operation as shown in FIG. 2 may be performed.

2 is a flow diagram illustrating a random access aiding operation in accordance with the present disclosure;

The target frame can be determined based on the scheduling information for one or more multimedia contents in step S210 of FIG.

Here, the scheduling information may include identification information of one or more multimedia contents, a reproduction order, a branching time, and the like. Here, the branching point may include at least one of a starting point (or a position or a frame) of decoding or reproduction of the multimedia contents, and an ending point (or a position or a frame) of decoding or reproducing.

For example, the necessity of preprocessing in certain multimedia contents can be determined by whether the reproduction start time determined based on the scheduling information of the multimedia contents is an interframe of the multimedia contents. That is, it can be determined that preprocessing is necessary when the start point of the decoding or reproduction of the multimedia content corresponds to the interframe.

Further, the determination as to whether or not the pre-processing is necessary can be performed before the reproduction of the entire program or during the reproduction.

If it is determined that preprocessing is necessary, that is, when the branching point (concretely, the start point of decoding or reproduction) corresponds to an interframe of a multimedia content, the intra frame associated with the interframe is determined as a preprocessing target frame .

Pre-processing for the target frame determined in step S220 may be performed.

Here, an intra frame associated with an interframe determined as a preprocessing target frame may be an intra frame to which the interframe is originally referenced (i.e., an intra frame referred to by the interframe in the same multimedia content to which the interframe belongs) have.

As described above, the intra frame associated with the inter frame is not actually reproduced, but can be preprocessed for correct decoding and reproduction of the inter frame.

Preprocessing for an intra frame may include one or more of decoding, storing, and managing the intra frame. The preprocessing may include decoding for at least an intra frame.

In addition, an intra frame associated with an inter frame is not necessarily limited to a frame preceding the inter frame.

For example, if the interframe is a B-frame (Bidirectional predictive frame), preprocessing for the frames before and after the interframe may be performed.

In addition, an intra frame associated with an inter frame may not necessarily be an intra frame immediately before the inter frame, but may be a few previous intra frames or one or more intra frames among a plurality of intra frames before the inter frame.

The intra frame associated with the inter frame may be an intra frame directly referred to by the inter frame, or may be an indirect frame referred to indirectly (for example, a frame directly referred to by the inter frame is another inter frame, Or an intra frame referred to by the another interframe).

In the example of FIG. 1, two programs corresponding to the interframe at the program branching time are assumed on the assumption of scheduling for four programs for clarity, but the scope of the present disclosure is not limited thereto.

For example, scheduling for dozens of programs may have a more complex scheduling scheme, in which case there may be a greater need for preprocessing for intra frames associated with inter frames.

Therefore, if scheduling for the entire program is determined, it is necessary to perform preprocessing at the entire branching point based on the scheduling information before the program starts for the first time or after the initial program is started (i.e., when the decoding or playback start point of the multimedia contents is It may be necessary to confirm or check whether there is an inter frame.

If it is specified that preprocessing is required, an intra frame associated with the specified interframe of the specified multimedia content can be determined.

Accordingly, prior to the decoding or reproduction of the specified interframe, pre-processing of the determined intra frame is performed to thereby decode the intra frame, store the decoded information, and determine whether the stored information is used for decoding an inter frame Management operations such as configuring a mapping table can be performed.

In step S230, the pre-processing result can be used for inter-frame decoding.

The intra frame information previously decoded, stored, and managed in the above-described step S220 is stored in the intra frame buffer used in the immediately preceding program at the time of branching to the interframe (i.e., starting decoding or playback of the interframe) You can replace stored information. Such an operation may be expressed as loading preprocessed information into an intra frame buffer.

Accordingly, even when the arbitrary decoding or reproduction starting point of a certain multimedia content is an interframe, the interframe can be decoded without errors so that a natural program branch and a reproduced image can be provided to the user.

In addition, even when one or more multimedia contents to be scheduled conform to different coding standards and formats, an intra frame buffer temporarily storing information used for reproduction of the first program according to the above-described configuration of the present invention can be used as a second By filling in the preprocessed information with respect to the intra frame associated with the program's interframe, decoding of the second program with respect to the interframe can be correctly performed without delay.

3 is a block diagram illustrating a random access aiding device according to the present disclosure;

The random access support apparatus 300 of FIG. 3 may be implemented as one module constituting a general scheduler, or in the form of software, hardware, firmware, or a combination thereof.

The target frame determination unit 310 may perform an interframe branch inspection operation.

The random access support apparatus 300 of FIG. 3 may further include a scheduling information acquisition unit (not shown). Alternatively, the scheduling information obtaining unit may be implemented as an external memory of the random access support apparatus.

The target frame determination unit 310 can determine a target frame based on the scheduling information for one or more multimedia contents.

Here, the scheduling information may include at least one of identification information for at least one multimedia content, a playback order, a branching time (e.g., starting point (or position or frame) of decoding or playback of the multimedia content, Frame), etc.), and the like.

Specifically, it can be determined that pre-processing is necessary when the start time of decoding or playback of multimedia contents corresponds to an inter frame based on the scheduling information. If it is determined that preprocessing is necessary, an intra frame associated with the inter- It can be determined as a target frame.

Here, an intra frame associated with an interframe determined as a preprocessing target frame may be an intra frame to which the interframe is originally referenced (i.e., an intra frame referred to by the interframe in the same multimedia content to which the interframe belongs) have.

The line processing unit 320 may perform intra frame decoding, storage, management, and the like.

Specifically, the pre-processing unit 320 can perform at least one of a decoding function, a storage function, and a management function for an intra frame determined as a preprocessing target frame by the preprocessing target determination unit 310. Or the line processing unit 320 may be implemented separately in an intra frame decoding unit and an intra frame storing and managing unit.

In addition, the decoding function or decoding unit of the line processing unit 320 may include a plurality of decoding functions or decoding units that support various standards and formats.

The preprocessing result output unit 330 can output the intra frame information decoded and stored by the preprocessing unit 320 to the intra frame buffer at the decoding time or the immediately preceding time of the inter frame.

Accordingly, the preprocessed intra frame information is used by the scheduler to decode the corresponding interframe at a branching point that starts with an interframe, thereby providing a natural and seamless interprogram branching without a random access delay time.

4 is a flow chart for explaining a random access method according to the present disclosure.

In step S410, for the entire program in which one or more programs are scheduled, a branch point to an interframe can be checked. A branch point to an interframe may mean that a start point of one program corresponds to an interframe.

This interframe branch point check may be performed before starting the entire program reproduction. Or an operation of checking a turning point for the next one or more programs may be performed during reproduction of a program, and before reproduction of the next one or more programs.

Here, the initial value of n in step S410 may be set to one. For example, it is possible to check a branch point from an n = 1 program to an interframe among all the N programs, and sequentially check a branch point to an interframe in the next program.

Alternatively, if each of the first X programs always starts decoding and playback from the initial starting point, then the interframe coding check for these programs is skipped and the initial value of n in step S410 may be set to X + 1 .

It is possible to determine whether or not a branch point to an inter-frame exists in step S420.

If there is a branch point to one or more interframes, the flow advances to step S430 to pre-process the intra frame associated with the interframe.

If there is no branch point to the inter-frame in step S420, the flow directly advances to step S450.

If there is a branch point to an interframe in step S430, an intra frame to which the interframe is originally referenced can be preprocessed. That is, an intra frame referred to by the interframe in the same n-th program to which the interframe belongs may be an intra frame existing before and / or after the interframe. Preprocessing of such intra frames may include decoding, storing and managing intra frames.

In step S440, it may be determined whether or not a branch point from the nth program to the remaining interframe exists.

For example, assuming scheduling examples as in the case of program # 1, program # 2, program # 4, program # 2, program # 1, A branch to an arbitrary time point of the specific program may be performed again. That is, there may be a branch point from one program to one or more interframes.

Therefore, it is possible to improve the random access performance by checking all the inter-frame branch points for each program and pre-processing the intra frames associated with each of the one or more inter frames.

If it is determined in step S440 that the branch point to the interframe remains in the nth program, the flow advances to step S430 to perform preliminary processing on the intra frame associated with the interframe.

If there is no branch to the interframe in the nth program in step S440, the process may proceed to step S450.

It may be determined in step S450 whether or not the branch point inspection for interframes for all N programs has been completed.

Finally, if the program in which the branch point inspection into the inter-frame is performed is the Nth program, the exemplary operation of FIG. 4 may be terminated.

If the program to be inspected remains, the process proceeds to step S460 where the value of n is incremented by 1, and accordingly, the branch point inspection from the next program (i.e., the (n + 1) th program) to the interframe is performed at step S410 .

According to various examples of the present disclosure described above, one or more branches of the multimedia content to any point in time may be continuously reproduced and may support substantial random access.

Although the exemplary methods of this disclosure are represented by a series of acts for clarity of explanation, they are not intended to limit the order in which the steps are performed, and if necessary, each step may be performed simultaneously or in a different order. In order to implement the method according to the present disclosure, the illustrative steps may additionally include other steps, include the remaining steps except for some steps, or may include additional steps other than some steps.

The various embodiments of the disclosure are not intended to be all-inclusive and are intended to illustrate representative aspects of the disclosure, and the features described in the various embodiments may be applied independently or in a combination of two or more.

In addition, various embodiments of the present disclosure may be implemented by hardware, firmware, software, or a combination thereof. In the case of hardware implementation, one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays A general processor, a controller, a microcontroller, a microprocessor, and the like.

The scope of the present disclosure is to be accorded the broadest interpretation as understanding of the principles of the invention, as well as software or machine-executable instructions (e.g., operating system, applications, firmware, Instructions, and the like are stored and are non-transitory computer-readable medium executable on the device or computer.

100 program scheduling
110, 120, 130, 150 Intra frames
140, 160 Interframe
300 random access support device
310 target frame determination unit
320 line processor
330 Preliminary result output unit

Claims (1)

A scheduling-based content random access method,
Determining a branch point to an inter-frame based on the scheduling information;
Performing preliminary processing on an intra frame associated with the interframe corresponding to the determined branch point; And
And performing decoding on the interframe based on the preprocessed intra frame information.

Images