KR20220030736A - Method, system, and computer readable record medium to minimize delay in real-time live streaming - Google Patents

Abstract

A method, system, and computer-readable recording medium to minimize delay in real-time live streaming are disclosed. In a real-time live streaming environment, at least two groups of pictures (GOPs) are cached for content received from a streamer, and a completed GOP is transmitted to a client through a cache as a packet beginning with an I-frame for the request of the client.

Description

Method, system, and computer-readable recording medium for minimizing delay in real-time live streaming

The description below relates to a technology for improving the quality experienced by users in a real-time live streaming environment.

The real-time live streaming service may refer to a technology that processes various kinds of transmitted multimedia data as if it were a continuous and continuous flow of water.

Real-time live streaming technology is becoming more important with the growth of the Internet, one of the reasons is that most users do not have a fast enough connection line to download large amounts of content immediately.

Streaming technology in such a situation allows the client player to start presenting data even before all files have been transferred.

For example, Korean Patent Laid-Open Publication No. 10-2006-0068547 (published on June 21, 2006) discloses a technology for providing a real-time streaming service in consideration of a transmission rate between a streaming server and a client.

As one of the qualities experienced by users in a real-time live streaming platform, a method and system are provided that can minimize the delay time from when the content on the platform is selected to output on the actual screen of the player.

In the cache process to minimize latency, we analyze the video information delivered from the streamer to find the audio that should be played when the video is played, and a method and system that can store the video and audio together. to provide.

We provide a method and system for delivering video and audio that can be played immediately with minimal latency through cache for user-selected content, but without causing synchronization issues in the player.

A method executed on a computer-implemented server, the server comprising at least one processor configured to execute computer readable instructions contained in a memory, the method comprising: by the at least one processor, real-time live streaming caching at least two groups of pictures (GOPs) for content received from a streamer in the environment; and forwarding, by the at least one processor, from a completed GOP to the client through a cache as an I-frame-initiated packet for the client's request.

According to one aspect, the GOP cache size may be determined based on a setting of an administrator related to the server or a network state between the server and the client.

According to another aspect, a delivery target GOP may be determined based on a setting of an administrator related to the server or a network state between the server and the client.

According to another aspect, in the delivering, when there are a plurality of GOPs completed through the cache, the most recently completed GOPs may be transferred to the client based on the access point of the client.

According to another aspect, in the delivering, when there are a plurality of GOPs completed through the cache, any one GOP among the plurality of GOPs may be delivered to the client according to a network state between the server and the client.

According to another aspect, the transmitting may include transmitting the remaining frames after erasing or skipping some frames according to the GOP reception state of the client.

According to another aspect, the caching may include finding an audio section corresponding to a GOP through analysis of a video stream and an audio stream of the content and aligning the video data and the audio data with a timeline. .

According to another aspect, the caching includes grouping the video data and the audio data by analyzing the data type for each frame of the video and matching each I-frame with the timestamp of the audio corresponding to the corresponding I-frame. can do.

According to another aspect, in the transmitting, the video data and the audio data may be reconstructed and transmitted in a form in which data in a packet unit intersects.

According to another aspect, the number of packet intersections may be determined based on sizes of the video data and the audio data.

There is provided a computer-readable recording medium in which a program for executing the method in a computer is recorded.

A computer-implemented server comprising: at least one processor configured to execute computer readable instructions contained in a memory, the at least one processor being configured to: The process of caching more GOPs; and a server that handles a process of transferring a packet starting with an I-frame to the client from a completed GOP through a cache in response to the client's request.

1 is a diagram illustrating an example of a network environment according to an embodiment of the present invention.
2 is a block diagram for explaining the internal configuration of an electronic device and a server according to an embodiment of the present invention.
3 is a diagram illustrating an example of a real-time live streaming environment according to an embodiment of the present invention.
4 is a block diagram illustrating an example of components that a processor of a server may include according to an embodiment of the present invention.
5 is a flowchart illustrating an example of a method that a server may perform according to an embodiment of the present invention.
6 to 10 are exemplary diagrams for explaining a GOP delivery process through a stream cache in an embodiment of the present invention.
11 is an exemplary diagram for explaining a process of storing video and audio together according to an embodiment of the present invention.
12 is an exemplary diagram for explaining a process of reconstructing and transmitting video and audio in packet units according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

Embodiments of the present invention relate to a technology for supporting optimal Quality of Experience (QoE) and Quality of Service (QoS) in a real-time live streaming environment.

Embodiments including those specifically disclosed herein can minimize the delay time from when the content on the platform is selected as one of the important qualities felt by the user in the real-time live streaming platform to being output to the actual screen of the player.

1 is a diagram illustrating an example of a network environment according to an embodiment of the present invention. The network environment of FIG. 1 shows an example including a plurality of electronic devices 110 , 120 , 130 , 140 , a plurality of servers 150 , 160 , and a network 170 . 1 is an example for explaining the invention, and the number of electronic devices or the number of servers is not limited as in FIG. 1 .

The plurality of electronic devices 110 , 120 , 130 , and 140 may be a fixed terminal implemented as a computer system or a mobile terminal. Examples of the plurality of electronic devices 110 , 120 , 130 , 140 include a smart phone, a mobile phone, a navigation device, a computer, a notebook computer, a digital broadcasting terminal, a personal digital assistant (PDA), and a portable multimedia player (PMP). ), tablet PCs, game consoles, wearable devices, Internet of things (IoT) devices, virtual reality (VR) devices, augmented reality (AR) devices, and the like. As an example, in FIG. 1 , the shape of a smartphone is shown as an example of the electronic device 110 , but in embodiments of the present invention, the electronic device 110 is substantially different through the network 170 using a wireless or wired communication method. It may refer to one of various physical computer systems capable of communicating with the electronic devices 120 , 130 , 140 and/or the servers 150 and 160 .

The communication method is not limited, and a communication method using a communication network (eg, a mobile communication network, a wired Internet, a wireless Internet, a broadcasting network, a satellite network, etc.) that the network 170 may include, as well as a short-distance wireless communication between devices may be included. can For example, the network 170 may include a personal area network (PAN), a local area network (LAN), a campus area network (CAN), a metropolitan area network (MAN), a wide area network (WAN), and a broadband network (BBN). , the Internet, and the like. In addition, the network 170 may include any one or more of a network topology including a bus network, a star network, a ring network, a mesh network, a star-bus network, a tree or a hierarchical network, etc. not limited

Each of the servers 150 and 160 communicates with the plurality of electronic devices 110 , 120 , 130 , 140 and the network 170 through a computer device or a plurality of computers that provides commands, codes, files, contents, services, etc. It can be implemented in devices. For example, the server 150 may be a system that provides a first service to a plurality of electronic devices 110 , 120 , 130 , 140 connected through the network 170 , and the server 160 is also a network ( It may be a system that provides the second service to the plurality of electronic devices 110 , 120 , 130 , and 140 connected through 170 . As a more specific example, the server 150 is installed on the plurality of electronic devices 110 , 120 , 130 , and 140 through an application as a computer program that is driven, and a service (eg, real-time live streaming service, etc.) ) may be provided to the plurality of electronic devices 110 , 120 , 130 , and 140 as a first service. As another example, the server 160 may provide a service for distributing a file for installing and driving the above-described application to the plurality of electronic devices 110 , 120 , 130 , and 140 as the second service.

2 is a block diagram for explaining the internal configuration of an electronic device and a server according to an embodiment of the present invention. In FIG. 2 , the internal configuration of the electronic device 110 and the server 150 will be described as an example of the electronic device. In addition, other electronic devices 120 , 130 , 140 or server 160 may also have the same or similar internal configuration to the aforementioned electronic device 110 or server 150 .

The electronic device 110 and the server 150 may include memories 211 and 221 , processors 212 and 222 , communication modules 213 and 223 , and input/output interfaces 214 and 224 . The memories 211 and 221 are computer-readable recording media, and non-volatile mass storage such as random access memory (RAM), read only memory (ROM), disk drive, solid state drive (SSD), flash memory, etc. It may include a permanent mass storage device. Here, a non-volatile mass storage device such as a ROM, SSD, flash memory, disk drive, etc. may be included in the electronic device 110 or the server 150 as a separate permanent storage device distinct from the memories 211 and 221 . In addition, in the memories 211 and 221 , an operating system and at least one program code (eg, a browser installed and driven in the electronic device 110 or codes for an application installed in the electronic device 110 to provide a specific service) can be stored. These software components may be loaded from a computer-readable recording medium separate from the memories 211 and 221 . The separate computer-readable recording medium may include a computer-readable recording medium such as a floppy drive, a disk, a tape, a DVD/CD-ROM drive, and a memory card. In another embodiment, the software components may be loaded into the memories 211 and 221 through the communication modules 213 and 223 rather than the computer-readable recording medium. For example, the at least one program is a computer program installed by files provided through the network 170 by a file distribution system (eg, the above-described server 160 ) for distributing installation files of developers or applications. It may be loaded into the memories 211 and 221 based on (eg, the above-described application).

The processors 212 and 222 may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input/output operations. Commands may be provided to the processors 212 and 222 by the memories 211 and 221 or the communication modules 213 and 223 . For example, the processors 212 and 222 may be configured to execute received instructions according to program codes stored in a recording device such as the memories 211 and 221 .

The communication modules 213 and 223 may provide a function for the electronic device 110 and the server 150 to communicate with each other through the network 170 , and the electronic device 110 and/or the server 150 may communicate with each other through the network 170 . A function for communicating with an electronic device (eg, the electronic device 120 ) or another server (eg, the server 160 ) may be provided. For example, a request generated by the processor 212 of the electronic device 110 according to a program code stored in a recording device such as the memory 211 is transmitted to the server 150 through the network 170 under the control of the communication module 213 . ) can be transferred. Conversely, a control signal, command, content, file, etc. provided under the control of the processor 222 of the server 150 passes through the communication module 223 and the network 170 to the communication module 213 of the electronic device 110 . ) through the electronic device 110 . For example, a control signal, command, content, file, etc. of the server 150 received through the communication module 213 may be transmitted to the processor 212 or the memory 211 , and the content or file may be transmitted to the electronic device ( 110) may be stored as a storage medium (the above-described permanent storage device) that may further include.

The input/output interface 214 may be a means for interfacing with the input/output device 215 . For example, the input device may include a device such as a keyboard, mouse, microphone, camera, and the like, and the output device may include a device such as a display, a speaker, a haptic feedback device, and the like. As another example, the input/output interface 214 may be a means for an interface with a device in which functions for input and output are integrated into one, such as a touch screen. The input/output device 215 may be configured as one device with the electronic device 110 . In addition, the input/output interface 224 of the server 150 may be a means for interfacing with a device (not shown) for input or output that is connected to the server 150 or that the server 150 may include. As a more specific example, a service configured using data provided by the server 150 or the electronic device 120 when the processor 212 of the electronic device 110 processes a command of a computer program loaded into the memory 211 . A screen or content may be displayed on the display through the input/output interface 214 .

In addition, in other embodiments, the electronic device 110 and the server 150 may include more components than those of FIG. 2 . However, there is no need to clearly show most of the prior art components. For example, the electronic device 110 is implemented to include at least a portion of the above-described input/output device 215 or other components such as a transceiver, a global positioning system (GPS) module, a camera, various sensors, and a database. may include more. As a more specific example, when the electronic device 110 is a smart phone, an acceleration sensor or a gyro sensor, a camera module, various physical buttons, a button using a touch panel, an input/output port, and a vibrator for vibration generally included in the smart phone Various components such as may be implemented to be further included in the electronic device 110 .

Hereinafter, specific embodiments of a method and system capable of minimizing the delay time experienced by a user in a real-time live streaming environment will be described.

3 is a diagram illustrating an example of a real-time live streaming environment according to an embodiment of the present invention.

FIG. 3 illustrates a content delivery network (CDN) service environment including a client 310 , a content provider (CP) 320 , and a server 350 as an example of a real-time live streaming environment. The server 350 may correspond to the server 150 described with reference to FIGS. 1 and 2 , and the client 310 and the CP 320 are the electronic devices 110 , 120 , 130 , described with reference to FIGS. 1 and 2 . 140) and may correspond.

The CDN service is a service of a real-time live streaming environment, and after storing the contents provided by the CP 320 corresponding to the streamer in several servers 350 installed at the bottom of the network of the Internet Service Provider (ISP) in advance, When there is a request from the client 310 corresponding to a viewer, the system is operated to deliver the corresponding content to the client 310 .

A player on the client 310 side receives the content selected by the user on the platform from the server 350 and then outputs the received content through an output device (display, speaker, etc.) to play it.

As one of the qualities perceived by the user in the real-time live streaming environment, there is a delay time from when the content on the platform is selected to actually appearing on the screen of the player.

Embodiments of the present invention may include a technique capable of minimizing the delay time described above, and may also include a technique capable of removing a side effect of a technique applied to minimize the delay time.

4 is a block diagram illustrating an example of components that a processor of a server may include according to an embodiment of the present invention, and FIG. 5 is an example of a method that the server may perform according to an embodiment of the present invention is a flowchart showing

The server 350 according to the present embodiment serves as a cache server that delivers the content provided by the CP 320 to the client 310 in a real-time live streaming environment. The server 350 may be configured with a computer-implemented real-time live streaming system. The server 350 may provide a real-time live streaming service to the client 310 through a dedicated application installed on the client 310 or a web/mobile site connection related to the server 350 .

The processor 222 of the server 350 is a component for performing the real-time live streaming method according to FIG. 5 , and as shown in FIG. 4 , a stream receiving unit 410 , a stream cache unit 420 , and an audio video analyzing unit 430 , and a stream transmission unit 440 . Depending on the embodiment, components of the processor 222 may be selectively included in or excluded from the processor 222 . In addition, according to an embodiment, the components of the processor 222 may be separated or combined to express the functions of the processor 222 .

The processor 222 and components of the processor 222 may control the server 350 to perform steps S510 to S540 included in the real-time live streaming method of FIG. 4 . For example, the processor 222 and components of the processor 222 may be implemented to execute instructions according to the code of the operating system included in the memory 221 and the code of at least one program.

Here, the components of the processor 222 may be expressions of different functions performed by the processor 222 according to an instruction provided by the program code stored in the server 350 . For example, the stream receiving unit 410 may be used as a functional representation of the processor 222 that controls the server 350 according to the above-described command so that the server 350 receives a stream of content from the CP 320 . .

The processor 222 may read a necessary command from the memory 221 in which the command related to the control of the server 350 is loaded. In this case, the read command may include a command for controlling the processor 222 to execute steps S510 to S540 to be described later.

Steps S510 to S540 to be described later may be performed in an order different from that shown in FIG. 5 , and some of the steps S510 to S540 may be omitted or additional processes may be further included.

Referring to FIG. 5 , in step S510 , the stream receiving unit 410 may receive a stream of content provided by the streamer CP 320 through a real-time live streaming platform.

In operation S520 , the stream cache unit 420 may cache the video stream for the content received in operation S510 in units of group of pictures (GOP). A video consists of an I-frame (intra frame), a P-frame (predicted frame), and a B-frame (bidirectional frame), and at least one I-frame is absolutely necessary to start playback. The stream cache unit 420 may cache the video stream in units of GOPs in response to the request of the client 310 to deliver a packet starting with an I-frame.

In step S530, the audio/video analysis unit 430 may find an audio section corresponding to the GOP unit through analysis of the video stream and the audio stream and store it together with the corresponding video. In order to solve the side effects that may occur when the video stream is delivered to the client 310 in units of GOP starting with I-frames, the audio/video analysis unit 430 performs GOP unit video and audio corresponding to the video. You can find segments, align and archive video and audio.

In step S540 , the stream transmission unit 440 may transmit a GOP unit video stream together with audio to the player of the client 310 according to the request of the client 310 . The stream transmission unit 440 transmits the cached GOP unit video and the audio of the section corresponding to the video to the player of the client 310, in the form of cross-editing the video and audio in packet units, which are the smallest data transmission units. can transmit In order to solve a side effect that may occur when the video stream is delivered in units of GOPs, video and audio may be reconstructed so that they intersect in units of packets and delivered to the player of the client 310 .

Hereinafter, a real-time live streaming process will be described with a specific example.

In general, a video has a form in which I-frames, P-frames, and B-frames are repeated in a constant pattern, and may be encoded in one image frame group, that is, GOP unit. Referring to FIG. 6 , one GOP includes an I-frame, a P-frame, and a B-frame, and B-frames and P-frames are arranged between I-frames and I-frames. In other words, a set of consecutive frames starting with an I-frame is called a GOP. In some cases, the GOP may be performed only with the I-frame and the P-frame without the B-frame.

The I-frame means a key frame and corresponds to a key frame that is the basis of the GOP. The first frame of every GOP must start with an I-frame. I-frames are frames stored as-is without reference to other previous frames. In addition, the P-frame is a forward prediction frame, and is a frame in which partial data having a difference is predicted based on the I-frame located immediately before and stored. Finally, the B-frame is a bidirectional prediction frame, which is located between the I-frame and the P-frame and stores motion between the two frames as speculative data with reference to both frames.

As shown in FIG. 7 , the time point (ⓐ) at which the client 310 is connected to the server 350 for content viewing is mostly in the middle of the GOP being generated when the client 310 is connected, and from that point It is highly likely that you will receive a stream. For example, if GOP1 is connected while being generated, the client 310 receives the stream immediately after the access point (ⓐ), and thereafter, the video start packet of the new GOP (GOP2), that is, I-frame (I_2) The viewer experiences a delay as they cannot see anything through the player screen until they receive it.

In order to minimize this delay time, the server 350 mounts a GOP unit cache. In this embodiment, the delay time can be minimized by delivering the most recently cached complete GOP to the GOP starting with the I-frame through the video stream cache, not from the stream immediately after the connection point of the client 310. . The client 310 can immediately play the video in the player by receiving from I-frames as a video stream in GOP units.

In the present embodiment, a cache size for a video stream may be designated in consideration of a server administrator setting or a network state between the client 310 and the server 350 .

When updating a new GOP in the video stream caching process, the first completed GOP among the cached GOPs is discarded.

Referring to FIG. 8 , in the case of a cache 701 having a size of 1, if the client 310 is connected before one GOP is completed, that is, if it is connected during the GOP cache, the stream is not delivered until the GOP is completed. As a result, viewers still feel the delay. This is because, when one I-frame is accessed before being completed as a GOP, the delay from the time of access until the GOP is completed and delivered to the client 310 through the network is exposed as it is.

In order to prevent such a problem, in the present embodiment, a cache 702 having a size of 2 or more may be applied so that at least two or more GOPs can be cached. In other words, the size of the GOP cache may be specified as at least 2 or more, and it is also possible to set which GOP will be delivered based on the access point of the client 310 .

For example, it may be specified to set the GOP cache size to 2 and access from the last completed GOP. As shown in FIG. 9 , assuming that GOP1 is completed in the cache 702 and GOP2 is in the process of being cached, even if the client 310 approaches the time point ⓑ, it can receive from GOP1, which is the completed GOP, and play immediately, so the cache proceeds. The delay time until the in-progress GOP2 is completed and delivered can be eliminated.

Although it has been described that the most recently completed GOP is delivered based on the access point of the client 310, the delivery target GOP may be determined differently in some cases. For example, referring to FIG. 10 , when the GOP cache size is 3, GOP1 and GOP2 are completed first in chronological order, and when the client 310 is connected at the time ⓒ in the situation where the GOP3 is cached, it is connected with the client 310. It may be determined from which GOP of GOP1 and GOP2 to transmit in consideration of the network state. As an example, if the current network bandwidth is greater than or equal to the threshold based on the access point (ⓒ) of the client 310, the most recently completed GOP2 is transmitted, while if the current network bandwidth is less than the threshold, it is transmitted from the GOP1 completed before GOP2 there is.

As another example, an Adaptive Bitrate Publish (ABP) technology for transmitting data according to a network bandwidth may be applied. In a real-time live protocol (eg, real time messaging protocol (RTMP) environment), adaptive data transmission that measures network bandwidth and immediately changes bitrate or fps (frame per second) adaptively to the measured bandwidth technology can be used. Similarly, in a real-time live streaming environment using a stream cache, the most recently completed GOP is delivered based on the connection time of the client 310 , but as a result of monitoring the client 310 , the reception situation on the client 310 side is not good, so the GOP When is delayed, a method of dropping some packets (B-frames and/or P-frames) from the GOP or a method of skipping or jumping some GOPs according to the content transmission time may be applied. A frame length to be erased (skipped or jumped) may vary according to a monitoring period for the client 310 . For example, when the client 310 accesses to view a specific program, the GOP from the most recently completed GOP is delivered to the client 310 side based on the connection time, and as a result of the first monitoring, the GOP from the client 310 side If it is not received normally and is delayed, a frame corresponding to the first length is erased. Similarly in the secondary monitoring result, when the GOP is continuously pushed, frames of a second length greater than the first length are erased. In other words, for uninterrupted viewing from the client 310 side, some frames are erased based on the reception state of the client 310 and then the remaining frames are transmitted. In this case, the length of the frame to be erased can be adjusted according to the monitoring period.

Caching and delivering the video stream in units of GOPs starting with I-frames may minimize delay time, but may increase the possibility of malfunction of the player on the client 310 side. Since the GOP contains only video information, the audio/video synchronization mechanism built into the player may malfunction. For example, in the case of a method in which one GOP, which is video data, is first transmitted and then audio is transmitted, it is assumed that there is no audio data in the reception process due to the size of the video data and playback may be started.

In order to prevent such a problem, the processor 222 analyzes the image information (video stream and audio stream) received from the CP 320, so that not only the GOP but also the audio to be played when one GOP is played is also displayed on the timeline. It can be sorted and stored.

Referring to FIG. 11 , in the stream cache process, data types (I-frame, B-frame, P-frame) are analyzed for each video frame, and the corresponding I-frame for each of the I-frames (I_1, I_2, ??) Video and audio can be grouped according to the timestamp of audio (audio_1, audio_2, ??) corresponding to .

In addition, the processor 222 may cache video and audio together through the grouping described above and transmit the video and audio together in consideration of the player's audio-video synchronization mechanism when delivering to the player.

If you pass one GOP corresponding to video data and then audio, or audio first and then one GOP, depending on the data size of the video or audio being passed, there may be a problem with the audio-video synchronization mechanism of the player. There is a possibility.

In order to solve this problem, the processor 222 may reconstruct and transmit video and audio in a packet-by-packet cross-edited form. Referring to FIG. 12 , the processor 222 transmits video data and audio data audio_1 corresponding to one GOP (GOP1) to the player of the client 310, video (V) in packets and The audio (A) can be reconstructed and transmitted in an intersecting form.

In the packet crossing method, packets of video (V) and audio (A) may be cross-arranged one-to-one. As another example, the number of packet intersections is determined according to the data sizes of the video V and the audio A, and it is also possible to cross-arrange the packets in any one of one-to-one, one-to-many, many-to-one, and many-to-many.

By reconstructing and delivering video data and audio data in packet-by-packet cross-edited format, the player's audio-video synchronization mechanism works without any problem.

The player can play the screen only by receiving the I-frame. Since the access point of the user is unlikely to correspond to the I-frame location, it is possible to ensure that the player always receives the I-frame and plays it as quickly as possible by delivering the most recently completed GOP based on the access point.

Real-time live protocols such as RTMP run on top of TCP. In order to make the transmission speed sufficiently fast in TCP, sufficient bandwidth must be secured through a slow start process. By transmitting the GOP through the stream cache from the server 350 to the player, it is possible to obtain the effect of rapidly advancing the slow start process (warm-up).

In addition, the player generally starts playing when the internal buffer is filled with data of a certain size or more for stable playback. By sending the GOP through the stream cache from the server 350 to the player, the buffer can be quickly filled, so that not only the first screen according to the I-frame is drawn, but also the actual video playback thereafter can be quickly processed.

As described above, according to embodiments of the present invention, by using the GOP unit cache in a real-time live streaming environment to deliver video and audio in a form that can be played immediately to the player, delay time can be minimized to improve user experience quality. Furthermore, according to embodiments of the present invention, the video information transmitted from the streamer is analyzed in the cache process to minimize the delay time, and the audio to be played together when the video is played is found and stored together, and then the video is transmitted during the transmission process. and audio in a packet-by-packet cross-edited format, it is possible to effectively solve the synchronization issue in the player, which may occur as a side effect of the cache layer.

The device described above may be implemented as a hardware component, a software component, and/or a combination of the hardware component and the software component. For example, the apparatus and components described in the embodiments may include a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), and a programmable logic unit (PLU). It may be implemented using one or more general purpose or special purpose computers, such as a logic unit, microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications executed on the operating system. A processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For convenience of understanding, although one processing device is sometimes described as being used, one of ordinary skill in the art will recognize that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that can include For example, the processing device may include a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as parallel processors.

Software may comprise a computer program, code, instructions, or a combination of one or more thereof, which configures a processing device to operate as desired or is independently or collectively processed You can command the device. The software and/or data may be embodied in any tangible machine, component, physical device, computer storage medium or device for interpretation by or providing instructions or data to the processing device. there is. The software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored in one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. In this case, the medium may be to continuously store a program executable by a computer, or to temporarily store it for execution or download. In addition, the medium may be various recording means or storage means in the form of a single or several hardware combined, it is not limited to a medium directly connected to any computer system, and may exist distributed over a network. Examples of the medium include a hard disk, a magnetic medium such as a floppy disk and a magnetic tape, an optical recording medium such as CD-ROM and DVD, a magneto-optical medium such as a floppy disk, and those configured to store program instructions, including ROM, RAM, flash memory, and the like. In addition, examples of other media may include recording media or storage media managed by an app store that distributes applications, sites that supply or distribute other various software, and servers.

As described above, although the embodiments have been described with reference to the limited embodiments and drawings, various modifications and variations are possible from the above description by those skilled in the art. For example, the described techniques are performed in an order different from the described method, and/or the described components of the system, structure, apparatus, circuit, etc. are combined or combined in a different form than the described method, or other components Or substituted or substituted by equivalents may achieve an appropriate result.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims (20)

In the method executed on a server implemented by a computer,
the server comprises at least one processor configured to execute computer readable instructions contained in a memory;
The method is
caching, by the at least one processor, at least two or more groups of pictures (GOPs) for content received from a streamer in a real-time live streaming environment; and
forwarding, by the at least one processor, from a completed GOP to the client through a cache as an I-frame-initiated packet for the client's request;
How to include. According to claim 1,
The GOP cache size is determined based on the settings of the administrator related to the server or the network status between the server and the client
How to characterize. According to claim 1,
The delivery target GOP is determined based on the setting of the administrator related to the server or the network status between the server and the client
How to characterize. According to claim 1,
The delivering step is
When there are a plurality of GOPs completed through the cache, the most recently completed GOP is delivered to the client based on the access point of the client.
How to characterize. According to claim 1,
The delivering step is
When there are a plurality of GOPs completed through the cache, transferring from any one of the plurality of GOPs to the client according to the network state between the server and the client
How to characterize. According to claim 1,
The delivering step is
Transmitting the remaining frames after erasing or skipping some frames according to the GOP reception state of the client
How to include. According to claim 1,
The caching step is
Finding an audio section corresponding to the GOP through analysis of the video stream and the audio stream of the content and aligning the video data and the audio data with a timeline
How to include. According to claim 1,
The caching step is
Analyzing the data type for each frame of the video and grouping the video data and the audio data by matching each I-frame with the timestamp of the audio corresponding to the corresponding I-frame
How to include. According to claim 1,
The delivering step is
Reconstructing and transmitting video data and audio data in the form of intersecting packet unit data
How to characterize. 10. The method of claim 9,
determining the number of packet intersections based on the sizes of the video data and the audio data
How to characterize. A computer-readable recording medium in which a program for causing a computer to execute the method of any one of claims 1 to 10 is recorded. In the server implemented by a computer,
at least one processor configured to execute computer readable instructions contained in memory
including,
the at least one processor,
A process of caching at least two or more GOPs for content received from a streamer in a real-time live streaming environment; and
The process of transferring from a completed GOP to the client through a cache as a packet starting with an I-frame for a client's request
server that handles it. 13. The method of claim 12,
The GOP cache size is determined based on the settings of the administrator related to the server or the network status between the server and the client
Server characterized by. 13. The method of claim 12,
The delivery target GOP is determined based on the setting of the administrator related to the server or the network status between the server and the client
Server characterized by. 13. The method of claim 12,
The at least one processor,
When there are a plurality of GOPs completed through the cache, the most recently completed GOP is delivered to the client based on the access point of the client.
Server characterized by. 13. The method of claim 12,
the at least one processor,
When there are a plurality of GOPs completed through the cache, transferring from any one of the plurality of GOPs to the client according to the network state between the server and the client
Server characterized by. 13. The method of claim 12,
the at least one processor,
Transmitting the remaining frames after erasing or skipping some frames according to the GOP reception status of the client
Server characterized by. 13. The method of claim 12,
the at least one processor,
Finding the audio section corresponding to the GOP through the analysis of the video stream and the audio stream of the content and aligning the video data and the audio data with the timeline
Server characterized by. 13. The method of claim 12,
the at least one processor,
Analyzing data types for each frame of video and grouping video data and audio data by matching each I-frame with the timestamp of the audio corresponding to the corresponding I-frame
Server characterized by. 13. The method of claim 12,
The at least one processor,
Reconstructing and transmitting video data and audio data in the form of intersecting packet unit data
Server characterized by.

Images