KR20210110097A - The method for streaming a video and the electronic device supporting same - Google Patents

Abstract

The present invention relates to a video streaming method capable of reducing a network bandwidth used for video reproduction and an electronic device for supporting the same. According to one embodiment of the present invention, the electronic device transmit video to an external device and comprises: a communication circuit; a memory for storing a plurality of images corresponding to a plurality of viewpoints; and a processor. Each of the plurality of images includes a plurality of sub-tracks, each of the plurality of sub-tracks includes consecutive frames, each of the frames has one attribute of a first type frame or a second type frame, and the plurality of sub-tracks have different ratios of the first type frame among all frames. The processor transmits one of the plurality of sub-tracks of a first image for a first viewpoint among the plurality of images to the external device; receives a request for changing the viewpoint from the external device through the communication circuit; and transmits, at a first time, a first sub-track including the first type frame among a plurality of sub-tracks of a second image for a second time point corresponding to the request to the external device. Various other embodiments understood through the specification are possible.

Description

The method for streaming a video and the electronic device supporting same}

Various embodiments of the present document relate to a method of streaming an image and an electronic device supporting the same.

An image service for providing a multi-viewpoint image simultaneously captured using a plurality of cameras to a user's terminal device is being used. In the process of streaming an image corresponding to one viewpoint from the server to the terminal device, when a request to switch the viewpoint occurs in the terminal device, the server may stream an image of another viewpoint in response to the request. For example, in a baseball field, each image is taken using a plurality of cameras, and one image (eg, first-base side shot) is streamed by the user's terminal device (eg, smartphone, tablet PC, laptop PC) can be displayed through When the user generates an input for changing a viewpoint, an image captured from a different angle (eg, an image taken from the third base side) may be streamed.

A method of transmitting a multi-view video according to the prior art includes: i) a method of simultaneously receiving images from different viewpoints, ii) a method of additionally generating a switching frame, iii) a method of transmitting two streams in a hierarchical structure, iv ), a method of delivering an additional image for changing the viewpoint is being used. The above schemes may require an increase in network bandwidth or may increase encoding resources and cost due to additional frame generation.

The server device according to various embodiments of the present document may reduce network bandwidth required for streaming by using sub-tracks having different frame structures for one view.

The electronic device transmits an image to an external device and includes a communication circuit, a memory for storing a plurality of images corresponding to a plurality of viewpoints, and a processor, each of the plurality of images comprising a plurality of sub-tracks, Each of the plurality of sub-tracks consists of consecutive frames, each of the frames has one property of a first-type frame or a second-type frame, and the plurality of sub-tracks includes the first type of the entire frame. frame rates are different from each other, and the processor transmits one of a plurality of sub-tracks of a first image for a first viewpoint among the plurality of images to the external device, from the external device through the communication circuit Upon receiving a request for changing the view, the first sub-track including the first type frame from among a plurality of sub-tracks of a second image for a second view corresponding to the request is played by the external device at a first time can be sent to

The electronic device according to various embodiments disclosed herein may reduce a network bandwidth used for image reproduction in a multi-view video service by using sub-tracks having different frame structures.

The electronic device according to various embodiments disclosed in this document may reduce a response time to a user's request for changing a viewpoint by using sub-tracks having different frame structures.

1 illustrates a video streaming system according to various embodiments.
2 shows a streaming server according to an embodiment.
3 illustrates a storage and selection operation of a plurality of sub-tracks according to various embodiments of the present disclosure.
4 shows a video streaming method performed by a streaming server according to various embodiments.
5 illustrates a dynamic change of a frame structure over time according to various embodiments of the present disclosure;
6 illustrates a video streaming method through transmission of frame structure information according to various embodiments.
7 is a block diagram of an electronic device in a network environment according to various embodiments of the present disclosure;
In connection with the description of the drawings, the same or similar reference numerals may be used for the same or similar components.

Hereinafter, various embodiments of the present document will be described with reference to the accompanying drawings. However, it is not intended to limit the technology described in this document to specific embodiments, and it should be understood to include various modifications, equivalents, and/or alternatives of the embodiments of this document. . In connection with the description of the drawings, like reference numerals may be used for like components.

1 illustrates a video streaming system according to various embodiments.

Referring to FIG. 1 , the video streaming system 100 may include a source device 110 , a streaming server 120 , and a terminal device 130 . The video streaming system 100 may include other devices among the components of FIG. 1 . For example, a separate association server may be included between the source device 110 and the streaming server 120 . For another example, a separate relay device (eg, a desktop PC) may be included between the streaming server 120 and the terminal device 130 .

According to various embodiments, the source device 110 may collect or store multi-view images. For example, the source device 110 may be a camera device capable of capturing an image, a storage device capable of storing an image captured by the camera device, or a plurality of image processing devices (eg, a plurality of smart phones). have. In an embodiment, the multi-viewpoint image may be an image obtained by photographing the same object from various angles, or may be an image photographed in various directions within the same or adjacent space. For example, the multi-view image may be an image captured by installing a plurality of cameras at various angles in a baseball field. The images captured by each camera may be stored in association with each other according to the shooting time, or may be streamed through a network.

According to various embodiments, the streaming server 120 (eg, the server 708 of FIG. 7 ) may receive an image from the source device 110 . The streaming server 120 may stream the received image to the terminal device 130 . The streaming server 120 may receive a real-time image from the image source device 110 or a stored image (eg, a video on demand (VOD) image).

In an embodiment, the streaming server 120 may receive a multi-view image from the source device 110 . The multi-view image may be an image obtained by photographing the same object from various angles, or may be an image photographed in various directions within the same or adjacent space. For example, the streaming server 120 may receive images (or image sources) corresponding to N views by mapping the N tracks.

According to various embodiments, the streaming server 120 transmits the received multi-view image to a plurality of sub-tracks (or a plurality of sub-views) having different GOP structures for an image (hereinafter, a viewpoint image) corresponding to each view. ) can be converted to For example, the first view image of the first view may be converted into a plurality of sub-tracks having different GOP structures. When each sub-track is decoded, the same image as the first view image may be restored.

According to various embodiments, the streaming server 120 may transmit the received multi-view image to the terminal device 130 . The streaming server 120 may stream a viewpoint image requested by the terminal device 130 . For example, the streaming server 120 may stream a fifth viewpoint image (or a fifth track) corresponding to the fifth viewpoint to the terminal device 130 according to the request of the terminal device 130 .

According to various embodiments, when the streaming server 120 receives a request to stream a different viewpoint image from the terminal device 130 while streaming one viewpoint image (hereinafter, referred to as a viewpoint switching request), a viewpoint switching request It is possible to stream a viewpoint image corresponding to . For example, the streaming server 120 may receive a view switching request to change to the fifth view from the terminal device 130 while streaming the 17th view image to the terminal device 130 . The streaming server 120 may stream the fifth viewpoint image to the terminal device 130 in response to the viewpoint switching request.

According to various embodiments, the streaming server 120 may provide a streaming image to the plurality of terminal devices 130 when there are a plurality of terminal devices 130 . In this case, since it is necessary to provide a streaming service using a limited network resource (eg, bandwidth), the streaming server 120 needs to efficiently transmit streaming data. In addition, the streaming server 120 quickly responds to the request for changing the viewpoint requested by the terminal device 130 in order to satisfy the user's service quality requirements, and it is necessary to quickly transmit the viewpoint image requested by each terminal device 130 . there is The streaming server 120 can efficiently manage a network resource (eg, bandwidth) by using a plurality of sub-tracks having different frame structures for one viewpoint image, and request a viewpoint change of the terminal device 130 . can react quickly to

According to various embodiments, the streaming server 120 may include a processor 150 , a memory 160 and a communication circuit 190 .

The processor 150 may perform various operations necessary for the operation of the streaming server 120 . The processor 150 may receive a multi-view image from the source device 110 through the communication circuit 190 . The processor 150 may transmit a viewpoint image requested by the terminal device 130 from among a plurality of viewpoint images constituting the received multi-view image.

According to various embodiments, the processor 150 may transmit one of the multi-view images to the terminal device 130 through the communication circuit 190 . For example, the streaming server 120 may stream a fifth viewpoint image (or a fifth track) corresponding to the fifth viewpoint to the terminal device 130 according to the request of the terminal device 130 .

According to various embodiments, the processor 150, the streaming server 120, in response to the request for changing the viewpoint of the terminal device 130, may change the streaming viewpoint image. For example, the processor 150 may receive a view switching request to change to the fifth view from the terminal device 130 while streaming the 17th view image to the terminal device 130 . The processor 150 may stream the fifth viewpoint image to the terminal device 130 in response to the viewpoint switching request.

According to various embodiments, the processor 150 may store a plurality of sub-tracks (or a plurality of sub-images) having different frame structures for one view image.

In an embodiment, a plurality of sub-tracks for one view image may have different group of pictures (GOP) structures. A plurality of sub-tracks constituting one view image may have different I-frame rates. For example, when one view image is composed of first to third sub-tracks, all frames of the first sub-track may be composed of I-frames. The second sub-track may have an I-frame rate of 1/4 and a P-frame rate of 3/4 among all frames. The third sub-track may have an I-frame rate of 1/8 of the entire frame and a P-frame rate of 7/8.

According to various embodiments, the processor 150 may check various conditions such as a network situation and the number of terminal devices 130 , select one sub-track from among a plurality of sub-tracks, and transmit the selected sub-track to the terminal device 130 . have.

According to various embodiments, the memory 160 may store various information necessary for the operation of the streaming server 120 . According to an embodiment, the memory 160 may include an input buffer 170 and an output buffer 180 . The input buffer 170 and the output buffer 180 may be set to be distinguished by physical or logical addresses, or may be implemented with different storage devices. The input buffer 170 may store a multi-view image. The input buffer 170 may store a plurality of sub-tracks for each viewpoint image. The plurality of sub-tracks may have different frame structures, and when decoded, may be converted into images having the same quality. The output buffer 180 may store an image to be transmitted to the terminal device 130 . The output buffer 180 may temporarily store a viewpoint image (or a sub-track selected from among a plurality of sub-tracks constituting the viewpoint image) by switching of the processor 150 .

According to various embodiments, the communication circuit 190 may transmit/receive data to and from an external device (eg, the source device 110 or the terminal device 130) through wireless communication or wired communication. The communication circuit 190 may receive the viewpoint image from the source device 110 and transmit it to be stored in the input buffer 170 . Alternatively, the communication circuit 190 may transmit the image stored in the output buffer 180 to the terminal device 130 .

According to various embodiments, the terminal device 130 (eg, the electronic device 701 of FIG. 7 ) may receive data regarding a single viewpoint image through a network and display it through a display. There may be a plurality of terminal devices 130 , and may receive and output a viewpoint image desired by each user from the streaming server 120 . The terminal device 130 may be outputting images from different viewpoints. For example, when the terminal device 130 includes the first to fifth terminal devices, the first to fifth terminal devices may output images from different viewpoints, respectively. Alternatively, the first to third terminal devices may output a first viewpoint image, the fourth terminal device may output a second viewpoint image, and the fifth terminal device may output a fourth viewpoint image. Viewpoint images selected by each user of the terminal device 130 may be variously output.

2 shows a streaming server according to an embodiment.

Referring to FIG. 2 , the streaming server 120 according to various embodiments may include an input buffer 170 , a track selector 220 and an output buffer 180 . can In FIG. 2 , the streaming server 120 is classified according to functions, but is not limited thereto. Some components may be added or omitted. Alternatively, the components may be integrated or separate. The streaming server 120 may not be a single device or a single server device, and a plurality of devices or a plurality of server devices may be combined. Alternatively, at least some functions of the streaming server 120 may be performed in another device or the terminal device 130 .

The input buffer 170 according to various embodiments may store a multi-view image. The input buffer 170 may store a plurality of sub-tracks for the viewpoint images 250-1, 250-2, ... 250-N corresponding to each viewpoint. The plurality of sub-tracks may have different frame structures, and when decoded, may be converted into images having the same quality. For example, the first viewpoint image 250 - 1 may include three sub-tracks, and each sub-track may have a different I-frame rate.

2 exemplarily illustrates a case in which one viewpoint image is composed of three sub-tracks, but the present invention is not limited thereto. Various embodiments of the plurality of sub-tracks stored in the input buffer 170 may be further provided with reference to FIG. 3 .

The track selector 220 according to various embodiments transmits the image and the first selector (or first switch) 221 that selects a view image (or track) in response to a view change request from the terminal device 130 . A second selector (or second switch) 222 for selecting one of a plurality of sub-tracks (or sub-tracks) may be included for efficiency.

The switching operation of the track selector 220 according to various embodiments may be an operation of a processor inside the streaming server 120 (eg, the processor 150 of FIG. 1 ) or an operation under the control of the processor 150 . . Hereinafter, the operation of the track selector 220 (the first selector 221 and the second selector 222 ) may be an operation under the control of the processor 150 of FIG. 1 .

The switching operation in FIG. 2 is exemplary and not limited thereto. The switching operation of the track selector 220 may be switching by software or switching by a hardware element.

According to an embodiment, the switching of the first selector 221 and the second selector 222 may be controlled by the same arithmetic element or may be controlled by different arithmetic elements. The track selection unit 220 may operate through communication with each other in different containers/virtualization servers/physical servers according to the method in which the streaming server 120 is installed.

The first selector 221 according to various embodiments of the present disclosure may switch (or select) a track to be streamed in response to a view change request from the terminal device 130 . For example, while streaming the first viewpoint image 250-1, when receiving a viewpoint change request for switching to the N-th viewpoint from the terminal device 130, the first selector 221 controls the streaming track may be switched from a first track through which the first view image 250-1 is transmitted to an N-th track through which the N-th view image 201-N is transmitted.

The second selector 222 according to various embodiments may select one of a plurality of sub-tracks according to a specified condition. For example, when a track is changed by the first selector 221 , a sub-track having the highest I-frame ratio among a plurality of sub-tracks constituting the switched viewpoint image may be selected. Thereafter, the second selector 222 may select a sub-track having a gradually lower I-frame ratio among the plurality of sub-tracks (eg, refer to FIG. 3 ).

The output buffer 180 according to various embodiments may store an image to be transmitted to the terminal device 130 . The output buffer 180 may temporarily store an image transmitted through a track selected by the first selector 221 and the second selector 222 .

3 illustrates a storage and selection operation of a plurality of sub-tracks according to various embodiments of the present disclosure.

Referring to FIG. 3 , the input buffer 170 according to various embodiments may store a multi-viewpoint image. The input buffer 170 may store a plurality of sub-tracks for each viewpoint image 250 - 1 , 250 - 2 , ... 250 -N. The plurality of sub-tracks may have different group of pictures (GOP) structures.

)(250-1-1)은 전체 프레임이 모두 I-프레임으로 구성될 수 있다(I-프레임 비율(RI)=1). 제1 시점 영상(250-1)에 대한 제2 서브 트랙(

)(250-1-2)은 4개의 프레임 당 1개의 I 프레임을 포함하고, 나머지는 P 프레임으로 구성될 수 있다(I-프레임 비율(RI)=1/4). 제1 시점 영상(250-1)에 대한 제3 서브 트랙(

)(250-1-3)은 8개의 프레임 당 하나의 I-프레임을 포함하고, 나머지는 P 프레임으로 구성될 수 있다(I-프레임 비율(RI)=1/8). For example, the first sub-track (

) (250-1-1), the entire frame may consist of all I-frames (I-frame ratio (RI)=1). A second sub-track (for the first viewpoint image 250-1)

) (250-1-2) may include 1 I frame per 4 frames, and the rest may be composed of P frames (I-frame ratio (RI)=1/4). A third sub-track (for the first viewpoint image 250-1)

) (250-1-3) may include one I-frame per 8 frames, and the rest may consist of P frames (I-frame ratio (RI)=1/8).

)(250-N-1)은 전체 프레임이 모두 I-프레임으로 구성될 수 있다(I-프레임 비율(RI)=1). 제N 시점 영상(250-N)에 대한 제2 서브 트랙(

)(250-N-2)은 4개의 프레임 당 4개의 I 프레임을 포함하고, 나머지는 P 프레임으로 구성될 수 있다(I-프레임 비율(RI)=1/4). 제N 시점 영상(250-N)에 대한 제3 서브 트랙(

)(250-N-3)은 8개의 프레임 당 하나의 I-프레임을 포함하고, 나머지는 P 프레임으로 구성될 수 있다(I-프레임 비율(RI)=1/8). As another example, the first sub-track (

) (250-N-1), the entire frame may consist of all I-frames (I-frame ratio (RI)=1). A second sub-track for the N-th view image 250-N (

) (250-N-2) may include 4 I frames per 4 frames, and the rest may consist of P frames (I-frame ratio (RI)=1/4). A third sub-track for the N-th view image 250-N (

) (250-N-3) may include one I-frame per 8 frames, and the rest may consist of P frames (I-frame ratio (RI)=1/8).

According to various embodiments, the processor (eg, the processor 150 of FIG. 1 ) of the streaming server 120 receives the source image received from the source device (eg, the source device 110 of FIG. 1 ) with a different GOP structure. Branches can be converted into a plurality of sub-tracks and stored. Alternatively, the processor 150 may receive and store the source image converted into a plurality of sub-tracks for each viewpoint image from the source device 110 .

According to various embodiments, all of the plurality of sub-tracks for one view image may provide the same image quality, and due to the characteristics of inter-frame compression, the bitrate of each sub-track is the third It may increase in the order of sub-track 250-N-3 < second sub-track 250-N-2 < first sub-track 250-N-1. Accordingly, the network bandwidth required to reproduce an image of the same quality is determined by the ratio of the third sub-track 250-N-3 < the second sub-track 250-N-2 < the first sub-track 250-N-1. may increase in sequence. In the case of efficiently using a limited network bandwidth in order to provide the same streaming service to a plurality of users, a sub-video having a lower I-frame ratio may have a higher network priority. For example, the network priority may be higher in the order of the first sub-track 250-N-1 < the second sub-track 250-N-2 < the third sub-track 250-N-3.

According to various embodiments, each of the plurality of sub-tracks constituting the viewpoint image may be configured by successive video frames. When receiving a view change request from the terminal device 130 while an image is being streamed, the processor 150 (eg, the first selector 221 of FIG. 2 ) performs the streaming first view image 250-1 , may switch to the N-th view image (or track) 250 -N corresponding to the view change request. The processor 150 (eg, the second selector 222 of FIG. 2 ) may select one of a plurality of sub tracks (or sub images) according to a specified condition.

According to various embodiments, at the first time when the viewpoint image is switched (or the time when the processor 150 performs switching of the viewpoint image), the types of frames of each sub-track may be different from each other. For example, when the first view image 250-1 being streamed at time t0 is switched to the N-th view image 250-N, the first to third views constituting the N-th view image 250-N Frames of the sub tracks 250-N-1, 250-N-2, and 250-N-3) may be I-frames, P-frames, and P-frames, respectively.

According to various embodiments, at a first time (eg, t0) when a viewpoint image is switched, the processor 150 (eg, the second selector 222 of FIG. 2 ) selects a sub-track to be streamed with an I-frame. Streaming may be started by determining the first sub-track 250-N-1. In a comparative example, the second sub-track 250-N-2 or the third sub-track 250-N in which the terminal device (eg, the terminal device 130 of FIG. 1 ) has a P-frame from the streaming server 120 . When -3) is received, the terminal device 130 may not output a normal screen because it does not have a reference frame capable of decoding the received frame. On the other hand, in an embodiment of the present invention, when the first sub-track 250-N-1 having an I-frame is primarily transmitted, a reference frame may not be required and the terminal device ( 130) may provide a normal screen to the user.

According to various embodiments, when a viewpoint image is maintained at a second time (eg, t1) when the second sub-track 250 -N-2 has an I-frame (when a separate viewpoint switching request is not received) , the processor 150 (eg, the second selector 222 of FIG. 2 ) may select and stream the second sub-track 250-N-2. Similarly, when a viewpoint image is maintained at a third time (eg, t5) when the third sub-track 250-N-3 has an I-frame, the processor 150 (eg, the second selector 222) ) can be streamed by selecting the third sub-track 250-N-3. By sequential switching of the processor 150 (eg, the second selector 222 ), the ratio of I-frames of the streaming sub-track may be gradually lowered, and network efficiency may be increased.

According to various embodiments, the processor 150 may quickly set a change time between sub-tracks according to network conditions. For example, when the communication quality of the network is less than or equal to a specified value, the processor 150 (eg, the second selector 222 of FIG. 2 ) determines that the second sub-track 250-N-2 has an I-frame. At the second time (eg, t1), the second sub-track 250-N-2 may be selected and streamed. For another example, when the communication quality of the network exceeds a specified value, the processor 150 (eg, the second selector 222 of FIG. 2 ) determines that the second sub-track 250-N-2 is displayed for the second time. It is possible to select and stream the second sub-track 250-N-2 at a time other than (eg, t1).

)(250-1-3)(I-프레임 비율(RI)=1/8)으로 스트리밍 하는 중, 시점 전환 요청에 의해 제N 시점 영상(250-N)에 대한 제1 서브 트랙(

)(250-N-1)(I-프레임 비율(RI)=1)로 전환되는 경우, 프로세서(150)는 화질 변경이 발생할 수 있다는 메시지를 단말 장치(130)에 제공할 수 있다.According to various embodiments, the processor 150 may provide related information to the terminal device 130 when the frame structure before and after switching the viewpoint is changed. For example, the third sub-track (

) (250-1-3) (I-frame rate (RI) = 1/8) during streaming, the first sub-track (

) (250-N-1) (I-frame rate (RI) = 1), the processor 150 may provide a message indicating that the quality change may occur to the terminal device 130 .

According to various embodiments, the processor 150 may set a section limiting the change between sub-tracks according to a specified condition. The condition may be based on information about the user's history related to viewpoint switching, or information previously analyzed related to viewpoint switching.

4 shows a video streaming method performed by a streaming server according to various embodiments.

Referring to FIG. 4 , according to various embodiments, in operation 410 , the processor of the streaming server 120 (eg, the processor 150 of FIG. 1 ) performs a plurality of sub-tracks constituting a first view image in a multi-view image. One of them may be used to stream to the terminal device 130 . For example, the processor may be in progress streaming to the sub-track having the lowest ratio of I-frames among the plurality of sub-tracks.

According to various embodiments, the processor 150 may perform streaming through an operation of enqueueing the frame to the output buffer 180 after dequeueing the corresponding frame in the input buffer 170 .

According to various embodiments, in operation 420 , the processor 150 of the streaming server 120 may determine whether a viewpoint switching request is received from the terminal device 130 . When the view switching request is not received (No in operation 420 ), the processor 150 may maintain the streaming state or may continue streaming by changing to a sub-track having a higher network priority.

According to various embodiments, in operation 430, when the processor of the streaming server 120 receives a view change request from the terminal device (Yes in operation 420), a first time to switch to a second view image corresponding to the request For example, a sub-track including a first type frame (eg, an I-frame) among the second view image may be streamed.

For example, in the example of FIG. 3 , the first to third subs that are switched from the streaming view image to the N-th view image 250-N, and that constitute the N-th view image 250-N at the time of switching Among the tracks 250-N-1, 250-N-2, and 250-N-3, when only the first sub-track 250-N-1 includes an I-frame, the processor 150 performs the first sub-track Track 250-N-1 may be streamed.

As another example, the first to third sub-tracks 250-N that are switched from the streaming view image to the N-th view image 250-N, and that constitute the N-th view image 250-N at the time of conversion -1, 250-N-2, and 250-N-3), when both the first sub-track 250-N-1 and the second sub-track 250-N-2 include I-frames, The processor 150 may stream the second sub-track 250 -N-2 having a high network priority.

As another example, the first to third sub-tracks 250- that are switched from the streaming view image to the N-th view image 250-N, and constitute the N-th view image 250-N at the time of the conversion. When all of N-1, 250-N-2, and 250-N-3) include I-frames, the processor 150 streams the third sub-track 250-N-3 having the highest network priority. can

According to various embodiments, the processor 150 may select a sub-track by reflecting a network situation at a first time when the first-view image is switched to the second-view image. For example, when the number of terminal devices 130 receiving streaming data is drastically reduced or when the bandwidth allocated to the network is expanded, a sub-track having a relatively high I-frame ratio can be selected and streamed. .

5 illustrates a dynamic change of a frame structure over time according to various embodiments of the present disclosure;

Referring to FIG. 5 , an input buffer (eg, the input buffer 170 of FIG. ) according to various embodiments may store a multi-view image. The input buffer 170 may store a plurality of sub-tracks 550-1-1 to 550-1-3 for the viewpoint image 550-1 corresponding to one viewpoint.

According to various embodiments, the plurality of sub-tracks 550-1-1 to 550-1-3 may have different group of pictures (GOP) structures. For example, the plurality of sub-tracks 550-1-1 to 550-1-3 may be images having different I-frame rates.

According to various embodiments, the processor (eg, the processor 150 of FIG. 1 ) of the streaming server 120 is I- of a plurality of sub-tracks (550-1-1 to 550-1-3) according to a specified condition. The frame rate (RI) can be dynamically changed. For example, the processor 150 may increase or decrease the I-frame rate RI according to a relationship between a computing resource or network bandwidth and the number of users using a service. Alternatively, the processor 150 may change the I-frame rate RI according to the user's track switching pattern or the attribute (or characteristic) of the image to be transmitted.

)(550-1-1)은 모두 I-프레임으로 구성될 수 있다(I-프레임 비율(RI)=1). 제1 시점 영상(550-1)에 대한 제2 서브 트랙(

)(550-1-2)은 8개의 프레임 당 2개의 I 프레임을 포함하고, 나머지는 P 프레임으로 구성될 수 있다(I-프레임 비율(RI)=1/4). 제1 시점 영상(550-1)에 대한 제3 서브 트랙(

)(550-1-3)은 8개의 프레임 당 하나의 I-프레임을 포함하고, 나머지는 P 프레임으로 구성될 수 있다(I-프레임 비율(RI)=1/8). According to various embodiments, in the first time period T1, the first sub-track (

) (550-1-1) may all be composed of I-frames (I-frame ratio (RI)=1). A second sub-track (for the first viewpoint image 550-1)

) (550-1-2) includes 2 I frames per 8 frames, and the rest may be composed of P frames (I-frame ratio (RI)=1/4). A third sub-track (for the first viewpoint image 550-1)

) (550-1-3) may include one I-frame per 8 frames, and the rest may consist of P frames (I-frame ratio (RI)=1/8).

)(550-1-1)은 계속적으로 모두 I-프레임으로 구성될 수 있다(I-프레임 비율(RI)=1). 제1 시점 영상(550-1)에 대한 제2 서브 트랙(

)(550-1-2)은 8개의 프레임 당 4개의 I 프레임을 포함하고, 나머지는 P 프레임으로 구성될 수 있다(I-프레임 비율(RI)=1/2). 제1 시점 영상(550-1)에 대한 제3 서브 트랙(

)(550-1-3)은 8개의 프레임 당 2개의 I-프레임을 포함하고, 나머지는 P 프레임으로 구성될 수 있다(I-프레임 비율(RI)=1/4). 예를 들어, 제2 시간 구간(T2)에서 사용자의 트랙 전환 요청이 빈번할 것으로 예측되는 경우, 프로세서(150)는 네트워크 우선 순위가 상대적으로 높은 서브 트랙에 대한 I-프레임 비율을 높여, 사용자의 연속된 트랙 전환 요청에 빠르게 대응할 수 있다.According to various embodiments, in the second time section T2 after the first time section T1, the first sub-track (

) (550-1-1) may continuously consist of all I-frames (I-frame ratio (RI)=1). A second sub-track (for the first viewpoint image 550-1)

) (550-1-2) includes 4 I frames per 8 frames, and the rest may be composed of P frames (I-frame ratio (RI)=1/2). A third sub-track (for the first viewpoint image 550-1)

) (550-1-3) may include 2 I-frames per 8 frames, and the rest may consist of P frames (I-frame ratio (RI)=1/4). For example, if it is predicted that the user's request for track switching is frequent in the second time period T2, the processor 150 increases the I-frame rate for the sub-track having a relatively high network priority, so that the user's It can quickly respond to continuous track switching requests.

According to various embodiments, in the third time period T3 after the second time period T2, the same I-frame rate as the first time period T1 may be maintained. For example, when it is predicted that the frequency of the track change request becomes lower again, the processor 150 sets the I-frame rate of the plurality of sub-tracks 550-1-1 to 550-1-3 to the previous first. It is possible to return to the state in the time period T1.

According to various embodiments, the processor 150 determines at least one of a network state, the number of terminal devices 130 receiving a streaming service, whether a user frequently requests a view change, and a characteristic of a view image to be provided. Accordingly, the I-frame rate can be dynamically changed.

6 shows a video streaming method through transmission of frame structure information according to various embodiments. Referring to FIG. 6 , a processor (eg, the processor 150 of FIG. 1 ) of the streaming server 120 according to various embodiments. may transmit information (hereinafter, frame structure information) 610 about the frame structure of the multi-view image to the terminal device 130 . The frame structure information 610 for the multi-view image may be provided through the same channel as the channel providing the streaming service to the terminal device 130 or through a different channel.

According to various embodiments, the frame structure information 610 may include information on the entire multi-view image or information on a part of the multi-view image. For example, in a state of receiving the first view image, the terminal device 130 may receive the frame structure information 610 in the second view image with a high possibility of switching.

According to various embodiments, the processor of the terminal device 130 (eg, the processor 720 of FIG. 7 ) may effectively respond to the user's request for changing the viewpoint by using the received frame structure information 610 . For example, the time it takes for the actual track to change from the time the user generates an input related to viewpoint switching (e.g., touch input, voice input) (hereinafter, reaction time (d_react)) may be an important quality factor for streaming services. have. The reaction time d_react may be managed to maintain a specified value or less, and the reaction time d_react may be changed according to a service or content type.

According to various embodiments, the processor of the terminal device 130 (eg, the processor 720 of FIG. 7 ) may change the actual track change time according to the value of the reaction time d_react. For example, when a time point change request occurs at time t0 and the reaction time d_react is set to 1, the terminal device 130 may request that the track be switched at time t1. As another example, when a time point change request occurs at time t0 and the reaction time d_react is set to 2, the terminal device 130 may request that the track be switched at time t2. In this case, the second sub-track may also include I-frames, so that the processor of the terminal device 130 (eg, the processor 720 of FIG. 7 ) directly switches to the second sub-track without switching to the first sub-track. can be switched As another example, when a time point change request occurs at time t0 and the reaction time d_react is set to 4, the terminal device 130 may request that the track be switched at time t4. In this case, the third sub-track may also include an I-frame, so that the processor of the terminal device 130 (eg, the processor 720 of FIG. 7 ) can immediately switch to the first sub-track or the second sub-track without switching to the first sub-track or the second sub-track. You can switch to the 3rd sub-track. As another example, when a time point change request occurs at time t0 and the reaction time d_react is set to 8, the terminal device 130 may request that the track be switched at time t8. In this case, the fourth sub-track may also include an I-frame, so that the processor of the terminal device 130 (eg, the processor 720 of FIG. 7 ) performs the first sub-track, the second sub-track, or the third sub-track. You can switch directly to the 4th sub-track without switching to . The processor of the terminal device 130 (eg, the processor 720 of FIG. 7 ) may select a sub-track having a high network priority, and thus communication efficiency may be improved.

According to various embodiments, when an input for changing a user's viewpoint occurs, the processor (eg, the processor 720 of FIG. 7 ) of the terminal device 130 , a time designated in response to a set value of the reaction time d_react After waiting, it is possible to transmit a request for switching the viewpoint to the streaming server 120 .

According to various embodiments, the streaming server 120 may transmit a multi-view image using various transport protocols such as a real-time streaming protocol (eg, RTSP, RTMP) or an adaptive streaming protocol (eg, HLS, DASH).

According to various embodiments, in the case of a real-time streaming protocol (eg, RTSP, RTMP), the streaming server 120 may store the state of the terminal device 130 and transmit an image in units of frames. The streaming server 120 may manage information on the track being played by the terminal device 130 connected to the streaming service. When a viewpoint switching request occurs, the streaming server 120 may select and transmit a track to be transmitted according to the characteristics of each frame of the multi-track. The view change request may be transmitted in the form of a message of a protocol for video transmission, or may be transmitted in another form. When the frame structure information 610 is used by the terminal device 130 , the streaming server 120 may delay the view switching request or attempt to directly switch to an appropriate track.

According to various embodiments, in the case of an adaptive streaming protocol (eg, HLS, DASH), the streaming server 120 may transmit a manifest file that is frame structure information 610 regarding a multi-view image to the terminal device 130 . . For example, in the case of DASH, the address of all multi-tracks may be transmitted in xml format in Media Presentation Description (MPD). Upon receiving the manifest file, the terminal device 130 may request and reproduce a current segment of a decodable track.

According to various embodiments, the terminal device 130 may directly control the track selection. When a view change request occurs, the terminal device 130 may select a plurality of tracks to be played during the track change and may request segments of the tracks. When the user quickly requests for continuous track switching, the terminal device 130 may select a discontinuous track by selecting an appropriate speed.

According to various embodiments, the streaming server 120 may store information about the reaction time (d_react) in the terminal device 130 in advance. In this case, the streaming server 120 may switch the viewpoint image after waiting a specified time when the viewpoint switching request occurs in consideration of the reaction time (d_react) in the terminal device 130 .

According to various embodiments, the streaming server 120 may receive viewpoint images from different source devices 120 . For example, the streaming server 120 receives, among the first to fifteenth view images constituting the multi-view image, first to fifth view images from the first source device, and the sixth to fifteenth view images 2 Receive from the source device.

According to various embodiments, the streaming server 120 or the terminal device 130 is a mobile edge computing (MEC) server inside a base station that supports a network for some images in a section in which a viewpoint change occurs frequently or is highly likely to occur among streaming images. can be requested to be stored in . When a view point switching request occurs, the terminal device 130 may display some images stored in the MEC server by first being transmitted from the MEC server. The terminal device 130 may receive and display an image after the image stored in the MEC server from the streaming server 120 . The streaming server 120 may transmit an image after the image stored in the MEC server to the terminal device 130 .

According to various embodiments, the streaming server 120 or the terminal device 130 may request each source device 130 to prefetch data for a section in which switching occurs or an expected section. . In the previous example, when the possibility of switching from the fifth viewpoint image received from the first source device to the tenth viewpoint image received from the second source device is high, the streaming server 120 or the terminal device 130 When the 5-view image is being streamed, the second source device may request to increase the prefetched data of the 10-view image.

7 is a block diagram of an electronic device 701 in a network environment 700 according to various embodiments of the present disclosure. Electronic devices according to various embodiments disclosed in this document may be devices of various types. Electronic devices include, for example, portable communication devices (eg, smartphones), computer devices (eg, personal digital assistants), tablet PCs (tablet PCs), laptop PCs (desktop PCs, workstations, or servers); It may include at least one of a portable multimedia device (eg, an e-book reader or an MP3 player), a portable medical device (eg, a heart rate, blood sugar, blood pressure, or body temperature monitor), a camera, or a wearable device. (e.g. watches, rings, bracelets, anklets, necklaces, eyeglasses, contact lenses, or head-mounted-devices (HMDs)); : skin pad or tattoo), or a bioimplantable circuit.In some embodiments, the electronic device is, for example, a television, a digital video disk (DVD) player, an audio device, an audio accessory device. (e.g. speakers, headphones, or headsets), refrigerators, air conditioners, vacuum cleaners, ovens, microwaves, washing machines, air purifiers, set-top boxes, home automation control panels, security control panels, game consoles, electronic dictionaries, electronic keys, camcorders , or at least one of an electronic picture frame.

In another embodiment, the electronic device may include a navigation device, a global navigation satellite system (GNSS), an event data recorder (EDR) (eg, a black box for a vehicle/vessel/airplane), and an automotive infotainment device. (e.g. head-up displays for vehicles), industrial or domestic robots, drones, automated teller machines (ATMs), point of sales (POS) instruments, metering instruments (e.g. water, electricity, or gas metering instruments); Alternatively, it may include at least one of an IoT device (eg, a light bulb, a sprinkler device, a fire alarm, a thermostat, or a street lamp). The electronic device according to the embodiment of this document is not limited to the above-described devices, and, for example, as in the case of a smartphone equipped with a function of measuring personal biometric information (eg, heart rate or blood sugar), a plurality of electronic devices The functions of the devices may be provided in a complex manner. In this document, the term user may refer to a person who uses an electronic device or a device (eg, an artificial intelligence electronic device) using the electronic device.

Referring to FIG. 7 , in a network environment 700 , the electronic device 701 communicates with the electronic device 702 through a first network 798 (eg, a short-range wireless communication network) or a second network 799 . It may communicate with the electronic device 704 or the server 708 through (eg, a remote wireless communication network). According to an embodiment, the electronic device 701 may communicate with the electronic device 704 through the server 708 . According to an embodiment, the electronic device 701 includes a processor 720 , a memory 730 , an input device 750 , a sound output device 755 , a display device 760 , an audio module 770 , and a sensor module ( 776 , interface 777 , haptic module 779 , camera module 780 , power management module 788 , battery 789 , communication module 790 , subscriber identification module 796 , or antenna module 797 . ) may be included. In some embodiments, at least one of these components (eg, the display device 760 or the camera module 780 ) may be omitted or one or more other components may be added to the electronic device 701 . In some embodiments, some of these components may be implemented as one integrated circuit. For example, the sensor module 776 (eg, a fingerprint sensor, an iris sensor, or an illuminance sensor) may be implemented while being embedded in the display device 760 (eg, a display).

The processor 720, for example, executes software (eg, a program 740) to execute at least one other component (eg, a hardware or software component) of the electronic device 701 connected to the processor 720 . It can control and perform various data processing or operations. According to one embodiment, as at least part of data processing or computation, the processor 720 converts commands or data received from other components (eg, the sensor module 776 or the communication module 790 ) to the volatile memory 732 . may load into the volatile memory 732 , process the commands or data stored in the volatile memory 732 , and store the resulting data in the non-volatile memory 734 . According to an embodiment, the processor 720 includes a main processor 721 (eg, a central processing unit or an application processor), and a secondary processor 723 (eg, a graphics processing unit, an image signal processor) that can be operated independently or in conjunction with the main processor 721 (eg, a graphics processing unit or an image signal processor). , a sensor hub processor, or a communication processor). Additionally or alternatively, the auxiliary processor 723 may be configured to use less power than the main processor 721 or to be specialized for a designated function. The coprocessor 723 may be implemented separately from or as part of the main processor 721 .

The coprocessor 723 may, for example, act on behalf of the main processor 721 while the main processor 721 is in an inactive (eg, sleep) state, or when the main processor 721 is active (eg, executing an application). ), together with the main processor 721, at least one of the components of the electronic device 701 (eg, the display device 760, the sensor module 776, or the communication module 790) It is possible to control at least some of the related functions or states. According to one embodiment, coprocessor 723 (eg, image signal processor or communication processor) may be implemented as part of another functionally related component (eg, camera module 780 or communication module 790 ). have.

The memory 730 may store various data used by at least one component (eg, the processor 720 or the sensor module 776 ) of the electronic device 701 . The data may include, for example, input data or output data for software (eg, the program 740 ) and instructions related thereto. The memory 730 may include a volatile memory 732 or a non-volatile memory 734 .

The program 740 may be stored as software in the memory 730 , and may include, for example, an operating system 742 , middleware 744 , or an application 746 .

The input device 750 may receive a command or data to be used by a component (eg, the processor 720 ) of the electronic device 701 from the outside (eg, a user) of the electronic device 701 . The input device 750 may include, for example, a microphone, a mouse, a keyboard, or a digital pen (eg, a stylus pen).

The sound output device 755 may output a sound signal to the outside of the electronic device 701 . The sound output device 755 may include, for example, a speaker or a receiver. The speaker can be used for general purposes such as multimedia playback or recording playback, and the receiver can be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from or as part of the speaker.

The display device 760 may visually provide information to the outside (eg, a user) of the electronic device 701 . The display device 760 may include, for example, a display, a hologram device, or a projector and a control circuit for controlling the corresponding device. According to an embodiment, the display device 760 may include a touch circuitry configured to sense a touch or a sensor circuit (eg, a pressure sensor) configured to measure the intensity of a force generated by the touch. have.

The audio module 770 may convert a sound into an electric signal or, conversely, convert an electric signal into a sound. According to an embodiment, the audio module 770 obtains a sound through the input device 750 or an external electronic device (eg, a sound output device 755 ) directly or wirelessly connected to the electronic device 701 . The electronic device 702) (eg, a speaker or headphones) may output sound.

The sensor module 776 detects an operating state (eg, power or temperature) of the electronic device 701 or an external environmental state (eg, a user state), and generates an electrical signal or data value corresponding to the sensed state. can do. According to an embodiment, the sensor module 776 may include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, It may include a temperature sensor, a humidity sensor, or an illuminance sensor.

The interface 777 may support one or more specified protocols that may be used for the electronic device 701 to be directly or wirelessly connected to an external electronic device (eg, the electronic device 702 ). According to an embodiment, the interface 777 may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

The connection terminal 778 may include a connector through which the electronic device 701 can be physically connected to an external electronic device (eg, the electronic device 702 ). According to an embodiment, the connection terminal 778 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (eg, a headphone connector).

The haptic module 779 may convert an electrical signal into a mechanical stimulus (eg, vibration or movement) or an electrical stimulus that the user can perceive through tactile or kinesthetic sense. According to an embodiment, the haptic module 779 may include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

The camera module 780 may capture still images and moving images. According to one embodiment, the camera module 780 may include one or more lenses, image sensors, image signal processors, or flashes.

The power management module 788 may manage power supplied to the electronic device 701 . According to an embodiment, the power management module 788 may be implemented as at least part of a power management integrated circuit (PMIC), for example.

The battery 789 may supply power to at least one component of the electronic device 701 . According to one embodiment, battery 789 may include, for example, a non-rechargeable primary cell, a rechargeable secondary cell, or a fuel cell.

The communication module 790 is a direct (eg, wired) communication channel or a wireless communication channel between the electronic device 701 and an external electronic device (eg, the electronic device 702 , the electronic device 704 , or the server 708 ). It can support establishment and communication through the established communication channel. The communication module 790 may include one or more communication processors that operate independently of the processor 720 (eg, an application processor) and support direct (eg, wired) communication or wireless communication. According to one embodiment, the communication module 790 is a wireless communication module 792 (eg, a cellular communication module, a short-range communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module 794 (eg, : It may include a local area network (LAN) communication module, or a power line communication module). Among these communication modules, a corresponding communication module is a first network 798 (eg, a short-range communication network such as Bluetooth, WiFi direct, or infrared data association (IrDA)) or a second network 799 (eg, a cellular network, the Internet, or It may communicate with the external electronic device 704 through a computer network (eg, a telecommunication network such as a LAN or WAN). These various types of communication modules may be integrated into one component (eg, a single chip) or may be implemented as a plurality of components (eg, multiple chips) separate from each other. The wireless communication module 792 uses the subscriber information (eg, International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module 796 within a communication network, such as the first network 798 or the second network 799 . The electronic device 701 may be identified and authenticated.

The antenna module 797 may transmit or receive a signal or power to the outside (eg, an external electronic device). According to an embodiment, the antenna module 797 may include one antenna including a conductor formed on a substrate (eg, a PCB) or a radiator formed of a conductive pattern. According to an embodiment, the antenna module 797 may include a plurality of antennas. In this case, at least one antenna suitable for a communication scheme used in a communication network such as the first network 798 or the second network 799 is connected from the plurality of antennas by, for example, the communication module 790 . can be selected. A signal or power may be transmitted or received between the communication module 790 and the external electronic device through the selected at least one antenna. According to some embodiments, other components (eg, RFIC) other than the radiator may be additionally formed as a part of the antenna module 797 .

At least some of the components are connected to each other through a communication method between peripheral devices (eg, a bus, general purpose input and output (GPIO), serial peripheral interface (SPI), or mobile industry processor interface (MIPI)) and a signal ( e.g. commands or data) can be exchanged with each other.

According to an embodiment, the command or data may be transmitted or received between the electronic device 701 and the external electronic device 704 through the server 708 connected to the second network 799 . Each of the external electronic devices 702 and 1204 may be the same as or different from the electronic device 701 . According to an embodiment, all or part of the operations performed by the electronic device 701 may be executed by one or more of the external electronic devices 702 , 704 , or 708 . For example, when the electronic device 701 needs to perform a function or service automatically or in response to a request from a user or other device, the electronic device 701 may instead of executing the function or service itself. Alternatively or additionally, one or more external electronic devices may be requested to perform at least a part of the function or the service. The one or more external electronic devices that have received the request may execute at least a part of the requested function or service, or an additional function or service related to the request, and transmit a result of the execution to the electronic device 701 . The electronic device 701 may process the result as it is or additionally and provide it as at least a part of a response to the request. For this purpose, for example, cloud computing, distributed computing, or client-server computing technology may be used.

An electronic device (eg, the streaming server 120 of FIG. 1 , the server 708 of FIG. 7 ) according to various embodiments of the present disclosure is an external device (eg, the terminal device 130 of FIG. 1 , the electronic device 701 of FIG. 7 ) ), a communication circuit (eg, the communication circuit 190 of FIG. 1 ), a memory that stores a plurality of images corresponding to a plurality of viewpoints (eg, the memory 160 of FIG. 1 ), and a processor ( Example: the processor 150 of FIG. 1 ), wherein each of the plurality of images consists of a plurality of sub-tracks, each of the plurality of sub-tracks consists of successive frames, and each of the frames has one attribute of a first type frame or a second type frame, the plurality of sub-tracks have different ratios of the first type frame among all frames, and the processor (eg, the processor 150 in FIG. 1 ) transmits one of a plurality of sub-tracks of a first image for a first viewpoint among the plurality of images to the external device (eg, the terminal device 130 of FIG. 1 , the electronic device 701 of FIG. 7 ) and, through the communication circuit (eg, the communication circuit 190 of FIG. 1 ), a request for viewpoint switching from the external device (eg, the terminal device 130 of FIG. 1 , the electronic device 701 of FIG. 7 ) and, at a first time, transmit a first sub-track including the first type frame among a plurality of sub-tracks of a second image for a second time point corresponding to the request to the external device (eg, FIG. 1 ). may be transmitted to the terminal device 130 of , the electronic device 701 of FIG. 7 ).

According to various embodiments, the first sub-track may be a sub-track having the highest rate of inclusion of the first type frame among a plurality of sub-tracks of the second image.

According to various embodiments, the first sub-track may be an image in which all frames are composed of the first type frame.

According to various embodiments, the first time is a time for switching to the second image in response to the request, and the processor (eg, the processor 150 of FIG. 1 ) is a second time after the first time. For example, the second image may be changed to a second sub-track including the first type frame among the plurality of sub-tracks and transmitted.

According to various embodiments, the second sub-track may have a lower proportion of the first type frame than the first sub-track.

According to various embodiments, the second sub-track may have a higher image compression ratio than the first sub-track.

According to various embodiments, the second sub-track may have a higher network priority than the first sub-track.

According to various embodiments, after the first time, the processor (eg, the processor 150 of FIG. 1 ) selects a sub-track having a low ratio of the first type frame among the plurality of sub-tracks of the second image. It can be sequentially changed and transmitted.

According to various embodiments, the processor (eg, the processor 150 of FIG. 1 ) is the external device (eg, the terminal device 130 of FIG. 1 , the electronic device 701 of FIG. 7 ) before the first time information on the frame configuration of the first image or the second image may be transmitted to the .

According to various embodiments, the processor (eg, the processor 150 of FIG. 1 ) may change the frame configuration of the plurality of sub-tracks of the second image according to time.

According to various embodiments, the processor (eg, the processor 150 of FIG. 1 ) may include a network condition, the number of the external devices (eg, the terminal device 130 of FIG. 1 , the electronic device 701 of FIG. 7 ); The frame configuration may be changed based on one of the properties of the first image or the second image.

According to various embodiments, the first image may have the same frame configuration as the second image.

According to various embodiments, the processor (eg, the processor 150 of FIG. 1 ) may receive the request, wait a specified time, and then switch the image in response to the request.

According to various embodiments, the first type frame may be an independently decodable frame, and the second type frame may be a decodable frame using the second type frame.

According to various embodiments, the processor (eg, the processor 150 of FIG. 1 ) may include a first switch for switching corresponding to the request and a first switch for selecting one of the plurality of sub-tracks of the second image. 2 switches can be controlled.

According to various embodiments, when all of the plurality of sub-tracks of the second image are of the second type frame at the first time, the processor (eg, the processor 150 of FIG. 1 ) is configured to control the plurality of sub-tracks. It may wait until at least one of them is changed to the first type frame.

An image transmission method according to various embodiments includes an electronic device (eg, the electronic device of FIG. 1 ) that transmits one of a plurality of images to an external device (eg, the terminal device 130 of FIG. 1 and the electronic device 701 of FIG. 7 ). Streaming server 120, performed in the server 708 of FIG. 7), one of the plurality of sub-tracks of the first image for the first view among the plurality of images to the external device (eg, the terminal of FIG. 1) The operation of transmitting to the device 130 (the electronic device 701 of FIG. 7 ), each of the plurality of sub-tracks is composed of consecutive frames, and each of the frames is one of a first type frame or a second type frame The plurality of sub-tracks have one property, the ratio of the first type frame among the entire frame is different from each other, and the external device (eg, the terminal device 130 of FIG. 1 , the electronic device 701 of FIG. 7 ) a plurality of sub-tracks of the second image in the operation of receiving a request for viewpoint switching from Among them, the operation may include transmitting the first sub-track including the first type frame to the external device (eg, the terminal device 130 of FIG. 1 , the electronic device 701 of FIG. 7 ).

According to various embodiments, the operation of transmitting the first sub-track may include the operation of transmitting the sub-track having the highest rate of inclusion of the first type frame among a plurality of sub-tracks of the second image.

According to various embodiments, the operation of transmitting the first sub-track may include an operation of transmitting the first sub-track in which all frames are configured as the first type frame.

According to various embodiments of the present disclosure, the method of transmitting the image may further include the operation of changing and transmitting the second image to a second sub-track including the first type frame among the plurality of sub-tracks.

The electronic device according to various embodiments disclosed in this document may have various types of devices. The electronic device may include, for example, a portable communication device (eg, a smart phone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a home appliance device. The electronic device according to the embodiment of the present document is not limited to the above-described devices.

It should be understood that the various embodiments of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, and include various modifications, equivalents, or substitutions of the embodiments. In connection with the description of the drawings, like reference numerals may be used for similar or related components. The singular form of the noun corresponding to the item may include one or more of the item, unless the relevant context clearly dictates otherwise. As used herein, “A or B”, “at least one of A and B”, “at least one of A or B”, “A, B or C”, “at least one of A, B and C”, and “A , B, or C" each may include any one of the items listed together in the corresponding one of the phrases, or all possible combinations thereof. Terms such as “first”, “second”, or “first” or “second” may simply be used to distinguish the component from other components in question, and may refer to components in other aspects (e.g., importance or order) is not limited. that one (eg first) component is “coupled” or “connected” to another (eg, second) component with or without the terms “functionally” or “communicatively” When referenced, it means that one component can be connected to the other component directly (eg by wire), wirelessly, or through a third component.

As used herein, the term “module” may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as, for example, logic, logic block, component, or circuit. A module may be an integrally formed part or a minimum unit or a part of the part that performs one or more functions. For example, according to an embodiment, the module may be implemented in the form of an application-specific integrated circuit (ASIC).

Various embodiments of the present document include one or more instructions stored in a storage medium (eg, internal memory 736 or external memory 738) readable by a machine (eg, electronic device 701). may be implemented as software (eg, the program 740) including For example, a processor (eg, processor 720 ) of a device (eg, electronic device 701 ) may call at least one of one or more instructions stored from a storage medium and execute it. This makes it possible for the device to be operated to perform at least one function according to the at least one command called. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' only means that the storage medium is a tangible device and does not contain a signal (eg, electromagnetic wave), and this term refers to the case where data is semi-permanently stored in the storage medium and It does not distinguish between temporary storage cases.

According to one embodiment, the method according to various embodiments disclosed in this document may be provided in a computer program product (computer program product). Computer program products may be traded between sellers and buyers as commodities. The computer program product is distributed in the form of a machine-readable storage medium (eg compact disc read only memory (CD-ROM)), or via an application store (eg Play Store ^TM ) or on two user devices ( It can be distributed (eg downloaded or uploaded) directly or online between smartphones (eg: smartphones). In the case of online distribution, at least a part of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium such as a memory of a server of a manufacturer, a server of an application store, or a relay server.

According to various embodiments, each component (eg, a module or a program) of the above-described components may include a singular or a plurality of entities. According to various embodiments, one or more components or operations among the above-described corresponding components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (eg, a module or a program) may be integrated into one component. In this case, the integrated component may perform one or more functions of each component of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. . According to various embodiments, operations performed by a module, program, or other component are executed sequentially, in parallel, repeatedly, or heuristically, or one or more of the operations are executed in a different order, or omitted. or one or more other operations may be added.

Claims (20)

An electronic device for transmitting an image to an external device, the electronic device comprising:
communication circuit;
a memory for storing a plurality of images corresponding to a plurality of viewpoints; and
processor; including;
Each of the plurality of images is composed of a plurality of sub-tracks,
each of the plurality of sub-tracks consists of consecutive frames, each of which has an attribute of one of a first type frame or a second type frame;
The plurality of sub-tracks have different ratios of the first type frame among all frames,
the processor
transmitting one of a plurality of sub-tracks of a first image for a first viewpoint among the plurality of images to the external device;
receiving a request for changing a viewpoint from the external device through the communication circuit;
The electronic device transmits, to the external device, a first sub-track including the first type frame among a plurality of sub-tracks of a second image for a second time point corresponding to the request at a first time. The method of claim 1, wherein the first sub-track is
an electronic device, which is a sub-track having the highest rate of inclusion of the first type frame among a plurality of sub-tracks of the second image. The method of claim 1, wherein the first sub-track is
An electronic device in which all frames are images composed of the first type frame. According to claim 1, wherein the first time,
It is the time to switch to the second image in response to the request,
the processor
The electronic device transmits a second sub-track including the first type frame among the plurality of sub-tracks of the second image at a second time after the first time. 5. The method of claim 4, wherein the second sub-track is
An electronic device having a lower inclusion ratio of the first type frame than the first sub-track. 5. The method of claim 4, wherein the second sub-track is
An electronic device having a higher image compression ratio than the first sub-track. 5. The method of claim 4, wherein the second sub-track is
An electronic device having a higher network priority than the first sub-track. The method of claim 1, wherein the processor is
After the first time, the electronic device sequentially changes and transmits a sub-track having a low ratio of the first type frame among the plurality of sub-tracks of the second image. The method of claim 1, wherein the processor is
The electronic device transmits information on a frame configuration of the first image or the second image to the external device before the first time. The method of claim 1, wherein the processor is
An electronic device for changing a frame configuration of the plurality of sub-tracks of the second image according to time. 10. The method of claim 9, wherein the processor
An electronic device for changing the frame configuration based on one of a network situation, the number of external devices, and an attribute of the first image or the second image. The method of claim 1, wherein the first image is
An electronic device having the same frame configuration as the second image. The method of claim 1, wherein the processor is
After receiving the request and waiting for a specified time, the electronic device switches an image in response to the request. The method of claim 1, wherein the first type frame comprises:
The electronic device is a frame that can be independently decoded, and is a frame that can be decoded using the second type frame by using the second type frame. The method of claim 1, wherein the processor is
a first switch for switching in response to the request;
An electronic device for controlling a second switch for selecting one of the plurality of sub-tracks of the second image. The method of claim 1, wherein the processor is
In the first time, when all of the plurality of sub-tracks of the second image are second-type frames, the electronic device waits until at least one of the plurality of sub-tracks is changed to the first-type frame. An image transmission method performed in an electronic device for transmitting one of a plurality of images to an external device, the method comprising:
An operation of transmitting one of a plurality of sub-tracks of a first image for a first viewpoint among a plurality of images to the external device, each of the plurality of sub-tracks being constituted by successive frames, each of the frames has one property of a first type frame or a second type frame, and the plurality of sub-tracks have different ratios of the first type frame among all frames;
receiving a request for changing a viewpoint from the external device;
switching to a second image for a second viewpoint corresponding to the request; and
and transmitting a first sub-track including the first type frame among a plurality of sub-tracks of the second image to the external device at a first time when the second image is switched. 18. The method of claim 17, wherein the transmitting of the first sub-track comprises:
and transmitting a sub-track having the highest rate of inclusion of the first type frame among a plurality of sub-tracks of the second image. 18. The method of claim 17, wherein the transmitting of the first sub-track comprises:
transmitting the first sub-track in which all frames are composed of the first type frame. 18. The method of claim 17,
transmitting a second sub-track including the first type frame among the plurality of sub-tracks of the second image at a second time after the first time.

Images