KR20200112518A - Method and Apparatus for Displaying Streaming Video Transmitted from Network - Google Patents

Abstract

According to the present embodiment, an apparatus for reproducing a streaming video transmitted from a network includes: a video information obtaining part obtaining information about a video which can be provided by a streaming server connected with the network; a video request part obtaining a zoom magnification for a video being displayed, creating an image quality selection command corresponding to the zoom magnification when the zoom magnification reaches a preset size, and requesting the streaming server for a video having an image quality different from the image quality of the video being displayed in order to respond to the image quality selection command based on the information; a video receiving part receiving the video of the different image quality corresponding to the request from the streaming server; and a video reproduction part reproducing the video of the different image quality by decoding the same. Therefore, the apparatus is capable of providing services and customer experience which are differentiated from existing online broadcasting.

Description

Method and Apparatus for Reproducing Streaming Video Transmitted from Network {Method and Apparatus for Displaying Streaming Video Transmitted from Network}

The present embodiment relates to a method and apparatus for reproducing a streaming video transmitted from a network.

The contents described below merely provide background information related to the present embodiment and do not constitute the prior art.

When a user watches a sports game that is broadcasted online using a terminal, the screen size of the terminal is limited, but the size of the entire stadium is very large. It is common that the detailed appearance of the stadium, such as the movements and facial expressions of the players, cannot be viewed in detail on the screen.

If the location is enlarged and relayed to the terminal screen so that a specific player or the shape of a specific location in the stadium can be identified in detail with the terminal, the user wants to check the appearance of the player playing at another location in the stadium at the same time. In this case, it cannot be checked on the corresponding terminal screen.

For example, if player A's playing field is enlarged in high definition and relayed to the terminal screen, if the terminal user wants to watch player B’s playing field at the same time that player A’s main scene was relayed, the player A’s main scene is broadcasted. Player B's stadium surface at the same viewpoint as the viewpoint is enlarged in high-definition, and the recorded broadcast image must be separately transmitted to the terminal to view the stadium surface of athlete B at the time point.

Accordingly, in order to solve the above problems associated with viewing high-definition images on a terminal having a limited screen size, there is a need for a method of expanding and viewing a display location or target scene selected by a viewer in high definition.

An object of the present embodiment is to provide a location and quality of an image desired by a user in reproducing a streaming image transmitted from a network.

According to an embodiment of the present invention, there is provided an apparatus for reproducing a streaming video transmitted from a network, comprising: an image information acquisition unit for acquiring information on an image that can be provided by a streaming server connected to the network; When the zoom magnification of the image being displayed is acquired and the zoom magnification reaches a preset size, a quality selection command corresponding to the zoom magnification is generated, and based on the information, the image being displayed is An image requesting unit for requesting an image of a different quality from that of the streaming server; An image receiver configured to receive an image of the different quality corresponding to the request from the streaming server; And an image reproducing unit that decodes and reproduces the image of the different quality.

According to another embodiment of the present invention, there is provided a method of playing a streaming video transmitted from a network, comprising: an image information acquisition process of obtaining information on an image that can be provided by a streaming server connected to the network; Obtaining a zoom magnification of the image being displayed and generating a quality selection command corresponding to the zoom magnification when the zoom magnification reaches a preset size; An image requesting process of requesting an image of a quality different from that of the image being displayed from the streaming server to correspond to the image quality selection command based on the information; An image receiving process of receiving an image of the different quality corresponding to the request from the streaming server; And an image reproducing process of decoding and reproducing the image of the different quality.

According to another embodiment of the present invention, there is provided an image information acquisition process of acquiring information on an image that can be provided by a streaming server connected to a network in a data processing device; Obtaining a zoom magnification of the image being displayed and generating a quality selection command corresponding to the zoom magnification when the zoom magnification reaches a preset size; An image requesting process of requesting an image of a quality different from that of the image being displayed from the streaming server to correspond to the image quality selection command based on the information; An image receiving process of receiving an image of the different quality corresponding to the request from the streaming server; And a computer-readable recording medium in which a program for realizing an image reproducing process for decoding and reproducing images of different quality is recorded.

The present invention allows the viewer to freely move to the main scene at a desired time or a desired location in the photographing object by the viewer's selection while viewing the entire photographing object on one screen with respect to an image of a specific photographing object broadcasted online. By doing so, there is an effect of providing a service and customer experience differentiated from conventional online broadcasting.

1 is a diagram showing the overall configuration of a streaming video service system according to the present embodiment.
2 is a diagram illustrating an image encoded in an HEVC tile structure.
3 is a diagram illustrating a high-definition coded image and a low-quality coded image, respectively.
4 is a diagram showing a streaming server according to the present embodiment.
5 is a diagram showing the overall configuration of a streaming video reproducing apparatus according to the present embodiment.
6 is a diagram illustrating an example of an encoded image segment generated by an image stream generator.
FIG. 7 is a diagram illustrating a region of a streaming image decoded according to a total reproduction size of a streaming image and a size playable in a display device.
Figure 8 (a) is a view showing a state before resetting the display position using a touch screen, Figure 8 (b) is a view showing the display position after the display position is reset using a touch screen to be.
9 is a diagram illustrating a change in service quality when a user wants to enlarge service quality using a touch screen.
FIG. 10 is a diagram illustrating an example in which a video segment transmitted to a corresponding terminal device varies according to a change in quality of an encoded video selected as a streaming service target by a terminal device.
11 is a diagram illustrating a streaming video reproducing method according to the present embodiment.

Hereinafter, some embodiments of the present invention will be described in detail through exemplary drawings. In adding reference numerals to elements of each drawing, it should be noted that the same elements are assigned the same numerals as possible even if they are indicated on different drawings. In addition, in describing the present invention, if it is determined that a detailed description of a related known configuration or function may obscure the subject matter of the present invention, a detailed description thereof will be omitted.

1 is a diagram showing the overall configuration of a streaming video service system according to the present embodiment.

As shown in Fig. 1, the streaming video service system 100 includes a plurality of cameras 101, 102, 103, an audio receiver 120, an image stitching unit 130, and an image stream generator 140. ), a streaming server 150, a network 160, and a terminal device 170.

In the present embodiment, an example is described that the target of the video streaming service is a sports game, but the present invention is not limited thereto and is applicable to providing video streaming services by photographing various targets.

The plurality of cameras 101, 102, and 103 each generate high-quality images at a relay site, for example, a sports stadium. The plurality of cameras 101, 102, 103 captures, for example, left, center, and right images of a sports stadium to generate a left high definition image, a center high definition image, and a right high definition image.

The sound receiver 120 receives sound generated during a sports event relay process. The sound signal received here may be a sound of a spectator generated in a stadium, a sound of a sound source tool, a voice of a player, and a voice of a broadcast broadcast announcer.

The image stitching unit 130 receives high-definition images captured by each of the plurality of cameras 101, 102, and 103, synchronizes them with each other, and synthesizes them into one high-definition image to generate a stitched image. In addition, the image stitching unit 130 synchronizes the stitched image with an acoustic signal corresponding thereto.

On the other hand, in the present embodiment, the timing of synchronizing the captured image and the corresponding sound signal is not limited to that after generating the stitched image, and when generating the coded image for the stitched image described below, etc. Synchronization is possible at this point. In addition, since the present embodiment is the core of a video streaming service method according to image quality conversion, an additional description of the sound signal will be omitted below.

The image stream generator 140 receives the stitched image and encodes it at a plurality of different bit rates to generate a plurality of encoded images each having a different bit rate. Here, the video stream generator 140 may encode using High Efficiency Video Codec (HEVC), but the present invention is not limited thereto.

If the image stitched by the image stitching unit 130 is a high-definition image, the plurality of encoded images generated by the image stream generation unit 140 are encoded in high-definition, low-quality, and standard quality, respectively. It may be an old video.

Here, the high-definition encoded image is a high-definition image stitched by the image stitching unit 130 is encoded, and the standard-quality encoded image is down-scaled to a lower resolution than the high-definition encoded image to be encoded. An image that has been encoded with a low-quality image and is downscaled to a lower resolution than an image encoded with a standard-quality image and is encoded. For example, an image encoded with high quality means a 4K (4096×2160) resolution coded image, an image coded with standard quality means a coded image of 2048×1080 resolution, and an image coded with low quality means 1280×720 resolution. Means the coded image of.

The image stream generator 140 may be implemented to encode each of a plurality of encoded images in a form divided into a tile structure.

Here, a tile means a data processing unit, which is a set of coding tree blocks (CTBs) that are the largest coding units, and there are m horizontal and n vertical coding tree blocks in the tile. Here, m and n may be the same or different from each other.

2 is a diagram illustrating an image encoded in an HEVC tile structure.

The image stream generation unit 140 implements a single encoded image to have a shape having 3 horizontal (column) and 1 vertical (row) tile, as shown in FIG. 2(a), or FIG. 2(b) As shown in ), it can be implemented to have a shape having 3 horizontal and 2 vertical tiles. In the present embodiment, when generating an encoded image, it is not limited to the 3×1 tile structure of FIG. 2A and the 3×2 tile structure of FIG. 2B, and may be generated in various types of tile structures. .

The information related to the tile structure in the encoded image can be in various forms such as a video parameter set (VPS), a sequence parameter set (SPS), a picture parameter set (PPS), or their extension data. It may be included in the header information and transmitted to the terminal device 170.

In addition, the header information may include information indicating that each tile is predictively encoded without referring to a reference image of another tile.

For example, a flag of loop_filter_across_tiles_enabled_flag may be included in the PPS header as a flag indicating whether to apply inter-tile loop filtering, for example.

loop_filter_across_tiles_enabled_flag indicates whether an in-loop filter crossing a tile boundary can be applied in a corresponding picture. The in-loop filter is a filter applied to a reconstructed picture, and is used for output of an encoded image, and a deblocking filter and a sample adaptive offset (SAO) filter are included in the in-loop filter.

For example, if the value of the loop_filter_across_tiles_enabled_flag flag is 1, it indicates that in-loop filtering across the tile boundary is performed for pictures referencing the PPS. If the flag value is 0, the picture referring to the PPS Indicates that in-loop filtering across tile boundaries is not allowed for s.

In addition, when referring to a reference picture for inter-prediction of a coding tree block in one tile during tile-based encoding, a reference picture at the same position as its own tile position to be encoded The encoding may be performed by excluding reference to data of other tiles other than the inner tile data, or encoding may be performed without excluding referencing data of tiles different from one's own tile position in the reference picture.

For example, in a coded image having a tile structure as shown in FIG. 2A, when inter-predicting a coding tree block in tile 2, only data existing in a tile at the same position as the tile 2 position in the reference picture is referred to. Alternatively, it may be implemented so that data existing at the location of tile 1 or tile 3, which is a tile at a location different from the location of tile 2, can be referred.

In the present embodiment, it is described as an example that the image stream generator 140 encodes each of the three levels of high-definition, standard-quality, and low-quality images as a plurality of encoded images, unless otherwise specified, but the present invention is limited thereto. It is not possible, and it is possible to encode an image having a quality level of various resolutions, such as encoding a two-stage quality image of high and low quality.

3 is a diagram illustrating a high-definition coded image and a low-quality coded image, respectively.

As shown in FIG. 3, when the streaming video service system 100 is implemented to provide a two-level video quality service, the video stream generator 140 uses the stitched high-definition video 301 to provide a plurality of coded video images. As a result, a high-quality coded image 302 and a low-quality coded image 303 are respectively generated.

The streaming server 150 receives a service request message from the terminal device 170 connected to the network 160 and transmits one of a plurality of encoded images to the network 160 according to the service quality of the terminal device 170 included in the service request message. ) To the terminal device 170 as a streaming video. In addition, the streaming server 150 transmits header information to the terminal device 170 together with the streaming image.

As a method in which the streaming server 150 transmits one of a plurality of encoded images to the terminal device 170 by streaming, DASH (Dynamic Adaptive Streaming over HTTP) may be used as a representative method. Hereinafter, in the present embodiment, it will be described that an encoded video is streamed using DASH, but the present invention is not limited thereto and various methods can be used.

4 is a diagram showing a streaming server according to the present embodiment.

The streaming server 150 according to the present embodiment includes an access information generating unit 410, a quality information receiving unit 420, and an image transmitting unit 430.

The access information generation unit 410 receives a service request message from the terminal device 170 connected to the network 160 and generates access information for a plurality of encoded images based on the received service request message, and the terminal device 170 Send to

When receiving a service request message from the terminal device 170, the access information generating unit 410 defines a multimedia presentation description (MPD) as access information for a plurality of encoded images, and the terminal device 170 Send to

The multimedia presentation description information includes image quality information of a coded image that can be provided and image profile information such as a URL address of a corresponding coded image.

The terminal device 170 may receive adaptive streaming of an encoded image based on the multimedia presentation description received from the streaming server 150 according to the network environment, terminal device specifications, and user's image quality selection information. have.

The image quality information receiving unit 420 acquires image quality selection information transmitted from the terminal device 170.

When the terminal device 170 is connected to the streaming server 150 to receive the video streaming service from the streaming server 150, the terminal device 170 may be configured according to the network environment, terminal device specifications, and image quality selection information of the user. A message requesting an encoded video of the selected service quality is transmitted to the quality information receiving unit 420.

The image transmission unit 430 transmits the coded image of service quality requested from the terminal device 170 to the terminal device 170 via the network 160.

5 is a diagram showing a streaming video reproducing apparatus according to the present embodiment.

The streaming image reproducing apparatus 500 according to the present embodiment of FIG. 5 is an example of implementing the terminal device 170 of FIG. 1.

The streaming image reproducing apparatus 500 according to this embodiment includes a service request unit 510, an image information acquisition unit 520, an image request unit 530, an image reception unit 540, a header information reception unit 550, and an image playback A unit 560, a user interface 570, a display position setting unit 580, and an image quality setting unit 590 are included. Here, the components of the streaming image reproducing apparatus 500 shown in FIG. 5 are not all essential components, and some components may be omitted or implemented by adding other components not shown.

When the streaming video playback device 500 is connected to the streaming server 150 to receive the video streaming service, the service requesting unit 510 determines the serviceable quality according to the network environment and terminal device specifications, and the determined serviceable quality information A service request message including a is transmitted to the streaming server 150.

When the service requesting unit 510 transmits the service request message to the streaming server 150, the streaming server 150 corresponds to the serviceable quality information included in the service request message, and the encoded video having the available quality as a streaming video. It transmits the access information for the streaming video playback device 500.

The image information acquisition unit 520 acquires access information for an encoded image of a quality available for presentation from the streaming server 150.

Here, the access information for the encoded image includes information on segments of a plurality of encoded images having a quality capable of streaming service, and the streaming server 150 defines a multimedia presentation description as access information for the encoded image to provide a streaming image. It is transmitted to the playback device 500.

The multimedia presentation description information includes information such as image quality information on a service-providable coded video segment and a URL (Uniform Resource Locator) address of the coded video segment.

6 is a diagram illustrating an example of an encoded image segment generated by an image stream generator.

As shown in FIG. 6, a coded image of each quality is divided into image segments, and one image segment refers to one or more coded image pictures generated during a predetermined time unit. For example, one video segment may be variously defined according to embodiments, such as a set of coded video pictures generated for 2 seconds or a set of coded video pictures of a preset number.

The video request unit 530 requests a video request message from the streaming server 150 for requesting a video of a quality corresponding to the service target quality information.

Here, the service target quality information is determined according to a network environment and display specifications of the streaming image playback device 500 when the streaming image playback device 500 is first connected to the streaming server 150, or a preset quality, for example, , It may be determined as a low picture quality corresponding to the lowest picture quality among serviceable picture quality.

A case in which the service target quality is changed by the user's selection while the streaming service is provided will be described later.

The image receiving unit 540 receives a streaming image of service quality requested by the image requesting unit 530 from the streaming server 150.

The header information receiving unit 550 receives header information transmitted along with the streaming image.

The header information includes information indicating whether each picture of the streaming video is encoded by dividing the region in units of tiles, and for example, the tiles_enabled_flag flag included in the PPS corresponds to this. For example, if the tiles_enabled_flag flag value is 1, it indicates that each picture is encoded by dividing the region in units of tiles, and if the tiles_enabled_flag flag value is 0, it indicates that each picture is encoded without dividing the region in units of tiles.

As described above, the header information may include information on the tile structure of the streaming video. In addition, the header information may include encoding information indicating that each tile is predictively encoded without referring to a reference image of a tile at a different location. As described above, a flag of loop_filter_across_tiles_enabled_flag may be included in the PPS header as a flag indicating whether to apply inter-tile loop filtering.

The video reproducing unit 560 divides and decodes the coded video in tile units according to the tiles_enabled_flag flag included in the header information and information related to the tile structure in the coded video, and in-loop filtering across the tile boundary according to the loop_filter_across_tiles_enabled_flag flag value. Selectively perform.

For example, when the value of the loop_filter_across_tiles_enabled_flag flag is 1, it indicates that in-loop filtering can be performed across the tile boundary for pictures referencing the corresponding PPS, and when this flag value is 0, the corresponding PPS is This indicates that in-loop filtering across a tile boundary is not allowed for reference pictures.

The image reproducing unit 560 refers to the header information received by the header information receiving unit 550 to decode the streaming image received by the image receiving unit 540 and display it on a display device (not shown).

The video reproducing unit 560 divides and decodes the streaming video in tile units according to the tiles_enabled_flag flag included in the header information and information related to the tile structure in the encoded video. For example, if the tiles_enabled_flag flag value is 1, the video playback unit 560 divides and decodes the streaming video in tile units, and if the tiles_enabled_flag flag value is 0, the video playback unit 560 does not split the streaming video in tile units. Without decryption.

The image reproducing unit 560, according to a result of comparing the total reproduction size of the received streaming image with the size reproducible on the display device of the streaming image reproducing apparatus 500, the bitstream corresponding to a partial coding region in the streaming image. The bitstream of the remaining area is decoded and reproduced.

FIG. 7 is a diagram illustrating a region of a streaming image decoded according to a total reproduction size of a streaming image and a size playable in a display device.

As shown in (a) of FIG. 7, the total playback size of tiles 1, 2 and 3 constituting the streaming image is equal to or smaller than the size that can be played on the display device 710 of the streaming image playback device 500. In this case, all tiles 1, 2, and 3 constituting the streaming image are decoded and displayed on the display device 710.

FIG. 7B is a diagram illustrating a case in which the total playback size 720 of tiles 1, 2, and 3 constituting a streaming image is larger than the size that can be played by the display device 710.

In the case of FIG. 7B, the image reproducing unit 560 does not decode the streaming image of the region corresponding to the tile 1 that is not displayed on the display device 710. On the other hand, the image reproducing unit 560 decodes the streaming image of the tile 2 and the tile 3 corresponding to the area displayed on the display device 710 and displays it on the display device 710.

The user interface 570 is a device for performing functions including starting and ending a streaming service, setting a service quality, and setting a display location.

The user interface 570 is generally implemented as a touch screen, but may be implemented by other means such as a mouse or a button mounted on the streaming image reproducing apparatus 500 according to an embodiment.

In this embodiment, a case in which a touch screen is used as the user interface 570 is described as an example.

The touch screen has a function as a display device for the image reproducing unit 560 to decode and reproduce a streaming image and a function as a user interface 570 in which a touch input detection function is embedded therein.

Hereinafter, a method of setting a display position and a method of setting service quality using a touch screen will be described.

Figure 8 (a) is a view showing a state before resetting the display position using a touch screen, Figure 8 (b) is a view showing the display position after the display position is reset using a touch screen to be.

FIG. 8A shows a tile corresponding to a position displayed in the streaming image by the video player 560 when the area displayed on the touch screen 810 in the streaming image spans tiles 2 and 3 A diagram illustrating a case of decoding 2 and 3 tiles. In this case, only a displayable area is displayed on the touch screen 810 from among images corresponding to the decoded tile 2 and tile 3 areas.

The touch screen 810 functions as a device for the image playback unit 560 to decode and display a streaming image, and a touch input detection function is built into the touch screen 810 so that the user By touching 830, it is possible to change the display position of the streaming image.

When the user touches the surface of the touch screen 810 using the finger 830, the touch screen 810 senses the touch of the user's finger 830.

When detecting the first touch by the user's finger 830 and the drag following the first touch, the touch screen 810 transmits information on a drag distance following the first touch to the display positioning unit 580.

The display position setting unit 580 receives information on the drag distance following the first touch from the touch screen 810 and sets a new display position.

On the other hand, since it is obvious to those skilled in the art to determine the distance of a touch and a subsequent drag on the touch screen 810, further detailed description will be omitted.

As shown in (a) of FIG. 8, when the user's finger 830 touches the surface of the touch screen 810 on which the high-definition streaming image is being displayed and drags it in the right direction, the display screen is the touch screen 810 It is moved to the right on the screen, and the touch screen 810 is moved to the left relative to the high-definition streaming image, as shown in (b) of FIG. 8.

When the display area of the touch screen 810 in the high-definition streaming image is changed from the first position in FIG. 8 to the second position in FIG. 8 by touching and dragging the surface of the touch screen 810 by the user's finger 830, The display position setting unit 580 resets the display position of the high-definition streaming image and transmits information on the reset display position to the image reproducing unit 560.

The image reproducing unit 560 decodes and reproduces the streaming image according to the information on the reset display position.

In (b) of FIG. 8, since an area displayed on the touch screen 810 among the entire streaming image area 820 exists across tiles 1 and 2, the image playback unit 560 is positioned in the streaming image. The tiles 1 and 2 corresponding to are decoded, and the tile 3 corresponding to a position that is not displayed may be implemented not to be decoded. In addition, the image reproducing unit 560 displays only a displayable area of the decoded image corresponding to the area of tile 1 and tile 2 on the touch screen 810.

In this way, according to the display position reset of the display position setting unit 580, the image reproducing unit 560 determines a partial coding region in the streaming image from which decoding is excluded and the remaining regions in which decoding is performed.

On the other hand, when the image receiving unit 540 receives a streaming image of service quality requested by the image requesting unit 530 from the streaming server 150, some coding regions excluded from the display target (ie, tile regions excluded from the display target) ) And the remaining bitstream corresponding to the encoding area to be displayed (i.e., the tile area to be displayed), and the bitstream corresponding to some encoding areas excluded from the display object to be displayed is an image reproducing unit 560 It may be implemented to select only the rest of the display target bitstream and transmit it to the image reproducing unit 560 without transmitting to ).

When the image receiving unit 540 is implemented as described above, the image receiving unit 540 performs a function of comparing the total reproduction size of the received streaming image with the size playable in the display device of the streaming image reproducing apparatus 500. In this case, the display position setting unit 580 resets the display position of the high-definition streaming image and transmits information on the reset display position to the image receiving unit 540.

9 is a diagram illustrating a change in service quality when a user wants to enlarge service quality using a touch screen.

9 is a diagram illustrating that the touch screen 810 detects a change in image zoom magnification using a pinch zoom function of the touch screen and a corresponding image quality change rate.

In FIG. 9, while the standard quality streaming image is being played, the touch screen 910 detects the first touch and the second touch by the touch of the user's finger 930, and the dragged position of the first touch and the second touch When detected, the touch screen 910 transmits information on the first touch and the second touch point and information on the dragged position of the first touch and the second touch to the image quality setting unit 590.

The image quality setting unit 590 receives information on the first touch and the second touch point and information on the dragged position of the first touch and the second touch, determines a zoom factor, and selects a service quality based on the information. . Here, the zoom magnification refers to a ratio of expanding the displayed screen (zoom-in ratio) or a ratio of reducing the screen (zoom-out ratio).

As described above, it is obvious to those skilled in the art to determine the zoom magnification using the touch and drag distances on the touch screen by using the pinch zoom function, and thus a detailed description thereof will be omitted.

The image playback unit 560 continues to play the image currently being displayed until the zoom factor determined by the image quality setting unit 590 reaches a certain size. However, the image quality of the image on which the display continues to be maintained remains the same. Only the size is enlarged and displayed.

When the zoom magnification reaches a certain size, the image quality setting unit 590 determines the zoom magnification as the image quality enlargement ratio or the image quality reduction ratio. For example, when the zoom magnification is greater than 1, the corresponding zoom magnification becomes the image quality enlargement ratio, and when the zoom magnification is less than 1, the zoom magnification becomes the image quality reduction ratio.

When the image quality expansion ratio determined by the image quality setting unit 590 is greater than or equal to the threshold value or has a value in a predetermined range as a quality expansion ratio, which means that the image quality is expanded to the next higher image quality than the image quality currently in service. The unit 530 transmits a video request message requesting the streaming server 150 for a higher resolution streaming video.

For example, there are three types of images that the streaming server 150 can provide: a low-quality image, a standard-quality image, and a high-definition image, and the quality resolution ratio of the three images is [low-quality resolution: standard-quality resolution: high-quality resolution = 1: 2:4] is assumed.

In this case, while the image reproducing unit 560 decodes and displays a low-quality streaming image, the image quality enlargement ratio calculated by the image quality setting unit 590 according to the user's touch result on the touch screen 910 is, for example, a first If the threshold value is 1.5 or more, a standard quality video request message for requesting a standard quality video service corresponding to a video having twice the resolution quality is transmitted to the streaming server 150 by the video request unit 530.

In addition, according to an embodiment, when the image quality expansion ratio has a value between 1.5 and 3, the image requesting unit 530 sends a standard quality image request message for requesting a service of a standard quality image corresponding to an image having the next higher resolution image quality. It may be implemented to be transmitted to the streaming server 150 by.

In addition, the image quality setting unit 590 is an image quality reduction ratio that means that the determined image quality reduction ratio (that is, when the zoom factor is less than 1) is reduced to a lower image quality that is one level lower than the image quality of the currently serving image or less than the threshold value. When it has a value in a predetermined range, the video requesting unit 530 transmits a video request message requesting the streaming server 150 for a lower resolution high-quality streaming video.

For example, while the image reproducing unit 560 decodes and displays a high-quality streaming image, the image quality reduction ratio calculated by the image quality setting unit 590 according to the result of the user's touch on the touch screen 910 is the second threshold value. If it is less than or equal to 0.7, a message requesting a standard-quality video service having a resolution image quality of 1/2 times is transmitted to the streaming server 150 by the video requesting unit 530.

In addition, according to an embodiment, when the image quality reduction ratio has a value between 0.35 and 0.7, a standard quality image request message for requesting a service of a standard quality image having a lower resolution (1/2 resolution) image quality is sent to the image request unit. It may be implemented to be transmitted to the streaming server 150 by 530.

Meanwhile, the image quality setting unit 590 may determine the service quality based on the previously received profile without calculating the image quality expansion ratio or the image quality reduction ratio. For example, when a user double-clicks a specific point on the display screen using a mouse or touches a specific point on the touch screen twice in a row, the service quality is determined as the next higher quality than the quality of the video currently being serviced. Can be implemented. Here, the next higher image quality may be defined by the received image profile information as multimedia presentation description information.

The streaming server 150 transmits a high-definition streaming video to the streaming video reproducing apparatus 500 according to the high-definition video request message received from the video requesting unit 530.

When the image receiving unit 540 receives a high-definition streaming image from the streaming image reproducing apparatus 500, the received high-definition streaming image is decoded and displayed on the touch screen 910. In this case, the area displayed on the touch screen 910 corresponds to a partial area 910_1 of the entire received high-definition image area 920, and the position of the displayed partial area 910_1 in the streaming image is the first touch and the first touch. 2 After the drag following the touch ends, it is determined according to the dragged position.

In this way, the image quality setting unit 590 refers to information on the types of image quality that can be provided by the streaming server 150 in determining the service quality according to the image quality expansion ratio. That is, the image quality setting unit 590 selects image quality information corresponding to any one of image quality information listed in the multimedia presentation description information including information such as image quality information of a coded image that can be provided and a URL address of the coded image. .

As described above, the image request unit 530 requests the streaming server 150 to provide a streaming image corresponding to the service quality determined by the image quality setting unit 590.

The streaming server 150 transmits a streaming video of a corresponding quality to the streaming video reproducing apparatus 500 according to the video request message received from the video requesting unit 530. For example, when the streaming server 150 receives a high-definition video request message from the video requesting unit 530, the streaming server 150 transmits the high-definition streaming video to the streaming video reproducing apparatus 500.

When the image receiving unit 540 receives a high-definition streaming image from the streaming image reproducing apparatus 500, the image reproducing unit 560 decodes the received high-definition streaming image and displays it on the touch screen 910. In this case, an area displayed on the touch screen 910 in the high-definition streaming image corresponds to a partial area 910_1 of the received high-definition image 920.

FIG. 10 is a diagram illustrating an example in which a video segment transmitted to a corresponding terminal device varies according to a change in quality of an encoded video selected as a streaming service target by a terminal device.

FIG. 10 shows that after displaying four low-quality streaming images, a user sequentially sets image quality expansion using a touch screen to display three standard-quality encoded image segments and three high-quality encoded image segments, and then touched by the user. This is an example of a case in which two standard quality coded image segments are sequentially displayed when image quality reduction is set to standard quality using a screen.

11 is a diagram illustrating a streaming video reproducing method according to the present embodiment.

Hereinafter, a streaming video playback method according to the present embodiment will be described with reference to FIGS. 1 to 11 together.

In order to reproduce the streaming image according to the present embodiment, the image requesting unit 530 requests the streaming server 150 for an image of image quality corresponding to the selected image quality information (S1110). Before step S1110, the image information acquisition unit 520 obtains information about the image that can be provided by the streaming server 150 connected to the network from the streaming server 150, and the image request unit 530 A process of receiving an image quality selection command may be added.

Here, the service target quality information is determined according to a network environment and display specifications of the streaming image playback device 500 when the streaming image playback device 500 is first connected to the streaming server 150, or a preset quality, for example, , May be determined as the lowest picture quality corresponding to the lowest serviceable picture quality.

The video receiving unit 540 receives a streaming video of a quality corresponding to the service quality requested by the video requesting unit 530 from the streaming server 150, and the header information receiving unit 550 receives header information necessary for decoding the streaming video. Do (S1120).

The video reproducing unit 560 checks whether all tiles belonging to the received streaming video can be played (S1130). That is, the video playback unit 560 compares the total playback size corresponding to the received streaming video with the size playable on the touch screen 910 of the streaming video playback device 500, and according to the result Check if the tile is playable.

When it is determined that some tiles in the received streaming image are not playable, the image reproducing unit 560 excludes bitstreams corresponding to some encoding regions from among the received streaming images from being decoded (S1140). When the received streaming image is encoded in a tile structure, tiles that are not reproduced on the touch screen 910 are excluded from the decoding target, and only the remaining tiles are set as the decoding target.

For example, when the size that can be reproduced in the streaming image reproducing apparatus 500 is smaller than the total reproduction size corresponding to the streaming image, the video reproducing unit 560 classifies the region of the streaming image into a decoding target region and a non-decoding target region. .

In addition, when receiving an image of a second quality having a higher resolution than the image of the first quality being played from the streaming server 150, the image reproducing unit 560 determines the resolution according to the resolution size of the image of the second quality. 2 Among the sub-regions of the streaming video corresponding to image quality, bitstreams corresponding to some sub-regions are excluded from the decoding target. That is, as illustrated in FIG. 8, when a high-quality image is received while displaying the entire standard-quality streaming image on the touch screen 910, the tile 1 region that does not correspond to the location of the displayed image is excluded from the decoding target.

In addition, when the sub-regions of the streaming image are decoded in units of tiles, the image reproducing unit 560 excludes reference images of tiles corresponding to some sub-regions excluded from the decoding target from the reference object.

The video reproducing unit 560 decodes the decoding target bitstream based on the header information received by the header information receiving unit 550 (S1150). For example, when it is determined that all tiles in the received streaming image can be played on the touch screen 910, the entire streaming image is decoded, and it is confirmed that some tiles in the received streaming image are not playable on the touch screen 910. If so, the bitstream corresponding to the remaining tile regions excluding the tiles that are not playable is decoded.

The image reproducing unit 560 outputs a screen of the reproducible region on the touch screen 910 to the touch screen 910 after decoding the reproducible tile region (S1160).

While the streaming image of a predetermined quality is being played back on the touch screen 910, the quality setting unit 590 stores information on the touch and drag input distances of the plurality of user fingers 930 with respect to the touch screen 910 on the touch screen ( 910), it is checked whether or not the service quality has changed (S1170).

The image quality setting unit 590 receives information on the location of the first touch and the second touch point, and information on the drag distance following the first and second touches, and calculates the image quality enlargement ratio, based on the calculation result. The service quality changes accordingly. For example, if the acquired image quality enlargement ratio is greater than or equal to the threshold value, the image request unit 530 transmits a high definition image request message requesting a higher definition high definition streaming image to the streaming server 150 (S1180). .

If the image quality enlargement ratio obtained by the image quality setting unit 590 is less than or equal to the threshold value, the display position setting unit 580 checks whether the display position on the touch screen 910 for the streaming image is changed (S1190).

The display position setting unit 580 sets a new display position according to a result of receiving one touch to the touch screen 810 and a drag length following the first touch from the touch screen 810.

If the display position setting unit 580 detects a change in the display position of the decoded image and sets a new display position, the process proceeds to step S1120 to receive a new streaming image, and all tiles belonging to the newly received streaming image are Repeating the step of checking whether or not it can be reproduced (S1130).

If the display position setting unit 580 does not detect a change in the display position of the decoded image, the image receiving unit 540 continues to receive the streaming image of the quality currently being serviced (S1192), and the image reproducing unit 560 ) Proceeds to a step of decoding a bitstream set as a decoding target from among a region corresponding to the entire streaming image (S1150).

The streaming image reproducing method illustrated in the present embodiment of FIG. 11 includes an image requesting unit 530, an image receiving unit 540, a header information receiving unit 550, an image playing unit 560, a display position setting unit 580, Since the image quality setting unit 590 and the touch screen 910 have been described above, further detailed descriptions are omitted.

Meanwhile, the method according to the embodiment illustrated in FIG. 11 may be implemented as a program and recorded on a computer-readable recording medium. The recording medium on which a program for implementing the method according to the present embodiment is recorded and that can be read by a computer includes all types of recording devices that store data that can be read by a computer system. That is, the computer-readable recording media include storage media such as magnetic storage media (eg, ROM, floppy disk, hard disk, etc.) and optical reading media (eg, CD-ROM, DVD, etc.). In addition, the computer-readable recording medium can be distributed over a computer system connected through a network to store and execute computer-readable codes in a distributed manner.

In addition, the method according to the embodiment illustrated in FIG. 11 may be implemented in the form of a program such as a mobile web or an application program installed on a terminal. For reference, when the method according to the present embodiment is implemented as an app program, the operating system of the terminal can be implemented to receive a quality selection command using the user interface of the terminal and the operating system of the terminal transmit the quality selection command to the app program. .

In the above description, the description was made on the assumption that all bitstreams included in the image of the corresponding quality are transmitted from the streaming server 150, but according to the embodiment, the region not reproduced by the streaming image reproducing apparatus 500 is the streaming server. It may be implemented such that 150 is excluded from the bitstream transmitted to the streaming image reproducing apparatus 500.

In order for the streaming server 150 to exclude a bitstream corresponding to a region that is not reproduced by the streaming image reproducing apparatus 500 from a transmission target, the streaming server 150 receives information corresponding to the transmission exclusion target in advance.

At this time, the display position setting unit 580 transmits information on the new display position determined by the user's touch screen touch to the image request unit 530, and the image request unit 530 transmits information on the received display position. When a video request message for requesting a service target quality video is transmitted, it is implemented to be transmitted to the streaming server 150 together.

The above description is merely illustrative of the technical idea of the present embodiment, and those of ordinary skill in the technical field to which the present embodiment belongs will be able to make various modifications and variations without departing from the essential characteristics of the present embodiment. Accordingly, the present exemplary embodiments are not intended to limit the technical idea of the present exemplary embodiment, but are illustrative, and the scope of the technical idea of the present exemplary embodiment is not limited by these exemplary embodiments. The scope of protection of this embodiment should be interpreted by the following claims, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present embodiment.

100: streaming video service system
101, 102, 103: multiple cameras 120: sound receiver
130: image stitching unit 140: image stream generation unit
150: streaming server 160: network
170: terminal device 301: stitched high-definition image
302: high quality coded image 303: low quality coded image
410: access information generation unit 420: image quality information receiving unit
430: video transmission unit
500: streaming video playback device
510: service request unit 520: image information acquisition unit
530: image requesting unit 540: image receiving unit
550: header information receiving unit 560: video reproducing unit
570: user interface 580: display positioning unit
590: image quality setting unit

Claims (14)

In the device for playing a streaming video (Streaming Video) transmitted from a network,
An image information acquisition unit that acquires information on an image that can be provided by a streaming server connected to the network;
When the zoom magnification of the image being displayed is acquired and the zoom magnification reaches a preset size, a quality selection command corresponding to the zoom magnification is generated, and based on the information, the image being displayed is An image requesting unit for requesting an image of a different quality from that of the streaming server;
An image receiver configured to receive an image of the different quality corresponding to the request from the streaming server; And
An image reproducing unit that decodes and reproduces the image of different quality
Streaming video playback device comprising a. The method of claim 1,
The video information acquisition unit, wherein the streaming video reproducing apparatus obtains information on a type of image quality that can be provided by the streaming server from the streaming server as information on the video. The method of claim 1,
The image receiving unit,
Comparing the total reproduction size corresponding to the image of the different image quality and the size playable by the streaming image reproducing apparatus, according to the comparison result, the bitstream of the remaining region of the image of the different image quality excluding some coding regions to the image reproducing unit A streaming video playback device, characterized in that the transmission. The method of claim 3,
The streaming image reproducing apparatus further comprises a display position setting unit configured to reset a display position of the image of the different quality by detecting a display position setting state using a user interface. The method of claim 4,
And the partial encoding region is an region selected according to resetting of the display position. The method of claim 1,
While the streaming image of the first quality is being played, the video requesting unit receives a quality selection command indicating selection of a second quality having a higher resolution than the first quality, and sends the streaming image of the second quality to the streaming server. If requested to
The image receiving unit may receive the streaming image of the second quality from the streaming server, and according to the size of the streaming image of the second quality, a bitstream corresponding to a partial coding region of the streaming image of the second quality to be decoded. A streaming video playback device, characterized in that it determines whether to exclude. The method of claim 1,
And a header information receiving unit for receiving header information indicating that the display area of the image of different quality is divided and encoded in a tile unit. The method of claim 7,
And the header information includes encoding information indicating whether each tile is encoded without referring to an image of another tile. The method of claim 1,
And the image reproducing unit continues to play the image being displayed until the zoom magnification reaches a predetermined size. In a method of playing a streaming video transmitted from a network,
An image information acquisition process of obtaining information on an image that can be provided by a streaming server connected to the network;
Obtaining a zoom magnification of the image being displayed and generating a quality selection command corresponding to the zoom magnification when the zoom magnification reaches a preset size;
An image requesting process of requesting an image of a quality different from that of the image being displayed from the streaming server to correspond to the image quality selection command based on the information;
An image receiving process of receiving an image of the different quality corresponding to the request from the streaming server; And
Image reproducing process of decoding and reproducing the image of different quality
Streaming video playback method comprising a. The method of claim 10,
In the video playback process,
A streaming video, characterized in that, when a size playable by the streaming video reproducing apparatus is smaller than the total playback size corresponding to the video of the different quality, the area of the video of the different quality is classified into a decoding target area and a non-decoding target area How to play. The method of claim 10,
In the video playback process,
When a second quality image having a higher resolution than the first quality image being played is received from the streaming server, a sub-area of the streaming image corresponding to the second quality according to the resolution size of the second quality image A method of reproducing a streaming video, comprising excluding bitstreams corresponding to some sub-regions from among the decoding targets. The method of claim 12,
In the video playback process,
When the sub-regions of the streaming image corresponding to the second image quality are each decoded in tile units, the reference image of the tile corresponding to the partial sub-regions is excluded from a reference object. Data processing equipment,
An image information acquisition process of obtaining information on an image that can be provided by a streaming server connected to the network;
Obtaining a zoom magnification of the image being displayed and generating a quality selection command corresponding to the zoom magnification when the zoom magnification reaches a preset size;
An image requesting process of requesting an image of a quality different from that of the image being displayed from the streaming server to correspond to the image quality selection command based on the information;
An image receiving process of receiving an image of the different quality corresponding to the request from the streaming server; And
Image reproducing process of decoding and reproducing the image of different quality
A recording medium that can be read by a computer that records a program for realizing it.

Images