WO2016002747A1

WO2016002747A1 - Content transmission device

Info

Publication number: WO2016002747A1
Application number: PCT/JP2015/068765
Authority: WO
Inventors: 琢也岩波; 渡部　秀一; 徳毛　靖昭
Original assignee: シャープ株式会社
Priority date: 2014-07-01
Filing date: 2015-06-30
Publication date: 2016-01-07
Also published as: JP2017152750A

Abstract

Provided is a content transmission device that reproduces and distributes content and that improves quality while suppressing network bandwidth, by enabling a client-side user to report, to a server, subjective evaluation data pertaining to the perceived resolution and smoothness of movement with respect to content, and by aggregating the data obtained by the server. This content transmission device, which transmits content comprising multiple segments to a content reproduction device on a segment-by-segment basis, is characterized by a transmission unit that, as a response to a request from the content reproduction device requesting the transmission of each segment, transmits to the content reproduction device a response containing the relevant segments, and a processing unit that processes subjective evaluation data obtained from the client, with the content being reproduced again when a processing amount measured by the processing unit is equal to or greater than a prescribed value.

Description

Content transmission device

The present invention relates to a content transmission apparatus that transmits content.

With the spread of the Internet and higher performance of computers, distribution of large-capacity content such as moving images via the Internet is widely performed. For example, there is a service called VOD (Video On Demand) that provides content such as a moving image in response to a user request. In VOD, for example, as described in Patent Document 1, data is transmitted and received between a server (content providing apparatus) and a client (content reproducing apparatus) using HTTP (HyperText Transfer Protocol).

Here, various technologies have been developed for content distribution by HTTP. For example, MPEG (Motion Picture Experts Group) is promoting international standardization using an adaptive streaming technology using HTTP as an MPEG-DASH (Dynamic Adaptive Streaming over HTTP) standard.

In MPEG-DASH, content is time-divided into a plurality of segments and transmitted in segment units. Each segment is composed of one or a plurality of fragments. The content is composed of one or a plurality of periods, and one period includes one or a plurality of segments.

Also, in MPEG-DASH, a plurality of representations having different quality types (reproduction quality such as bit rate, image resolution, and data format) are prepared for one content. For example, a plurality of segment data encoded at different bit rates for each segment is prepared. Accordingly, a client that receives and reproduces content can execute adaptive streaming by changing the bit rate of the requested content (segment) in accordance with the reception status of the content.

In MPEG-DASH, MPD (Media Presentation Description) is associated with content, and the content is managed by MPD. MPD is content metadata and describes content management information in XML format. In other words, MPD is information used by the client when acquiring / reproducing content.

A specific description example of MPD will be described with reference to FIG. FIG. 5 is a diagram illustrating a description example of MPD. As shown in FIG. 5, the MPD 200 describes acquisition destination information 201 indicating a content acquisition destination. More specifically, in the example shown in FIG. 5, the URL of the server is described as the acquisition source information 201. Here, URLs of two servers (cdn1 and cdn2) are described, and the client can select a content acquisition destination by referring to the MPD 200 as described above.

Also, here, it is assumed that a high-quality representation with a bit rate of 1024 kbps and a low-quality representation with a bit rate of 512 kbps are prepared as contents. Therefore, in the MPD 200, high-quality segment information 202 indicating a high-quality segment and low-quality segment information 203 indicating a low-quality segment are described as segments included in a certain period (from reproduction time 0 seconds to 600 seconds). Has been. The high quality segment information 202 describes the ID and bit rate of the high quality representation included in the period. In addition, the length and URL of each segment included in the period are described. The same applies to the low quality segment information 203. The period is composed of 60 segments from the first segment to the 60th segment.

Next, the operation sequence and HTTP message when switching the quality type of the content to be acquired based on the reception status of the content (when switching the representation to be acquired) will be described based on FIG. 6 and FIG. FIG. 6 is a diagram illustrating an example of an operation sequence in the case of switching the quality type of content acquired based on the content reception status. FIG. 7 is a diagram illustrating an example of an HTTP message when the quality type of content acquired based on the content reception status is switched.

Here, it is assumed that the MPD 200 shown in FIG. 5 is associated with the content, and the client 302 receives a high quality Representation (rep1) or a low quality Representation (rep2) from the server (cdn1) 301 according to the reception status. Shall be acquired.

As shown in FIGS. 6 and 7, first, the client 302 refers to the MPD 200 and transmits a request message 311 for requesting transmission of the Nth segment of high quality representation to the server 301. As a response to the request message 311, the server 301 transmits a response message 312 including the data body of the Nth segment of high quality representation to the client 302.

Here, when the client 302 detects the deterioration of the reception status when receiving the Nth segment of the high quality representation (process 313), the client 302 searches for an alternative representation with reference to the MPD 200 (process 314). Here, since the low quality representation (rep2) is described as an alternative representation in the MPD 200, the client 302 refers to the MPD 200 and requests the server 301 to transmit the (N + 1) th segment of the low quality representation. A request message 315 is transmitted to the server 301. As a response to the request message 315, the server 301 transmits a response message 316 including the data body of the (N + 1) th segment of the low quality representation to the client 302.

Japanese Published Patent Publication “Japanese Patent Laid-Open No. 2005-110244 (Publication Date: April 21, 2005)”

In conventional adaptive streaming, the client measures the content reception bandwidth status for a limited network bandwidth, and requests the server for the quality of received content (such as bit rate) based on the measurement result. There is a problem that the quality of receivable content is biased depending on the client request timing and the quality determination algorithm.

On the other hand, in order to solve this problem, if content of the same quality is distributed to any client, the network bandwidth is limited, so that low-quality content is distributed to each client.

The present invention has been made in view of the above problems, and its purpose is to notify the server of subjective evaluation data relating to the resolution and smoothness of movement perceived by the user on the client side. The content transmission device, the content reproduction device, the content distribution system, the control method for the content transmission device, the content transmission device that generates and distributes the content that improves the quality while suppressing the network bandwidth by aggregating the obtained data. To realize a control method, a data structure, a control program, and a recording medium.

In order to solve the above-described problem, the content transmission device according to the present invention is a content transmission device that transmits content composed of a plurality of segments to the content reproduction device for each segment, and each segment from the content reproduction device. As a response to the request for requesting transmission, a transmission unit that transmits a response including the segment to the content reproduction device, a processing unit that processes subjective evaluation data acquired from the client, and a processing amount measured by the processing unit In the case where the value is equal to or greater than the predetermined value, the content is generated again.

Also, the content transmission apparatus according to the present invention is characterized in that the subjective evaluation data is added to the header of the request in order to solve the above problem.

According to one aspect of the present invention, it is possible to stably realize live streaming with content in consideration of video quality perceived by the user.

1, showing an embodiment of the present invention, is a block diagram illustrating an example of a configuration of a content distribution system. FIG. 1, showing an embodiment of the present invention, is another block diagram illustrating an example of a configuration of a content distribution system. FIG. It is a figure which shows an example of the syntax of event information. It is another figure which shows an example of the syntax of event information. It is a figure which shows the example of description of MPD. It is a figure which shows an example of the operation | movement sequence in the case of switching the quality classification of the content acquired based on the reception condition of a content. It is a figure which shows an example of the HTTP message in the case of switching the quality classification of the content acquired based on the reception status of a content.

An embodiment of the present invention will be described with reference to FIGS. 1 to 4 as follows. First, an outline of the content distribution system of the present embodiment will be described based on FIG.

[Outline of content distribution system]
FIG. 1 is a diagram showing an outline of a content distribution system 1 according to the present embodiment. As shown in FIG. 1, the content distribution system 1 includes a server 100, a client 110, an imaging unit 105, and a display unit 116.

As shown in FIG. 1, the client 110 is connected to the server 100 via a network. Each device is connected by an arbitrary network of wired communication or wireless communication.

Further, the configuration of the content distribution system 1 is not limited to the example shown in FIG. For example, the content distribution system 1 may include two or more servers 100 and may include two or more clients 110. Further, the content distribution system 1 may include a proxy (not shown) between the server 100 and the client 110.

Further, in this embodiment, it is assumed that HTTP, which is widely used as a hypertext transfer protocol, is used as a transmission protocol on the network in the content distribution system 1. The content distributed by the server 100 is video content, and the content is segmented ISOBFF data. That is, in the present embodiment, the content distribution system 1 distributes content based on the above-described MPEG-DASH standard.

[Configuration of each device]
Next, the main configuration of the server 100 and the client 110 will be described with reference to FIG. FIG. 1 is a diagram illustrating an example of a main configuration of the server 100 and the client 110.

(About the server)
As illustrated in FIG. 1, the server 100 includes an encoding unit 101, a sending unit 102, a receiving unit 103, and a totaling / processing unit 104.

The encoding unit 101 encodes video and audio data input from the imaging unit 105.

The sending unit 102 communicates with the client 110 by wireless communication means or wired communication means, and exchanges data in accordance with instructions from the server 100. The receiving unit 103 communicates with the client 110 by wireless communication means or wired communication means, and exchanges data according to instructions from the client 110.

The tabulation / processing unit 104 tabulates user subjective evaluation data received by the receiving unit 103. When the absolute amount or relative amount of the subjective evaluation data is equal to or larger than the predetermined amount, the aggregation / processing unit 104 corrects the encoding parameter and outputs the corrected correction parameter to the encoding unit 101. On the other hand, when the absolute amount or the relative amount of the subjective evaluation data is less than the predetermined amount, the aggregation / processing unit 104 does not correct the encoding parameter and does not output to the encoding unit 101.

Further, the aggregation / processing unit 104 may generate the content of the encoding parameter according to the processing amount notified by the receiving unit 103.

(About the client)
The client 110 is a content reproduction device that reproduces content acquired from another device such as the server 100 or content stored in the device itself. The client 110 is, for example, a digital television, a recorder, an STB (Set Top Box), a PC, a mobile phone, a game machine, a PDA (Personal Digital Assistant), a digital camera, a digital video, or the like.

As shown in FIG. 1, the client 110 includes a receiving unit 111, a decoding / reproducing unit 112, an input unit 113, a metrics generation unit 114, and a transmission unit 115.

The receiving unit 111 acquires segment data based on an instruction from the sending unit 102. When the receiving unit 111 acquires segment data, the receiving unit 111 outputs the acquired segment data to the decoding / reproducing unit 112. Note that the receiving unit 111 may acquire segment data regardless of whether there is an instruction from the sending unit 102.

The decoding / reproducing unit 112 decodes the video data input from the receiving unit 111 and reproduces it.

The input unit 113 is for the user to input an instruction signal to the client 110. The input unit 113 may be configured with input devices such as a keyboard, a mouse, a keypad, and operation buttons. Further, a touch panel in which the input unit 113 and the display unit 116 are integrated may be used. The input unit 113 may be a remote control device such as a remote controller that is separate from the client 110. The input unit 113 may be a gesture recognition device that recognizes a user's gesture.

The display unit 116 is for receiving a video signal from the decoding / reproducing unit 112 and displaying the video. For example, an LCD (liquid crystal display), an organic EL display, a plasma display, or the like can be applied.

When the metrics generation unit 114 receives the subjective evaluation information from the input unit 113, the metrics generation unit 114 notifies the transmission unit 115 of the correspondence between the subjective evaluation type and the subjective evaluation value indicated by the received subjective evaluation information.

Specifically, when the metrics generation unit 114 receives subjective evaluation information from the input unit 113, the metrics generation unit 114 determines whether or not the user event indicated by the received subjective evaluation information matches the subjective evaluation type in the metrics generation unit 114. . When it is determined that the user event indicated by the received subjective evaluation information matches the subjective evaluation type in the metrics generation unit 114, the subjective evaluation information including the subjective evaluation type and the subjective evaluation value corresponding to the matched user event is used as event information. The data is transmitted to the server 100 via the sending unit 115. Details of the event information will be described later.

In other words, the metrics generation unit 114 waits for a user event indicating subjective evaluation information to be input from the input unit 113, detects a user event indicating subjective evaluation information, and sets the subjective evaluation type corresponding to the detected user event. Subjective evaluation information including the subjective evaluation value is transmitted to the server 100 via the sending unit 115.

In addition, when a user event other than the user event indicated by the subjective evaluation information is input from the input unit 113, the metrics generation unit 114 performs processing corresponding to the input user event based on a rule predetermined in the client 110. May be executed. The Metrics generation unit 114 may invalidate the input user event when a user event other than the user event indicated by the subjective evaluation information is input from the input unit 113.

The sending unit 115 communicates with the server 100 by wireless communication means or wired communication means, and exchanges subjective evaluation information generated by the metrics generation unit 114.

[When acquiring and playing back content using MPD]
Next, in the content distribution system 2, a case where content is acquired / reproduced using MPD between the server and the client will be described. For convenience of explanation, members having the same functions as those shown in the above embodiment are given the same reference numerals, and the explanation thereof is omitted. Hereinafter, differences from the above-described embodiment will be mainly described.

Referring to FIG. 2, the configuration of main parts of the server 200 and the client 210 will be described. FIG. 2 is a diagram illustrating an example of a main configuration of the server 200 and the client 210.

(About the server)
The server 200 is a content transmission device that receives a request message (request) requesting transmission of content from the client 210 and transmits a response message (response) to the received request message. Specifically, when a request for requesting transmission of content management information (MPD) is received from the client 210, a response including the MPD of the content is transmitted to the client 210. Further, when a request for transmitting a segment constituting the content is received from the client 210, a response including the segment is transmitted to the client 210.

2, the server 200 includes an encoding unit 201, a sending unit 202, a receiving unit 203, a totaling / processing unit 204, and a distribution information generating unit 205.

The encoding unit 201 encodes the video data input from the imaging unit 105. At this time, if an encoding parameter is input from the totalization / processing unit 104, the video or audio data is encoded according to the input encoding parameter, sent to the sending unit 202, and encoded. Information on the generated content is sent to the distribution information generation unit 205.

The sending unit 202 communicates with the client 210 by wireless communication means or wired communication means, and exchanges data according to instructions from the server 100. Here, when the sending unit 202 receives a request from the client 210, the sending unit 202 sends a response to the request to the client 210. Specifically, when the receiving unit 203 receives a request for requesting transmission of content management information (MPD) from the client 210, it instructs the distribution information generating unit 205 to generate an MPD of the content, and the distribution information generating unit When MPD data is acquired from 205, a response including the acquired MPD data is transmitted to the client 210. In addition, when a request for transmitting a segment constituting the content is received from the client 210, the sending unit 202 is instructed to acquire the segment, and a response including the segment data is transmitted to the client 210.

The receiving unit 203 communicates with the client 210 by wireless communication means or wired communication means, and exchanges data in accordance with instructions from the client 210. Specifically, the subjective evaluation information acquired from the sending unit 215 of the client 210 is not only sent to the aggregation / processing unit 204, but also the MPD acquisition request acquired from the sending unit 215 of the client 210 is sent to the sending unit 202.

The tabulation / processing unit 204 tabulates the user's subjective evaluation data received by the receiving unit 203. When the absolute amount or relative amount of the subjective evaluation data is equal to or larger than the predetermined amount, the aggregation / processing unit 204 corrects the encoding parameter and outputs it to the encoding unit 201. On the other hand, when the absolute amount or the relative amount of the subjective evaluation data is less than the predetermined amount, the aggregation / processing unit 204 does not correct the encoding parameter and does not output to the encoding unit 201. In addition, the aggregation / processing unit 204 may send the aggregated subjective evaluation data to the distribution information generation unit 205, and in this case, the distribution information generation unit 205 may use it as reference information when creating the MPD. Good.

The distribution information generation unit 205 creates an MPD based on the content information input from the encoding unit 201 and sends it to the sending unit 202. When creating the MPD, the subjective evaluation data input from the tabulation / processing unit 204 may be referred to.

(About the client)
The receiving unit 211 acquires segment data based on an instruction from the sending unit 202. When receiving the segment data, the receiving unit 211 outputs the acquired segment data to the decoding / reproducing unit 212. Note that the reception unit 211 may acquire segment data regardless of whether there is an instruction from the transmission unit 202.

The decoding / reproducing unit 212 decodes the video data input from the receiving unit 111 and reproduces it.

The sending unit 215 communicates with the server 100 by wireless communication means or wired communication means, and exchanges subjective evaluation information generated by the metrics generation unit 114. In addition, a request is transmitted from the transmission unit 215 to the server 200, and content (an MPD associated with the content and a segment constituting the content) is acquired from the server 200.

Specifically, when a content acquisition (playback) instruction is input from the user via the input unit 113, the transmission unit 215 transmits a request for requesting transmission of management information (MPD) of the content to the server 200. To do. Then, the receiving unit 211 receives the MPD data of the content as a response to the request. The received MPD data is referred to by the distribution information processing unit 217, and a request for requesting transmission of the segments constituting the content is transmitted to the server 200 via the transmission unit. Then, the reception unit 211 acquires the segment data of the content as a response to the request. The acquired segment data is output to the decoding / reproducing unit 212.

As described above, the distribution information processing unit 217 refers to the MPD input from the reception unit 211 and sends a request for transmission of a segment constituting the content to the transmission unit 215.

〔event information〕
3 and 4 are diagrams showing examples of event information syntax, respectively.

In FIG. 3, the sensation (“subjective_score”), resolution (“sublective_score_resolutional”), and motion blur (“subjective_score_bluring”) received from the entire content are set as the subjective evaluation types. Further, five levels from 5 (excellent) to 1 (poor) are set as subjective evaluation values.

FIG. 4 is another example showing the subjective evaluation types and subjective evaluation values described in FIG.

[Server processing]
Next, the content transmission process of the server 200 will be described.

The server 200 waits for a request from the client 210 to be transmitted. Here, when the reception unit 203 receives a request for requesting transmission of content management information (MPD) from the client 210, the reception unit 203 associates the distribution information generation unit 205 with the content indicated by the received request. Instruct to obtain the MPD.

Upon receiving the instruction from the reception unit 203, the distribution information generation unit 205 acquires the MPD data from the encoding unit, obtains the MPD data, creates the MPD, and outputs the MPD data to the transmission unit 202. Then, the transmission unit 202 transmits a response including the MPD data acquired from the distribution information generation unit 205 to the client 210.

When the receiving unit 203 receives a request for transmitting a segment from the client 210, the receiving unit 203 instructs the sending unit 202 to transmit the segment indicated by the request.

Also, when the receiving unit 203 acquires event information related to subjective evaluation information from the client 210, the receiving unit 203 totals it in the totaling / processing unit 204 and outputs the encoding parameters to the encoding unit 201.

When the event information is acquired, the aggregation / processing unit 204 counts the number of subjective evaluation values for each subjective evaluation type. Here, for example, when the subjective evaluation value of the resolution is high as a result of aggregation, but the subjective evaluation value of the moving image blur is low, the encoding unit instructs the encoding parameter of the content to increase the frame rate from the current level while reducing the resolution. 201 is notified. The encoding unit 201 performs content encoding processing based on the acquired encoding parameter, transmits video data to the transmission unit 202, and transmits parameter information to the distribution information generation unit 205.

The distribution information generation unit 205 updates the MPD and transmits it to the transmission unit 202.

The sending unit 202 sends the updated video data and MPD to the client 210 via the network.

The processing variation of the server 200 is not limited to the above. For example, the server 200 may distribute a message for acquiring an MPD describing different acquisition destination information to each client owned by a plurality of users. In this case, each client may acquire the MPD from the address described in the message, and may further acquire the content from the address described in the MPD. In this way, the server 200 controls the content acquisition destination for each client, thereby realizing data acquisition while suppressing congestion on the network.

As described above, the content is regenerated based on the subjective evaluation from the user, thereby realizing the content satisfying the user while suppressing the network load.

[Supplement]
The present invention is not limited to the above-described embodiments, and various modifications can be made within the scope shown in the claims. That is, embodiments obtained by combining technical means appropriately modified within the scope of the claims are also included in the technical scope of the present invention.

Finally, the blocks of the

servers

100 and 200 and the

clients

110 and 210 may be configured by hardware logic, or may be realized by software using a CPU as follows.

That is, the

servers

100 and 200 and the

clients

110 and 210 include a CPU (central processing unit) that executes instructions of a control program that realizes each function, a ROM (read only memory) that stores the program, and a RAM that expands the program. (Randomaccess memory), a storage device (recording medium) such as a memory for storing the program and various data. An object of the present invention is a recording medium in which program codes (execution format program, intermediate code program, source program) of control programs for the server 1 and the client 2 which are software for realizing the functions described above are recorded so as to be readable by a computer. This can also be achieved by supplying the above to the

servers

100 and 200 and the

clients

110 and 210, and reading and executing the program code recorded on the recording medium by the computer (or CPU or MPU).

Examples of the recording medium include tapes such as magnetic tapes and cassette tapes, magnetic disks such as floppy (registered trademark) disks / hard disks, and disks including optical disks such as CD-ROM / MO / MD / DVD / CD-R. Card system such as IC card, IC card (including memory card) / optical card, or semiconductor memory system such as mask ROM / EPROM / EEPROM (registered trademark) / flash ROM.

Further, the server 1 and the client 2 may be configured to be connectable to a communication network, and the program code may be supplied via the communication network. The communication network is not particularly limited. For example, the Internet, intranet, extranet, LAN, ISDN, VAN, CATV communication network, virtual private network, telephone line network, mobile communication network, satellite communication. A net or the like is available. Further, the transmission medium constituting the communication network is not particularly limited. For example, even in the case of wired such as IEEE 1394, USB, power line carrier, cable TV line, telephone line, ADSL line, etc., infrared rays such as IrDA and remote control, Bluetooth ( (Registered trademark), 802.11 wireless, HDR, mobile phone network, satellite line, terrestrial digital network, and the like can also be used. The present invention can also be realized in the form of a computer data signal embedded in a carrier wave in which the program code is embodied by electronic transmission.

The present invention can be used for a content transmission device that transmits content of the MPEG-DASH standard and a content playback device that acquires and plays back the content.

1, 2

Content distribution system

100, 200

Server

101, 201

Encoding unit

102, 202 Sending

unit

103, 203

Receiving unit

104, 204 Totaling / processing unit 105

Imaging unit

110, 201

Client

111, 211 Receiving unit 112 Decoding / reproducing unit 113, 213 Input unit 114 Metrics generating unit 115 Sending unit 116 Display unit 205 Distribution information generating unit 212 Decoding / reproducing unit 217 Distribution information processing unit

Claims

A content transmission device that transmits content composed of a plurality of segments to a content reproduction device for each segment,
As a response to a request for requesting transmission of each segment from the content reproduction device, a transmission unit that transmits a response including the segment to the content reproduction device;
A processing unit for processing subjective evaluation data acquired from a client;
A content transmitting apparatus, wherein content is generated again when a processing amount measured by the processing unit is equal to or greater than a predetermined value.
The content transmission apparatus according to claim 1, wherein the subjective evaluation data is added to a header of the request.