US10382832B2 - Information processing apparatus and information processing method - Google Patents

Information processing apparatus and information processing method Download PDF

Info

Publication number
US10382832B2
US10382832B2 US15/205,546 US201615205546A US10382832B2 US 10382832 B2 US10382832 B2 US 10382832B2 US 201615205546 A US201615205546 A US 201615205546A US 10382832 B2 US10382832 B2 US 10382832B2
Authority
US
United States
Prior art keywords
bit rate
video
unit
display region
bandwidth
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
US15/205,546
Other versions
US20170251274A1 (en
Inventor
Akira Fujii
Suresh MURALI
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Fujifilm Business Innovation Corp
Original Assignee
Fuji Xerox Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fuji Xerox Co Ltd filed Critical Fuji Xerox Co Ltd
Assigned to FUJI XEROX CO., LTD. reassignment FUJI XEROX CO., LTD. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: FUJII, AKIRA, MURALI, Suresh
Publication of US20170251274A1 publication Critical patent/US20170251274A1/en
Application granted granted Critical
Publication of US10382832B2 publication Critical patent/US10382832B2/en
Assigned to FUJIFILM BUSINESS INNOVATION CORP. reassignment FUJIFILM BUSINESS INNOVATION CORP. CHANGE OF NAME (SEE DOCUMENT FOR DETAILS). Assignors: FUJI XEROX CO., LTD.
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
    • H04N21/637—Control signals issued by the client directed to the server or network components
    • H04N21/6373—Control signals issued by the client directed to the server or network components for rate control, e.g. request to the server to modify its transmission rate
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/266—Channel or content management, e.g. generation and management of keys and entitlement messages in a conditional access system, merging a VOD unicast channel into a multicast channel
    • H04N21/2662—Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
    • H04N21/47202—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for requesting content on demand, e.g. video on demand
    • H04L65/4092—
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H04L65/61—Network streaming of media packets for supporting one-way streaming services, e.g. Internet radio
    • H04L65/613—Network streaming of media packets for supporting one-way streaming services, e.g. Internet radio for the control of the source by the destination
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21—Server components or server architectures
    • H04N21/218—Source of audio or video content, e.g. local disk arrays
    • H04N21/2187—Live feed
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/234363—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by altering the spatial resolution, e.g. for clients with a lower screen resolution
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/234381—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by altering the temporal resolution, e.g. decreasing the frame rate by frame skipping
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/23439—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements for generating different versions
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/238—Interfacing the downstream path of the transmission network, e.g. adapting the transmission rate of a video stream to network bandwidth; Processing of multiplex streams
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/238—Interfacing the downstream path of the transmission network, e.g. adapting the transmission rate of a video stream to network bandwidth; Processing of multiplex streams
    • H04N21/2385—Channel allocation; Bandwidth allocation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/258—Client or end-user data management, e.g. managing client capabilities, user preferences or demographics, processing of multiple end-users preferences to derive collaborative data
    • H04N21/25808—Management of client data
    • H04N21/25825—Management of client data involving client display capabilities, e.g. screen resolution of a mobile phone
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/258—Client or end-user data management, e.g. managing client capabilities, user preferences or demographics, processing of multiple end-users preferences to derive collaborative data
    • H04N21/25808—Management of client data
    • H04N21/25833—Management of client data involving client hardware characteristics, e.g. manufacturer, processing or storage capabilities
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/262—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists
    • H04N21/26258—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists for generating a list of items to be played back in a given order, e.g. playlist, or scheduling item distribution according to such list
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/4104—Peripherals receiving signals from specially adapted client devices
    • H04N21/4126—The peripheral being portable, e.g. PDAs or mobile phones
    • H04N21/41265—The peripheral being portable, e.g. PDAs or mobile phones having a remote control device for bidirectional communication between the remote control device and client device
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/414—Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance
    • H04N21/41407—Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/422—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS]
    • H04N21/42204—User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor
    • H04N21/42206—User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor characterized by hardware details
    • H04N21/42224—Touch pad or touch panel provided on the remote control
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/433—Content storage operation, e.g. storage operation in response to a pause request, caching operations
    • H04N21/4331—Caching operations, e.g. of an advertisement for later insertion during playback
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/437—Interfacing the upstream path of the transmission network, e.g. for transmitting client requests to a VOD server
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/44004—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving video buffer management, e.g. video decoder buffer or video display buffer
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440236—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by media transcoding, e.g. video is transformed into a slideshow of still pictures, audio is converted into text
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440281—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by altering the temporal resolution, e.g. by frame skipping
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/442—Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk
    • H04N21/44209—Monitoring of downstream path of the transmission network originating from a server, e.g. bandwidth variations of a wireless network
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/45—Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
    • H04N21/462—Content or additional data management, e.g. creating a master electronic program guide from data received from the Internet and a Head-end, controlling the complexity of a video stream by scaling the resolution or bit-rate based on the client capabilities
    • H04N21/4621—Controlling the complexity of the content stream or additional data, e.g. lowering the resolution or bit-rate of the video stream for a mobile client with a small screen
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
    • H04N21/47217—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for controlling playback functions for recorded or on-demand content, e.g. using progress bars, mode or play-point indicators or bookmarks
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/482—End-user interface for program selection
    • H04N21/4825—End-user interface for program selection using a list of items to be played back in a given order, e.g. playlists
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/61—Network physical structure; Signal processing
    • H04N21/6106—Network physical structure; Signal processing specially adapted to the downstream path of the transmission network
    • H04N21/6125—Network physical structure; Signal processing specially adapted to the downstream path of the transmission network involving transmission via Internet
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/61—Network physical structure; Signal processing
    • H04N21/6106—Network physical structure; Signal processing specially adapted to the downstream path of the transmission network
    • H04N21/6131—Network physical structure; Signal processing specially adapted to the downstream path of the transmission network involving transmission via a mobile phone network
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/81—Monomedia components thereof
    • H04N21/8146—Monomedia components thereof involving graphical data, e.g. 3D object, 2D graphics
    • H04N21/8153—Monomedia components thereof involving graphical data, e.g. 3D object, 2D graphics comprising still images, e.g. texture, background image
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
    • H04N21/845—Structuring of content, e.g. decomposing content into time segments
    • H04N21/8456—Structuring of content, e.g. decomposing content into time segments by decomposing the content in the time domain, e.g. in time segments

Definitions

  • the present invention relates to an information processing apparatus and an information processing method.
  • a playback quality such as a resolution level of a video and playback continuity (in which the playback is not interrupted) is important.
  • an information processing device including: a first specifying unit that specifies a size of a display region of a video delivered by streaming, the video having a bit rate that is changeable during a playback; a second specifying unit that specifies a data amount of data temporarily stored for a period of the video to be next played back; a third specifying unit that specifies a bandwidth in reception of the data; and a determining unit that determines a bit rate of the video that is played back based on the size, the data amount, and the bandwidth that are specified.
  • FIG. 1 is a diagram illustrating an overall configuration of a video delivery system according to an exemplary embodiment
  • FIG. 2 is a diagram illustrating a stream delivery mechanism
  • FIG. 3 is a diagram illustrating a hardware configuration of a smart phone
  • FIG. 4 is a diagram illustrating a hardware configuration of a video delivering apparatus
  • FIG. 5 is a diagram illustrating functional configurations implemented by respective apparatuses
  • FIG. 6A is a diagram illustrating an exemplary video displayed on a display region
  • FIG. 6B is a diagram illustrating an exemplary video displayed on a display region
  • FIG. 6C is a diagram illustrating an exemplary video displayed on a display region
  • FIG. 7 is a diagram illustrating an exemplary bit rate table
  • FIG. 8 is a diagram illustrating an exemplary operation process of respective apparatuses in a stream delivery process
  • FIG. 9 is a diagram illustrating a functional configuration implemented by a smart phone according to a modified example.
  • FIG. 10 is a diagram illustrating functional configurations implemented by respective apparatuses according to a modified example
  • FIG. 11 is a diagram illustrating an exemplary file format table
  • FIG. 12 is a diagram illustrating an overall configuration of a video delivery system according to a modified example
  • FIG. 13 is a diagram illustrating a hardware configuration of a server apparatus
  • FIG. 14 is a diagram illustrating functional configurations implemented by respective apparatuses according to a modified example
  • FIG. 15 is a diagram illustrating a functional configuration implemented by a smart phone according to a modified example.
  • FIG. 16 is a diagram illustrating an exemplary displayed stream playback state.
  • FIG. 1 illustrates an overall configuration of a video delivery system 1 according to an exemplary embodiment.
  • the video delivery system 1 is a system that performs stream delivery of a video to a playback apparatus used by the user.
  • the stream delivery of the video refers to delivery of a video by a mechanism that sequentially plays back segment data indicating a part of a video while transceiving the segment data in a playback order.
  • the video delivery system 1 includes a communication line 2 , a smart phone 10 , and a video delivering apparatus 20 .
  • the communication line 2 is a system in which exchange of data is relayed between device such as a mobile communication network or the Internet.
  • the smart phone 10 is connected to the communication line 2 in a wireless manner
  • the video delivering apparatus 20 is connected to the communication line 2 in a wired manner (it is an example, and they may be connected by any of wireless and wired manners).
  • the smart phone 10 is a playback device that plays back a video that is delivered by streaming in the video delivery system 1 .
  • the smart phone 10 is carried by the user and used in various places.
  • the video delivering apparatus 20 is an information processing apparatus that stores a plurality of pieces of segment data indicating one video as a whole and performs stream delivery of a video by sequentially transmitting the segment data in the playback order.
  • One segment data are data indicating a portion of, for example, about 1 to 10 seconds among the entire the video.
  • a video that conforms to a standard called, for example, Moving Picture Experts Group (MPEG)-Dynamic Adaptive Streaming over HTTP (DASH) and has a bit rate that can be changed during the playback thereof is delivered by streaming.
  • MPEG Moving Picture Experts Group
  • DASH Dynamic Adaptive Streaming over HTTP
  • a plurality of bit rates are determined in advance.
  • FIG. 2 illustrates a stream delivery mechanism.
  • the video delivering apparatus 20 stores segment data groups A 1 , A 2 , and A 3 indicating videos having “high,” “intermediate,” and “low” bit rates, respectively.
  • the bit rate refers to a data amount (having a unit of megabits per seconds (Mbps) or megabytes (MB/sec) that is transceived at intervals of one second.
  • the smart phone 10 designates the bit rate of the video to be delivered when requesting the stream delivery of the video.
  • the video delivering apparatus 20 sequentially reads the segment data from the segment data group of the designated bit rate, and transmits the read segment data.
  • the smart phone 10 temporarily stores (caches) the received segment data, and reads and plays back the cached segment data.
  • the video delivering apparatus 20 receives the request, and transmits the segment data of the designated new bit rate.
  • the smart phone 10 plays back the video indicated by the segment data of the new bit rate, subsequently to the video indicated by the segment data of the current bit rate. Accordingly, the smart phone 10 displays the video in which the bit rate can be changed during the playback.
  • One of the features of the exemplary embodiment of the present invention lies in the bit rate decision method. The decision method will be described later in detail.
  • FIG. 3 illustrates a hardware configuration of the smart phone 10 .
  • the smart phone 10 is a computer that includes a central processing unit (CPU) 11 , a random access memory (RAM) 12 , a read only memory (ROM) 13 , a network interface card (NIC) 14 , a flash memory 15 , a touch screen 16 , a speaker 17 , and a sensor 18 .
  • the CPU 11 controls operations of the respective units by executing a program stored in the ROM 13 or the flash memory 15 using the RAM 12 as a work area.
  • the NIC 14 includes an antenna, a communication circuit, and the like, and performs wireless communication that conforms to, for example, a mobile communication standard.
  • the flash memory 15 stores data and a program which the CPU 11 uses for control.
  • a browser program is included as the program, and a function of the exemplary embodiment of the present invention which will be described later is implemented by Java (a registered trademark) script executed on the browser.
  • the present invention is not limited to this exemplary embodiment, and an application program that implements the function of the present invention may be stored.
  • the touch screen 16 includes a display serving as a display unit and a touch panel installed on the surface of the display, and displays an image and receives an operation from the user.
  • the speaker 17 converts audio data indicating a sound into an analog signal and outputs a sound.
  • the sensor 18 is, for example, a three-axis geomagnetic sensor, and supplies data indicating an inclination of its own apparatus to the CPU 11 .
  • FIG. 4 illustrates a hardware configuration of the video delivering apparatus 20 .
  • the video delivering apparatus 20 is a computer that includes a CPU 21 , a RAM 22 , a ROM 23 , a NIC 24 , and a hard disk drive (HDD) 25 .
  • the CPU 21 to the ROM 23 are the same hardware as the components having the same names illustrated in FIG. 3 .
  • the NIC 24 includes a communication circuit, and performs communication through the communication line 2 .
  • the HDD 25 stores data and a program which the CPU 21 uses for control.
  • FIG. 5 illustrates functional configurations implemented by the respective apparatuses.
  • the smart phone 10 includes a bit rate information acquiring unit 101 , a display region size specifying unit 102 , a cache amount specifying unit 103 , a bandwidth specifying unit 104 , a bit rate determining unit 105 , a delivery requesting unit 106 , a segment data receiving unit 107 , a temporary storage unit 108 , and a video playing-back unit 109 .
  • the video delivering apparatus 20 includes a segment data storage unit 201 , a bit rate information storage unit 202 , a bit rate information transmitting unit 203 , a segment data reading unit 204 , and a segment data transmitting unit 205 .
  • the segment data storage unit 201 of the video delivering apparatus 20 stores the segment data of the video delivered by streaming by its own apparatus.
  • the segment data storage unit 201 stores the segment data indicating the video of the plurality of bit rates as described above with reference to FIG. 2 . Since each piece of the segment data has a small communication amount, each piece of the segment data is encoded (compressed), for example, as data indicating a reference frame and a difference with the frame.
  • the bit rate information storage unit 202 stores bit rate information (for example, a media presentation description (MPD) in MPEG-Dash) indicating the plurality of bit rates of the video stored in the segment data storage unit 201 and delivered by streaming by its own apparatus.
  • bit rate information for example, a media presentation description (MPD) in MPEG-Dash
  • the bit rate information storage unit 202 stores the bit rate information indicating the bit rate of the video indicated by each of the segment data groups A 1 , A 2 , and A 3 .
  • the bit rate information is created and stored by the operator.
  • the bit rate information acquiring unit 101 of the smart phone 10 acquires the bit rate information indicating the plurality of bit rates of the video. For example, a link to the video that is delivered by streaming by the video delivering apparatus 20 is attached in a web page displayed on the smart phone 10 , and when the user performs an operation of selecting the link, the bit rate information acquiring unit 101 transmits request data for requesting the bit rate information of the video to the video delivering apparatus 20 .
  • the request data include information (for example, a uniform resource locator (URL) of a file (for example, an MPD in MPEG-DASH)) specifying the video.
  • URL uniform resource locator
  • the bit rate information transmitting unit 203 of the video delivering apparatus 20 Upon receiving the request data transmitted from the smart phone 10 , the bit rate information transmitting unit 203 of the video delivering apparatus 20 reads the bit rate information of the video specified by the request data, that is, the bit rate information of the video stored in its own apparatus from the bit rate information storage unit 202 and transmits the read bit rate information of the video to the request source (the smart phone 10 in this exemplary embodiment).
  • the bit rate information acquiring unit 101 acquires the bit rate information transmitted as described above.
  • the bit rate information acquiring unit 101 supplies the acquired bit rate information to the bit rate determining unit 105 .
  • the bit rate determining unit 105 determines any one of the plurality of bit rates indicated by the bit rate information acquired by the bit rate information acquiring unit 101 as the bit rate of the video to be played back. The bit rate decision method will be described later in detail.
  • the bit rate determining unit 105 repeatedly determines the bit rate at determined time intervals. For example, an interval of a time (for example, intervals of one second when a playback time is 2 seconds or the like) shorter than a playback time of the video indicated by one piece of segment data is used as the time interval. Each time the bit rate is determined, the bit rate determining unit 105 notifies the delivery requesting unit 106 of the determined bit rate.
  • the delivery requesting unit 106 requests an apparatus (the video delivering apparatus 20 in this exemplary embodiment) that delivers the video by streaming to deliver the video at the bit rate determined by the bit rate determining unit 105 .
  • an apparatus the video delivering apparatus 20 in this exemplary embodiment
  • the bit rate of the video that is delivered by streaming by the delivery requesting unit 106 is also changed.
  • the segment data reading unit 204 of the video delivering apparatus 20 reads the segment data of the requested video.
  • the segment data reading unit 204 Upon receiving designation data transmitted from the smart phone 10 , the segment data reading unit 204 reads a set number of pieces of segment data from the segment data group of the bit rate designated by the designation data in order from the beginning of the video.
  • the segment data reading unit 204 repeatedly performs the process of reading a set number of pieces of segment data subsequently to the read segment data at intervals of a playback time of a part of the video indicated by the segment data, which is read one time, or a time shorter than the playback time of the part.
  • the segment data reading unit 204 reads a set number of pieces of segment data from a part from which a playback position is continued among the segment data of the new bit rate.
  • the segment data reading unit 204 supplies the read segment data to the segment data transmitting unit 205 .
  • the segment data transmitting unit 205 transmits the supplied segment data to the designation data transmission source, that is, the request source (the smart phone 10 in this exemplary embodiment) that has requested the stream delivery of the video.
  • the segment data receiving unit 107 of the smart phone 10 receives the segment data transmitted from the video delivering apparatus 20 .
  • the segment data receiving unit 107 supplies the received segment data to the temporary storage unit 108 .
  • the temporary storage unit 108 temporarily stores the segment data received by the segment data receiving unit 107 until the part of the video indicated by the segment data is played back.
  • the video playing-back unit 109 sequentially reads the stored segment data, and sequentially plays back the video indicated by the read segment data. Since the segment data are encoded as described above, the video playing-back unit 109 decodes and plays back the segment data.
  • the video playing-back unit 109 causes the played-back video to be displayed on a display region but causes the video to be displayed on a different display region according to a direction of its own apparatus.
  • FIGS. 6A to 6C illustrate an exemplary video displayed on the display region.
  • the video is displayed on a display surface 161 of the touch screen 16 illustrated in FIG. 2 .
  • the display surface 161 is a rectangular surface surrounded by long sides 162 and short sides 163 .
  • FIG. 6A illustrates a display region B 1 when the smart phone 10 is disposed so that the long side 162 of the display surface 161 is along the vertical direction
  • FIG. 6B illustrates a display region B 2 when the smart phone 10 is disposed so that the long side 162 of the display surface 161 is along the horizontal direction.
  • a video C 1 in which a person is shown is displayed on both of the display regions B 1 and B 2 .
  • the video playing-back unit 109 causes the video to be displayed on the display region B 1 or B 2 according to an inclination of its own device.
  • the display region size specifying unit 102 of the smart phone 10 specifies the size of the display region of the video that is delivered by streaming.
  • the display region size specifying unit 102 is an example of a “first specifying unit” according to the exemplary embodiment of the present invention.
  • the display region size specifying unit 102 transmits an inquiry about a type of the display region (one of the display regions B 1 and B 2 ) in which the video is displayed to the video playing-back unit 109 . Upon receiving the inquiry, the video playing-back unit 109 notifies the display region size specifying unit 102 of the type of the display region in which the video is displayed.
  • the display region size specifying unit 102 specifies the size corresponding to the type of the display region that is notified of as the size of the display region.
  • the display region size specifying unit 102 specifies a value obtained by multiplying a length L 1 of the long side by a length L 2 of the short side illustrated in FIG. 6A , that is, an area of the display region as the size of the display region. Further, when the display region B 2 is notified of as the type of the display region, the display region size specifying unit 102 specifies a value obtained by multiplying a length L 3 of the long side by a length L 4 of the short side illustrated in FIG. 6B as the size of the display region.
  • the short side of the display region B 1 contacts with the long side 162 of the display surface 161
  • the short side of the display region B 2 contacts with the short side 163 of the display surface 161
  • the video playing-back unit 109 may cause the video C 1 to be displayed on a display region B 3 in which none of the long side and the short side contacts with the long side 162 and the short side 163 of the display surface 161 , for example, as illustrated in FIG. 6C .
  • the display region size specifying unit 102 specifies a value obtained by multiplying a length L 5 of the long side and a length L 6 of the short side of the display region B 3 as the display region size.
  • the display region size specifying unit 102 supplies size information (information indicating the area of the display region in this exemplary embodiment) indicating the size of the display region specified as described above to the bit rate determining unit 105 .
  • the cache amount specifying unit 103 of the smart phone 10 specifies a cache amount in the stream playback, that is, a data amount of the segment data temporarily stored for a period of the video to be next played back, in the video that is played back by streaming.
  • the cache amount specifying unit 103 is an example of a “second specifying unit” according to the exemplary embodiment of the present invention.
  • the cache amount specifying unit 103 refers to the segment data temporarily stored in the temporary storage unit 108 , and specifies a playback time in which the segment data that are referred to is played back as the cache amount.
  • the cache amount specifying unit 103 specifies that the video of 1.5 seconds is cached, and supplies information indicating 1.5 seconds to the bit rate determining unit 105 as cache amount information.
  • the bandwidth specifying unit 104 of the smart phone 10 specifies a bandwidth in reception of the segment data of the video delivered by streaming.
  • the bandwidth specifying unit 104 is an example of a “third specifying unit” according to the exemplary embodiment of the present invention.
  • the bandwidth specifying unit 104 monitors the segment data receiving unit 107 , and specifies a data size of the segment data received per unit time by the segment data receiving unit 107 as the bandwidth. For example, when the segment data of 10 Mbps are received for one second, the segment data receiving unit 107 specifies 10 Mbps as the bandwidth.
  • the segment data receiving unit 107 supplies bandwidth information indicating the specified bandwidth to the bit rate determining unit 105 .
  • the display region size specifying unit 102 , the cache amount specifying unit 103 , and the bandwidth specifying unit 104 perform the specifying process as the same time intervals as the time intervals at which the bit rate determining unit 105 repeatedly determines the bit rate in this exemplary embodiment.
  • the bit rate determining unit 105 determines the bit rate of the video that is played back based on the size of the display region specified by the display region size specifying unit 102 , the cache amount specified by the cache amount specifying unit 103 , and the bandwidth specified by the bandwidth specifying unit 104 .
  • the bit rate determining unit 105 uses a bit rate table in which the size of the display region is associated with a bit rate range.
  • FIG. 7 illustrates an exemplary bit rate table.
  • five bit rates such as 4.0 Mbps, 2.0 Mbps, 1.0 Mbps, 0.5 Mbps, and 0.3 Mbps are assumed to be defined for the stream delivery.
  • a bit rate ranges of “2.0, 1.0, 0.5, 0.3” (whose unit is all Mbps) is associated with a size of the display region of “less than threshold value Th 1 .”
  • a bit rate range of “4.0, 2.0, 1.0, 0.5” is associated with a size of the display region of “equal to or larger than threshold value Th 1 and less than threshold value Th 2 ,” and a bit rate range of “4.0, 2.0, 1.0” is associated with a size of the display region of “equal to or larger than threshold value Th 2 .”
  • the bit rate determining unit 105 reads the bit rate range associated with the size of the display region specified by the display region size specifying unit 102 from the bit rate table.
  • the bit rate determining unit 105 narrows the read bit rate range down to a range that does not exceed the bandwidth specified by the bandwidth specifying unit 104 . For example, when the size of the display region less than the threshold value Th 1 is specified, the bit rate determining unit 105 reads the bit rate range of “2.0, 1.0, 0.5, 0.3,” and when the bandwidth of 1.2 Mbps is specified, since the bit rate of 2.0 Mbps exceeds the specified bandwidth, the bit rate determining unit 105 narrows the bit rate range down to “1.0, 0.5, 0.3.”
  • the bit rate determining unit 105 selects any one of the bit rates of up to a set number in a descending order within the narrowed-down range (i.e., selects any one of the bit rate(s) from a set number of highest bit rates within the narrowed-down range), and determines the selected bit rate as the bit rate of the video that is played back.
  • the bit rate determining unit 105 regards 1 as the set number, that is, selects the highest bit rate and determines the highest bit rate as the bit rate of the video that is played back.
  • the narrowed-down range is “1.0, 0.5, 0.3” as in the above example
  • the bit rate determining unit 105 determines “1.0” which is the highest bit rate among them as the bit rate of the video to be played back.
  • the bit rate determining unit 105 selects any one of the bit rates from a set number in an ascending order within the narrowed-down range (i.e., selects any one of the bit rate(s) from a set number of lowest bit rates within the narrowed-down range), and determines the selected bit rate as the bit rate of the video that is played back.
  • a number that causes all the bit rates excluding the highest bit rate to be included is set.
  • the bit rate determining unit 105 selects any one of the bit rates excluding the highest one in the narrowed-down range, and determines the selected bit rate as the bit rate of the video that is played back.
  • the bit rate determining unit 105 selects any one of “0.5, 0.3” excluding “1.0” which is the highest bit rate among them, and determines the selected bit rate as the bit rate of the video to be played back.
  • the bit rate determining unit 105 may consistently select the highest bit rate among them when importance is placed on the resolution of the video or may consistently select the lowest bit rate among them when importance is placed on the continuity of the video.
  • the bit rate determining unit 105 determines the bit rate without using the cache amount specified by the cache amount specifying unit 103 and the bandwidth specified by the bandwidth specifying unit 104 until an initial response period set as a period in which the decision method at the initial stage of the playback is used elapses. In other words, the bit rate determining unit 105 determines the bit rate based on only the size of the display region specified by the display region size specifying unit 102 until the initial response period elapses.
  • the bit rate determining unit 105 selects any one of the bit rates in the bit rate range which is associated with the specified size of the display region in the bit rate table illustrated in FIG. 7 , and determines the selected bit rate as the bit rate of the video to be played back.
  • the bit rate determining unit 105 may select the highest bit rate from the range when the importance is placed on the resolution of the video or may select the lowest bit rate from the range when the importance is placed on the continuity of the video.
  • the present invention is not limited to this example, and, for example, when the frame rate desired by the user is set in advance, the bit rate determining unit 105 may select a bit rate of a value obtained by multiplying the set frame rate by the specified size of the display region and a coefficient or a bit rate of a value closest to the value and determine the selected bit rate as the bit rate of the video to be played back.
  • the respective apparatuses included in the video delivery system 1 perform a stream delivery process on the video based on the above-described configuration.
  • FIG. 8 illustrates an exemplary operation process of the respective apparatuses in the stream delivery process.
  • the operation process starts.
  • the smart phone 10 receives the video playback operation (step S 11 ). Then, the smart phone 10 (the bit rate information acquiring unit 101 ) requests the video delivering apparatus 20 to transmit the bit rate information of the video for which the playback operation is received (step S 12 ). The video delivering apparatus 20 (the bit rate information transmitting unit 203 ) reads the requested bit rate information of the video, and transmits the read bit rate information of the video to the smart phone 10 (step S 13 ). The smart phone 10 (the bit rate information acquiring unit 101 ) acquires the transmitted bit rate information (step S 14 ).
  • the smart phone 10 (the display region size specifying unit 102 ) specifies the size of the display region of the video for which the playback operation is received (step S 15 ). Then, the smart phone 10 (the bit rate determining unit 105 ) determines the bit rate of the video according to the specified size (step S 16 ). Then, the smart phone 10 (the delivery requesting unit 106 ) requests the video delivering apparatus 20 to perform the stream delivery of the video at the bit rate determined in step S 16 (step S 17 ).
  • the video delivering apparatus 20 (the segment data reading unit 204 ) reads the segment data indicating the video of the bit rate requested in step S 17 (step S 21 ). Then, the video delivering apparatus 20 (the segment data transmitting unit 205 ) transmits the segment data read in step S 21 to the smart phone 10 (step S 22 ).
  • the smart phone 10 (the segment data receiving unit 107 ) receives the segment data transmitted in step S 22 .
  • the smart phone 10 (the temporary storage unit 108 ) temporarily stores the received segment data.
  • the smart phone 10 (the video playing-back unit 109 ) reads and plays back the segment data temporarily stored, and causes the segment data to be displayed on the display region (step S 23 ). Then, the smart phone 10 (the display region size specifying unit 102 ) specifies the size of the display region of the played-back video (step S 24 ). Then, the smart phone 10 (the cache amount specifying unit 103 ) specifies a data amount of the segment data temporarily stored, that is, the cache amount (step S 25 ).
  • the smart phone 10 (the bandwidth specifying unit 104 ) specifies the bandwidth in the reception of the segment data (step S 26 ). Then, the smart phone 10 (the bit rate determining unit 105 ) determines the bit rate based on the size of the display region specified in step S 24 , the cache amount specified in step S 25 , and the bandwidth specified in step S 26 (step S 27 ). The smart phone 10 (the delivery requesting unit 106 ) requests the video delivering apparatus 20 to deliver the video of the bit rate determined in step S 27 (step S 28 ).
  • the video delivering apparatus 20 (the segment data reading unit 204 ) returns to step S 21 , and reads the segment data indicating the video of the bit rate requested in step S 28 . Thereafter, the operation of steps S 21 to S 28 is repeated until the user ends the playback operation or the video is played back up to the end, and so the playback of the video ends.
  • the bit rate In the playback of the video, unless the bit rate is changed, as the display region decreases, the resolution of the video increases. In other words, when the display region is small, although the bit rate is low, the decrease in the resolution of the video is suppressed compared to when the display region is large. In this exemplary embodiment, since the bit rate corresponding to the specified size of the display region is determined, compared to when the size of the display region is not considered, a phenomenon that, when the display region is small, the video of the resolution higher than necessary is displayed, and the wireless communication band is compressed is suppressed.
  • the bit rate is determined based on the specified cache amount as well. For example, when the cache amount is relatively large (when the cache playback time is equal to or larger than the threshold value Th 3 ), a high bit rate is selected, and when the cache amount is relatively small (when the cache playback time is less than the threshold value Th 3 ), a low bit rate is selected.
  • the playback quality refers to a comprehensive quality including the level of the resolution of the video and the playback continuity (in which the playback is not interrupted).
  • the playback of the video is more likely to be interrupted than when the cache amount is relatively large, and thus when the low bit rate is selected as described above, it takes a time to receive new segment data, but the playback of the video indicated by the data temporarily stored is unlikely to be interrupted.
  • the situation in which the playback of the video indicated by the segment data temporarily stored ends before the reception of the new segment data ends, and thus the video is interrupted is more likely to occur than when the bit rate is selected by a different method from that of this exemplary embodiment.
  • the bit rate is determined based on the specified bandwidth as well.
  • the bandwidth is high, and the video is unlikely to be interrupted, the high bit rate is selected so that the resolution of the video is increased, and the playback quality of the video is improved by improving the resolution of the video, whereas when the bandwidth is low, the low bit rate is selected so that the video is unlikely to be interrupted, and the playback quality of the video is improved by improving the continuity of the video.
  • the bit rate is determined based on the size of the display region and the cache amount in addition to the bandwidth as described above, and thus the playback quality is improved to be higher than, for example, when the bit rate is determined in view of only the bandwidth. Further, in this exemplary embodiment, in the initial response period, the bit rate is determined without using the cache amount and the bandwidth. As a result, the bit rate is determined even in the period in which the cache amount and the bandwidth are not specified yet, and thus the playback quality of the video is improved to be higher than when the bit rate is determined without considering the size of the display region in the initial response period.
  • the bit rate decision method by the bit rate determining unit 105 is not limited to the above-described example.
  • the bit rate determining unit 105 determines the bit rate that leads an increase in the data amount of the segment data temporarily stored in the temporary storage unit 108 among the bit rates in the bit rate range narrowed down similarly to the example as the bit rate of the video to be played back.
  • the bit rate range is assumed to be narrowed down to “1.0, 0.5, 0.3” (Mbps) based on the size of the display region and the bandwidth using the bit rate table illustrated in FIG. 7 . Since the data amount is decreased when 0.3 Mbps among them is selected, the data amount does not change when 0.5 Mbps is selected, but the data amount is increased when 1.0 Mbps is selected, the bit rate determining unit 105 selects 1.0 Mbps and determines 1.0 Mbps as the bit rate of the video to be played back.
  • the bit rate determining unit 105 may select the highest bit rate and determine the highest bit rate as the bit rate of the video to be played back when the importance is placed on the resolution of the video or may select the lowest bit rate and determine the lowest bit rate as the bit rate of the video to be played back when the importance is placed on the continuity of the video.
  • the data amount of the segment data stored in the temporary storage unit 108 is increased due to the change in the bit rate, and thus the playback of the video is more unlikely to be interrupted than when the bit rate that does not lead to the increase in the data amount is selected.
  • the bit rate determining unit 105 may determine any one of the bit rates that lead to an increase in the playback time by the segment data temporarily stored in the temporary storage unit 108 in the above-described range as the bit rate of the video to be played back.
  • an increase in the bit rate leads to an increase in the frame rate and an increase in the resolution as in the example
  • the data amount is mostly increased as well, but, for example, there are cases in which the bit rate that does not lead to the increase in the data amount by decreasing the resolution of the video instead of increasing the number of frames is used as well.
  • the bit rate at which the data amount is not increased, but the number of frames is increased is determined, the playback of the video is more unlikely to be interrupted, and the communication load is likely to be reduced.
  • the bit rate determining unit 105 may change the bit rate decision method according to the playback speed of the video.
  • the level of the bit rate is assumed to correspond to at least one of the level of the resolution of the video and the level of the frame rate of the video.
  • the resolution may not change in a certain bit rate range although the bit rate is increased, for example, when the bit rate is increased to 0.5 Mbps, 1.0 Mbps, 2.0 Mbps, and 4.0 Mbps, the resolution may be increased to 640 ⁇ 360, 854 ⁇ 480, 854 ⁇ 480, and 1280 ⁇ 720.
  • FIG. 9 illustrates a functional configuration implemented by a smart phone 10 a according to the present modified example.
  • the smart phone 10 a includes a playback speed change operation receiving unit 110 in addition to the units illustrated in FIG. 5 .
  • the playback speed change operation receiving unit 110 receives an operation of changing the playback speed of the video that is being played back by streaming. Examples of the operation of changing the playback speed include a fast-forward operation, a slow playback operation, and a rewind operation.
  • the playback speed change operation receiving unit 110 Upon receiving the playback speed changing operation, the playback speed change operation receiving unit 110 notifies the bit rate determining unit 105 of content of the received operation.
  • the bit rate determining unit 105 determines the bit rate based on the content of the operation that is reported from the playback speed change operation receiving unit 110 . For example, when the video is played back at a speed (first-forward playback), the bit rate determining unit 105 determines the bit rate in which the resolution or the frame rate is lower than that at the time of the normal playback.
  • the bit rate determining unit 105 is assumed to select the highest bit rate (4.0 Mbps) in the normal playback and determines the highest bit rate (4.0 Mbps) as the bit rate of the video to be played back.
  • the bit rate determining unit 105 selects the second highest bit rate (2.0 Mbps) or the third highest bit rate (1.0 Mbps) in which the resolution and the frame rate are lower than at the time of the normal playback and determines the selected bit rate as the bit rate of the video to be played back. Since the user mostly performs the fast-forward playback when the user search for a scene which the user wants to view, in this case, it is desirable that a scene that is played back can be roughly understood, and thus the high resolution and the high frame rate are mostly not necessary.
  • the resolution and the frame rate are lower than at the time of the normal playback, but the user does not feel uncomfortable mostly due to the above reason.
  • the communication load of the segment data is reduced to be smaller, and it is easier to secure a band for other stream delivery than when the normal playback is performed, and the bit rate decision method is not changed.
  • a time per unit time required to receive the segment data is shorter, and the playback time by the segment data temporarily stored when the normal playback is performed after the fast-forward playback ends is longer, and thus the playback of the video is unlikely to be interrupted after the fast-forward playback ends.
  • the playback quality is improved to be higher than when the normal playback is performed, and the bit rate decision method is not changed.
  • the bit rate determining unit 105 may determine the bit rate based on a magnification ratio in the fast-forward playback. For example, the bit rate determining unit 105 determines the bit rate such that the bit rate that decreases as the magnification ratio increases is selected. As the magnification ratio increases, the data amount of the transmitted segment data increases, and thus a time required until the reception is completed increases, and the playback is likely to be interrupted. In this regard, since the bit rate is determined based on the magnification ratio of the fast-forward playback, the fast-forward playback is more unlikely to be interrupted, and the playback quality of the video is improved to be higher than when the magnification ratio is not considered.
  • the bit rate determining unit 105 determines the bit rate in which the resolution or the frame rate is higher than at the time of the normal playback. For example, when the bit rate range is narrowed down to the three bit rates of 4.0 Mbps, 2.0 Mbps, and 1.0 Mbps according to the size of the display region and the bandwidth, and the cache amount is equal to or larger than the threshold value Th 3 , the bit rate determining unit 105 is assumed to select the lowest bit rate (1.0 Mbps) in the normal playback and determines the lowest bit rate (1.0 Mbps) as the bit rate of the video to be played back.
  • the bit rate determining unit 105 selects the second highest bit rate (2.0 Mbps) or the highest bit rate (4.0 Mbps) in which the resolution and the frame rate are higher than at the time of the normal playback and determines the selected bit rate as the bit rate of the video to be played back. Since the user performs the slow playback when the user wants to view the scene more carefully, it is desirable that the resolution and the frame rate be high. Using this decision method, at the time of the slow playback, the video in which the resolution or the frame rate is higher than at the time of the normal playback is played back. As a result, the playback quality is improved to be higher than when the normal playback is performed, and the bit rate decision method is not changed.
  • the bit rate determining unit 105 may determine the bit rate based on the magnification ratio in the slow playback. For example, the bit rate determining unit 105 determines the bit rate such that the bit rate that increases as the magnification ratio decreases (that is, the speed decreases) is selected. In the case of the slow playback, as the magnification ratio decreases, a period until a next frame is played back increases, and thus although the bit rate is increased, the playback (the slow playback) of the video is unlikely to be interrupted. In this regard, since the bit rate is determined based on the magnification ratio of the slow playback, the resolution or the frame rate of the video and the playback quality of the video are improved to be higher than when the magnification ratio is not considered.
  • a mechanism of acquiring a still image indicating a certain scene of the video that is played back may be provided.
  • FIG. 10 illustrates functional configurations implemented by the respective apparatuses according to the present modified example.
  • a smart phone 10 b including a still image requesting unit 111 and a still image display unit 112 in addition to the units illustrated in FIG. 5 is illustrated.
  • a video delivering apparatus 20 b including a still image generating unit 206 and a still image transmitting unit 207 in addition to the units illustrated in FIG. 5 is illustrated.
  • the still image requesting unit 111 of the smart phone 10 b requests the video delivering apparatus 20 b to transmit the still image of the scene.
  • the still image requesting unit 111 requests the still image of the scene in a video of a second bit rate higher than the first bit rate.
  • the still image requesting unit 111 requests the still image using a bit rate that is one step higher than the first bit rate as the second bit rate.
  • the second bit rate is not limited to this example and may be the highest bit rate of the video that is delivered by streaming.
  • the still image requesting unit 111 makes the request by transmitting request data indicating the video in which the still image is requested, the scene of the still image, and the second bit rate to the video delivering apparatus 20 b .
  • request data indicating the video in which the still image is requested, the scene of the still image, and the second bit rate
  • a URL of the video is used as information specifying the video
  • a playback time of the scene is used as information specifying the scene of the still image.
  • the still image generating unit 206 of the video delivering apparatus 20 b Upon receiving the transmitted request data, the still image generating unit 206 of the video delivering apparatus 20 b generates the still image of the scene indicated by the request data with reference to the segment data of the video of the bit rate indicated by the request data.
  • the still image generating unit 206 decodes the segment data including the target scene, and generates the still image of the scene. For example, the still image generating unit 206 generates the still image at a predetermined file format (joint photographic experts group (JPEG), portable network graphics (PNG), or the like).
  • JPEG photographic experts group
  • PNG portable network graphics
  • the still image generating unit 206 supplies the generated still image to the still image transmitting unit 207 .
  • the still image transmitting unit 207 transmits the supplied still image, that is, the still image requested from the still image requesting unit 111 to the smart phone 10 b serving as the request source apparatus.
  • the still image display unit 112 of the smart phone 10 b causes the still image transmitted from the video delivering apparatus 20 b to be displayed on the display unit (the display of the touch screen 16 ) of its own apparatus.
  • the resolution mostly increases as well.
  • the still image of the video of the second bit rate is requested, and thus the resolution of the still image to be displayed is more easily increased than when the still image of the video of the first bit rate is requested.
  • the data size of the still image is smaller than the video that is delivered by streaming, and thus even when the resolution is increased, the communication load is small, and even when the still image having the high resolution is acquired, influence on the communication is suppressed to be small.
  • the still image transmitting unit 207 transmits the still image generated from the segment data, but the present invention is not limited to this example, and, for example, data of a still image of each scene may be stored in a storage unit of its own apparatus in advance, and the requested still image of the scene may be read from the storage unit and transmitted.
  • the number of formats of the still image to be transmitted is not limited to one, and two or more formats (for example, both JPEG and PNG or the like) may be used, and any one of the formats may be selected by the user. Further, when two or more still images are transmitted, the still image transmitting unit 207 may transmit the still images at the same time or different time (different timings).
  • the file format of the still image is set in advance, but the present invention is not limited to this example, and the file format of the still image may be variable.
  • the still image requesting unit 111 requests the still image of the file format corresponding to the bandwidth specified by the bandwidth specifying unit 104 .
  • the still image requesting unit 111 uses, for example, a file format table in which the bandwidth is associated with the file format.
  • FIG. 11 illustrates an exemplary file format table.
  • “JPEG” is associated with the bandwidth of “less than threshold value Th 11 ”
  • “PNG” is associated with the bandwidth of “equal to or larger than threshold value Th 11 .”
  • the bandwidth specifying unit 104 supplies the bandwidth information indicating the specified bandwidth to the still image requesting unit 111 .
  • the still image requesting unit 111 requests the video delivering apparatus 20 b to transmit the still image of the file format that is associated with the bandwidth indicated by the supplied bandwidth information in the file format table.
  • the file format may be set according to the purpose of the still image.
  • the purpose is to enable the general user to view a video of a movie, a television, or the like
  • JPEG is desirable, but when the purpose is to check a video of a monitoring camera, the video may be enlarged in order to discern an object that is shown in the video, PNG in which a fine image can be expressed by a lossless compression scheme is used.
  • the purpose of the video is set in the smart phone 10 b through the operation of the user or the like, and the still image requesting unit 111 requests the video delivering apparatus 20 b to transmit the still image of the file format corresponding to the set purpose of the video.
  • the still image of the file format suitable for the purpose of a video is displayed.
  • the user may select the bit rate and the file format serving as the basis of the still image.
  • the still image requesting unit 111 displays a selection screen in which the bit rate and the file format are selected, and requests the video delivering apparatus 20 b to transmit the still image of the bit rate and the file format selected by the operation performed on the selection screen.
  • the still image of the file format desired by the user is displayed.
  • the bit rate may be determined by a server apparatus.
  • FIG. 12 illustrates an overall configuration of a video delivery system 1 c according to the present modified example.
  • the video delivery system 1 c includes a communication line 2 , a smart phone 10 c , a video delivering apparatus 20 , and a server apparatus 30 .
  • FIG. 13 illustrates a hardware configuration of the server apparatus 30 .
  • the server apparatus 30 is a computer that includes a CPU 31 , a RAM 32 , a ROM 33 , a NIC 34 , and a HDD 35 .
  • the CPU 31 to the HDD 35 are the same hardware as the components having the same names illustrated in FIG. 4 .
  • FIG. 14 illustrates functional configurations implemented by the respective apparatuses according to the present modified example.
  • the smart phone 10 c includes a playback operation notifying unit 113 in addition to the segment data receiving unit 107 , the temporary storage unit 108 , and the video playing-back unit 109 illustrated in FIG. 5 .
  • the server apparatus 30 includes the bit rate information acquiring unit 101 , the display region size specifying unit 102 , the cache amount specifying unit 103 , the bandwidth specifying unit 104 , the bit rate determining unit 105 , and the delivery requesting unit 106 illustrated in FIG. 5 .
  • the playback operation notifying unit 113 of the smart phone 10 c gives a notification indicating that the operation is performed to the server apparatus 30 .
  • the notification is received by the bit rate information acquiring unit 101 , the display region size specifying unit 102 , the cache amount specifying unit 103 , and the bandwidth specifying unit 104 of the server apparatus 30 .
  • the bit rate information acquiring unit 101 acquires the bit rate information, and the specifying units perform communication with the corresponding units of the smart phone 10 c (the display region size specifying unit 102 performs communication with the video playing-back unit 109 , the cache amount specifying unit 103 performs communication with the temporary storage unit 108 , and the bandwidth specifying unit 104 performs communication with the segment data receiving unit 107 ), and start the specifying operation.
  • the bit rate determining unit 105 determines the bit rate, and the delivery requesting unit 106 requests the delivery of the video.
  • the delivery requesting unit 106 sets the smart phone 10 c as the delivery destination of the video, and makes the request.
  • the segment data transmitting unit 205 of the video delivering apparatus 20 transmits the segment data of the requested video to the smart phone 10 c .
  • the video of the bit rate determined by the server apparatus 30 as described above is delivered to the smart phone 10 c .
  • the server apparatus 30 determines the bit rate similarly to the smart phone 10 of the example, and thus the bit rate is used such that the playback quality is improved to be higher than when the bit rate is determined in view of only the bandwidth.
  • the information processing apparatus such as the decision apparatus that determines the bit rate or the playback apparatus that plays back the video is not limited to the smart phone or the server apparatus.
  • the information processing apparatus may be a tablet device, a laptop PC, or a desktop PC.
  • the smart phone functions as both the decision apparatus and the playback device, but the server apparatus functions as the decision apparatus but does not function as the playback device and thus requests the video delivering apparatus to deliver the video of the determined bit rate to an external playback device by streaming.
  • the video of the determined bit rate is delivered by streaming, and thus the playback quality is improved to be higher than when the bit rate is determined in view of only the bandwidth.
  • the specified information described above or the determined bit rate may be displayed on the playback apparatus that plays back the video.
  • FIG. 15 illustrates a functional configuration implemented by a smart phone 10 d according to the present modified example.
  • the smart phone 10 d includes a stream playback state display unit 114 in addition to the units illustrated in FIG. 5 .
  • the display region size specifying unit 102 , the cache amount specifying unit 103 , and the bandwidth specifying unit 104 supply specifying result information indicating specified results to the stream playback state display unit 114 , and notify the stream playback state display unit 114 of the bit rate determined by the bit rate determining unit 105 .
  • the stream playback state display unit 114 causes a stream playback state indicated by the supplied specifying result information and the bit rate that is notified to be displayed on the display unit of its own apparatus.
  • FIG. 16 illustrates an exemplary displayed stream playback state.
  • character strings such as a “display region size,” a “cache amount,” a “bandwidth,” and a “bit rate” and images D 1 , D 2 , D 3 , and D 4 indicating states are displayed on the display surface 161 of the touch screen 16 illustrated in FIG. 2 .
  • Each of the images indicates a state according to a position of a circular image.
  • the image D 1 indicates whether the display region size is large or small (as the circular image gets closer to “small” in the drawing, the size of the display region is decreased, and as the circular image gets closer to “large” in the drawing, the size of the display region is increased), and the image D 2 indicates whether the cache amount is large or small.
  • the image D 3 indicates whether the bandwidth is high or low, and the image D 4 indicates whether the bit rate is high or low.
  • the image D 2 may be an operator for receiving an operation of changing the cache amount. For example, as the circular image moves to the left, the cache amount decreases, whereas as the circular image moves to the right, the cache amount increases. As the cache amount is increased, the cache playback time is likely to become equal to or larger than the threshold value Th 3 , and thus the bit rate increases. As described above, since the stream delivery state is displayed, the user can take an action of increasing or decreasing the bit rate.
  • the present invention is not limited to this exemplary embodiment.
  • the bit rate increases, only the frame rate may increase with no change in the resolution, or only the resolution may increase with no change in the frame rate.
  • the bit rate determining unit 105 determines the bit rate selected from predetermined plurality of bit rates as the bit rate of the video to be played back, but the present invention is not limited to this exemplary embodiment. For example, when a bit rate in a set range is designated, and the video delivering apparatus has a function of performing a process of converting video of a reference bit rate that is stored to a video of a designated bit rate, the bit rate determining unit 105 may determine the bit rate within the set range.
  • the bit rate determining unit 105 may determine a value of the specified size of the display region, a value of the cache amount, or a value obtained by substituting a value of the bandwidth into a set formula as the bit rate. As a result, the stream delivery of the videos of the bit rates of more steps than when the plurality of bit rates are determined in advance is performed.
  • the present invention is applicable as the video delivering apparatus or the video delivery system including the video delivering apparatus in addition to the information processing apparatus such as the smart phone or the server apparatus.
  • the present invention is applicable as an information processing method of carrying out the process implemented by the respective apparatuses and applicable as a program causing a computer controlling the respective apparatuses to operate.
  • the program may be provided in the form of a recording medium such as an optical disk storing the program or may be provided in a form in which the program is downloaded to a computer via a communication line such as the Internet and installed for use.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Databases & Information Systems (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Human Computer Interaction (AREA)
  • Computer Graphics (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Security & Cryptography (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Information Transfer Between Computers (AREA)

Abstract

An information processing apparatus includes: a first specifying unit that specifies a size of a display region of a video delivered by streaming, the video having a bit rate that is changeable during a playback; a second specifying unit that specifies a data amount of data temporarily stored for a period of the video to be next played back; and a third specifying unit that specifies a bandwidth in reception of the data. The information processing apparatus further includes a determining unit that determines a bit rate of the video that is played back based on the size, the data amount, and the bandwidth that are specified.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS
This application is based on and claims priority under 35 USC 119 from Japanese Patent Application No. 2016-36943 filed on Feb. 29, 2016.
BACKGROUND
1. Technical Field
The present invention relates to an information processing apparatus and an information processing method.
2. Related Art
In a technique of playing back a video delivered by streaming, a playback quality such as a resolution level of a video and playback continuity (in which the playback is not interrupted) is important.
SUMMARY
According to an aspect of the invention, there is provided an information processing device, including: a first specifying unit that specifies a size of a display region of a video delivered by streaming, the video having a bit rate that is changeable during a playback; a second specifying unit that specifies a data amount of data temporarily stored for a period of the video to be next played back; a third specifying unit that specifies a bandwidth in reception of the data; and a determining unit that determines a bit rate of the video that is played back based on the size, the data amount, and the bandwidth that are specified.
BRIEF DESCRIPTION OF THE DRAWINGS
Exemplary embodiments of the present invention will be described in detail based on the following figures, wherein:
FIG. 1 is a diagram illustrating an overall configuration of a video delivery system according to an exemplary embodiment;
FIG. 2 is a diagram illustrating a stream delivery mechanism;
FIG. 3 is a diagram illustrating a hardware configuration of a smart phone;
FIG. 4 is a diagram illustrating a hardware configuration of a video delivering apparatus;
FIG. 5 is a diagram illustrating functional configurations implemented by respective apparatuses;
FIG. 6A is a diagram illustrating an exemplary video displayed on a display region;
FIG. 6B is a diagram illustrating an exemplary video displayed on a display region;
FIG. 6C is a diagram illustrating an exemplary video displayed on a display region;
FIG. 7 is a diagram illustrating an exemplary bit rate table;
FIG. 8 is a diagram illustrating an exemplary operation process of respective apparatuses in a stream delivery process;
FIG. 9 is a diagram illustrating a functional configuration implemented by a smart phone according to a modified example;
FIG. 10 is a diagram illustrating functional configurations implemented by respective apparatuses according to a modified example;
FIG. 11 is a diagram illustrating an exemplary file format table;
FIG. 12 is a diagram illustrating an overall configuration of a video delivery system according to a modified example;
FIG. 13 is a diagram illustrating a hardware configuration of a server apparatus;
FIG. 14 is a diagram illustrating functional configurations implemented by respective apparatuses according to a modified example;
FIG. 15 is a diagram illustrating a functional configuration implemented by a smart phone according to a modified example; and
FIG. 16 is a diagram illustrating an exemplary displayed stream playback state.
 DETAILED DESCRIPTION [1] Exemplary Embodiments
FIG. 1 illustrates an overall configuration of a video delivery system 1 according to an exemplary embodiment. The video delivery system 1 is a system that performs stream delivery of a video to a playback apparatus used by the user. The stream delivery of the video refers to delivery of a video by a mechanism that sequentially plays back segment data indicating a part of a video while transceiving the segment data in a playback order. The video delivery system 1 includes a communication line 2, a smart phone 10, and a video delivering apparatus 20.
The communication line 2 is a system in which exchange of data is relayed between device such as a mobile communication network or the Internet. The smart phone 10 is connected to the communication line 2 in a wireless manner, and the video delivering apparatus 20 is connected to the communication line 2 in a wired manner (it is an example, and they may be connected by any of wireless and wired manners).
The smart phone 10 is a playback device that plays back a video that is delivered by streaming in the video delivery system 1. The smart phone 10 is carried by the user and used in various places. The video delivering apparatus 20 is an information processing apparatus that stores a plurality of pieces of segment data indicating one video as a whole and performs stream delivery of a video by sequentially transmitting the segment data in the playback order. One segment data are data indicating a portion of, for example, about 1 to 10 seconds among the entire the video.
In the video delivery system 1, a video that conforms to a standard called, for example, Moving Picture Experts Group (MPEG)-Dynamic Adaptive Streaming over HTTP (DASH) and has a bit rate that can be changed during the playback thereof is delivered by streaming. In the stream delivery of the exemplary embodiment, a plurality of bit rates are determined in advance.
FIG. 2 illustrates a stream delivery mechanism. The video delivering apparatus 20 according to the exemplary embodiment stores segment data groups A1, A2, and A3 indicating videos having “high,” “intermediate,” and “low” bit rates, respectively. The bit rate refers to a data amount (having a unit of megabits per seconds (Mbps) or megabytes (MB/sec) that is transceived at intervals of one second.
In the videos indicated by the segment data groups, as the bit rate increases, the resolution and the frame rate of the video increase. The smart phone 10 designates the bit rate of the video to be delivered when requesting the stream delivery of the video. The video delivering apparatus 20 sequentially reads the segment data from the segment data group of the designated bit rate, and transmits the read segment data. The smart phone 10 temporarily stores (caches) the received segment data, and reads and plays back the cached segment data.
Further, when the smart phone 10 designates a different bit rate from that of a video being played back during the playback of the video and requests delivery of segment data of the different bit rate, the video delivering apparatus 20 receives the request, and transmits the segment data of the designated new bit rate. Upon receiving the segment data, the smart phone 10 plays back the video indicated by the segment data of the new bit rate, subsequently to the video indicated by the segment data of the current bit rate. Accordingly, the smart phone 10 displays the video in which the bit rate can be changed during the playback. One of the features of the exemplary embodiment of the present invention lies in the bit rate decision method. The decision method will be described later in detail.
FIG. 3 illustrates a hardware configuration of the smart phone 10. The smart phone 10 is a computer that includes a central processing unit (CPU) 11, a random access memory (RAM) 12, a read only memory (ROM) 13, a network interface card (NIC) 14, a flash memory 15, a touch screen 16, a speaker 17, and a sensor 18. The CPU 11 controls operations of the respective units by executing a program stored in the ROM 13 or the flash memory 15 using the RAM 12 as a work area. The NIC 14 includes an antenna, a communication circuit, and the like, and performs wireless communication that conforms to, for example, a mobile communication standard.
The flash memory 15 stores data and a program which the CPU 11 uses for control. In this exemplary embodiment, a browser program is included as the program, and a function of the exemplary embodiment of the present invention which will be described later is implemented by Java (a registered trademark) script executed on the browser. The present invention is not limited to this exemplary embodiment, and an application program that implements the function of the present invention may be stored. The touch screen 16 includes a display serving as a display unit and a touch panel installed on the surface of the display, and displays an image and receives an operation from the user. The speaker 17 converts audio data indicating a sound into an analog signal and outputs a sound. The sensor 18 is, for example, a three-axis geomagnetic sensor, and supplies data indicating an inclination of its own apparatus to the CPU 11.
FIG. 4 illustrates a hardware configuration of the video delivering apparatus 20. The video delivering apparatus 20 is a computer that includes a CPU 21, a RAM 22, a ROM 23, a NIC 24, and a hard disk drive (HDD) 25. The CPU 21 to the ROM 23 are the same hardware as the components having the same names illustrated in FIG. 3. The NIC 24 includes a communication circuit, and performs communication through the communication line 2. The HDD 25 stores data and a program which the CPU 21 uses for control.
By controlling the respective units by executing the programs through the CPUs of the respective apparatuses included in the video delivery system 1, functions to be described below are implemented.
FIG. 5 illustrates functional configurations implemented by the respective apparatuses. The smart phone 10 includes a bit rate information acquiring unit 101, a display region size specifying unit 102, a cache amount specifying unit 103, a bandwidth specifying unit 104, a bit rate determining unit 105, a delivery requesting unit 106, a segment data receiving unit 107, a temporary storage unit 108, and a video playing-back unit 109. The video delivering apparatus 20 includes a segment data storage unit 201, a bit rate information storage unit 202, a bit rate information transmitting unit 203, a segment data reading unit 204, and a segment data transmitting unit 205.
The segment data storage unit 201 of the video delivering apparatus 20 stores the segment data of the video delivered by streaming by its own apparatus. The segment data storage unit 201 stores the segment data indicating the video of the plurality of bit rates as described above with reference to FIG. 2. Since each piece of the segment data has a small communication amount, each piece of the segment data is encoded (compressed), for example, as data indicating a reference frame and a difference with the frame.
The bit rate information storage unit 202 stores bit rate information (for example, a media presentation description (MPD) in MPEG-Dash) indicating the plurality of bit rates of the video stored in the segment data storage unit 201 and delivered by streaming by its own apparatus. In the example of FIG. 2, the bit rate information storage unit 202 stores the bit rate information indicating the bit rate of the video indicated by each of the segment data groups A1, A2, and A3. For example, when the segment data of the video are stored by an operator of the video delivering apparatus 20, the bit rate information is created and stored by the operator.
In the case of requesting the stream delivery of the video, the bit rate information acquiring unit 101 of the smart phone 10 acquires the bit rate information indicating the plurality of bit rates of the video. For example, a link to the video that is delivered by streaming by the video delivering apparatus 20 is attached in a web page displayed on the smart phone 10, and when the user performs an operation of selecting the link, the bit rate information acquiring unit 101 transmits request data for requesting the bit rate information of the video to the video delivering apparatus 20. The request data include information (for example, a uniform resource locator (URL) of a file (for example, an MPD in MPEG-DASH)) specifying the video.
Upon receiving the request data transmitted from the smart phone 10, the bit rate information transmitting unit 203 of the video delivering apparatus 20 reads the bit rate information of the video specified by the request data, that is, the bit rate information of the video stored in its own apparatus from the bit rate information storage unit 202 and transmits the read bit rate information of the video to the request source (the smart phone 10 in this exemplary embodiment). The bit rate information acquiring unit 101 acquires the bit rate information transmitted as described above. The bit rate information acquiring unit 101 supplies the acquired bit rate information to the bit rate determining unit 105.
The bit rate determining unit 105 determines any one of the plurality of bit rates indicated by the bit rate information acquired by the bit rate information acquiring unit 101 as the bit rate of the video to be played back. The bit rate decision method will be described later in detail. The bit rate determining unit 105 repeatedly determines the bit rate at determined time intervals. For example, an interval of a time (for example, intervals of one second when a playback time is 2 seconds or the like) shorter than a playback time of the video indicated by one piece of segment data is used as the time interval. Each time the bit rate is determined, the bit rate determining unit 105 notifies the delivery requesting unit 106 of the determined bit rate.
The delivery requesting unit 106 requests an apparatus (the video delivering apparatus 20 in this exemplary embodiment) that delivers the video by streaming to deliver the video at the bit rate determined by the bit rate determining unit 105. When the bit rate determined by the bit rate determining unit 105 is changed, the bit rate of the video that is delivered by streaming by the delivery requesting unit 106 is also changed.
When the stream delivery of the video is requested from an external apparatus, the segment data reading unit 204 of the video delivering apparatus 20 reads the segment data of the requested video. Upon receiving designation data transmitted from the smart phone 10, the segment data reading unit 204 reads a set number of pieces of segment data from the segment data group of the bit rate designated by the designation data in order from the beginning of the video.
Thereafter, the segment data reading unit 204 repeatedly performs the process of reading a set number of pieces of segment data subsequently to the read segment data at intervals of a playback time of a part of the video indicated by the segment data, which is read one time, or a time shorter than the playback time of the part. When the designated bit rate is changed in the middle, the segment data reading unit 204 reads a set number of pieces of segment data from a part from which a playback position is continued among the segment data of the new bit rate. Each time the segment data are read, the segment data reading unit 204 supplies the read segment data to the segment data transmitting unit 205.
The segment data transmitting unit 205 transmits the supplied segment data to the designation data transmission source, that is, the request source (the smart phone 10 in this exemplary embodiment) that has requested the stream delivery of the video. The segment data receiving unit 107 of the smart phone 10 receives the segment data transmitted from the video delivering apparatus 20. The segment data receiving unit 107 supplies the received segment data to the temporary storage unit 108.
The temporary storage unit 108 temporarily stores the segment data received by the segment data receiving unit 107 until the part of the video indicated by the segment data is played back. When the segment data are stored in the temporary storage unit 108, the video playing-back unit 109 sequentially reads the stored segment data, and sequentially plays back the video indicated by the read segment data. Since the segment data are encoded as described above, the video playing-back unit 109 decodes and plays back the segment data. The video playing-back unit 109 causes the played-back video to be displayed on a display region but causes the video to be displayed on a different display region according to a direction of its own apparatus.
FIGS. 6A to 6C illustrate an exemplary video displayed on the display region. In FIGS. 6A to 6C, the video is displayed on a display surface 161 of the touch screen 16 illustrated in FIG. 2. The display surface 161 is a rectangular surface surrounded by long sides 162 and short sides 163. FIG. 6A illustrates a display region B1 when the smart phone 10 is disposed so that the long side 162 of the display surface 161 is along the vertical direction, and FIG. 6B illustrates a display region B2 when the smart phone 10 is disposed so that the long side 162 of the display surface 161 is along the horizontal direction. A video C1 in which a person is shown is displayed on both of the display regions B1 and B2. As described above, the video playing-back unit 109 causes the video to be displayed on the display region B1 or B2 according to an inclination of its own device.
The display region size specifying unit 102 of the smart phone 10 specifies the size of the display region of the video that is delivered by streaming. The display region size specifying unit 102 is an example of a “first specifying unit” according to the exemplary embodiment of the present invention. The display region size specifying unit 102 transmits an inquiry about a type of the display region (one of the display regions B1 and B2) in which the video is displayed to the video playing-back unit 109. Upon receiving the inquiry, the video playing-back unit 109 notifies the display region size specifying unit 102 of the type of the display region in which the video is displayed. The display region size specifying unit 102 specifies the size corresponding to the type of the display region that is notified of as the size of the display region.
When the display region B1 is notified of as the type of the display region, the display region size specifying unit 102 specifies a value obtained by multiplying a length L1 of the long side by a length L2 of the short side illustrated in FIG. 6A, that is, an area of the display region as the size of the display region. Further, when the display region B2 is notified of as the type of the display region, the display region size specifying unit 102 specifies a value obtained by multiplying a length L3 of the long side by a length L4 of the short side illustrated in FIG. 6B as the size of the display region.
The short side of the display region B1 contacts with the long side 162 of the display surface 161, and the short side of the display region B2 contacts with the short side 163 of the display surface 161, but the size and the arrangement of the display region are not limited to this example. The video playing-back unit 109 may cause the video C1 to be displayed on a display region B3 in which none of the long side and the short side contacts with the long side 162 and the short side 163 of the display surface 161, for example, as illustrated in FIG. 6C. In this case, the display region size specifying unit 102 specifies a value obtained by multiplying a length L5 of the long side and a length L6 of the short side of the display region B3 as the display region size. The display region size specifying unit 102 supplies size information (information indicating the area of the display region in this exemplary embodiment) indicating the size of the display region specified as described above to the bit rate determining unit 105.
The cache amount specifying unit 103 of the smart phone 10 specifies a cache amount in the stream playback, that is, a data amount of the segment data temporarily stored for a period of the video to be next played back, in the video that is played back by streaming. The cache amount specifying unit 103 is an example of a “second specifying unit” according to the exemplary embodiment of the present invention. The cache amount specifying unit 103 refers to the segment data temporarily stored in the temporary storage unit 108, and specifies a playback time in which the segment data that are referred to is played back as the cache amount. For example, when the cached segment data indicate a video of 2 seconds, and 0.5 seconds have been already played back, the cache amount specifying unit 103 specifies that the video of 1.5 seconds is cached, and supplies information indicating 1.5 seconds to the bit rate determining unit 105 as cache amount information.
The bandwidth specifying unit 104 of the smart phone 10 specifies a bandwidth in reception of the segment data of the video delivered by streaming. The bandwidth specifying unit 104 is an example of a “third specifying unit” according to the exemplary embodiment of the present invention. The bandwidth specifying unit 104 monitors the segment data receiving unit 107, and specifies a data size of the segment data received per unit time by the segment data receiving unit 107 as the bandwidth. For example, when the segment data of 10 Mbps are received for one second, the segment data receiving unit 107 specifies 10 Mbps as the bandwidth. The segment data receiving unit 107 supplies bandwidth information indicating the specified bandwidth to the bit rate determining unit 105.
The display region size specifying unit 102, the cache amount specifying unit 103, and the bandwidth specifying unit 104 perform the specifying process as the same time intervals as the time intervals at which the bit rate determining unit 105 repeatedly determines the bit rate in this exemplary embodiment.
The bit rate determining unit 105 determines the bit rate of the video that is played back based on the size of the display region specified by the display region size specifying unit 102, the cache amount specified by the cache amount specifying unit 103, and the bandwidth specified by the bandwidth specifying unit 104. For example, the bit rate determining unit 105 uses a bit rate table in which the size of the display region is associated with a bit rate range.
FIG. 7 illustrates an exemplary bit rate table. In an example of FIG. 7, five bit rates such as 4.0 Mbps, 2.0 Mbps, 1.0 Mbps, 0.5 Mbps, and 0.3 Mbps are assumed to be defined for the stream delivery. In this case, a bit rate ranges of “2.0, 1.0, 0.5, 0.3” (whose unit is all Mbps) is associated with a size of the display region of “less than threshold value Th1.” A bit rate range of “4.0, 2.0, 1.0, 0.5” is associated with a size of the display region of “equal to or larger than threshold value Th1 and less than threshold value Th2,” and a bit rate range of “4.0, 2.0, 1.0” is associated with a size of the display region of “equal to or larger than threshold value Th2.”
The bit rate determining unit 105 reads the bit rate range associated with the size of the display region specified by the display region size specifying unit 102 from the bit rate table. The bit rate determining unit 105 narrows the read bit rate range down to a range that does not exceed the bandwidth specified by the bandwidth specifying unit 104. For example, when the size of the display region less than the threshold value Th1 is specified, the bit rate determining unit 105 reads the bit rate range of “2.0, 1.0, 0.5, 0.3,” and when the bandwidth of 1.2 Mbps is specified, since the bit rate of 2.0 Mbps exceeds the specified bandwidth, the bit rate determining unit 105 narrows the bit rate range down to “1.0, 0.5, 0.3.”
When a playback time (hereinafter, referred to as a “cache playback time”), for which the video indicated by the temporarily stored segment data having the cache amount specified by the cache amount specifying unit 103 is played back, is equal to or larger than a set threshold value Th3, the bit rate determining unit 105 selects any one of the bit rates of up to a set number in a descending order within the narrowed-down range (i.e., selects any one of the bit rate(s) from a set number of highest bit rates within the narrowed-down range), and determines the selected bit rate as the bit rate of the video that is played back. In this exemplary embodiment, the bit rate determining unit 105 regards 1 as the set number, that is, selects the highest bit rate and determines the highest bit rate as the bit rate of the video that is played back. When the narrowed-down range is “1.0, 0.5, 0.3” as in the above example, the bit rate determining unit 105 determines “1.0” which is the highest bit rate among them as the bit rate of the video to be played back.
Further, when the cache playback time is less than the threshold value Th3, the bit rate determining unit 105 selects any one of the bit rates from a set number in an ascending order within the narrowed-down range (i.e., selects any one of the bit rate(s) from a set number of lowest bit rates within the narrowed-down range), and determines the selected bit rate as the bit rate of the video that is played back. In this exemplary embodiment, a number that causes all the bit rates excluding the highest bit rate to be included is set. Thus, the bit rate determining unit 105 selects any one of the bit rates excluding the highest one in the narrowed-down range, and determines the selected bit rate as the bit rate of the video that is played back.
When the narrowed-down range is “1.0, 0.5, 0.3” as in the above-described example, the bit rate determining unit 105 selects any one of “0.5, 0.3” excluding “1.0” which is the highest bit rate among them, and determines the selected bit rate as the bit rate of the video to be played back. When any one of two or more bit rates is selected, for example, the bit rate determining unit 105 may consistently select the highest bit rate among them when importance is placed on the resolution of the video or may consistently select the lowest bit rate among them when importance is placed on the continuity of the video.
However, immediately after a video playback operation is received, since the segment data are not received yet, the bandwidth is not specified by the bandwidth specifying unit 104 yet, and the segment data are not cached yet, and thus the cache amount is neither specified by the cache amount specifying unit 103. In this regard, when the playback is performed from the beginning of the video, the bit rate determining unit 105 determines the bit rate without using the cache amount specified by the cache amount specifying unit 103 and the bandwidth specified by the bandwidth specifying unit 104 until an initial response period set as a period in which the decision method at the initial stage of the playback is used elapses. In other words, the bit rate determining unit 105 determines the bit rate based on only the size of the display region specified by the display region size specifying unit 102 until the initial response period elapses.
For example, a period until the cache amount and the bandwidth are initially specified is set as the initial response period. Further, a period in which the specifying of the cache amount and the bandwidth is expected to be completed, which is set based on a record of a time required for the specifying, may be used as the initial response period. In the initial response period, for example, the bit rate determining unit 105 selects any one of the bit rates in the bit rate range which is associated with the specified size of the display region in the bit rate table illustrated in FIG. 7, and determines the selected bit rate as the bit rate of the video to be played back.
The bit rate determining unit 105 may select the highest bit rate from the range when the importance is placed on the resolution of the video or may select the lowest bit rate from the range when the importance is placed on the continuity of the video. The present invention is not limited to this example, and, for example, when the frame rate desired by the user is set in advance, the bit rate determining unit 105 may select a bit rate of a value obtained by multiplying the set frame rate by the specified size of the display region and a coefficient or a bit rate of a value closest to the value and determine the selected bit rate as the bit rate of the video to be played back.
The respective apparatuses included in the video delivery system 1 perform a stream delivery process on the video based on the above-described configuration.
FIG. 8 illustrates an exemplary operation process of the respective apparatuses in the stream delivery process. In an example of FIG. 8, when the user performs an operation (for example, an operation of selecting a link of the video) of causing the smart phone 10 to play back the video that is delivered by streaming, the operation process starts.
First, the smart phone 10 receives the video playback operation (step S11). Then, the smart phone 10 (the bit rate information acquiring unit 101) requests the video delivering apparatus 20 to transmit the bit rate information of the video for which the playback operation is received (step S12). The video delivering apparatus 20 (the bit rate information transmitting unit 203) reads the requested bit rate information of the video, and transmits the read bit rate information of the video to the smart phone 10 (step S13). The smart phone 10 (the bit rate information acquiring unit 101) acquires the transmitted bit rate information (step S14).
Then, the smart phone 10 (the display region size specifying unit 102) specifies the size of the display region of the video for which the playback operation is received (step S15). Then, the smart phone 10 (the bit rate determining unit 105) determines the bit rate of the video according to the specified size (step S16). Then, the smart phone 10 (the delivery requesting unit 106) requests the video delivering apparatus 20 to perform the stream delivery of the video at the bit rate determined in step S16 (step S17).
The video delivering apparatus 20 (the segment data reading unit 204) reads the segment data indicating the video of the bit rate requested in step S17 (step S21). Then, the video delivering apparatus 20 (the segment data transmitting unit 205) transmits the segment data read in step S21 to the smart phone 10 (step S22). The smart phone 10 (the segment data receiving unit 107) receives the segment data transmitted in step S22. The smart phone 10 (the temporary storage unit 108) temporarily stores the received segment data.
Then, the smart phone 10 (the video playing-back unit 109) reads and plays back the segment data temporarily stored, and causes the segment data to be displayed on the display region (step S23). Then, the smart phone 10 (the display region size specifying unit 102) specifies the size of the display region of the played-back video (step S24). Then, the smart phone 10 (the cache amount specifying unit 103) specifies a data amount of the segment data temporarily stored, that is, the cache amount (step S25).
Then, the smart phone 10 (the bandwidth specifying unit 104) specifies the bandwidth in the reception of the segment data (step S26). Then, the smart phone 10 (the bit rate determining unit 105) determines the bit rate based on the size of the display region specified in step S24, the cache amount specified in step S25, and the bandwidth specified in step S26 (step S27). The smart phone 10 (the delivery requesting unit 106) requests the video delivering apparatus 20 to deliver the video of the bit rate determined in step S27 (step S28).
The video delivering apparatus 20 (the segment data reading unit 204) returns to step S21, and reads the segment data indicating the video of the bit rate requested in step S28. Thereafter, the operation of steps S21 to S28 is repeated until the user ends the playback operation or the video is played back up to the end, and so the playback of the video ends.
In the playback of the video, unless the bit rate is changed, as the display region decreases, the resolution of the video increases. In other words, when the display region is small, although the bit rate is low, the decrease in the resolution of the video is suppressed compared to when the display region is large. In this exemplary embodiment, since the bit rate corresponding to the specified size of the display region is determined, compared to when the size of the display region is not considered, a phenomenon that, when the display region is small, the video of the resolution higher than necessary is displayed, and the wireless communication band is compressed is suppressed.
In this exemplary embodiment, the bit rate is determined based on the specified cache amount as well. For example, when the cache amount is relatively large (when the cache playback time is equal to or larger than the threshold value Th3), a high bit rate is selected, and when the cache amount is relatively small (when the cache playback time is less than the threshold value Th3), a low bit rate is selected.
When the cache amount is relatively large, a time taken to download the segment data is longer than when the cache amount is relatively small, but the playback of the video is more unlikely to be interrupted, and thus when the high bit rate is selected as described above, the resolution of the video is improved while maintaining the continuity of the video at a predetermined level. At least the playback quality of the video is improved to be higher than when the cache amount (the data amount of the segment data temporarily stored) is not considered. Here, the playback quality refers to a comprehensive quality including the level of the resolution of the video and the playback continuity (in which the playback is not interrupted).
On the other hand, when the cache amount is relatively small, if a time taken to download the segment data is long, the playback of the video is more likely to be interrupted than when the cache amount is relatively large, and thus when the low bit rate is selected as described above, it takes a time to receive new segment data, but the playback of the video indicated by the data temporarily stored is unlikely to be interrupted. In other words, the situation in which the playback of the video indicated by the segment data temporarily stored ends before the reception of the new segment data ends, and thus the video is interrupted is more likely to occur than when the bit rate is selected by a different method from that of this exemplary embodiment.
Further, in this exemplary embodiment, the bit rate is determined based on the specified bandwidth as well. When the bandwidth is high, and the video is unlikely to be interrupted, the high bit rate is selected so that the resolution of the video is increased, and the playback quality of the video is improved by improving the resolution of the video, whereas when the bandwidth is low, the low bit rate is selected so that the video is unlikely to be interrupted, and the playback quality of the video is improved by improving the continuity of the video.
In this exemplary embodiment, the bit rate is determined based on the size of the display region and the cache amount in addition to the bandwidth as described above, and thus the playback quality is improved to be higher than, for example, when the bit rate is determined in view of only the bandwidth. Further, in this exemplary embodiment, in the initial response period, the bit rate is determined without using the cache amount and the bandwidth. As a result, the bit rate is determined even in the period in which the cache amount and the bandwidth are not specified yet, and thus the playback quality of the video is improved to be higher than when the bit rate is determined without considering the size of the display region in the initial response period.
[2] Modified Example
The above exemplary embodiment is an example of the present invention, and modifications can be made as follows. The exemplary embodiment and modified examples may be carried out in combination with each other as necessary.
[2-1] Bit Rate Decision Method
The bit rate decision method by the bit rate determining unit 105 is not limited to the above-described example. For example, the bit rate determining unit 105 determines the bit rate that leads an increase in the data amount of the segment data temporarily stored in the temporary storage unit 108 among the bit rates in the bit rate range narrowed down similarly to the example as the bit rate of the video to be played back.
For example, in the state in which the stream delivery is being performed at the bit rate of 0.5 Mbps, the bit rate range is assumed to be narrowed down to “1.0, 0.5, 0.3” (Mbps) based on the size of the display region and the bandwidth using the bit rate table illustrated in FIG. 7. Since the data amount is decreased when 0.3 Mbps among them is selected, the data amount does not change when 0.5 Mbps is selected, but the data amount is increased when 1.0 Mbps is selected, the bit rate determining unit 105 selects 1.0 Mbps and determines 1.0 Mbps as the bit rate of the video to be played back.
For example, when there are a plurality of bit rates that lead the increase in the data amount, the bit rate determining unit 105 may select the highest bit rate and determine the highest bit rate as the bit rate of the video to be played back when the importance is placed on the resolution of the video or may select the lowest bit rate and determine the lowest bit rate as the bit rate of the video to be played back when the importance is placed on the continuity of the video. Using this decision method, the data amount of the segment data stored in the temporary storage unit 108 is increased due to the change in the bit rate, and thus the playback of the video is more unlikely to be interrupted than when the bit rate that does not lead to the increase in the data amount is selected.
The bit rate determining unit 105 may determine any one of the bit rates that lead to an increase in the playback time by the segment data temporarily stored in the temporary storage unit 108 in the above-described range as the bit rate of the video to be played back. In the case in which an increase in the bit rate leads to an increase in the frame rate and an increase in the resolution as in the example, if the number of frames is increased, the data amount is mostly increased as well, but, for example, there are cases in which the bit rate that does not lead to the increase in the data amount by decreasing the resolution of the video instead of increasing the number of frames is used as well. In this case, using this decision method, since the bit rate at which the data amount is not increased, but the number of frames is increased is determined, the playback of the video is more unlikely to be interrupted, and the communication load is likely to be reduced.
[2-2] Playback Speed
The bit rate determining unit 105 may change the bit rate decision method according to the playback speed of the video. In the present modified example, the level of the bit rate is assumed to correspond to at least one of the level of the resolution of the video and the level of the frame rate of the video. Specifically, as the bit rate is increased, either or both of the resolution and the frame rate is increased. For example, the resolution may not change in a certain bit rate range although the bit rate is increased, for example, when the bit rate is increased to 0.5 Mbps, 1.0 Mbps, 2.0 Mbps, and 4.0 Mbps, the resolution may be increased to 640×360, 854×480, 854×480, and 1280×720.
FIG. 9 illustrates a functional configuration implemented by a smart phone 10 a according to the present modified example. The smart phone 10 a includes a playback speed change operation receiving unit 110 in addition to the units illustrated in FIG. 5. The playback speed change operation receiving unit 110 receives an operation of changing the playback speed of the video that is being played back by streaming. Examples of the operation of changing the playback speed include a fast-forward operation, a slow playback operation, and a rewind operation. Upon receiving the playback speed changing operation, the playback speed change operation receiving unit 110 notifies the bit rate determining unit 105 of content of the received operation.
The bit rate determining unit 105 determines the bit rate based on the content of the operation that is reported from the playback speed change operation receiving unit 110. For example, when the video is played back at a speed (first-forward playback), the bit rate determining unit 105 determines the bit rate in which the resolution or the frame rate is lower than that at the time of the normal playback. For example, when the bit rate range is narrowed down to the three bit rates of 4.0 Mbps, 2.0 Mbps, and 1.0 Mbps according to the size of the display region and the bandwidth, and the cache amount is equal to or larger than the threshold value Th3, the bit rate determining unit 105 is assumed to select the highest bit rate (4.0 Mbps) in the normal playback and determines the highest bit rate (4.0 Mbps) as the bit rate of the video to be played back.
In this case, when the notification indicating that the fast-forward playback is performed is received from the playback speed change operation receiving unit 110, the bit rate determining unit 105 selects the second highest bit rate (2.0 Mbps) or the third highest bit rate (1.0 Mbps) in which the resolution and the frame rate are lower than at the time of the normal playback and determines the selected bit rate as the bit rate of the video to be played back. Since the user mostly performs the fast-forward playback when the user search for a scene which the user wants to view, in this case, it is desirable that a scene that is played back can be roughly understood, and thus the high resolution and the high frame rate are mostly not necessary.
Using this decision method, at the time of the fast-forward playback, the resolution and the frame rate are lower than at the time of the normal playback, but the user does not feel uncomfortable mostly due to the above reason. On the other hand, the communication load of the segment data is reduced to be smaller, and it is easier to secure a band for other stream delivery than when the normal playback is performed, and the bit rate decision method is not changed. Further, compared to when the normal playback is performed, and the bit rate decision method is not changed, a time per unit time required to receive the segment data is shorter, and the playback time by the segment data temporarily stored when the normal playback is performed after the fast-forward playback ends is longer, and thus the playback of the video is unlikely to be interrupted after the fast-forward playback ends. As a result, the playback quality is improved to be higher than when the normal playback is performed, and the bit rate decision method is not changed.
The bit rate determining unit 105 may determine the bit rate based on a magnification ratio in the fast-forward playback. For example, the bit rate determining unit 105 determines the bit rate such that the bit rate that decreases as the magnification ratio increases is selected. As the magnification ratio increases, the data amount of the transmitted segment data increases, and thus a time required until the reception is completed increases, and the playback is likely to be interrupted. In this regard, since the bit rate is determined based on the magnification ratio of the fast-forward playback, the fast-forward playback is more unlikely to be interrupted, and the playback quality of the video is improved to be higher than when the magnification ratio is not considered.
Further, when the video is played back at a slow speed (slow playback), the bit rate determining unit 105 determines the bit rate in which the resolution or the frame rate is higher than at the time of the normal playback. For example, when the bit rate range is narrowed down to the three bit rates of 4.0 Mbps, 2.0 Mbps, and 1.0 Mbps according to the size of the display region and the bandwidth, and the cache amount is equal to or larger than the threshold value Th3, the bit rate determining unit 105 is assumed to select the lowest bit rate (1.0 Mbps) in the normal playback and determines the lowest bit rate (1.0 Mbps) as the bit rate of the video to be played back.
In this case, when the notification indicating that the slow playback is performed is received from the playback speed change operation receiving unit 110, the bit rate determining unit 105 selects the second highest bit rate (2.0 Mbps) or the highest bit rate (4.0 Mbps) in which the resolution and the frame rate are higher than at the time of the normal playback and determines the selected bit rate as the bit rate of the video to be played back. Since the user performs the slow playback when the user wants to view the scene more carefully, it is desirable that the resolution and the frame rate be high. Using this decision method, at the time of the slow playback, the video in which the resolution or the frame rate is higher than at the time of the normal playback is played back. As a result, the playback quality is improved to be higher than when the normal playback is performed, and the bit rate decision method is not changed.
The bit rate determining unit 105 may determine the bit rate based on the magnification ratio in the slow playback. For example, the bit rate determining unit 105 determines the bit rate such that the bit rate that increases as the magnification ratio decreases (that is, the speed decreases) is selected. In the case of the slow playback, as the magnification ratio decreases, a period until a next frame is played back increases, and thus although the bit rate is increased, the playback (the slow playback) of the video is unlikely to be interrupted. In this regard, since the bit rate is determined based on the magnification ratio of the slow playback, the resolution or the frame rate of the video and the playback quality of the video are improved to be higher than when the magnification ratio is not considered.
[2-3] Acquisition of Still Image
A mechanism of acquiring a still image indicating a certain scene of the video that is played back may be provided.
FIG. 10 illustrates functional configurations implemented by the respective apparatuses according to the present modified example. In an example of FIG. 10, a smart phone 10 b including a still image requesting unit 111 and a still image display unit 112 in addition to the units illustrated in FIG. 5 is illustrated. Further, a video delivering apparatus 20 b including a still image generating unit 206 and a still image transmitting unit 207 in addition to the units illustrated in FIG. 5 is illustrated.
When the user performs an operation for acquiring a still image of a certain scene of the video that is played back, the still image requesting unit 111 of the smart phone 10 b requests the video delivering apparatus 20 b to transmit the still image of the scene. In further detail, when an operation for acquiring a still image of a certain scene of the video is performed while the playback of the video is being performed at a first bit rate, the still image requesting unit 111 requests the still image of the scene in a video of a second bit rate higher than the first bit rate.
For example, when the first bit rate (the bit rate of the video during the playback) is 1.0 Mbps, the still image requesting unit 111 requests the still image using a bit rate that is one step higher than the first bit rate as the second bit rate. The second bit rate is not limited to this example and may be the highest bit rate of the video that is delivered by streaming.
The still image requesting unit 111 makes the request by transmitting request data indicating the video in which the still image is requested, the scene of the still image, and the second bit rate to the video delivering apparatus 20 b. For example, a URL of the video is used as information specifying the video, and for example, a playback time of the scene is used as information specifying the scene of the still image.
Upon receiving the transmitted request data, the still image generating unit 206 of the video delivering apparatus 20 b generates the still image of the scene indicated by the request data with reference to the segment data of the video of the bit rate indicated by the request data. The still image generating unit 206 decodes the segment data including the target scene, and generates the still image of the scene. For example, the still image generating unit 206 generates the still image at a predetermined file format (joint photographic experts group (JPEG), portable network graphics (PNG), or the like).
When the still image requested from the still image requesting unit 111 is generated as described above, the still image generating unit 206 supplies the generated still image to the still image transmitting unit 207. The still image transmitting unit 207 transmits the supplied still image, that is, the still image requested from the still image requesting unit 111 to the smart phone 10 b serving as the request source apparatus. The still image display unit 112 of the smart phone 10 b causes the still image transmitted from the video delivering apparatus 20 b to be displayed on the display unit (the display of the touch screen 16) of its own apparatus.
As the bit rate increases, the resolution mostly increases as well. For this reason, in the present modified example, the still image of the video of the second bit rate is requested, and thus the resolution of the still image to be displayed is more easily increased than when the still image of the video of the first bit rate is requested. On the other hand, the data size of the still image is smaller than the video that is delivered by streaming, and thus even when the resolution is increased, the communication load is small, and even when the still image having the high resolution is acquired, influence on the communication is suppressed to be small.
In the above example, the still image transmitting unit 207 transmits the still image generated from the segment data, but the present invention is not limited to this example, and, for example, data of a still image of each scene may be stored in a storage unit of its own apparatus in advance, and the requested still image of the scene may be read from the storage unit and transmitted. The number of formats of the still image to be transmitted is not limited to one, and two or more formats (for example, both JPEG and PNG or the like) may be used, and any one of the formats may be selected by the user. Further, when two or more still images are transmitted, the still image transmitting unit 207 may transmit the still images at the same time or different time (different timings).
[2-4] File Format of Still Image
In the above example, the file format of the still image is set in advance, but the present invention is not limited to this example, and the file format of the still image may be variable. For example, the still image requesting unit 111 requests the still image of the file format corresponding to the bandwidth specified by the bandwidth specifying unit 104. For this request, the still image requesting unit 111 uses, for example, a file format table in which the bandwidth is associated with the file format.
FIG. 11 illustrates an exemplary file format table. In an example of FIG. 11, “JPEG” is associated with the bandwidth of “less than threshold value Th11,” and “PNG” is associated with the bandwidth of “equal to or larger than threshold value Th11.” When the bandwidth is specified, the bandwidth specifying unit 104 supplies the bandwidth information indicating the specified bandwidth to the still image requesting unit 111. The still image requesting unit 111 requests the video delivering apparatus 20 b to transmit the still image of the file format that is associated with the bandwidth indicated by the supplied bandwidth information in the file format table.
In the present modified example, in a situation in which the bandwidth is high (equal to or larger than the threshold value Th11), and it is easy to perform communication of data having a large size, a still image of a file format of a lossless compression format such as PNG is requested, and thus a communication amount is large, but an excellent image quality is secured even when the displayed still image is enlarged. On the other hand, in a situation in which the bandwidth is low (less than the threshold value Th11), and it is not easy to perform communication of data having a large size, a still image of a file format of a lossy compression format such as JPEG is requested, and thus an image quality is bad when the displayed still image is enlarged, but the communication amount is small, and the time taken until the still image is displayed is reduced. As described above, in the present modified example, the still image of the file format suitable for the bandwidth is displayed.
In addition to the bandwidth, for example, the file format may be set according to the purpose of the still image. For example, when the purpose is to enable the general user to view a video of a movie, a television, or the like, JPEG is desirable, but when the purpose is to check a video of a monitoring camera, the video may be enlarged in order to discern an object that is shown in the video, PNG in which a fine image can be expressed by a lossless compression scheme is used. In this case, for example, the purpose of the video is set in the smart phone 10 b through the operation of the user or the like, and the still image requesting unit 111 requests the video delivering apparatus 20 b to transmit the still image of the file format corresponding to the set purpose of the video. Thus, the still image of the file format suitable for the purpose of a video is displayed.
Further, the user may select the bit rate and the file format serving as the basis of the still image. In this case, the still image requesting unit 111 displays a selection screen in which the bit rate and the file format are selected, and requests the video delivering apparatus 20 b to transmit the still image of the bit rate and the file format selected by the operation performed on the selection screen. Thus, the still image of the file format desired by the user is displayed.
[2-5] Server Apparatus
The bit rate may be determined by a server apparatus.
FIG. 12 illustrates an overall configuration of a video delivery system 1 c according to the present modified example. The video delivery system 1 c includes a communication line 2, a smart phone 10 c, a video delivering apparatus 20, and a server apparatus 30.
FIG. 13 illustrates a hardware configuration of the server apparatus 30. The server apparatus 30 is a computer that includes a CPU 31, a RAM 32, a ROM 33, a NIC 34, and a HDD 35. The CPU 31 to the HDD 35 are the same hardware as the components having the same names illustrated in FIG. 4.
FIG. 14 illustrates functional configurations implemented by the respective apparatuses according to the present modified example. The smart phone 10 c includes a playback operation notifying unit 113 in addition to the segment data receiving unit 107, the temporary storage unit 108, and the video playing-back unit 109 illustrated in FIG. 5. The server apparatus 30 includes the bit rate information acquiring unit 101, the display region size specifying unit 102, the cache amount specifying unit 103, the bandwidth specifying unit 104, the bit rate determining unit 105, and the delivery requesting unit 106 illustrated in FIG. 5.
When the operation of playing back the video that is delivered by streaming is performed on its own apparatus, the playback operation notifying unit 113 of the smart phone 10 c gives a notification indicating that the operation is performed to the server apparatus 30. The notification is received by the bit rate information acquiring unit 101, the display region size specifying unit 102, the cache amount specifying unit 103, and the bandwidth specifying unit 104 of the server apparatus 30. When the notification is received, the bit rate information acquiring unit 101 acquires the bit rate information, and the specifying units perform communication with the corresponding units of the smart phone 10 c (the display region size specifying unit 102 performs communication with the video playing-back unit 109, the cache amount specifying unit 103 performs communication with the temporary storage unit 108, and the bandwidth specifying unit 104 performs communication with the segment data receiving unit 107), and start the specifying operation.
Thus, the bit rate determining unit 105 determines the bit rate, and the delivery requesting unit 106 requests the delivery of the video. At this time, the delivery requesting unit 106 sets the smart phone 10 c as the delivery destination of the video, and makes the request. The segment data transmitting unit 205 of the video delivering apparatus 20 transmits the segment data of the requested video to the smart phone 10 c. The video of the bit rate determined by the server apparatus 30 as described above is delivered to the smart phone 10 c. In the present modified example, the server apparatus 30 determines the bit rate similarly to the smart phone 10 of the example, and thus the bit rate is used such that the playback quality is improved to be higher than when the bit rate is determined in view of only the bandwidth.
[2-6] Information Processing Apparatus
The information processing apparatus such as the decision apparatus that determines the bit rate or the playback apparatus that plays back the video is not limited to the smart phone or the server apparatus. For example, the information processing apparatus may be a tablet device, a laptop PC, or a desktop PC. The smart phone functions as both the decision apparatus and the playback device, but the server apparatus functions as the decision apparatus but does not function as the playback device and thus requests the video delivering apparatus to deliver the video of the determined bit rate to an external playback device by streaming.
Even in the case of the information processing apparatus that is used in a stationary state such as the desktop PC, the display region size, the cache amount, and the bandwidth change, and thus similarly to the above example, the video of the determined bit rate is delivered by streaming, and thus the playback quality is improved to be higher than when the bit rate is determined in view of only the bandwidth.
[2-7] Display of Specified Information
The specified information described above or the determined bit rate may be displayed on the playback apparatus that plays back the video.
FIG. 15 illustrates a functional configuration implemented by a smart phone 10 d according to the present modified example. The smart phone 10 d includes a stream playback state display unit 114 in addition to the units illustrated in FIG. 5. In the present modified example, the display region size specifying unit 102, the cache amount specifying unit 103, and the bandwidth specifying unit 104 supply specifying result information indicating specified results to the stream playback state display unit 114, and notify the stream playback state display unit 114 of the bit rate determined by the bit rate determining unit 105.
The stream playback state display unit 114 causes a stream playback state indicated by the supplied specifying result information and the bit rate that is notified to be displayed on the display unit of its own apparatus.
FIG. 16 illustrates an exemplary displayed stream playback state. In an example of FIG. 16, character strings such as a “display region size,” a “cache amount,” a “bandwidth,” and a “bit rate” and images D1, D2, D3, and D4 indicating states are displayed on the display surface 161 of the touch screen 16 illustrated in FIG. 2. Each of the images indicates a state according to a position of a circular image.
The image D1 indicates whether the display region size is large or small (as the circular image gets closer to “small” in the drawing, the size of the display region is decreased, and as the circular image gets closer to “large” in the drawing, the size of the display region is increased), and the image D2 indicates whether the cache amount is large or small. The image D3 indicates whether the bandwidth is high or low, and the image D4 indicates whether the bit rate is high or low. By viewing the stream playback state, for example, the user notices that the bandwidth is low, and the bit rate is low and takes an action of moving to a place in which a state of the radio wave is good. If the user understands that as the display region size decreases, the bit rate decreases, the user takes an action of increasing the bit rate by increasing the display region size.
The image D2 may be an operator for receiving an operation of changing the cache amount. For example, as the circular image moves to the left, the cache amount decreases, whereas as the circular image moves to the right, the cache amount increases. As the cache amount is increased, the cache playback time is likely to become equal to or larger than the threshold value Th3, and thus the bit rate increases. As described above, since the stream delivery state is displayed, the user can take an action of increasing or decreasing the bit rate.
[2-8] Bit Rate
In this exemplary embodiment, as the bit rate increases, the resolution of the video increases, and the frame rate increases as well, but the present invention is not limited to this exemplary embodiment. For example, although the bit rate increases, only the frame rate may increase with no change in the resolution, or only the resolution may increase with no change in the frame rate.
[2-9] Bit Rate Determining Unit
In this exemplary embodiment, the bit rate determining unit 105 determines the bit rate selected from predetermined plurality of bit rates as the bit rate of the video to be played back, but the present invention is not limited to this exemplary embodiment. For example, when a bit rate in a set range is designated, and the video delivering apparatus has a function of performing a process of converting video of a reference bit rate that is stored to a video of a designated bit rate, the bit rate determining unit 105 may determine the bit rate within the set range.
At this time, for example, the bit rate determining unit 105 may determine a value of the specified size of the display region, a value of the cache amount, or a value obtained by substituting a value of the bandwidth into a set formula as the bit rate. As a result, the stream delivery of the videos of the bit rates of more steps than when the plurality of bit rates are determined in advance is performed.
[2-10] Category of Invention
The present invention is applicable as the video delivering apparatus or the video delivery system including the video delivering apparatus in addition to the information processing apparatus such as the smart phone or the server apparatus. The present invention is applicable as an information processing method of carrying out the process implemented by the respective apparatuses and applicable as a program causing a computer controlling the respective apparatuses to operate. The program may be provided in the form of a recording medium such as an optical disk storing the program or may be provided in a form in which the program is downloaded to a computer via a communication line such as the Internet and installed for use.
The foregoing description of the exemplary embodiments of the present invention has been provided for the purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise forms disclosed. Obviously, many modifications and variations will be apparent to practitioners skilled in the art. The embodiments were chosen and described in order to best explain the principles of the invention and its practical applications, thereby enabling others skilled in the art to understand the invention for various embodiments and with the various modifications as are suited to the particular use contemplated. It is intended that the scope of the invention be defined by the following claims and their equivalents.

Claims (14)

What is claimed is:
1. An information processing apparatus comprising at least one hardware processor configured to play a video delivered by streaming, a bit rate of the video being changeable during playback, and the at least one hardware processor comprising:
a first specifying unit that specifies a size of a display region of the video;
a second specifying unit that specifies an amount of data temporarily stored for a period of the video to be next played back;
a third specifying unit that specifies a bandwidth by which the data is received;
a determining unit that determines the bit rate of the video delivered by streaming and played back based on the size, the data amount, and the bandwidth that are specified; and
a still image requesting unit that, when an operation of requesting a still image of a certain scene of the video is performed while the video is being played back at a first bit rate, requests the still image of the certain scene in a video at a second bit rate higher than the first bit rate,
wherein a plurality of bit rates are defined for stream delivery,
wherein when a playback time of the video corresponding to the temporarily stored data is equal to or larger than a threshold value, the determining unit selects in descending order, from a bit rate range of the plurality of bit rates that corresponds to the specified size of the display region, any one of the bit rates that does not exceed the specified bandwidth, the determining unit selecting a highest bit rate within the specified bandwidth from a bit rate range associated with the specified size of the display region and
wherein the determining unit determines the selected bit rate as the bit rate of the video that is played back.
2. The information processing apparatus according to claim 1,
wherein the determining unit determines, as the bit rate of the video to be next played back, a bit rate within the bit rate range that leads to an increase in the amount of the temporarily stored data.
3. The information processing apparatus according to claim 1,
wherein a level of the bit rate corresponds to a level of a resolution of the video and a level of a frame rate of the video, and
wherein when the video is played back at a fast speed, the determining unit determines, as the bit rate of the video, a bit rate in which the resolution or the frame rate is lower than that at a time of normal playback.
4. The information processing apparatus according to claim 1,
wherein a level of the bit rate corresponds to a level of a resolution of the video and a level of a frame rate of the video, and
wherein when the video is played back at a slow speed, the determining unit determines, as the bit rate of the video, a bit rate in which the resolution or the frame rate is higher than that at a time of normal playback.
5. The information processing apparatus according to claim 1,
wherein the still image requesting unit requests the still image of a file format corresponding to the specified bandwidth.
6. The information processing apparatus according to claim 1,
wherein the still image requesting unit requests the still image of a file format corresponding to a purpose of the video, which is set by the at least one hardware processor.
7. The information processing apparatus according to claim 1,
wherein, when the playback starts from beginning of the video, the determining unit determines the bit rate without using the data amount and the bandwidth until a set period elapses.
8. The information processing apparatus according to claim 1, wherein the at least one hardware processor further comprises:
an acquiring unit that acquires bit rate information indicating the plurality of bit rates; and
a delivery requesting unit that requests delivery of the video by streaming at the determined bit rate.
9. An information processing method for an information processing apparatus comprising a at least one hardware processor configured to play a video delivered by streaming, a bit rate of the video being changeable during playback, the information processing method comprising:
specifying a size of a display region of the video;
specifying an amount of data temporarily stored for a period of the video to be next played back;
specifying a bandwidth by which the data is received;
determining the bit rate of the video delivered by streaming and played back based on the size, the data amount, and the bandwidth that are specified; and
when an operation of requesting a still image of a certain scene of the video is performed while the video is being played back at a first bit rate, requesting the still image of the certain scene in a video at a second bit rate higher than the first bit rate,
wherein a plurality of bit rates are defined for stream delivery,
wherein when a playback time of the video corresponding to the temporarily stored data is equal to or larger than a threshold value, the determining step includes selecting in descending order, from a bit rate range of the plurality of bit rates that corresponds to the specified size of the display region, any one of the bit rates that does not exceed the specified bandwidth, the determining step including selecting a highest bit rate within the specified bandwidth from a bit rate range associated with the specified size of the display region, and
wherein the determining step also includes determining the selected bit rate as the bit rate of the video that is played back.
10. An information processing apparatus comprising at least one hardware processor configured to play a video delivered by streaming, a bit rate of the video being changeable during playback, and the at least one hardware processor comprising:
a first specifying unit that specifies a size of a display region of the video;
a second specifying unit that specifies an amount of data temporarily stored for a period of the video to be next played back;
a third specifying unit that specifies a bandwidth by which the data is received;
a determining unit that determines the bit rate of the video delivered by streaming and played back based on the size, the data amount, and the bandwidth that are specified, the determining unit selecting a highest bit rate within the specified bandwidth from a bit rate range associated with the specified size of a display region, and
a still image requesting unit that, when an operation of requesting a still image of a certain scene of the video is performed while the video is being played back at a first bit rate, requests the still image of the certain scene in a video at a second bit rate higher than the first bit rate.
11. The information processing apparatus according to claim 1,
wherein a plurality of bit rates are defined in a stream delivery, and
wherein when a playback time, for which the video indicated by the temporarily stored data having the specified data amount is played back, is less than a threshold value, the determining unit selects any one of bit rates of up to a set number in an ascending order within a bit rate range that corresponds to the specified size of the display region and does not exceed the specified bandwidth, and determines the selected bit rate as the bit rate of the video that is played back.
12. The information processing apparatus according to claim 1, wherein the display region is configured to be variable.
13. The information processing method according to claim 9, wherein the display region is configured to be variable.
14. The information processing apparatus according to claim 10, wherein the display region is configured to be variable.
US15/205,546 2016-02-29 2016-07-08 Information processing apparatus and information processing method Active US10382832B2 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2016-036943 2016-02-29
JP2016036943A JP2017157903A (en) 2016-02-29 2016-02-29 Information processor

Publications (2)

Publication Number Publication Date
US20170251274A1 US20170251274A1 (en) 2017-08-31
US10382832B2 true US10382832B2 (en) 2019-08-13

Family

ID=56684463

Family Applications (1)

Application Number Title Priority Date Filing Date
US15/205,546 Active US10382832B2 (en) 2016-02-29 2016-07-08 Information processing apparatus and information processing method

Country Status (5)

Country Link
US (1) US10382832B2 (en)
EP (1) EP3211906A1 (en)
JP (1) JP2017157903A (en)
KR (1) KR101918017B1 (en)
CN (1) CN107135422B (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190387271A1 (en) * 2017-01-30 2019-12-19 Sony Corporation Image processing apparatus, image processing method, and program
US20210235139A1 (en) * 2018-08-17 2021-07-29 Sony Interactive Entertainment Inc. Image generation apparatus, image generation method, and program
US11108984B2 (en) * 2018-03-01 2021-08-31 Canon Kabushiki Kaisha Image processing device and control method therefor, and storage medium

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019054360A1 (en) 2017-09-12 2019-03-21 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ Image display method, image delivery method, image display device, and image delivery device
CN111698536B (en) * 2019-03-15 2023-03-28 瑞昱半导体股份有限公司 Video processing method and system
EP3955634A4 (en) * 2019-04-12 2022-06-08 Sony Group Corporation Control device, wireless communication device, and control method for wireless communication device
KR20210050752A (en) * 2019-10-29 2021-05-10 삼성전자주식회사 Electronic device and control method thereof
EP4074058A1 (en) * 2019-12-11 2022-10-19 Google LLC Methods, systems, and media for selecting formats for streaming media content items
CN113141514B (en) * 2020-01-17 2022-07-22 北京达佳互联信息技术有限公司 Media stream transmission method, system, device, equipment and storage medium
WO2022092760A1 (en) * 2020-10-26 2022-05-05 주식회사 프로젝트한 Reproduction method for image
KR102281128B1 (en) * 2020-10-26 2021-07-26 한창엽 Reproduction method of graphic data
US11979767B2 (en) * 2020-11-27 2024-05-07 At&T Intellectual Property I, L.P. Automatic adjustment of throughput rate to optimize wireless device battery performance
CN114363674B (en) * 2021-12-09 2024-02-09 北京快乐茄信息技术有限公司 Video playing method, device, terminal and storage medium
CN114979726B (en) * 2022-06-30 2023-09-26 重庆紫光华山智安科技有限公司 Code rate adjusting method, device, server and storage medium

Citations (37)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002344941A (en) 2001-05-14 2002-11-29 Sony Corp Contents quality control system
JP2002344913A (en) 2001-05-16 2002-11-29 Nec Yonezawa Ltd Conversion processing device and conversion processing method for video data in network, and conversion processing service
JP2002369052A (en) 2001-06-06 2002-12-20 Nikon Corp Electronic picture device
US20030012560A1 (en) 2001-07-12 2003-01-16 Masashi Mori Portable image recording apparatus
US20030222974A1 (en) * 2002-05-30 2003-12-04 Kddi Corporation Picture transmission system
JP2004088480A (en) 2002-08-27 2004-03-18 Canon Inc Image pickup device and method for controlling data transmission
US20040145660A1 (en) 2001-06-06 2004-07-29 Yosuke Kusaka Electronic imaging apparatus and electronic imaging system
JP2006092341A (en) 2004-09-24 2006-04-06 Victor Co Of Japan Ltd Content delivery server
JP2007036666A (en) 2005-07-27 2007-02-08 Onkyo Corp Contents distribution system, client, and client program
JP2008109434A (en) 2006-10-26 2008-05-08 Mitsubishi Electric Corp Stream data reproducing system, transmitter for stream data reproducing system, receiver for stream data reproducing system and program for stream data reproducing system
US20080143877A1 (en) 2006-12-19 2008-06-19 Canon Kabushiki Kaisha Video processing apparatus and control method for the video processing apparatus
JP2008270939A (en) 2007-04-17 2008-11-06 Victor Co Of Japan Ltd Motion image distribution system, and server device and client device
JP2011180810A (en) 2010-03-01 2011-09-15 Nec Corp System, method and apparatus for delivery of content, content measurement apparatus and program
US20110268428A1 (en) 2009-09-09 2011-11-03 Eli Chen Accelerated playback of streaming media
US20120195388A1 (en) 2011-01-27 2012-08-02 Brother Kogyo Kabushiki Kaisha Encoding Apparatus, Encoding Method, and Storage Medium
US20130028576A1 (en) * 2011-07-29 2013-01-31 Kabushiki Kaisha Toshiba Reproduction control device and reproduction control method
US20130091248A1 (en) 2011-10-05 2013-04-11 Alcatel-Lucent Bell N.V. Method and apparatus for improving adaptive streaming video quality by optimizing resource allocation
US20130124748A1 (en) * 2008-05-30 2013-05-16 Microsoft Corporation Media streaming with enhanced seek operation
US20130195204A1 (en) 2012-01-19 2013-08-01 Vid Scale Inc. Methods and Systems for Video Delivery Supporting Adaptation to Viewing Conditions
US20130227080A1 (en) * 2012-02-27 2013-08-29 Qualcomm Incorporated Dash client and receiver with playback rate selection
JP2013214800A (en) 2012-03-30 2013-10-17 Ntt Communications Kk Streaming media reproduction device, streaming media reproduction method, and program
US20130282917A1 (en) 2012-04-24 2013-10-24 Vid Scale, Inc. Method and apparatus for smooth stream switching in mpeg/3gpp-dash
US20130279565A1 (en) * 2010-12-29 2013-10-24 British Telecommunications Public Limited Company Video coding
US20140040959A1 (en) 2012-08-03 2014-02-06 Ozgur Oyman Device orientation capability exchange signaling and server adaptation of multimedia content in response to device orientation
US20140258552A1 (en) * 2011-12-28 2014-09-11 Ozgur Oyman Video adaptation for content-aware wireless streaming
JP2014183488A (en) 2013-03-19 2014-09-29 Enwa Co Ltd Streaming receiver
CN104079954A (en) 2013-03-26 2014-10-01 索尼公司 Image processing device, image processing method, and computer program
US20140365677A1 (en) 2013-06-11 2014-12-11 bitmovin Softwareentwicklung OG Adaptation logic for varying a bitrate
US20150026358A1 (en) 2013-07-19 2015-01-22 Futurewei Technologies, Inc. Metadata Information Signaling And Carriage In Dynamic Adaptive Streaming Over Hypertext Transfer Protocol
US20150032899A1 (en) 2011-11-14 2015-01-29 Telefonaktiebolaget L M Ericsson (Publ) Media Streaming in Mobile Networks with Improved Efficiency
US20150049249A1 (en) 2013-08-15 2015-02-19 Lei Shao Apparatus, system and method of controlling wireless transmission of video streams
US20150156238A1 (en) * 2010-10-06 2015-06-04 Microsoft Technology Licensing, Llc Rapidly initializing and dynamically adjusting media streams
EP2924999A1 (en) 2014-03-25 2015-09-30 Intel IP Corporation Context-aware streaming of digital content
JP2015194832A (en) 2014-03-31 2015-11-05 パイオニア株式会社 Content output device, content distribution server, content output method and content output program
US20160103830A1 (en) * 2013-05-28 2016-04-14 Samsung Electronics Co., Ltd. User interface method and device for searching for multimedia content
US20160366454A1 (en) * 2015-06-15 2016-12-15 Intel Corporation Adaptive data streaming based on virtual screen size
US20170041238A1 (en) * 2014-04-03 2017-02-09 Orbital Multi Media Holdings Corporation Data flow control method

Patent Citations (47)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002344941A (en) 2001-05-14 2002-11-29 Sony Corp Contents quality control system
JP2002344913A (en) 2001-05-16 2002-11-29 Nec Yonezawa Ltd Conversion processing device and conversion processing method for video data in network, and conversion processing service
JP2002369052A (en) 2001-06-06 2002-12-20 Nikon Corp Electronic picture device
US20040145660A1 (en) 2001-06-06 2004-07-29 Yosuke Kusaka Electronic imaging apparatus and electronic imaging system
US20080068486A1 (en) 2001-06-06 2008-03-20 Nikon Corporation Digital image apparatus and digital image system
US20030012560A1 (en) 2001-07-12 2003-01-16 Masashi Mori Portable image recording apparatus
JP2003032629A (en) 2001-07-12 2003-01-31 Mitsubishi Electric Corp Portable type image recorder
US20030222974A1 (en) * 2002-05-30 2003-12-04 Kddi Corporation Picture transmission system
JP2004088480A (en) 2002-08-27 2004-03-18 Canon Inc Image pickup device and method for controlling data transmission
JP2006092341A (en) 2004-09-24 2006-04-06 Victor Co Of Japan Ltd Content delivery server
JP2007036666A (en) 2005-07-27 2007-02-08 Onkyo Corp Contents distribution system, client, and client program
JP2008109434A (en) 2006-10-26 2008-05-08 Mitsubishi Electric Corp Stream data reproducing system, transmitter for stream data reproducing system, receiver for stream data reproducing system and program for stream data reproducing system
US20080143877A1 (en) 2006-12-19 2008-06-19 Canon Kabushiki Kaisha Video processing apparatus and control method for the video processing apparatus
JP2008152114A (en) 2006-12-19 2008-07-03 Canon Inc Video processing apparatus and method for controlling video processing apparatus
JP2008270939A (en) 2007-04-17 2008-11-06 Victor Co Of Japan Ltd Motion image distribution system, and server device and client device
US20130124748A1 (en) * 2008-05-30 2013-05-16 Microsoft Corporation Media streaming with enhanced seek operation
US20110268428A1 (en) 2009-09-09 2011-11-03 Eli Chen Accelerated playback of streaming media
JP2011180810A (en) 2010-03-01 2011-09-15 Nec Corp System, method and apparatus for delivery of content, content measurement apparatus and program
US20150156238A1 (en) * 2010-10-06 2015-06-04 Microsoft Technology Licensing, Llc Rapidly initializing and dynamically adjusting media streams
US20130279565A1 (en) * 2010-12-29 2013-10-24 British Telecommunications Public Limited Company Video coding
US20120195388A1 (en) 2011-01-27 2012-08-02 Brother Kogyo Kabushiki Kaisha Encoding Apparatus, Encoding Method, and Storage Medium
JP2012156829A (en) 2011-01-27 2012-08-16 Brother Ind Ltd Encoder, encoding method, and encoding program
US20130028576A1 (en) * 2011-07-29 2013-01-31 Kabushiki Kaisha Toshiba Reproduction control device and reproduction control method
KR20140073546A (en) 2011-10-05 2014-06-16 알까뗄 루슨트 Improving adaptive streaming video quality by optimizing resource allocation
US20130091248A1 (en) 2011-10-05 2013-04-11 Alcatel-Lucent Bell N.V. Method and apparatus for improving adaptive streaming video quality by optimizing resource allocation
US20150032899A1 (en) 2011-11-14 2015-01-29 Telefonaktiebolaget L M Ericsson (Publ) Media Streaming in Mobile Networks with Improved Efficiency
US20140258552A1 (en) * 2011-12-28 2014-09-11 Ozgur Oyman Video adaptation for content-aware wireless streaming
US20130195204A1 (en) 2012-01-19 2013-08-01 Vid Scale Inc. Methods and Systems for Video Delivery Supporting Adaptation to Viewing Conditions
US20130227080A1 (en) * 2012-02-27 2013-08-29 Qualcomm Incorporated Dash client and receiver with playback rate selection
JP2013214800A (en) 2012-03-30 2013-10-17 Ntt Communications Kk Streaming media reproduction device, streaming media reproduction method, and program
US20130282917A1 (en) 2012-04-24 2013-10-24 Vid Scale, Inc. Method and apparatus for smooth stream switching in mpeg/3gpp-dash
JP2015518350A (en) 2012-04-24 2015-06-25 ヴィド スケール インコーポレイテッド Method and apparatus for smooth stream switching in MPEG / 3GPP-DASH
JP2015531903A (en) 2012-08-03 2015-11-05 インテル コーポレイション Device orientation capability exchange signaling and server adaptation of multimedia content in response to device orientation
US20140040959A1 (en) 2012-08-03 2014-02-06 Ozgur Oyman Device orientation capability exchange signaling and server adaptation of multimedia content in response to device orientation
JP2014183488A (en) 2013-03-19 2014-09-29 Enwa Co Ltd Streaming receiver
CN104079954A (en) 2013-03-26 2014-10-01 索尼公司 Image processing device, image processing method, and computer program
US20140298394A1 (en) 2013-03-26 2014-10-02 Sony Corporation Image processing device, image processing method, and computer program
US20160103830A1 (en) * 2013-05-28 2016-04-14 Samsung Electronics Co., Ltd. User interface method and device for searching for multimedia content
US20140365677A1 (en) 2013-06-11 2014-12-11 bitmovin Softwareentwicklung OG Adaptation logic for varying a bitrate
US20150026358A1 (en) 2013-07-19 2015-01-22 Futurewei Technologies, Inc. Metadata Information Signaling And Carriage In Dynamic Adaptive Streaming Over Hypertext Transfer Protocol
CN105230024A (en) 2013-07-19 2016-01-06 华为技术有限公司 Based on the metadata information in the dynamic self-adapting Streaming Media of HTML (Hypertext Markup Language) instruction and carry
US20150049249A1 (en) 2013-08-15 2015-02-19 Lei Shao Apparatus, system and method of controlling wireless transmission of video streams
WO2015023832A1 (en) 2013-08-15 2015-02-19 Intel Corporation Apparatus, system and method of controlling wireless transmission of video streams
EP2924999A1 (en) 2014-03-25 2015-09-30 Intel IP Corporation Context-aware streaming of digital content
JP2015194832A (en) 2014-03-31 2015-11-05 パイオニア株式会社 Content output device, content distribution server, content output method and content output program
US20170041238A1 (en) * 2014-04-03 2017-02-09 Orbital Multi Media Holdings Corporation Data flow control method
US20160366454A1 (en) * 2015-06-15 2016-12-15 Intel Corporation Adaptive data streaming based on virtual screen size

Non-Patent Citations (11)

* Cited by examiner, † Cited by third party
Title
Brendan Long. "Proposed Study Text FO 23009-3 for Low Latency". 113. MPEG Meeting; Oct. 19, 2015-Oct. 23, 2015; Geneva; (Motion Picture Expert Group or ISO/IEC JTC1/SC29/WG11), No. m37491, Oct. 22, 2015, XP030065859.
BRENDAN LONG: "Proposed study text for 23009-3 for low latency", 113. MPEG MEETING; 19-10-2015 - 23-10-2015; GENEVA; (MOTION PICTURE EXPERT GROUP OR ISO/IEC JTC1/SC29/WG11), 22 October 2015 (2015-10-22), XP030065859
Contribution to the Server and Network Assisted DASH Operation CE, ISO/IEC JTC1/SC29/WG11, MPEG2013/m30357, Jul. 2013). *
Feb. 14, 2017 Search Report issued in European Patent Application No. 16182138.4.
May 28, 2019 Office Action issued in Chinese Patent Application No. 201610643376.3.
May 28, 2019 Office Action issued in Japanese Patent Application No. 2016-036943.
Ozgur Oyman et al. "On Sand CE: QoS Signaling for Dash and Proposed Client Adaptation Guidelines". 107. MPEG Meeting, Jan. 13, 2014-Jan. 17, 2014; San Jose; (Motion Picture Expert Group or ISO/IEC JTC1/SC29/WG11), No. m32325, Jan. 9, 2014, XP030060777.
OZGUR OYMAN; VISHWANATH RAMAMURTHI: "On SAND CE: QoS Signaling for DASH and Proposed Client Adaptation Guidelines", 107. MPEG MEETING; 13-1-2014 - 17-1-2014; SAN JOSE; (MOTION PICTURE EXPERT GROUP OR ISO/IEC JTC1/SC29/WG11), 9 January 2014 (2014-01-09), XP030060777
Sep. 12, 2017 Office Action issued in Korean Application No. 10-2016-0095304.
Xin Wang et al. "Contribution to the Server and Network Assisted Dash Operation CE". 105. MPEG Meeting; Jul. 29, 2013-Aug. 2, 2013; Vienna; (Motion Picture Expert Group or ISO/IEC JTC1/SC29/WG11), No. m30357, Jul. 28, 2013, XP030058884.
XIN WANG; SHAOBO ZHANG; YONGLIANG LIU; LEI ZHOU; YUQUN FAN; YANGPO XU; XIN LIU; PEIYUN DI; CHANGQUAN AI: "Contribution to the Server and Network Assisted DASH Operation CE", 105. MPEG MEETING; 29-7-2013 - 2-8-2013; VIENNA; (MOTION PICTURE EXPERT GROUP OR ISO/IEC JTC1/SC29/WG11), 28 July 2013 (2013-07-28), XP030058884

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190387271A1 (en) * 2017-01-30 2019-12-19 Sony Corporation Image processing apparatus, image processing method, and program
US11108984B2 (en) * 2018-03-01 2021-08-31 Canon Kabushiki Kaisha Image processing device and control method therefor, and storage medium
US20210235139A1 (en) * 2018-08-17 2021-07-29 Sony Interactive Entertainment Inc. Image generation apparatus, image generation method, and program
US11962820B2 (en) * 2018-08-17 2024-04-16 Sony Interactive Entertainment Inc. Image generation apparatus, image generation method, and program indicating game play status

Also Published As

Publication number Publication date
CN107135422B (en) 2021-02-26
US20170251274A1 (en) 2017-08-31
EP3211906A1 (en) 2017-08-30
KR101918017B1 (en) 2018-11-13
KR20170101755A (en) 2017-09-06
CN107135422A (en) 2017-09-05
JP2017157903A (en) 2017-09-07

Similar Documents

Publication Publication Date Title
US10382832B2 (en) Information processing apparatus and information processing method
US8495495B2 (en) Information processing apparatus, bookmark setting method, and program
CN113141514B (en) Media stream transmission method, system, device, equipment and storage medium
US8886765B2 (en) System and method for predicitive trick play using adaptive video streaming
US20150156557A1 (en) Display apparatus, method of displaying image thereof, and computer-readable recording medium
JP2016538657A (en) Browse videos by searching for multiple user comments and overlaying content
US20200099973A1 (en) Information processing system, terminal apparatus, and non-transitory computer readable medium
KR102487000B1 (en) Advertisement display method, advertisement display device, advertisement display program
EP3211912B1 (en) Information processing apparatus
JP2013214800A (en) Streaming media reproduction device, streaming media reproduction method, and program
WO2021143388A1 (en) Bitrate switching method and device
US9866786B2 (en) Display apparatus
JP6891497B2 (en) Information processing equipment, information processing systems and programs
JP6099715B2 (en) Streaming media playback apparatus, streaming media playback method, and program
US20180048727A1 (en) Transmission apparatus and method, and storage medium
US9672194B2 (en) Presenting images from slow image-event stream
US20240098333A1 (en) Video Playback based on an HTML iframe and a Headless Browser
JP4902326B2 (en) Video transmission server and control method thereof
JPWO2008069032A1 (en) Moving image distribution system, moving image distribution apparatus, and moving image distribution method
US20080298771A1 (en) Method for acquiring reduced image and video apparatus using the same
JP2015220609A (en) Terminal device, control method, and program

Legal Events

Date Code Title Description
AS Assignment

Owner name: FUJI XEROX CO., LTD., JAPAN

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:FUJII, AKIRA;MURALI, SURESH;REEL/FRAME:039109/0852

Effective date: 20160705

STPP Information on status: patent application and granting procedure in general

Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS

STPP Information on status: patent application and granting procedure in general

Free format text: AWAITING TC RESP., ISSUE FEE NOT PAID

STCF Information on status: patent grant

Free format text: PATENTED CASE

AS Assignment

Owner name: FUJIFILM BUSINESS INNOVATION CORP., JAPAN

Free format text: CHANGE OF NAME;ASSIGNOR:FUJI XEROX CO., LTD.;REEL/FRAME:058287/0056

Effective date: 20210401

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

Year of fee payment: 4