Summary of the invention
Technical problems to be solved in this application are to provide a kind of Streaming Media processing method, comprising:
Start the process that generates streaming media image at server end, keep described process in running status;
Detect in real time the request that client is obtained streaming media image, in the time receiving described request, from the described process in running status, obtain described streaming media image, and be back to described client;
Receive the operational order that described client is sent, and generate corresponding streaming media image according to described operational order.
Preferably, the described process of described maintenance further comprises the streaming media image status transmission detecting between described client and described server in running status, in the time detecting the transmission of streaming media image between described client and described server and disconnect, described server restarts the program of described generation streaming media image.
Preferably, streaming media image status transmission between the described client of described detection and described server further comprises that described client timing sends a packet to described server, described server returns to a packet for described packet, if the packet that described client is returned described in receiving in threshold time, carries out depending on the streaming media image transmission between described client and described server; If the packet that described client is returned described in not receiving in threshold time, the streaming media image transmission of looking described between client and described server disconnects.
Preferably, also comprise that the instruction of the process of closing playing stream media image is sent to described server by described client, described server restarts the process of described generation streaming media image according to described instruction.
Preferably, describedly from the described process in running status, obtain described streaming media image, and be back to described client and further comprise from the described process in running status and to capture, compress described streaming media image, and described streaming media image is back to described client; Described client decompress(ion) contracts and shows described streaming media image.
Correspondingly, the application also provides a kind of Streaming Media treatment system, comprising:
Starting drive, for start the process that generates streaming media image at server end, keeps described process in running status;
Streaming media image acquisition device for detecting in real time the request that client is obtained streaming media image, in the time receiving described request, obtains described streaming media image, and is back to described client from the described process in running status;
Operational order processing unit, the operational order sending for receiving described client, and generate corresponding streaming media image according to described operational order.
Preferably, described starting drive further comprises detection module, for detection of the streaming media image status transmission between described client and described server; And restart module, in the time detecting streaming media image transmission between described client and described server and disconnect, described server restarts the program of described generation streaming media image.
Preferably, described streaming media image acquisition device further comprises that streaming media image returns to module, for capturing, compress described streaming media image from the described process in running status, and described streaming media image is back to described client; And streaming media image display module, show described streaming media image for contracing at described client decompress(ion).
Compared with prior art, the application has the following advantages:
1. generate the process of streaming media image at server end all the time in opening, in the time that user opens a certain Streaming Media, server can be transferred to client by streaming media image after receiving request immediately, the time that the process initiation of streaming media image to be generated expends such as greatly shorten when user opens Streaming Media, promoted user's experience.
2. the application is detecting the streaming media image status transmission between client and server by transmitting packet between client and server, in the time detecting streaming media image transmission disconnection, server can restart the process of described generation streaming media image, thereby has effectively ensured that the process that generates streaming media image remains opening on server.
Embodiment
For the above-mentioned purpose, the feature and advantage that make the application can become apparent more, below in conjunction with the drawings and specific embodiments, the application is described in further detail.
With reference to Fig. 1, show the embodiment flow chart of the application's Streaming Media processing method.
Step 101: start the process that generates streaming media image at server end, keep described process in running status.
Described streaming media image comprises sound stream, video flowing, text flow, image stream, animation streams etc., as video conference, game, Broadcast Journalism, video display, meeting, speech.Streaming Media is not downloaded whole file before broadcasting, only deposits beginning content in internal memory, and the data flow of streaming video transmits at any time at any time and plays, and just in the time starting, has some delays.The transmission of Streaming Media mainly contains two kinds of methods: real-time streaming transmission (Real time streaming) and progressive streaming transmission (progressive streaming).In general, if video is real-time broadcast, or use stream transmission media server, or application is as the real-time protocol (RTP) of RTSP, is real-time streaming transmission.As using http server, file sends by sequential flow.
Described server can be common server, also can be Cloud Server, wherein Cloud Server is a kind of Intel Virtualization Technology of similar VPS server, VPS adopts virtual software, VZ or VM fictionalize the part of multiple similar separate servers on a station server, the mirror image that each part has Cloud Server, can do independent operating system, and management method is the same with server.
In an embodiment of the present invention, Streaming Media is for wandering about play.Cloud game server end comprises two main modular, has stored cloud games in the first module, generates cloud game picture by moving this program; The second module is used for receiving client instruction, captures cloud game picture according to this instruction, this picture is compressed and transfer to client.
At server end, by the second module, the first module is sent to enabled instruction, thereby start the process that generates streaming media image.What wherein, this instruction was used is the api function that windows system carries.Start after the process of this generation streaming media image, if this process is not sent to instruction, this process will be all the time in running status, is equivalent to server end all the time in opening the state of game initial page.With reference to Fig. 2, show the reciprocal process schematic diagram between the application's client, web server and streaming media server.
In an embodiment of the present invention, server is divided into two types: web server and streaming media server.When user is by after being arranged on the player or the selected streaming media image of web browser in client, the ID of this streaming media image is sent to web server by client, web server sends to client according to the ID of streaming media image by the streaming media server address of this streaming media image of operation, client according to this address to streaming media server request streaming media image, this streaming media image is transferred to client by the streaming media server that is moving this streaming media image, and by this streaming media image of player plays.
Web server is also referred to as web page server, and major function is to provide network information browsing service.Web server can be resolved http protocol.When Web server receives a HTTP request, can return to a http response, for example send a html page back to.In order to process a request, Web server can respond a static page or picture, carry out page jump, or the generation of dynamic response is entrusted to some other such as CGI scripting of program, JSP (JavaServer Pages) script, servlets, ASP (Active Server Pages) script, server end JavaScript, or some other server end technology.Regardless of the object of these scripts, browser can be browsed in the response that the program of these server ends produces a HTML conventionally.The agent model of Web server is in the time that a request is sent in Web server, and it is the simple program (server side scripts) that can well process request that request is passed to only.Web server only provides the environment that can carry out the response that server and return information produce, and can not exceed term of reference.Server has transaction conventionally, the functions such as database connection and message.In an embodiment of the present invention, web server is responsible for controlling mutual between client and server, comprises ID and this server address of the streaming media image of the operation that reception server sends, and provides server address to the client of the request of sending.
Step 102: the request that detects in real time client and obtain streaming media image, in the time receiving described request, from the described process in running status, obtain described streaming media image, and be back to described client.
Described client comprises that PC, TV, portable terminal device, flat board, projector etc. have the device of demonstration and decoding function.
Obtaining streaming media image request can send by the player being arranged in client, also can send by web browser, for example, by loading on the flash player plays streaming media image on the web browser page or passing through the link on webpage clicking browser page, be connected to flash player, thereby obtain streaming media image.
Describedly from the described process in running status, obtain described streaming media image, and be back to described client and further comprise from the described process in running status and to capture, compress described streaming media image, and described streaming media image is back to described client; Described client decompress(ion) contracts and shows described streaming media image.
Server end comprises detection module, for detecting in real time the request that client is obtained streaming media image.In the time receiving this request, the second module captures initial cloud game picture according to this request, this picture is compressed and transfer to client.After client decoding, present to user by user interface.
Each piece image is compressed and comprises by certain frequency and respectively the described image block that belongs to same two field picture is carried out to intraframe predictive coding and inter prediction encoding.
For example, carry out intraframe predictive coding every 10 two field pictures, boil down to I frame, carries out inter prediction encoding between I frame and I frame, and boil down to P frame, encodes to image according to the coding form of IPPPPPPPPPPIPPPPPPPPPPI.Can use MPEG series standard (MPEG-1-MPEG-7), H.261, H.262, H.263, H.264 etc. standard is encoded, preferably, use and H.264 encode.
In an embodiment of the present invention, initial pictures is RGB image, when image is compressed, this two field picture is divided into R frame, G frame and B frame three two field pictures and processes respectively, is combined into a two field picture when decompression.Also can first Initial R GB image be converted to YCrCb image, then YCrCb image is carried out to compression and decompression processing.
Particularly, I two field picture being carried out to intraframe predictive coding comprises each the piece image division macro block being partitioned into.As shown in Figure 3, each piece image can be divided into the macro block of 16 × 16 pixels, 16 × 8 pixels or 8 × 8 pixels, or be divided into less sub-macro block 8 × 4 pixels, 4 × 8 pixels, 4 × 4 pixels, then predict the pixel value of each macro block.To be predicted as example in 16 × 16 macroblock frame, use 33 the pixel generation forecast values adjacent with current macro.As shown in Figure 4, have 4 kinds of predictive modes: pattern 0(vertical prediction), pattern 1(horizontal forecast), pattern 2(DC prediction), mode 3 (planar prediction), the pixel of utilizing respectively current macro top (H), left (V) and upper left side to encode is predicted the pixel value of current macro.
P two field picture is carried out to inter prediction encoding and comprise in the mode of Fig. 3 each the piece image being partitioned into is divided to macro block, preferably, each the piece image being partitioned into is divided into sub-macro block, estimator macro block is the motion vector to current frame image by former frame image.After estimation completes, the motion vector information of acquisition and prediction residual quantization encoding are sent to decompressor end, together send to the mark that also has macro-block partition mode of decompressor end.
Convert afterwards, transform method comprises that Fourier transform (DFT), Wo-Ha convert (WHT) or discrete cosine transform (DCT), preferably, carry out discrete cosine transform (DCT) to each macro block.Dct transform is to be spatial frequency distribution by how much distribution transformations of space pixel, for example, for the two-dimensional dct transform formula of N × N image f (x, y) is:
Subsequently, carry out entropy coding for above-mentioned result data.Entropy coding can adopt the unified variable length code based on exponential type Golomb code to encode to all grammer stream elements except conversion coefficient, or adopts based on contextual adaptive binary arithmetic coding.Finally macro block is carried out to filtering processing.
After multiple image block compression codings of one two field picture are completed, the identification information of described multiple image blocks and described packed data are packaged into packet and are sent to decompressor end.
After data compression is complete, packed data is packaged into packet, this packet comprises the mark of each image block, as ID, and packed data, this packed data is represented by information such as data packet length, initial address, timestamps.
After current frame image is handled, server automatically or by program setting, next frame image is compressed with said method, to realize the Real Time Compression of video flowing.This server can be common server or server.
After client packet, packet is taken apart, obtained the ID of packed data and each image block.
To packed data decompress process be the reverse process of compression process, to packed data respectively according to entropy decoding, re-quantization, inverse transformation, go back original image.
Step 103: receive the operational order that described client is sent, and generate corresponding streaming media image according to described operational order.
The process that generates streaming media image is that the operational order sending according to client generates corresponding streaming media image.In an embodiment of the present invention, this Streaming Media is for wandering about play, and the operational order that client is sent is game command, in the time that user starts to play, client sends to user's instruction the process that generates streaming media image, and this process generates corresponding game picture according to this instruction.This instruction can be mouse or keyboard instruction, for the client that windows operating system is installed, Cloud Server just can obtain the coordinate of cursor in user interface and the physical coordinates of mouse by getdevicestate function and the general order of relevant windows system of calling in directX storehouse.
First the request that client is obtained streaming media image sends to web server; The server address of the process that generates streaming media image described in real time execution is sent to described client by described web server; Described client sends the request of obtaining described streaming media image to described server according to described server address; Described server returns to described streaming media image according to described request.
Web server is responsible for controlling mutual between client and server, and what client sent to web server obtains the ID that has carried streaming media image in request, the address of the continuous receiving stream media server report of web server simultaneously and.The ID of the streaming media image that web server sends client contrasts with the ID of the streaming media image moving of streaming media server report, selects the server address of the streaming media image that is moving identical ID, and this address is sent to client.Client sends the request of obtaining this streaming media image to streaming media server according to this address, this streaming media server returns to described streaming media image according to described request.
In the time running into user and initiatively exit the situation such as process or network disconnection that generates streaming media image, the process of the generation streaming media image of streaming media server can stop, therefore need to restart the process of this generation streaming media image, so that this user or other users obtain this streaming media image.
Also comprise that the instruction of the process of closing playing stream media image is sent to described server by described client, described server restarts the process of described generation streaming media image according to described instruction.
In the time that user initiatively exits the process that generates streaming media image, the order of closing the process that generates streaming media image is sent to described server by described client, and described server restarts the process of described generation streaming media image according to described order.
In the time that user closes streaming media playing window, windows system can be called onclose function and close a series of correlation function of broadcast window, comprising the function that the order closing window is sent to server, server can restart streaming media playing program after receiving this order.
When network disconnects, server can detect by heartbeat mechanism, restarts afterwards streaming media playing program.
With reference to Fig. 3, show the application and detect the schematic diagram of streaming media image status transmission between client and server.
The present invention is provided with heartbeat mechanism between client and server, for detection of the streaming media image status transmission between described client and described server, in the time detecting the transmission of streaming media image between described client and described server and disconnect, described server restarts the process of described generation streaming media image.Heartbeat mechanism is generally used in Transmission Control Protocol, can be initiated by client, also can be initiated by server end.
In the time that heartbeat mechanism is initiated by client, client timing sends a packet to described server, described server returns to a packet for described packet, if the packet that described client is returned described in receiving in threshold time, carries out depending on the streaming media image transmission between described client and described server; If the packet that described client is returned described in not receiving in threshold time, the streaming media image transmission of looking described between client and described server disconnects.
In described packet, comprising fix information, the not special regulation of fix information, packet is all generally very little bag, or only comprises an empty bag in packet header.Client sends a fix information at set intervals to streaming media server, streaming media server is received fix information of rear reply, if streaming media server is in threshold time, the packet that the packet that client sends is replied with streaming media server can be identical, also can be different.In 3 minutes, do not receive the information that client sends, the streaming media image transmission of looking between client and this streaming media server disconnects.
In the time that heartbeat mechanism is initiated by streaming media server, described server timing sends a packet to described client, described client is returned to a packet for described packet, if the packet that described server returns described in receiving in threshold time, carries out depending on the streaming media image transmission between described client and described server; If the packet that described server returns described in not receiving in threshold time, the streaming media image transmission of looking described between client and described server disconnects.
The principle that heartbeat mechanism is initiated by streaming media server is similar with the principle of being initiated by client, preferably, initiates heartbeat mechanism by client.
With reference to Fig. 4, show the embodiment schematic diagram of the application's Streaming Media treatment system.
This Streaming Media treatment system 400 comprises:
Starting drive 401, for start the process that generates streaming media image at server end, keeps described process in running status;
Streaming media image acquisition device 402 for detecting in real time the request that client is obtained streaming media image, in the time receiving described request, obtains described streaming media image, and is back to described client from the described process in running status;
Operational order processing unit 403, the operational order sending for receiving described client, and generate corresponding streaming media image according to described operational order.
Preferably, described starting drive 401 further comprises detection module 404, for detection of the streaming media image status transmission between described client and described server; And restart module 405, in the time detecting streaming media image transmission between described client and described server and disconnect, described server restarts the program of described generation streaming media image.
Preferably, described streaming media image acquisition device 402 further comprises that streaming media image returns to module 406, for capturing, compress described streaming media image from the described process in running status, and described streaming media image is back to described client; And streaming media image display module 407, show described streaming media image for contracing at described client decompress(ion).
For system embodiment, because it is substantially similar to embodiment of the method, so description is fairly simple, relevant part is referring to the part explanation of embodiment of the method.
The present invention can be used in numerous general or special purpose computing system environment or configuration.For example: personal computer, server computer, handheld device or portable set, laptop device, multicomputer system, system based on microprocessor, network PC, minicom, mainframe computer, the distributed computing environment (DCE) that comprises above any system or equipment etc.
A kind of Streaming Media processing method and the system that above the application are provided are described in detail, applied principle and the execution mode of specific case to the application herein and set forth, the explanation of above embodiment is just for helping to understand the application's method and core concept thereof; , for one of ordinary skill in the art, according to the application's thought, all will change in specific embodiments and applications, in sum, this description should not be construed as the restriction to the application meanwhile.