CN106791973B - data processing method and device based on smart television - Google Patents

Abstract

The application provides a data processing method and device based on a smart television. In the method, the client and the server communicate through the self-defined communication identifier, cross-platform interactive data processing between the smart television and the network is realized, in addition, when the client sends the video mail, only one packaged HTML page is sent, in addition, the method does not need a user to spend a large amount of time to upload a large-capacity attachment, the uploading can be finished when the video is recorded, the mail sending efficiency is improved, and the condition that a receiver can download and browse while subsequently using a webpage to browse the video can also be ensured.

Description

Data processing method and device based on smart television

the application is as follows: 12/27/2012, with application number: 201210583823.2, the name of invention creation: a divisional application of a data processing method and device based on a smart television is provided.

Technical Field

the present application relates to data processing technologies, and in particular, to a data processing method and apparatus based on a smart television.

background

The intelligent television is gradually popularized according to the trends of high-definition, networking and intellectualization of the television. The intelligent television can realize various application services such as network search, IP television, BBTV network video communication and the like.

At present, with the continuous development of the smart television technology, the smart television is becoming a third information access terminal following a computer and a mobile phone, so that a user can access information required by the user at any time by using the smart television conveniently.

Disclosure of Invention

the application provides a data processing method and device based on an intelligent television, and aims to achieve cross-platform interactive data processing between the intelligent television and a network.

The technical scheme provided by the application comprises the following steps:

a data processing method based on a smart television comprises the following steps:

The intelligent television serves as a client and is communicated with the server through a self-defined communication identifier so as to obtain network authorization of the server;

when the client needs to send the video mail, logging in a server by using the network identification ID authorized by the server, and acquiring an HTML page embedded into the client by the server after logging in; the HTML page at least comprises: a receiving and sending bar, a toolbar and a mail editing area;

The client at least adds recipient information, sender information, mail title and mail sending time in a receiving and sending column of the HTML page;

The client side inserts a video option in a toolbar of the HTML page so as to enable a video selection area to be added to the mail editing area of the HTML page;

the client side writes mail content in a mail editing area of the HTML page, uploads a local video file to a server when a video selection area selects to send the local video file, adjusts the size of a video area when a recorded video file is selected to be sent, opens a recording device to start recording the video file by mapping the video resolution with the size of the video area, and uploads the recorded video file to the server;

And after uploading the video file to the server, the client informs the server to send the video file, packages the HTML page by the server according to the format understood by the mail generator to generate a video mail, and sends the video mail to a receiver.

a data processing method based on a smart television comprises the following steps:

The server carries out network authorization on the intelligent television as the client;

The server receives a connection request with a self-defined mail receiving identifier sent by the client through HTTP, identifies whether a mail requested by the connection request is expired, if not, returns a response with a self-defined authorized mail receiving identifier to the client, and if so, returns a response with a self-defined refused mail receiving identifier to the client;

The server receives a playing request with a playing adjustment identifier sent by the client, wherein the playing request is sent when the client receives a response with an authorized mail receiving identifier;

And the server controls the multimedia stream to be played at the client according to the parameters in the playing request, wherein the parameters comprise a delay parameter and a network speed parameter.

a data processing device based on an intelligent television is an intelligent television and comprises:

The communication module is used for communicating with the server through a self-defined communication identifier so as to acquire the network authorization of the server;

The login module is used for logging in a server by using the network identification ID authorized by the server when the device needs to send the video mail, and acquiring an HTML (hypertext markup language) page embedded into the client by the server after logging in; the HTML page at least comprises: a receiving and sending bar, a toolbar and a mail editing area;

a page processing module, configured to perform the following processing on the HTML page: at least adding receiver information, sender information, a mail title and mail sending time in a receiving and sending column of the HTML page, selecting and inserting a video option in a toolbar of the HTML page to enable a mail editing area of the HTML page to add a video selection area, writing mail content in the mail editing area of the HTML page, uploading a local video file to a server when the video selection area selects to send the local video file, adjusting the size of the video area when the recorded video file is selected to be sent, opening a recording device to start recording the video file by mapping the video resolution with the size of the video area, and uploading the recorded video file to the server;

and the notification module is used for notifying the server to send the video file after the page processing module finishes uploading the video file to the server, packaging the HTML page by the server according to a format understood by the mail generator to generate a video mail, and sending the video mail to a receiver.

A data processing device based on a smart television is a server and comprises:

The network processing module is used for performing network authorization on the intelligent television as the client;

The mail request processing module is used for receiving a connection request with a self-defined mail receiving identifier sent by the client through HTTP, identifying whether a mail requested by the connection request is expired, if not, returning a response with a self-defined authorized mail receiving identifier to the client, and if so, returning a response with a self-defined refused mail receiving identifier to the client;

the playing request processing module is used for receiving a playing request with a playing adjustment identifier sent by the client and controlling the multimedia stream to be played at the client according to parameters in the playing request; wherein the play request is sent when the client receives a response with an authorized mail receiving identifier, and the parameters comprise a delay parameter and a network speed parameter.

according to the technical scheme, the data processing of cross-platform interaction between the intelligent television and the network is realized through the interaction between the client and the server, and in addition, when the client sends the video mail, compared with the traditional email sending process, the method does not need a user to spend a large amount of time to upload a large-capacity attachment, the uploading can be finished when the video is recorded, and the mail sending efficiency is improved;

Furthermore, in the invention, the server only sends one packaged HTML page, that is, the receiver only receives one HTML page, which can ensure that the receiver can download and browse while browsing videos by using the webpage.

Drawings

FIG. 1 is a flow chart of a method provided by an embodiment of the present invention;

fig. 2 is a flow chart of network authorization provided by the embodiment of the present invention;

Fig. 3 is a flowchart of recording authorization provided by the embodiment of the present invention;

FIG. 4 is a schematic diagram of a server dynamically allocating a storage space according to an embodiment of the present invention;

Fig. 5 is a flowchart of recording a video file according to an embodiment of the present invention;

FIG. 6 is a flow chart of the audio and video data composition according to the embodiment of the present invention;

FIG. 7 is a flow chart of a video mail notification transmission according to an embodiment of the present invention;

FIG. 8 is a flowchart illustrating a process of sending a video mail to a client by a server according to an embodiment of the present invention;

FIG. 9 is a flowchart of an implementation of step 805 provided by an embodiment of the invention;

FIG. 10 is a diagram illustrating server-selective multimedia streaming according to an embodiment of the present invention;

FIG. 11 is a block diagram of an apparatus according to an embodiment of the present invention;

Fig. 12 is a diagram illustrating an apparatus according to an embodiment of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in detail with reference to the accompanying drawings and specific embodiments.

The invention provides a data processing method based on a smart television, which comprises the following steps as shown in figure 1:

And step 101, the intelligent television serves as a client and communicates with the server through a customized communication identifier to obtain the network authorization of the server.

here, the step 101 can be specifically realized by the flow shown in fig. 2:

fig. 2 is a flowchart of network authorization provided in the embodiment of the present invention. As shown in fig. 2, the process includes the following steps:

In step 201, the client sends a connection request with a customized application ACCESS identifier [ CL _ USER _ ACCESS ] through HTTP.

here, the connection request carries the following parameters:

CLIENT _ ID: when the client ID is transmitted, the private key of the client is encrypted, the public key is used for decryption at the server, the real identity of the client is verified, and 11 bytes are occupied.

TIMESTAMP: the time for sending the message is also encrypted by a private key to verify the real identity of the client, and 11 bytes are occupied.

In step 202, the server matches the information of the client sending the connection request to determine whether to authorize the client to access.

here, the matched client information includes at least a user name and a password.

In step 203, the server sends a response with a customized ACCESS authorization identifier [ SR _ ACCESS _ ACCEPT ] to the client when authorized, or sends a response with a customized ACCESS denial identifier [ SR _ ACCESS _ REJECT ] to the client when not authorized.

Here, the response with [ SR _ ACCESS _ ACCEPT ] carries TIMESTAMP and SERVER _ ID. And the SERVER ID is the SERVER ID, the SERVER sends confirmation information to the client, the SERVER ID is encrypted by a SERVER private key, and the client uses a public key for decryption and is used for verifying the real identity of the SERVER and occupies 11 bytes.

in step 204, the client sends a capability message with a customized capability parameter identifier [ CL _ PROFILE ] via HTTP upon receiving a response with [ SR _ ACCESS _ ACCEPT ].

Here, the purpose of sending the capability message is to facilitate informing the server of the actual situation of the client for server management. Wherein, the performance parameters carried by the performance message at least comprise:

the CPU master frequency (CPU _ FREQ) of the client occupies 10 bytes.

the maximum memory size (MEM) supported by the client player, takes 5 bytes.

the maximum Resolution (RES) supported by the client, takes 5 bytes.

network RATE (RATE) of the client, takes 6 bytes.

in step 205, the server records the performance parameters in the performance message and returns a response with the customized performance validation identifier [ SR _ PROFILE _ DONE ].

Thus, the flow shown in fig. 2 is completed.

The network authorization is completed through the process shown in fig. 2.

And 102, when the client needs to send the video mail, logging in a server by using a network Identification (ID) authorized by the server, and acquiring an HTML page embedded into the client by the server after logging in.

And 103, processing the HTML page by the client.

Preferably, in the invention, the HTML page mainly comprises a toolbar, a mail editing area, a transceiving bar and a video library. Based on this, in this step 103, the client processing the HTML page specifically includes:

in addition to the options of inserting pictures or attachments and the like, the invention also adds the options of inserting videos and canceling transmission in the toolbar part of the HTML page.

1) For the video insertion option, a video selection area is added to the mail editing area, and the video selection area provides a tag for the user to select to add a local video or record a video, which is described in detail below.

2) for the option of canceling the sending, the option is selectable before the server delivers the mail, otherwise the option is not selectable.

in the mail edit area part of the HTML page, the content of the mail can be composed, and when the user selects to send the local video file in the video selection area, the local video file is uploaded to the server, and when the user selects to send the recorded video file, the size of the video area is adjusted, the recording device is opened to record the video file with the video area size mapping video resolution, and the recorded video file is uploaded to the server.

in the receiving and dispatching bar part of the HTML page, preferably, the receiving and dispatching bar consists of a sending bar and an receiving bar, and a user can click on the components according to the receiving and dispatching requirements.

1) For the mail sending column, the user can add the address of the receiver, the address of the transcriber and the title content, and click a sending button to send the mail.

2) The recipient column displays information such as the sender, the recipient, the transcription person, the title content, and the transmission time of the mail.

In the video library part of the HTML page, a video to be sent can be displayed on a video preview interface in a thumbnail mode, a user can enter the video preview interface by clicking, and a trigger mode is set to trigger, for example, a long-time press is carried out to insert the video into a mail editing area.

Thus, the processing of the HTML page can be completed as described above.

and 104, after uploading the video file to the server, the client informs the server to send the video file, packages the HTML page according to the format understood by the mail generator by the server to generate a video mail, and sends the video mail to a receiver.

here, the server packages the HTML page in a format understood by the mail generator in step 104 to generate a video mail, as described in detail below.

thus, the flow shown in fig. 1 is completed.

As can be seen from the flow of sending the video mail by the client shown in fig. 1, in the present invention, the user does not need to spend a lot of time uploading a large capacity of attachments, and the uploading can be completed when the video is recorded.

In the flow shown in fig. 1, in step 103, before the client starts recording the video, it is further required to communicate with the server through the customized communication identifier to confirm whether the server authorizes recording the video.

The communication flow mainly includes the steps shown in fig. 3:

In step 301, the client sends a connection request with a custom recorded VIDEO identifier [ CL _ a _ NEW _ VIDEO ] via HTTP.

step 302, after receiving the request, the server dynamically allocates a storage space for the client.

wherein, the dynamically allocated storage space may be: firstly, a storage space with L, for example, 20M, and a buffer storage with L, for example, 20M, are preset, and after the recorded content exceeds the preset storage space, the storage space is doubled and increased, but the size of the buffer storage keeps the L, for example, 20M, that is, the allocated storage space mm _ storage is M +20, (M is 20, 40, 80, …, 10x2n, n is 1, 2, …), and taking the L as 20M as an example, the allocated storage space is as shown in fig. 4. By allocating the storage space in this way, it can be ensured that the storage space is the size of the recorded video file after the recording is completed.

and step 303, the server sends a response with a self-defined authorized recording identifier [ SR _ PVR _ ACCEPT ] or a recording refusal identifier [ SR _ PVR _ REJECT ] to the client according to whether the recording condition is met.

and step 304, recording the video file when the client receives the response with the [ SR _ PVR _ ACCEPT ], and closing recording communication when the client receives the response with the [ SR _ PVR _ REJECT ].

The flow shown in fig. 3 is completed.

Preferably, in the present invention, the recording of the video file by the client in step 103 mainly includes the process shown in fig. 5:

Referring to fig. 5, fig. 5 is a flowchart for recording a video file according to an embodiment of the present invention. As shown in fig. 5, the process may include the following steps:

step 501, collecting audio and video data.

here, the audio and video data may be collected through a microphone and a camera connected to the client, respectively.

When audio and video data are collected, audio and video collecting parameters can be configured according to input of a user, and for videos, the collecting parameters can comprise: configuration parameters such as resolution, YUV format, brightness, chroma, saturation and the like; for audio, its acquisition parameters may include: sampling rate, bit depth, etc.

It should be noted that, in the present invention, the acquisition parameters of the audio and video may be adjusted by the user according to the actual requirements.

In addition, it should be noted that, in the present invention, it is preferable that the audio and video acquisition interval is smaller than the synchronization constraint interval.

step 502, encoding the collected audio and video data.

in this step 502, when encoding audio and video data, the encoding parameters of the audio and video may be configured according to the input of the user, and for the video, the encoding parameters may include: encoding format, frame rate, GOP, bit rate of code stream, etc.; for audio, its encoding parameters may include: audio format, bit rate of the code stream, etc.; it should be noted that, in the present invention, the encoding parameters of the audio and video can be adjusted by the user according to the actual requirements.

Step 503, the encoded audio and video are composited.

in this step 503, when the audio and video are combined, the video file duration or the total file length limit may be set according to the container type of the video file input by the user, and the original data packet of the audio and video is packed. In order to ensure the synchronization of audio and video coding, the time stamp of the audio and video should be recorded and added to the header of the audio and video frame.

in the invention, because the video data is larger, the audio data is much smaller, and the time consumed by collecting and coding the video frame is not completely the same each time, so that the video data can also be collected for a complete number of frames when the collection of one frame of audio data is not finished, in the invention, the system time is used as a reference clock, the video frame is selected as a reference point, the processed audio frames are sequentially cached in an audio buffer area in the process of collecting and coding one frame of video data, after the collection and coding of one frame of video data are finished, the system reference clock is used as a time stamp and added into a video frame header, all the audio data in the audio buffer area are obtained as one audio frame, and the same system reference clock is added into the frame header as time stamp information, and the specific flow is shown in fig. 6.

The flow shown in fig. 5 is completed.

In the present invention, in the above step 104, the client notifying the server to send the video mail can be implemented by the flow shown in fig. 7:

Referring to fig. 7, fig. 7 is a flowchart of sending a notification of a video mail according to an embodiment of the present invention. As shown in fig. 7, the process may include the following steps:

In step 701, the client sends a connection request with a customized sending email identifier [ CL _ a _ NEW _ VE ] via HTTP.

In step 702, the server determines whether to deliver the video mail to the mailbox of the recipient according to the HTML page processed in step 102.

in step 703, the server sends a response with the customized transmission authorization identifier [ SR _ VE _ ACCEPT ] to the client if the determination result of step 702 is yes, and sends a response with the customized transmission rejection identifier [ SR _ VE _ REJECT ] to the client if the determination result of step 702 is no.

In step 704, the client clicks the send option on the triggered HTML page to notify the server to send the video mail when receiving the response with [ SR _ VE _ ACCEPT ].

the flow shown in fig. 7 is completed.

In the present invention, in step 104, the step of packing the HTML page by the server according to the format understood by the mail generator to generate a video mail may specifically be:

Under the mail generating device, mail information files, video files, script updating files, temporary files and the like are defined, after a client finishes writing mails and uploading videos, a video mail generator automatically acquires mail contents to the mail information files and the video files, updates the script of the video mail generator and finally generates new mails, namely: html.

Txt, the mail content is obtained and is arranged into a file format understood by a video mail generator, such as a txt file format, and the txt file format mainly comprises a sender, a receiver, a copying person, a title, content and the like.

txt, video file defines mediainfo, obtains video path and video format information, and the like.

And the script updating file defines VideoE mail.sh, and updates the contents of the mail information file and the video file to a new mail.

Updating mail information, defining variable FileC to inquire mail information file, and inquiring txt file in mail directory by searching suffix name method, namely: and $ { FileC: -4} -, ". txt", and updating the mail content step by step according to a predefined variable Level, wherein the Level value is defined by whether the updated content is NULL, and the specific implementation is as follows:

level1, sender information update: html is updated to addfrom.html;

Level2, recipient information update: html is updated to addfrom.

Level3, updating the information of the carbon copy: html is updated to addcc.html;

Level4, title information update: html is updated to addsubject.html;

Level5, mail content update: html is updated to addproject html;

Updating video file information, defining a variable URL to be responsible for video linking, defining a variable MediaC to be responsible for video content, and continuously updating mail content, wherein the specific implementation is as follows:

Level6, video information update: html is updated to video _ email _ id.

When the video mail is generated, because various templates are defined to control and update the mail content, only necessary data needs to be filled, for example, a video is inserted into an HTML page, only the attribute and the value of the video tag in table 1 need to be packed and sent to a Flash player, which is specifically as follows:

TABLE 1

this completes the description of how the server packages the HTML page in a format understood by the mail generator to generate a video mail.

In the above description, how the client sends the video mail is described, in the present invention, how the server sends the video mail to the client is also described, and the following description is given:

Referring to fig. 8, fig. 8 is a flowchart illustrating a process of sending a video mail to a client by a server according to an embodiment of the present invention. As shown in fig. 8, the process may include the following steps:

step 801, the server performs network authorization on the smart television serving as the client.

for the specific implementation of step 801, reference may be made to the flow shown in fig. 2, which is not described herein again.

in step 802, the server receives a connection request with a customized receiving mail identifier [ CL _ VE _ PATH ] sent by the client via HTTP.

Step 803, the server judges whether the mail requested by the connection request is expired or not, if not, returns a response with a self-defined authorization receiving mail identifier [ SR _ VE _ PATH _ ACCEPT ] to the client, and if so, returns a response with a self-defined refusal receiving mail identifier [ SR _ VE _ PATH _ REJECT ] to the client.

step 804, the server receives a play request with a play adjustment identifier [ CL _ ADJUST ] sent by the client, where the play request is sent when the client receives a response with [ SR _ VE _ PATH _ ACCEPT ].

step 805, the server controls the multimedia stream to be played at the client according to the parameters in the playing request, where the parameters include a delay parameter and a network speed parameter. The method specifically comprises the following steps: the server analyzes the DELAY parameter FRAME _ DELAY and the network speed parameter RATE _ CHANGE in the playing request, and adjusts the code RATE of multimedia stream transmission in real time according to the FRAME _ DELAY, RATE _ CHANGE and the performance parameters of the client so as to control the smooth playing of the transmitted multimedia stream at the client.

here, FRAME _ DELAY is a decoding DELAY time generated by the client during playing, and occupies 13 bytes. And RATE _ CHANGE is the actual network speed of the client during the playing process, and occupies 13 bytes.

The flow shown in fig. 8 is completed.

it should be noted that, in the present invention, all the custom identifiers between the client and the server are added through the data segment in the packet header, the custom identifier value occupies a width of 16 bits, and the size of the identifier is set according to the bit width value.

In the present invention, the server is based on the cloud storage mode, and in step 805, it may adjust the bitrate of the multimedia stream transmission in real time according to the FRAME _ DELAY, the RATE _ CHANGE, and the performance parameter of the client, and may be implemented by the process shown in fig. 9:

referring to fig. 9, fig. 9 is a flowchart of step 805 implemented by an embodiment of the present invention. As shown in fig. 9, the process may include the following steps:

step 901, when the FRAME _ DELAY is greater than the maximum value of the decoding time per FRAME, step 902 is executed, when the FRAME _ DELAY is less than 1/2fps, step 903 is executed, and when the RATE _ CHANGE is less than the code RATE of the current layer, step 904 is executed.

in the invention, a server side supports a scalable multimedia stream with N spatial layers in a spatial domain and each spatial layer has no more than 3 temporal layers in a temporal domain, if the spatial layer of the multimedia stream is { L0, L1, L2, L3, …, LN-1}, and if each spatial layer is attached with 3 temporal layers and the maximum frame rate is fps, fps/2, fps/4 respectively at each spatial level.

wherein, each layer in the multimedia stream is determined by the following decisions:

Let RX be the code rate of layer X, if the spatial resolutions from L0 to LN-1 are larger and larger, the code rate from L0 to LN-1 is increased, if the network speed is R, the maximum resolution supported by the device is res, and the initial layer number K is determined as follows.

When a layer k exists, the resolution ratio of the layer k is closest to the resolution ratio of equipment, the code rate is smaller than the current network speed multiplied by lambda, and the lambda is a coefficient larger than 1, the lambda value can ensure the transmission of the audio part and the smooth playing under a certain network speed fluctuation condition.

if no layer exists, selecting the layer with code rate smaller than lambdar and closest to lambdar.

and if the lambada R is smaller than the code rate of the basic layer, selecting the basic layer and the lowest time domain layering layer as the initial layering.

In the invention, if the FRAME _ DELAY is greater than the maximum value of each FRAME decoding time, it indicates that the video cannot be smoothly played at the client, and the server adjusts the transmitted multimedia stream, specifically see step 902; when the FRAME _ DELAY is smaller than 1/2fps, it means that the client CPU still has a large amount of idle time, and the code stream quality can be improved accordingly, as described in step 903. When the RATE _ CHANGE is smaller than the code RATE of the current layer, it means that there is no need to adjust the code stream at this time, and the code stream transmission can be continued according to the original mode.

and 902, comparing the recorded CPU _ FREQ and QLow in the client performance parameters, wherein the QLow is a CPU main frequency value required by the current layer to support smooth playing, if the former is smaller than the latter, selecting multimedia stream transmission of a layer lower in a space domain relative to the current layer, and if the former is larger than or equal to the latter, selecting multimedia stream transmission of a layer lower in a time domain relative to the current layer.

Step 903, when the RATE _ CHANGE is greater than R 'and greater than R ″, selecting a higher-layer multimedia stream transmission in the spatial domain relative to the current layer, and when the RATE _ CHANGE is greater than R' and less than R ″, selecting a higher-layer multimedia stream transmission in the temporal domain relative to the current layer; wherein, R 'is a code rate one layer higher in time domain relative to the current layer, and R' is a code rate one layer higher in space domain relative to the current layer.

Step 904, end adjustment.

The flow shown in fig. 9 is completed.

In the flow shown in fig. 9, the server selecting the multimedia streaming can be implemented by the following steps:

reading the selected multimedia stream through a file reader and putting the multimedia stream into a memory;

then separating the video data and the audio data in the multimedia stream by using a file demultiplexer;

inputting only video data into a NAL packet selector, so that the video data is organized into a plurality of NAL units after being input into the NAL packet selector;

and selecting the NAL unit input by the NAL packet selector, then respectively sending the audio data and the processed video data into a network packet packing device, packing the audio data and the processed video data, and finally outputting the audio data and the processed video data as a network data stream.

Fig. 10 shows a schematic diagram of server-selective multimedia streaming.

in the present invention, the lower the number of layers is, the smaller the code rate is, in the decoding process, the higher layer decoding may depend on the lower layer data, and the lower layer data does not depend on the higher layer data, therefore, the selection of the NAL packet input by the NAL packet selector may specifically be: mapping the specific rate specification of the multimedia stream decided by the server to be transmitted to corresponding target dependency _ id (denoted by U) and target temporal _ id (denoted by V), reading NAL units with NAL Type being a first value from all NAL units input by the NAL packet selector, wherein the first value represents the NAL unit of an SVC extension header, which can be specifically 14, if the dependency _ id of the read NAL unit is less than or equal to U, and the temporal _ id is less than or equal to V, indicating that all NAL units after the NAL unit are required to be selected and sent, otherwise, skipping all NAL packets after the NAL unit. If a NAL packet with the NAL Type equal to the first value is encountered again, the judgment is carried out again.

Thus, the description of the method provided by the present invention is completed.

the following describes the apparatus provided by the present invention:

referring to fig. 11, fig. 11 is a structural diagram of an apparatus according to an embodiment of the present invention. The device is an intelligent television, is applied to the process shown in fig. 1, and can include:

The communication module is used for communicating with the server through a self-defined communication identifier so as to acquire the network authorization of the server;

the login module is used for logging in a server by using the network identification ID authorized by the server when the device needs to send the video mail, and acquiring an HTML (hypertext markup language) page embedded into the client by the server after logging in; the HTML page at least comprises: a receiving and sending bar, a toolbar and a mail editing area;

a page processing module, configured to perform the following processing on the HTML page: at least adding receiver information, sender information, a mail title and mail sending time in a receiving and sending column of the HTML page, selecting and inserting a video option in a toolbar of the HTML page to enable a mail editing area of the HTML page to add a video selection area, writing mail content in the mail editing area of the HTML page, uploading a local video file to a server when the video selection area selects to send the local video file, adjusting the size of the video area when the recorded video file is selected to be sent, opening a recording device to start recording the video file by mapping the video resolution with the size of the video area, and uploading the recorded video file to the server;

and the notification module is used for notifying the server to send the video file after the page processing module finishes uploading the video file to the server, packaging the HTML page by the server according to a format understood by the mail generator to generate a video mail, and sending the video mail to a receiver.

preferably, in the present invention, the page processing module records the video file by the following steps:

Collecting audio and video data;

Encoding the collected audio and video data;

The system time is used as a reference clock, video frames are selected as reference points, processed audio frames are sequentially cached in an audio buffer area in the process of collecting and coding a frame of video data, after the collection and coding of the frame of video data are completed, the current system time is used as a time stamp to be added into a video frame head, all audio data in the audio buffer area are obtained to be used as an audio frame, and the same system time is added into the audio frame head to be used as the time stamp.

In addition, the present invention also provides another data processing apparatus based on a smart television, as shown in fig. 12, where the apparatus is a server, and includes:

the network processing module is used for performing network authorization on the intelligent television as the client;

The mail request processing module is used for receiving a connection request with a self-defined mail receiving identifier sent by the client through HTTP, identifying whether a mail requested by the connection request is expired, if not, returning a response with a self-defined authorized mail receiving identifier to the client, and if so, returning a response with a self-defined refused mail receiving identifier to the client;

The playing request processing module is used for receiving a playing request with a playing adjustment identifier sent by the client and controlling the multimedia stream to be played at the client according to parameters in the playing request; wherein the play request is sent when the client receives a response with an authorized mail receiving identifier, and the parameters comprise a delay parameter and a network speed parameter.

the playing request processing module controls the multimedia stream to be played at the client according to the parameters in the playing request, and specifically includes: analyzing the DELAY parameter FRAME _ DELAY and the network speed parameter RATE _ CHANGE in the playing request, and adjusting the code RATE of multimedia stream transmission in real time according to the FRAME _ DELAY, RATE _ CHANGE and the performance parameters of the client so as to control the transmitted multimedia stream to be played normally at the client.

In the invention, the multimedia stream has N layers of space in a space domain, and each space layer has no more than three time layers in a time domain;

based on this, the playing request processing module adjusts the transmission code RATE in real time according to the FRAME _ DELAY, the RATE _ CHANGE, and the performance parameter of the client, and includes:

if the FRAME DELAY is greater than the maximum value of the decoding time per FRAME, then,

comparing the recorded CPU _ FREQ and QLow in the client performance parameters, wherein the QLow is a CPU main frequency value required by the support of smooth playing of the current layer, if the former is smaller than the latter, the multimedia stream transmission of a layer lower than the current layer on a spatial domain is selected, and if the former is larger than or equal to the latter, the multimedia stream transmission of a layer lower than the current layer on a temporal domain is selected;

if the FRAME _ DELAY is less than 1/2fps, selecting one-layer higher multimedia stream transmission in spatial domain relative to the current layer when the RATE _ CHANGE is greater than R 'and greater than R ″, and selecting one-layer higher multimedia stream transmission in temporal domain relative to the current layer when the RATE _ CHANGE is greater than R' and less than R ″; wherein, R 'is a code rate one layer higher in time domain relative to the current layer, and R' is a code rate one layer higher in space domain relative to the current layer.

The multimedia stream includes an audio stream and/or a video stream.

In particular, in a preferred embodiment, the video stream has N spatial layers in spatial domain, each spatial layer having no more than three temporal layers in temporal domain;

The playing request processing module adjusts the transmission code RATE in real time according to the FRAME _ DELAY, the RATE _ CHANGE and the performance parameter of the client, and comprises the following steps:

if the FRAME DELAY is greater than the maximum value of the decoding time per FRAME, then,

Comparing the recorded CPU _ FREQ and QLow in the client performance parameters, wherein the QLow is a CPU main frequency value required by the support of smooth playing of the current layer, if the former is smaller than the latter, video streaming transmission of a layer lower than the current layer on a spatial domain is selected, and if the former is larger than or equal to the latter, video streaming transmission of a layer lower than the current layer on a temporal domain is selected; the video stream has N layers of space in a spatial domain, and each spatial layer has no more than three temporal layers in a temporal domain;

If the FRAME _ DELAY is less than 1/2fps, selecting video stream transmission of one layer higher in spatial domain relative to the current layer when the RATE _ CHANGE is greater than R 'and greater than R ″, and selecting video stream transmission of one layer higher in temporal domain relative to the current layer when the RATE _ CHANGE is greater than R' and less than R ″; wherein, R 'is a code rate one layer higher in time domain relative to the current layer, and R' is a code rate one layer higher in space domain relative to the current layer.

thus, the description of the apparatus provided by the present invention is completed.

The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.

Claims (6)

1. a data processing method based on a smart television is characterized by comprising the following steps:

the server carries out network authorization on the intelligent television as the client;

the server receives a connection request with a self-defined mail receiving identifier sent by the client through HTTP, identifies whether a mail requested by the connection request is expired, if not, returns a response with a self-defined authorized mail receiving identifier to the client, and if so, returns a response with a self-defined refused mail receiving identifier to the client;

The server receives a playing request with a playing adjustment identifier sent by the client, wherein the playing request is sent when the client receives a response with an authorized mail receiving identifier;

the server controls the multimedia stream to be played at the client according to the parameters in the playing request, and the method comprises the following steps:

the server analyzes a DELAY parameter FRAME _ DELAY and a network speed parameter RATE _ CHANGE in the playing request, and adjusts the code RATE of multimedia stream transmission in real time according to the FRAME _ DELAY, RATE _ CHANGE and the performance parameters of the client so as to control the normal playing of the transmitted multimedia stream at the client.

2. The method according to claim 1, characterized in that said performance parameters comprise at least: CPU master frequency CPU _ FREQ, maximum memory size MEM supported by a client player, maximum resolution RES supported by a client and network RATE RATE of the client;

The multimedia stream has a space with N layers in a spatial domain, and each spatial layer has no more than three temporal layers in a temporal domain;

the adjusting the transmission code RATE in real time according to the FRAME _ DELAY, the RATE _ CHANGE, and the performance parameter of the client includes:

if the FRAME DELAY is greater than the maximum value of the decoding time per FRAME, then,

Comparing the recorded CPU _ FREQ and QLow in the client performance parameters, wherein the QLow is a CPU main frequency value required by the support of smooth playing of the current layer, if the former is smaller than the latter, the multimedia stream transmission of a layer lower than the current layer in a space domain is selected, and if the former is larger than or equal to the latter, the multimedia stream transmission of a layer lower than the current layer in a time domain is selected;

if the FRAME _ DELAY is less than 1/(2fps), selecting one-layer higher multimedia stream transmission in spatial domain relative to the current layer when the RATE _ CHANGE is greater than R 'and greater than R ″, and selecting one-layer higher multimedia stream transmission in temporal domain relative to the current layer when the RATE _ CHANGE is greater than R' and less than R ″; wherein, R 'is a code rate one layer higher in time domain relative to the current layer, R' is a code rate one layer higher in space domain relative to the current layer, and fps is the maximum frame rate.

3. the method of claim 2, wherein selecting multimedia streaming comprises:

Reading the selected multimedia stream and putting the multimedia stream into a memory;

distinguishing video data and audio data in the multimedia stream, performing decision processing on the video data, and selecting the video data to be transmitted;

and packaging and transmitting the selected video data and the audio data.

4. The method of claim 1, wherein the server performing network authorization on the smart television as the client comprises:

the server receives a connection request with a self-defined application access identifier sent by the client through HTTP;

the server matches the information of the client sending the connection request, if the matching is successful, a response with a self-defined access authorization identifier is sent to the client, and if the matching is failed, a response with a self-defined access denial identifier is sent to the client;

The server receives a performance message which is sent by the client through HTTP and has a self-defined performance parameter identifier, wherein the performance message is sent when the client receives a response with an access authorization identifier and carries the performance parameter of the client;

And the server records the performance parameters carried by the performance message and returns a performance confirmation to the client.

5. A data processing device based on an intelligent television is characterized in that the device is a server and comprises:

the network processing module is used for performing network authorization on the intelligent television as the client;

the mail request processing module is used for receiving a connection request with a self-defined mail receiving identifier sent by the client through HTTP, identifying whether a mail requested by the connection request is expired, if not, returning a response with a self-defined authorized mail receiving identifier to the client, and if so, returning a response with a self-defined refused mail receiving identifier to the client;

The playing request processing module is used for receiving a playing request with a playing adjustment identifier sent by the client and controlling the multimedia stream to be played at the client according to parameters in the playing request; wherein the play request is sent when the client receives a reply with an authorized collection mail identifier; the controlling the multimedia stream to be played at the client according to the parameters in the playing request specifically includes: analyzing a DELAY parameter FRAME _ DELAY and a network speed parameter RATE _ CHANGE in the playing request, and adjusting a transmission code RATE in real time according to the FRAME _ DELAY, the RATE _ CHANGE and the performance parameters of the client so as to control the playing smoothness of the client.

6. The apparatus of claim 5,

the playing request processing module adjusts the transmission code RATE in real time according to the FRAME _ DELAY, the RATE _ CHANGE and the performance parameter of the client, and comprises the following steps:

if the FRAME DELAY is greater than the maximum value of the decoding time per FRAME, then,

Comparing the recorded CPU _ FREQ and QLow in the client performance parameters, wherein the QLow is a CPU main frequency value required by the support of smooth playing of the current layer, if the former is smaller than the latter, the multimedia stream transmission of a layer lower than the current layer in a space domain is selected, and if the former is larger than or equal to the latter, the multimedia stream transmission of a layer lower than the current layer in a time domain is selected; the multimedia stream has a space with N layers in a spatial domain, and each spatial layer has no more than three temporal layers in a temporal domain;

If the FRAME _ DELAY is less than 1/(2fps), selecting one-layer higher multimedia stream transmission in spatial domain relative to the current layer when the RATE _ CHANGE is greater than R 'and greater than R ″, and selecting one-layer higher multimedia stream transmission in temporal domain relative to the current layer when the RATE _ CHANGE is greater than R' and less than R ″; wherein, R 'is a code rate one layer higher in time domain relative to the current layer, R' is a code rate one layer higher in space domain relative to the current layer, and fps is the maximum frame rate.