CN106534859B - Image transmission method and device based on SPICE protocol - Google Patents
Image transmission method and device based on SPICE protocol Download PDFInfo
- Publication number
- CN106534859B CN106534859B CN201510580296.3A CN201510580296A CN106534859B CN 106534859 B CN106534859 B CN 106534859B CN 201510580296 A CN201510580296 A CN 201510580296A CN 106534859 B CN106534859 B CN 106534859B
- Authority
- CN
- China
- Prior art keywords
- macro block
- image
- frame image
- current frame
- server
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The invention discloses an image transmission method based on an SPICE protocol, which comprises the following steps: when receiving a change image of a current frame image and a previous frame image sent by a server based on an SPICE protocol, synthesizing the change image and the previous frame image to obtain the current frame image; dividing the obtained current frame image into a plurality of macro blocks, and respectively executing corresponding coding compression processing on each macro block; and transmitting the coded and compressed data after the plurality of macro blocks are subjected to coding and compression processing to the server so that the server can send the coded and compressed data to a terminal, and the terminal can decode the coded and compressed data to obtain the current frame image when receiving the coded and compressed data. The invention also discloses an image transmission device based on the SPICE protocol. The invention reduces the bandwidth occupied by the server when transmitting the image based on the SPICE protocol.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to an image transmission method and device based on an SPICE protocol.
Background
With the continuous development of computer and network technologies, cloud computing becomes more and more important, and a virtual desktop is a typical application of the cloud computing technology and can provide remote computer desktop services for users. The SPICE (Simple Protocol for Independent Computing Environment) Protocol is a Protocol for a remote desktop, and a change partial image of a desktop image can be directly transmitted to a user terminal based on the SPICE Protocol without transmitting the whole desktop image. But because of adopting the mode of directly transmitting the image, the occupied bandwidth is high when the image is transmitted.
Disclosure of Invention
The invention mainly aims to provide an image transmission method and device based on an SPICE protocol, and aims to solve the technical problem of high occupied bandwidth when images are transmitted based on the SPICE protocol.
In order to achieve the above object, the present invention provides an image transmission method based on SPICE protocol, which comprises the following steps:
when receiving a change image of a current frame image and a previous frame image sent by a server based on an SPICE protocol, synthesizing the change image and the previous frame image to obtain the current frame image;
dividing the obtained current frame image into a plurality of macro blocks, and respectively executing corresponding coding compression processing on each macro block;
and transmitting the coded and compressed data after the plurality of macro blocks are subjected to coding and compression processing to the server so that the server can send the coded and compressed data to a terminal, and the terminal can decode the coded and compressed data to obtain the current frame image when receiving the coded and compressed data.
Preferably, the step of dividing the obtained current frame image into a plurality of macroblocks, and performing the corresponding encoding compression processing on each macroblock respectively comprises:
dividing the obtained current frame image into a plurality of macro blocks, and respectively judging whether each macro block is a P-SKIP forward prediction macro block;
when the macro block is a P-SKIP macro block, performing P-SKIP macro block coding compression processing on the macro block;
and when the macro block is not the P-SKIP macro block, performing P macro block coding compression processing on the macro block.
Preferably, the step of dividing the obtained current frame image into a plurality of macroblocks and respectively determining whether each macroblock is a P-SKIP macroblock includes:
determining a macro block coordinate range corresponding to the changed image;
dividing the obtained current frame image into a plurality of macro blocks, and determining the reference coordinate of each macro block;
sequentially judging whether the reference coordinate of each macro block is in the macro block coordinate range corresponding to the changed image;
when the reference coordinate of the macro block is in the macro block coordinate range corresponding to the changed image, judging that the macro block is a P-SKIP macro block;
and when the reference coordinate of the macro block is not in the macro block coordinate range corresponding to the changed image, judging that the macro block is not a P-SKIP macro block.
Preferably, the step of dividing the obtained current frame image into a plurality of macro blocks and determining the reference coordinate of each macro block includes:
dividing the obtained current frame image into a plurality of macro blocks, and determining a reference point of each macro block;
and acquiring the reference point coordinate of each macro block, and setting the reference point coordinate of each macro block as the reference coordinate of the macro block.
Preferably, before the step of synthesizing the change image with the previous frame image to obtain the current frame image when receiving the change image of the current frame image and the previous frame image sent by the server based on the SPICE protocol, the image transmission method based on the SPICE protocol further includes the steps of:
when a frame of complete image sent by a server based on an SPICE protocol and an IDR instant decoding refresh frame coding instruction corresponding to the complete image are received, carrying out IDR frame coding on the complete image;
and transmitting the encoded data to the server so that the server can send the encoded data to a terminal, and the terminal can decode the encoded data to obtain the complete image when receiving the encoded data.
In addition, in order to achieve the above object, the present invention further provides an image transmission apparatus based on SPICE protocol, including:
the synthesis module is used for synthesizing a change image of a current frame image and a previous frame image when receiving the change image of the current frame image and the previous frame image which are sent by a server based on the SPICE protocol so as to obtain the current frame image;
the coding module is used for dividing the obtained current frame image into a plurality of macro blocks and respectively executing corresponding coding compression processing on each macro block;
and the sending module is used for transmitting the coded compressed data after the plurality of macro blocks perform the coding compression processing to the server so that the server sends the coded compressed data to a terminal, and the terminal decodes the coded compressed data to obtain the current frame image when receiving the coded compressed data.
Preferably, the encoding module comprises:
the judging unit is used for dividing the obtained current frame image into a plurality of macro blocks and respectively judging whether each macro block is a P-SKIP macro block;
the coding unit is used for executing P-SKIP macro block coding compression processing on the macro block when the macro block is the P-SKIP macro block; and when the macro block is not the P-SKIP macro block, performing P macro block coding compression processing on the macro block.
Preferably, the judging unit includes:
the first determining subunit is used for determining a macroblock coordinate range corresponding to the changed image;
a second determining subunit, configured to divide the obtained current frame image into a plurality of macroblocks, and determine a reference coordinate of each macroblock;
the first judgment subunit is used for sequentially judging whether the reference coordinate of each macro block is in the macro block coordinate range corresponding to the changed image;
the second judgment subunit is used for judging that the macro block is a P-SKIP macro block when the reference coordinate of the macro block is in the macro block coordinate range corresponding to the changed image; and when the reference coordinate of the macro block is not in the macro block coordinate range corresponding to the changed image, judging that the macro block is not a P-SKIP macro block.
Preferably, the second determining subunit is configured to:
dividing the obtained current frame image into a plurality of macro blocks, and determining a reference point of each macro block;
and acquiring the reference point coordinate of each macro block, and setting the reference point coordinate of each macro block as the reference coordinate of the macro block.
Preferably, the encoding module is further configured to perform IDR frame encoding on a complete picture when receiving a frame of complete picture sent by a server based on a SPICE protocol and an IDR frame encoding instruction corresponding to the complete picture;
the sending module is further configured to transmit the encoded data to the server, so that the server sends the encoded data to a terminal, and the terminal decodes the encoded data to obtain the complete image when receiving the encoded data.
In the image transmission method based on the SPICE protocol, when a server sends a current frame image and a previous frame image based on the SPICE protocol, the changed image and the previous frame image are synthesized to obtain the current frame image, the obtained current frame image is divided into a plurality of macro blocks, corresponding coding compression processing is respectively executed on each macro block, after the corresponding coding compression processing is executed on the macro blocks, the coding compression data of the macro blocks after the coding compression processing is executed are transmitted to the server, the server sends the received coding compression data to a terminal, and the terminal can obtain the current frame image after decoding. Therefore, the server does not need to directly send the image to the terminal, and a mode of sending the coded compressed data is adopted, so that the bandwidth occupied by the server is reduced.
Drawings
FIG. 1 is a schematic flow chart of a first embodiment of an image transmission method based on the SPICE protocol according to the present invention;
FIG. 2 is a detailed flowchart of the image transmission method based on SPICE protocol according to the second embodiment of the present invention, in which the corresponding encoding compression processing is performed on each macro block;
FIG. 3 is a flowchart illustrating a third embodiment of an image transmission method based on SPICE protocol according to the present invention;
FIG. 4 is a functional block diagram of a first embodiment of an image transmission apparatus based on SPICE protocol according to the present invention;
fig. 5 is a schematic diagram of a detailed functional module of an encoding module in a second embodiment of the image transmission apparatus based on the SPICE protocol.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The invention provides an image transmission method based on a SPICE protocol, and referring to FIG. 1, FIG. 1 is a flow diagram of a first embodiment of the image transmission method based on the SPICE protocol.
In this embodiment, the image transmission method based on SPICE protocol includes the following steps:
step S10, when receiving a changed image of a current frame image and a previous frame image sent by a server based on the SPICE protocol, synthesizing the changed image and the previous frame image to obtain a current frame image;
when the server sends the current frame image to the user terminal based on the SPICE Protocol (Simple Protocol for Independent Computing Environment), the server only needs to send the change image of the change part of the current frame image and the previous frame image to the terminal, and does not need to send the complete current frame image to the terminal. In this embodiment, when the server sends the changed image of the changed part of the current frame image and the previous frame image to the user terminal based on the SPICE protocol, the server first sends the changed image to the encoder, and the encoder is preferably an h.264 encoder. When the H.264 encoder receives a change image of a current frame image and a previous frame image which are sent by the server based on the SPICE protocol, the received change image and the stored previous frame image are synthesized to obtain the current frame image corresponding to the change image. And, the h.264 encoder replaces the saved previous frame image with the obtained current frame image, and saves the obtained current frame image. When a next frame image and a change image of the current frame image which are sent by a server based on the SPICE protocol are received, the change images of the next frame image and the current frame image are synthesized with the stored current frame image, and therefore the next frame image is obtained.
Step S20, dividing the obtained current frame image into a plurality of macroblocks, and performing corresponding encoding and compression processing on each macroblock;
the h.264 encoder divides the current frame image into a plurality of macroblocks after obtaining the current frame image in step S10. For example, the current frame image is divided into several macroblocks of 16 × 16 pixels. Then, all the macroblocks are traversed, and corresponding encoding compression processing is performed on each macroblock in turn, for example, P macroblock processing of h.264 standard is performed on each macroblock.
Step S30, transmitting the encoded compressed data obtained by performing the encoding and compressing process on the macro blocks to the server, so that the server sends the encoded compressed data to a terminal, and the terminal decodes the encoded compressed data when receiving the encoded compressed data to obtain the current frame image.
And when the corresponding coding and compression processing on all the macro blocks is finished, namely the macro block traversal is finished, transmitting the coding and compression data after the corresponding coding and compression processing on all the macro blocks to the server. And when the server receives the coded compressed data, sending the coded compressed data to a user terminal. And when the user terminal receives the coded compressed data, decoding the coded compressed data to obtain the current frame image.
The following describes the image transmission method based on the SPICE protocol in detail by taking a cloud desktop system as an example. When the cloud desktop server sends the change desktop image of the change part of the current frame desktop image and the previous frame desktop image to the user terminal based on the SPICE protocol, the cloud desktop server firstly sends the change desktop image to the H.264 encoder. And when the H.264 encoder receives the change desktop image sent by the cloud desktop server based on the SPICE protocol, synthesizing the received change desktop image with the stored previous frame desktop image to obtain a current frame desktop image corresponding to the change desktop image. Then, the current frame desktop image is divided into a plurality of macro blocks, and corresponding coding compression processing is performed on each macro block in sequence. And after the corresponding coding compression processing is performed on all the macro blocks, transmitting the coding compression data after the corresponding coding compression processing is performed on all the macro blocks to the cloud desktop server. And when the cloud desktop server receives the coded compressed data, the coded compressed data is sent to a thin terminal of a user. And when the thin terminal of the user receives the coded compressed data, decoding the coded compressed data to obtain the current frame desktop image.
According to the scheme provided by the embodiment, when an encoder receives a changed image of a current frame image and a previous frame image sent by a server based on a SPICE protocol, the changed image and the previous frame image are synthesized to obtain the current frame image, the obtained current frame image is divided into a plurality of macro blocks, corresponding coding compression processing is respectively executed on each macro block, after the corresponding coding compression processing is executed on the macro blocks, the coded compressed data obtained after the coding compression processing is executed on the macro blocks is transmitted to the server, the server sends the received coded compressed data to a user terminal, and the current frame image can be obtained after the terminal decodes the coded compressed data, so that the image does not need to be directly sent to the user terminal, and the bandwidth occupied by the terminal is reduced by adopting a mode of sending the coded compressed data.
Further, as shown in fig. 2, a second embodiment of the image transmission method based on SPICE protocol of the present invention is proposed based on the first embodiment, and in this embodiment, the step S20 includes:
step S21, dividing the obtained current frame image into a plurality of macroblocks, and respectively determining whether each macroblock is a P-SKIP forward prediction macroblock;
in the present embodiment, in order to simplify the encoding compression process of the current frame image, after dividing the obtained current frame image into a plurality of macroblocks, different encoding compression processes are performed for different types of macroblocks. Specifically, after dividing the obtained current frame image into a plurality of macroblocks, first, whether each macroblock is a P-SKIP forward prediction macroblock is determined respectively. Specifically, the step S21 includes:
step a, determining a macro block coordinate range corresponding to the changed image;
b, dividing the obtained current frame image into a plurality of macro blocks, and determining the reference coordinate of each macro block;
c, sequentially judging whether the reference coordinate of each macro block is in the macro block coordinate range corresponding to the changed image;
d, when the reference coordinate of the macro block is in the macro block coordinate range corresponding to the changed image, judging that the macro block is a P-SKIP macro block;
and e, when the reference coordinate of the macro block is not in the macro block coordinate range corresponding to the changed image, judging that the macro block is not the P-SKIP macro block.
In this embodiment, when a changed image of a current frame image and a previous frame image sent by the server based on the SPICE protocol is received, a macroblock coordinate range corresponding to the changed image is determined. Preferably, the variant image is a rectangular region, and the image information of the variant image includes parameter information such as vertex coordinates (x, y), a width value, and a height value of the rectangular region, and YUV image data of the variant image. Preferably, the vertex coordinates (x, y) of the rectangular region are the coordinates of the upper left corner of the rectangular region. And calculating an x-axis coordinate range [ x _ Start, x _ End ] of the changed image after being converted into the macro block of 16 x 16 pixels according to the x coordinate value and the width value of the vertex coordinates (x, y), and calculating a y-axis coordinate range [ y _ Start, y _ End ] of the changed image after being converted into the macro block of 16 x 16 pixels according to the y coordinate value and the height value of the vertex coordinates (x, y).
After synthesizing the change image with the previous frame image to obtain a current frame image, dividing the obtained current frame image into a plurality of 16-by-16 pixel macro blocks, and determining the reference coordinate (x) of each macro blocki、yi). Specifically, in this embodiment, the step b includes:
f, dividing the obtained current frame image into a plurality of macro blocks, and determining a reference point of each macro block;
and g, acquiring the reference point coordinates of each macro block, and setting the reference point coordinates of each macro block as the reference coordinates of the macro block.
In the process of obtainingAfter the current frame image is divided into a plurality of 16-by-16 pixel macro blocks, the reference point of each macro block is determined. Preferably, the upper left corner of each macroblock is determined as the reference point for the macroblock. Then, the coordinates of the reference point of each macro block, that is, the coordinates of the upper left corner of each macro block are obtained. Setting the reference point coordinate of each macro block as the reference coordinate (x) of the macro blocki、yi) That is, it is preferable to set the coordinate of the upper left corner of each macroblock as the reference coordinate (x) of the macroblocki、yi)。
Then, according to the reference coordinate (x) of each macro blocki、yi) And the coordinate range [ x _ Start, x _ End ] of the macro block corresponding to the changed image]And [ y _ Start, y _ End]Determining the reference coordinate (x) of each macro blocki、yi) Whether the current macroblock coordinate range is in the macroblock coordinate range [ x _ Start, x _ End ] corresponding to the changed image]And [ y _ Start, y _ End]And (4) the following steps. If x _ Start is less than or equal to xiX _ End is less than or equal to x _ End and y _ Start is less than or equal to yiIf y _ End is less than or equal to y _ End, determining that the reference coordinate of the macro block is in the macro block coordinate range corresponding to the changed image; otherwise, determining that the reference coordinate of the macro block is out of the macro block coordinate range corresponding to the changed image. When the reference coordinate of the macro block is in the macro block coordinate range corresponding to the changed image, judging that the macro block is a P-SKIP macro block; and when the reference coordinate of the macro block is out of the macro block coordinate range corresponding to the changed image, judging that the macro block is not a P-SKIP macro block.
Step S22, when the macro block is a P-SKIP macro block, executing P-SKIP macro block coding compression processing on the macro block;
in step S23, when the macroblock is not a P-SKIP macroblock, a P macroblock coding compression process is performed on the macroblock.
When the macro block is judged not to be the P-SKIP macro block, the P macro block coding compression processing of the H.264 standard is executed on the macro block, wherein the P macro block coding compression processing comprises inter-frame prediction, motion estimation, DCT (Discrete Cosine Transform), entropy coding and the like. And when the macro block is judged to be the P-SKIP macro block, executing P-SKIP macro block coding compression processing of H.264 standard on the macro block, setting the motion vector MV of the macro block to be 0, skipping interframe prediction, motion estimation, entropy coding and the like, wherein only a few placeholders with bit as a unit exist in the output coding code stream after coding compression processing.
And after the corresponding coding compression processing is carried out on the macro blocks, the coding compression data after the coding compression processing is carried out on the macro blocks is transmitted to the server. And when the server receives the coded compressed data, sending the coded compressed data to a user terminal. And when the user terminal receives the coded compressed data, decoding the coded compressed data to obtain the current frame image.
In the scheme provided by this embodiment, after dividing the obtained current frame image into a plurality of macro blocks, it is first determined whether each macro block is a P-SKIP macro block, and when a macro block is a P-SKIP macro block, the P-SKIP macro block is encoded and compressed; when the macro block is not the P-SKIP macro block, the P macro block coding compression processing is executed to the macro block, namely, the corresponding coding compression processing is executed according to the type of the macro block.
Further, as shown in fig. 3, a third embodiment of the image transmission method based on SPICE protocol according to the present invention is proposed based on the first embodiment or the second embodiment, and in this embodiment, before the step S10, the method further includes the steps of:
step S40, when receiving a frame of complete image sent by a server based on SPICE protocol and an IDR instant decoding refresh frame coding instruction corresponding to the complete image, performing IDR frame coding on the complete image;
step S50, transmitting the encoded data to the server, so that the server sends the encoded data to a terminal, and the terminal decodes the encoded data to obtain the complete image when receiving the encoded data.
When the server is to send the first frame image to the h.264 encoder based on the SPICE protocol, the h.264 encoder is started first, and relevant parameters including image resolution, frame rate, code rate and the like are initialized. Since higher real-time is required, the encoding level is set to BaseLine. When the server sends a first frame image to an h.264 encoder based on a SPICE protocol or sends a frame reference image, the server sends a frame of complete image and an IDR (Instantaneous Decoding Refresh) frame encoding instruction corresponding to the complete image to the h.264 encoder. And when receiving a frame of complete image sent by the server based on the SPICE protocol and an IDR frame coding instruction corresponding to the complete image, the H.264 encoder executes IDR frame coding of the H.264 standard on the complete image and transmits coded data to the server. The encoded data includes data information such as SPS (sequence parameter set) and PPS (picture parameter set). And after the server receives the coded data, the coded data are sent to the user terminal. And when the user terminal receives the coded data, decoding the coded data to obtain the complete image.
In the scheme provided by this embodiment, when an encoder receives a frame of complete image sent by a server based on a SPICE protocol and an IDR frame coding instruction corresponding to the complete image, the IDR frame coding is performed on the complete image, and then coded data is transmitted to the server, the server sends the received coded data to a user terminal, and the terminal decodes the coded data to obtain the complete image. Therefore, in the embodiment, the complete image can be coded and then sent to the user terminal, and the user experience is further improved.
The invention further provides an image transmission device based on SPICE protocol, as shown in fig. 4, fig. 4 is a functional module schematic diagram of the first embodiment of the image transmission device based on SPICE protocol.
In this embodiment, the image transmission apparatus based on SPICE protocol includes:
a synthesis module 10, configured to, when receiving a change image of a current frame image and a previous frame image sent by a server based on a SPICE protocol, synthesize the change image and the previous frame image to obtain the current frame image;
when the server sends the current frame image to the user terminal based on the SPICE Protocol (Simple Protocol for Independent Computing Environment), the server only needs to send the change image of the change part of the current frame image and the previous frame image to the terminal, and does not need to send the complete current frame image to the terminal. In this embodiment, when the server sends the changed image of the changed part of the current frame image and the previous frame image to the user terminal based on the SPICE protocol, the server first sends the changed image to the encoder, and the encoder is preferably an h.264 encoder. When the h.264 encoder receives a changed image of a current frame image and a previous frame image sent by the server based on the SPICE protocol, the synthesis module 10 synthesizes the received changed image and the stored previous frame image to obtain the current frame image corresponding to the changed image. Further, the image transmission device based on the SPICE protocol further comprises a storage module, and the storage module stores the obtained current frame image to replace the stored previous frame image. When receiving a next frame image and a changed image of the current frame image sent by the server based on the SPICE protocol, the combining module 10 combines the changed image of the next frame image and the current frame image with the saved current frame image, so as to obtain the next frame image.
An encoding module 20, configured to divide the obtained current frame image into a plurality of macroblocks, and perform corresponding encoding and compression processing on each macroblock;
after the current frame image is obtained by the synthesis module 10, the encoding module 20 divides the current frame image into a plurality of macroblocks. For example, the current frame image is divided into several macroblocks of 16 × 16 pixels. Then, all the macroblocks are traversed, and corresponding encoding compression processing is performed on each macroblock in turn, for example, P macroblock processing of h.264 standard is performed on each macroblock.
A sending module 30, configured to transmit the encoded compressed data obtained by performing the encoding and compressing process on the multiple macro blocks to the server, so that the server sends the encoded compressed data to a terminal, and the terminal decodes the encoded compressed data when receiving the encoded compressed data to obtain the current frame image.
After the encoding module 20 completes the corresponding encoding and compressing process on all the macro blocks, the sending module 30 transmits the encoded and compressed data obtained by performing the corresponding encoding and compressing process on all the macro blocks to the server. And when the server receives the coded compressed data, sending the coded compressed data to a user terminal. And when the user terminal receives the coded compressed data, decoding the coded compressed data to obtain the current frame image.
The following describes the image transmission apparatus based on SPICE protocol in detail by taking a cloud desktop system as an example. When the cloud desktop server sends the change desktop image of the change part of the current frame desktop image and the previous frame desktop image to the user terminal based on the SPICE protocol, the cloud desktop server firstly sends the change desktop image to the H.264 encoder. When the h.264 encoder receives the change desktop image sent by the cloud desktop server based on the SPICE protocol, the synthesis module 10 synthesizes the received change desktop image with the stored previous frame desktop image to obtain a current frame desktop image corresponding to the change desktop image. Then, the encoding module 20 divides the current frame desktop image into a plurality of macro blocks, and sequentially performs corresponding encoding compression processing on each macro block. After the corresponding encoding and compressing processes are performed on all the macro blocks, the sending module 30 transmits the encoded and compressed data obtained by performing the corresponding encoding and compressing processes on all the macro blocks to the cloud desktop server. And when the cloud desktop server receives the coded compressed data, the coded compressed data is sent to a thin terminal of a user. And when the thin terminal of the user receives the coded compressed data, decoding the coded compressed data to obtain the current frame desktop image.
The present embodiment provides a scheme that, when the encoder receives the changed image of the current frame image and the previous frame image sent by the server based on the SPICE protocol, the synthesis module 10 synthesizes the changed image with the previous frame image to obtain the current frame image, the encoding module 20 then divides the obtained current frame image into a plurality of macroblocks, performs a corresponding encoding compression process on each macroblock, after the corresponding encoding and compressing processes are performed on the macro blocks, the sending module 30 transmits the encoded and compressed data after the encoding and compressing processes are performed on the macro blocks to the server, the server sends the received coded compressed data to a user terminal, the terminal can obtain the current frame image after decoding, therefore, the image does not need to be directly sent to the user terminal, and the bandwidth occupied by the method of sending the coded compressed data is reduced.
Further, as shown in fig. 5, a second embodiment of the image transmission apparatus based on SPICE protocol according to the present invention is proposed based on the first embodiment, in this embodiment, the encoding module 20 includes:
a determining unit 21, configured to divide the obtained current frame image into a plurality of macro blocks, and respectively determine whether each macro block is a P-SKIP macro block;
in the present embodiment, in order to simplify the encoding compression process of the current frame image, after dividing the obtained current frame image into a plurality of macroblocks, different encoding compression processes are performed for different types of macroblocks. Specifically, after dividing the obtained current frame image into a plurality of macroblocks, the determining unit 21 first determines whether each macroblock is a P-SKIP macroblock. Specifically, the determination unit 21 includes:
the first determining subunit is used for determining a macroblock coordinate range corresponding to the changed image;
a second determining subunit, configured to divide the obtained current frame image into a plurality of macroblocks, and determine a reference coordinate of each macroblock;
the first judgment subunit is used for sequentially judging whether the reference coordinate of each macro block is in the macro block coordinate range corresponding to the changed image;
the second judgment subunit is used for judging that the macro block is a P-SKIP macro block when the reference coordinate of the macro block is in the macro block coordinate range corresponding to the changed image; and when the reference coordinate of the macro block is not in the macro block coordinate range corresponding to the changed image, judging that the macro block is not a P-SKIP macro block.
In this embodiment, when a changed image of a current frame image and a previous frame image sent by the server based on the SPICE protocol is received, the first determining subunit determines a macroblock coordinate range corresponding to the changed image. Preferably, the variant image is a rectangular region, and the image information of the variant image includes parameter information such as vertex coordinates (x, y), a width value, and a height value of the rectangular region, and YUV image data of the variant image. Preferably, the vertex coordinates (x, y) of the rectangular region are the coordinates of the upper left corner of the rectangular region. The first determining subunit calculates an x-axis coordinate range [ x _ Start, x _ End ] of the changed image after being converted into a macroblock of 16 × 16 pixels based on the x-coordinate value and the width value of the vertex coordinates (x, y), and calculates a y-axis coordinate range [ y _ Start, y _ End ] of the changed image after being converted into a macroblock of 16 × 16 pixels based on the y-coordinate value and the height value of the vertex coordinates (x, y).
After the synthesis module 10 synthesizes the change image with the previous frame image to obtain a current frame image, the second determining subunit divides the obtained current frame image into a plurality of 16 × 16 pixel macro blocks, and determines the reference coordinate (x) of each macro blocki、yi). Specifically, in this embodiment, the second determining subunit is configured to:
dividing the obtained current frame image into a plurality of macro blocks, and determining a reference point of each macro block;
and acquiring the reference point coordinate of each macro block, and setting the reference point coordinate of each macro block as the reference coordinate of the macro block.
After dividing the obtained current frame image into a plurality of macroblocks of 16 × 16 pixels, the second determining subunit determines the reference point of each macroblock first. Preferably, the upper left corner of each macroblock is determined as the reference point for the macroblock. Then, the reference point coordinates of each macro block are obtained, that is, each macro block is obtainedCoordinates of the upper left corner of the block. The second determining subunit sets the reference point coordinates of each macro block as the reference coordinates (x) of the macro blocki、yi) That is, it is preferable to set the coordinate of the upper left corner of each macroblock as the reference coordinate (x) of the macroblocki、yi)。
Then, the first judging subunit judges the reference coordinate (x) of each macro blocki、yi) And the coordinate range [ x _ Start, x _ End ] of the macro block corresponding to the changed image]And [ y _ Start, y _ End]Determining the reference coordinate (x) of each macro blocki、yi) Whether the current macroblock coordinate range is in the macroblock coordinate range [ x _ Start, x _ End ] corresponding to the changed image]And [ y _ Start, y _ End]And (4) the following steps. If x _ Start is less than or equal to xiX _ End is less than or equal to x _ End and y _ Start is less than or equal to yiIf y _ End is less than or equal to y _ End, determining that the reference coordinate of the macro block is in the macro block coordinate range corresponding to the changed image; otherwise, determining that the reference coordinate of the macro block is out of the macro block coordinate range corresponding to the changed image. When the reference coordinate of the macro block is in the macro block coordinate range corresponding to the changed image, the second judging subunit judges that the macro block is a P-SKIP macro block; and when the reference coordinate of the macro block is out of the macro block coordinate range corresponding to the changed image, the second judging subunit judges that the macro block is not the P-SKIP macro block.
The encoding unit 22 is configured to, when a macroblock is a P-SKIP macroblock, perform P-SKIP macroblock encoding compression processing on the macroblock; and when the macro block is not the P-SKIP macro block, performing P macro block coding compression processing on the macro block.
When the second judging subunit judges that the macroblock is not a P-SKIP macroblock, the encoding unit 22 performs, on the macroblock, a P macroblock encoding compression process of the h.264 standard, including inter-frame prediction, motion estimation, DCT (Discrete Cosine Transform), entropy encoding, and the like. When the second judging subunit judges that the macro block is a P-SKIP macro block, the encoding unit 22 performs the P-SKIP macro block encoding compression processing of the h.264 standard on the macro block, sets the motion vector MV of the macro block to 0, and SKIPs inter-frame prediction, motion estimation, entropy encoding and the like, and only some placeholders with bit as a unit are in the encoded code stream output after the encoding compression processing.
In the scheme provided by this embodiment, after dividing the obtained current frame image into a plurality of macroblocks, the determining unit 21 first determines whether each macroblock is a P-SKIP macroblock, and when a macroblock is a P-SKIP macroblock, the encoding unit 22 performs a P-SKIP macroblock encoding compression process on the macroblock; when the macro block is not a P-SKIP macro block, the encoding unit 22 performs P macro block encoding and compressing processing on the macro block, that is, the encoding unit 22 performs corresponding encoding and compressing processing according to the type of the macro block, and since the encoding unit 22 adopts P-SKIP macro block encoding and compressing processing, the encoding and compressing processing process of the current frame image is simplified, thereby reducing the cpu occupancy rate during encoding.
Further, a third embodiment of the image transmission apparatus based on the SPICE protocol according to the present invention is proposed based on the first embodiment or the second embodiment, in this embodiment, the encoding module 20 is further configured to: when a frame of complete image sent by a server based on an SPICE protocol and an IDR frame coding instruction corresponding to the complete image are received, carrying out IDR frame coding on the complete image;
the sending module 30 is further configured to: and transmitting the encoded data to the server so that the server can send the encoded data to a terminal, and the terminal can decode the encoded data to obtain the complete image when receiving the encoded data.
When the server is to send the first frame image to the h.264 encoder based on the SPICE protocol, the h.264 encoder is started first, and relevant parameters including image resolution, frame rate, code rate and the like are initialized. Since higher real-time is required, the encoding level is set to BaseLine. When the server sends a first frame image to an H.264 encoder based on a SPICE protocol or sends a frame of reference image, the server sends a frame of complete image and an IDR frame encoding instruction corresponding to the complete image to the H.264 encoder. When the h.264 encoder receives a frame of complete image and an IDR frame encoding instruction corresponding to the complete image, which are sent by the server based on the SPICE protocol, the encoding module 20 executes IDR frame encoding of the h.264 standard on the complete image, and the sending module 30 transmits encoded data to the server. The encoded data includes data information such as SPS (sequence parameter set) and PPS (picture parameter set). And after the server receives the coded data, the coded data are sent to the user terminal. And when the user terminal receives the coded data, decoding the coded data to obtain the complete image.
In the scheme provided by this embodiment, when receiving a frame of complete image sent by a server based on a SPICE protocol and an IDR frame coding instruction corresponding to the complete image, a coding module 20 performs IDR frame coding on the complete image, a sending module 30 transmits coded data to the server, the server sends the received coded data to a user terminal, and the terminal decodes the coded data to obtain the complete image. Therefore, in the embodiment, the complete image can be coded and then sent to the user terminal, and the user experience is further improved.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (10)
1. An image transmission method based on SPICE independent computing environment simple protocol, characterized in that the image transmission method based on SPICE protocol comprises the following steps:
when receiving a change image of a current frame image and a previous frame image sent by a server based on an SPICE protocol, synthesizing the change image and the previous frame image to obtain the current frame image;
dividing the obtained current frame image into a plurality of macro blocks, and respectively executing corresponding coding compression processing on each macro block;
transmitting the encoded compressed data after the plurality of macro blocks perform encoding compression processing to the server, so that the server can send the encoded compressed data to a terminal, and the terminal can decode the encoded compressed data to obtain the current frame image when receiving the encoded compressed data;
wherein, after the step of synthesizing the changed image with the previous frame image to obtain the current frame image, the method further comprises:
replacing the stored previous frame image with the current frame image, and storing the obtained current frame image;
and when receiving a next frame image and a change image of the current frame image which are sent by a server based on the SPICE protocol, synthesizing the change images of the next frame image and the current frame image with the stored current frame image to obtain the next frame image.
2. The SPICE protocol-based image transmission method according to claim 1, wherein the step of dividing the obtained current frame image into a plurality of macroblocks and performing a corresponding encoding compression process on each macroblock respectively comprises:
dividing the obtained current frame image into a plurality of macro blocks, and respectively judging whether each macro block is a P-SKIP forward prediction macro block;
when the macro block is a P-SKIP macro block, performing P-SKIP macro block coding compression processing on the macro block;
and when the macro block is not the P-SKIP macro block, performing P macro block coding compression processing on the macro block.
3. The SPICE protocol-based image transmission method of claim 2, wherein the step of dividing the obtained current frame image into a plurality of macroblocks, and respectively determining whether each macroblock is a P-SKIP forward prediction macroblock comprises:
determining a macro block coordinate range corresponding to the changed image;
dividing the obtained current frame image into a plurality of macro blocks, and determining the reference coordinate of each macro block;
sequentially judging whether the reference coordinate of each macro block is in the macro block coordinate range corresponding to the changed image;
when the reference coordinate of the macro block is in the macro block coordinate range corresponding to the changed image, judging that the macro block is a P-SKIP macro block;
and when the reference coordinate of the macro block is not in the macro block coordinate range corresponding to the changed image, judging that the macro block is not a P-SKIP macro block.
4. The SPICE protocol-based image transmission method according to claim 3, wherein the obtained current frame image is divided into a plurality of macroblocks, and the step of determining the reference coordinates of each macroblock comprises:
dividing the obtained current frame image into a plurality of macro blocks, and determining a reference point of each macro block;
and acquiring the reference point coordinate of each macro block, and setting the reference point coordinate of each macro block as the reference coordinate of the macro block.
5. The SPICE protocol-based image transmission method according to any one of claims 1 to 4, wherein, before the step of synthesizing a current frame image and a previous frame image sent by a server based on SPICE protocol to obtain the current frame image, the SPICE protocol-based image transmission method further comprises the steps of:
when a frame of complete image sent by a server based on an SPICE protocol and an IDR instant decoding refresh frame coding instruction corresponding to the complete image are received, carrying out IDR frame coding on the complete image;
and transmitting the encoded data to the server so that the server can send the encoded data to a terminal, and the terminal can decode the encoded data to obtain the complete image when receiving the encoded data.
6. An image transmission device based on SPICE protocol, characterized in that the image transmission device based on SPICE protocol comprises:
the synthesis module is used for synthesizing a change image of a current frame image and a previous frame image when receiving the change image of the current frame image and the previous frame image which are sent by a server based on the SPICE protocol so as to obtain the current frame image;
the coding module is used for dividing the obtained current frame image into a plurality of macro blocks and respectively executing corresponding coding compression processing on each macro block;
a sending module, configured to transmit the encoded compressed data obtained by performing the encoding and compressing process on the multiple macro blocks to the server, so that the server sends the encoded compressed data to a terminal, and the terminal decodes the encoded compressed data when receiving the encoded compressed data to obtain the current frame image;
the synthesis module is also used for replacing the previous frame image stored by the current frame image and storing the obtained current frame image;
and when receiving a next frame image and a change image of the current frame image which are sent by a server based on the SPICE protocol, synthesizing the change images of the next frame image and the current frame image with the stored current frame image to obtain the next frame image.
7. The SPICE protocol-based image transmission apparatus of claim 6, wherein the encoding module comprises:
the judging unit is used for dividing the obtained current frame image into a plurality of macro blocks and respectively judging whether each macro block is a P-SKIP forward prediction macro block;
the coding unit is used for executing P-SKIP macro block coding compression processing on the macro block when the macro block is the P-SKIP macro block; and when the macro block is not the P-SKIP macro block, performing P macro block coding compression processing on the macro block.
8. The SPICE protocol-based image transmission apparatus according to claim 7, wherein the judgment unit comprises:
the first determining subunit is used for determining a macroblock coordinate range corresponding to the changed image;
a second determining subunit, configured to divide the obtained current frame image into a plurality of macroblocks, and determine a reference coordinate of each macroblock;
the first judgment subunit is used for sequentially judging whether the reference coordinate of each macro block is in the macro block coordinate range corresponding to the changed image;
the second judgment subunit is used for judging that the macro block is a P-SKIP macro block when the reference coordinate of the macro block is in the macro block coordinate range corresponding to the changed image; and when the reference coordinate of the macro block is not in the macro block coordinate range corresponding to the changed image, judging that the macro block is not a P-SKIP macro block.
9. The SPICE protocol-based image transmission apparatus of claim 8, wherein the second determining subunit is operable to:
dividing the obtained current frame image into a plurality of macro blocks, and determining a reference point of each macro block;
and acquiring the reference point coordinate of each macro block, and setting the reference point coordinate of each macro block as the reference coordinate of the macro block.
10. The SPICE protocol-based image transmission device according to any one of claims 6-9, wherein the encoding module is further configured to, upon receiving a frame of a complete image sent by a server based on SPICE protocol and an IDR frame encoding instruction corresponding to the complete image, perform IDR frame encoding on the complete image;
the sending module is further configured to transmit the encoded data to the server, so that the server sends the encoded data to a terminal, and the terminal decodes the encoded data to obtain the complete image when receiving the encoded data.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510580296.3A CN106534859B (en) | 2015-09-11 | 2015-09-11 | Image transmission method and device based on SPICE protocol |
| PCT/CN2016/084775 WO2017041551A1 (en) | 2015-09-11 | 2016-06-03 | Spice protocol-based image transmission method and apparatus |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510580296.3A CN106534859B (en) | 2015-09-11 | 2015-09-11 | Image transmission method and device based on SPICE protocol |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106534859A CN106534859A (en) | 2017-03-22 |
| CN106534859B true CN106534859B (en) | 2021-01-15 |
Family
ID=58239146
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510580296.3A Active CN106534859B (en) | 2015-09-11 | 2015-09-11 | Image transmission method and device based on SPICE protocol |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN106534859B (en) |
| WO (1) | WO2017041551A1 (en) |
Families Citing this family (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106982374A (en) * | 2017-03-28 | 2017-07-25 | 山东超越数控电子有限公司 | A kind of SPICE video transmission methods and device based on HEVC optimized algorithms |
| CN107682694B (en) * | 2017-07-07 | 2020-02-04 | 西安万像电子科技有限公司 | Image coding and decoding method, device and system |
| CN110545417B (en) * | 2018-05-28 | 2023-07-14 | 深信服科技股份有限公司 | Image encoding and decoding method and related device of desktop scene |
| CN110297680B (en) * | 2019-06-03 | 2023-02-17 | 北京星网锐捷网络技术有限公司 | Method and device for transmitting virtual desktop image |
| CN111131831B (en) * | 2019-12-20 | 2022-11-15 | 西安万像电子科技有限公司 | Data transmission method and device |
| CN111447453B (en) * | 2020-03-31 | 2024-05-17 | 西安万像电子科技有限公司 | Image processing method and device |
| CN113319850A (en) * | 2021-03-25 | 2021-08-31 | 弘丰塑胶制品（深圳）有限公司 | Manipulator, clamping suite matched with multiple manipulators and control system |
| CN115865909B (en) * | 2023-01-13 | 2023-06-02 | 中科方德软件有限公司 | SPICE protocol-based data transmission method and device and readable storage medium |
| CN115955573A (en) * | 2023-03-15 | 2023-04-11 | 广州思涵信息科技有限公司 | Real-time remote synchronous projection method for two-dimensional image |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101783949A (en) * | 2010-02-22 | 2010-07-21 | 深圳市融创天下科技发展有限公司 | Mode selection method of skip block |
| CN101977322A (en) * | 2010-11-10 | 2011-02-16 | 上海交通大学 | Screen coding system based on universal video coding standard |
| CN102244786A (en) * | 2011-07-12 | 2011-11-16 | 深圳市万兴软件有限公司 | Method and device for compressing and decompressing video data and mobile terminal |
| EP2574056A2 (en) * | 2010-05-19 | 2013-03-27 | Sk Telecom Co., LTD | Image encoding/decoding device and method |
| CN103227926A (en) * | 2013-05-27 | 2013-07-31 | 合肥优尔电子科技有限公司 | Video coding device and method applicable to low-bandwidth wireless communication channel |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP4792001B2 (en) * | 2007-04-27 | 2011-10-12 | 株式会社東芝 | Moving picture decoding apparatus, broadcast receiving apparatus, moving picture decoding method |
| US20100226441A1 (en) * | 2009-03-06 | 2010-09-09 | Microsoft Corporation | Frame Capture, Encoding, and Transmission Management |
| CN104253804B (en) * | 2013-06-28 | 2017-12-12 | 广州华多网络科技有限公司 | A kind of method and device for transmitting view data |
| US9335964B2 (en) * | 2013-07-18 | 2016-05-10 | Nvidia Corporation | Graphics server for remotely rendering a composite image and method of use thereof |
| CN104768009B (en) * | 2015-03-19 | 2018-01-05 | 中国科学院信息工程研究所 | A kind of image transfer method under desktop virtualization SPICE protocol |
-
2015
- 2015-09-11 CN CN201510580296.3A patent/CN106534859B/en active Active
-
2016
- 2016-06-03 WO PCT/CN2016/084775 patent/WO2017041551A1/en active Application Filing
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101783949A (en) * | 2010-02-22 | 2010-07-21 | 深圳市融创天下科技发展有限公司 | Mode selection method of skip block |
| EP2574056A2 (en) * | 2010-05-19 | 2013-03-27 | Sk Telecom Co., LTD | Image encoding/decoding device and method |
| CN101977322A (en) * | 2010-11-10 | 2011-02-16 | 上海交通大学 | Screen coding system based on universal video coding standard |
| CN102244786A (en) * | 2011-07-12 | 2011-11-16 | 深圳市万兴软件有限公司 | Method and device for compressing and decompressing video data and mobile terminal |
| CN103227926A (en) * | 2013-05-27 | 2013-07-31 | 合肥优尔电子科技有限公司 | Video coding device and method applicable to low-bandwidth wireless communication channel |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2017041551A1 (en) | 2017-03-16 |
| CN106534859A (en) | 2017-03-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106534859B (en) | Image transmission method and device based on SPICE protocol | |
| US8644381B2 (en) | Apparatus for reference picture resampling generation and method thereof and video decoding system using the same | |
| US11172220B2 (en) | Video encoding method, and storage medium thereof | |
| US20200374555A1 (en) | Video encoding method and apparatus, video decoding method and apparatus, computer device, and readable storage medium | |
| US9538239B2 (en) | Decoder and method for decoding encoded input data containing a plurality of blocks or packets | |
| US10051271B2 (en) | Coding structure | |
| KR101266667B1 (en) | Dual-mode compression of images and videos for reliable real-time transmission | |
| CN112533059B (en) | Image rendering method and device, electronic equipment and storage medium | |
| US20200374511A1 (en) | Video encoding method and apparatus, video decoding method and apparatus, computer device, and storage medium | |
| CN115866356A (en) | Video watermark adding method, device, equipment and storage medium | |
| WO2021004153A1 (en) | Image prediction method, coder, decoder, and storage medium | |
| US9967581B2 (en) | Video quality adaptation with frame rate conversion | |
| CN110692241A (en) | Diversified motion using multiple global motion models | |
| CN108401185B (en) | Reference frame selection method, video transcoding method, electronic device and storage medium | |
| WO2021057686A1 (en) | Video decoding method and apparatus, video encoding method and apparatus, storage medium and electronic device | |
| CN110572677B (en) | Video encoding and decoding method and device, storage medium and electronic device | |
| US20060209962A1 (en) | Video encoding method and video encoder for improving performance | |
| CN110798705A (en) | Video image transmission method, device and equipment | |
| CN115442617A (en) | Video processing method and device based on video coding | |
| CN111193926B (en) | Encoded data processing method, apparatus, computer device and storage medium | |
| CN111988621A (en) | Video processor training method and device, video processing device and video processing method | |
| CN113794887A (en) | Method and related equipment for video coding in game engine | |
| US20030223498A1 (en) | Video encoding method and video encoder for improving performance | |
| CN110636295B (en) | Video encoding and decoding method and device, storage medium and electronic device | |
| CN101860711A (en) | Method and device for displaying small video on digital television |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |