WO2014057809A1 - Système et méthode de transmission de vidéo de mouvement - Google Patents

Système et méthode de transmission de vidéo de mouvement Download PDF

Info

Publication number
WO2014057809A1
WO2014057809A1 PCT/JP2013/075965 JP2013075965W WO2014057809A1 WO 2014057809 A1 WO2014057809 A1 WO 2014057809A1 JP 2013075965 W JP2013075965 W JP 2013075965W WO 2014057809 A1 WO2014057809 A1 WO 2014057809A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
screen
block
drawing command
image data
Prior art date
Application number
PCT/JP2013/075965
Other languages
English (en)
Japanese (ja)
Inventor
一範 小澤
Original Assignee
日本電気株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 日本電気株式会社 filed Critical 日本電気株式会社
Publication of WO2014057809A1 publication Critical patent/WO2014057809A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/65—Transmission of management data between client and server
    • H04N21/654—Transmission by server directed to the client
    • H04N21/6543—Transmission by server directed to the client for forcing some client operations, e.g. recording
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
    • H04N21/845—Structuring of content, e.g. decomposing content into time segments
    • H04N21/8456—Structuring of content, e.g. decomposing content into time segments by decomposing the content in the time domain, e.g. in time segments
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H04N19/61—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding

Definitions

  • the present invention relates to transmission of moving images over a data communication network.
  • a terminal such as a mobile terminal accesses a server remotely via a network such as a mobile network and operates a virtual client on the server
  • the screen of the virtual client is transmitted from the server to the terminal. It relates to the transmission of moving images.
  • Some systems of this type have a configuration in which a screen is always compressed and encoded using an image codec regardless of the screen status.
  • the amount of data per unit time transferred from the server to the terminal is approximately proportional to the screen size of the terminal. For this reason, when the network bandwidth is narrow relative to the screen size of the terminal, data transfer is likely to be delayed, resulting in delays in screen update on the terminal side and reaction delays on the terminal side. May occur.
  • the time variation of the network bandwidth is likely to increase and the above-described delay is likely to occur. .
  • a method is also known in which a screen drawing command for generating a screen at a terminal is transferred from the server without transferring the screen itself from the server.
  • this method when the screen movement is small, the amount of data transferred from the server to the terminal can be reduced compared to the former case, but when the screen is complicated and the movement is large, the data is transferred from the server to the terminal. Since the number of commands is greatly increased, the processing load on the terminal is greatly increased compared to the former in order to draw the screen on the terminal. For this reason, the processing time in a terminal increases and responsiveness falls. Also, the amount of power used by the terminal increases due to an increase in processing load at the terminal.
  • Patent Document 1 is cited as a document describing a technique related to the present invention.
  • Patent Document 1 describes a server that instructs another client device to transmit drawing information for updating the screen of one client device.
  • Patent Document 2 is cited as a document in which other techniques related to the present invention are described.
  • Japanese Patent Application Laid-Open No. 2004-228561 describes a method in which a screen is divided into a plurality of blocks, compression-coded for each block, and transmitted to a client device with priority from a block having a small image size after compression coding.
  • the present invention has been made in view of such a situation, and the problem to be solved by the present invention is based on the communication status of the network and the nature of the image to be transmitted when transmitting an image constituting a moving image. Accordingly, it is possible to perform transmission in an appropriate transmission form.
  • the present invention has, as one aspect thereof, a transmission-side device that transmits a moving image for each screen via a data communication network and a reception-side device that receives the screen via the data communication network.
  • the transmission side device is any one of means for dividing the screen into a plurality of blocks, image data obtained by encoding the block for each of the plurality of blocks, and a drawing command for drawing the block Selecting means, means for encoding a block into image data, means for determining a drawing command for drawing the block from the block, and receiving the image data and drawing command via the data communication network Means for transmitting to the side device, the receiving side device from the transmitting side device to the data communication network.
  • Means for discriminating the image data and drawing command received via the network means for decoding the image data and outputting as an image signal constituting part of the screen, and part of the screen according to the drawing command
  • a moving image transmission system comprising means for outputting an image signal of an image drawn.
  • the present invention provides, as another aspect, means for dividing one screen of a moving image into a plurality of blocks, image data obtained by encoding the block for each of the plurality of blocks, and a drawing command for drawing the block
  • a data communication apparatus comprising means for transmitting to another data communication apparatus.
  • the present invention corresponds to one of a plurality of blocks obtained by dividing one screen of a moving image, and an image received from the other data communication apparatus via the data communication network.
  • Means for discriminating data or a drawing command, means for decoding the image data and outputting it as an image signal constituting a part of the screen, and an image signal of an image obtained by drawing a part of the screen according to the drawing command Provided is a data communication device comprising a means for outputting.
  • the present invention provides, as another aspect, means for dividing one screen of a moving image into a plurality of blocks, image data obtained by encoding the block for each of the plurality of blocks, and a drawing command for drawing the block
  • a program for causing a computer to function as means for transmitting to another data communication apparatus is provided.
  • the transmitting side device divides one screen of the moving image into a plurality of blocks, and the transmitting side device uses the image data of the block for one of the plurality of blocks. Selecting any one of encoding to and determination of a drawing command for drawing the block, executing one of the above in response to the selection at the transmitting side device, the image data, and A step of transmitting any of the drawing commands from the transmission side device to the reception side device via the data communication network, and the reception side device according to the decoding of the image data and the drawing command A moving image transmission method comprising the step of generating an image signal constituting a part of the screen by performing any one of the drawn images.
  • one block of a screen when transmitted, it is transmitted as either image data or a drawing command, so that the block can be transmitted in a transmission form according to the situation.
  • FIG. 1 is a block diagram of a moving picture transmission system 1 according to the first embodiment of the present invention.
  • FIG. 2 is a block diagram of a thin client system 100 according to the second embodiment of the present invention.
  • FIG. 3 is a block diagram of the server device 110 of the thin client system 100.
  • FIG. 4 is a block diagram of the determination unit 185 of the server device 110.
  • FIG. 5 is a block diagram of the image encoding unit 186 of the server apparatus 110.
  • FIG. 6 is a block diagram of the portable terminal 170 of the thin client system 100.
  • the moving image transmission system 1 is a system that transmits a moving image for each screen from the transmission side device 3 to the reception side device 4 via the network 2.
  • the network 2 is a data communication network, and is particularly suitable for a network whose communication status changes according to time, such as a mobile communication network such as a mobile phone network as at least a part thereof.
  • the transmission side device 3 is a device that transmits a time-changing screen to the reception side device 4.
  • a server of a thin client system as in the second embodiment to be described later, that is, a server that executes a virtual client and provides a screen of the virtual client to the thin client is a preferable example of the transmission side device 3.
  • the transmission side device 3 includes a dividing unit 5, a determination unit 6, an image encoder 7, a drawing command encoder 8, and a transmission unit 9.
  • the dividing unit 5 divides a screen input from a moving image photographing function (not shown) into a plurality of areas, for example, m ⁇ n (m and n are natural numbers) areas.
  • each divided area is referred to as a block.
  • the discriminating unit 6 selects one of the image encoder 7 and the drawing command encoder 8 according to the block itself input from the dividing unit 5 and the communication status of the network 2, and passes the block to the selected encoder.
  • the image encoder 7 encodes the transferred block into image data of a predetermined format.
  • the image encoder 7 may encode the block according to any of a plurality of types of compression encoding methods.
  • the drawing command encoder 8 obtains a drawing command group necessary for drawing the input block, and encodes the obtained drawing command group according to a predetermined compression encoding method.
  • the transmission unit 9 transmits the block encoded by either the image encoder 7 or the drawing command encoder 8 to the reception side device 4 via the network 2.
  • the receiving side device 4 is a device that receives and displays a screen from the transmitting side device 3.
  • the terminal of the thin client system of the second embodiment to be described later that is, a terminal that receives a virtual client screen from a server that executes the virtual client, particularly a mobile communication terminal such as a mobile phone terminal, although it is a suitable example, it may be applied to other devices that receive and display a moving image from a network, such as an information processing device, a workstation, a server, or a personal computer having a data communication function, and a moving image display function.
  • the receiving side device 4 includes a receiving unit 10, an image decoder 11, a drawing command decoder 12, a screen display unit 13, and a screen drawing unit 14.
  • the receiving unit 10 determines whether the encoded data is encoded as image data of a predetermined format or the drawing command group, and the former In this case, the image is output to the image decoder 11, and in the latter case, the image is output to the drawing command decoder 12.
  • the image decoder 11 decodes the image data according to the encoding method used by the image encoder 7 for encoding, and displays the image data in a corresponding area on the screen of the screen display unit 13.
  • the corresponding area is an area occupied by the block in the screen before the screen including the block corresponding to the image data is input to the dividing unit 5.
  • the drawing command decoder 12 decodes the encoded drawing command group and outputs it.
  • the screen drawing unit 14 draws an image in a corresponding area on the screen of the screen display unit 13 in accordance with the input drawing command group.
  • the screen display unit 13 is a display device, and specifically, there are a liquid crystal display device, a plasma display, a cathode ray tube, and the like, but the type thereof is not limited. In a configuration in which a block is transmitted only as image data regardless of the block or communication status as in the past, for example, even a block that can be drawn with a simple drawing command group is transmitted as image data.
  • the amount of data increases, and the display on the screen display unit 13 may be delayed depending on the communication status of the network.
  • the block and communication status in the configuration in which the block is transmitted only as a drawing command, the number of commands is greatly increased when converting a block of a screen that is particularly complex and intensely moving into a drawing command. Not only does the amount of data to be transmitted increase, but also the processing time required to execute all the increased drawing commands in the screen drawing unit 14 becomes longer.
  • the block is encoded using either the image encoder 7 or the drawing command encoder 8 according to the communication status of the network 2 and the block itself to be transmitted.
  • a thin client system 100 according to a second embodiment of the present invention will be described with reference to FIG.
  • a mobile network 150 is used as the network.
  • the structure in the case of using a SGSN / GGSNN apparatus as a packet transfer apparatus is shown.
  • the SGSN / GGSN device means a device in which an SGSN (Serving GPRS Support Node) device and a GGSN (Gateway GPRS Support Node) device are integrated.
  • SGSN Serving GPRS Support Node
  • GGSN Gateway GPRS Support Node
  • the thin client server device 110 is arranged on the cloud network 130 and the cloud network 130 and the mobile network 150 are connected.
  • the end user connects the mobile terminal 170 to the virtual client of the server apparatus 110 arranged in the cloud network 130 and operates the virtual client as if operating the real terminal.
  • the client software of the mobile terminal 170 transmits a packet storing the operation signal to the server device 110 via the base station 194, the RNC device 195, and the SGSN / GGSN device 190 on the mobile network 150.
  • the operation signal means a signal transmitted from the mobile terminal 170 to the server device 110 by an operation such as a key operation on the mobile terminal 170, a touch operation on the screen, character input, or scrolling.
  • the operation signal packet transmitted from the packet transmission unit in the client software installed in the mobile terminal 170 is transmitted to the cloud network 130 via the base station device 194, the RNC device 195, and the SGSN / GGSN device 190 on the mobile network 150.
  • the server device 110 receives the operation signal.
  • a well-known protocol can be used for sending the operation signal, but UDP / IP is used, but TCP / IP or the like can also be used.
  • FIG. 3 is a block diagram illustrating a configuration of the server device 110.
  • the operation signal packet receiving unit 182 receives a packet storing the operation signal from the mobile terminal 170 via the base station 194, the RNC device 195, and the SGSN / GGSN device 190.
  • the operation signal packet receiving unit 182 extracts an operation signal from the received operation signal UDP / IP packet and outputs the operation signal to the virtual client unit 211.
  • the virtual client unit 211 includes application software corresponding to various services, a control unit, a screen generation unit, a cache memory, and the like. In addition, the application software can be easily updated from the outside of the server device 110.
  • the virtual client unit 211 analyzes the operation signal input from the operation signal packet receiving unit 182, activates the application software specified by the operation signal, and displays a screen drawn by the application software and the OS as a predetermined screen. The image is generated at the resolution and output to the screen capture unit 180. Further, a drawing command executed when generating and drawing the screen is output to the drawing command collecting unit 181.
  • the drawing command collection unit 181 collects the drawing command group output from the virtual client unit 211 for each screen, temporarily saves it for each screen, and outputs it to the determination unit 185.
  • the screen capture unit 180 captures and outputs a screen at a predetermined screen resolution and frame rate.
  • the dividing unit 184 divides the captured screen into a plurality of blocks having a predetermined size.
  • the block size is, for example, 16 pixels ⁇ 16 lines, but other sizes, for example, 8 pixels ⁇ 8 lines, 4 pixels ⁇ 4 lines, and the like can be used. The smaller the block size, the better the discrimination accuracy in the discriminator, but the processing amount increases.
  • the dividing unit 184 outputs the divided blocks to the determining unit 185.
  • the configuration of the determination unit 185 is shown in FIG.
  • the determination unit 185 determines whether to transfer the screen based on at least one of the feature amount obtained from the image signal of the screen and the network delay amount, or to transfer the drawing command. Determine either.
  • a motion vector is used as the image feature amount used by the determination unit 185.
  • other well-known feature amounts for example, the sum of absolute values of difference values between frames, the motion compensation prediction residual error, and the like are used. An absolute value sum can also be used.
  • the motion vector calculation unit 201 performs, for example, D in the following Expression 1 for each block.
  • k Vector V to minimize k (Dx, dy) is calculated.
  • f n, k (Xi, Yj), f n-1, k (Xi, Yj) represents a pixel that enters the kth block of the nth frame and a pixel that enters the kth block of the (n-1) th frame.
  • the motion vector calculation unit 201 obtains the magnitude and direction of the motion vector for each block by the following formulas 2 and 3, and outputs these to the discrimination selection unit 202.
  • V k Indicates the magnitude of the motion vector in the kth block
  • ⁇ k Indicates the angle (direction) of the motion vector in the k-th block.
  • the discrimination / selection unit 202 applies V to a plurality of consecutive blocks.
  • the moving image area is shaped so as to be a rectangular area, and the area range includes the number of pixels in the horizontal direction and the number of lines in the vertical direction of the rectangular area, and the blocks included in the area. Number and block size.
  • the network bandwidth estimation unit 203 periodically inputs information necessary for network delay estimation from the first packet transmission / reception unit 176 of FIG.
  • Necessary information includes, for example, the data size transmitted from the server, the time transmitted from the server, the time received by the terminal, and the like. Based on these pieces of information, for example, the network band estimated value B is periodically calculated based on the following equations 4 and 5.
  • D (j) indicates the data size of the jth packet transmitted from the first packet transmitting / receiving unit 176 to the mobile terminal 170
  • R (j) is the jth packet received by the mobile terminal 170. Is the time of reception.
  • the band estimation value W calculated by Expression 4 is temporally smoothed using Expression 5 below.
  • B (n) represents a network band estimation value after smoothing at the nth time
  • ⁇ is a constant in a range of 0 ⁇ ⁇ 1.
  • the network bandwidth estimation unit 203 periodically calculates B (n) and outputs it to the discrimination selection unit 202.
  • the discrimination / selection unit 202 obtains the ratio ⁇ between the area of the moving image area and the area of the entire screen by the following Expression 6.
  • the discrimination / selection unit 202 inputs the network band estimation value ⁇ from the network band estimation unit 203. If ⁇ exceeds a predetermined threshold value Th1, it is determined that the screen is transferred, the discrimination flag F is set to 0, and the discrimination flag, the moving picture area information, and the movement area information are sent to the image encoding unit 186 in FIG. Alternatively, information on the static area is output.
  • Th1 a predetermined threshold value
  • Th2 it is determined that the command is transferred, the determination flag F is set to 1, and the determination flag F and the drawing command group are set as shown in FIG. Output to the drawing command encode 183.
  • the discrimination selection unit 202 outputs the discrimination flag F to the first packet transmission / reception unit 176.
  • the configuration of the image encoding unit 186 will be described with reference to FIG.
  • a first image encoder 227 inputs an image signal of a moving image area and a determination flag, and when the determination flag is 0, that is, in the case of screen transfer, a compression code using a predetermined moving image encoder is used for the image signal.
  • the compressed bit stream is output to the first packet transmitting / receiving unit 176 in FIG.
  • a predetermined moving image encoder H.264 is used.
  • H.264 is used, but other well-known moving image codecs, for example, MPEG-4 is also available for HEVC (High Efficiency Video Coding).
  • the region information of the moving image region is output to the first packet transmitting / receiving unit 176 in FIG.
  • the second image encoder unit 228 inputs a captured image from the image capture unit 180, inputs a determination flag from the determination unit 185, and when the determination flag is 0, other regions (for example, a moving region or a still region)
  • the range is input, and in the case of a still image, the image is compressed and encoded using a still image codec and output to the first packet transmitting / receiving unit 176 in FIG.
  • JPEG2000 is used as the still image codec, but other well-known codecs such as JPEG can also be used. Further, information on other areas is also output to the first packet transmission unit 176 in FIG.
  • a bit stream obtained by compression-encoding an image before movement with a still image codec and one representative motion vector are output to the first packet transmitting / receiving unit 176 in FIG. Further, information on other areas is also output to the first packet transmitting / receiving unit 176 in FIG.
  • the discrimination flag is 1, that is, when the drawing command is transferred
  • the drawing command encoding unit 183 inputs a drawing command group for each screen, performs lossless encoding of the drawing command group by a predetermined compression method, and results of compression encoding Is output to the first packet transmitting / receiving unit 176.
  • the predetermined compression method a known lossless encoding method such as a Zip compression method can be used.
  • the audio encoding unit 187 inputs an audio signal attached to the screen from the screen capture unit 180, performs compression encoding with the audio encoder, and performs the second packet transmission unit 177 in FIG. Output to.
  • MPEG-4 AAC is used as the audio encoder, but other known audio encoders can also be used.
  • the first packet transmission / reception unit 176 inputs flag information from the determination unit. When the flag information is 0, the first packet transmission / reception unit 176 inputs the compression encoded bitstream and region information from the image encoding unit 186.
  • a compression-encoded drawing command is input from the drawing command encoding unit 183, and flag information, area information, a compression-encoded bit stream, and a compression-encoded drawing command are stored in the payload portion of the packet.
  • a packet according to a predetermined protocol is constructed and sent to the SGSN / GGSN apparatus 190 in FIG.
  • a well-known protocol such as RTP / UDP / IP, UDP / IP, TCP / IP, or the like can be used as a predetermined protocol, but UDP / IP is used as an example.
  • the area information can be stored in an unused area of the RTP header part or the UDP header part.
  • the first packet transmission / reception unit 176 extracts information necessary for delay amount calculation from the response signal received from the mobile terminal 170 and outputs the information to the network bandwidth estimation unit 203 in FIG.
  • the second packet transmission unit 177 stores the compression-coded bit stream for the audio signal in the payload of the packet, constructs a packet according to a predetermined protocol, and transmits the packet to the SGSN / GGSN device 190.
  • a well-known protocol such as RTP / UDP / IP, UDP / IP, TCP / IP, or the like can be used as a predetermined protocol, but UDP / IP is used as an example.
  • the SGSN / GGSN device 190 tunnels the packet received from the server device 110 using the GTP-U protocol and transfers the packet to the RNC device 195.
  • the RNC device 195 transmits the packet to the mobile terminal 170 wirelessly through the base station device 194.
  • client software for sending to the mobile terminal 170 an operation signal when the user operates the terminal to the server, receiving packets from the server, and decoding and displaying the compressed encoded stream is displayed. 171 is mounted.
  • the configuration of the client software 171 is shown in FIG.
  • the first packet transmission / reception / selection unit 250 receives a packet, and extracts a discrimination flag, compression-encoded bitstream and area information, and compression-encoded drawing command information stored in the packet.
  • the size of the moving image area and the compression-encoded bit stream encoded by the first image encoder are selected and extracted as moving image area information, and the first image decoder 252 Output.
  • the other region information and the compression-encoded bit stream encoded by the other encoder in the region are output to the second image decoder 253.
  • the first image decoder 252 receives the moving image area information and the compression-encoded stream, decodes the compression-encoded stream, and outputs the decoded stream to the screen display unit 256.
  • the moving image area information is also output to the screen display unit 256.
  • the first image decoder for example, H.264 is used.
  • the H.264 decoder is used, but other well-known image decoders such as an MPEG-4 decoder can also be used. However, it goes without saying that the same type as the first image encoder 227 in the server is used.
  • the second image decoder 253 receives the other region information and the compression-encoded stream, decodes the compression-encoded stream for the other region, and outputs the decoded image to the screen display unit 256. Further, other area information is output to the screen display unit 256.
  • the screen display unit 256 receives the first area information and the image signal in the first area from the first image decoder 252, and receives the other area information and the image signal in the other area from the second image decoder 253.
  • the output image from the first image decoder 252 is displayed in the first area using the first area information, and the other area information is used to display the output image from the second image decoder 253 in the other area.
  • a display screen is generated by combining the image signals of the respective regions and output.
  • the first packet transmission / reception / selection unit 250 selects and extracts the compression-coded drawing command information and outputs it to the coder 259 with the drawing command.
  • the drawing command decoder 259 performs, for example, Zip compression decoding and outputs a drawing command group to the screen drawing unit 260 for each screen.
  • the screen drawing unit 260 inputs a drawing command group for each screen, draws and generates a screen, and outputs the generated screen to the screen display unit 256. Further, the first packet transmission / reception / selection unit 250 sends a response signal packet for the received packet to the network. Information on data size, reception time, and transmission time is described in the response signal packet.
  • the second packet receiving unit 251 receives the packet, extracts a compression-coded bit stream related to audio stored in the packet, and outputs the compressed bit stream to the audio decoder 255.
  • the audio decoder 255 receives and decodes the compression-coded stream, and outputs it in synchronization with the screen portion.
  • the operation signal generation unit 257 detects an operation input by the user to the mobile terminal 170, for example, screen touch, screen scroll, icon touch, character input, and the like, generates an operation signal for each, and sends the operation signal to the packet transmission unit 258. Output.
  • the packet transmission unit 258 receives an operation signal, stores it in a packet according to a predetermined protocol, and transmits it to the network.
  • TCP / IP, UDP / IP, etc. can be used as a predetermined protocol.
  • an image feature amount is obtained for a screen generated on the server side, and at least one of the feature amount and a network bandwidth estimation value or a network delay amount is obtained. Based on one, it is determined whether to transfer the screen or drawing command, and these are switched and transferred, so it is realized by balancing the amount of data, the amount of network delay, and the load on the terminal As a result, it is possible to prevent a significant increase in data acid, a significant delay in response time, and a significant increase in processing load at the terminal. While the present invention has been described with reference to the embodiment, the present invention is not limited to this. There may be three or more types of screen areas determined by the determination / selection unit 202.
  • a feature quantity other than a motion vector can be used as an image feature quantity used for region determination, or a plurality of types of feature quantities can be used in combination.
  • Other known methods can be used as the bandwidth estimation method in the network bandwidth estimation unit 203.
  • the delay amount can be estimated, and the delay selection unit 202 can use the delay amount estimation value to determine whether the screen transfer or the drawing command transfer.
  • Other known methods can be used as the determination method in the determination / selection unit 202.
  • the mobile network 150 can be a mobile LTE / EPC network, or can be a WiMax network or a WiFi network. Further, it can be a fixed network, an NGN network, or an Internet network.
  • the server device is arranged in the cloud network, but can be arranged in the Internet network.
  • the server device can be arranged in the company network.
  • the server device 110 can be arranged on the mobile network 150, a fixed network, or an NGN network.
  • a transmission-side device that transmits a moving image for each screen via a data communication network, and a reception-side device that receives the screen via the data communication network
  • the transmitting device is: Means for dividing the screen into a plurality of blocks; Means for selecting at least one of the plurality of blocks, either image data obtained by encoding the block, or a drawing command for drawing the block; Means for encoding the block into image data; Means for determining a drawing command for drawing the block from the block; and Means for transmitting the image data and the drawing command to the receiving side device via the data communication network;
  • the receiving side device Means for discriminating the image data and drawing command received from the transmission side device via the data communication network; Means for decoding the image data and outputting as an image signal constituting a part of the screen; and Means for outputting an image signal of an image obtained by drawing a part of the screen in accordance with the drawing command;
  • a video transmission system characterized by the above.
  • the transmitting device operates as the server of a thin client system including a server that executes a virtual client and provides a screen of a virtual client to a client terminal, and a client terminal that displays a screen provided from the server.
  • the receiving device operates as the client terminal;
  • the data communication network includes a mobile communication network
  • the moving image transmission system according to supplementary note 1, wherein: (Appendix 3) The system according to any one of Supplementary Note 1 and Supplementary Note 2, wherein the means for selecting one of the image data and the drawing command performs selection according to one or more image feature amounts obtained from the block. .
  • the transmission side device includes a plurality of image encoders having different encoding methods as means for encoding the image data
  • the receiving-side apparatus includes a plurality of image decoders corresponding to the encoding methods of the plurality of image encoders as means for decoding the image data.
  • the transmission-side apparatus includes, as means for encoding the image data, a first image encoder for encoding a moving image area and a second image encoder for encoding a non-moving area
  • the receiving-side apparatus includes, as means for decoding the image data, a first image encoder for decoding a moving image area and a second image encoder for decoding a non-moving area.
  • Appendix 7 A way to divide one screen of a video into multiple blocks, Means for selecting at least one of the plurality of blocks, either image data obtained by encoding the block, or a drawing command for drawing the block; Means for encoding the block into image data; Means for determining a drawing command for drawing the block from the block; and A data communication apparatus comprising means for transmitting the image data and the drawing command to another data communication apparatus via a data communication network.
  • the computer operates as the server of a thin client system including a server that executes a virtual client and provides a screen of the virtual client to the client terminal, and a client terminal that displays the screen provided from the server,
  • the other data communication device operates as the client terminal
  • the data communication network includes a mobile communication network
  • the data communication device according to appendix 7, wherein (Appendix 9) The data according to any one of appendix 7 and appendix 8, wherein the means for selecting one of the image data and the drawing command performs selection according to one or more image feature amounts obtained from the block. Communication device. (Appendix 10) 10.
  • the data communication apparatus according to any one of appendix 7 to appendix 9, wherein the means for selecting one of the image data and the drawing command performs selection according to a communication status of the data communication network.
  • Appendix 11 11.
  • Appendix 12 Additional means 11 for encoding the image data functions as a first image encoder for encoding a moving image area and a second image encoder for encoding a non-moving area.
  • (Appendix 13) Means corresponding to one of a plurality of blocks formed by dividing one screen of a moving image, and means for discriminating image data or a drawing command received from the other data communication device via a data communication network; Means for decoding the image data and outputting as an image signal constituting a part of the screen; and Means for outputting an image signal of an image obtained by drawing a part of the screen in accordance with the drawing command; A data communication device.
  • the other data communication device is a server of a thin client system including a server that executes a virtual client and provides a screen of the virtual client to the client terminal, and a client terminal that displays the screen provided from the server.
  • the data communication device operates as the client terminal;
  • the data communication network includes a mobile communication network 14.
  • (Appendix 16) (Supplementary note 15)
  • the means for decoding the image data includes a first image encoder for decoding a moving image area and a second image encoder for decoding a non-moving area.
  • Appendix 17 A way to divide one screen of a video into multiple blocks, Means for selecting at least one of the plurality of blocks, either image data obtained by encoding the block, or a drawing command for drawing the block; Means for encoding the block into image data; Means for determining a drawing command for drawing the block from the block; and A program for causing a computer to function as means for transmitting the image data and the drawing command to another data communication apparatus via a data communication network.
  • the computer operates as the server of a thin client system including a server that executes a virtual client and provides a screen of the virtual client to the client terminal, and a client terminal that displays the screen provided from the server,
  • the other data communication device operates as the client terminal,
  • the data communication network includes a mobile communication network
  • Appendix 20 The program according to any one of appendix 17 to appendix 19, wherein the means for selecting one of the image data and the drawing command performs selection according to a communication status of the data communication network.
  • Appendix 21 The program according to any one of appendix 17 to appendix 20, wherein the means for encoding the image data functions as a plurality of image encoders having different encoding methods.
  • Appendix 22 The appendix 21 is characterized in that the means for encoding the image data functions as a first image encoder for encoding a moving image area and a second image encoder for encoding a non-moving area. The listed program.
  • (Appendix 23) Means corresponding to one of a plurality of blocks formed by dividing one screen of a moving image, and means for discriminating image data or a drawing command received from the other data communication device via a data communication network; Means for decoding the image data and outputting as an image signal constituting a part of the screen; and A program for causing a computer to function as means for outputting an image signal of an image obtained by drawing a part of the screen in accordance with the drawing command.
  • the other data communication device is a server of a thin client system including a server that executes a virtual client and provides a screen of the virtual client to the client terminal, and a client terminal that displays the screen provided from the server.
  • the data communication device operates as the client terminal;
  • the data communication network includes a mobile communication network
  • (Appendix 25) 25 The program according to any one of Supplementary Note 23 and Supplementary Note 24, wherein the means for decoding the image data functions as a plurality of image decoders corresponding to a plurality of image encoder encoding methods.
  • the listed program wherein the means for decoding the image data functions as a first image encoder for decoding a moving image area and a second image encoder for decoding a non-moving area
  • (Appendix 27) Dividing one screen of the video into a plurality of blocks at the transmission side device; In the transmitting device, for one of the plurality of blocks, selecting one of encoding of the block into image data and determination of a drawing command for drawing the block; Performing any one of the above in response to the selection at the transmitting device; Transmitting either the image data or the drawing command from the transmitting device to the receiving device via the data communication network; and A step of generating an image signal constituting a part of the screen by performing any one of decoding of the image data and drawing of an image according to the drawing command in the receiving side device.
  • a moving picture transmission method comprising: (Appendix 28)
  • the transmitting device operates as the server of a thin client system including a server that executes a virtual client and provides a screen of a virtual client to a client terminal, and a client terminal that displays a screen provided from the server.
  • the receiving device operates as the client terminal;
  • the data communication network includes a mobile communication network
  • the method according to appendix 27, wherein: (Appendix 29) The step of selecting either one of the encoding to the image data and the determination of the drawing command is performed according to one or more image feature amounts obtained from the block.
  • the method according to any one. (Appendix 30) 30.
  • the method of. (Appendix 31)
  • the image data is encoded using one image encoder selected from a plurality of image encoders having different encoding methods,
  • the decoding of the image data is performed using an image decoder corresponding to the encoding method of the selected one image encoder. 31.

Abstract

Selon l'invention, un dispositif côté transmission segmente un écran en une pluralité de blocs, et transmet soit comme des données d'image qui encodent le bloc soit comme une commande de rendu qui effectue le rendu du bloc. Avec un dispositif côté réception, un signal d'image d'un bloc est produit en effectuant le rendu en fonction soit du décodage des données d'image soit de la commande de rendu.
PCT/JP2013/075965 2012-10-12 2013-09-18 Système et méthode de transmission de vidéo de mouvement WO2014057809A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2012226654 2012-10-12
JP2012-226654 2012-10-12

Publications (1)

Publication Number Publication Date
WO2014057809A1 true WO2014057809A1 (fr) 2014-04-17

Family

ID=50477280

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2013/075965 WO2014057809A1 (fr) 2012-10-12 2013-09-18 Système et méthode de transmission de vidéo de mouvement

Country Status (1)

Country Link
WO (1) WO2014057809A1 (fr)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2004312584A (ja) * 2003-04-10 2004-11-04 Matsushita Electric Ind Co Ltd 画像処理方法および画像表示システム
JP2010232891A (ja) * 2009-03-26 2010-10-14 Nec Personal Products Co Ltd サーバ、リモート操作システム、伝送方式制御方法、プログラム及び記録媒体

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2004312584A (ja) * 2003-04-10 2004-11-04 Matsushita Electric Ind Co Ltd 画像処理方法および画像表示システム
JP2010232891A (ja) * 2009-03-26 2010-10-14 Nec Personal Products Co Ltd サーバ、リモート操作システム、伝送方式制御方法、プログラム及び記録媒体

Similar Documents

Publication Publication Date Title
KR101443070B1 (ko) 짧은 대기 시간의 전송 프로토콜을 위한 방법 및 시스템
US10756997B2 (en) Bandwidth adjustment for real-time video transmission
JP5983761B2 (ja) サーバ装置、端末、シンクライアントシステム、画面送信方法およびプログラム
US20170094301A1 (en) Initial Bandwidth Estimation For Real-time Video Transmission
WO2011142311A1 (fr) Système de communication mobile à distance, dispositif de serveur et procédé de commande de système de communication mobile à distance
US10516892B2 (en) Initial bandwidth estimation for real-time video transmission
CN104509048B (zh) 服务器装置、通信系统及通信方法
US20170094296A1 (en) Bandwidth Adjustment For Real-time Video Transmission
WO2013030166A2 (fr) Procédé permettant de transmettre à un dispositif client des signaux vidéo à partir d'une application s'exécutant sur un serveur par l'intermédiaire d'un réseau ip
JP5854247B2 (ja) 画像情報伝送方法及びパケット通信システム
CN116134474A (zh) 在支持混合现实/增强现实的通信系统中相对于三维媒体数据使用延迟补偿姿势预测来执行渲染的方法和装置
JPWO2012029648A1 (ja) サーバ装置、映像品質計測システム、映像品質計測方法およびプログラム
WO2014057809A1 (fr) Système et méthode de transmission de vidéo de mouvement
KR101632012B1 (ko) 통신 시스템, 서버 장치, 서버 장치의 제어 방법 및 프로그램을 저장하는 컴퓨터 판독 가능한 기록 매체
WO2014042137A1 (fr) Système et procédé de communication, et dispositif de serveur et terminal
KR101251879B1 (ko) 멀티미디어 클라우드 시스템에서 화면변화에 따른 광고영상 출력 장치 및 방법

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 13844726

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 13844726

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: JP