WO2018171084A1 - 一种虚拟现实图像发送方法及装置 - Google Patents
一种虚拟现实图像发送方法及装置 Download PDFInfo
- Publication number
- WO2018171084A1 WO2018171084A1 PCT/CN2017/092285 CN2017092285W WO2018171084A1 WO 2018171084 A1 WO2018171084 A1 WO 2018171084A1 CN 2017092285 W CN2017092285 W CN 2017092285W WO 2018171084 A1 WO2018171084 A1 WO 2018171084A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- virtual reality
- motion information
- modulation
- channel quality
- compression ratio
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T9/00—Image coding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
- H04N13/194—Transmission of image signals
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
- A63F13/20—Input arrangements for video game devices
- A63F13/21—Input arrangements for video game devices characterised by their sensors, purposes or types
- A63F13/211—Input arrangements for video game devices characterised by their sensors, purposes or types using inertial sensors, e.g. accelerometers or gyroscopes
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
- A63F13/20—Input arrangements for video game devices
- A63F13/21—Input arrangements for video game devices characterised by their sensors, purposes or types
- A63F13/212—Input arrangements for video game devices characterised by their sensors, purposes or types using sensors worn by the player, e.g. for measuring heart beat or leg activity
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/0093—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
- G06F3/0346—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
- G06F3/038—Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/006—Mixed reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/20—Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/20—Scenes; Scene-specific elements in augmented reality scenes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L1/00—Arrangements for detecting or preventing errors in the information received
- H04L1/0001—Systems modifying transmission characteristics according to link quality, e.g. power backoff
- H04L1/0002—Systems modifying transmission characteristics according to link quality, e.g. power backoff by adapting the transmission rate
- H04L1/0003—Systems modifying transmission characteristics according to link quality, e.g. power backoff by adapting the transmission rate by switching between different modulation schemes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L1/00—Arrangements for detecting or preventing errors in the information received
- H04L1/0001—Systems modifying transmission characteristics according to link quality, e.g. power backoff
- H04L1/0023—Systems modifying transmission characteristics according to link quality, e.g. power backoff characterised by the signalling
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
- H04N13/106—Processing image signals
- H04N13/161—Encoding, multiplexing or demultiplexing different image signal components
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/115—Selection of the code volume for a coding unit prior to coding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/136—Incoming video signal characteristics or properties
- H04N19/137—Motion inside a coding unit, e.g. average field, frame or block difference
- H04N19/139—Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/164—Feedback from the receiver or from the transmission channel
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/177—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a group of pictures [GOP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/20—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video object coding
- H04N19/27—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video object coding involving both synthetic and natural picture components, e.g. synthetic natural hybrid coding [SNHC]
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F2300/00—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
- A63F2300/80—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game specially adapted for executing a specific type of game
- A63F2300/8082—Virtual reality
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0101—Head-up displays characterised by optical features
- G02B2027/014—Head-up displays characterised by optical features comprising information/image processing systems
Definitions
- the present application relates to the field of virtual reality technologies, and in particular, to a method and an apparatus for transmitting a virtual reality image.
- VR technology is an emerging multimedia technology in recent years.
- VR technology is a computer simulation technology that can create and experience virtual worlds.
- the user typically wears a virtual reality device to experience a VR scene that enables the user to obtain an interactive immersive experience through an integrated graphics system, optical system, and gesture tracking system.
- virtual reality devices have limited processing power, and virtual reality devices may not be affordable if some large-scale games or video applications are to be implemented.
- the industry has proposed to set up a processor for performing a large number of calculations on other devices such as a desktop computer, a mobile phone, or a game console.
- These devices are called virtual reality hosts, and virtual reality hosts generate virtual reality images, and The virtual reality image is wirelessly transmitted to the virtual reality device, and the virtual reality device is only used to receive the virtual reality image sent by the virtual reality host and display the virtual reality image, thereby helping to reduce the weight and volume of the virtual reality device.
- the current VR system is generally composed of a virtual reality device and a virtual reality host, and the virtual reality host is used to create a virtual world, and the user experiences the virtual world by wearing a virtual reality device.
- the number of virtual reality images transmitted in the VR system is very large, that is, the amount of data is very large, and the transmission rate of the wireless channel between the virtual reality host and the virtual reality device is usually required to be 5 Gbps or more.
- Existing commercial wireless channels have been unable to meet this demand.
- the industry proposes a method of compressing virtual reality images before transmission to reduce the amount of data transmitted to meet bandwidth requirements.
- the image compression ratio used for compressing the virtual reality image is determined according to the channel quality of the wireless channel. For example, if the current channel quality is poor, in order to reduce the amount of data transmitted, one can be determined. Larger image compression ratio. However, the image compression ratio determined in this way is not accurate enough. For example, since the switching speed of the virtual reality image displayed by the virtual reality device is fast and slow, for a slow motion or even a static virtual reality image, the user tends to be more critical of the image quality, so in this case, in order to ensure image quality, Virtual reality images cannot be compressed with a large image compression ratio. However, if the channel quality is always poor, the virtual reality host will always compress the virtual reality image with a larger image compression ratio, resulting in poor image quality. It can be seen that the current method of determining the image compression ratio is less accurate.
- the present application provides a virtual reality image transmitting method and apparatus for improving the accuracy of an image compression ratio determined in a VR system.
- the present application provides a virtual reality image transmitting method.
- the method can be implemented by a virtual reality host in a VR system.
- the method includes: receiving, by the virtual reality host, first motion information sent by the virtual reality device, where the first motion information is used to indicate a motion state of the virtual reality device.
- the virtual reality host determines an image compression ratio based on the first motion information, and compresses the virtual reality image to be transmitted according to the first image compression.
- the virtual reality host sends the compressed virtual reality image to the virtual reality device.
- the virtual reality host determines an image compression ratio according to the motion information of the virtual reality device. For example, when the virtual reality device moves at a slow rate, that is, the virtual reality image displayed by the virtual reality device has a slower switching rate, in order to ensure virtual The quality of the actual image determines a smaller image compression ratio. When the virtual reality device moves at a faster rate, that is, the virtual reality image displayed by the virtual reality device has a faster switching rate, and a larger image compression ratio is determined in order to ensure the continuity of the virtual reality image. In this way, the accuracy of the determined image compression ratio is improved, and the user's needs can be better met.
- the method before the virtual reality host determines the first image compression ratio according to the first motion information, the method further includes: the virtual reality host detecting the current channel quality of the wireless channel, and according to the channel quality and image of the wireless channel Determining, by the mapping relationship between the compression ratios, an image compression ratio corresponding to the detected channel quality; the virtual reality host determining the first image compression ratio according to the first motion information, including: the virtual reality host determining, according to the first motion information
- the image compression ratio corresponding to the detected channel quality is a first image compression ratio, or the virtual reality host adjusts an image compression ratio corresponding to the detected channel quality according to the first motion information, and compresses the adjusted image compression ratio as the first image compression ratio.
- the virtual reality host initially determines an image compression ratio according to the current channel quality of the wireless channel, and then the virtual reality host determines, according to the first motion information, whether the initially determined image compression ratio can be used as the first image compression ratio.
- the virtual reality image is compressed. For example, when the virtual reality device is in a static state, the virtual reality host may directly determine that the initially determined image compression ratio is a first image compression ratio, and when the virtual reality device is in a motion state, the virtual reality host may adjust the initially determined according to the first motion information.
- the image compression ratio is used as the first image compression ratio. In this way, it helps to improve the accuracy of the determined image compression ratio.
- the virtual reality host determines, according to the first motion information, an image compression ratio corresponding to the detected channel quality as the first image compression ratio, including: the virtual reality host according to the first motion information and the history of the virtual reality device The motion information determines that the image compression ratio corresponding to the detected channel quality is the first image compression ratio if it is determined that the virtual reality device is in a stationary state for the first time period in the future.
- the channel quality of the wireless channel between the virtual reality device and the virtual reality host has a certain correlation with the motion information of the virtual reality device. For example, when the virtual reality device moves at a faster rate, the channel quality is often unstable, and the virtual reality device motion rate is When it is slow, the channel quality is relatively stable. Therefore, in the present application, the virtual reality host can predict the motion state of the virtual reality device in the future first time period, and if it is determined that the virtual reality device is in a static state for the first time in the future, that is, the virtual reality device and the virtual reality The channel quality of the wireless channel between the hosts is relatively stable, so the virtual reality host can directly determine the image compression ratio corresponding to the channel quality as the first image compression ratio. In this way, the accuracy of the determined image compression ratio is improved.
- the virtual reality host adjusts an image compression ratio corresponding to the detected channel quality according to the first motion information, including: the virtual reality host according to the first motion information and the historical motion information of the virtual reality device, if determined in the future The virtual reality device is in a motion state within the first time period, and the image compression ratio corresponding to the detected channel quality is adjusted according to the first motion information.
- the virtual reality host can predict the motion state of the virtual reality device in the future for the first time period. If it is determined that the virtual reality device is in the motion state for the first time in the future, then the virtual reality host directly The image compression ratio corresponding to the detected channel quality is inaccurate as the first image compression ratio, so the virtual reality host can adjust the image compression ratio corresponding to the detected channel quality according to the first motion information. In this way, the accuracy of the determined image compression ratio is improved.
- the virtual reality host adjusts an image compression ratio corresponding to the detected channel quality according to the first motion information, including: if the first motion information is used to indicate that the moving rate of the virtual reality device is less than or equal to the first rate, The virtual reality host reduces an image compression ratio corresponding to the detected channel quality; or if the first motion information is used to indicate that the moving rate of the virtual reality device is greater than the second rate, the virtual reality host increases image compression corresponding to the detected channel quality Ratio; wherein the first rate is less than or equal to the second rate.
- the virtual reality host may adjust an image compression ratio corresponding to the detected channel quality according to the moving rate of the virtual reality device. For example, when the moving rate of the virtual reality device is large, the image compression ratio corresponding to the detected channel quality is increased to ensure the continuity of the virtual reality image. When the moving rate of the virtual reality device is small, the image compression ratio corresponding to the detected channel quality is reduced to ensure the quality of the virtual reality image. In this way, the accuracy of the determined image compression ratio is improved.
- the method further includes: determining, by the virtual reality host, the channel quality according to the mapping relationship between the channel quality of the wireless channel and the modulation and coding mode. Corresponding modulation coding mode; the virtual reality host determines, according to the first motion information, a modulation coding mode corresponding to the detected channel quality as a first modulation coding mode, or a channel quality adjusted and detected by the virtual reality host according to the first motion information.
- the method further includes: a virtual reality host
- the compressed virtual reality image is modulated and encoded according to the first modulation and coding mode; the virtual reality host sends the compressed virtual reality image to the virtual reality device, and the virtual reality host sends the modulated and encoded compressed virtual reality image to the virtual reality host.
- the virtual reality host initially determines a modulation and coding mode according to the channel quality of the wireless channel, and then the virtual reality host determines, according to the first motion information, whether the initially determined modulation and coding mode can be compressed as the first modulation and coding mode.
- the virtual reality image is modulated and encoded. For example, when the virtual reality device is in a static state, the virtual reality host can directly determine that the initially determined modulation and coding mode is the first modulation and coding mode, and when the virtual reality device is in the motion state, the virtual reality host can adjust the initially determined according to the first motion information.
- the modulation coding method uses the adjusted modulation and coding method as the first modulation and coding method. In this way, the accuracy of the determined modulation coding method is improved.
- the virtual reality host determines, according to the first motion information, a modulation coding mode corresponding to the detected channel quality as the first modulation and coding mode, including: the virtual motion host according to the first motion information and the historical motion of the virtual reality device The information, if it is determined that the virtual reality device is in a static state for the first time period in the future, determines that the modulation and coding mode corresponding to the detected channel quality is the first modulation and coding mode.
- the channel quality of the wireless channel between the virtual reality device and the virtual reality host has a certain correlation with the motion information of the virtual reality device. For example, when the virtual reality device moves at a faster rate, the channel quality is often unstable, and the virtual reality device motion rate is When it is slow, the channel quality is relatively stable. Therefore, in the present application, the virtual reality host can predict the motion state of the virtual reality device in the future first time period, and if the virtual reality device is determined to be in a static state for the first time in the future, that is, the virtual reality device and the virtual reality Wireless mail between hosts The channel quality of the channel is relatively stable, so the virtual reality host can directly determine the modulation and coding mode corresponding to the detected channel quality as the first modulation and coding mode. In this way, the accuracy of the determined modulation coding method is improved.
- the virtual reality host adjusts a modulation and coding manner corresponding to the detected channel quality according to the first motion information, including: the virtual reality host according to the first motion information and the historical motion information of the virtual reality device, if determined in the future The virtual reality device is in a motion state within the first time period, and then the modulation and coding mode corresponding to the detected channel quality is adjusted according to the first motion information.
- the virtual reality host can predict the motion state of the virtual reality device in the future for the first time period. If it is determined that the virtual reality device is in the motion state for the first time in the future, then the virtual reality host directly The modulation and coding scheme corresponding to the detected channel quality is inaccurate as the first modulation and coding scheme, so the virtual reality host adjusts the modulation and coding scheme corresponding to the detected channel quality according to the first motion information. In this way, the accuracy of the determined modulation coding method is improved.
- the virtual reality host adjusts the modulation and coding mode corresponding to the detected channel quality according to the first motion information, including: if the first motion information is used to indicate that the moving rate of the virtual reality device is less than or equal to the first rate, The virtual reality host adjusts the modulation and coding mode corresponding to the detected channel quality to the second modulation and coding mode, and the modulation and coding rate of the second modulation and coding mode is greater than the modulation and coding rate of the modulation and coding mode corresponding to the detected channel quality; The motion information is used to indicate that the moving rate of the virtual reality device is greater than the second rate, and the virtual reality host adjusts the modulation and coding mode corresponding to the detected channel quality to the third modulation and coding mode, and the modulation and coding rate of the third modulation and coding mode is less than and detection.
- the channel coding quality corresponds to a modulation coding rate of the modulation coding mode; wherein the first rate is smaller than the second rate.
- the virtual reality host may adjust the adjustment coding mode corresponding to the detected channel quality according to the moving rate of the virtual reality device. For example, when the moving rate of the virtual reality device is large, an adjustment coding mode with a small modulation coding rate is determined to ensure the continuity of the virtual reality image. When the moving rate of the virtual reality device is small, an adjustment coding mode with a large modulation coding rate is determined to ensure the quality of the virtual reality image. In this way, the accuracy of the determined modulation coding method is improved.
- the first motion information is used to indicate current location information of the virtual reality device, and the virtual reality host detects the current channel quality of the wireless channel, including: the virtual reality host according to the location information in the first angle range from the virtual reality Determining, by the device, the first wireless transmission direction in at least one wireless transmission direction distributed within a range of 360 degrees; the first angle is greater than 0 degrees and less than 360 degrees; the virtual reality host detects a current channel of the wireless channel in the first wireless transmission direction quality.
- the virtual reality host may determine the first wireless transmission direction from the at least one wireless transmission direction according to the location information of the virtual reality device, and the wireless channel in the wireless transmission direction is a virtual reality.
- the communication channel between the host and the virtual reality device In this way, the virtual reality host does not need to find the wireless transmission direction within a range of 360 degrees, thereby improving the transmission efficiency of the virtual reality image.
- the first motion information includes one or more of speed information, acceleration information, angular velocity information, and angular acceleration information of the virtual reality device.
- the first motion information may also be other information, and the above several are only examples, and the present application does not specifically limit this.
- the present application also provides a virtual reality image transmitting apparatus.
- the virtual reality image transmitting apparatus has a function of realizing a virtual reality host in the above method design.
- These functions can be implemented in hardware or in software by executing the corresponding software.
- the hardware or software includes one or more of the functions corresponding to the above functions unit.
- the specific structure of the virtual reality image transmitting apparatus may include a receiving unit, a processing unit, and a transmitting unit. These units may perform the respective functions of the methods provided by any of the possible aspects of the first aspect or the first aspect described above.
- the present application further provides a virtual reality image transmitting apparatus.
- the virtual reality image transmitting apparatus has a function of realizing a virtual reality host in the above method design. These features can be implemented in hardware.
- the virtual reality image transmitting apparatus includes a memory for storing computer executable program code, a transceiver, and a processor coupled to the memory and the transceiver.
- the program code stored in the memory includes instructions that, when executed by the processor, cause the virtual reality image transmitting device to perform the virtual reality host executed by any one of the first aspect or the first aspect of the first aspect of the design Methods.
- the present application further provides a computer readable storage medium, wherein the computer readable storage medium stores instructions, when executed on a computer, causing the computer to execute the virtual reality image transmitting method according to the first aspect described above .
- the present application further provides a computer program product comprising instructions for causing a computer to execute the virtual reality image transmitting method of the first aspect described above when it is run on a computer.
- FIG. 1 is an architectural diagram of a VR system provided by the present application
- FIG. 2 is a flowchart of a method for transmitting a virtual reality image provided by the present application
- FIG. 3 is a schematic diagram of beamforming of a virtual reality host and a virtual reality device provided by the present application
- FIG. 4 is a schematic structural diagram of a virtual reality image transmitting apparatus provided by the present application.
- FIG. 5 is a schematic structural diagram of another virtual reality image transmitting apparatus provided by the present application.
- This application describes a VR system including a virtual reality host and a virtual reality device as an example.
- a VR system including a virtual reality host and a virtual reality device as an example.
- a virtual reality host is an important device in a VR system for generating a virtual reality image, or acquiring a virtual reality image from a local, other electronic device or the cloud, and transmitting the virtual reality image to the virtual reality device.
- the virtual reality host can be an electronic device such as a television, a notebook computer, or a smart phone.
- a virtual reality host can also be referred to as a virtual reality host or the like.
- a virtual reality image which may be a three-dimensional stereoscopic image, such as a virtual reality host generated from a geometric model or transformed from a solid model in a two-dimensional image.
- the virtual reality device is a Head Mounted Display (HMD).
- HMD Head Mounted Display
- the virtual reality display is disposed directly in front of the user's eyes, and the virtual reality display usually includes two left and right glasses, each of which is separately separated.
- the left and right eye screens respectively display different images of the left and right eyes, and the human eye obtains the information with the difference and generates a stereoscopic effect in the mind.
- Beam is a communication resource.
- the beam can be a wide beam, or a narrow beam, or Other types of beams. Different beams can be considered as different communication resources.
- the same information or different information can be transmitted through different beams.
- Beamforming is to form a specific beam in a specified direction when transmitting a wireless signal through multiple antennas, thereby obtaining a large signal gain in a desired direction, thereby reducing power consumption and ensuring signal quality.
- the transmit beams generated by the transmitting end may be distributed in different directions of 360 degrees in the space, and the receiving beam generated at the receiving end may also be distributed in different directions of 360 degrees in the space.
- the beam training may be a process of beam alignment between a transmit beam generated by a transmitting end and a receive beam generated by a receiving end, from which an optimal transmit-receive beam is determined, and the transmit-receive beam is between a transmitting end and a receiving end.
- Wireless channel may be a process of beam alignment between a transmit beam generated by a transmitting end and a receive beam generated by a receiving end, from which an optimal transmit-receive beam is determined, and the transmit-receive beam is between a transmitting end and a receiving end.
- the "beam” may also be referred to as a "transmission resource” or the like. That is to say, the name of the "beam” is not limited herein, as long as the above concept is expressed.
- the VR system includes a virtual reality host and a virtual reality device, wherein the virtual reality device takes a head mounted device as an example.
- the virtual reality host generates a virtual reality image, and then the virtual reality host sends the generated virtual reality image to the HMD.
- the HMD receives the virtual reality image, the virtual reality image is displayed, so the user wearing the HMD can view the virtual reality image.
- a virtual reality host generally transmits a virtual reality image to a virtual reality device through a wireless channel, but the amount of data of the virtual reality image that needs to be transmitted is very large.
- the bandwidth of the wireless channel is 4M, then the transmission of the VR image requires a minimum of 360 hours, that is, 15 days. It can be seen that the virtual reality image needs to be performed due to the limitation of the bandwidth of the wireless channel and the huge amount of data. Compression to reduce the amount of data transmitted and improve transmission efficiency.
- the virtual reality host needs to determine the image compression ratio.
- the following describes the current VR system to determine the image compression ratio.
- a virtual reality host determines an image compression ratio based on the channel quality of the wireless channel.
- the virtual reality host detects the current channel quality of the wireless channel, for example, the current channel quality is poor, and in order to reduce the amount of data transmitted, a larger image compression ratio is determined.
- the image compression ratio determined in this way is not accurate enough.
- the user due to the switching rate of the virtual reality image, that is, the content changes quickly and slowly, for slow motion or even static virtual reality images, the user tends to be more critical about the image quality, so in this case, in order to ensure image quality, it cannot be used.
- a larger image compression ratio compresses the virtual reality image.
- the virtual reality host will always compress the virtual reality image with a larger image compression ratio, resulting in poor image quality. It can be seen that the current method of determining the image compression ratio is less accurate.
- the present application provides a virtual reality image transmitting method. That is, the virtual reality host determines an image compression ratio according to the motion information of the virtual reality device, and by using the method, the determined image pressure is improved. The accuracy of the reduction ratio.
- FIG. 2 is a flowchart of a method for transmitting a virtual reality image provided by the present application.
- FIG. 2 can also be understood as the information interaction between the virtual reality host and the HMD in the VR architecture shown in FIG. 1 .
- schematic diagram. The process of this method is described as follows:
- the motion information is used to indicate the current state of motion of the virtual reality device.
- S201 may be completed by a virtual reality device or by a virtual reality host.
- a motion sensor such as a gyroscope or an accelerometer can be set on the virtual reality device to detect the motion information.
- the camera or other optical sensor may be set on the virtual reality host, and the positional movement size of the virtual reality device per unit time is detected by the camera or the optical sensor, thereby determining the motion information of the virtual reality device.
- the motion information is detected by using a virtual reality device as an example, and the content described below will continue to use the virtual reality device to detect motion information as an example.
- the motion information may include one or more of position information, speed information, acceleration information, angular velocity information, and angular acceleration information of the virtual reality device.
- the virtual reality device can detect motion information periodically or in real time.
- the virtual reality device may also detect the motion information only after receiving the request for detecting the motion information sent by the virtual reality host, and does not detect the request without receiving the request.
- the virtual reality host may send a request for requesting detection of motion information to the virtual reality host, and the virtual reality device detects the motion information after receiving the request.
- the virtual reality host may send a request for requesting detection of the motion information to the virtual reality device each time the image compression ratio is determined, and the virtual reality device detects the motion after receiving the request. information.
- the virtual reality device sends the motion information to the virtual reality host, and correspondingly, the virtual reality host receives the motion information.
- the virtual reality device may detect the motion information periodically or in real time, or may detect the motion information after receiving the request for detecting the motion information sent by the virtual reality host. Then, if the virtual reality device detects the motion information after receiving the request for detecting the motion information sent by the virtual reality host, the virtual reality device may send the motion information to the virtual reality host after detecting the motion information. If the virtual reality device detects the motion information periodically or in real time, the virtual reality device may send the motion information to the virtual reality host every time the motion information is detected, or may receive the motion information sent by the virtual reality host. The newly detected motion information or all motion information detected during the most recent time period is sent to the virtual reality host after the request.
- the virtual reality device when transmitting the motion information, may simultaneously send the time information of the virtual reality device to generate the motion information and the device identifier of the virtual reality device.
- the virtual reality device may simultaneously send the time information of the virtual reality device to generate the motion information and the device identifier of the virtual reality device.
- the virtual reality host usually one virtual reality host needs to communicate with multiple HMDs, that is, the virtual reality host needs to send virtual reality images to multiple HMDs, and the virtual reality host can also receive motion information sent by multiple HMDs.
- the HMD device ID of the HMD that generates the motion information may be sent together by the HMD to identify the motion information that is sent by the HMD of the plurality of HMDs.
- the virtual reality host may receive multiple pieces of motion information sent by the same HMD, so each HMD may send the time information of the motion information generated by the HMD together when transmitting its own motion information, so as to be virtualized.
- the real host recognizes the generation time of each of the plurality of pieces of motion information.
- the virtual reality host Take the virtual reality host as an example of communication with only one virtual reality device. If the virtual reality device sends the motion information together with the time information for generating the motion information, the virtual reality host may store the motion information and the time information correspondingly after receiving the motion information and the time information. If the virtual reality host receives the plurality of motion information, the generation time of each motion information is different. When the virtual reality host needs to use the motion information, the latest time motion information may be selected from the plurality of motion information, or may be taken within a certain period of time. The average of all the motion information, and the average is used as the required motion information.
- the three motion information sent by the virtual reality host to the virtual reality device is a rate of 3 m/s, a rate of 4 m/s, and a rate of 3 m/s, and the time information for generating the three pieces of motion information is separately received, respectively. It is 00:10, 00:15, and 00:20.
- the time information of the latest time can be directly selected, that is, the virtual reality device moves at a speed of 3 m/s at 00:20, and can also take a period of time, for example, 00:15-00:20.
- the average of the motion information that is, an average of 3 m/s and 4 m/s, that is, 3.5 m/s
- the virtual reality host determines 3.5 m/s as the required motion information.
- the period of time mentioned here may be determined according to actual conditions, and the application is not limited.
- the virtual reality host determines an image compression ratio according to the motion information.
- a possible implementation manner of S203 is: the virtual reality host firstly determines an image compression ratio according to the channel quality of the wireless channel, and then determines a final image compression ratio according to the motion information and the initially determined image compression ratio. . In this way, considering two different factors, namely channel quality and motion information, the determined image compression ratio is more accurate.
- the implementation process of this method is specifically described below.
- the virtual reality host determines the final image compression ratio, which can be done in three steps.
- the first step is to determine a wireless channel between the virtual reality host and the virtual reality device; in the second step, the image compression ratio is initially determined according to the channel quality of the wireless channel; and the third step is to determine the final based on the motion information and the initially determined image compression ratio.
- Image compression ratio is to determine a wireless channel between the virtual reality host and the virtual reality device.
- the current VR system can be roughly divided into two categories: the first type, a VR system that uses beamforming technology for data transmission, for example, a VR system that implements data transmission using a high frequency band (for example, 60 GHz) communication method.
- the second category VR systems that do not utilize beamforming techniques for data transmission, such as VR systems that implement low-band (eg, 30 GHz) communication techniques for data transmission.
- the wireless channel between the virtual reality host and the virtual reality device may vary.
- the virtual reality host and the virtual reality device can perform beam training before establishing communication. Since the virtual reality host in the current VR system is not clear about the location of the virtual reality device. Therefore, beam training between the virtual reality host and the virtual reality device requires a 360-degree search, that is, a transmit beam generated by the virtual reality host (distributed at different angles within 360 degrees) and a receive beam generated by the virtual reality device.
- the alignment process distributed at different angles within 360 degrees
- the transmit-receive beam formed after the alignment is the wireless channel between the virtual reality host and the virtual reality device. It can be seen that the beam training between the virtual reality host and the virtual reality device in the current first type of VR system takes a long time, resulting in an excessive image transmission delay.
- the virtual reality host may determine the optimal wireless transmission direction from the virtual reality device to generate at least one wireless transmission direction distributed within a range of 360 degrees according to the location information of the virtual reality device. It is only necessary to detect the channel quality of the wireless channel in the optimal wireless transmission direction, and it is not necessary to search within a range of 360 degrees, which shortens the delay caused by beam training.
- the first angle range may be determined according to the moving distance between the location information of the current time and the location information of the previous moment by the virtual reality device, or may be based on the historical location information of the virtual reality device and the current
- the location information predicts location information of the virtual reality device at the next moment, and is determined according to the predicted moving distance between the location information of the next moment and the location information of the current moment.
- the transmit beam generated by the virtual reality host is divided into 8 directions within a range of 360 degrees.
- the virtual reality device moves from the first position to the second position, and the first The moving distance from the position to the second position, that is, the moving range is in two directions of the transmitting beam generated by the virtual reality host, that is, the direction t1-t2.
- the virtual reality host only needs to search for the receiving beam generated by the virtual reality device in the direction t1-t2.
- the optimal wireless channel determined by the search result for the virtual reality host is the transmit-receive beam (t2, r8).
- the VR system since the VR system does not adopt beamforming technology, there is no need to search for the beamforming direction.
- the wireless channel between the virtual reality host and the virtual reality device is often set in advance. Therefore, the virtual reality host can directly execute the second step of S203.
- the virtual reality host After determining the wireless channel, the virtual reality host proceeds to the second step, that is, the virtual reality host initially determines the image compression ratio according to the channel quality of the wireless channel.
- the virtual reality host needs to detect the channel quality of the wireless channel.
- the specific implementation manner may be multiple.
- the virtual reality host detects the channel quality in real time or periodically, and the virtual reality host can store the detected channel quality.
- the virtual reality host can determine the channel quality of the last detection in the stored channel quality. This method is relatively simple, and the virtual reality host does not need to specifically detect the channel quality, which helps improve efficiency; or the virtual reality host can directly The current channel quality is detected so that the determined channel quality is more accurate.
- the virtual reality host initially determines the image compression ratio according to the mapping relationship between the previously stored channel quality and the image compression ratio.
- mapping relationships between channel quality and image compression ratio of a wireless channel there are various mapping relationships between channel quality and image compression ratio of a wireless channel, and Table 1 is an example.
- the channel quality of the wireless channel in Table 1 is exemplified by the transmission rate of the wireless channel.
- the image compression ratio can be initially determined to be 4:1 according to Table 1.
- the virtual reality host determines the final image compression ratio based on the motion information and the initially determined image compression ratio.
- the virtual reality host may first predict the motion state of the virtual reality device in the future first time period according to the received motion information. The reason for the virtual reality host to predict the motion state of the virtual reality device within the first time period in the future will be introduced later. The virtual reality host then determines the final image compression ratio based on the predicted motion state of the virtual reality device within the first time period of the future and the initially determined image compression ratio.
- the state of motion of a virtual reality device can usually be divided into two situations, static or moving.
- the implementation process of the third step is divided into two cases: In the first case, how the virtual reality host determines the final image when the virtual reality device is in a stationary state for the first time period in the future. Compression ratio; second, how the virtual reality host determines the final image compression ratio when the virtual reality device is in motion for the first time period in the future.
- the user's motion state changes at any time, that is, the motion state of the virtual reality device changes at any time.
- the virtual reality device detects that the current state is static, that is, the rate is 0 m/s
- the virtual reality device sends the motion information with the rate of 0 m/s to the virtual reality host, and the virtual reality device receives the compression sent by the virtual reality host.
- This process up to the virtual reality image may take a certain amount of time after which the virtual reality device may have changed from a stationary state to a motion state. Therefore, in order to improve the accuracy of the determined image compression ratio, the virtual reality host can predict the motion state within the first duration of the virtual reality device in the future.
- the virtual reality host receives the motion information sent by the virtual reality device and the time information of the virtual reality device to generate the motion information at 00:10, specifically, the rate of the virtual reality device is 0 m/s at 00:08. . Then the virtual reality host can only determine that the virtual reality device is still at 00:08. If the virtual reality host compresses the virtual reality image, and the process of transmitting the compressed virtual reality image to the virtual reality device takes 5 seconds, the virtual reality device receives the compressed virtual reality image at 00:15, but the virtual reality at 00:15. The device may have changed from a stationary state to a motion state. Therefore, in order to improve the accuracy of the image compression ratio, the virtual reality host may predict the motion state of the virtual reality device within the first time duration, for example, within 5 seconds, according to the received motion information.
- the first duration may be determined empirically or may be determined by calculation. If it is determined by calculation, the reason for predicting the motion state of the virtual reality device in the future first time period by the foregoing virtual reality host is that the first time length may be that the virtual reality device starts from the time when the certain motion information is generated to the receiving virtual
- the duration can be specifically divided into three sums of time.
- the first duration may be the length of time required for the virtual reality device to send the motion information to the virtual reality host
- the second duration may be the duration required for the virtual reality host to compress the virtual reality image
- the third duration may be the virtual reality host. The length of time it takes to send a compressed virtual reality image to a virtual reality device.
- the first duration may be a time difference between a time when the virtual reality host receives the motion information and a time when the virtual reality device generates the motion information.
- the second duration may be that the total number of frames of the virtual reality image that needs to be transmitted is multiplied by the length of time required to compress one frame of the virtual reality image.
- the length of time required to compress one frame of virtual reality image can be obtained through testing.
- the virtual reality host can record the first time before compressing one frame of the virtual reality image and the second time when the compression is completed, and the difference between the second time and the first time is the length of time required to compress one frame of the virtual reality image. .
- the third duration may be that the amount of data transmitted at a time is divided by the transmission rate of the wireless channel.
- the amount of data transmitted at one time can be determined by the total number of frames of the transmitted virtual reality image and the amount of data included in each frame of the virtual reality image. After determining the three durations, the three durations are added to obtain the first duration.
- the first duration may be other algorithms for the first duration, which is not specifically limited in this application.
- the virtual reality host may determine, according to the received motion information and the historical motion information of the virtual reality device, whether the virtual reality device is in a stationary state or in a motion state in the future first time period.
- the virtual reality device sends the motion information together, the time information for generating the motion information is also sent, and the virtual reality host can receive the motion information and generate the time information of the motion information.
- the motion information and the time information are stored correspondingly, thereby obtaining information that the virtual reality device motion information changes with time, that is, obtaining historical motion information of the virtual reality device.
- the historical motion information of the virtual reality device stored in the virtual reality host is: at time 00:00, the rate of the virtual reality device is zero.
- the rate of the virtual reality device is 1 m/s.
- the virtual reality device rate is 2 m/s.
- the rate of the virtual reality device is 3 m/s.
- the rate of the virtual reality device is 2 m/s.
- the rate of the virtual reality device is 1 m/s.
- the rate of the virtual reality device is 0 m/s.
- the rate of the virtual reality device is 0 m/s.
- the rate of the virtual reality device is 0 m/s.
- the rate of the virtual reality device is 0 m/s.
- the rate of the virtual reality device is 0 m/s.
- the rate of the virtual reality device is 0 m/s and so on.
- the virtual reality host receives the motion information sent by the virtual reality device and the time information of the motion information at 00:14, specifically, the speed of the virtual reality device is 2 m/ at time 00:12. s. If the first duration is 3s, the virtual reality host according to the historical motion information of the virtual reality device, it can be known that the virtual reality host rate is 3 m/s in the next 3 s, that is, at 00:15, that is, the virtual reality device is determined to be in motion. . For another example, the virtual reality host receives the motion information sent by the virtual reality host and the time information of the motion information at 00:30. Specifically, the virtual reality device has a rate of 0 m/s at 00:28.
- the virtual reality host according to the historical motion information of the virtual reality device, it can be seen that in the next 3s, that is, at 00:15, the virtual reality host rate is 0 m/s, that is, the virtual reality device is in a static state.
- the channel quality of the wireless channel between the virtual reality host and the virtual reality device has a certain correlation with the motion state of the virtual reality device. For example, when the virtual reality device moves rapidly, the channel stability is poor, and the channel quality is poor. The change will be relatively large, and when the virtual reality device is stationary, the channel quality is generally stable and the change is not large. Therefore, when the virtual reality host determines that the virtual reality device is in a static state for the first time period in the future, it can be determined that the channel quality is relatively stable, and the virtual reality host can directly determine that the image compression ratio corresponding to the detected channel quality of the wireless channel is The final image compression ratio, that is, the initially determined image compression ratio is directly used as the final image compression ratio.
- the motion information may be used according to the motion information (the motion information here is used to indicate the current motion state of the virtual reality device, not the aforementioned Historical motion information) adjusts the initially determined image compression ratio. For example, if the motion information is used to indicate that the moving speed of the virtual reality device is slow, the virtual reality host appropriately reduces the image compression ratio based on the initially determined image compression ratio. If the motion information is used to indicate that the moving speed of the virtual reality device is faster, the virtual reality host appropriately increases the image compression ratio based on the initially determined image compression ratio.
- the moving speed of the virtual reality device is fast, that is, the switching rate of the virtual reality image displayed by the virtual reality device, that is, the content changes rapidly.
- the switching rate of the virtual reality image displayed by the virtual reality device that is, the content changes rapidly.
- the content of the virtual reality image displayed by the virtual reality device changes slowly.
- the user often has high quality requirements for the image. For example, the image clarity, resolution, and the like are required to be high. At this time, the image compression ratio is reduced, and the image quality can be improved, thereby improving the user experience.
- the virtual reality host may also consider the trend of the channel quality of the wireless channel in the future first time period when adjusting the initially determined image compression ratio according to the motion information. That is, the virtual reality host can adjust the initially determined image compression ratio based on the motion information and the predicted change trend of the channel quality in the first time duration in the future. details as follows:
- the virtual reality host If the motion information is used to indicate that the moving rate of the virtual reality device is slow, and the channel quality in the first time period is good, the virtual reality host appropriately reduces the image compression ratio based on the initially determined image compression ratio. If the motion information is used to indicate that the moving speed of the virtual reality device is faster, and the channel quality deteriorates in the first time period in the future, the virtual reality host appropriately increases the image compression ratio based on the initially determined image compression ratio.
- the virtual reality host increases the image compression ratio based on the initially determined image compression ratio, but increasing the image compression ratio will greatly reduce the amount of data transmitted. The transmission requirement can be satisfied without being too good, so if the channel quality is deteriorated, the transmission of the virtual reality image will not be affected.
- the moving speed of the virtual reality device is slow, that is, the content of the virtual reality image displayed by the virtual reality device changes slowly.
- the virtual reality host reduces the image compression ratio based on the initially determined image compression ratio, but after reducing the image compression ratio, the amount of data transmitted may be greatly increased, if the channel of the wireless channel Better quality will not affect the transmission of increased data volume. That is, the virtual reality host adjusts the image compression ratio according to the trend of the motion information and the channel quality in the first time period in the future, so that the accuracy of determining the image compression ratio can be further improved.
- the virtual reality host may choose to adjust the image compression ratio only according to the motion information, or may select to adjust the image compression ratio according to the motion information and the trend of the channel quality of the wireless channel in the future first time period.
- S204 The virtual reality host compresses the virtual reality image to be sent according to the image compression.
- the virtual reality host may compress each frame of the virtual reality image according to the determined final image compression ratio, or may compress the multi-frame virtual reality image package together each time.
- the virtual reality host determines the image compression ratio according to the motion information of the virtual reality device. For example, when the virtual reality device moves at a slow rate, that is, the content of the virtual reality image displayed by the virtual reality device changes slowly, in order to ensure virtual reality. The quality of the image determines a smaller image compression ratio. When the virtual reality device moves at a faster rate, that is, the content of the virtual reality image displayed by the virtual reality device changes rapidly, and a large image compression ratio is determined in order to ensure the consistency of the virtual reality image. In this way, the accuracy of the determined image compression ratio is improved, and the user's needs can be better met.
- the virtual reality host needs to determine the modulation and coding mode before transmitting the virtual reality image, and then modulates and encodes the virtual reality image according to the modulation and coding mode.
- the following describes how the current VR system determines the modulation and coding method.
- the virtual reality host determines the modulation and coding mode according to the channel quality of the wireless channel. For example, if the channel quality of the wireless channel is good, a modulation and coding mode with a fast modulation and coding rate is determined.
- the modulation coding method determined in this way is not accurate enough. This is because, in fact, each modulation coding mode corresponds to a respective modulation coding rate, and the modulation coding rate is related to the frame error rate generated by the virtual reality image during transmission, for example, when the modulation coding rate is fast, although transmission The efficiency is higher, but the probability of frame error and frame loss is higher. Big.
- the moving speed of the virtual reality device is fast, the content of the virtual reality image displayed by the virtual reality device changes rapidly.
- Modulation coding method with a coding rate too fast If the channel quality of the wireless channel is always good, the virtual reality host will always adopt the modulation and coding mode with a faster modulation and coding rate, that is, the probability of frame loss and frame error is large, and once the frame is dropped, the user sees the virtual Realistic images will be incoherent. It can be seen that at present, in the VR system, the modulation and coding mode determined by the virtual reality host according to the channel quality is not accurate enough. The process of determining the modulation coding method in this application is described below.
- the virtual reality host determines a modulation and coding manner according to the motion information.
- S204 is not limited as long as it occurs after S202.
- FIG. 2 is an example in which S205 occurs after S204.
- a possible implementation manner of S205 is that the virtual reality host firstly determines a modulation and coding mode according to the channel quality of the wireless channel, and then determines the final modulation and coding mode according to the motion information and the initially determined modulation and coding mode. In this way, considering two different factors, namely channel quality and motion information, the determined modulation and coding method will be more accurate. The implementation process of this method is specifically described below.
- the virtual reality host determines the final modulation and coding scheme, which can be done in two steps.
- the first step is to determine a wireless channel between the virtual reality host and the virtual reality device; in the second step, the modulation and coding mode is initially determined according to the channel quality of the wireless channel; and the third step is to determine the final according to the motion information and the initially determined modulation and coding mode.
- Modulation coding method is to determine a wireless channel between the virtual reality host and the virtual reality device.
- the virtual reality host first needs to determine the mapping relationship between channel quality and channel quality and modulation and coding mode.
- the virtual reality host can be pre-stored. Therefore, the main requirement of the virtual reality host is the channel quality.
- the manner in which the virtual reality host determines the channel quality has been previously described.
- the virtual reality host initially determines the modulation and coding mode according to the mapping relationship between the stored channel quality and the modulation and coding mode.
- Table 2 is an example.
- Table 2 is only an example of the mapping relationship between the channel quality and the modulation and coding scheme, and is not a limitation of the mapping relationship between the channel quality and the modulation and coding scheme.
- the channel quality of the wireless channel in Table 1 is exemplified by the transmission rate of the wireless channel.
- the mapping relationship between the channel quality of the wireless channel and the modulation and coding mode may be a modulation and coding scheme (MCS) index table in the prior art, or may be a user-defined mapping relationship. .
- MCS modulation and coding scheme
- the modulation and coding mode is 16-QAM.
- the implementation process of the third step is described below, that is, the virtual reality host determines the final modulation and coding mode according to the motion information and the initially determined modulation and coding mode. Similar to S203, the virtual reality host predicts the motion state of the virtual reality device in the future first time period according to the received motion information. The final modulation and coding scheme is then determined based on the motion state and the initially determined modulation and coding scheme. Therefore, the implementation process of the third step is also divided into two cases: In the first case, how the virtual reality host determines the final modulation and coding mode when the virtual reality device is in a stationary state for the first time period in the future.
- the virtual reality host determines the final modulation and coding mode when the virtual reality device is in motion for the first time period in the future.
- the reason and implementation manner of the virtual reality host predicting the motion state of the virtual reality device in the first time period in the future has been introduced, and will not be repeatedly described herein. Therefore, the following two cases of the implementation process of the second step are directly introduced.
- the channel quality of the wireless channel between the virtual reality host and the virtual reality device has a certain correlation with the motion state of the virtual reality device, and the virtual reality device determines that the virtual reality device is in the first time period in the future. In the quiescent state, it can be determined that the channel quality is relatively stable, and the virtual reality host can directly determine the modulation and coding mode corresponding to the channel quality of the detected wireless channel as the final modulation and coding mode.
- the second case is how the virtual reality host determines the final modulation and coding mode when the virtual reality device is in motion for the first time in the future.
- the initially determined modulation and coding mode may be adjusted according to the motion information. For example, if the motion information is used to indicate that the moving rate of the virtual reality device is slow, the virtual reality host adjusts the initially determined modulation and coding mode to a first modulation and coding mode, and the modulation and coding rate of the first modulation code is greater than or equal to a preliminary determination. The modulation coding rate of the modulation coding mode.
- the virtual reality host modulates the initially determined modulation and coding mode into a second modulation and coding mode, where the modulation coding rate of the second modulation and coding mode is smaller than the initially determined modulation code.
- the modulation coding rate of the mode is smaller than the initially determined modulation code.
- the virtual reality host can determine the modulation and coding mode with a lower modulation and coding rate.
- the moving rate of the virtual reality device is slow, that is, the content of the virtual reality image displayed by the virtual reality device changes slowly, that is, the content of the virtual reality image transmitted each time changes less, and the number of frames of the virtual reality image that needs to be transmitted each time is smaller.
- a modulation and coding method that does not require a modulation coding rate to be too low at this time can also be implemented, so the virtual reality host can determine a modulation coding mode with a higher modulation coding rate, thereby improving transmission efficiency.
- the moving rate of the virtual reality device is slow, since the wireless channel changes slowly, the detected channel quality is relatively accurate, so the virtual reality host can directly determine the modulation and coding mode corresponding to the detected channel quality as the final modulation. Encoding.
- the virtual reality host modulates and encodes the compressed virtual reality image according to a modulation and coding manner.
- the virtual reality host may modulate and encode each frame of the compressed virtual reality image according to the final modulation and coding manner. If the virtual reality host is packaged and compressed with multi-frame virtual reality images, then the virtual reality host can be based on the most The final modulation coding method modulates and encodes the virtual reality image compression package together.
- the virtual reality host sends the modulated and encoded compressed virtual reality image to the virtual reality device, and correspondingly, the virtual reality device receives the modulated and encoded virtual reality image.
- the virtual reality device After receiving the modulated encoded virtual reality image, the virtual reality device demodulates and decodes the virtual reality image to obtain a compressed virtual reality image, and then decompresses the compressed virtual reality image, and finally displays the virtual reality image.
- the virtual reality host determines the modulation and coding mode according to the motion information of the virtual reality device. For example, when the moving speed of the virtual reality device is fast, that is, the switching speed of the virtual reality image displayed by the virtual reality device is fast, in order to ensure the consistency and fluency of the virtual reality image, the probability of occurrence of frame error and frame loss is reduced, and the probability of avoiding frame loss is avoided.
- the image is retransmitted, so the virtual reality host can determine the modulation coding mode with a lower modulation coding rate.
- the virtual reality host may determine a modulation coding mode with a higher modulation coding rate or directly determine a modulation and coding mode according to the channel quality of the wireless channel. , thereby improving transmission efficiency. In this way, the accuracy of the determined modulation and coding method is improved, and the user's needs can be better met.
- the virtual reality host determines an image compression ratio and a modulation and coding manner according to motion information of the virtual reality device. For example, when the virtual reality device moves at a slow rate, that is, the virtual reality image content displayed by the virtual reality device changes slowly, a smaller image compression ratio is determined, and the modulation and coding mode can be directly determined according to the current channel quality. When the virtual reality device moves at a faster rate, that is, the virtual reality image displayed by the virtual reality device changes rapidly, and for the flash lens, the image quality of the human eye is reduced, so in order to ensure the continuity of the virtual reality image, A larger image compression ratio can be determined.
- the virtual reality host determines the modulation coding mode with a lower modulation coding rate, which double ensures the continuity of the virtual reality image. Therefore, in the method for transmitting a virtual reality image provided by the present application, the virtual reality host can determine the image compression ratio and the modulation and coding mode according to the motion information of the virtual reality device, thereby ensuring the quality of the virtual reality image, improving the transmission efficiency, and being better. To meet the needs of users.
- FIG. 4 shows a schematic structural diagram of a virtual reality image transmitting apparatus 400.
- the virtual reality image transmitting apparatus 400 can implement the functions of the virtual reality host referred to above.
- the virtual reality image transmitting apparatus 400 may include a receiving unit 401, a processing unit 402, and a transmitting unit 403.
- the receiving unit 401 can be used to execute S202 in the embodiment shown in FIG. 2, and/or other processes for supporting the techniques described herein.
- Processing unit 402 may be used to perform S203-S206 in the embodiment shown in FIG. 2, and/or other processes for supporting the techniques described herein.
- Transmitting unit 403 can be used to perform S207 in the embodiment shown in FIG. 2, and/or other processes for supporting the techniques described herein. All the related content of the steps involved in the foregoing method embodiments may be referred to the functional descriptions of the corresponding functional modules, and details are not described herein again.
- the above-described virtual reality image transmitting apparatus is displayed in the form of a functional unit.
- the term "unit” as used herein, without limitation, may refer to an application-specific integrated circuit (ASIC), electronic circuit, (shared, dedicated, or group) that executes one or more software or firmware programs.
- a processor and memory, combinational logic, and/or other suitable components that provide the described functionality.
- the virtual reality image transmitting apparatus 400 can also be implemented by the structure as shown in FIG.
- the virtual reality image transmitting apparatus 500 may include: a memory 501, a processor 502, and a connection. Receiver 503, transmitter 504 and bus 505.
- the memory 501, the processor 502, the receiver 503, and the transmitter 504 can be connected by a bus 505.
- the receiver 503 is configured to receive first motion information that is sent by the virtual reality device, where the first motion information is used to indicate a current motion state of the virtual reality device, and the processor 502 is configured to determine a first image compression ratio according to the first motion information. And compressing the virtual reality image to be sent according to the first image compression; the transmitter 504 is configured to send the compressed virtual reality image to the virtual reality device.
- the operations performed by the processor 502, the receiver 503, and the transmitter 504 reference may be made to the corresponding steps performed by the virtual reality host in the embodiment of FIG. 2, and details are not described herein again.
- the memory 501 is configured to store a computer execution instruction.
- the processor 502 executes a computer execution instruction stored in the memory 501 to implement the foregoing functions, thereby implementing the virtuality provided by the embodiment shown in FIG. 2.
- Realistic image transmission method For a specific image capturing method, reference may be made to the related descriptions in the above and the drawings, and details are not described herein again.
- the processor 502 may be a field-programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processing unit. (central processor unit, CPU), network processor (NP), digital signal processor (DSP), microcontroller (micro controller unit (MCU), can also use programmable controller (programmable Logic device, PLD) or other integrated chip.
- the bus 505 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus.
- PCI peripheral component interconnect
- EISA extended industry standard architecture
- the bus 505 can be divided into an address bus, a data bus, a control bus, and the like. For ease of representation, only one thick line is shown in Figure 5, but it does not mean that there is only one bus or one type of bus.
- the invention may be implemented in whole or in part by software, hardware, firmware or any combination thereof.
- software it may be implemented in whole or in part in the form of a computer program product.
- the computer program product includes one or more computer instructions.
- the computer program instructions When the computer program instructions are loaded and executed on a computer, the processes or functions described in accordance with the present application are generated in whole or in part.
- the computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable device.
- the computer instructions can be stored in a computer readable storage medium or transferred from one computer readable storage medium to another readable storage medium, for example, the computer instructions can be passed from a website site, computer, server or data center Wired (eg, coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (eg, infrared, wireless, microwave, etc.) to another website site, computer, server, or data center.
- the computer readable storage medium can be any available media that can be accessed by a computer or a data storage device such as a server, data center, or the like that includes one or more available media.
- the usable medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a DVD), or a semiconductor medium (eg, a solid state disk (SSD)) or the like.
- a magnetic medium eg, a floppy disk, a hard disk, a magnetic tape
- an optical medium eg, a DVD
- a semiconductor medium eg, a solid state disk (SSD)
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Signal Processing (AREA)
- Human Computer Interaction (AREA)
- Software Systems (AREA)
- Computer Hardware Design (AREA)
- Optics & Photonics (AREA)
- Computer Graphics (AREA)
- Quality & Reliability (AREA)
- Computer Networks & Wireless Communication (AREA)
- Architecture (AREA)
- Biophysics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Cardiology (AREA)
- General Health & Medical Sciences (AREA)
- Heart & Thoracic Surgery (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
- User Interface Of Digital Computer (AREA)
- Compression, Expansion, Code Conversion, And Decoders (AREA)
- Detection And Prevention Of Errors In Transmission (AREA)
- Mobile Radio Communication Systems (AREA)
Abstract
Description
Claims (25)
- 一种虚拟现实图像发送方法,其特征在于,包括:虚拟现实主机接收虚拟现实设备发送的第一运动信息,所述第一运动信息用于指示所述虚拟现实设备的当前运动状态;所述虚拟现实主机根据所述第一运动信息确定第一图像压缩比;所述虚拟现实主机根据所述第一图像压缩比对待发送的虚拟现实图像进行压缩;所述虚拟现实主机将压缩的所述虚拟现实图像发送给所述虚拟现实设备。
- 根据权利要求1所述的方法,其特征在于,在所述虚拟现实主机根据所述第一运动信息确定第一图像压缩比之前,所述方法还包括:所述虚拟现实主机检测无线信道当前的信道质量,其中,所述无线信道为所述虚拟现实设备与所述虚拟现实主机之间的通信信道;所述虚拟现实主机根据无线信道的信道质量与图像压缩比之间的映射关系,确定与检测的信道质量对应的图像压缩比;所述虚拟现实主机根据所述第一运动信息确定第一图像压缩比,包括:所述虚拟现实主机根据所述第一运动信息,确定与检测的信道质量对应的图像压缩比为所述第一图像压缩比,或所述虚拟现实主机根据所述第一运动信息调整与检测的信道质量对应的图像压缩比,并将调整后的图像压缩比作为所述第一图像压缩比。
- 根据权利要求2所述的方法,其特征在于,所述虚拟现实主机根据所述第一运动信息,确定与检测的信道质量对应的图像压缩比为所述第一图像压缩比,包括:所述虚拟现实主机根据所述第一运动信息和所述虚拟现实设备的历史运动信息,若确定在未来的第一时长内所述虚拟现实设备处于静止状态,则确定与检测的信道质量对应的图像压缩比为所述第一图像压缩比。
- 根据权利要求2所述的方法,其特征在于,所述虚拟现实主机根据所述第一运动信息调整与检测的信道质量对应的图像压缩比,包括:所述虚拟现实主机根据所述第一运动信息和所述虚拟现实设备的历史运动信息,若确定在未来的第一时长内所述虚拟现实设备处于运动状态,则根据所述第一运动信息调整与检测的信道质量对应的图像压缩比。
- 根据权利要求2或4所述的方法,其特征在于,所述虚拟现实主机根据所述第一运动信息调整与检测的信道质量对应的图像压缩比,包括:若所述第一运动信息用于指示所述虚拟现实设备的移动速率小于或等于第一速率,所述虚拟现实主机减小与检测的信道质量对应的图像压缩比;或若所述第一运动信息用于指示所述虚拟现实设备的移动速率大于第二速率,所述虚拟现实主机增大与检测的信道质量对应的图像压缩比;其中,所述第一速率小于或等于所述第二速率。
- 根据权利要求2-5任一所述的方法,其特征在于,在所述虚拟现实主机检测无线信道当前的信道质量之后,所述方法还包括:所述虚拟现实主机根据无线信道的信道质量与调制编码方式之间的映射关系,确定与检测的信道质量对应的调制编码方式;所述虚拟现实主机根据所述第一运动信息确定与检测的信道质量对应的调制编码方式为第一调制编码方式,或,所述虚拟现实主机根据所述第一运动信息调整与检测 的信道质量对应的调制编码方式,并将调整后的调制编码方式作为第一调制编码方式;在所述虚拟现实主机将压缩的所述虚拟现实图像发送给所述虚拟现实设备之前,所述方法还包括:所述虚拟现实主机根据所述第一调制编码方式对所述压缩的虚拟现实图像进行调制编码;所述虚拟现实主机将压缩后所述虚拟现实图像发送给所述虚拟现实设备,包括:所述虚拟现实主机将调制编码后的压缩的所述虚拟现实图像发送给所述虚拟现实设备。
- 根据权利要求6所述的方法,其特征在于,所述虚拟现实主机根据所述第一运动信息确定与检测的信道质量对应的调制编码方式为第一调制编码方式,包括:所述虚拟现实主机根据所述第一运动信息和所述虚拟现实设备的历史运动信息,若确定在未来的第一时长内所述虚拟现实设备处于静止状态,则确定与检测的信道质量对应的调制编码方式为所述第一调制编码方式。
- 根据权利要求6所述的方法,其特征在于,所述虚拟现实主机根据所述第一运动信息调整与检测的信道质量对应的调制编码方式,包括:所述虚拟现实主机根据所述第一运动信息和所述虚拟现实设备的历史运动信息,若确定在未来的第一时长内所述虚拟现实设备处于运动状态,则根据所述第一运动信息调整与检测的信道质量对应的调制编码方式。
- 根据权利要求8所述的方法,其特征在于,所述虚拟现实主机根据所述第一运动信息调整与检测的信道质量对应的调制编码方式,包括:若所述第一运动信息用于指示所述虚拟现实设备的移动速率小于或等于第一速率,所述虚拟现实主机将与检测的信道质量对应的调制编码方式调整为第二调制编码方式,所述第二调制编码方式的调制编码速率大于与检测的信道质量对应的调制编码方式的调制编码速率;若所述第一运动信息用于指示所述虚拟现实设备的移动速率大于第二速率,所述虚拟现实主机将所述与检测的信道质量对应的调制编码方式调整为第三调制编码方式,所述第三调制编码方式的调制编码速率小于与检测的信道质量对应的调制编码方式的调制编码速率;其中,所述第一速率小于所述第二速率。
- 根据权利要求2所述的方法,其特征在于,所述第一运动信息用于指示所述虚拟现实设备的当前位置信息,所述虚拟现实主机检测无线信道当前的信道质量,包括:所述虚拟现实主机根据所述位置信息在第一角度范围内从所述虚拟现实设备产生的分布于360度范围内的至少一个无线传输方向中确定第一无线传输方向;所述第一角度大于0度小于360度;所述虚拟现实主机检测所述第一无线传输方向上的无线信道的当前的信道质量。
- 根据权利要求1-9任一所述的方法,其特征在于,所述第一运动信息包括所述虚拟现实设备的速度信息、加速度信息、角速度信息以及角加速度信息中的一种或多种。
- 一种虚拟现实图像发送装置,其特征在于,包括:接收单元,用于接收虚拟现实设备发送的第一运动信息,所述第一运动信息用于指示所述虚拟现实设备的当前运动状态;处理单元,用于根据所述第一运动信息确定第一图像压缩比;并根据所述第一图像压缩比对待发送的虚拟现实图像进行压缩;发送单元,用于将压缩的所述虚拟现实图像发送给所述虚拟现实设备。
- 根据权利要求12所述的装置,其特征在于,所述处理单元还用于:检测无线信道当前的信道质量,其中,所述无线信道为所述虚拟现实设备与所述虚拟现实图像发送装置之间的通信信道;根据无线信道的信道质量与图像压缩比之间的映射关系,确定与检测的信道质量对应的图像压缩比;所述处理单元具体用于:根据所述第一运动信息,确定与检测的信道质量对应的图像压缩比为所述第一图像压缩比,或根据所述第一运动信息调整与检测的信道质量对应的图像压缩比,并将调整后的图像压缩比作为所述第一图像压缩比。
- 根据权利要求13所述的装置,其特征在于,所述处理单元具体用于:根据所述第一运动信息和所述虚拟现实设备的历史运动信息,若确定在未来的第一时长内所述虚拟现实设备处于静止状态,则确定与检测的信道质量对应的图像压缩比为所述第一图像压缩比。
- 根据权利要求13所述的装置,其特征在于,所述处理单元具体用于:根据所述第一运动信息和所述虚拟现实设备的历史运动信息,若确定在未来的第一时长内所述虚拟现实设备处于运动状态,则根据所述第一运动信息调整与检测的信道质量对应的图像压缩比。
- 根据权利要求13或15所述的装置,其特征在于,所述处理单元具体用于:若所述第一运动信息用于指示所述虚拟现实设备的移动速率小于或等于第一速率,则减小与检测的信道质量对应的图像压缩比;或若所述第一运动信息用于指示所述虚拟现实设备的移动速率大于第二速率,则增大与检测的信道质量对应的图像压缩比;其中,所述第一速率小于或等于所述第二速率。
- 根据权利要求13-16任一所述的装置,其特征在于,所述处理单元还用于:根据无线信道的信道质量与调制编码方式之间的映射关系,确定与检测的信道质量对应的调制编码方式;根据所述第一运动信息确定与检测的信道质量对应的调制编码方式为第一调制编码方式,或,根据所述第一运动信息调整与检测的信道质量对应的调制编码方式,并将调整后的调制编码方式作为第一调制编码方式;根据所述第一调制编码方式对所述压缩的虚拟现实图像进行调制编码;所述发送单元具体用于:将调制编码后的压缩的所述虚拟现实图像发送给所述虚拟现实设备。
- 根据权利要求17所述的装置,其特征在于,所述处理单元具体用于:根据所述第一运动信息和所述虚拟现实设备的历史运动信息,若确定在未来的第一时长内所述虚拟现实设备处于静止状态,则确定与检测的信道质量对应的调制编码 方式为所述第一调制编码方式。
- 根据权利要求17所述的装置,其特征在于,所述处理单元具体用于:根据所述第一运动信息和所述虚拟现实设备的历史运动信息,若确定在未来的第一时长内所述虚拟现实设备处于运动状态,则根据所述第一运动信息调整与检测的信道质量对应的调制编码方式。
- 根据权利要求19所述的装置,其特征在于,所述处理单元具体用于:若所述第一运动信息用于指示所述虚拟现实设备的移动速率小于或等于第一速率,将与检测的信道质量对应的调制编码方式调整为第二调制编码方式,所述第二调制编码方式的调制编码速率大于与检测的信道质量对应的调制编码方式的调制编码速率;若所述第一运动信息用于指示所述虚拟现实设备的移动速率大于第二速率,将所述与检测的信道质量对应的调制编码方式调整为第三调制编码方式,所述第三调制编码方式的调制编码速率小于与检测的信道质量对应的调制编码方式的调制编码速率;其中,所述第一速率小于所述第二速率。
- 根据权利要求13所述的装置,其特征在于,所述第一运动信息用于指示所述虚拟现实设备的当前位置信息,所述处理单元具体用于:根据所述位置信息在第一角度范围内从所述虚拟现实设备产生的分布于360度范围内的至少一个无线传输方向中确定第一无线传输方向;所述第一角度大于0度小于360度;检测所述第一无线传输方向上的无线信道的当前的信道质量。
- 根据权利要求12-20任一所述的装置,其特征在于,所述第一运动信息包括所述虚拟现实设备的速度信息、加速度信息、角速度信息以及角加速度信息中的一种或多种。
- 一种虚拟现实图像发送装置,其特征在于,包括:处理器和存储器;所述存储器用于存储计算机执行指令,当所述处理器执行所述指令时,使所述图像拍摄装置执行如权利要求1-11任意一项所述的方法。
- 一种计算机可读存储介质,其特征在于,包括指令,当其在计算机上运行时,使得计算机执行如权利要求1-11任意一项所述的方法。
- 一种包含指令的计算机程序产品,其特征在于,当其在计算机上运行时,使得计算机执行如权利要求1-11任意一项所述的方法。
Priority Applications (7)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US16/495,858 US11402894B2 (en) | 2017-03-22 | 2017-07-07 | VR image sending method and apparatus |
AU2017405914A AU2017405914B2 (en) | 2017-03-22 | 2017-07-07 | Virtual reality image sending method and apparatus |
CA3057180A CA3057180C (en) | 2017-03-22 | 2017-07-07 | Method and apparatus for sending virtual reality image |
KR1020197030873A KR102296139B1 (ko) | 2017-03-22 | 2017-07-07 | 가상 현실 이미지를 송신하는 방법 및 장치 |
CN201780012081.9A CN109074152B (zh) | 2017-03-22 | 2017-07-07 | 一种虚拟现实图像发送方法及装置 |
JP2019552177A JP6873268B2 (ja) | 2017-03-22 | 2017-07-07 | 仮想現実画像を送信するための方法および装置 |
EP17901675.3A EP3591502B1 (en) | 2017-03-22 | 2017-07-07 | Virtual reality image sending method and apparatus |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201710175242.8 | 2017-03-22 | ||
CN201710175242 | 2017-03-22 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2018171084A1 true WO2018171084A1 (zh) | 2018-09-27 |
Family
ID=63586109
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2017/092285 WO2018171084A1 (zh) | 2017-03-22 | 2017-07-07 | 一种虚拟现实图像发送方法及装置 |
Country Status (8)
Country | Link |
---|---|
US (1) | US11402894B2 (zh) |
EP (1) | EP3591502B1 (zh) |
JP (1) | JP6873268B2 (zh) |
KR (1) | KR102296139B1 (zh) |
CN (1) | CN109074152B (zh) |
AU (1) | AU2017405914B2 (zh) |
CA (1) | CA3057180C (zh) |
WO (1) | WO2018171084A1 (zh) |
Families Citing this family (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110391831B (zh) * | 2019-09-04 | 2021-09-14 | 联想(北京)有限公司 | 一种建立波束连接的方法、装置及电子设备 |
US20230120092A1 (en) * | 2020-03-06 | 2023-04-20 | Sony Group Corporation | Information processing device and information processing method |
US11652513B2 (en) * | 2021-08-17 | 2023-05-16 | Cisco Technology, Inc. | Wireless reliability for devices in motion |
CN113824746B (zh) * | 2021-11-25 | 2022-02-18 | 山东信息职业技术学院 | 一种虚拟现实信息传输方法及虚拟现实系统 |
US11870852B1 (en) * | 2023-03-31 | 2024-01-09 | Meta Platforms Technologies, Llc | Systems and methods for local data transmission |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103248883A (zh) * | 2012-02-13 | 2013-08-14 | 联想(北京)有限公司 | 传输方法和电子设备 |
CN105117111A (zh) * | 2015-09-23 | 2015-12-02 | 小米科技有限责任公司 | 虚拟现实交互画面的渲染方法和装置 |
US9460351B2 (en) * | 2012-11-28 | 2016-10-04 | Electronics And Telecommunications Research Institute | Image processing apparatus and method using smart glass |
CN106293043A (zh) * | 2015-06-29 | 2017-01-04 | 北京智谷睿拓技术服务有限公司 | 可视化内容传输控制方法、发送方法、及其装置 |
Family Cites Families (13)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP0973129A3 (en) * | 1998-07-17 | 2005-01-12 | Matsushita Electric Industrial Co., Ltd. | Motion image data compression system |
US6351499B1 (en) | 1999-12-15 | 2002-02-26 | Iospan Wireless, Inc. | Method and wireless systems using multiple antennas and adaptive control for maximizing a communication parameter |
KR100619031B1 (ko) | 2004-06-11 | 2006-08-31 | 삼성전자주식회사 | 부가 데이터의 인터랙티브한 이용방법 및 장치, 그에 따른수신장치 |
US8749556B2 (en) * | 2008-10-14 | 2014-06-10 | Mixamo, Inc. | Data compression for real-time streaming of deformable 3D models for 3D animation |
US9897805B2 (en) | 2013-06-07 | 2018-02-20 | Sony Interactive Entertainment Inc. | Image rendering responsive to user actions in head mounted display |
US10585472B2 (en) * | 2011-08-12 | 2020-03-10 | Sony Interactive Entertainment Inc. | Wireless head mounted display with differential rendering and sound localization |
KR101546654B1 (ko) * | 2012-06-26 | 2015-08-25 | 한국과학기술원 | 웨어러블 증강현실 환경에서 증강현실 서비스 제공 방법 및 장치 |
JP6397243B2 (ja) * | 2014-07-22 | 2018-09-26 | キヤノン株式会社 | 表示装置、制御方法、及びプログラム |
KR20160092136A (ko) * | 2015-01-26 | 2016-08-04 | 한국전자통신연구원 | 가상 데스크탑 제공 방법 및 가상 데스크탑 제공 서버 |
US9840003B2 (en) | 2015-06-24 | 2017-12-12 | Brain Corporation | Apparatus and methods for safe navigation of robotic devices |
WO2017044795A1 (en) | 2015-09-10 | 2017-03-16 | Google Inc. | Playing spherical video on a limited bandwidth connection |
US20170287097A1 (en) * | 2016-03-29 | 2017-10-05 | Ati Technologies Ulc | Hybrid client-server rendering in a virtual reality system |
US9965899B2 (en) * | 2016-04-28 | 2018-05-08 | Verizon Patent And Licensing Inc. | Methods and systems for minimizing pixel data transmission in a network-based virtual reality media delivery configuration |
-
2017
- 2017-07-07 CA CA3057180A patent/CA3057180C/en active Active
- 2017-07-07 CN CN201780012081.9A patent/CN109074152B/zh active Active
- 2017-07-07 EP EP17901675.3A patent/EP3591502B1/en active Active
- 2017-07-07 AU AU2017405914A patent/AU2017405914B2/en active Active
- 2017-07-07 KR KR1020197030873A patent/KR102296139B1/ko active IP Right Grant
- 2017-07-07 WO PCT/CN2017/092285 patent/WO2018171084A1/zh unknown
- 2017-07-07 JP JP2019552177A patent/JP6873268B2/ja active Active
- 2017-07-07 US US16/495,858 patent/US11402894B2/en active Active
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103248883A (zh) * | 2012-02-13 | 2013-08-14 | 联想(北京)有限公司 | 传输方法和电子设备 |
US9460351B2 (en) * | 2012-11-28 | 2016-10-04 | Electronics And Telecommunications Research Institute | Image processing apparatus and method using smart glass |
CN106293043A (zh) * | 2015-06-29 | 2017-01-04 | 北京智谷睿拓技术服务有限公司 | 可视化内容传输控制方法、发送方法、及其装置 |
CN105117111A (zh) * | 2015-09-23 | 2015-12-02 | 小米科技有限责任公司 | 虚拟现实交互画面的渲染方法和装置 |
Non-Patent Citations (1)
Title |
---|
See also references of EP3591502A4 |
Also Published As
Publication number | Publication date |
---|---|
KR20190132434A (ko) | 2019-11-27 |
JP2020520133A (ja) | 2020-07-02 |
AU2017405914A1 (en) | 2019-10-17 |
JP6873268B2 (ja) | 2021-05-19 |
EP3591502B1 (en) | 2024-03-13 |
KR102296139B1 (ko) | 2021-08-30 |
US20200064907A1 (en) | 2020-02-27 |
CN109074152A (zh) | 2018-12-21 |
CA3057180A1 (en) | 2018-09-27 |
CA3057180C (en) | 2022-06-28 |
US11402894B2 (en) | 2022-08-02 |
EP3591502A1 (en) | 2020-01-08 |
CN109074152B (zh) | 2021-01-05 |
EP3591502A4 (en) | 2020-05-06 |
AU2017405914B2 (en) | 2021-06-10 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10861215B2 (en) | Asynchronous time and space warp with determination of region of interest | |
WO2018171084A1 (zh) | 一种虚拟现实图像发送方法及装置 | |
US11050810B2 (en) | Method and apparatus for transmitting and receiving image data for virtual-reality streaming service | |
US10776992B2 (en) | Asynchronous time warp with depth data | |
CN111052750A (zh) | 用于点云流传输的方法和装置 | |
WO2020060984A1 (en) | Cross layer traffic optimization for split xr | |
US20160378177A1 (en) | Visualized content transmission control method, sending method and apparatuses thereof | |
US20210240257A1 (en) | Hiding latency in wireless virtual and augmented reality systems | |
EP2849080B1 (en) | Image display method and device | |
US20220301184A1 (en) | Accurate optical flow interpolation optimizing bi-directional consistency and temporal smoothness | |
US20160378178A1 (en) | Visualized content transmission control method, sending method and apparatuses thereof | |
CN106919376B (zh) | 动态画面传输方法、伺服器装置及用户装置 | |
US20230079171A1 (en) | Image content transmitting method and device using edge computing service | |
US20230206575A1 (en) | Rendering a virtual object in spatial alignment with a pose of an electronic device | |
US20230215075A1 (en) | Deferred rendering on extended reality (xr) devices | |
EP4202611A1 (en) | Rendering a virtual object in spatial alignment with a pose of an electronic device | |
KR20240083484A (ko) | Vr 모션 예측 장치 | |
KR20240083468A (ko) | Vr 모션 예측 장치 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 17901675 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 3057180 Country of ref document: CA |
|
ENP | Entry into the national phase |
Ref document number: 2019552177 Country of ref document: JP Kind code of ref document: A |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
ENP | Entry into the national phase |
Ref document number: 2017901675 Country of ref document: EP Effective date: 20191004 |
|
ENP | Entry into the national phase |
Ref document number: 2017405914 Country of ref document: AU Date of ref document: 20170707 Kind code of ref document: A |
|
ENP | Entry into the national phase |
Ref document number: 20197030873 Country of ref document: KR Kind code of ref document: A |