WO2023217198A1 - 用于在头戴显示设备显示画面的方法、装置及电子设备 - Google Patents

用于在头戴显示设备显示画面的方法、装置及电子设备 Download PDF

Info

Publication number
WO2023217198A1
WO2023217198A1 PCT/CN2023/093339 CN2023093339W WO2023217198A1 WO 2023217198 A1 WO2023217198 A1 WO 2023217198A1 CN 2023093339 W CN2023093339 W CN 2023093339W WO 2023217198 A1 WO2023217198 A1 WO 2023217198A1
Authority
WO
WIPO (PCT)
Prior art keywords
head
display device
mounted display
posture
attitude
Prior art date
Application number
PCT/CN2023/093339
Other languages
English (en)
French (fr)
Inventor
吕宪伟
Original Assignee
闪耀现实(无锡)科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 闪耀现实(无锡)科技有限公司 filed Critical 闪耀现实(无锡)科技有限公司
Publication of WO2023217198A1 publication Critical patent/WO2023217198A1/zh

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30—Image reproducers
    • H04N13/332—Displays for viewing with the aid of special glasses or head-mounted displays [HMD]
    • H04N13/344—Displays for viewing with the aid of special glasses or head-mounted displays [HMD] with head-mounted left-right displays
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/106—Processing image signals
    • H04N13/122—Improving the 3D impression of stereoscopic images by modifying image signal contents, e.g. by filtering or adding monoscopic depth cues
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30—Image reproducers
    • H04N13/398—Synchronisation thereof; Control thereof

Definitions

  • the present disclosure relates to the technical field of head-mounted display devices, and in particular, to a method, device and electronic device for displaying images on a head-mounted display device.
  • head-mounted display devices are increasingly used in applications. There may be bumps, shakes, etc. during use of the head-mounted display device, which will affect the display effect of the head-mounted display device. How to ensure the display effect of the head-mounted display device to improve the user experience is an urgent problem for those skilled in the art.
  • Embodiments of the present disclosure provide a method, device, and electronic device for displaying images on a head-mounted display device.
  • a method for displaying a picture on a head-mounted display device including: based on the angular velocity data of the head-mounted display device at the current moment, and for analyzing the head-mounted display device at the previous moment.
  • the first correction posture for processing the display screen determines the reference posture used for processing the display screen at the current moment; the posture error between the reference posture and the preset posture used for processing the display screen is processed to obtain Correct the posture difference; use the correction posture difference to compensate the reference posture to obtain the second correction posture used to process the picture to be displayed at the current moment; render the picture to be displayed on the head-mounted display device based on the second correction posture, To display the screen to be displayed on the head mounted display device.
  • a computer-readable storage medium stores a computer program.
  • the computer program is used to execute the above method for displaying a picture on a head-mounted display device.
  • an electronic device including: a processor; a memory for storing executable instructions by the processor; and a processor for reading executable instructions from the memory and executing the instructions to implement The above-mentioned method for displaying images on a head-mounted display device.
  • FIG. 1 is a schematic flowchart of a method for displaying a picture on a head-mounted display device according to an exemplary embodiment of the present disclosure.
  • FIG. 2 is a schematic diagram of a processing method for a display screen of a head-mounted display device in an embodiment of the present disclosure.
  • FIG. 3 is a schematic diagram of another processing method for the display screen of the head-mounted display device in an embodiment of the present disclosure.
  • Figure 4 is a schematic diagram of a scenario to which embodiments of the present disclosure are applicable.
  • FIG. 5 is a flowchart of a method for displaying a picture on a head-mounted display device according to another exemplary embodiment of the present disclosure. intention.
  • FIG. 6 is a schematic flowchart of determining the second correction posture in an embodiment of the present disclosure.
  • FIG. 7 is a schematic flowchart of a method for displaying a picture on a head-mounted display device according to yet another exemplary embodiment of the present disclosure.
  • FIG. 8 is a schematic flowchart of a method for displaying a picture on a head-mounted display device according to yet another exemplary embodiment of the present disclosure.
  • FIG. 9 is a schematic flowchart of a method for displaying a picture on a head-mounted display device according to yet another exemplary embodiment of the present disclosure.
  • FIG. 10 is a schematic flowchart of a method for displaying a picture on a head-mounted display device according to yet another exemplary embodiment of the present disclosure.
  • FIG. 11-1 is a schematic flowchart of a method for displaying a picture on a head-mounted display device according to another exemplary embodiment of the present disclosure.
  • FIG. 11-2 is a schematic flowchart of a method for displaying a picture on a head-mounted display device according to another exemplary embodiment of the present disclosure.
  • FIG. 11-3 is a schematic flowchart of a method for displaying a picture on a head-mounted display device according to another exemplary embodiment of the present disclosure.
  • FIG. 11-4 is a schematic flowchart of a method for displaying a picture on a head-mounted display device according to another exemplary embodiment of the present disclosure.
  • FIG. 12 is a schematic flowchart of a method for displaying a picture on a head-mounted display device according to yet another exemplary embodiment of the present disclosure.
  • FIG. 13 is a schematic structural diagram of a device for displaying images on a head-mounted display device according to an exemplary embodiment of the present disclosure.
  • FIG. 14 is a schematic structural diagram of an apparatus for displaying a picture on a head-mounted display device according to another exemplary embodiment of the present disclosure.
  • FIG. 15 is a schematic structural diagram of an apparatus for displaying a picture on a head-mounted display device according to yet another exemplary embodiment of the present disclosure.
  • FIG. 16 is a schematic structural diagram of a device for displaying images on a head-mounted display device according to yet another exemplary embodiment of the present disclosure.
  • FIG. 17 is a schematic structural diagram of an apparatus for displaying images on a head-mounted display device according to yet another exemplary embodiment of the present disclosure.
  • FIG. 18 is a schematic structural diagram of an apparatus for displaying a picture on a head-mounted display device according to yet another exemplary embodiment of the present disclosure.
  • FIG. 19 is a structural diagram of an electronic device provided by an exemplary embodiment of the present disclosure.
  • plural may refer to two or more than two, and “at least one” may refer to one, two, or more than two.
  • Embodiments of the present disclosure may be applied to electronic devices such as terminal devices, computer systems, servers, etc., which may operate with numerous other general or special purpose computing system environments or configurations.
  • Examples of well-known terminal devices, computing systems, environments and/or configurations suitable for use with terminal devices, computer systems, servers and other electronic devices include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients Computers, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, networked personal computers, small computer systems, mainframe computer systems and distributed cloud computing technology environments including any of the above systems, etc.
  • Electronic devices such as terminal devices, computer systems, servers, etc. may be described in the general context of computer system executable instructions (such as program modules) being executed by the computer system.
  • program modules may include routines, programs, object programs, components, logic, data structures, etc., that perform specific tasks or implement specific abstract data types.
  • the computer system/server may be implemented in a distributed cloud computing environment where tasks are performed by remote processing devices linked through a communications network.
  • program modules may be located on local or remote computing system storage media including storage devices.
  • FIG. 1 is a schematic flowchart of a method for displaying a picture on a head-mounted display device according to an exemplary embodiment of the present disclosure.
  • the method shown in Figure 1 may include step 110, step 120, step 130 and step 140. Each step will be described separately below.
  • Step 110 Based on the angular velocity data of the head-mounted display device at the current moment and the first correction posture used to process the display screen of the head-mounted display device at the previous moment, determine the current moment for processing the display screen to be displayed. Reference posture.
  • the head-mounted display device can also be called a head-mounted display (Head-Mounted Display, HMD) or head-mounted display.
  • Head-mounted display devices can be used to achieve extended reality (XR) effects, such as augmented reality (Augmented Reality, AR) effects, virtual reality (VR) effects, and mixed reality (Mixed Reality, MR) effects. Effects etc.
  • XR extended reality
  • AR Augmented Reality
  • VR virtual reality
  • MR mixed reality
  • a head-mounted display device can be used to achieve an AR effect.
  • the head-mounted display device can be AR glasses.
  • a head-mounted display device may include an inertial measurement unit (Inertial Measurement Unit, IMU) and an optical machine.
  • IMUs can include accelerometers, gyroscopes, magnetometers, etc.
  • the accelerometer is used to collect acceleration data
  • the gyroscope is used to collect angular velocity data.
  • the optical engine is the imaging system of the head-mounted display device, and the optical engine can include a display screen and optical components.
  • the display screen is provided to the user through the following solution.
  • the display screen is used to emit light to display the picture, and the optical element can process the above-mentioned light, so that the light of the display picture is projected into the user's eyes, and finally the user can see the display picture. Since when the user wears the head-mounted display device, there is almost no relative displacement or rotation between the head-mounted display device and the user, and the head-mounted display device will move along with the user's movements.
  • the display screen may be necessary to process the display screen based on the position and posture of the head-mounted display device (which can actually be regarded as the user), so that the display screen can always be positioned at a certain position or within a certain area. , or can move with the user's movements, etc., to match the corresponding application scenarios or user needs.
  • the angular velocity data of the IMU at the current moment can be obtained, such as the angular velocity data collected by the gyroscope in the IMU, and the first correction attitude used to process the display screen of the head-mounted display device at the previous moment can be obtained.
  • the first correction posture can be obtained in a manner similar to the second correction posture below. For clarity of layout, the method of obtaining the first correction posture will not be described here.
  • step 110 you can refer to the integration method from the gyroscope angular velocity to the angular position, and use the angular velocity data at the current moment and the first correction posture to perform integration processing to obtain the picture to be displayed at the current moment (the picture to be displayed can be The reference posture for processing the picture to be displayed on the head-mounted display device).
  • the angular velocity data at the current moment is expressed as Gyro k
  • the first correction attitude is expressed as Pose k-1
  • the reference attitude is expressed as Then there are:
  • Step 120 Process the posture error between the reference posture and the preset posture used for processing the display screen to obtain a corrected posture difference.
  • the preset posture may be a posture that is preset for the head-mounted display device and can ensure the display effect of the head-mounted display device. That is to say, the preset posture may be a posture used to process the display screen in order to ensure the display effect.
  • the preset posture is a posture used to cause the display screen of the head-mounted display device to be displayed at a preset position in the coordinate system of the head-mounted display device. It is understandable that it can also be a preset position in the IMU coordinate system.
  • the preset position in the coordinate system of the head-mounted display device can be a fixed position in the coordinate system of the head-mounted display device, for example, the display screen is displayed directly in front, upper left, or right of the head-mounted display device. Below and so on, the position can be observed by the user. It should be noted that the usage habit of most users is to display the display screen in the center in front of them. In embodiments of the present disclosure, the display effect of the head-mounted display device can be made to meet the needs of most users as much as possible by setting the preset posture to a posture that matches the usage habits of most users, which is conducive to improving the user experience. .
  • the preset posture is a posture used to align the display screen of the head-mounted display device with the direction of gravity (in this posture, the heading angle (Yaw) of the head-mounted display device may be 0).
  • the alignment of the display screen of the head-mounted display device with the direction of gravity can be understood as meaning that when the user observes the display screen through the head-mounted display device, the extension direction of the plane of the display screen is the same as the direction of gravity in the real world. This can keep the display screen basically It is perpendicular to the user's line of sight and conforms to viewing habits. It should be noted that the usage habit of most users is to align the display screen of the head-mounted display device with the direction of gravity. In embodiments of the present disclosure, by setting the preset posture to a posture that matches the usage habits of most users, the display effect of the head-mounted display device can meet the needs of most users as much as possible, which is conducive to improving the user experience. .
  • the user can also set a preset posture that enables the display screen of the head-mounted display device to be displayed at a customized position according to needs.
  • a difference between the reference posture and the preset posture may be made to obtain the posture error between the reference posture and the preset posture.
  • the attitude error can be in the form of a 3 ⁇ 1 axis angle. It should be noted that the difference here is not a simple subtraction, but a generalized function.
  • the "difference" in the difference refers to the difference. The difference can refer to finding the phase between two angles. pair rotation. Assume that the reference attitude is expressed as The default attitude is represented by Target, and the attitude error is represented by error, then there are:
  • the attitude error After the attitude error is obtained, the attitude error can be corrected by processing the attitude error, thereby obtaining the corrected attitude difference.
  • the preset posture may be a posture preset for the head-mounted display device.
  • the preset posture and the reference posture are in the same coordinate system, and the preset posture is a zero-rotation posture, in order to save the program, there is no need to make a difference between the reference posture and the preset posture, and the reference posture is directly used as The subsequent volume to be processed. That is, in subsequent steps, the reference posture is directly processed to correct the posture error, thereby obtaining the corrected posture difference.
  • Step 130 Use the correction posture difference to compensate the reference posture to obtain the second correction posture used for processing the to-be-displayed picture at the current moment.
  • the correction posture difference value can be updated into the reference posture as a compensation value to obtain the second correction posture used for processing the to-be-displayed picture at the current moment.
  • the second corrected posture is expressed as Pose k , then there is:
  • the compensation value here can represent a relative rotation
  • updating the compensation value to the reference posture can be a relative rotation superimposed on the reference posture to obtain another posture (i.e., the second corrected posture ).
  • the corrected posture difference is used to update the reference posture to obtain the second corrected posture. Regardless of whether the corrected posture difference is obtained by processing the reference posture or by processing the posture error between the reference posture and the preset posture, the second corrected posture is updated based on the corrected posture difference.
  • Step 140 Render the image to be displayed on the head-mounted display device based on the second correction posture to display the image to be displayed on the head-mounted display device.
  • both the first correction posture and the second correction posture are postures used to process the display screen of the head-mounted display device, and can also be called correction postures.
  • the correction posture at the previous moment can also be the real posture of the head-mounted display device or the preset for processing the display screen. Set posture, or other preset postures, etc.
  • the rendering engine of the head-mounted display device can use the second correction posture or the correction posture to render the picture to obtain the corresponding display picture, and the obtained display picture can be updated to the head-mounted display device.
  • this implementation uses software to process the display screen of the head-mounted display device. For example, this solution can adjust the overall parameters of the picture to be displayed based on the second correction posture or the correction posture, and the overall parameters can include the three-dimensional size of the display picture, etc.
  • picture rendering can be performed based on the second correction posture (such as Pose k in Figure 2) or the correction posture, and the light of the display picture on the head-mounted display device can be adjusted.
  • the display position in the display screen in the machine (such as the Display in Figure 2) is used to correct the display screen of the head-mounted display device.
  • This implementation form is used to process the display screen of the head-mounted display device.
  • the display screen of the head-mounted display device can adjust the display position up, down, left, and right, etc.
  • this solution can adjust the two-dimensional size of the picture to be displayed based on the second correction posture or the correction posture.
  • picture rendering can be performed based on the second correction posture or the correction posture, and the optical engine in the head-mounted display device can be configured.
  • the overall position of the optical engine in the head-mounted display module can be adjusted through the electronic control chassis or the position of the display screen in the optical engine can be adjusted to achieve correction of the display screen of the head-mounted display device. (This is equivalent to holding the display in front of the user's eyes and making corresponding shock-absorbing adjustments). It is easy to see that this implementation uses hardware to process the display screen of the head-mounted display device.
  • the execution logic of this implementation can be: second correction posture ⁇ electronic control chassis ⁇ optical-mechanical rotation.
  • the angular velocity data of the head-mounted display device at the current moment and the first correction posture used to process the display screen of the head-mounted display device at the previous moment it can be determined that the display image to be displayed at the current moment is The reference posture for processing the picture, and then the posture error between the reference posture and the preset posture used for processing the display picture can be processed, or the reference posture can be processed to obtain the corrected posture difference, and the correction can be used
  • the posture difference compensates the reference posture to obtain the second corrected posture used for processing the to-be-displayed picture at the current moment.
  • the second correction posture can be considered to be determined by referring to the angular velocity data of the head-mounted display device at the current moment, the first correction posture and the preset posture, and is adapted to the head-mounted display device (actually, it can also be regarded as the user) at the current moment.
  • the posture based on which the image to be displayed on the head-mounted display device is rendered, is conducive to correcting the display effect of the head-mounted display device according to the display effect of the head-mounted display device in the preset posture.
  • the head-mounted display device adjusts the display effect correspondingly to the preset posture, thereby improving the user experience.
  • each posture involved in the embodiment of the present disclosure can be in the form of a rotation matrix (Rotation matrix); each correction posture involved in the embodiment of the present disclosure can be used for the rendering engine of the head-mounted display device.
  • the gesture of screen rendering can be used for the rendering engine of the head-mounted display device.
  • the head-mounted display device when presenting content, there are currently multiple modes, which may include a first mode and a second mode.
  • the first mode may be a 3-degree of freedom (dof) mode.
  • the head-mounted display device performs positioning processing on the display screen.
  • the content of the display screen of the head-mounted display device is consistent with the content of the display screen.
  • the display screen of the head-mounted display device can remain rotated with the world coordinate system.
  • the second mode is similar to the screen projection mode from a mobile phone to a TV. In the second mode, the head-mounted display device does not perform positioning processing on the display screen.
  • the content of the display screen of the head-mounted display device can be related to the content playback progress.
  • the display screen of the head-mounted display device moves with the movement of the user's head.
  • the second mode may be called a 0 degree of freedom (dof) mode.
  • 0dof mode if the mobile platform runs smoothly, the display effect of the head-mounted display device is acceptable. Once bumps and shakes occur, for example, due to uneven road surfaces, a moving vehicle bumps and shakes, and for example, due to the influence of airflow, an aircraft in progress bumps and shakes, causing users to shake or shake while wearing a head-mounted display device. Shake etc. In this 0dof mode, because the display screen will also shake or shake, in this case, the user will not be able to observe the content of the display screen of the head-mounted display device stably and clearly, which will reduce the user experience.
  • the display effect of the head-mounted display device can be corrected based on the display effect of the head-mounted display device in the preset posture.
  • the display screen of the head-mounted display device can be processed based on the correction posture so that when the head-mounted display device is used with small angle changes, the display effect is similar to 3dof mode, and when the head-mounted display device is used at large angle changes, the display effect is similar to 0dof mode, thus improving the user experience.
  • the head mounted display device is configured to be coupled to a mobile platform and moveable relative to the mobile platform. move.
  • the head-mounted display device can be coupled to the mobile platform through the user, and the mobile platform can be in motion.
  • the head-mounted display device can be worn on the head of a user (such as a passenger in a vehicle), and the head-mounted display device and the user remain relatively stationary.
  • the user can sit or stand inside the vehicle, and when the vehicle moves, the user will move with the vehicle.
  • the user can also move within the vehicle (for example, from one location to another, or from one direction to another, etc.).
  • the mobile platform is of other types, such as a boat, the user can also sit or stand on the side of the boat.
  • the head-mounted display device is configured to be coupled to the mobile platform and movable relative to the mobile platform
  • the mobile platform may bump or shake during movement, or the mobile platform may move during movement.
  • the method steps of the present disclosure can be executed for this scenario to ensure the display effect of the head-mounted display device and improve the user experience.
  • step 120 includes:
  • Step 1201 Perform low-pass filtering (Low Pass Filter) processing on the attitude error corresponding to the current moment to obtain the corrected attitude difference.
  • Low Pass Filter Low Pass Filter
  • the attitude error obtained by difference between the reference attitude determined in step 110 and the preset attitude is the attitude error corresponding to the current moment.
  • the preset posture and the reference posture are postures in the same coordinate system, and the preset posture is a zero-rotation posture, low-pass filtering can be performed on the reference posture to obtain the corrected posture difference.
  • the attitude error corresponding to the current moment can be low-pass filtered through a low-pass filter such as an Infinite Impulse Response (IIR) digital filter or a Finite Impulse Response (FIR) digital filter. deal with.
  • IIR Infinite Impulse Response
  • FIR Finite Impulse Response
  • Each output value of the IIR digital filter can only rely on the previous sample value, so engineering applications are very convenient.
  • the low-pass filtering process of the FIR digital filter involves a sliding window.
  • the value of the corrected attitude difference error 1 is not 0, that is, the second corrected attitude after using the corrected attitude difference error 1 to compensate the reference attitude. Different from the reference pose. If the data higher than the cutoff frequency cannot pass, the value of the corrected attitude difference error 1 is 0, that is, the second corrected attitude after subsequently using the corrected attitude difference error 1 to compensate the reference attitude is the same as the reference attitude. However, under actual working conditions, there may be very few cases where the value of the corrected attitude difference error 1 is 0, and it may be such a small value that it cannot be noticed by the user.
  • the angular velocity data Gyro k collected by the gyroscope in the IMU at the current moment after obtaining the angular velocity data Gyro k collected by the gyroscope in the IMU at the current moment, the angular velocity data Gyro k at the current moment and the angular velocity data Gyro k collected at the previous moment can be used for head-mounted
  • the first corrected posture Pose k-1 for processing the display screen of the display device is integrated to obtain the reference posture used for processing the display screen at the current moment.
  • the attitude error error corresponding to the current moment can be obtained.
  • the corrected attitude difference error 1 can be obtained.
  • Pose k can be considered as a posture that can achieve anti-shake, and Pose k can be used to process the screen to be displayed at the current moment. It should be noted that what is shown in Figure 6 is the method of obtaining Pose k by taking Pose k-1 as input. A similar method can be used later to convert Pose k As input, we get Pose k+1 , and by taking Pose k+1 as input, we get Pose k+2 , etc.
  • the display screen of the head-mounted display device will tend to remain motionless, thus facilitating Users can clearly and stably observe the content of the display screen of the head-mounted display device, which can improve the user experience.
  • low-frequency movements (such as long-term large-angle movements) will be retained. If the user turns his head or the mobile platform turns (for example, the mobile platform turns and the head follows the turn), the head-mounted display device will move at a large angle. After compensation, the display screen of the head-mounted display device will be positioned following the user's head, so that the user can observe the required display screen, which can also improve the user experience.
  • step 1201 includes:
  • Step 12011 Perform low-pass filtering processing on the attitude error corresponding to the current moment through a low-pass filter.
  • the method also includes steps 150 and 160.
  • Step 150 Receive user input operation.
  • user input operations include but are not limited to voice input operations, touch input operations, etc.
  • the user input operation may be an operation in which the user's voice says "Adjust the cutoff frequency to 2hz (it can also be 5hz or other frequencies)."
  • the head-mounted display device may have a paired handle, and the user input operation may be an operation of the user clicking a cutoff frequency adjustment button on the handle.
  • a cutoff frequency selection box or input box can also be displayed on the display screen, and the user can directly select or input the required cutoff frequency.
  • Step 160 Adjust the cutoff frequency of the low-pass filter in response to user input operation.
  • step 160 the cutoff frequency required by the user can be determined according to the user input operation, and then the cutoff frequency of the low-pass filter is adjusted to the cutoff frequency required by the user in any implementable manner.
  • the display screen of the head-mounted display device follows the head-mounted display
  • the display screen of the head-mounted display device tends to remain stationary when the motion frequency of the head-mounted display device is higher than the cutoff frequency of the low-pass filter. It is understandable that this method may be less sensitive to motion closer to the cut-off frequency. In this way, the higher the motion frequency of the head-mounted display device is than the cut-off frequency, the closer the actual mode of the head-mounted display device is to the 3dof mode.
  • the headset's motion frequency is below the cutoff frequency
  • this solution will perform most movements of the head-mounted display device so that the display screen follows the movement of the head-mounted display device.
  • the compensation processing the closer the actual mode of the head-mounted display device is to the 0dof mode.
  • the lower the cut-off frequency the frequency of most movements may be higher than the cut-off frequency. Therefore, this solution will perform compensation processing on most movements of the head-mounted display device to make the display screen tend to remain motionless.
  • the actual mode is closer to the 3dof mode.
  • the user can make the actual mode of the head-mounted display device closer to the 0dof mode or closer to the 3dof mode according to actual needs.
  • the cutoff frequency of the low-pass filter is adjustable.
  • the cutoff frequency of the low-pass filter can also be set to a non-adjustable fixed value.
  • step 120 includes:
  • Step 1203 Use the correction proportional coefficient to correct the attitude error corresponding to the current moment to obtain the corrected attitude difference.
  • the correction proportional coefficient is greater than 0 and less than 1.
  • the correction proportional coefficient may be 0.2, 0.5, 0.6, 0.8 or other values between 0 and 1, which are not listed here.
  • the correction proportion coefficient can be used to correct the reference posture to obtain the corrected posture difference.
  • step 1203 the attitude error corresponding to the current moment may be multiplied by the correction proportion coefficient, and the multiplication result may be used as the corrected attitude difference.
  • the display screen of the head-mounted display device will tend to remain motionless, thereby making it easier for the user to clearly and stably observe the content of the display screen of the head-mounted display device, which can improve the user experience.
  • the actual posture of the head-mounted display device is always positive or negative, and the posture error is always positive or negative. burden.
  • the display screen of the head-mounted display device will follow the direction of the user's head, so that the user can observe the required display screen, which can also improve the user experience.
  • the corrected posture difference is used to determine the second corrected posture, the second corrected posture will be closer to the preset posture.
  • the correction speed of the display screen of the head-mounted display device will be faster.
  • the larger the correction proportion coefficient is the closer the actual mode of the head-mounted display device is to the 0dof mode.
  • the corrected posture of the head-mounted display device can be close to the preset posture, and the corrected posture is used for the display screen rendering of the head-mounted display device, which is beneficial to the head-mounted display device.
  • the display effect of the head-mounted display device is corrected based on the display effect of the head-mounted display device in the preset posture. In this way, whether there are small angle changes during the use of the head-mounted display device (bumps, shaking, jitter, etc.), or large-angle changes during the use of the head-mounted display device (such as large-angle turns), it can be Ensure that the head-mounted display device adjusts the display effect correspondingly to the preset posture, thereby improving the user experience.
  • users can make the actual mode of the head-mounted display device closer to the 0dof mode or closer to the 3dof mode according to actual needs.
  • the method further includes:
  • Step 115 Determine the correction proportion coefficient based on the compensation reference information.
  • the compensation reference information includes at least one of the following: the posture error corresponding to the current moment, the cumulative value of the posture error, the environmental conditions of the mobile platform where the head-mounted display device is located, and the movement of the head-mounted display device. Platform redirection information.
  • step 115 may be performed before step 120, that is, after determining the correction proportional coefficient based on the compensation reference information, the determined correction proportional coefficient is then used to correct the attitude error.
  • step 115 may be performed after step 120, that is, after correcting the attitude error using the correction proportional coefficient, the correction proportional coefficient may be determined based on the compensation reference information. In order to use the determination results to dynamically update the correction proportion coefficient. In this way, the updated correction proportion coefficient can subsequently be used to correct the attitude error.
  • the embodiments of the present disclosure do not place any restrictions on the execution order of step 115 and step 120. Therefore, it can be understood that the correction proportional coefficient in the embodiment of the present disclosure may be dynamically adjusted based on the above factors.
  • the posture error corresponding to the current moment and the cumulative value of the posture error can be understood as: the corresponding posture error at the current moment The reference attitude and the accumulated value of the reference attitude.
  • the environmental conditions of the mobile platform where the head-mounted display device is located can be determined based on information collected by devices installed on the platform (such as cameras, radars, etc.).
  • the environmental state of the mobile platform on which the head-mounted display device is located may include road surface information of the road where the vehicle is located, and the road surface information may include road surface smoothness.
  • the environmental state of the mobile platform on which the head-mounted display device is located may include the airflow intensity at the location of the airplane.
  • the steering information of the mobile platform where the head-mounted display device is located may include the steering degree, and the steering degree may be characterized by the steering angle.
  • the steering information can be detected by the vehicle-mounted steering system.
  • the correction proportion coefficient is positively correlated with the attitude error corresponding to the current moment.
  • the correction proportional coefficient is also larger.
  • a larger correction proportional coefficient is conducive to more quickly correcting the display screen of the head-mounted display device according to the preset posture, thereby having the advantage of It is beneficial to ensure the display effect of the head-mounted display device.
  • the correction proportional coefficient is positively correlated with the cumulative value of the attitude error.
  • the accumulated value of the attitude error may be the accumulated value of the attitude error within a preset time period, or the accumulated value of the continuous attitude error.
  • the correction proportion coefficient is determined by the overall motion trend before the screen is displayed, which can make the correction proportion coefficient more closely match the actual attitude error. For example, when the mobile platform makes a large-angle turn, the display screen of the head-mounted display device can be quickly corrected according to the preset posture, thereby ensuring the display effect of the head-mounted display device.
  • the value of the attitude error can be a positive number or a negative number.
  • the value of the attitude error toward the first direction is a positive number
  • the value of the attitude error toward the second direction opposite to the first direction is a negative number. Both positive attitude errors and negative attitude errors can be processed.
  • the absolute value of the attitude error can be used for evaluation.
  • the environmental conditions include road surface smoothness
  • the correction proportional coefficient is positively related to the road surface smoothness.
  • the more uneven the road surface the smaller the correction proportional coefficient.
  • the smaller correction proportional coefficient makes the display screen of the head-mounted display device correct more slowly according to the preset posture, which is conducive to ensuring the display screen of the head-mounted display device. stability.
  • the steering information includes a steering degree
  • the correction proportional coefficient is positively correlated with the steering degree.
  • the greater the change in the direction of the mobile platform the greater the correction proportion coefficient.
  • a larger correction proportion coefficient is conducive to more quickly correcting the display screen of the head-mounted display device according to the preset posture, thereby helping to ensure the display screen of the head-mounted display device.
  • the correction proportion coefficient can be reasonably determined with reference to at least one of the posture error corresponding to the current moment, the environmental conditions of the mobile platform where the head-mounted display device is located, and the steering information of the mobile platform, so that the determination The correction proportion coefficient is adapted to the actual situation.
  • the user can also adjust the above correction proportional coefficient according to the usage requirements.
  • the method also includes step 180 and step 190.
  • Step 180 Determine the change pattern of the attitude error based on the attitude error sequence.
  • the attitude error sequence includes attitude errors corresponding to multiple times.
  • the posture error corresponding to the current moment can be determined by making a difference between the reference posture corresponding to the current moment and the preset posture.
  • the corresponding attitude error can be determined for each moment, so that multiple attitude errors corresponding to multiple moments can be obtained.
  • Multiple attitude errors can be arranged in order from earliest to latest corresponding time to form an attitude error sequence.
  • the posture error change rule can be determined based on the reference posture sequence.
  • the attitude error sequence may include 20 attitude errors arranged in sequence, namely R1, R2, R3, ..., R20.
  • the time corresponding to R1 may be t1, and the time corresponding to R2 may be t2.
  • the time corresponding to R3 can be t3,...
  • the time corresponding to R20 can be t20.
  • t20 can be the current time.
  • the change rules of attitude error include but are not limited to the following information: whether the overall change trend of attitude error is increasing or decreasing; which moment the maximum and minimum values of attitude error correspond to; how many maximum values and how many maximum values of attitude error there are Minimum value; each maximum value and each minimum value correspond to which moment; the size between the attitude error corresponding to each moment after a certain moment (which can be considered as the base moment) and the attitude error corresponding to the base moment relation.
  • Step 190 Adjust the correction proportion coefficient based on the change pattern of the attitude error.
  • step 190 can be implemented in various forms, and examples are given below.
  • step 190 includes:
  • Step 1901 If it is determined based on the attitude error change law that the attitude error corresponding to each moment in the target time period is greater than or equal to the attitude error corresponding to the target time, increase the correction proportion coefficient.
  • the target time, the starting time of the target time period, and the end time of the target time period are each one of multiple moments.
  • the starting time of the target time period is later than the target time, and the duration of the target time period is greater than or equal to the preset time. duration.
  • the preset time length may be 3 seconds, 4 seconds, 5 seconds, 8 seconds or other time lengths, which are not listed here.
  • the attitude error sequence may include 20 attitude errors arranged in sequence, namely R1, R2, R3, ..., R20.
  • the time corresponding to R1 may be t1, and the time corresponding to R2 may be t2.
  • the time corresponding to R3 can be t3,...
  • the time corresponding to R20 can be t20.
  • t20 can be the current time.
  • the attitude error change law may include the size relationship between each of R14 to R20 and R13. Assume that the interval between any two adjacent moments is 0.5 seconds, the preset duration is 3 seconds, the starting moment of the target time period is t14, and the end moment of the target time period is t20, then each of R14 to R20 is greater than or When equal to R13, it can be considered that the attitude error continues to be high from the moment t13. That is, when the attitude error is corrected based on the current correction proportion coefficient, the correction effect is not ideal. In this case, the correction proportion coefficient can be increased, for example, the correction proportion coefficient can be adjusted from 0.5 to 0.6 or 0.7. By increasing the correction proportion coefficient, the display screen of the head-mounted display device can be corrected more quickly according to the preset posture, thereby ensuring the display effect of the head-mounted display device.
  • step 190 includes:
  • Step 1903 If it is determined based on the attitude error change law that the attitude error corresponding to each moment in the target time period is smaller than the attitude error corresponding to the target time, reduce the correction proportion coefficient.
  • the target time, the starting time of the target time period, and the end time of the target time period are each one of multiple moments.
  • the starting time of the target time period is later than the target time, and the duration of the target time period is greater than or equal to the preset time. duration.
  • the preset time length may be 3 seconds, 4 seconds, 5 seconds, 8 seconds or other time lengths, which are not listed here.
  • the correction proportion coefficient can be reduced, for example, the correction proportion coefficient can be adjusted from 0.5 to 0.3 or 0.4.
  • step 190 includes:
  • Step 1905 If it is determined that the change trend of the attitude error is an increasing trend based on the change law of the attitude error, increase the correction proportion coefficient.
  • the correction proportion coefficient can be increased.
  • the correction proportion coefficient can be adjusted from 0.5 to 0.6 or 0.7.
  • step 190 includes:
  • Step 1907 If it is determined that the change trend of the attitude error is a decreasing trend based on the change law of the attitude error, reduce the correction proportion coefficient.
  • the change trend of attitude error is a decreasing trend. For example, continuing the example in the first implementation form, if R1, R2, R3,..., R20 decrease in sequence, or although R1, R2 , R3,..., R20 do not decrease in sequence, but the overall trend is to gradually decrease (for example, the first 10 attitude errors among the 20 attitude errors decrease in sequence, and the 11th attitude error is the same as the 12th attitude error. The last 8 attitude errors decrease sequentially), then
  • the correction scaling factor can be reduced.
  • the correction proportion coefficient can be adjusted from 0.5 to 0.3 or 0.4. By reducing the correction proportion coefficient, the display screen of the head-mounted display device can be corrected more slowly according to the preset posture, thereby helping to ensure the stability of the display screen of the head-mounted display device.
  • the display screen of the head-mounted display device can be corrected at a more appropriate speed according to the actual situation, thereby ensuring that the head-mounted display device The display effect and display screen stability.
  • the method also includes step 102, step 104 and step 106.
  • Step 102 Identify the type of platform on which the head-mounted display device is located.
  • the head-mounted display device can establish a communication connection with the platform where it is located, and in the process of establishing the communication connection, relevant information of the platform can be obtained.
  • Platform-related information includes but is not limited to identification information, type information, etc.
  • the head-mounted display device can also include a camera.
  • the head-mounted display device can collect images of its own environment through the camera and perform target detection on the images to determine whether the images match mobile platforms such as vehicles. . In the case of matching, it can be determined that the type of the platform on which the head-mounted display device is located is a mobile platform; otherwise, it can be determined that the type of the platform on which the head-mounted display device is located is not a mobile platform.
  • the acceleration data collected by the accelerometer or the angular velocity data collected by the gyroscope in the IMU of the head-mounted display device can be obtained and compared with a preset threshold. If it is determined based on the acceleration data that the acceleration of the head mounted display device is greater than the set acceleration threshold, or if it is determined based on the angular velocity data that the angular velocity of the head mounted display device is greater than the set angular velocity threshold, then it can be determined that the type of platform on which the head mounted display device is located is: mobile platform. Otherwise, it can be determined that the type of platform on which the head-mounted display device is located is not a mobile platform.
  • Step 104 When the type of the platform on which the head-mounted display device is located is a mobile platform, control the head-mounted display device to switch to the target mode.
  • the head-mounted display device may include at least three modes, and the at least three modes may include the above 0dof mode, 3dof mode, and the target mode in step 104.
  • Step 106 Determine whether the head-mounted display device is in the target mode. If so, trigger step 110.
  • the type of platform on which the head-mounted display device is located can be identified. If the platform type is not a mobile platform, the head-mounted display device can be placed in a conventional 0dof mode or 3dof mode. When the type of the platform is a mobile platform, the head-mounted display device can be automatically switched to the target mode, so that step 110 is triggered in the target mode to implement the execution of steps 110 to 140 above.
  • the target mode is a mode between the 0dof mode and the 3dof mode. In target mode, when the mobile platform bumps and shakes, by adjusting the display screen of the head-mounted display device, the bumps and shakes can be compensated to keep the screen stable. Moreover, when the mobile platform or the user's head turns, the display screen of the head-mounted display device can be kept positioned following the user's head, thereby improving the user's experience.
  • the mobile platform involved in the embodiments of the present disclosure can also be called a moving vehicle; the angular velocity data in the embodiments of the present disclosure may not be collected by the gyroscope in the IMU, but by other devices that can obtain the angular velocity data. Sensor collection.
  • the display effect of the head-mounted display device can always be based on the preset posture of the head-mounted display device.
  • the display effect can be guaranteed whether it is a short-term small-angle movement or a long-term large-angle movement.
  • Any method for displaying images on a head-mounted display device provided by embodiments of the present disclosure can be executed by any appropriate device with data processing capabilities, including but not limited to: terminal devices and servers.
  • any of the methods for displaying images on a head-mounted display device provided by the embodiments of the present disclosure can be executed by the processor.
  • the processor executes any of the methods mentioned in the embodiments of the present disclosure by calling corresponding instructions stored in the memory.
  • a method for displaying images on a head-mounted display device No further details will be given below.
  • Embodiments of the present disclosure also provide a method for displaying images on a head-mounted display device.
  • the method includes: obtaining angular velocity data of the head-mounted display device;
  • the first corrected posture for processing the display screen of the display device determines the reference posture used for processing the to-be-displayed picture; processes the reference posture to obtain a corrected posture difference; and uses the corrected posture difference to compensate the Reference posture to obtain a second corrected posture for processing the picture to be displayed; based on the second corrected posture, render the picture to be displayed of the head-mounted display device to display the picture to be displayed of the head-mounted display device display screen.
  • Embodiments of the present disclosure also provide a method for displaying a picture on a head-mounted display device.
  • the method includes: obtaining angular velocity data of the head-mounted display device, and based on the angular velocity data and the previous step used to calculate the head-mounted display device.
  • the first correction posture for processing the display screen of the display device determines the reference posture for processing the display screen; processing the posture error between the reference posture and the preset posture for processing the display screen, to obtain a corrected posture difference; use the corrected posture difference to compensate the reference posture to obtain a second corrected posture for processing the to-be-displayed picture; based on the second corrected posture, perform the head-mounted display device
  • the picture to be displayed is rendered to display the picture to be displayed of the head mounted display device.
  • Embodiments of the present disclosure also provide a method for displaying a picture on a head-mounted display device.
  • the method includes: based on the angular velocity data of the head-mounted display device at the current moment, and at the previous moment, used to display the image on the head-mounted display device.
  • the first correction posture for processing the display screen of the head-mounted display device determines the reference posture used for processing the display screen at the current moment; processing the reference posture to obtain the correction posture difference; using the correction posture
  • the difference value compensates the reference posture to obtain the second corrected posture used to process the picture to be displayed at the current moment; based on the second corrected posture, renders the picture to be displayed on the head-mounted display device to display The screen to be displayed on the head-mounted display device.
  • the head-mounted display device display mode is switched to one of a 0-DOF mode, a target mode, and a 3-DOF mode.
  • the head-mounted display device is in the target mode, perform based on the angular velocity data of the head-mounted display device at the current moment and the first correction posture used to process the display screen of the head-mounted display device at the previous moment, and determine the The current moment is used to process the reference posture of the image to be displayed.
  • the method for displaying a picture on a head-mounted display device further includes: receiving a user input operation; and adjusting the above-mentioned correction scale coefficient in response to the user input operation.
  • the method for displaying a picture on a head-mounted display device further includes: after using the correction scale coefficient to correct the posture error, the correction scale coefficient can be determined based on the compensation reference information, so as to use the determined As a result, dynamic updating of the correction proportional coefficient is performed.
  • Figure 13 is a schematic structural diagram of a device for displaying images on a head-mounted display device according to an exemplary embodiment of the present disclosure.
  • the device shown in Figure 13 includes a first determination module 1310, a first processing module 1320, and a compensation module. 1330 and the second processing module 1340.
  • the first determination module 1310 is configured to determine, based on the angular velocity data of the head-mounted display device at the current moment and the first correction posture used to process the display screen of the head-mounted display device at the previous moment, the method used to treat the head-mounted display device at the current moment. Display the reference posture for processing the screen;
  • the first processing module 1320 is used to process the posture error between the reference posture and the preset posture used to process the display screen to obtain the corrected posture difference;
  • the compensation module 1330 is used to compensate the reference posture using the correction posture difference to obtain the second correction posture used for processing the to-be-displayed picture at the current moment;
  • the second processing module 1340 is configured to render the image to be displayed on the head-mounted display device based on the second correction posture, so as to display the image to be displayed on the head-mounted display device.
  • the first processing module 1320 is used for:
  • the first processing module 1320 is used for:
  • the device also includes:
  • Receiving module 1350 used to receive user input operations
  • the first adjustment module 1360 is configured to adjust the cutoff frequency of the low-pass filter in response to user input operations.
  • the first processing module 1320 is used for:
  • the correction proportional coefficient is used to correct the attitude error corresponding to the current moment to obtain the corrected attitude difference, where the correction proportional coefficient is greater than 0 and less than 1.
  • the device further includes:
  • the second determination module 1315 is used to determine the correction proportion coefficient based on the compensation reference information.
  • the compensation reference information includes at least one of the following: the posture error corresponding to the current moment, the cumulative value of the posture error, and the location of the mobile platform where the head-mounted display device is located. environmental conditions and steering information of the mobile platform.
  • the correction proportion coefficient is positively correlated with the attitude error corresponding to the current moment
  • the correction proportional coefficient is positively correlated with the cumulative value of attitude error
  • Environmental conditions include road surface smoothness, and the correction proportion coefficient is positively correlated with road surface smoothness;
  • the steering information includes the degree of steering, and the correction proportion coefficient is positively related to the degree of steering.
  • the device further includes:
  • the third determination module 1380 is used to determine the change pattern of the attitude error based on the attitude error sequence.
  • the attitude error sequence includes corresponding attitude errors at multiple times;
  • the second adjustment module 1390 is used to adjust the correction proportion coefficient based on the attitude error change law.
  • the second adjustment module 1390 includes at least one of the first adjustment unit 13901, the second adjustment unit 13903, the third adjustment unit 13905 and the fourth adjustment unit 13907 shown in Figure 16;
  • the first adjustment unit 13901 is used to increase the correction proportion coefficient when it is determined that the attitude error corresponding to each moment in the target time period is greater than or equal to the attitude error corresponding to the target time based on the attitude error change law;
  • the second adjustment unit 13903 is used to reduce the correction proportion coefficient when it is determined that the attitude error corresponding to each moment in the target time period is smaller than the attitude error corresponding to the target time based on the attitude error change law;
  • the third adjustment unit 13905 is configured to increase the correction proportion coefficient when it is determined that the change trend of the attitude error is an increasing trend based on the change pattern of the attitude error;
  • the fourth adjustment unit 13907 is used to determine the change trend of the attitude error to decrease based on the attitude error change law. In the case of a small trend, reduce the correction proportion coefficient;
  • the target time, the starting time of the target time period, and the end time of the target time period are each one of multiple times.
  • the starting time of the target time period is later than the target time, and the duration of the target time period is greater than or equal to Default duration.
  • the device also includes:
  • the identification module 1302 is used to identify the type of platform on which the head-mounted display device is located;
  • the control module 1304 is used to control the head-mounted display device to switch to the target mode when the type is a mobile platform;
  • the trigger module 1306 is configured to trigger the first determination module 1310 when the head-mounted display device is in the target mode.
  • the head-mounted display device is configured to be coupled to a mobile platform and moveable relative to the mobile platform.
  • the preset posture is a posture used to display the display screen of the head-mounted display device at a preset position in the coordinate system of the head-mounted display device;
  • the preset posture is a posture for aligning the display screen of the head-mounted display device with the direction of gravity.
  • the device further includes:
  • the configuration module 1395 is configured to configure the optical machine in the head-mounted display device based on the second correction posture.
  • the electronic device may be either or both of the first device and the second device, or a stand-alone device independent of them.
  • the stand-alone device may communicate with the first device and the second device to receive the collected information from them. input signal.
  • Figure 19 illustrates a block diagram of an electronic device 1900 according to an embodiment of the present disclosure.
  • electronic device 1900 includes one or more processors 1910 and memory 1920.
  • Processor 1910 may be a central processing unit (CPU) or other form of processing unit having data processing capabilities and/or instruction execution capabilities, and may control other components in electronic device 1900 to perform desired functions.
  • CPU central processing unit
  • Processor 1910 may be a central processing unit (CPU) or other form of processing unit having data processing capabilities and/or instruction execution capabilities, and may control other components in electronic device 1900 to perform desired functions.
  • Memory 1920 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory.
  • the volatile memory may include, for example, random access memory (RAM) and/or cache memory (cache).
  • the non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc.
  • One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 1910 may execute the program instructions to implement various embodiments of the present disclosure described above for use in a head-mounted display device. Method of displaying the screen and/or other desired functionality.
  • Various contents such as input signals, signal components, noise components, etc. may also be stored in the computer-readable storage medium.
  • the electronic device 1900 may also include an input device 1930 and an output device 1940, these components being interconnected by a bus system and/or other forms of connection mechanisms (not shown).
  • the input device 1930 may be a microphone or a microphone array.
  • the input device 1930 may be a communication network connector for receiving the collected input signals from the first device and the second device.
  • the input device 1930 may also include, for example, a keyboard, a mouse, and the like.
  • the output device 1940 can output various information to the outside.
  • the output device 1940 may include, for example, a display, a speaker, a printer, a communication network and remote output devices connected thereto, and the like.
  • the electronic device 1900 may also include any other appropriate components depending on the specific application.
  • embodiments of the present disclosure may also be a computer program product, which includes computer program instructions that, when executed by a processor, cause the processor to perform the “exemplary method” described above in this specification Steps in a method for displaying a picture on a head-mounted display device according to various embodiments of the present disclosure are described in section .
  • the computer program product may be written with program code for performing operations of embodiments of the present disclosure in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc. , also includes conventional procedural programming languages, such as the "C" language or similar programming languages.
  • the program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server execute on.
  • embodiments of the present disclosure may also be a computer-readable storage medium having computer program instructions stored thereon.
  • the computer program instructions when executed by a processor, cause the processor to execute the above-mentioned “example method” part of this specification.
  • the steps in the method for displaying a picture on a head-mounted display device according to various embodiments of the present disclosure are described in .
  • the computer-readable storage medium may be any combination of one or more readable media.
  • the readable medium may be a readable signal medium or a readable storage medium.
  • the readable storage medium may include, for example, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: electrical connection with one or more conductors, portable disk, hard disk, random access memory (RAM), read only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.
  • the methods and apparatus of the present disclosure may be implemented in many ways.
  • the methods and devices of the present disclosure may be implemented through software, hardware, firmware, or any combination of software, hardware, and firmware.
  • the above order for the steps of the methods is for illustration only, and the steps of the methods of the present disclosure are not limited to the order specifically described above unless otherwise specifically stated.
  • the present disclosure may also be implemented as programs recorded in recording media, and these programs include machine-readable instructions for implementing methods according to the present disclosure.
  • the present disclosure also covers recording media storing programs for executing methods according to the present disclosure.
  • each component or each step can be decomposed and/or recombined. These decompositions and/or recombinations should be considered equivalent versions of the present disclosure.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Controls And Circuits For Display Device (AREA)

Abstract

本申请公开了一种用于在头戴显示设备显示画面的方法、装置及电子设备。具体实现方案为：基于头戴显示设备的当前时刻的角速度数据，以及在上一时刻用于对头戴显示设备的显示画面进行处理的第一校正姿态，确定在当前时刻用于对待显示画面进行处理的参考姿态（110）；对参考姿态与用于对显示画面进行处理的预设姿态之间的姿态误差进行处理，以得到校正姿态差值（120）；利用校正姿态差值补偿参考姿态，以得到在当前时刻用于对待显示画面进行处理的第二校正姿态（130）；基于第二校正姿态，对头戴显示设备的待显示画面进行渲染，以显示头戴显示设备的待显示画面（140）。

Description

用于在头戴显示设备显示画面的方法、装置及电子设备
本公开要求在2022年05月11日提交中国专利局、申请号为CN202210510750.8、发明名称为“用于在头戴显示设备显示画面的方法、装置及电子设备”的中国专利申请的优先权，其全部内容通过引用结合在本公开中。
技术领域
本公开涉及头戴显示设备技术领域，尤其涉及一种用于在头戴显示设备显示画面的方法、装置及电子设备。
背景技术
目前，头戴显示设备的应用越来越广泛。头戴显示设备使用过程中可能存在颠簸、晃动等情况，这会影响到头戴显示设备的显示效果。如何保证头戴显示设备的显示效果，以提升用户的使用体验对于本领域技术人员而言是一个亟待解决的问题。
发明内容
本公开的实施例提供了一种用于在头戴显示设备显示画面的方法、装置及电子设备。
根据本公开的一个方面，提供了一种用于在头戴显示设备显示画面的方法，包括：基于头戴显示设备的当前时刻的角速度数据，以及在上一时刻用于对头戴显示设备的显示画面进行处理的第一校正姿态，确定在当前时刻用于对待显示画面进行处理的参考姿态；对参考姿态与用于对显示画面进行处理的预设姿态之间的姿态误差进行处理，以得到校正姿态差值；利用校正姿态差值补偿参考姿态，以得到在当前时刻用于对待显示画面进行处理的第二校正姿态；基于第二校正姿态，对头戴显示设备的待显示画面进行渲染，以显示头戴显示设备的待显示画面。
根据本公开的再一个方面，提供了一种计算机可读存储介质，该存储介质存储有计算机程序，计算机程序用于执行上述用于在头戴显示设备显示画面的方法。
根据本公开的又一个方面，提供了一种电子设备，包括：处理器；用于存储处理器可执行指令的存储器；处理器，用于从存储器中读取可执行指令，并执行指令以实现上述用于在头戴显示设备显示画面的方法。
下面通过附图和实施例，对本公开的技术方案做进一步的详细描述。
附图说明
通过结合附图对本公开实施例进行更详细的描述，本公开的上述以及其他目的、特征和优势将变得更加明显。附图用来提供对本公开实施例的进一步理解，并且构成说明书的一部分，与本公开实施例一起用于解释本公开，并不构成对本公开的限制。在附图中，相同的参考标号通常代表相同部件或步骤。
图1是本公开一示例性实施例提供的用于在头戴显示设备显示画面的方法的流程示意图。
图2是本公开的实施例中对头戴显示设备的显示画面的一种处理方式的示意图。
图3是本公开的实施例中对头戴显示设备的显示画面的另一种处理方式的示意图。
图4是本公开的实施例所适用的场景的示意图。
图5是本公开另一示例性实施例提供的用于在头戴显示设备显示画面的方法的流程示 意图。
图6是本公开的实施例中确定第二校正姿态的流程示意图。
图7是本公开再一示例性实施例提供的用于在头戴显示设备显示画面的方法的流程示意图。
图8是本公开又一示例性实施例提供的用于在头戴显示设备显示画面的方法的流程示意图。
图9是本公开又一示例性实施例提供的用于在头戴显示设备显示画面的方法的流程示意图。
图10是本公开又一示例性实施例提供的用于在头戴显示设备显示画面的方法的流程示意图。
图11-1是本公开又一示例性实施例提供的用于在头戴显示设备显示画面的方法的流程示意图。
图11-2是本公开又一示例性实施例提供的用于在头戴显示设备显示画面的方法的流程示意图。
图11-3是本公开又一示例性实施例提供的用于在头戴显示设备显示画面的方法的流程示意图。
图11-4是本公开又一示例性实施例提供的用于在头戴显示设备显示画面的方法的流程示意图。
图12是本公开又一示例性实施例提供的用于在头戴显示设备显示画面的方法的流程示意图。
图13是本公开一示例性实施例提供的用于在头戴显示设备显示画面的装置的结构示意图。
图14是本公开另一示例性实施例提供的用于在头戴显示设备显示画面的装置的结构示意图。
图15是本公开再一示例性实施例提供的用于在头戴显示设备显示画面的装置的结构示意图。
图16是本公开又一示例性实施例提供的用于在头戴显示设备显示画面的装置的结构示意图。
图17是本公开又一示例性实施例提供的用于在头戴显示设备显示画面的装置的结构示意图。
图18是本公开又一示例性实施例提供的用于在头戴显示设备显示画面的装置的结构示意图。
图19是本公开一示例性实施例提供的电子设备的结构图。
具体实施方式
下面，将参考附图详细地描述根据本公开的示例实施例。显然，所描述的实施例仅仅是本公开的一部分实施例，而不是本公开的全部实施例，应理解，本公开不受这里描述的示例实施例的限制。
应注意到：除非另外具体说明，否则在这些实施例中阐述的部件和步骤的相对布置、数字表达式和数值不限制本公开的范围。
本领域技术人员可以理解，本公开实施例中的“第一”、“第二”等术语仅用于区别不同步骤、设备或模块等，既不代表任何特定技术含义，也不表示它们之间的必然逻辑顺序。
还应理解，在本公开实施例中，“多个”可以指两个或两个以上，“至少一个”可以指一个、两个或两个以上。
还应理解，对于本公开实施例中提及的任一部件、数据或结构，在没有明确限定或者在前后文给出相反启示的情况下，一般可以理解为一个或多个。
另外，本公开中术语“和/或”，仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。另外，本公开中字符“/”，一般表示前后关联对象是一种“或”的关系。
还应理解，本公开对各个实施例的描述着重强调各个实施例之间的不同之处，其相同或相似之处可以相互参考，为了简洁，不再一一赘述。
以下对至少一个示例性实施例的描述实际上仅仅是说明性的，决不作为对本公开及其应用或使用的任何限制。
对于相关领域普通技术人员已知的技术、方法和设备可能不作详细讨论，但在适当情况下，所述技术、方法和设备应当被视为说明书的一部分。
应注意到：相似的标号和字母在下面的附图中表示类似项，因此，一旦某一项在一个附图中被定义，则在随后的附图中不需要对其进行进一步讨论。
本公开实施例可以应用于终端设备、计算机系统、服务器等电子设备，其可与众多其它通用或专用计算系统环境或配置一起操作。适于与终端设备、计算机系统、服务器等电子设备一起使用的众所周知的终端设备、计算系统、环境和/或配置的例子包括但不限于：个人计算机系统、服务器计算机系统、瘦客户机、厚客户机、手持或膝上设备、基于微处理器的系统、机顶盒、可编程消费电子产品、网络个人电脑、小型计算机系统、大型计算机系统和包括上述任何系统的分布式云计算技术环境，等等。
终端设备、计算机系统、服务器等电子设备可以在由计算机系统执行的计算机系统可执行指令(诸如程序模块)的一般语境下描述。通常，程序模块可以包括例程、程序、目标程序、组件、逻辑、数据结构等等，它们执行特定的任务或者实现特定的抽象数据类型。计算机系统/服务器可以在分布式云计算环境中实施，分布式云计算环境中，任务是由通过通信网络链接的远程处理设备执行的。在分布式云计算环境中，程序模块可以位于包括存储设备的本地或远程计算系统存储介质上。
示例性方法
图1是本公开一示例性实施例提供的用于在头戴显示设备显示画面的方法的流程示意图。图1所示的方法可以包括步骤110、步骤120、步骤130和步骤140，下面对各步骤分别进行说明。
步骤110，基于头戴显示设备的当前时刻的角速度数据，以及在上一时刻用于对头戴显示设备的显示画面进行处理的第一校正姿态，确定在当前时刻用于对待显示画面进行处理的参考姿态。
需要说明的是，头戴显示设备也可以称为头戴显示器(Head-Mounted Display，HMD)或者头显。头戴显示设备可以用于实现扩展现实(Extended Reality，XR)效果，例如用于实现增强现实(Augmented Reality，AR)效果、虚拟现实(Virtual Reality，VR)效果、混合现实(Mixed Reality，MR)效果等。可选地，头戴显示设备可以用于实现AR效果，此时，头戴显示设备可以为AR眼镜。
一般而言，头戴显示设备可以包括惯性测量单元(Inertial Measurement Unit，IMU)和光机。IMU可以包括加速度计、陀螺仪和磁力仪等。加速度计用于采集加速度数据，陀螺仪用于采集角速度数据。光机是头戴显示设备的成像系统，光机可以包括显示屏和光学元件。
通常情况下，用户可能无法直接看到头戴显示设备的显示画面，而是通过如下的方案向用户提供显示画面。显示屏用于发出显示画面的光线，光学元件可对上述光线进行处理，使得显示画面的光线投射入用户的眼睛，最终使得用户能够看到显示画面。而由于用户在佩戴头戴显示设备的时候，头戴显示设备与用户之间几乎不发生相对的位移和旋转等，头戴显示设备会随着用户的运动而运动。因此，可能需要依据头戴显示设备(其实也可以被看作是用户)的位置和姿态等对显示画面进行处理，以使得显示画面能够一直定位在某个位置、或是能在某个区域内、或是能随用户运动而运动等等，以匹配相应的应用场景或是用户的需求。
在步骤110之前，可以获取IMU的当前时刻的角速度数据，例如IMU中的陀螺仪采集的角速度数据，并获取在上一时刻用于对头戴显示设备的显示画面进行处理的第一校正姿态。第一校正姿态可以采用与下文中的第二校正姿态类似的方式得到，为了布局清楚，在此不对第一校正姿态的获得方式做展开介绍。
在步骤110中，可以参照由陀螺仪角速度至角位置的积分方式，使用当前时刻的角速度数据和第一校正姿态做积分处理，以得到在当前时刻用于对待显示画面(该待显示画面可以为头戴显示设备的待显示画面)进行处理的参考姿态。假设当前时刻的角速度数据表示为Gyrok，第一校正姿态表示为Posek-1，参考姿态表示为则有：
步骤120，对参考姿态与用于对显示画面进行处理的预设姿态之间的姿态误差进行处理，以得到校正姿态差值。
需要说明的是，预设姿态可以为预先为头戴显示设备设定的、能够保证头戴显示设备的显示效果的姿态。也就是说预设姿态可以是为了保证显示效果而用于对显示画面进行处理的姿态。
在一个可选示例中，预设姿态为用于使头戴显示设备的显示画面显示在头戴显示设备的坐标系中预设位置的姿态。可以理解的，也可以为IMU坐标系下的预设位置。
可以理解的，头戴显示设备的坐标系中预设位置可以是在头戴显示设备的坐标系中的一个固定的位置，例如使显示画面显示在头戴显示设备的正前方、左上方、右下方等等能被用户观察到的位置。需要说明的是，大多数用户的使用习惯是使显示画面显示在面前中心的位置。本公开的实施例中，可以通过将预设姿态设置为与大多数用户的使用习惯匹配的姿态，使头戴显示设备的显示效果尽可能满足大多数用户需求，这样有利于提升用户的使用体验。
在一个可选示例中，预设姿态为用于使头戴显示设备的显示画面与重力方向对齐的姿态(该姿态下，头戴显示设备的航向角(Yaw)可以为0)。
头戴显示设备的显示画面与重力方向对齐可以理解为，用户通过头戴显示设备观察显示画面时，显示画面的平面的延伸方向和真实世界中的重力方向是相同的，这样能保持显示画面基本与用户平视视线垂直，符合观看习惯。需要说明的是，大多数用户的使用习惯是使头戴显示设备的显示画面与重力方向对齐。本公开的实施例中，通过将预设姿态设置为与大多数用户的使用习惯匹配的姿态，能够使头戴显示设备的显示效果尽可能满足大多数用户需求，这样有利于提升用户的使用体验。
当然，用户也可以根据需求来设置能使头戴显示设备的显示画面显示在定制位置的预设姿态。
在步骤120中，可以将参考姿态与预设姿态做差，以得到参考姿态与预设姿态之间的姿态误差。姿态误差可以呈3×1的轴角形式。需要说明的是，这里的做差并不是简单的减法，而是广义的函数，做差中的“差”指的是差异，做差可以是指求两个角度之间的相 对旋转。假设参考姿态表示为预设姿态表示为Target，姿态误差表示为error，则有：
在得到姿态误差之后，可以通过对姿态误差进行处理，实现姿态误差的校正，从而得到校正姿态差值。
可以理解地，预设姿态可以为预先为头戴显示设备设定的姿态。在预设姿态与参考姿态为同一坐标系下的姿态，预设姿态为零旋转的姿态的情况下，为了程序节约，可以没有将参考姿态与预设姿态做差的步骤，直接将参考姿态作为后续的待处理量。即，在后续步骤中，直接对参考姿态进行处理，实现姿态误差的校正，从而得到校正姿态差值。
步骤130，利用校正姿态差值补偿参考姿态，以得到在当前时刻用于对待显示画面进行处理的第二校正姿态。
在步骤130中，可以将校正姿态差值作为补偿值更新到参考姿态中，以得到在当前时刻用于对待显示画面进行处理的第二校正姿态。第二校正姿态表示为Posek，则有：
需要说明的是，这里的补偿值表示的可以是一个相对旋转，将补偿值更新到参考姿态中，可以是在参考姿态的基础上叠加一个相对旋转，从而得到另一个姿态(即第二校正姿态)。
可以理解地，校正姿态差值是用于更新参考姿态，以得到第二校正姿态的。不论校正姿态差值是通过处理参考姿态得到的，还是处理参考姿态与预设姿态之间的姿态误差得到的，第二校正姿态都是基于校正姿态差值更新的。
步骤140，基于第二校正姿态，对头戴显示设备的待显示画面进行渲染，以显示头戴显示设备的待显示画面。
可以理解的，第一校正姿态和第二校正姿态均是用于对头戴显示设备的显示画面进行处理的姿态，也可以称之为校正姿态。例如，在头戴显示设备在首次执行本方法而没有上一时刻的校正姿态的情况下，上一时刻的校正姿态也可以采用头戴显示设备的真实姿态、用于对显示画面进行处理的预设姿态、或是一个预先设定的其他姿态等。
需要说明的是，基于第二校正姿态或是校正姿态，对头戴显示设备的显示画面进行处理的具体实现形式多样，下面进行举例介绍。
在一种可选的实现形式中，头戴显示设备的渲染引擎可以使用第二校正姿态或是校正姿态做画面渲染，以得到相应的显示画面，得到的显示画面可以更新到头戴显示设备的光机中显示，以实现头戴显示设备的显示画面的修正。容易看出，这种实现形式采用软件方式实现头戴显示设备的显示画面的处理。例如，该方案可以基于第二校正姿态或是校正姿态调整待显示画面的整体参数，整体参数可以包括显示画面的三维尺寸等。
在另一种可选的实现形式中，如图2所示，可以基于第二校正姿态(例如图2中的Posek)或是校正姿态做画面渲染，调整显示画面在头戴显示设备的光机中的显示屏(例如图2中的Display)中的显示位置，以实现头戴显示设备的显示画面的修正。采用这种实现形式来实现头戴显示设备的显示画面的处理，例如，头戴显示设备的显示画面可以上下左右调整显示位置等。例如，该方案可以基于第二校正姿态或是校正姿态调整待显示画面的二维尺寸。
在再一种可选的实现形式中，如图3所示，可以基于第二校正姿态或是校正姿态做画面渲染，并配置头戴显示设备中的光机。例如，可以通过电控机箱调整头戴显示模组中的光机的整体位置或是调整光机中的显示屏的位置，以实现头戴显示设备的显示画面的修正 (这相当于在用户眼前拿着显示屏做相应的减震调整)。容易看出，这种实现形式采用硬件方式实现头戴显示设备的显示画面的处理，这种实现形式的执行逻辑可以为：第二校正姿态→电控机箱→光机旋转。
本公开的实施例中，结合头戴显示设备的当前时刻的角速度数据，以及上一时刻用于对头戴显示设备的显示画面进行处理的第一校正姿态，可以确定在当前时刻用于对待显示画面进行处理的参考姿态，接下来可以对参考姿态与用于对显示画面进行处理的预设姿态之间的姿态误差进行处理，或是对参考姿态进行处理，得到校正姿态差值，并利用校正姿态差值补偿参考姿态，以得到在当前时刻用于对待显示画面进行处理的第二校正姿态。第二校正姿态可以认为是参考头戴显示设备的当前时刻的角速度数据、第一校正姿态和预设姿态确定的，适配于当前时刻头戴显示设备(其实也可以被看作是用户)的姿态，基于该姿态对头戴显示设备的待显示画面进行渲染，有利于使头戴显示设备的显示效果依据头戴显示设备在预设姿态下的显示效果做校正。这样，不管是头戴显示设备使用过程中发生小角度变化的情况(颠簸、晃动、抖动等)，或是头戴显示设备使用过程中发生大角度变化的情况(例如大角度转弯)，均能够保证头戴显示设备向预设姿态对应地调整显示效果，从而能够提升用户的使用体验。
可选地，本公开的实施例中涉及的各个姿态均可以呈旋转矩阵(Rotation matrix)的形式；本公开的实施例中涉及的各个校正姿态均可以是用于头戴显示设备的渲染引擎的画面渲染的姿态。
需要说明的是，对于头戴显示设备而言，在做内容呈现时，目前已有的模式有多种，可以包括第一模式和第二模式。例如，第一模式可以为3自由度(degree of freedom，dof)模式，第一模式下，头戴显示设备对显示画面进行定位方面的处理，此时，头戴显示设备的显示画面的内容与用户的姿态相关，头戴显示设备的显示画面可以与世界坐标系保持旋转不变。第二模式与手机至电视的投屏模式类似，第二模式下，头戴显示设备不对显示画面进行定位方面的处理，此时，头戴显示设备的显示画面的内容可以与内容播放进度有关，而与用户的姿态无关，头戴显示设备的显示画面随用户头部运动而运动，为了便于说明，第二模式可以称为0自由度(dof)模式。
可以理解的是，用户可以在车辆、船、飞机或者其它类型的移动平台上使用头戴显示设备。在0dof模式下，如果移动平台的运行平稳，头戴显示设备的显示效果尚可接受。而一旦出现颠簸、晃动，例如，由于路面不平整，行进中的车辆颠簸、晃动，再例如，由于气流影响，进行中的飞机颠簸、晃动，导致用户会带着头戴显示设备进行抖动或是晃动等。在该0dof模式下，由于显示画面也会随之抖动或是晃动，这种情况下用户将无法稳定清晰地观察头戴显示设备的显示画面的内容，这样会降低用户的使用体验。
在3dof模式下，一旦出现上述颠簸、晃动的情况，由于对显示画面进行了定位方面的处理，即使头戴显示设备抖动或是晃动，显示画面的定位可以不发生变化，用户可以看清头戴显示设备的显示画面的内容。但是在移动平台方向变化的情况下(比如转90度)，由于用户(头戴显示设备)也会随着移动平台变化相同的方向，而显示画面的定位还是不发生变化的话，用户需要转变为发生方向变化前的姿态才能观察所需的显示画面，这样也会降低用户的使用体验。
因此，通过本公开的实施例，可以实现头戴显示设备的显示效果依据头戴显示设备在预设姿态下的显示效果做校正。例如，可以依据校正姿态对头戴显示设备的显示画面进行处理使得，在头戴显示设备使用过程中小角度变化的情况下，显示效果类似3dof模式，在大角度变化的情况下，显示效果类似于0dof模式，从而能够提升用户的使用体验。
在一个可选示例中，头戴显示设备配置为与所在的移动平台藕接并相对移动平台可移 动。
需要说明的是，头戴显示设备可以通过用户与所在的移动平台藕接，移动平台可以处于运动状态。可选地，如图4所示，头戴显示设备可以佩戴于用户(例如车辆上的乘客)头部，头戴显示设备与用户之间保持相对静止。用户可以坐在或站在车辆内部，在车辆运动的时候，用户会随车辆运动。用户还可以在车辆内部运动(例如由一个位置换到另一个位置，或者由朝一个方向换到朝另一个方向等)。另外，移动平台为其他类型时，例如船，用户也可以坐在或站在船舷上。
需要说明的是，对于头戴显示设备配置为与所在的移动平台藕接并相对移动平台可移动的场景，可能出现移动平台在运动的过程中颠簸、晃动的情况，或者移动平台在运动的过程中转向的情况，有鉴于此，可以针对该场景执行本公开的方法步骤，以保证头戴显示设备的显示效果，提升用户的使用体验。
需要指出的是，针对移动平台上的手机(或电脑)的显示画面防抖，目前存在一种解决方案，即用户手动调节手机的真实显示屏幕与用户眼睛之间的相对位置和角度。也即，手机的真实显示屏幕相对于用户眼睛具有相对位移和旋转。相比较而言，采用本公开的实施例中的防抖方案，头戴显示设备相对于用户眼睛几乎没有相对位移和旋转。
在图1所示实施例的基础上，如图5所示，步骤120，包括：
步骤1201，对当前时刻对应的姿态误差进行低通滤波(Low Pass Filter)处理，以得到校正姿态差值。
需要说明的是，通过将步骤110确定的参考姿态与预设姿态做差得到的姿态误差即为当前时刻对应的姿态误差。而在预设姿态与参考姿态为同一坐标系下的姿态，预设姿态为零旋转的姿态的情况下，可以对参考姿态进行低通滤波处理，以得到校正姿态差值。
可选地，可以通过无限脉冲响应(Infinite Impulse Response，IIR)数字滤波器、有限脉冲响应(Finite Impulse Response，FIR)数字滤波器等低通滤波器，对当前时刻对应的姿态误差进行低通滤波处理。IIR数字滤波器的每一个输出值可以仅依赖之前的一个采样值，因此工程应用非常便捷。FIR数字滤波器的低通滤波处理过程涉及滑动窗口，FI R数字滤波器的每一个输出值可以通过对之前若干数量的采样值进行计算得到。假设当前时刻对应的姿态误差表示为error，校正姿态差值表示为error1，则有：
error1＝LowPassfilter(error)
可以理解的，在理想状态下，只有低于截止频率的数据能够通过，则校正姿态差值error1的值不为0，即后续利用校正姿态差值error1补偿参考姿态后的第二校正姿态与参考姿态不同。而高于截止频率的数据不能通过，则校正姿态差值error1的值为0，即后续利用校正姿态差值error1补偿参考姿态后的第二校正姿态与参考姿态相同。然而实际工况下，校正姿态差值error1的值为0的情况可能很少，可能会是一个很小的值，以至于并不能被用户觉察到。
本公开的实施例中，如图6所示，在获取IMU中的陀螺仪采集的当前时刻的角速度数据Gyrok之后，可以使用当前时刻的角速度数据Gyrok和在上一时刻用于对头戴显示设备的显示画面进行处理的第一校正姿态Posek-1做积分处理，以得到在当前时刻用于对待显示画面进行处理的参考姿态通过将与预设姿态Target做差，可以得到当前时刻对应的姿态误差error。例如通过对error进行低通滤波处理，可以得到校正姿态差值error1。error1可以作为补偿值更新到中，以最终得到第二校正姿态Posek。Posek可以认为是能够实现防抖的姿态，Posek可以用于当前时刻对待显示画面的处理。需要说明的是，图6中呈现的是将Posek-1作为输入得到Posek的方式，后续还可以采用类似的方式，将Posek 作为输入得到Posek+1，将Posek+1作为输入得到Posek+2等。这样，通过对姿态误差进行低通滤波处理，可以将高频的抖动和晃动(例如是短时间内的小角度的运动)过滤掉，头戴显示设备的显示画面会趋于不动，从而便于用户清晰稳定地观察头戴显示设备的显示画面的内容，这样能够提升用户的使用体验。另外，低频的运动(例如长时间的大角度的运动)会被保留，如果用户转头或者移动平台转向(例如移动平台转向导致头跟着转向)，导致头戴显示设备发生大角度的运动，经过补偿后，头戴显示设备的显示画面会跟随用户头部而定位，以便于用户观察所需的显示画面，这样也能够提升用户的使用体验。
在图5所示实施例的基础上，如图7所示，步骤1201，包括：
步骤12011，通过低通滤波器对当前时刻对应的姿态误差进行低通滤波处理。
该方法还包括步骤150和步骤160。
步骤150，接收用户输入操作。
可选地，用户输入操作包括但不限于语音输入操作、触控输入操作等。例如，用户输入操作可以为用户语音说“将截止频率调整为2hz(也可以为5hz或者其它频率)”的操作。或者，头戴显示设备可以具有配对的手柄，用户输入操作可以为用户点击手柄上的截止频率调整按钮的操作。或者，显示画面上还能显示截止频率选择框或是输入框，用户可以直接选择或是输入所需的截止频率。
步骤160，响应于用户输入操作，对低通滤波器的截止频率进行调整。
在步骤160中，可以根据用户输入操作，确定用户所需的截止频率，然后采用任意可实施的方式，将低通滤波器的截止频率调整为用户所需的截止频率。
结合上文中的描述可知，通过低通滤波器的低通滤波处理，对于头戴显示设备的运动频率不高于低通滤波器的截止频率的情况，头戴显示设备的显示画面跟随头戴显示设备的运动而变化，而对于头戴显示设备的运动频率高于低通滤波器的截止频率的情况，头戴显示设备的显示画面趋于不动。可以理解的，该方法可能对越接近截止频率的运动越不敏感，这样，头戴显示设备的运动频率比截止频率高得越多，头戴显示设备的实际模式越接近3dof模式。头戴显示设备的运动频率比截止频率低得越多，头戴显示设备的实际模式越接近0dof模式。因此，可以理解的是，截止频率越高，则大部分运动的频率可能低于该截止频率，则该方案对头戴显示设备的大部分的运动都会进行使得显示画面跟随头戴显示设备的运动的补偿处理，则头戴显示设备的实际模式越接近0dof模式。而截止频率越低，则大部分运动的频率可能高于该截止频率，则该方案对头戴显示设备的大部分的运动都会进行使得显示画面趋于不动的补偿处理，头戴显示设备的实际模式越接近3dof模式。
本公开的实施例中，通过对低通滤波器的截止频率进行调整，用户可以根据实际需求，使头戴显示设备的实际模式更接近0dof模式或更接近3dof模式。
需要指出的是，以上介绍的是低通滤波器的截止频率可调的情况，具体实现时，低通滤波器的截止频率也可以设置为一不可调的定值。
在图1所示实施例的基础上，如图8所示，步骤120，包括：
步骤1203，利用校正比例系数校正当前时刻对应的姿态误差，以得到校正姿态差值，校正比例系数大于0且小于1。
可选地，校正比例系数可以为0.2、0.5、0.6、0.8或者0至1之间的其它数值，在此不再一一列举。
可以理解地，在预设姿态与参考姿态为同一坐标系下的姿态，预设姿态为零旋转的姿态的情况下，可以利用校正比例系数校正参考姿态，以得到校正姿态差值。
在步骤1203中，可以将当前时刻对应的姿态误差乘以校正比例系数，并将相乘结果作为校正姿态差值。
可以理解的是，存在抖动和晃动的情况下，使得用户的头部往复运动，导致头戴显示 设备的实际姿态有正有负，姿态误差也有正有负。经过补偿后，头戴显示设备的显示画面会趋于不动，从而便于用户清晰稳定地观察头戴显示设备的显示画面的内容，这样能够提升用户的使用体验。另外，如果用户转头或者移动平台转向(例如移动平台转向导致用户的头跟着转向)，导致头戴显示设备发生转向运动，头戴显示设备的实际姿态始终是正或者负，姿态误差也始终是正或者负。经过补偿后，头戴显示设备的显示画面会跟随用户头部的朝向，以便于用户观察所需的显示画面，这样也能够提升用户的使用体验。
需要说明的是，校正比例系数越大，校正姿态差值与姿态误差越接近，在将校正姿态差值用于第二校正姿态的确定时，会使第二校正姿态与预设姿态更接近。相应地，头戴显示设备的显示画面的修正速度会越快，这样，校正比例系数越大，头戴显示设备的实际模式越接近0dof模式，校正比例系数越小，头戴显示设备的实际模式越接近3dof模式。
本公开的实施例中，通过引入校正比例系数对姿态误差做校正，可以使头戴显示设备的校正姿态接近预设姿态，将校正姿态用于头戴显示设备的显示画面渲染，有利于使头戴显示设备的显示效果依据头戴显示设备在预设姿态下的显示效果做校正。这样，不管是头戴显示设备使用过程中发生小角度变化的情况(颠簸、晃动、抖动等)，或是头戴显示设备使用过程中发生大角度变化的情况(例如大角度转弯)，均能够保证头戴显示设备向预设姿态对应地调整显示效果，从而能够提升用户的使用体验。并且，通过控制校正比例系数，用户可以根据实际需求，使头戴显示设备的实际模式更接近0dof模式或更接近3dof模式。
在图8所示实施例的基础上，如图9所示，该方法还包括：
步骤115，基于补偿参考信息，确定校正比例系数，补偿参考信息包括以下至少一项：当前时刻对应的姿态误差、姿态误差的累计值、头戴显示设备所在的移动平台所处的环境状况、移动平台的转向信息。
可选地，步骤115可以在步骤120之前执行，也即，在基于补偿参考信息，确定校正比例系数之后，再利用确定好的校正比例系数，进行姿态误差的校正。或者，步骤115可以在步骤120之后执行，也即，在利用校正比例系数，进行姿态误差的校正之后，可以基于补偿参考信息，确定校正比例系数。以便利用确定结果，进行校正比例系数的动态更新。这样，经更新后的校正比例系数后续可以用于姿态误差的校正。本公开的实施例对步骤115和步骤120的执行顺序不做任何限定。因此，可以理解的，本公开的实施例中的校正比例系数可以是基于上述因素动态调整的。
可以理解地，在预设姿态与参考姿态为同一坐标系下的姿态，预设姿态为零旋转的姿态的情况下，当前时刻对应的姿态误差和姿态误差的累计值可以理解为：当前时刻对应的参考姿态和参考姿态的累计值。
头戴显示设备所在的移动平台所处的环境状况可以基于安装在平台上的装置(例如摄像头、雷达等)采集的信息确定。在头戴显示设备所在的移动平台为车辆的情况下，头戴显示设备所在的移动平台所处的环境状态可以包括车辆所处路面的路面信息，路面信息可以包括路面平整度。在头戴显示设备所在的移动平台为飞机的情况下，头戴显示设备所在的移动平台所处的环境状态可以包括飞机所处位置的气流强度。
头戴显示设备所在的移动平台的转向信息可以包括转向程度，转向程度可以利用转向角度进行表征。在移动平台为车辆的情况下，转向信息可以由车载转向系统检测得到。
可选地，校正比例系数与当前时刻对应的姿态误差呈正相关。
在参考姿态与预设姿态之间的差异越大的情况下，校正比例系数也越大，较大的校正比例系数有利于更快速地依据预设姿态修正头戴显示设备的显示画面，从而有利于保证头戴显示设备的显示效果。
可选地，校正比例系数与姿态误差的累计值呈正相关。姿态误差的累计值可以是预设时间段内的姿态误差的累计值，也可以是持续的姿态误差的累计值。与上述示例相比，校正比例系数由画面显示之前的整体运动趋势确定，能使得校正比例系数更匹配实际的姿态误差。例如在移动平台进行大角度的转弯时，能够快速地依据预设姿态修正头戴显示设备的显示画面，从而有利于保证头戴显示设备的显示效果。
可以理解的是，姿态误差的值可以是一个正数，也可以是一个负数。例如定义朝向第一方向的姿态误差的值是正数的话，则朝向与第一方向相反的第二方向的姿态误差的值则是负数。正数的姿态误差和负数的姿态误差都可以进行运算处理。当评价姿态误差的“大”、“小”和“正相关”时，可以参照姿态误差的绝对值进行评价。
可选地，环境状况包括路面平整度，校正比例系数与路面平整度呈正相关。这样，路面越不平整，校正比例系数越小，较小的校正比例系数使得头戴显示设备的显示画面依据预设姿态进行修正的速度更为和缓，从而有利于保证头戴显示设备的显示画面稳定性。
可选地，转向信息包括转向程度，校正比例系数与转向程度呈正相关。这样，移动平台方向变化越大，校正比例系数越大，较大的校正比例系数有利于更快速地依据预设姿态修正头戴显示设备的显示画面，从而有利于保证头戴显示设备的显示画面跟随效果。
本公开的实施例中，可以参考当前时刻对应的姿态误差、头戴显示设备所在的移动平台所处的环境状况、移动平台的转向信息中的至少一项，合理确定校正比例系数，以使确定的校正比例系数与实际情况适配。
当然可以理解的，类似于截止频率的调整方案，用户也可以根据使用需求调整上述校正比例系数。
在图8所示实施例的基础上，如图10所示，该方法还包括步骤180和步骤190。
步骤180，基于姿态误差序列，确定姿态误差变化规律，姿态误差序列包括多个时刻各自对应的姿态误差。
需要说明的是，在通过执行步骤110确定出当前时刻对应的参考姿态之后，通过将当前时刻对应的参考姿态与预设姿态做差，可以确定当前时刻对应的姿态误差。按照类似的方式，可以针对每个时刻确定对应的姿态误差，这样可以得到与多个时刻一一对应的多个姿态误差。多个姿态误差可以按照所对应时刻由早至晚的顺序进行排列，以形成姿态误差序列。
可以理解地，在预设姿态与参考姿态为同一坐标系下的姿态，预设姿态为零旋转的姿态的情况下，可以基于参考姿态序列确定姿态误差变化规律。
在一个可选的例子中，姿态误差序列中可以包括顺序排列的20个姿态误差，分别是R1、R2、R3、……、R20，R1对应的时刻可以为t1，R2对应的时刻可以为t2，R3对应的时刻可以为t3，……，R20对应的时刻可以为t20。可选地，t20可以为当前时刻。
通过对姿态误差序列进行分析，可以确定姿态误差变化规律。姿态误差变化规律包括但不限于以下信息：姿态误差整体的变化趋势是增大趋势还是减小趋势；姿态误差的最大值和最小值分别对应哪个时刻；姿态误差有多少个极大值和多少个极小值；各个极大值和各个极小值分别对应哪个时刻；位于某一时刻(其可以认为是基准时刻)之后的每个时刻对应的姿态误差与基准时刻对应的姿态误差之间的大小关系。
步骤190，基于姿态误差变化规律，对校正比例系数进行调整。
需要说明的是，步骤190的实现形式多样，下面进行举例介绍。
在一种可选的实现形式中，如图11-1所示，步骤190，包括：
步骤1901，在基于姿态误差变化规律，确定目标时间段中的每个时刻对应的姿态误差均大于或等于目标时刻对应的姿态误差的情况下，增大校正比例系数。
目标时刻、目标时间段的起始时刻、目标时间段的结束时刻分别为多个时刻中的一个时刻，目标时间段的起始时刻晚于目标时刻，且目标时间段的时长大于或等于预设时长。
可选地，预设时长可以为3秒、4秒、5秒、8秒或者其他时长，在此不再一一列举。
在一个可选的例子中，姿态误差序列中可以包括顺序排列的20个姿态误差，分别是R1、R2、R3、……、R20，R1对应的时刻可以为t1，R2对应的时刻可以为t2，R3对应的时刻可以为t3，……，R20对应的时刻可以为t20。可选地，t20可以为当前时刻。
假设t1至t20中的t13作为目标时刻，则姿态误差变化规律中可以包括R14至R20中的每一者与R13之间的大小关系。假设任意相邻两个时刻间隔0.5秒，预设时长为3秒，目标时间段的起始时刻为t14，目标时间段的结束时刻为t20，则在R14至R20中的每一者均大于或等于R13的情况下，可以认为从t13这个时刻起，姿态误差持续偏高。也即，基于当前的校正比例系数进行姿态误差的校正时，校正效果不够理想，这时可以增大校正比例系数，例如可以将校正比例系数由0.5调节至0.6或0.7。通过校正比例系数的增大，可以更快速地依据预设姿态修正头戴显示设备的显示画面，从而有利于保证头戴显示设备的显示效果。
在另一种可选的实现形式中，如图11-2所示，步骤190，包括：
步骤1903，在基于姿态误差变化规律，确定目标时间段中的每个时刻对应的姿态误差均小于目标时刻对应的姿态误差的情况下，减小校正比例系数。
目标时刻、目标时间段的起始时刻、目标时间段的结束时刻分别为多个时刻中的一个时刻，目标时间段的起始时刻晚于目标时刻，且目标时间段的时长大于或等于预设时长。
可选地，预设时长可以为3秒、4秒、5秒、8秒或者其他时长，在此不再一一列举。
延续前一种实现形式中的例子，在R14至R20中的每一者均小于R13的情况下，可以认为从t13这个时刻起，姿态误差持续偏低。也即，基于当前的校正比例系数进行姿态误差的校正时，校正效果可以满足要求，这时可以减小校正比例系数，例如可以将校正比例系数由0.5调节至0.3或0.4。通过校正比例系数的减小，可以使得头戴显示设备的显示画面依据预设姿态进行修正的速度更为和缓，从而有利于保证头戴显示设备的显示画面稳定性。
在再一种可选的实现形式中，如图11-3所示，步骤190，包括：
步骤1905，在基于姿态误差变化规律，确定姿态误差的变化趋势为增大趋势的情况下，增大校正比例系数。
假设基于姿态误差变化规律，确定姿态误差的变化趋势为增大趋势，例如，延续第一种实现形式中的例子，如果R1、R2、R3、……、R20依次增大，或者虽然R1、R2、R3、……、R20并不是依次增大，但是整体趋势是逐渐增大(例如20个姿态误差中的前10个姿态误差依次增大，第11个姿态误差和第12个姿态误差相同，后8个姿态误差依次增大)，则可以增大校正比例系数。例如可以将校正比例系数由0.5调节至0.6或0.7。通过校正比例系数的增大，可以更快速地依据预设姿态修正头戴显示设备的显示画面，从而有利于保证头戴显示设备的显示效果。
在又一种可选的实现形式中，如图11-4所示，步骤190，包括：
步骤1907，在基于姿态误差变化规律，确定姿态误差的变化趋势为减小趋势的情况下，减小校正比例系数。
假设基于姿态误差变化规律，确定姿态误差的变化趋势为减小趋势，例如，延续第一种实现形式中的例子，如果R1、R2、R3、……、R20依次减小，或者虽然R1、R2、R3、……、R20并不是依次减小，但是整体趋势是逐渐减小(例如20个姿态误差中的前10个姿态误差依次减小，第11个姿态误差和第12个姿态误差相同，后8个姿态误差依次减小)，则 可以减小校正比例系数。例如可以将校正比例系数由0.5调节至0.3或0.4。通过校正比例系数的减小，可以使得头戴显示设备的显示画面依据预设姿态进行修正的速度更为和缓，从而有利于保证头戴显示设备的显示画面稳定性。
可见，本公开的实施例中，通过参考姿态误差变化规律，对校正比例系数进行调整，可以依据实际情况，以更为合适的速度进行头戴显示设备的显示画面修正，从而保证头戴显示设备的显示效果和显示画面稳定性。
在图1所示实施例的基础上，如图12所示，该方法还包括步骤102、步骤104和步骤106。
步骤102，识别头戴显示设备所在的平台的类型。
需要说明的是，识别头戴显示设备所在的平台的类型的实现形式多样，下面进行举例介绍。
在一种可选的实现形式中，头戴显示设备可以与所在的平台建立通信连接，在建立通信连接的过程中，可以获取该平台的相关信息。平台的相关信息包括但不限于标识信息、类型信息等。通过从获取的相关信息中提取类型信息，并识别该类型信息所表征的类别，可以确定该平台的类型是否为移动平台。
在另一种可选的实现形式中，头戴显示设备还可以包括摄像头，头戴显示设备可以通过摄像头采集自身所处环境的图像，对图像进行目标检测，以确定图像是否匹配车辆等移动平台。在匹配的情况下，则可以确定头戴显示设备所在的平台的类型为移动平台，否则，可以确定头戴显示设备所在的平台的类型不为移动平台。
在再一种可选的实现形式中，可以获取头戴显示设备的IMU中的加速度计采集的加速度数据或者陀螺仪采集的角速度数据并和预先设定的阈值进行对比。若基于加速度数据确定头戴显示设备的加速度大于设定的加速度阈值，或者，基于角速度数据确定头戴显示设备的角速度大于设定的角速度阈值，则可以确定头戴显示设备所在的平台的类型为移动平台。否则，可以确定头戴显示设备所在的平台的类型不为移动平台。
步骤104，在头戴显示设备所在的平台的类型为移动平台的情况下，控制头戴显示设备切换为目标模式。
本公开的实施例中，头戴显示设备可以包括至少三种模式，至少三种模式中可以包括上文中的0dof模式、3dof模式，以及步骤104中的目标模式。
步骤106，判断头戴显示设备是否处于目标模式，若是，触发步骤110。
本公开的实施例中，可以识别头戴显示设备所在的平台的类型，在该平台的类别不为移动平台的情况下，可以将头戴显示设备置于常规的0dof模式或者3dof模式。而在该平台的类别为移动平台的情况下，可以自动将头戴显示设备切换为目标模式，以在目标模式下，触发步骤110，实现上文中的步骤110至步骤140的执行。如上所述的，目标模式为介于0dof模式与3dof模式之间的一种模式。在目标模式下，在移动平台颠簸、晃动时，通过调整头戴显示设备的显示画面，可以补偿颠簸、晃动以保持画面稳定。并且，在移动平台或用户头部转向时，可以保持头戴显示设备的显示画面跟随用户头部而定位，从而能够提升用户的使用体验。
需要指出的是，以上描述的是在头戴显示设备位于移动平台上的情况下，自动将头戴显示设备切换为目标模式。当然用户也可以根据实际需求，手动选择将头戴显示设备置于0dof模式、3dof模式或者目标模式中的任一模式。
需要说明的是，本公开的实施例中涉及的移动平台也可以称为运动载具；本公开的实施例中的角速度数据也可以不由IMU中的陀螺仪采集，而由其它能够获得角速度数据的传感器采集。
综上，本公开的实施例中，在不利用IMU采集的加速度数据的前提下，通过利用IMU采集的角速度数据，可以令头戴显示设备的显示效果一直依据头戴显示设备在预设姿态下的显示效果做校正，对于头戴显示设备而言，不管是短时间内小角度的运动还是长时间的大角度的运动都能保证显示效果。
本公开的实施例提供的任一种用于在头戴显示设备显示画面的方法可以由任意适当的具有数据处理能力的设备执行，包括但不限于：终端设备和服务器等。或者，本公开实施例提供的任一种用于在头戴显示设备显示画面的方法可以由处理器执行，如处理器通过调用存储器存储的相应指令来执行本公开实施例提及的任一种用于在头戴显示设备显示画面的方法。下文不再赘述。
本公开的实施例还提供了一种用于在头戴显示设备显示画面的方法，该方法包括：获取头戴显示设备的角速度数据；基于所述角速度数据以及上一用于对所述头戴显示设备的显示画面进行处理的第一校正姿态，确定用于对待显示画面进行处理的参考姿态；对所述参考姿态进行处理，以得到校正姿态差值；利用所述校正姿态差值补偿所述参考姿态，以得到用于对待显示画面进行处理的第二校正姿态；基于所述第二校正姿态，对所述头戴显示设备的待显示画面进行渲染，以显示所述头戴显示设备的待显示画面。
本公开的实施例还提供了一种用于在头戴显示设备显示画面的方法，该方法包括：获取头戴显示设备的角速度数据，基于所述角速度数据以及上一用于对所述头戴显示设备的显示画面进行处理的第一校正姿态，确定用于对待显示画面进行处理的参考姿态；对所述参考姿态与用于对显示画面进行处理的预设姿态之间的姿态误差进行处理，以得到校正姿态差值；利用所述校正姿态差值补偿所述参考姿态，以得到用于对待显示画面进行处理的第二校正姿态；基于所述第二校正姿态，对所述头戴显示设备的待显示画面进行渲染，以显示所述头戴显示设备的待显示画面。
本公开的实施例还提供了一种用于在头戴显示设备显示画面的方法，该方法包括：基于所述头戴显示设备的当前时刻的角速度数据，以及在上一时刻用于对所述头戴显示设备的显示画面进行处理的第一校正姿态，确定在当前时刻用于对待显示画面进行处理的参考姿态；对所述参考姿态进行处理，以得到校正姿态差值；利用所述校正姿态差值补偿所述参考姿态，以得到在当前时刻用于对待显示画面进行处理的第二校正姿态；基于所述第二校正姿态，对所述头戴显示设备的待显示画面进行渲染，以显示所述头戴显示设备的待显示画面。
在一些可选的实施例中，响应于用户选择，将头戴显示设备显示画面模式切换为0自由度模式，目标模式，和3自由度模式中的一种。在头戴显示设备处于目标模式的情况下，执行基于头戴显示设备的当前时刻的角速度数据，以及在上一时刻用于对头戴显示设备的显示画面进行处理的第一校正姿态，确定在当前时刻用于对待显示画面进行处理的参考姿态的步骤。
在一些可选的实施例中，用于在头戴显示设备显示画面的方法还包括：接收用户输入操作；响应于用户输入操作，调整上述校正比例系数。
在一些可选的实施例中，用于在头戴显示设备显示画面的方法还包括：在利用校正比例系数，进行姿态误差的校正之后，可以基于补偿参考信息，确定校正比例系数，以便利用确定结果，进行校正比例系数的动态更新。
示例性装置
图13是本公开一示例性实施例提供的用于在头戴显示设备显示画面的装置的结构示意图，图13所示的装置包括第一确定模块1310、第一处理模块1320、补偿模块 1330和第二处理模块1340。
第一确定模块1310，用于基于头戴显示设备的当前时刻的角速度数据，以及在上一时刻用于对头戴显示设备的显示画面进行处理的第一校正姿态，确定在当前时刻用于对待显示画面进行处理的参考姿态；
第一处理模块1320，用于对参考姿态与用于对显示画面进行处理的预设姿态之间的姿态误差进行处理，以得到校正姿态差值；
补偿模块1330，用于利用校正姿态差值补偿参考姿态，以得到在当前时刻用于对待显示画面进行处理的第二校正姿态；
第二处理模块1340，用于基于第二校正姿态，对头戴显示设备的待显示画面进行渲染，以显示头戴显示设备的待显示画面。
在一个可选示例中，第一处理模块1320，用于：
对当前时刻对应的姿态误差进行低通滤波处理，以得到校正姿态差值。
在一个可选示例中，
第一处理模块1320，用于：
通过低通滤波器对当前时刻对应的姿态误差进行低通滤波处理；
如图14所示，该装置还包括：
接收模块1350，用于接收用户输入操作；
第一调整模块1360，用于响应于用户输入操作，对低通滤波器的截止频率进行调整。
在一个可选示例中，第一处理模块1320，用于：
利用校正比例系数校正当前时刻对应的姿态误差，以得到校正姿态差值，其中，校正比例系数大于0且小于1。
在一个可选示例中，如图15所示，该装置还包括：
第二确定模块1315，用于基于补偿参考信息，确定校正比例系数，补偿参考信息包括以下至少一项：当前时刻对应的姿态误差、姿态误差的累计值、头戴显示设备所在的移动平台所处的环境状况、移动平台的转向信息。
在一个可选示例中，以下四项中的至少一项满足：
校正比例系数与当前时刻对应的姿态误差呈正相关；
校正比例系数与姿态误差的累计值呈正相关；
环境状况包括路面平整度，校正比例系数与路面平整度呈正相关；
转向信息包括转向程度，校正比例系数与转向程度呈正相关。
在一个可选示例中，如图16所示，该装置还包括：
第三确定模块1380，用于基于姿态误差序列，确定姿态误差变化规律，姿态误差序列包括多个时刻各自对应的姿态误差；
第二调整模块1390，用于基于姿态误差变化规律，对校正比例系数进行调整。
在一个可选示例中，第二调整模块1390包括图16所示的第一调整单元13901、第二调整单元13903、第三调整单元13905和第四调整单元13907中的至少一项；
第一调整单元13901，用于在基于姿态误差变化规律，确定目标时间段中的每个时刻对应的姿态误差均大于或等于目标时刻对应的姿态误差的情况下，增大校正比例系数；
第二调整单元13903，用于在基于姿态误差变化规律，确定目标时间段中的每个时刻对应的姿态误差均小于目标时刻对应的姿态误差的情况下，减小校正比例系数；
第三调整单元13905，用于在基于姿态误差变化规律，确定姿态误差的变化趋势为增大趋势的情况下，增大校正比例系数；
第四调整单元13907，用于在基于姿态误差变化规律，确定姿态误差的变化趋势为减 小趋势的情况下，减小校正比例系数；
其中，目标时刻、目标时间段的起始时刻、目标时间段的结束时刻分别为多个时刻中的一个时刻，目标时间段的起始时刻晚于目标时刻，且目标时间段的时长大于或等于预设时长。
在一个可选示例中，
如图17所示，该装置还包括：
识别模块1302，用于识别头戴显示设备所在的平台的类型；
控制模块1304，用于在类型为移动平台的情况下，控制头戴显示设备切换为目标模式；
触发模块1306，用于在头戴显示设备处于目标模式的情况下，触发第一确定模块1310。
在一个可选示例中，头戴显示设备配置为与所在的移动平台藕接并相对移动平台可移动。
在一个可选示例中，以下两项中的一项满足：
预设姿态为用于使头戴显示设备的显示画面显示在头戴显示设备的坐标系中预设位置的姿态；
预设姿态为用于使头戴显示设备的显示画面与重力方向对齐的姿态。
在一个可选示例中，如图18所示，该装置还包括：
配置模块1395，用于基于第二校正姿态，配置头戴显示设备中的光机。
示例性电子设备
下面，参考图19来描述根据本公开实施例的电子设备。该电子设备可以是第一设备和第二设备中的任一个或两者、或与它们独立的单机设备，该单机设备可以与第一设备和第二设备进行通信，以从它们接收所采集到的输入信号。
图19图示了根据本公开实施例的电子设备1900的框图。
如图19所示，电子设备1900包括一个或多个处理器1910和存储器1920。
处理器1910可以是中央处理单元(CPU)或者具有数据处理能力和/或指令执行能力的其他形式的处理单元，并且可以控制电子设备1900中的其他组件以执行期望的功能。
存储器1920可以包括一个或多个计算机程序产品，所述计算机程序产品可以包括各种形式的计算机可读存储介质，例如易失性存储器和/或非易失性存储器。所述易失性存储器例如可以包括随机存取存储器(RAM)和/或高速缓冲存储器(cache)等。所述非易失性存储器例如可以包括只读存储器(ROM)、硬盘、闪存等。在所述计算机可读存储介质上可以存储一个或多个计算机程序指令，处理器1910可以运行所述程序指令，以实现上文所述的本公开的各个实施例的用于在头戴显示设备显示画面的方法以及/或者其他期望的功能。在所述计算机可读存储介质中还可以存储诸如输入信号、信号分量、噪声分量等各种内容。
在一个示例中，电子设备1900还可以包括：输入装置1930和输出装置1940，这些组件通过总线系统和/或其他形式的连接机构(未示出)互连。
例如，在电子设备1900是第一设备或第二设备时，该输入装置1930可以是麦克风或麦克风阵列。在电子设备1900是单机设备时，该输入装置1930可以是通信网络连接器，用于从第一设备和第二设备接收所采集的输入信号。
此外，该输入装置1930还可以包括例如键盘、鼠标等等。
该输出装置1940可以向外部输出各种信息。该输出装置1940可以包括例如显示器、扬声器、打印机、以及通信网络及其所连接的远程输出装置等等。
当然，为了简化，图19中仅示出了该电子设备1900中与本公开有关的组件中的一些， 省略了诸如总线、输入/输出接口等等的组件。除此之外，根据具体应用情况，电子设备1900还可以包括任何其他适当的组件。
示例性计算机程序产品和计算机可读存储介质
除了上述方法和设备以外，本公开的实施例还可以是计算机程序产品，其包括计算机程序指令，所述计算机程序指令在被处理器运行时使得所述处理器执行本说明书上述“示例性方法”部分中描述的根据本公开各种实施例的用于在头戴显示设备显示画面的方法中的步骤。
所述计算机程序产品可以以一种或多种程序设计语言的任意组合来编写用于执行本公开实施例操作的程序代码，所述程序设计语言包括面向对象的程序设计语言，诸如Java、C++等，还包括常规的过程式程序设计语言，诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算设备上执行、部分地在用户设备上执行、作为一个独立的软件包执行、部分在用户计算设备上部分在远程计算设备上执行、或者完全在远程计算设备或服务器上执行。
此外，本公开的实施例还可以是计算机可读存储介质，其上存储有计算机程序指令，所述计算机程序指令在被处理器运行时使得所述处理器执行本说明书上述“示例性方法”部分中描述的根据本公开各种实施例的用于在头戴显示设备显示画面的方法中的步骤。
所述计算机可读存储介质可以采用一个或多个可读介质的任意组合。可读介质可以是可读信号介质或者可读存储介质。可读存储介质例如可以包括但不限于电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。可读存储介质的更具体的例子(非穷举的列表)包括：具有一个或多个导线的电连接、便携式盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、光纤、便携式紧凑盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。
以上结合具体实施例描述了本公开的基本原理，需要指出的是，本公开中提及的优点、优势、效果等仅是示例而非限制，不能认为这些优点、优势、效果等是本公开的各个实施例必须具备的。上述公开的具体细节仅是为了示例和便于理解的作用，而非限制，上述细节并不限制本公开为必须采用上述具体的细节实现。
本说明书中各个实施例均采用递进的方式描述，每个实施例重点说明的都是与其它实施例的不同之处，各个实施例之间相同或相似的部分相互参见即可。对于系统实施例而言，由于其与方法实施例基本对应，所以描述的比较简单，相关之处参见方法实施例的部分说明即可。
本公开中涉及的器件、装置、设备、系统的方框图仅作为例示性的例子并且不意图要求或暗示必须按照方框图示出的方式进行连接、布置、配置。如本领域技术人员将认识到的，可以按任意方式连接、布置、配置这些器件、装置、设备、系统。诸如“包括”、“包含”、“具有”等等的词语是开放性词汇，指“包括但不限于”，且可与其互换使用。这里所使用的词汇“或”和“和”指词汇“和/或”，且可与其互换使用，除非上下文明确指示不是如此。这里所使用的词汇“诸如”指词组“诸如但不限于”，且可与其互换使用。
可能以许多方式来实现本公开的方法和装置。例如，可通过软件、硬件、固件或者软件、硬件、固件的任何组合来实现本公开的方法和装置。用于所述方法的步骤的上述顺序仅是为了进行说明，本公开的方法的步骤不限于以上具体描述的顺序，除非以其它方式特别说明。此外，在一些实施例中，还可将本公开实施为记录在记录介质中的程序，这些程序包括用于实现根据本公开的方法的机器可读指令。因而，本公开还覆盖存储用于执行根据本公开的方法的程序的记录介质。
还需要指出的是，在本公开的装置、设备和方法中，各部件或各步骤是可以分解和/或重新组合的。这些分解和/或重新组合应视为本公开的等效方案。
提供所公开的方面的以上描述以使本领域的任何技术人员能够做出或者使用本公开。对这些方面的各种修改对于本领域技术人员而言是非常显而易见的，并且在此定义的一般原理可以应用于其他方面而不脱离本公开的范围。因此，本公开不意图被限制到在此示出的方面，而是按照与在此公开的原理和新颖的特征一致的最宽范围。
为了例示和描述的目的已经给出了以上描述。此外，此描述不意图将本公开的实施例限制到在此公开的形式。尽管以上已经讨论了多个示例方面和实施例，但是本领域技术人员将认识到其某些变型、修改、改变、添加和子组合。

Claims (15)

  1. 一种用于在头戴显示设备显示画面的方法，包括：
    基于所述头戴显示设备的当前时刻的角速度数据，以及在上一时刻用于对所述头戴显示设备的显示画面进行处理的第一校正姿态，确定在当前时刻用于对待显示画面进行处理的参考姿态；
    对所述参考姿态与用于对显示画面进行处理的预设姿态之间的姿态误差进行处理，以得到校正姿态差值；
    利用所述校正姿态差值补偿所述参考姿态，以得到在当前时刻用于对待显示画面进行处理的第二校正姿态；
    基于所述第二校正姿态，对所述头戴显示设备的待显示画面进行渲染，以显示所述头戴显示设备的待显示画面。
  2. 根据权利要求1所述的方法，其中，所述对所述参考姿态与用于对显示画面进行处理的预设姿态之间的姿态误差进行处理，以得到校正姿态差值，包括：
    利用校正比例系数校正当前时刻对应的姿态误差，以得到校正姿态差值，其中，所述校正比例系数大于0且小于1。
  3. 根据权利要求2所述的方法，还包括：
    基于补偿参考信息，确定所述校正比例系数，所述补偿参考信息包括以下至少一项：当前时刻对应的姿态误差、姿态误差的累计值、所述头戴显示设备所在的移动平台所处的环境状况、所述移动平台的转向信息。
  4. 根据权利要求3所述的方法，其中，以下四项中的至少一项满足：
    所述校正比例系数与当前时刻对应的姿态误差呈正相关；
    所述校正比例系数与所述姿态误差的累计值呈正相关；
    所述环境状况包括路面平整度，所述校正比例系数与所述路面平整度呈正相关；
    所述转向信息包括转向程度，所述校正比例系数与所述转向程度呈正相关。
  5. 根据权利要求2所述的方法，还包括：
    基于姿态误差序列，确定姿态误差变化规律，所述姿态误差序列包括多个时刻各自对应的姿态误差；
    基于所述姿态误差变化规律，对所述校正比例系数进行调整。
  6. 根据权利要求5所述的方法，其中，所述基于所述姿态误差变化规律，对所述校正比例系数进行调整，包括以下四项中的至少一项：
    在基于所述姿态误差变化规律，确定目标时间段中的每个时刻对应的姿态误差均大于或等于目标时刻对应的姿态误差的情况下，增大所述校正比例系数；
    在基于所述姿态误差变化规律，确定目标时间段中的每个时刻对应的姿态误差均小于目标时刻对应的姿态误差的情况下，减小所述校正比例系数；
    在基于所述姿态误差变化规律，确定姿态误差的变化趋势为增大趋势的情况下，增大所述校正比例系数；
    在基于所述姿态误差变化规律，确定姿态误差的变化趋势为减小趋势的情况下，减小所述校正比例系数；
    其中，所述目标时刻、所述目标时间段的起始时刻、所述目标时间段的结束时刻分别为所述多个时刻中的一个时刻，所述目标时间段的起始时刻晚于所述目标时刻，且所述目标时间段的时长大于或等于预设时长。
  7. 根据权利要求2所述的方法，还包括：
    接收用户输入操作；
    响应于用户输入操作，调整所述校正比例系数。
  8. 根据权利要求1所述的方法，其中，所述对所述参考姿态与用于对显示画面进行处理的预设姿态之间的姿态误差进行处理，以得到校正姿态差值，包括：
    对当前时刻对应的姿态误差进行低通滤波处理，以得到所述校正姿态差值。
  9. 根据权利要求8所述的方法，其中，
    所述对当前时刻对应的姿态误差进行低通滤波处理，包括：
    通过低通滤波器对当前时刻对应的姿态误差进行低通滤波处理；
    所述方法还包括：
    接收用户输入操作；
    响应于所述用户输入操作，对所述低通滤波器的截止频率进行调整。
  10. 根据权利要求1至9中任一项所述的方法，还包括：
    响应于用户选择，将头戴显示设备显示画面模式切换为0自由度模式，目标模式，和3自由度模式中的一种，
    在所述头戴显示设备处于所述目标模式的情况下，执行所述基于所述头戴显示设备的当前时刻的角速度数据，以及在上一时刻用于对所述头戴显示设备的显示画面进行处理的第一校正姿态，确定在当前时刻用于对待显示画面进行处理的参考姿态的步骤。
  11. 根据权利要求1至9中任一项所述的方法，还包括：
    识别所述头戴显示设备所在的平台的类型；
    在所述类型为移动平台的情况下，控制所述头戴显示设备切换为目标模式；
    在所述头戴显示设备处于所述目标模式的情况下，执行所述基于所述头戴显示设备的当前时刻的角速度数据，以及在上一时刻用于对所述头戴显示设备的显示画面进行处理的第一校正姿态，确定在当前时刻用于对待显示画面进行处理的参考姿态的步骤。
  12. 根据权利要求1至9中任一项所述的方法，其中，所述头戴显示设备配置为与所在的移动平台藕接并相对所述移动平台可移动。
  13. 根据权利要求1至12中任一项所述的方法，其中，以下两项中的至少一项满足：
    所述预设姿态为用于使所述头戴显示设备的显示画面显示在所述头戴显示设备的坐标系中预设位置的姿态；
    所述预设姿态为用于使所述头戴显示设备的显示画面与重力方向对齐的姿态。
  14. 一种电子设备，包括：
    存储器，用于存储计算机程序产品；
    处理器，用于执行所述存储器中存储的计算机程序产品，且所述计算机程序产品被执行时，实现上述权利要求1至13中任一项所述的用于在头戴显示设备显示画面的方法。
  15. 一种计算机可读存储介质，其上存储有计算机程序指令，其特征在于，所述计算机程序指令被处理器执行时，实现上述权利要求1至13中任一项所述的用于在头戴显示设备显示画面的方法。
PCT/CN2023/093339 2022-05-11 2023-05-10 用于在头戴显示设备显示画面的方法、装置及电子设备 WO2023217198A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210510750.8A CN114979615A (zh) 2022-05-11 2022-05-11 用于在头戴显示设备显示画面的方法、装置及电子设备
CN202210510750.8 2022-05-11

Publications (1)

Publication Number Publication Date
WO2023217198A1 true WO2023217198A1 (zh) 2023-11-16

Family

ID=82980607

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/093339 WO2023217198A1 (zh) 2022-05-11 2023-05-10 用于在头戴显示设备显示画面的方法、装置及电子设备

Country Status (2)

Country Link
CN (1) CN114979615A (zh)
WO (1) WO2023217198A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114979615A (zh) * 2022-05-11 2022-08-30 闪耀现实(无锡)科技有限公司 用于在头戴显示设备显示画面的方法、装置及电子设备

Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2011257342A (ja) * 2010-06-11 2011-12-22 Nsk Ltd ヘッドトラッキング装置及びヘッドトラッキング方法
EP3054266A1 (en) * 2013-12-25 2016-08-10 Sony Corporation Orientation measurement device, orientation measurement method, image processing device, image processing method, display device, display method, computer program, and image display system
CN106998409A (zh) * 2017-03-21 2017-08-01 华为技术有限公司 一种图像处理方法、头戴显示器以及渲染设备
WO2017156741A1 (zh) * 2016-03-17 2017-09-21 深圳多哚新技术有限责任公司 一种头部姿态补偿方法及相关设备
CN107204044A (zh) * 2016-03-17 2017-09-26 深圳多哚新技术有限责任公司 一种基于虚拟现实的画面显示方法及相关设备
US20180253142A1 (en) * 2015-11-02 2018-09-06 Sony Interactive Entertainment Inc. Information Processing Apparatus, Information Processing System, And Information Processing Method
US20200098302A1 (en) * 2018-09-25 2020-03-26 Chengdu Boe Optoelectronics Technology Co., Ltd. Method of compensating in display panel, driving unit and display panel
CN112308981A (zh) * 2020-11-09 2021-02-02 Oppo广东移动通信有限公司 图像处理方法、装置、电子设备及存储介质
US20210124413A1 (en) * 2018-03-29 2021-04-29 Displaylink (Uk) Limited Position error measurement in an extended reality mobile display device
CN113888685A (zh) * 2021-09-29 2022-01-04 青岛歌尔声学科技有限公司 一种头戴设备的控制方法、一种画面渲染方法
CN114979615A (zh) * 2022-05-11 2022-08-30 闪耀现实(无锡)科技有限公司 用于在头戴显示设备显示画面的方法、装置及电子设备

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP6533590B2 (ja) * 2015-10-14 2019-06-19 アルプスアルパイン株式会社 ウェアラブル装置とその姿勢測定方法及びプログラム
CN107560637B (zh) * 2017-08-21 2021-02-19 歌尔光学科技有限公司 头戴显示设备校准结果验证方法及头戴显示设备
CN107801014B (zh) * 2017-10-25 2019-11-08 深圳岚锋创视网络科技有限公司 一种全景视频防抖的方法、装置及便携式终端
US10768695B2 (en) * 2019-02-01 2020-09-08 Facebook Technologies, Llc Artificial reality system having adaptive degrees of freedom (DOF) selection
JP7150134B2 (ja) * 2019-02-22 2022-10-07 株式会社ソニー・インタラクティブエンタテインメント ヘッドマウントディスプレイおよび画像表示方法
CN110850961B (zh) * 2019-09-30 2023-11-28 深圳市瑞立视多媒体科技有限公司 一种头戴式显示设备的校准方法及头戴式显示设备
WO2021083584A1 (en) * 2019-10-29 2021-05-06 Holoride Gmbh Method and control device for operating a virtual reality headset in a vehicle
CN112015269A (zh) * 2020-08-03 2020-12-01 深圳市瑞立视多媒体科技有限公司 头显设备的显示校正方法、设备及存储介质

Patent Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2011257342A (ja) * 2010-06-11 2011-12-22 Nsk Ltd ヘッドトラッキング装置及びヘッドトラッキング方法
EP3054266A1 (en) * 2013-12-25 2016-08-10 Sony Corporation Orientation measurement device, orientation measurement method, image processing device, image processing method, display device, display method, computer program, and image display system
US20180253142A1 (en) * 2015-11-02 2018-09-06 Sony Interactive Entertainment Inc. Information Processing Apparatus, Information Processing System, And Information Processing Method
WO2017156741A1 (zh) * 2016-03-17 2017-09-21 深圳多哚新技术有限责任公司 一种头部姿态补偿方法及相关设备
CN107204044A (zh) * 2016-03-17 2017-09-26 深圳多哚新技术有限责任公司 一种基于虚拟现实的画面显示方法及相关设备
CN106998409A (zh) * 2017-03-21 2017-08-01 华为技术有限公司 一种图像处理方法、头戴显示器以及渲染设备
US20210124413A1 (en) * 2018-03-29 2021-04-29 Displaylink (Uk) Limited Position error measurement in an extended reality mobile display device
US20200098302A1 (en) * 2018-09-25 2020-03-26 Chengdu Boe Optoelectronics Technology Co., Ltd. Method of compensating in display panel, driving unit and display panel
CN112308981A (zh) * 2020-11-09 2021-02-02 Oppo广东移动通信有限公司 图像处理方法、装置、电子设备及存储介质
CN113888685A (zh) * 2021-09-29 2022-01-04 青岛歌尔声学科技有限公司 一种头戴设备的控制方法、一种画面渲染方法
CN114979615A (zh) * 2022-05-11 2022-08-30 闪耀现实(无锡)科技有限公司 用于在头戴显示设备显示画面的方法、装置及电子设备

Also Published As

Publication number Publication date
CN114979615A (zh) 2022-08-30

Similar Documents

Publication Publication Date Title
US10330940B1 (en) Content display methods
JP5869177B1 (ja) 仮想現実空間映像表示方法、及び、プログラム
EP2616907B1 (en) Control of applications on a head-mounted display using gesture and motion commands
EP3449624B1 (en) Electronic image stabilization frequency estimator
KR102612988B1 (ko) 디스플레이 장치 및 디스플레이 장치의 영상 처리 방법
US20150130839A1 (en) Display control device, display control method, and program
WO2019126958A1 (zh) 偏航姿态控制方法、无人机、计算机可读存储介质
WO2023217198A1 (zh) 用于在头戴显示设备显示画面的方法、装置及电子设备
WO2022022141A1 (zh) 图像显示方法、装置、计算机设备及存储介质
CN108881703A (zh) 防抖控制方法和装置
CN106445334B (zh) 显示界面的调节方法及调节系统
CN109302563B (zh) 防抖处理方法、装置、存储介质及移动终端
WO2020075825A1 (ja) 動き推定装置、電子機器、制御プログラム及び動き推定方法
US20180024661A1 (en) Method for performing display stabilization control in an electronic device with aid of microelectromechanical systems, and associated apparatus
CN108196701B (zh) 确定姿态的方法、装置及vr设备
KR102521557B1 (ko) 스크롤 입력에 기반하여 이미지 표시를 제어하는 전자 장치 및 방법
CN114040113A (zh) 图像处理方法及其装置
US20190212834A1 (en) Software gyroscope apparatus
CN111429519B (zh) 三维场景显示方法、装置、可读存储介质及电子设备
CN110035231B (zh) 一种拍摄方法、装置、设备和介质
WO2017156741A1 (zh) 一种头部姿态补偿方法及相关设备
CN113204285A (zh) 具有快速启动恢复的指向电子设备和对应方法
CN114201028A (zh) 扩增实境系统与其锚定显示虚拟对象的方法
US20240126369A1 (en) Information processing system and information processing method
US11847750B2 (en) Smooth object correction for augmented reality devices

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23802967

Country of ref document: EP

Kind code of ref document: A1