WO2023056803A1 - Procédé et appareil de présentation holographique - Google Patents

Procédé et appareil de présentation holographique Download PDF

Info

Publication number
WO2023056803A1
WO2023056803A1 PCT/CN2022/116191 CN2022116191W WO2023056803A1 WO 2023056803 A1 WO2023056803 A1 WO 2023056803A1 CN 2022116191 W CN2022116191 W CN 2022116191W WO 2023056803 A1 WO2023056803 A1 WO 2023056803A1
Authority
WO
WIPO (PCT)
Prior art keywords
user
display device
frequency
head
control pen
Prior art date
Application number
PCT/CN2022/116191
Other languages
English (en)
Chinese (zh)
Inventor
谢国勇
刘嘉兴
许秋子
Original Assignee
深圳市瑞立视多媒体科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市瑞立视多媒体科技有限公司 filed Critical 深圳市瑞立视多媒体科技有限公司
Publication of WO2023056803A1 publication Critical patent/WO2023056803A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30—Image reproducers
    • H04N13/398—Synchronisation thereof; Control thereof
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30—Image reproducers
    • H04N13/332—Displays for viewing with the aid of special glasses or head-mounted displays [HMD]
    • H04N13/344—Displays for viewing with the aid of special glasses or head-mounted displays [HMD] with head-mounted left-right displays
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30—Image reproducers
    • H04N13/366—Image reproducers using viewer tracking
    • H04N13/383—Image reproducers using viewer tracking for tracking with gaze detection, i.e. detecting the lines of sight of the viewer's eyes

Definitions

  • the present application relates to the technical field of data processing, and in particular to a holographic display method and device.
  • Holographic display technology (Front-Projected Holographic Display), also known as virtual imaging technology, is a technology that uses the principles of interference and diffraction to record and reproduce real three-dimensional images of objects. , glasses and other auxiliary devices to watch the advantages of three-dimensional images. With the continuous development of display technology, holographic display technology has gained more and more attention.
  • the present application provides a holographic display method and device to solve the problem in the prior art that it is difficult to realize the interaction between a user and a holographic image.
  • the present application provides a holographic display method, the method is based on a holographic display system, and the holographic display system includes: a display processing device, a sand table display device, a head-mounted display device and an interactive control pen;
  • the display processing device is electrically connected to the sand table display device, the head-mounted display device, and the interactive control pen respectively;
  • the holographic display method is executed by the display processing device, and the method includes:
  • the viewpoint information shows the binocular position of the first user and the binocular angle of view of the first user, and the first user holds the interactive control pen user;
  • the head-mounted display device adjusts the output frequency of the head-mounted display device to the first frequency; if the first angle is not greater than the first angle threshold, the head-mounted display The output frequency of the display device is adjusted to a second frequency, wherein the first frequency is greater than the second frequency.
  • the output frequency of the head-mounted display device is adjusted to the first frequency; if the first included angle is smaller than the The first angle threshold, then adjusting the output frequency of the head-mounted display device to a second frequency, before the first frequency is greater than the second frequency, the method further includes:
  • the users include the first user and a second user, and the second user is a user who wears the head-mounted display device and does not hold the interactive control pen;
  • the output frequency of the head-mounted display device is adjusted to the first frequency; if the first included angle is smaller than the The first angle threshold, then adjusting the output frequency of the head-mounted display device to a second frequency, before the first frequency is greater than the second frequency, the method further includes:
  • viewpoint information of each second user wherein the viewpoint information shows the binocular position of the second user and the binocular angle of view of the second user, and the second user wears the headset A user of a display device who does not hold the interactive control pen;
  • the specified environment is the environment where the sand table display device is located
  • the second included angle between the pointing and the specified line is greater than the second angle threshold, then adjust the output frequency of the sand table display device to a third frequency, if the second included angle is not greater than the The second angle threshold is to adjust the output frequency of the sand table display device to a fourth frequency, wherein the third frequency is greater than the fourth frequency.
  • the viewpoint information includes six degrees of freedom information of a head-mounted display device worn by the first user, wherein the viewpoint information is obtained according to a viewpoint information tracking algorithm.
  • the pose information of the interactive control pen includes six degrees of freedom information of the interactive control pen, wherein the position three degrees of freedom in the six degrees of freedom information of the interactive control pen
  • the information is obtained according to the image information collected by the dynamic auxiliary camera, and the rotation 3-degree-of-freedom information in the six-degree-of-freedom information of the interactive control pen is obtained according to the data collected by the IMU module in the interactive control pen.
  • the auxiliary camera is set on the sand table display device.
  • the head-mounted display device includes shutter-type active 3D glasses and at least three reflective marking points embedded in the casing of the shutter-type active 3D glasses.
  • the shutter type active 3D glasses are used to receive the holographic image transmitted by the sand table type display device, so as to obtain the correct left and right eye images.
  • the head-mounted display device is provided with a first pressure sensor, and when the head-mounted display device is worn on the user's head, the first pressure sensor generates the first A signal is sent to the display processing device;
  • the interactive control pen is provided with an IMU module, and when the posture of the interactive control pen changes, a second signal is generated and sent to the display processing device; wherein, obtaining Viewpoint information of the first user, including:
  • the viewpoint information of the first user is acquired.
  • the present application provides a holographic display device, including:
  • the first acquisition module is configured to: acquire the viewpoint information of the first user, wherein the viewpoint information shows the binocular position of the first user and the binocular angle of view of the first user, and the first user is a user holding the interactive control pen;
  • the second acquisition module is configured to: acquire the pose information of the interactive control pen, wherein the pose information shows the direction of the interactive control pen;
  • a first included angle determination module configured to: determine a first included angle between the viewing angle and the orientation
  • the first adjustment module is configured to: adjust the output frequency of the head-mounted display device to the first frequency if the first included angle is greater than the first angle threshold; if the first included angle is not greater than the first angle threshold, adjust the output frequency of the head-mounted display device to a second frequency, wherein the first frequency is greater than the second frequency.
  • the present application provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory complete mutual communication through the communication bus;
  • the processor is configured to implement the steps of any holographic display method in the first aspect when executing the program stored in the memory.
  • the present application provides a computer-readable storage medium on which a computer program is stored, wherein when the computer program is executed by a processor, the steps of any holographic presentation method in the first aspect are implemented.
  • the interaction between the first user and the holographic image is mainly realized through the interactive control pen.
  • An included angle to adjust the output frequency of the head-mounted display device If the first included angle is greater than the first angle threshold, it indicates that there is a large position difference between the parts of the content that the first user wishes to interact with in the holographic image. and/or angle differences may cause the first user to be unable to accurately select the part of the holographic image that he wishes to interact with. The part of the image that you want to interact with.
  • the first included angle is not greater than the first angle threshold, it indicates that the first user has good interaction conditions with the part of the content that he wants to interact with.
  • reducing the output frequency of the head-mounted display device can make more Users participate in the viewing of holographic images, which is conducive to the reasonable allocation of holographic image display resources.
  • the method in this application is applicable to teaching scenarios for multiple people.
  • FIG. 1 is a schematic diagram of a scene involved in a holographic presentation process provided by an embodiment of the present application
  • FIG. 2 is a schematic flowchart of a holographic display process provided by the embodiment of the present application.
  • FIG. 3 is a schematic diagram of a holographic display device corresponding to some steps in the process of Fig. 2;
  • FIG. 4 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.
  • the process in this specification is based on a holographic display system
  • the holographic display system includes: a display processing device, a sand table display device, a head-mounted display device, and an interactive control pen; the display processing device is connected with the sand table display device, the The head-mounted display device and the interactive control pen are electrically connected respectively, and an exemplary scene is shown in FIG. 1 .
  • the head-mounted display device can be in one-to-one correspondence with the user.
  • different head-mounted display devices output pictures to the user at the same frequency.
  • the display processing device outputs display signals to each head-mounted display device respectively.
  • the viewpoint information shows the binocular position of the first user and the binocular angle of view of the first user, and the first user is a user holding the interactive control pen.
  • the viewpoint information includes six degrees of freedom information of a head-mounted display device worn by the first user, wherein the viewpoint information is obtained according to a viewpoint information tracking algorithm.
  • the head-mounted display device includes shutter-type active 3D glasses and at least three reflective marking points, the at least three reflective marking points are embedded on the shell structure of the shutter-type active 3D glasses, and the shutter-type active The 3D glasses are used to receive the holographic images transmitted by the sand table display device, so as to obtain the correct images for the left and right eyes.
  • the process of determining the first user among the users may be as follows. For each head-mounted display device, determine the first position of the head-mounted display device in the specified environment (ie, the binocular position of the user wearing the head-mounted display device). And determine the second position of the interactive control pen in the specified environment. Among the head-mounted display devices, the head-mounted display devices whose first position is within a specified distance range (preset range, which can be 0.5 meters to 1.2 meters) from the second position are determined as the first pending devices.
  • a specified distance range preset range, which can be 0.5 meters to 1.2 meters
  • the first pending device For each first pending device, determine the first included angle corresponding to the first pending device. Determine the first undetermined device corresponding to the first included angle not greater than the third angle threshold (the third angle threshold is a preset value, greater than the first angle threshold), as the second undetermined device. Among the second undetermined devices, the second undetermined device with the closest distance to the designated point is determined as the head-mounted display device worn by the first user.
  • the third angle threshold is a preset value, greater than the first angle threshold
  • the pose information of the interactive control pen shows the direction of the interactive control pen.
  • the pose information of the interactive control pen includes six degrees of freedom information of the interactive control pen, wherein the position three-degree-of-freedom information in the six-degree-of-freedom information of the interactive control pen is Obtained according to the image information collected by the dynamic-assisted camera, the rotation 3-degree-of-freedom information in the six-degree-of-freedom information of the interactive control pen is obtained according to the data collected by the IMU module in the interactive control pen, the dynamic-assisted camera It is arranged on the sand table type display device.
  • angles such as the first included angle are determined.
  • a three-dimensional coordinate system may be established in advance in a designated environment, and the coordinate origin of the three-dimensional coordinate system is the center of the display screen of the sand table display device.
  • the output frequency of the head-mounted display device is adjusted according to the first included angle between the viewing angle of the first user and the direction of the interactive control pen.
  • the first included angle is greater than the first angle threshold, it means There is a large position difference and/or angle difference between the parts of the content that the first user wishes to interact with in the holographic image, which may cause the first user to be unable to accurately select the part of the content that the first user wishes to interact with in the holographic image.
  • increasing the output frequency of the head-mounted display device is beneficial for the first user to recognize part of the content in the holographic image that he wishes to interact with.
  • the first included angle is not greater than the first angle threshold, it indicates that the first user has good interaction conditions with the part of the content that he wants to interact with. At this time, reducing the output frequency of the head-mounted display device can make more Users participate in the viewing of holographic images, which is conducive to the reasonable allocation of holographic image display resources.
  • the number of users is acquired (the number of users may be acquired in real time, so as to adjust the first frequency in real time).
  • the user includes the first user and a second user
  • the second user is a user who wears the head-mounted display device and does not hold the interactive control pen. If the number of users is greater than the number threshold, adjust the value of the first frequency to a first value, and if the number of users is not greater than the number threshold, then adjust the value of the first frequency to a second value , wherein the first value is smaller than the second value.
  • Both the first value and the second value can be preset values, which can be set by the administrator of the holographic display system.
  • this specification not only adjusts the output frequency of the head-mounted display device to rationally allocate display resources, but also can reduce resource consumption while satisfying viewing needs of users.
  • the viewpoint information of each second user is obtained (it may be obtained in real time, so as to adjust the output frequency of the sand table display device in real time).
  • the viewpoint information shows the binocular position of the second user and the binocular angle of view of the second user, and the second user wears the head-mounted display device and does not hold the The user who controls the pen interactively. Determining the density center of the distribution of the binocular positions of each second user in a specified environment, wherein the specified environment is the environment where the sand table display device is located.
  • a line connecting the density center and a designated point is determined as a designated line, wherein the designated point is an intersection of the pointing point of the interactive control pen and the holographic image. If the second included angle between the pointing and the specified line is greater than the second angle threshold, then adjust the output frequency of the sand table display device to a third frequency, if the second included angle is not greater than the The second angle threshold is to adjust the output frequency of the sand table display device to a fourth frequency, wherein the third frequency is greater than the fourth frequency.
  • the head-mounted display device in order to determine whether to interact with the user, is provided with a first pressure sensor, and the head-mounted display device is worn on the head of the user.
  • the first pressure sensor When it is on, the first pressure sensor generates a first signal and sends it to the display processing device; the interactive control pen is provided with an IMU module, and when the posture of the interactive control pen changes, a second signal is generated, sent to the display processing device; wherein, obtaining the viewpoint information of the first user includes: if the first signal is detected at a specified moment, and the second signal is detected within a specified time range from the specified moment signal, the viewpoint information of the first user is acquired.
  • shutter-type active 3D glasses with a refresh rate of at least 240 Hz are required and support multi-viewpoint display functions.
  • the refresh rate of existing shutter-type active 3D glasses on the market is only 120Hz, and does not support multi-viewpoint display function.
  • the receiving frequency of the interface chip at the input end of the sending device cannot reach the minimum standard of 240 Hz.
  • the sending device communicates with the 3D display screen through a network cable, and the transmission frequency can be much higher than 240Hz.
  • this manual adopts an interface chip with a 4K bandwidth and a refresh rate of 60Hz, and the amount of input data is equivalent to 1080P with a bandwidth of 240Hz.
  • the sending device in this manual cuts the received 4K picture into 4 (corresponding to two shutter-type active 3D glasses) 1080P bandwidth pictures and transmits them to the LED screen receiving card for display at a refresh rate of 240Hz or higher. It is equivalent to realizing the LED display system capable of displaying 240Hz video stream.
  • multiple shutter-type active 3D glasses receive images from the same 3D display screen. For example, in the order of left 1, left 2, right 1, and right 2, two matching images are presented at an interval of 4.166ms. A total of 4 pictures of the shutter-type active 3D glasses.
  • the shutter-type active 3D glasses itself supports a display refresh rate of 120Hz, but the synchronization signal is a square wave signal, a 60Hz square wave signal, a single cycle already includes a peak half cycle and a valley half cycle, and there are 120 state switches per second , which can meet the 120Hz display refresh rate of ordinary shutter-type active 3D glasses.
  • this synchronization method can effectively distinguish the left and right eyes, for example, the peak is defined as the left eye, and the trough is defined as the right eye. Then, for a 240Hz 3D image display system, the frequency of the synchronous signal is 60Hz. Under the 60Hz synchronous signal, the peak and trough can still be used to distinguish the left and right eyes.
  • the shutter-type active 3D glasses are shutter-type active 3D glasses with adjustable signal delay. Every time a peak signal is received, the left eye becomes transparent for 4.166ms (the peak signal will last for 8.333ms, this The time-varying transparent state lasts only 4.166ms. When it becomes transparent, you can receive the picture on the 3D display screen. When it is not transparent, the display area can display black), and the right eye becomes transparent every time a trough is received 4.166 ms. Then adjust the signal delay of one of the glasses to 4.166ms to distinguish No. 1 and No. 2 glasses.
  • the way of interaction may be to enlarge, shrink, move, rotate, select, and replace the holographic image output by the sand table display device.
  • the sand table display device is an LED (Light Emitting Diode, light emitting diode) display device.
  • the image specification corresponding to the first display signal is 1920 pixels*1080 pixels, and the output frequency of the first display signal is 120 Hz, so as to realize alternately output images to the binoculars of the user at a frequency of 60 Hz.
  • the sand table display device After receiving the first display signal, the sand table display device outputs a holographic image according to the first display signal for viewing by a user wearing the head-mounted display device.
  • the display processing device In order to enable the sand table display device and the head-mounted display device to cooperate, the display processing device generates a synchronization signal according to the first display signal of the left eye and the first display signal of the right eye, so that the head-mounted display device collects
  • the holographic images output by the sand table display device are used to alternately display the collected holographic images to the user wearing the head-mounted display device. It can be seen that the synchronous signal is used to realize alternate binocular output images for the user.
  • the head-mounted display device is shutter-type 3D glasses.
  • the holographic display system in this specification provides holographic image display for multiple users at the same time, since the viewpoint of each user (that is, the viewpoint of the head-mounted display device worn by the user) is different to varying degrees,
  • the display processing device in this specification determines its viewpoint for each head-mounted display device. Since the number of head-mounted display devices is not unique, and the first display signal received by the sand table display device is not unique, the sand table display device needs to send information to different head-mounted display devices according to different first display signals.
  • Holographic display since the number of head-mounted display devices is not unique, and the first display signal received by the sand table display device is not unique, the sand table display device needs to send information to different head-mounted display devices according to different first display signals.
  • the holographic image displayed by the sand table display device will also cooperate with an interactive handle to provide users with a more interactive experience. Specifically, the display processing device judges whether the contour of the holographic image displayed by the sand table display device intersects with the indicating end of the interactive handle. If not, it means that the pointing end of the interactive handle is not pointing at the holographic image, causing the position indicated by the interactive handle on the holographic image to be unclear, then the holographic image displayed by the sand table display device can be enlarged until the outline of the holographic image is consistent with the holographic image. Intersect the indicating end of the interaction handle described above.
  • the holographic display in this description requires the cooperation of a sand table display device and a head-mounted display device. This coordination is achieved to a certain extent by means of synchronization signals.
  • a synchronous signal can be used to control all head-mounted display devices; in another optional embodiment of this specification, each head-mounted display device can be controlled separately The devices generate synchronous signals separately to control different head-mounted display devices separately.
  • the synchronization signal controls the head-mounted display device to collect the content displayed by the sand table display device when the sand table display device displays the holographic image under the head-mounted display device's viewpoint. , and display it to the user according to the collected results.
  • the head-mounted display device collects the holographic image output by the sand table display device according to the synchronization signal, and alternately displays the holographic image to the user wearing the head-mounted display device. Collected holographic images.
  • the holographic display method in this manual adopts the method of cooperating with the sand table display device and the head-mounted display device to display holographic images to the user.
  • the sand table display device is highly controllable, and the brightness of its display can be adjusted according to the The brightness of the scene is adjusted, and even in a dark scene, a better holographic image display effect can be obtained. Since the holographic images in this description are assisted by a sand table display device, compared with the existing VR glasses, the head-mounted display device in this manual does not need to perform more complicated data processing, which is beneficial to reduce the size of the head-mounted display. The volume and weight of the device make the user feel more comfortable.
  • the holographic display method in this specification adopts the combination of sand table display device and head-mounted display device, even if the user's point of view changes (which may be caused by the user's movement), it can also show the user a three-dimensional image.
  • the holographic image is conducive to improving the user experience.
  • the display processing device in the holographic display system controls the display of the sand table display device according to the viewpoint of the head mounted display device. content, and collect holographic images suitable for users to watch.
  • the holographic display system includes multiple head-mounted display devices. When different head-mounted display devices are worn by different users, the viewpoints of different head-mounted display devices will be different. Through the method in this specification, it is possible to realize Through the same sand table display device, multiple head-mounted display devices with different viewpoints can simultaneously display holographic images.
  • the display processing device After the display processing device generates the first display signals corresponding to each viewpoint, it performs Combining processing: outputting the composite signal obtained after the combining processing to the sand table display device.
  • the image specification corresponding to the composite signal is 1920 pixels*1080 pixels.
  • the output frequency of the composite signal is 120*k Hz.
  • k is the number of head-mounted display devices.
  • the output frequency of the composite signal is 240 Hz.
  • the holographic display system may also include receive card. After the display processing device generates the composite signal, it outputs the composite signal to the receiving card. The receiving card restores the composite signal to each first display signal, and drives the sand table type display device to display the holographic image according to each first display signal.
  • the receiving card outputs each first display signal to the sand table display device in time division, so that the sand table display device performs time division display of the holographic image according to each first display signal.
  • the sand table display device performs holographic image display according to the first display signal corresponding to the i-th head-mounted display device (i can be any integer, indicating any one)
  • the i-th head-mounted display device displays the sand table according to the synchronization signal. If the holographic image displayed by the head-mounted display device is collected, the collected result matches the viewpoint of the i-th head-mounted display device.
  • the receiving card in this specification has a certain interface function to cooperate with multiple sand table display devices to display holographic images simultaneously, and/or cooperate with different types of sand table display devices to display holographic images.
  • the holographic display method in this specification it is possible to display holographic images with strong stereoscopic effect and high image quality for users in various scenes with different brightness.
  • the user may move in the site where the sand table display device is installed according to his own needs, and there may be a mismatch between the holographic image displayed by the sand table display device and the user's actual viewpoint due to the user's movement.
  • the viewpoints of the multiple users are also different to some extent.
  • the holographic display system also includes: a motion capture camera .
  • the motion capture camera is arranged on the sand table display device.
  • the number of motion capture cameras in this manual, the installation positions and shooting angles on the sand table display device are determined according to the setting, shape, size, etc. of the sand table display device. Specifically, the number of motion capture cameras is four, which are respectively arranged at the corners of the sand table display device.
  • the motion capture camera is used for image acquisition (shooting) of the environment around the sand table display device, so as to capture the position of the user in the scene, and then obtain the viewpoint of the head-mounted display device.
  • the motion capture camera may be arranged above the sand table display device to collect images of the environment from a bird's-eye view.
  • the motion capture camera may perform image acquisition under the control of the display processing device.
  • the motion capture camera first acquires the 3D coordinate information of the head-mounted display device in space, and then combines the obtained human eye viewpoint of the user wearing the head-mounted display device in the motion capture space Coordinate information, so as to complete the tracking and positioning of the viewing angle of the user, and then transmit the positioning information to the display processing device.
  • the position signal of the display processing device in the motion capture space is synchronized to the holographic image, and the three-dimensional position of the head-mounted display device in the motion capture space is used as the position of the virtual camera in the holographic image (that is, the position of the head-mounted display device position) to calculate the viewpoint picture of the virtual camera.
  • the display processing device may include: a display server, a display processor, and a synchronization transmitter.
  • the first display signal is generated by the display server; the composite signal and/or synchronization signal is generated by the display processor; the synchronization signal is sent to the head-mounted display device by the synchronization transmitter.
  • the aforementioned left-eye display signal and right-eye display signal are generated by a display server, and the display server sends the left-eye display signal and right-eye display signal to the display processor after generating the left-eye display signal and right-eye display signal.
  • the display processor generates a first display signal according to the left-eye display signal and the right-eye display signal, and sends it to the sand table display device.
  • the synchronization signal is sent to the HMD by the synchronization transmitter.
  • the display server is electrically connected to the display processor through a DP or HDMI cable; the display processor is electrically connected to the receiving card through a network port; and the display processor is electrically connected to the synchronization transmitter through a cable.
  • the receiving card is electrically connected with the sand table display device through a cable.
  • the sand table display device may be composed of small-pitch LED display screens, and the sand table display device includes an LED display body, a 3D video fuser, a 3D signal transmitter, and the like.
  • the three-dimensional video fusion device is connected to the sending card of the LED display body.
  • the three-dimensional signal transmitter is connected to the three-dimensional video fuser, and the three-dimensional video fuser receives the viewpoint picture information of the virtual camera (that is, the head-mounted display device) processed by the display processor in the display processing device, adjusts the display frequency and The picture is displayed on the LED display body through the sending card to display the left and right eye pictures, and the display processor is equipped with a holographic three-dimensional display algorithm.
  • the three-dimensional signal transmitter synchronously sends the display frequency to the holographic 3D glasses, so as to realize real-time display of images adjusted according to different viewpoint positions of users wearing the holographic 3D glasses.
  • the three-dimensional position of the head-mounted display device in the motion capture space is used as the position of the virtual camera in the virtual space, so that the position of a virtual camera can be simulated to capture the location of the head-mounted display device.
  • the virtual three-dimensional scene is set according to the position of the virtual camera in the virtual space.
  • the picture taken by the virtual camera is the three-dimensional virtual scene picture (that is, a holographic image) seen by the user wearing the head-mounted display device.
  • the virtual three-dimensional scene captured by the camera is the 3D perspective picture viewed by the user.
  • the display processing device can process the picture to be displayed in real time according to the viewpoint position information of the head-mounted display device in the motion capture space, and at the same time use the sand table display device to display the viewpoint picture, which can achieve excellent holographic three-dimensional display effect and obtain excellent Excellent out-of-screen sense and holographic visual sense.
  • the holographic display system provided by this application has greatly reduced cost, more delicate and richer picture display, and richer colors.
  • the display effect is stable, the user is more convenient to use, and the operation control is simpler.
  • the sand table display device has delicate display effect, high brightness, and low cost. It adopts LED active 3D stereoscopic display technology combined with the spatial position information of the observation point, so as to realize the 3D stereoscopic effect of virtual objects suspended on the LED display screen, which can improve the perception of users. , to improve the effect of the holographic display.
  • the process in this specification can not only be used to display holographic images to users, but also realize interaction with users.
  • the display processing device receives the interaction instruction, it generates a second display signal and outputs it to the sand table display device, so that the sand table display device outputs a holographic image for displaying the interactive handle according to the second display signal, so that the user can Operate against the interactive handle to achieve interaction.
  • the interaction instruction is triggered by a specified action performed by the user through the interactive control pen, and the specified action may be collected by a motion capture camera.
  • the holographic image of the interactive handle can be a rod-shaped holographic image, and one end of the interactive handle cooperates with the user's hand movement, and moves in the holographic image generated by the sand table display device following the user's hand movement.
  • this specification further provides a holographic display device, as shown in Figure 3, the holographic display device includes one or more of the following modules:
  • the first acquisition module 300 is configured to: acquire the viewpoint information of the first user, wherein the viewpoint information shows the binocular position of the first user and the binocular angle of view of the first user, and the first user is the user holding the interactive control pen;
  • the second obtaining module 302 is configured to: obtain pose information of the interactive control pen, wherein the pose information shows the direction of the interactive control pen;
  • the first included angle determination module 304 is configured to: determine a first included angle between the viewing angle and the orientation;
  • the first adjustment module 306 is configured to: adjust the output frequency of the head-mounted display device to the first frequency if the first included angle is greater than the first angle threshold; if the first included angle is not greater than the first angle threshold When an angle threshold is reached, the output frequency of the head-mounted display device is adjusted to a second frequency, wherein the first frequency is greater than the second frequency.
  • the holographic display device further includes a second adjustment module.
  • the second adjustment module is configured to: obtain the number of users, wherein the users include the first user and the second user, and the second user wears the head-mounted display device and does not hold the Users who interact with the control pen; if the number of users is greater than the number threshold, adjust the value of the first frequency to the first value; if the number of users is not greater than the number threshold, then adjust the value of the first frequency The value is adjusted to a second value, wherein the first value is less than the second value.
  • the holographic display device further includes a third adjustment module, and the third adjustment module is configured to: acquire viewpoint information of each second user, wherein the viewpoint information shows that the first The binocular position of the second user and the binocular angle of view of the second user, the second user is a user who wears the head-mounted display device and does not hold the interactive control pen; determine each second user The density center of the distribution of the binocular position in the specified environment, wherein the specified environment is the environment where the sand table display device is located; determine the connection line between the density center and the specified point as the specified line, Wherein, the designated point is the intersection of the pointing point of the interactive control pen and the holographic image; if the second angle between the pointing point and the designated line is greater than a second angle threshold, the sand table display device The output frequency of the sand table display device is adjusted to the third frequency, and if the second included angle is not greater than the second angle threshold, the output frequency of the sand table display device is
  • the viewpoint information includes six degrees of freedom information of a head-mounted display device worn by the first user, wherein the viewpoint information is obtained according to a viewpoint information tracking algorithm.
  • the pose information of the interactive control pen includes six degrees of freedom information of the interactive control pen, wherein the position three degrees of freedom in the six degrees of freedom information of the interactive control pen
  • the information is obtained according to the image information collected by the dynamic auxiliary camera, and the rotation 3-degree-of-freedom information in the six-degree-of-freedom information of the interactive control pen is obtained according to the data collected by the IMU module in the interactive control pen.
  • the auxiliary camera is set on the sand table display device.
  • the head-mounted display device includes shutter-type active 3D glasses and at least three reflective marking points embedded in the casing of the shutter-type active 3D glasses.
  • the shutter type active 3D glasses are used to receive the holographic image transmitted by the sand table type display device, so as to obtain the correct left and right eye images.
  • the head-mounted display device is provided with a first pressure sensor, and when the head-mounted display device is worn on the user's head, the first pressure sensor generates the first A signal is sent to the display processing device;
  • the interactive control pen is provided with an IMU module, and when the posture of the interactive control pen changes, a second signal is generated and sent to the display processing device;
  • the first An acquisition module 300 specifically configured to acquire the viewpoint information of the first user if the first signal is detected at a specified time and the second signal is detected within a specified time range from the specified time.
  • Fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
  • the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory.
  • the memory may include a memory, such as a high-speed random-access memory (Random-Access Memory, RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.
  • RAM random-Access Memory
  • non-volatile memory such as at least one disk memory.
  • the electronic device may also include hardware required by other services.
  • the processor, the network interface and the memory can be connected to each other through an internal bus, which can be an ISA (Industry Standard Architecture, industry standard architecture) bus, a PCI (Peripheral Component Interconnect, peripheral component interconnection standard) bus or an EISA (Extended Industry Standard Architecture, extended industry standard architecture) bus, etc.
  • the bus can be divided into address bus, data bus, control bus and so on. For ease of representation, only one double-headed arrow is used in FIG. 4 , but it does not mean that there is only one bus or one type of bus.
  • Memory for storing programs.
  • the program may include program code, and the program code includes computer operation instructions.
  • Storage which can include internal memory and nonvolatile storage, provides instructions and data to the processor.
  • the processor reads the corresponding computer program from the non-volatile memory into the memory and runs it, forming a holographic display device and/or a second holographic display device on a logical level.
  • the processor executes the program stored in the memory, and is specifically used to execute any of the aforementioned holographic display processes.

Abstract

La présente demande concerne un procédé et un appareil de présentation holographique. Le procédé consiste à : acquérir des informations de point de vue d'un premier utilisateur (S200), les informations de point de vue montrant les positions binoculaires du premier utilisateur et des angles de vue binoculaires du premier utilisateur, le premier utilisateur étant un utilisateur tenant un stylo de commande d'interaction ; acquérir des informations de pose du stylo de commande d'interaction (S202) ; déterminer un premier angle inclus entre l'angle de vue et une direction de pointage (S204) ; et, si le premier angle inclus est supérieur à une première valeur de seuil d'angle, régler une fréquence de sortie d'un casque immersif sur une première fréquence, et si le premier angle inclus n'est pas supérieur à la première valeur de seuil d'angle, régler la fréquence de sortie du casque immersif sur une seconde fréquence, la première fréquence étant supérieure à la seconde fréquence (S206). Dans la présente description, une fréquence de sortie d'un casque immersif est réglée selon un premier angle inclus entre un angle de vue d'un premier utilisateur et une direction de pointage d'un stylo de commande d'interaction, permettant ainsi au premier utilisateur d'identifier plus facilement, à partir d'une image holographique, une partie de contenu avec laquelle le premier utilisateur s'attend à interagir. Le procédé dans la présente demande est approprié pour un scénario d'enseignement à plusieurs personnes.
PCT/CN2022/116191 2021-10-09 2022-08-31 Procédé et appareil de présentation holographique WO2023056803A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111179077.6A CN113891063B (zh) 2021-10-09 2021-10-09 一种全息展示方法及装置
CN202111179077.6 2021-10-09

Publications (1)

Publication Number Publication Date
WO2023056803A1 true WO2023056803A1 (fr) 2023-04-13

Family

ID=79005810

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/116191 WO2023056803A1 (fr) 2021-10-09 2022-08-31 Procédé et appareil de présentation holographique

Country Status (2)

Country Link
CN (1) CN113891063B (fr)
WO (1) WO2023056803A1 (fr)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113891063B (zh) * 2021-10-09 2023-09-01 深圳市瑞立视多媒体科技有限公司 一种全息展示方法及装置

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180077409A1 (en) * 2016-09-09 2018-03-15 Samsung Electronics Co., Ltd. Method, storage medium, and electronic device for displaying images
CN109445595A (zh) * 2018-11-02 2019-03-08 京东方科技集团股份有限公司 一种虚拟现实设备及其驱动方法
CN109714588A (zh) * 2019-02-16 2019-05-03 深圳市未来感知科技有限公司 多视点立体图像定位输出方法、装置、设备以及存储介质
CN110430420A (zh) * 2019-06-24 2019-11-08 南方电网调峰调频发电有限公司 一种基于小间距led的五面cave显示系统集成方法
CN113382224A (zh) * 2021-05-27 2021-09-10 深圳市瑞立视多媒体科技有限公司 一种基于全息沙盘的交互手柄展示方法及装置
CN113891063A (zh) * 2021-10-09 2022-01-04 深圳市瑞立视多媒体科技有限公司 一种全息展示方法及装置

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7268774B2 (en) * 1998-08-18 2007-09-11 Candledragon, Inc. Tracking motion of a writing instrument
US20160378176A1 (en) * 2015-06-24 2016-12-29 Mediatek Inc. Hand And Body Tracking With Mobile Device-Based Virtual Reality Head-Mounted Display
US10304247B2 (en) * 2015-12-09 2019-05-28 Microsoft Technology Licensing, Llc Third party holographic portal
CN106200985A (zh) * 2016-08-10 2016-12-07 北京天远景润科技有限公司 桌面型个人沉浸虚拟现实交互设备
US10698475B2 (en) * 2016-10-26 2020-06-30 Htc Corporation Virtual reality interaction method, apparatus and system
CN213934416U (zh) * 2021-01-29 2021-08-10 南京盛世光幻科技有限公司 一种mr头显全息装置的控制手柄
CN113206991A (zh) * 2021-04-23 2021-08-03 深圳市瑞立视多媒体科技有限公司 全息显示方法、系统、计算机程序产品及存储介质
CN113376858A (zh) * 2021-05-27 2021-09-10 深圳市瑞立视多媒体科技有限公司 一种基于全息沙盘的全息展示方法及装置
CN113485547A (zh) * 2021-06-17 2021-10-08 深圳市瑞立视多媒体科技有限公司 一种应用于全息沙盘的交互方法及装置

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180077409A1 (en) * 2016-09-09 2018-03-15 Samsung Electronics Co., Ltd. Method, storage medium, and electronic device for displaying images
CN109445595A (zh) * 2018-11-02 2019-03-08 京东方科技集团股份有限公司 一种虚拟现实设备及其驱动方法
CN109714588A (zh) * 2019-02-16 2019-05-03 深圳市未来感知科技有限公司 多视点立体图像定位输出方法、装置、设备以及存储介质
CN110430420A (zh) * 2019-06-24 2019-11-08 南方电网调峰调频发电有限公司 一种基于小间距led的五面cave显示系统集成方法
CN113382224A (zh) * 2021-05-27 2021-09-10 深圳市瑞立视多媒体科技有限公司 一种基于全息沙盘的交互手柄展示方法及装置
CN113891063A (zh) * 2021-10-09 2022-01-04 深圳市瑞立视多媒体科技有限公司 一种全息展示方法及装置

Also Published As

Publication number Publication date
CN113891063B (zh) 2023-09-01
CN113891063A (zh) 2022-01-04

Similar Documents

Publication Publication Date Title
CN106165415B (zh) 立体观看
US20150358539A1 (en) Mobile Virtual Reality Camera, Method, And System
TWI523488B (zh) 處理包含在信號中的視差資訊之方法
WO2017086263A1 (fr) Dispositif de traitement d'image et procédé de génération d'image
JP2017536565A (ja) ステレオ視のための広視野カメラ装置
US20060250391A1 (en) Three dimensional horizontal perspective workstation
WO2017173890A1 (fr) Casque à deux caméras
WO2023056803A1 (fr) Procédé et appareil de présentation holographique
CN114371779B (zh) 一种视线深度引导的视觉增强方法
CN113382224B (zh) 一种基于全息沙盘的交互手柄展示方法及装置
CN205510302U (zh) 一种三维立体投影仪
JP6207640B2 (ja) ２次元映像の立体映像化表示装置
KR20200115631A (ko) 멀티뷰잉 가상 현실 사용자 인터페이스
US11388378B2 (en) Image processing apparatus, image processing method and projection system
CN113376858A (zh) 一种基于全息沙盘的全息展示方法及装置
CN113382229B (zh) 一种基于全息沙盘的动辅相机调整方法及装置
Yan et al. Research summary on light field display technology based on projection
CN109313823A (zh) 信息处理装置、信息处理方法和程序
EP4030752A1 (fr) Système et procédé de génération d'image
WO2020135744A1 (fr) Appareil d'affichage de réalité virtuelle tridimensionnelle à l'oeil nu et procédé d'affichage
CN113382225A (zh) 一种基于全息沙盘的双目全息展示方法及装置
Chotrov Methods for Generating and Displaying Stereo Images on VR Systems using Quad-Buffered Graphics Adapters
Kyriakakis et al. Stereoscopic Video Acquisition, Display, Transmission and Interaction
CZ112598A3 (cs) Způsob vytváření stereoskopického obrazu a zařízení k prováděni tohoto způsobu

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22877825

Country of ref document: EP

Kind code of ref document: A1