EP4515322A1 - Apparatus and method for a volumetric display - Google Patents
Apparatus and method for a volumetric displayInfo
- Publication number
- EP4515322A1 EP4515322A1 EP23797210.4A EP23797210A EP4515322A1 EP 4515322 A1 EP4515322 A1 EP 4515322A1 EP 23797210 A EP23797210 A EP 23797210A EP 4515322 A1 EP4515322 A1 EP 4515322A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- display device
- rotation
- real
- visual representation
- world object
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B30/00—Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images
- G02B30/50—Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images the image being built up from image elements distributed over a three-dimensional [3D] volume, e.g. voxels
- G02B30/54—Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images the image being built up from image elements distributed over a three-dimensional [3D] volume, e.g. voxels the three-dimensional [3D] volume being generated by moving a two-dimensional [2D] surface, e.g. by vibrating or rotating the 2D surface
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
- G09G3/00—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes
- G09G3/001—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes using specific devices not provided for in groups G09G3/02 - G09G3/36, e.g. using an intermediate record carrier such as a film slide; Projection systems; Display of non-alphanumerical information, solely or in combination with alphanumerical information, e.g. digital display on projected diapositive as background
- G09G3/003—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes using specific devices not provided for in groups G09G3/02 - G09G3/36, e.g. using an intermediate record carrier such as a film slide; Projection systems; Display of non-alphanumerical information, solely or in combination with alphanumerical information, e.g. digital display on projected diapositive as background to produce spatial visual effects
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/30—Image reproducers
- H04N13/388—Volumetric displays, i.e. systems where the image is built up from picture elements distributed through a volume
- H04N13/393—Volumetric displays, i.e. systems where the image is built up from picture elements distributed through a volume the volume being generated by a moving, e.g. vibrating or rotating, surface
Definitions
- Embodiments of the present invention relate generally to computer output device technologies, and more specifically to visual display device technologies.
- a display device is an output device for presentation of information in visual form.
- LCD liquid crystal display
- LED light-emitting diode display
- PDP plasma display panel
- CRT cathode ray tube display
- Most display devices currently in use are limited to displaying two-dimensional (2D) images.
- One embodiment provides an apparatus, comprising: a display device that is at least partially transparent and that is capable of displaying images; a mount configured to support the display device and to facilitate continuous rotation of the display device completely around an axis of rotation; and a motion producing device associated with the mount that is configured to continuously rotate the display device around the axis of rotation.
- Another embodiment provides a method, comprising: continuously rotating a display device completely around an axis of rotation, wherein the display device is at least partially transparent and is capable of displaying images; and displaying images on the display device as it rotates in a manner that generates a visual representation of an object in three physical dimensions.
- Another embodiment provides a non-transitory computer readable storage medium storing one or more computer programs configured to cause a processor based system to execute steps comprising: continuously rotating a display device completely around an axis of rotation, wherein the display device is at least partially transparent and is capable of displaying images; and displaying images on the display device as it rotates in a manner that generates a visual representation of an object in three physical dimensions.
- Another embodiment provides a method, comprising: capturing a plurality of different views of a real -world object with a plurality of image capture devices; and generating a visual representation of the real -world object in three physical dimensions by displaying images on a transparent display device that continuously rotates completely around an axis of rotation; wherein the images displayed on the transparent display device are selected based on the plurality of different views of the real -world object.
- Another embodiment provides a system, comprising: a plurality of image capture devices arranged to capture a plurality of different views of a real -world object; a transparent display device configured to continuously rotate completely around an axis of rotation; and a processor based apparatus configured to generate a visual representation of the real -world object in three physical dimensions by displaying images on the transparent display device as it rotates, wherein the images displayed on the transparent display device are selected based on the plurality of different views of the real-world object.
- Another embodiment provides a non-transitory computer readable storage medium storing one or more computer programs configured to cause a processor based system to execute steps comprising: receiving data representing a plurality of different views of a real -world object that are captured with a plurality of image capture devices; and generating a visual representation of the real-world object in three physical dimensions by displaying images on a transparent display device that continuously rotates completely around an axis of rotation; wherein the images displayed on the transparent display device are selected based on the plurality of different views of the real -world object.
- FIG. l is a diagram illustrating a transparent display device operating in accordance with some embodiments of the present invention.
- FIG. 2 is a diagram illustrating an apparatus that operates in accordance with some embodiments of the present invention.
- FIG. 3 is a diagram illustrating an apparatus operating in accordance with some embodiments of the present invention.
- FIG. 4 is a top view of FIG. 3 illustrating the operation of an apparatus in accordance with some embodiments of the present invention
- FIG. 5 is a diagram illustrating a system that operates in accordance with some embodiments of the present invention.
- FIG. 6 is a block diagram illustrating a processor based apparatus/system that may be used to run, implement, and/or execute any of the methods, schemes, and techniques shown and described herein in accordance with some embodiments of the present invention.
- OLED organic light-emitting diode
- This technology has recently been used to make transparent display devices, also known as see-through display devices.
- Transparent display devices allow the viewer to see what is shown on the screen while also allowing the viewer to see through the display device itself. They display RGB color, but no black is displayed because, given they are transparent, there is no solid black background.
- transparent displays provide a source generated light but not a reflective light.
- MicroLED microscopic LEDs
- LCD liquid crystal display
- LED light-emitting diode
- Various different types of transparent display devices provide various different levels of transparency. But most transparent display devices can be said to be at least partially transparent.
- a volumetric display device is a graphic display device that forms a visual representation of an object in three physical dimensions. That is, a volumetric display device is capable of creating a three- dimensional (3D) image in space.
- a transparent display device such as a transparent OLED display
- a transparent display device is used to make a volumetric display.
- a transparent display device is mounted to a motor and then rotated. As is described herein, this creates a volumetric display that achieves a 3D effect through persistence of vision.
- FIG. 1 illustrates an example of a transparent display device 102 being rotated in accordance with an embodiment of the present invention.
- the transparent display device 102 is rotated around an axis of rotation 104.
- the display device 102 rotates completely and continuously around the axis of rotation 104. That is, the display device 102 rotates a full 360 degrees around the axis of rotation 104 continuously in one direction.
- the display device 102 is rotated at a constant rate.
- the display device 102 is rotated at a variable rate.
- the display device 102 is shown as rotating in the counterclockwise direction, but it should be understood that in some embodiments the display device 102 may instead rotate in the clockwise direction.
- the display device 102 includes a display surface 106 on which the images are displayed.
- the axis of rotation 104 either lies on the same plane as the display surface 106, or the axis of rotation 104 is parallel to the display surface 106. In the latter case, given that the display device 102 will typically have some thickness, the axis of rotation 104 may be somewhat offset from the plane of the display surface 106. More specifically, the display device 102 is typically very thin but still comprises a small thickness. The precise positioning of the axis of rotation 104 may take into consideration factors such as rotational balancing, etc.
- axis of rotation 104 lying on the same plane as the display surface 106 in some embodiments, or the axis of rotation 104 being offset from that plane and being parallel to the display surface 106 in some embodiments. It should be well understood, however, that many other positions and placements of the axis of rotation 104 are possible in accordance with some embodiments of the present invention.
- a 3D image is created through persistence of vision. This occurs at least in part because the transparent nature of the display device 102, along with the light from the pixels of the rotating display device, provide for the visual perception of a 3D image.
- the 3D image is semitranslucent because the color black cannot be actively generated with an LED.
- the created 3D image is similar to a hologram.
- the apparatuses and methods described herein may be considered as providing a persistence of vision holographic display.
- FIG. 2 illustrates an apparatus 110 that may be used for implementing a volumetric display in accordance with an embodiment of the present invention.
- the apparatus 110 includes a display device 112 that is at least partially transparent and that is capable of displaying images.
- the display device 112 may comprise a transparent OLED display or any other type of transparent or partially transparent display.
- the display device 112 may comprise an available off-the-shelf transparent display and/or any other conventional and/or available transparent display. Large format transparent OLED displays and other types of transparent displays are believed to be available.
- the display device 112 is not limited to any standard current form factors of displays or any currently available displays.
- the display device 112 may comprise a custom, unique, and/or specially built transparent display having any size, shape, form factor, etc.
- the display device 112 is rectangular in shape and is mounted in the portrait position. But the display device 112 may be mounted in the landscape position or any other position. And again, the display device 112 may comprise a display device having any shape, form factor, size, etc. For example, in some embodiments, the display device 112 may comprise a curved display device having any size.
- the display device 112 rotates completely around an axis of rotation 114 as described above.
- a mount 116 is configured to support the display device 112 and to facilitate continuous 360 degree rotation of the display device 112 around the axis of rotation 114.
- a motion producing device 118 such as a motor, is associated with the mount 116 and is configured to rotate the display device 112 around the axis of rotation 114.
- the display device 112 is rotated around the axis of rotation 114 at a constant rate. And in some embodiments, the display device 112 is rotated around the axis of rotation 114 at a variable rate.
- a constant rate of rotation may provide a more stable 3D image
- a variable rate of rotation may provide a 3D image that changes or fluctuates in some way.
- the display device 112 is shown as rotating in the clockwise direction, but it should be understood that in some embodiments the display device 112 may instead rotate in the counterclockwise direction.
- a shaft 122 or the like connects the motion producing device 118 to the display device 112 for driving the rotation.
- a sensing device or means is included in the apparatus 110 for sensing, detecting, and/or monitoring the current position of the display device 112 within the 360 degree rotation.
- the display device 112 includes a display surface 124 on which the images are displayed.
- the axis of rotation 1 14 is positioned substantially parallel to at least a portion of the display surface 124.
- the axis of rotation 114 is positioned such that it lies on the display surface 124, i.e. lies on the same plane as the display surface 124.
- the precise positioning of the axis of rotation 104 may take into consideration factors such as rotational balancing, etc.
- the precise positioning of the axis of rotation 114 may take into consideration factors such as rotational balancing, etc.
- the display device 112 is balanced so there is no vibration during rotation. That is, similar to a propeller or an automobile wheel/tire, balancing may be employed to help reduce or avoid vibration. Such balancing may be achieved by any suitable method or means.
- a balancer mechanism may be included that is configured to help reduce vibration as the display device 112 rotates.
- such balancer mechanism may be coupled to the rotational shaft 122 or to another element of a rotational housing to establish balance under rotation.
- such balancer mechanism may provide a counterbalance so the display device 112 does not, for example, act as an eccentric mass haptic and cause jitter on the display output.
- such balancer mechanism may comprise or provide any other suitable balancing technique.
- an enclosure 126 that is at least partially transparent encloses the display device 112 without obstructing its rotation.
- the enclosure 126 which remains stationary, prevents the rotation of the display device 112 from being obstructed or interfered with, e.g. prevents a person from touching it. That is, with the display device 112 enclosed inside the stationary transparent enclosure 126, the display device 112 can be spinning inside the enclosure, be visible to a viewer, and the viewer can touch the enclosure without getting hit by the rotating display device 112.
- the enclosure 126 may be constructed from plexiglass, polycarbonate, or any other transparent plastic or glass, which allows the display device 112 to be visible. In some embodiments, the enclosure 126 is highly transparent to provide for maximum visibility of the images displayed on the display device 112.
- the enclosure 126 is illustrated as being cylindrical in shape, but the enclosure 126 may comprise any suitable shape.
- the enclosure 126 is configured to enclose the display device 112 within a vacuum. Drawing a vacuum on the enclosure 126 will reduce the atmospheric friction and air resistance of the rotating display device 112 to facilitate more efficient rotation.
- power is supplied to the rotating display device 112 by way of a slip-ring type power delivery device.
- a slip-ring type power delivery device typically include a ring with multiple tracks in it, and then the portion that is spinning includes brushes that make contact with the track for power delivery. But other power delivery techniques may be used.
- the image and/or video data signals are provided to the rotating display device 112 by way of wireless communications.
- a 5G modem is used for such communications and may even be mounted to the display driver board for getting the image/video data into the display. But other data delivery techniques may be used.
- FIG. 3 illustrates the operation of the apparatus 110 in implementing a volumetric display in accordance with an embodiment of the present invention.
- a 3D object such as an egg 132
- the egg 132 may be an actual real-world egg, or it may be a virtual egg for which a 3D model has been rendered.
- the transparent display device 112 in order to display the egg 132, is continuously rotated in one direction completely around (i.e. 360 degrees) the axis of rotation 114. In some embodiments, such rotation may be at a constant rate. In some embodiments, such rotation may be at a variable rate. In FIG. 3 the transparent display device 112 inside the apparatus 110 is not shown because it is rotating, and given that it is substantially transparent, it is not clearly visible.
- the transparent display device 112 rotates, images are selected and displayed on the display device 112 in a manner that generates a visual representation 134 of the egg 132 in three physical dimensions.
- the visual representation 134 of the egg 132 comprises a full 360 degree viewing angle. That is, a viewer can see the visual representation 134 from all sides This is an advantage over many 3D display devices which have a limited viewing angle. With such devices a viewer cannot see anything if the viewer moves far enough to one side beyond the limited viewing angle.
- the visual representation 134 which exists in three physical dimensions, is similar to a hologram.
- the apparatuses and methods described herein may be considered as providing a persistence of vision holographic display.
- the images that are displayed on the display device 112 to generate the visual representation 134 of the egg 132 in three physical dimensions are selected from a plurality of views of the egg 132. That is, the images are selected from a plurality of actual views of the object for which the visual representation is generated.
- the views may be generated by cameras or other image capture devices, and for virtual objects the views may be generated by software from a rendered 3D model. This technique will be thoroughly discussed below.
- a visual representation of an object in three physical dimensions may be generated by displaying images other than actual views of the object.
- the visual representation 134 of the egg 132 may be generated by displaying a 2D oval shape on the display device 112 at all positions of the display device 112 in the full 360 degree rotation. That is, the image that is selected and displayed on the display device 112 is a 2D oval shape, and the image does not change as the display device rotates.
- the visual representation 134 of the egg 132 is generated because, as the display device 112 rotates, the viewer’s persistence of vision causes the 2D oval to appear to be a 3D egg.
- a visual representation of one-half of the egg 132 could be generated by displaying one-half of a 2D oval shape on one side of the display screen of the display device 112 for 180 degrees of each rotation, and leaving the entire display screen blank for the other 180 degrees of each rotation.
- the visual representation could be generated by alternating between displaying one-half of a 2D oval shape on one side of the display screen, and then displaying one-half of a 2D oval shape on the other side of the display screen, during the same 180 degrees of each rotation.
- a visual representation of a ring shape could be generated by lighting one pixel at the edge of the display screen of the display device 112, or by also lighting the corresponding pixel at the opposite edge of the display screen. As these one or two lit pixels spin they create a ring shape through persistence of vision.
- a visual representation of a cylinder could be generated by lighting several consecutive pixels to form a vertical line at either or both edges of the display screen. And a visual representation of one-half of that cylinder could be generated by turning those pixels off for one-half of the rotation (i.e. 180 degrees) of the display device 112.
- the system (example discussed below) driving the apparatus 110 needs to know when and what pixels on the display device 112 to turn on at what point in the rotation. That is, the system needs to know at what points in the rotation do certain pixels get turned on.
- FIG. 4 is a top view looking down on FIG. 3. That is, FIG. 4 is a top view looking down on the egg 132, the visual representation 134 of the egg 132, and the apparatus 110. Also shown is the display device 112, which is shown at one position in its rotation. The illustrated position of the display device 112 will be referred to as position A.
- the plurality of different positions of the display device 112 within the rotation comprises a plurality of equally spaced apart positions.
- the displayed image does not necessarily have to be changed, modified, and/or updated at each position if such is not needed. The displayed image could stay the same for more than one position.
- the number of different positions within the 360 degree rotation that the image displayed on the display device 112 can be changed, modified, and/or updated depends on the refresh rate of the display device 112 and the rate of rotation of the display device 112. This is because the image displayed on the display device 112 is changed, modified, and/or updated when the display device is refreshed, such as when a new image or video frame is displayed.
- the refresh rate of the transparent display device 112 is 120 Hertz, and the display device 112 is rotated at a rate of 1 revolution per second, then the display device 112 can be refreshed 120 times within one 360 degree rotation, which means the display device 112 can be refreshed every 3 degrees.
- the 120 equally spaced positions are each spaced apart by 3 degrees. This means that each 3 degree increment in the rotation circle comprises one frame of the display during the course of the rotation.
- the refresh rate of the transparent display device 112 is 120 Hertz, and the display device 112 is rotated at a rate of 2 revolutions per second, then the display device 112 can be refreshed 60 times within one 360 degree rotation, which means the display device 112 can be refreshed every 6 degrees.
- the 60 equally spaced positions are each spaced apart by 6 degrees, which means that each 6 degree increment in the rotation circle comprises one frame of the display during in the course of the rotation.
- the last of the above examples has the advantage of the fastest rate of rotation of the display device 112 at 6 revolutions per second.
- a faster rate of rotation of the display device 112 improves the persistence of vision of the volumetric display.
- the rate of rotation should be balanced with the number of positions within one 360 degree rotation that the display device 112 can be refreshed. For any given refresh rate of the transparent display device 112, the faster the rate of rotation, the fewer times that the display device 112 can be refreshed within one rotation/revolution, which means the fewer positions within one 360 degree rotation that the display device 112 can be refreshed. Having a larger number of closely spaced refresh positions within one 360 degree rotation improves the ability to provide a higher level of detail in the generated visual representation of an object. This is at least partly because the image displayed on the display device 112 can be changed, modified, and/or updated a greater number of times within the rotation.
- a rate of rotation of 6 revolutions per second which provides for a display refresh every 6 degrees, is believed to be a good balance of rotational speed and opportunities for refresh. It provides 60 equally spaced refresh positions within one 360 degree rotation, which provides that each 6 degree increment in the rotation circle comprises one frame of the display during in the course of the rotation. Stated differently, it provides 60 frames within a 360 degree revolution with each frame covering 6 degrees of movement of the display device 112. That is, when the display device 112 moves 6 degrees, there is a new frame. When the display device 112 moves another 6 degrees, there is another new frame, and so on.
- the size of the increments of movement for a new frame could be reduced without necessarily reducing the rate of rotation of the display device 112. For example, for very high refresh rates there could be a new frame for each 1 degree movement within the 360 degree rotation, or even more than one new frame per each 1 degree movement, depending upon the rate of rotation of the display device 112. Therefore, in some embodiments, higher refresh rates for the display device 112 are preferred because they can allow for faster rates of rotation and an increased number of refresh positions within one 360 degree rotation. It should be well understood, however, that any refresh rate, rate of rotation, and number of refresh positions may be used for the display device 112 in accordance with some embodiments of the present invention.
- the current position of the display device within the 360 degree rotation is sensed, detected, and/or monitored. This way, as the display device 112 rotates the system knows the exact position of the display as it is spinning and will be able to detect when one of the refresh positions is encountered. When the sensed current position is equal to one of the plurality of different refresh positions, the image corresponding to that position is displayed. Tn some embodiments, the position sensing, detecting, and/or monitoring is performed continuously while the display device 112 rotates.
- the sensing, detecting, and/or monitoring of the current position of the display device 112 may be performed with a sensing device configured to sense the position of the display device within a rotation.
- the sensing device may be associated with, or part of, the apparatus 110.
- a Hall effect sensor or an index wheel with an optical sensor may be used.
- a stepper motor may be used as the motion producing device 118 to rotate the display device 112 so that a very precise step corresponds to each of the predetermined refresh positions within the full 360 degree rotation.
- the images that are displayed on the display device 112 to generate a visual representation of an object in three physical dimensions are selected from a plurality of actual views of the object for which the visual representation is generated. More specifically, in some embodiments, in order to generate a visual representation of an object in three physical dimensions, images are displayed on the rotating transparent display device 112. In some embodiments, for each of a plurality of different positions of the display device 112 within a rotation, an image is displayed on the display device 112. And in some embodiments, for each of the plurality of different positions of the display device 112 within the rotation, the image that is displayed is selected based on a plurality of views of the object for which the visual representation is generated.
- the plurality of different positions of the display device 112 within a rotation corresponds to the equally spaced positions or increments in the rotation circle at which the display device 112 can be refreshed. As discussed above, the number of such equally spaced positions or increments depends on the refresh rate of the display device 112 and the rate of rotation of the display device 112.
- the egg 132 may be an actual real-world egg, or it may be a virtual egg for which a 3D model has been rendered.
- a 3D model of the object has been rendered
- a plurality of different views of the object can be created through software.
- a plurality of different views of the object can be captured with one or more cameras or other image capture devices.
- the number of refresh positions of the display device 112 is equal to the number of views of the object for which a visual representation is to be generated. That is, for each of the refresh positions of the display device 112, there is a corresponding view of the object for which a visual representation is to be generated.
- the 60 different views of the object are used as the actual images that are displayed on the display device 112. That is, for each of the 60 refresh positions in the rotation circle, the particular image that is selected and displayed on the display device 112 is the corresponding one of the 60 different views of the object.
- the image that is selected and displayed on the display device 1 12 comprises the corresponding view of the object for which the visual representation is generated.
- a 3D model of the object may be created in software.
- the 3D model is then sliced up like a pie into 60 slices (i.e. views), and each slice (i.e. view) gets displayed on the display device 112 during one of the 60 refresh positions in the rotation circle.
- the 3D model is rendered based on the camera position.
- the camera position changes according to the position of the screen.
- the camera position is changed 60 times to pre-render 60 different views, one for each refresh position in the rotation circle, which in this example is every 6 degrees. That is, 60 equally spaced different views of the object are created through software.
- the views of the object are preferably taken from the perspective of a circle around the object with the object at the center of the circle. This provides a view for each of the 60 different refresh positions in the rotation circle. Each of the 60 different views then gets displayed on the display device 112 during the corresponding one of the 60 refresh positions in the rotation circle. That is, each of the 60 different views is displayed at its particular point in the rotation.
- the egg 132 which is the object for which a visual representation is to be generated, has been sliced up like a pie into a plurality of slices (i.e. views), namely view A’, view B’, view C’, view D’, and so on. Tf the egg 132 is a virtual egg for which a 3D model has been rendered, then the plurality of different views can be created through software. If the egg 132 is an actual real -world egg, then the plurality of different views can be captured with one or more cameras or other image capture devices.
- the views A’, B’, C’, D’, and so on respectively correspond to the display device 112 refresh positions A, B, C, D, and so on. That is, when the display device 112 is located at position A, the view of the visual representation 134 that will be displayed on the display device 112 corresponds to view A’ of the egg 132. Similarly, when the display device 112 is located at position B, the view of the visual representation 134 that will be displayed on the display device 112 corresponds to view B’ of the egg 132. And similarly, when the display device 112 is located at position C, the view of the visual representation 134 that will be displayed on the display device 112 corresponds to view C’ of the egg 132. And the same for position D and view D’, and so on.
- the image that will be selected and displayed on the display device 112 is view A’ of the egg 132.
- the image that will be selected and displayed on the display device 112 is view B’ of the egg 132.
- the image that will be selected and displayed on the display device 112 is view C’ of the egg 132.
- the image that is selected and displayed on the display device 112 comprises the corresponding view of the object for which the visual representation is generated.
- images are displayed on the display device 112 as it rotates in a manner that generates a visual representation of an object in three physical dimensions. For each of a plurality of different positions of the display device 112 within a rotation, an image is displayed on the display device 112 that comprises a corresponding view of the object for which the visual representation is generated. Or, in some embodiments, an image is displayed on the display device 112 that comprises a corresponding one of the plurality of different views of the object
- this process may be used for any 3D object for which a visual representation is to be generated.
- the shape of the 3D object does not matter.
- the main objective is to display each view of the 3D object on the display device 112 at the corresponding refresh position in the rotation circle.
- a visual representation can be generated by the apparatus 110 for any shape 3D object by slicing up the 3D model as described herein.
- the 3D object for which a visual representation is to be generated comprises a real -world object
- a 3D software model may be created for the object, or the plurality of equally spaced different views of the object may be captured with one or more cameras or other image capture devices.
- the number of refresh positions of the display device 112 does not have to be equal to the number of views of the object for which a visual representation is to be generated. That is, in some embodiments, the number of different actual views of the object that are used for selecting the new image can be different than the number of refresh positions of the display device 112. For example, if there are a greater number of refresh positions than views of the object, then in some embodiments, some of the refresh positions do not have to be used. That is, the display device 112 does not have to be refreshed at every refresh position. The system can utilize only the refresh positions for which it has enough views of the object.
- some or all of the views of the object can be reused during the rotation circle of the display device 112 and displayed at more than one of the refresh positions.
- some of the views of the object can be skipped during the rotation circle of the display device 112 such that not all of the views get displayed.
- the plurality of views of the object are used to generate the particular image that is to be displayed on the display device 112 at each of the refresh positions in order to generate the visual representation of the object in three physical dimensions. That is, in some embodiments, the plurality of views of the object are not used as the actual images that are displayed on the display device 112. Stated differently, in some embodiments, when the display device 112 is located at a refresh position, the corresponding view of the object is not displayed on the display device 112. Instead, in some embodiments, the particular image that is to be displayed on the display device 112 at each refresh position is generated by using the plurality of views of the object for which the visual representation is to be generated.
- the plurality of views of the object are used to generate the particular image that is to be displayed on the display device 112 at each refresh position.
- the generated image is considered to be an image that is selected based on a plurality of views of the object for which the visual representation is generated.
- FIG. 5 illustrates a system 140 in accordance with another embodiment of the present invention.
- the apparatus 110 is used to generate a visual representation 144 of a human being 142 in three physical dimensions.
- the 3D object for which a visual representation is to be generated comprises a real-world object, namely the human being 142.
- the transparent display device 112 in the apparatus 110 is not limited to any standard current form factors of displays. Therefore, in some embodiments, the transparent display device 112 can, for example, be on the order of 6 feet tall by 2.5 feet wide, which can create the life-size visual representation 144 of the human being 142. Furthermore, the visual representation 144 of the human being 142 comprises a full 360 degree viewing angle, which allows a viewer to walk all the way around it, and which is advantageous over 3D display devices that have a limited viewing angle. Thus, in some embodiments, the visual representation 144 of the human being 142 provides a virtual person, virtual performer, virtual actor, etc., which viewers can view at very close distances (e.g. just inches) from all sides.
- the visual representation 144 of the human being 142 can be programmed or controlled in real time to interact with viewers.
- the visual representation 144 of the human being 142 can turn and talk to viewers as the viewers approach. This could be useful in museums or similar places where visual representations of famous or historical people could interact with viewers.
- the visual representation 144 which exists in three physical dimensions, is similar to a hologram.
- the visual representation 144 can be considered to be a holographic human being, and the apparatuses and methods described herein may be considered as providing a holographic display.
- the system 140 includes a plurality of image capture devices 146 arranged to capture a plurality of different views of the real-world object, i.e. the human being 142.
- the image capture devices 146 may comprises cameras or the like.
- the plurality of image capture devices 146 are arranged to encircle the human being 142. And in some embodiments, the plurality of image capture devices 146 are equally spaced apart. And in some embodiments, the plurality of image capture devices 146 continuously capture a plurality of different views of the human being 142.
- the plurality of image capture devices 146 are mounted to a ring 148 with the human being 142 located at the center of the ring 148.
- Each of the multiple image capture devices 146 mounted to the ring 148 around the circle represent a different point of view of the human being 142.
- the total number of image capture devices 146 used may depend at least in part on the refresh rate and rate of rotation of the display device 112, similar to as described above.
- the plurality of image capture devices 146 are arranged and configured to capture a plurality of equally spaced views of the human being 142.
- the views are used as described above to generate the visual representation 144 of the human being 142 in three physical dimensions in the apparatus 1 10.
- the number of image capture devices 146 that are used to encircle the human being 142 will be the same as the number of equally spaced refresh positions in the rotation circle of the display device 112 in the apparatus 110. But similar to as mentioned above, in some embodiments, the number of image capture devices 146 does not have to be the same as the number of equally spaced refresh positions.
- the number of refresh positions within the rotation circle depends on the refresh rate of the display device 112 and the rate of rotation of the display device 112.
- the refresh rate of the display device 112 is 360 Hertz, and the display device 112 is rotated at a rate of 6 revolutions per second, there will be 60 equally spaced refresh positions in the rotation circle.
- 60 different views of the human being 142 will be needed, which means that 60 image capture devices 146 are used to capture the different views of the human being 142.
- the ring 148 would have 60 image capture devices 146 mounted thereto with the human being 142 located at the center of the ring 148.
- all of the image capture devices 146 are preferably synchronized with each other and continuously capture images.
- the image capture devices 146 are preferably synchronized because then the first frame from all of the image captures devices 146 will occur at the same time, and the second frame from all of the image captures devices 146 will occur at the same time, and the third frame from all of the image captures devices 146 will occur at the same time, and so on. This way, the 60 different views of the human being 142 will be synchronized so that at one moment in time each view will show what occurred at that moment.
- a processor based apparatus or system may be used to control the apparatus 110.
- such processor based apparatus or system is configured to generate the visual representation 144 of the human being 142 in three physical dimensions by displaying images on the transparent display device 112 as it rotates.
- the images displayed on the display device 112 are selected based on the plurality of different views of the human being 142 captured by the image capture devices 146.
- the processor based apparatus or system receives the frames for the plurality of different views of the human being 142 captured by the image capture devices 146. The processor based apparatus or system then selects the appropriate frames to be displayed on the display device 112.
- the visual representation 144 of the human being 142 is generated in substantially real time with the capturing of the plurality of different views of the human being 142. This has the effect of the visual representation 144 of the human being 142 being generated “live” with the performance, movements, and/or other actions of the human being 142.
- the “live” visual representation 144 of the human being 142 comprises a full 360 degree viewing angle. In some embodiments, this results from using the plurality of synchronized image capture devices 146 that encircle the human being 142 and that continuously capture the plurality of different views of the human being 142. Thus, a viewer can see the visual representation 144 from all sides and will be able to see all the movements and actions made by the human being 142 from all sides in real time.
- the “live” visual representation 144 of the human being 142 is transmitted to anywhere in the world in real time.
- data representing the plurality of different views of the human being 142 captured by the image capture devices 146 is transmitted via a computer network 152, such as the Internet, to the apparatus 110 for use in generating the visual representation 144 of the human being 142.
- the apparatus 110 may be located many miles away from the human being 142 and the image capture devices 146, possibly in a different part of the world.
- transmitting the data over the network 152 in this way allows for the live generation of a visual representation of a performance that takes place many miles away.
- a famous or noteworthy person could be giving a speech in one geographic location, and then in another geographic location many miles away a visual representation of that person in three physical dimensions could be generated in real time by the apparatus 110.
- Viewers would be able to stand or sit within inches of the visual representation of the person to hear and see the speech in real time and would be able to walk all the way around the visual representation to view the person from all sides.
- the methods, schemes, and techniques described herein may be utilized, implemented and/or run on many different types of processor based apparatuses or systems.
- the methods, schemes, and techniques described herein may be utilized, implemented, and/or run in any type of communications systems or equipment, positional tracking systems, smartphones, game consoles, entertainment systems, portable devices, mobile devices, tablet devices, pad-like devices, computers, workstations, desktop computers, notebook computers, servers, etc.
- the methods, schemes, and techniques described herein may be utilized, implemented and/or run in online scenarios, networked scenarios, over the Internet, etc.
- FIG. 6 there is illustrated an example of a processor based system or apparatus 160 that may be used for any such implementations.
- the system or apparatus 160 may be used for implementing any method, scheme, technique, system, or device mentioned above.
- the system or apparatus 160 may be used for driving and/or controlling the apparatus 110 and thus the display device 112.
- the system or apparatus 160 may be used for driving and/or controlling all or part of the system 140.
- the use of the system or apparatus 160 or any portion thereof is certainly not required.
- the processor based system 160 may include, but is not required to include, a processor 162 (e.g. a central processing unit (CPU)), a memory 164, a wireless and/or wired network interface 166, access to a network 168, one or more displays 174, one or more input/output device interfaces 172, and a wireless interface 170.
- a processor 162 e.g. a central processing unit (CPU)
- a memory 164 e.g. a central processing unit (CPU)
- a wireless and/or wired network interface 166 access to a network 168
- a network 168 e.g. a central processing unit (CPU)
- a wireless and/or wired network interface 166 access to a network 168
- a wireless and/or wired network interface 166 access to a network 168
- displays 174 e.g. a display 174
- input/output device interfaces 172 e.g., a wireless interface 1
- connections and/or communications channels 176 may comprise wired connections, wireless connections, network connections, or a mixture or combination of both wired and wireless connections, communications channels, network connections, buses, etc.
- the processor 162 may be used to execute or assist in executing the steps of the methods, schemes, and techniques described herein, and various program content, images, video, overlays, UIs, assets, virtual worlds, menus, menu screens, interfaces, graphical user interfaces (GUIs), windows, tables, graphics, avatars, characters, players, video games, simulations, etc., may be rendered on the display(s) 174.
- the processor 162 executes code, software, or steps that implements the generation of visual representations of objects in three physical dimensions described above.
- the memory 164 may include or comprise any type of computer readable storage or recording medium or media.
- the memory 164 may include or comprise a tangible, physical memory.
- the memory 164 may be used for storing program or computer code or macros that implements the methods and techniques described herein, such as program code for running the methods, schemes, and techniques described herein.
- the memory 164 may serve as a tangible non-transitory computer readable storage medium for storing or embodying one or more computer programs or software applications for causing a processor based apparatus or system to execute or perform the steps of any of the methods, code, schemes, and/or techniques described herein.
- the memory 164 may be used for storing any needed database(s).
- the wireless and/or wired network interface 166 may be used for accessing the network 168 for obtaining any type of information, such as for example any information needed for implementing, running, and/or executing the methods, techniques, processes, etc., discussed herein.
- the network 168 may comprise the Internet, a local area network, an intranet, a wide area network, or any other network.
- the wireless interface 170 may be used for wirelessly communicating with other devices, such as for example any of the devices, apparatuses, systems, etc., described herein.
- the input/output device interface 172 may be used for interfacing with any devices or peripherals, such as for example any of the devices, apparatuses, systems, etc., described herein.
- the one or more displays 174 may comprises any type of display devices and may be used for implementing any needed environments.
- one or more displays 174 may be included in a head worn device such as a headset, glasses-type user device, head-mounted display (HMD), or the like.
- the one or more displays 174 may be included or associated with any type of virtual reality (VR) device, augmented reality (AR) device, or mixed reality (MR) device.
- the one or more displays 174 may be included in a device such as a smartphone, tablet computer, pad-like computer, notebook computer, etc.
- the one or more displays 174 may be associated with any type of computer such as desktop computers, etc.
- the one or more displays 174 may comprise any type of display or display device or apparatus, using any type of display technology.
- one or more of the embodiments, methods, approaches, schemes, and/or techniques described above may be implemented in one or more computer programs or software applications executable by a processor based apparatus or system.
- processor based system may comprise a smartphone, tablet computer, VR, AR, or MR system, entertainment system, game console, mobile device, computer, workstation, desktop computer, notebook computer, server, graphics workstation, client, portable device, pad-like device, communications device or equipment, etc.
- Such computer program(s) or software may be used for executing various steps and/or features of the above-described methods, schemes, and/or techniques.
- the computer program(s) or software may be adapted or configured to cause or configure a processor based apparatus or system to execute and achieve the functions described herein.
- such computer program(s) or software may be used for implementing any embodiment of the above-described methods, steps, techniques, schemes, or features.
- such computer program(s) or software may be used for implementing any type of tool or similar utility that uses any one or more of the above described embodiments, methods, approaches, schemes, and/or techniques.
- one or more such computer programs or software may comprise a VR, AR, or MR application, communications application, object positional tracking application, a tool, utility, application, computer simulation, computer game, video game, role-playing game (RPG), other computer simulation, or system software such as an operating system, BIOS, macro, or other utility.
- program code macros, modules, loops, subroutines, calls, etc., within or without the computer program(s) may be used for executing various steps and/or features of the above-described methods, schemes and/or techniques.
- such computer program(s) or software may be stored or embodied in a non-transitory computer readable storage or recording medium or media, such as a tangible computer readable storage or recording medium or media.
- such computer program(s) or software may be stored or embodied in transitory computer readable storage or recording medium or media, such as in one or more transitory forms of signal transmission (for example, a propagating electrical or electromagnetic signal).
- the present invention provides a computer program product comprising a medium for embodying a computer program for input to a computer and a computer program embodied in the medium for causing the computer to perform or execute steps comprising any one or more of the steps involved in any one or more of the embodiments, methods, approaches, schemes, and/or techniques described herein.
- the present invention provides one or more non-transitory computer readable storage mediums storing one or more computer programs adapted or configured to cause a processor based apparatus or system to execute steps comprising any one or more of the steps involved in any one or more of the embodiments, methods, approaches, schemes, and/or techniques described herein.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computer Hardware Design (AREA)
- Optics & Photonics (AREA)
Abstract
Description
Claims
Applications Claiming Priority (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US17/733,980 US11977244B2 (en) | 2022-04-29 | 2022-04-29 | Method and system for generating a visual representation of a real-world object in three physical dimensions |
| US17/733,981 US20230350223A1 (en) | 2022-04-29 | 2022-04-29 | Apparatus and method for a volumetric display |
| PCT/US2023/020002 WO2023212064A1 (en) | 2022-04-29 | 2023-04-26 | Apparatus and method for a volumetric display |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| EP4515322A1 true EP4515322A1 (en) | 2025-03-05 |
| EP4515322A4 EP4515322A4 (en) | 2026-04-15 |
Family
ID=88519586
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| EP23797210.4A Pending EP4515322A4 (en) | 2022-04-29 | 2023-04-26 | DEVICE AND METHOD FOR A VOLUMETRIC DISPLAY |
Country Status (3)
| Country | Link |
|---|---|
| EP (1) | EP4515322A4 (en) |
| CN (1) | CN119053905A (en) |
| WO (1) | WO2023212064A1 (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11977244B2 (en) | 2022-04-29 | 2024-05-07 | Sony Interactive Entertainment Inc. | Method and system for generating a visual representation of a real-world object in three physical dimensions |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6183088B1 (en) * | 1998-05-27 | 2001-02-06 | Actuality Systems, Inc. | Three-dimensional display system |
| JP3787841B2 (en) * | 2002-06-05 | 2006-06-21 | ソニー株式会社 | Display device and display method |
| US20050050778A1 (en) * | 2003-09-10 | 2005-03-10 | Seongukk Kim | Rotational information display device |
| US8233032B2 (en) * | 2008-06-09 | 2012-07-31 | Bartholomew Garibaldi Yukich | Systems and methods for creating a three-dimensional image |
| CN103823307B (en) * | 2014-02-14 | 2016-08-17 | 京东方科技集团股份有限公司 | True three-dimensional imaging device and display device |
| KR20170111131A (en) * | 2016-03-25 | 2017-10-12 | 전자부품연구원 | Volumetric Light Field Display System using an OLED Display |
| JP2022536992A (en) * | 2019-06-20 | 2022-08-22 | プリズム ホログラムス エルエルシー | 3D volumetric holographic display |
| CN112596263B (en) * | 2019-10-01 | 2023-03-31 | 财团法人工业技术研究院 | Space stereo imaging device and method |
-
2023
- 2023-04-26 WO PCT/US2023/020002 patent/WO2023212064A1/en not_active Ceased
- 2023-04-26 EP EP23797210.4A patent/EP4515322A4/en active Pending
- 2023-04-26 CN CN202380035555.7A patent/CN119053905A/en active Pending
Also Published As
| Publication number | Publication date |
|---|---|
| EP4515322A4 (en) | 2026-04-15 |
| CN119053905A (en) | 2024-11-29 |
| WO2023212064A1 (en) | 2023-11-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11977244B2 (en) | Method and system for generating a visual representation of a real-world object in three physical dimensions | |
| US20240037880A1 (en) | Artificial Reality System with Varifocal Display of Artificial Reality Content | |
| EP3814876B1 (en) | Placement and manipulation of objects in augmented reality environment | |
| JP7008730B2 (en) | Shadow generation for image content inserted into an image | |
| US20230350223A1 (en) | Apparatus and method for a volumetric display | |
| US11765335B2 (en) | Synthetic stereoscopic content capture | |
| EP4695667A1 (en) | Devices, methods, and graphical user interfaces for displaying views of physical locations | |
| WO2021007221A1 (en) | Virtual user interface using a peripheral device in artificial reality environments | |
| EP4705864A1 (en) | Methods for moving objects in a three-dimensional environment | |
| US20130050198A1 (en) | Multi-directional display | |
| EP3635686A1 (en) | Smoothly varying foveated rendering | |
| US20150370322A1 (en) | Method and apparatus for bezel mitigation with head tracking | |
| US8922591B2 (en) | Holographic display | |
| US20170293412A1 (en) | Apparatus and method for controlling the apparatus | |
| JP2022515608A (en) | Systems and / or methods for parallax correction in large area transparent touch interfaces | |
| EP4515322A1 (en) | Apparatus and method for a volumetric display | |
| US11048108B2 (en) | Multi-perspective display of an image using illumination switching | |
| WO2025080489A1 (en) | Display rotation to increase virtual fov for fish tank-style vr system | |
| US20240111390A1 (en) | Translating Interactions on a Two-Dimensional Interface to an Artificial Reality Experience | |
| EP4513308A1 (en) | Selectively upsampling visual sensor data for efficient display | |
| CN118842906A (en) | Space-time separable filter | |
| CN120687012A (en) | Wallpaper display method, device and electronic device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE |
|
| PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
| 17P | Request for examination filed |
Effective date: 20241126 |
|
| AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC ME MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
| DAV | Request for validation of the european patent (deleted) | ||
| DAX | Request for extension of the european patent (deleted) | ||
| A4 | Supplementary search report drawn up and despatched |
Effective date: 20260317 |
|
| RIC1 | Information provided on ipc code assigned before grant |
Ipc: G02B 30/54 20200101AFI20260311BHEP Ipc: G09G 3/00 20060101ALI20260311BHEP Ipc: H04N 13/393 20180101ALI20260311BHEP |