WO2024160814A1 - Multi-layer projection system for passive relief perception of three-dimensional media - Google Patents

Abstract

A projection system (1) comprising: • a projector (3) arranged to project an image in a main direction, referred to as the projection direction; • a projection surface (4) formed of a plurality of semi-transparent layers, said layers being oriented transversely to the projection direction, the projection surface forming a viewing support for an image projected by the projector.

Description

Multi-layer projection system for passive relief perception of three-dimensional media

The invention lies in the field of media visualization, and more specifically in that of three-dimensional media visualization.

State of the prior art

Media viewing can be accomplished in several ways.

Conventionally, an image from a media is projected onto a viewing surface by a projector.

Usually the viewing surface is a flat surface.

A disadvantage of this type of visualization is the lack of three-dimensional perception for a viewer.

Methods for restoring three-dimensional vision are known.

3D projection on a screen requires stereoscopic glasses and offers a limited viewing angle and false parallax.

Holograms can only display static three-dimensional media of moderate sizes, on the order of a few tens of centimeters, and are complex to produce.

Space scanning systems rely on the persistence of human vision to merge a series of slices of the three-dimensional object into a single three-dimensional image. Due to the high speed of rotation or translation of the moving part, this type of device is limited in size, of the order of a few tens of centimeters.

Devices based on micro-lens matrices are complex to produce and limited in size, of the order of a few tens of centimeters, their viewing angle is limited.

Devices based on semi-transparent mirrors and screens, the image of which is reflected in these mirrors, do not allow a high density of display planes, the size of the reflections being identical to that of the image displayed on the screen. As a result, these devices are more suitable for displaying distinct planes rather than for a supposedly continuous 3D volumetric media. In addition, these systems often have a limited viewing angle.

The invention aims to overcome all or part of the aforementioned drawbacks, in particular the possibility of displaying very large three-dimensional media (several meters on each side), of offering a wide viewing angle, of being compatible with calculated and real-time media and therefore potentially interactive, of not requiring the wearing of glasses and of being visible to a large number of spectators at the same time.

• un projecteur agencé pour projeter une image (ou succession d’image, fixe ou animée) selon une direction principale, dite direction de projection ;
• une surface de projection (continue ou discontinue) formée d’une pluralité de couches semi-transparentes, chacune des couches étant orientées transversalement à une direction de projection, dite locale sur ladite couche, la surface de projection formant support de visualisation pour une image projetée par le projecteur.

According to a first aspect of the invention, a system is proposed comprising:

• a projector designed to project an image (or succession of images, fixed or animated) in a main direction, called the projection direction;
• a projection surface (continuous or discontinuous) formed from a plurality of semi-transparent layers, each of the layers being oriented transversely to a projection direction, called local on said layer, the projection surface forming a viewing support for an image projected by the projector.

Preferably, each pixel projected by the projector onto the surface has a single point of contact with that surface. The point of contact determines the specific spatial position of the projected pixel. The location where the pixel touches the surface therefore defines the point in space from which that pixel can be observed. The set of projected pixels defines a set of points in space. These points in space can then be used to display representations of 3D media. Preferably, the projected media is a three-dimensional media. Preferably, the representation of the 3D media in the display system is also three-dimensional.

Thus, the projection surface can form a viewing support for three-dimensional media.

The system may be referred to as a volume display, spatial display, 3D display, three-dimensional display, 3D projection system, or three-dimensional display, or a combination of these terms. It enables a spatial representation of a 3D media by spreading it across multiple layers, thereby creating a distinct and visible three-dimensional visualization in space. This is not an illusion of relief, but a real spatial representation.

The viewer perceives the projected media in 3D through natural vision mechanisms, including convergence, accommodation, and binocular disparity, which allow him to distinguish the different depths and spatial positions of the images projected onto the layers.

Points displayed in the physical space of a three-dimensional display system can be retrieved by applying rotations, translations and homotheties to the original 3D media.

Three-dimensional media (or 3D media) is a form of content that presents information in three spatial dimensions, which may or may not be animated over time. Examples include 3D mesh and textured models, point clouds, or mathematical surfaces.

Displayed 3D media, projected 3D media, broadcast 3D media, representation of 3D media, projection of 3D media or display of 3D media are the representation of this same 3D media, resulting from the interaction of an image projected on multiple semi-transparent layers. The projected image is calculated according to the process described below.

Just as 3D media is defined by spatial coordinates, its representation in the three-dimensional display system also uses spatial coordinates.

The spatial configuration of these layers on which the image is projected defines the volume in which the 3D media can be displayed. Thus, the size and organization of these layers directly condition the dimensions of the 3D content that can be projected, by limiting or extending the three-dimensional space available for display.

The semi-transparent surface is characterized by its ability to diffuse light received on both sides, without sharp specular reflection (there are no semi-transparent mirrors), and without noticeable refraction effects (which means that it does not deflect light in such a way as to change its initial propagation direction), while allowing the other layers to be seen through it.

Semi-transparency means a surface that allows between 5% and 95% of the light to pass through, preferably between 10% and 40%. Preferably, the semi-transparent surface has a symmetrical light transmission characteristic, allowing identical transmission of light, regardless of the face through which it is incident. Preferably, the surface is formed of a single material.

Advantageously, the semi-transparent surface reflects the light received by the projector with the same intensity in all directions.

In one embodiment, the system further comprises a support structure comprising an upper frame and a lower frame, the frames forming two surfaces parallel to each other, each of the layers extending between an upper segment inscribed in the surface of the upper frame and a lower segment inscribed in the surface of the lower frame. The two surfaces are not necessarily parallel although this is the preferred production method.

Preferably, the projection surface is continuous, which allows for simplification of the manufacture of the projection surface.

According to one possibility, all layers have a normal forming an acute angle with the local projection direction. According to another possibility, all layers have a normal forming an obtuse angle with the local projection direction.

Alternatively, in which all layers have a normal forming alternately an acute angle and an obtuse angle, with the local projection direction.

Both of these possibilities allow for a better spatial distribution of the media than when the angle alternates between acute and obtuse between two consecutive layers.

Advantageously, each layer receives an identical quantity of pixels in projection, which ensures uniform spatial display quality across all layers.

Support wires can be stretched to position the layers in the upper or lower frame.

Advantageously, the projection surface is made of tulle.

In one embodiment, a lens and/or a mirror is operatively disposed between the projector and the projection surface.

A physical device can be placed and partially or completely hide the system.

An object can be placed between the projector and the projection surface or at the projection surface.

• génération d’un dispositif virtuel dual du système comportant :
  1. une caméra virtuelle présentant de mêmes caractéristiques que le projecteur,
  2. un maillage 3d, dit surface de projection virtuelle de mêmes caractéristiques que la surface de projection réelle,
• calcul d’une intersection tridimensionnelle virtuelle de la surface de projection virtuelle et de l’objet virtuel en volume,
• calcul d’une vue, aussi appelée rendu, de l’intersection tridimensionnelle par la caméra virtuelle,
• export d’une image ou vidéo et/ou envoi au projecteur du système.

According to a second aspect of the invention, a method is proposed for projecting a virtual object into volume by means of a system according to the first aspect of the invention, comprising the following steps:

• generation of a dual virtual device of the system comprising:
  1. a virtual camera with the same characteristics as the projector,
  2. a 3D mesh, called a virtual projection surface with the same characteristics as the real projection surface,
• calculation of a virtual three-dimensional intersection of the virtual projection surface and the virtual object in volume,
• calculation of a view, also called rendering, of the three-dimensional intersection by the virtual camera,
• export an image or video and/or send it to the system projector.

Preferably, the system has a predetermined main viewing direction and the 3D mesh of the virtual projection surface is thickened along the viewing direction prior to the step of calculating the virtual three-dimensional intersection.

The present invention enables the display of virtual 3D media by diffusing them within the volume defined by the semi-transparent layers. As a result, it is possible to observe the three-dimensional media from all angles, and from all distances. This observation capability is similar to that which an individual would experience when looking at a real object in physical space, thus providing an immersive and realistic visual experience.

In particular, the system is designed to allow viewers to position themselves at various points around the projection volume. They can be located behind the projection device, between the latter and the viewing volume, or facing the viewing volume. In addition, it is also possible to position themselves to the sides, as well as above or below the projection volume.

Description of figures

• est un schéma de principe d’un exemple de réalisation d’un système 1 selon l’invention ;
• est un schéma de principe d’un exemple de réalisation d’un système 1 selon l’invention comportant en outre une projection d’un média tridimensionnel et des positions d’observation ;
• illustre une construction d’une surface de projection d’un exemple de réalisation d’un système 1 selon l’invention ;
• illustre deux types de motifs distincts possibles pour la réalisation du système 1 de  ;
• illustre plusieurs variantes de réalisation du système de  ;
• illustre une variante de réalisation du système de , comportant un brise-vue et un fond noir ;
• illustre une variante de réalisation du système de , comportant un objet physique supplémentaire dans le champ de projection ;
• est un schéma de principe d’un exemple de réalisation d’un procédé selon l’invention ;
• est un schéma de principe d’une surface virtuelle épaissie selon une direction d’observation.

Other advantages and particularities of the invention will appear on reading the detailed description of implementations and embodiments which are in no way limiting, with regard to the appended drawings in which:

• is a block diagram of an exemplary embodiment of a system 1 according to the invention;
• is a block diagram of an exemplary embodiment of a system 1 according to the invention further comprising a projection of a three-dimensional media and observation positions;
• illustrates a construction of a projection surface of an exemplary embodiment of a system 1 according to the invention;
• illustrates two distinct types of possible patterns for the realization of system 1 of ;
• illustrates several variants of the system ;
• illustrates an alternative embodiment of the system of , including a privacy screen and a black background;
• illustrates an alternative embodiment of the system of , comprising an additional physical object in the projection field;
• is a block diagram of an exemplary embodiment of a method according to the invention;
• is a schematic diagram of a virtual surface thickened along an observation direction.

Description of embodiments.

The embodiments described below being in no way limiting, it will be possible in particular to consider variants of the invention comprising only a selection of the characteristics described, subsequently isolated from the other characteristics described, if this selection of characteristics is sufficient to confer a technical advantage or to differentiate the invention compared to the state of the prior art. This selection comprises at least one characteristic, preferably functional without structural details, or with only a part of the structural details if this part alone is sufficient to confer a technical advantage or to differentiate the invention compared to the state of the prior art.

In the figures, an element appearing in several figures retains the same reference.

Projection system

And represent a diagram of a projection system 1 according to one embodiment.

System 1 is intended to broadcast a 4’ three-dimensional media. The three-dimensional media can be a volume(s) and/or a surface(s) defined in space. The media can be fixed or animated.

• de formules mathématiques (surfaces, volumes….),
• d’objets 3D à base de faces (maillages 3D texturés),
• de nuages de points (issus de scanners lidar/photogrammétrie…),
• de cartes de profondeur, par exemple générées avec des caméras de profondeur de type Kinect®,
• de Gaussian Splatting,
• de champs de radiance neuronaux (NeRF).

The media can be generated from:

• mathematical formulas (surfaces, volumes, etc.),
• 3D face-based objects (textured 3D meshes),
• point clouds (from lidar/photogrammetry scanners, etc.),
• depth maps, for example generated with Kinect® type depth cameras,
• from Gaussian Splatting,
• of neural radiance fields (NeRF).

The 3D projection system allows the three-dimensional media to be displayed on multiple layers with a single projector. It allows multiple observers to view this object from different points of view, without the need to wear peripherals such as headsets, stereoscopic glasses.

The projection system is configured for passive relief perception of three-dimensional media from multiple viewpoints.

• un ordinateur 2, ou une autre source vidéo et un logiciel associé qui génère une image ;
• un projecteur 3 agencé pour projeter des images selon une direction principale, dite direction de projection, et présentant un cône de projection formé autour de ladite direction de projection ;
• une surface de projection 4 semi-transparente formée d’une pluralité de couches semi-transparentes, orientées transversalement à la direction de projection.

For this purpose, according to one embodiment, the system 1 comprises in particular:

• a computer 2, or other video source and associated software that generates an image;
• a projector 3 arranged to project images in a main direction, called the projection direction, and having a projection cone formed around said projection direction;
• a semi-transparent projection surface 4 formed from a plurality of semi-transparent layers, oriented transversely to the projection direction.

Thus, the projection surface can form a support for the three-dimensional media (4’) from projected images.

The spatial configuration of the semi-transparent layers defines the volume in which the 3D media can be displayed. The semi-transparent surface is characterized by its ability to diffuse light received on both sides, without sharp specular reflection, and without noticeable refraction effects (meaning that it does not deflect light in such a way as to change its initial propagation direction), while allowing the other layers to be seen through it.

The image may be generated by the software in real time or may be generated in advance and then broadcast later. In an alternative embodiment, the system may not include a computer, and the images may be prepared in advance and broadcast later by the projector. Very large three-dimensional media may be displayed.

Advantageously, the projection surface 4 forms a viewing support for all or part of an image projected by the projector 3 allowing the diffusion of the three-dimensional media 4’, visible from a multitude of points of view 9. The projection surface 4 is thus a diffusing surface.

The projection surface 4 can be arranged on a support structure 5 of the system, said support structure forming a frame for the projection surface.

The primary function of the frames is the adjustment of the thread tension.

More specifically, the support structure 5 may be formed of an upper frame and a lower frame each having taut support wires provided for positioning the layers.

Preferably, the frames form two surfaces parallel to each other, orthogonal and centered with respect to the projection direction. Other geometric constructions are possible: the frames may not be orthogonal to the projection direction, or may not be centered on the projection direction. It is also planned to manufacture a projection surface with symmetry of revolution around the main direction, thus offering greater regularity in the observation axis.

Each of the layers can be flat. Each of the layers can be a fabric, obtained from threads or fibers of one or more materials, synthetic or natural.

Each of the layers can be made of tulle. Tulle is a transparent and vaporous fabric formed by a network of regular meshes of fine threads of cotton, linen, silk, wool or, more recently, synthetic fibers.

The upper left subfigure of the represents a set Ts of upper segments forming a strip partition of the upper surface Ss. A set Ti of lower segments forms a strip partition of the upper surface Ss, each of the segments Ti being the projected image of a segment Ts when the projector is in operation.

A Ci layer is formed by associating an upper segment, Lsi, of the set Ts and a lower segment, Lii, of the set Tii.

The upper right subfigure of the represents a continuous projection surface defining the three-dimensional display volume. The lower part of illustrates the possibility of having non-parallel surfaces.

• dans l’exemple représenté sur la sous-figure a), la surface de projection est continue. Plus particulièrement, à l’exception des extrémités, chacun des segments inférieurs et segments supérieurs appartient à deux couches ; cette conformation étant dite en zigzag ; les couches présentent une normale formant un angle avec la direction de projection qui alterne entre angle aigu et angle obtus.
• dans l’exemple représenté sur la sous-figure b), la surface de projection est discontinue. Plus particulièrement, chacun des segments supérieurs et des segments inférieurs appartiennent à une seule couche. En outre, à l’exception des extrémités, le segment inférieur d’une couche est l’image par la projection du supérieur d’une autre couche ; les couches présentent toutes une normale formant un angle aigu avec la direction de projection.

As shown in , several layer conformations are planned:

• in the example shown in subfigure a), the projection surface is continuous. More particularly, with the exception of the ends, each of the lower segments and upper segments belongs to two layers; this conformation being said to be zigzag; the layers have a normal forming an angle with the projection direction which alternates between acute angle and obtuse angle.
• in the example shown in subfigure b), the projection surface is discontinuous. More particularly, each of the upper segments and the lower segments belong to a single layer. Furthermore, with the exception of the ends, the lower segment of a layer is the image by the projection of the upper one of another layer; the layers all have a normal forming an acute angle with the projection direction.

As shown in , according to a variant of the invention, the system 1 can further comprise a lens 6 functionally arranged between the projector and the projection surface. The lens parallelizes the optical paths, the semi-transparent surface offers better homogeneity of display volume density (and therefore a more homogeneous visual quality).

As shown in , according to a variant of the invention, the system 1 may further comprise an optical mirror 6 and/or a lens 6' functionally arranged between the projector and the projection surface. The mirror allows a return of the projection which may be useful in particular for reasons of space constraints or preferential orientation of the projector.

As shown in , the system 1 may further comprise a physical device that partially masks the system. The privacy screen 7 makes it possible to mask the projector, while the privacy screen 7' makes it possible to mask the residual light. The system 1 may further comprise a black background to trap and absorb the transmitted stray light.

As shown in , the system 1 may further comprise an object 8 placed between the projector and the projection surface, as illustrated in subfigure a, or at the projection surface, as illustrated in subfigure b. Either the object does not receive light from the projector, or the object is illuminated by the video projector, with conventional video mapping techniques.

Method of projecting a virtual object

• génération E1 d’un dispositif virtuel dual du système comportant :
  1. une caméra virtuelle présentant de mêmes caractéristiques que le projecteur,
  2. un maillage 3d, dit surface de projection virtuelle de mêmes caractéristiques que la surface de projection réelle ;
• calcul E2 d’une intersection tridimensionnelle virtuelle de la surface de projection virtuelle et du média en volume,
• calcul E3 d’une vue, aussi appelée rendu, de l’intersection tridimensionnelle par la caméra virtuelle,
• export E4 d’une image ou d’une vidéo et/ou envoi au projecteur du système.

illustrates a method of projecting a virtual object into volume using a system described above, comprising the following steps:

• E1 generation of a dual virtual device of the system comprising:
  1. a virtual camera with the same characteristics as the projector,
  2. a 3D mesh, called a virtual projection surface with the same characteristics as the real projection surface;
• E2 calculation of a virtual three-dimensional intersection of the virtual projection surface and the volume media,
• E3 calculation of a view, also called rendering, of the three-dimensional intersection by the virtual camera,
• E4 export of an image or video and/or sending to the system projector.

For example, if the 3D object is solid then the 3D intersection with each planar surface of the virtual projection surface will be a two-dimensional surface, if the 3D object is considered only as a surface in space, then the 3D intersection with each planar surface of the virtual projection surface will be a 1-dimensional curve of the thickness chosen to make it visible.

The characteristics of the virtual camera identical to those of the projector include: aperture, focal length, height and width ratio, horizontal and vertical shift, position and orientation relative to the virtual projection surface, optical distortion.

Advantageously, the method may comprise, when the system has a predetermined main observation direction, a 3D mesh of the virtual projection surface thickened according to the observation direction prior to the calculation step E2 of the virtual three-dimensional intersection. illustrates the thickening of each of the layers Ci to form a thickened virtual surface along a predetermined main observation direction d, as well as an object o to be projected. The determination of the projection of o is done from the calculation of the intersection of the object o with the thickened virtual surface, and not from the calculation of the intersection of the object o with the virtual surface.

The calibration of the system by aligning the projector beam with the physical system is remarkable.

• simulation temps réel en contexte de toute configuration du système pour évaluer sa qualité avant sa fabrication,
• affichage de la qualité visuelle du média en fonction de différents points d’observation,
• afficher des informations/dimensions sur les matériaux de construction,
• calibration du projecteur avec la surface de projection des images en temps réel.

The method may further comprise the following steps:

• real-time simulation in the context of any system configuration to assess its quality before manufacturing,
• display of the visual quality of the media based on different observation points,
• display information/dimensions on building materials,
• projector calibration with the image projection surface in real time.

Of course, the invention is not limited to the examples which have just been described and numerous arrangements can be made to these examples without departing from the scope of the invention. In addition, the various characteristics, forms, variants and embodiments of the invention can be associated with each other in various combinations to the extent that they are not incompatible or exclusive of each other.

Nomenclature

• 1: system
• 2: computer
• 3: projector
• 4: projection surface
• 4’: projected three-dimensional media
• 5: support structure
• 6: lens
• 6’: mirror
• 7: Screens
• 8: object
• 9: Observers

Claims (12)

1. Projection system (1) comprising:

   • a projector (3) arranged to project an image in a main direction, called the projection direction;
   • a projection surface (4) formed of a plurality of semi-transparent layers, each of the layers being oriented transversely to a projection direction, called local, on said layer, the projection surface forming a viewing support for an image projected by the projector.
2. System according to claim 1, further comprising a support structure (5) forming a frame for the projection surface, said structure comprising an upper frame and a lower frame, the frames forming two surfaces (Si, Ss), each of the layers (Ci) extending between an upper segment (Lsi) inscribed in the surface (Ss) of the upper frame and a lower segment (Lsi) inscribed in the surface (Si) of the lower frame.
3. System according to the preceding claim, in which the projection surface is continuous.
4. A system according to claim 1 or 2, wherein all layers have a normal forming an acute angle with the local projection direction or all layers have a normal forming an obtuse angle with the local projection direction.
5. System according to one of claims 1 or 2, in which all the layers have a normal forming alternately an acute angle and an obtuse angle, with the local projection direction.
6. A system according to any preceding claim, comprising taut support wires provided for positioning the layers.
7. System according to any one of the preceding claims, in which the projection surface (4) is made of tulle.
8. System according to any one of the preceding claims, further comprising a lens (6) and/or a mirror (6') operatively arranged between the projector and the projection surface.
9. System according to any one of the preceding claims, further comprising a physical device (7, 7') which partially or totally masks the system.
10. A system according to any preceding claim, further comprising an object placed between the projector and the projection surface or at the projection surface.
11. Method for projecting a virtual object into volume by means of a system according to any one of the preceding claims, comprising the following steps:

    • generation (E1) of a dual virtual device of the system comprising:
      1. a virtual camera with the same characteristics as the projector;
      2. a 3D mesh, called a virtual projection surface with the same characteristics as the real projection surface,
    • calculation (E2) of a virtual three-dimensional intersection of the virtual projection surface and the virtual object in volume,
    • calculation (E3) of a view, also called rendering, of the three-dimensional intersection by the virtual camera,
    • export (E4) of an image or video and/or sending to the system projector.
12. Method according to the preceding claim, in which the system has a predetermined main observation direction and the 3D mesh of the virtual projection surface is thickened according to the observation direction prior to the calculation step (E2) of the virtual three-dimensional intersection.