FR2889758A1 - Three dimensional virtual object e.g. digital model of vehicle, visualizing method for e.g. showroom, involves inserting two dimensional representation of object in real time in video flow, and transmitting created enriched video flow - Google Patents

Abstract

The method involves receiving a video flow and inserting a two dimensional representation of a virtual three dimensional object e.g. model, in real time in the received video flow. An enriched video flow is created, where the enriched flow comprises the received flow in which the two dimensional representation of the virtual object is inserted. The enriched video flow is transmitted. Another video flow is received, where the latter flow permits to determine reflections of a real environment on the virtual object and modifies design of environment of the object. Independent claims are also included for the following: (1) a device comprising a unit for implementation of a method for visualizing a virtual three dimensional object in a real environment (2) a computer program comprising instructions adapted to implement steps of a method for visualizing a virtual three dimensional object in a real environment.

Description

System for visualizing a digital mockup integrated into a

real place

  The present invention relates to a system for viewing an integrated digital model in a real place or "increased showroom".

  The inventor has developed the product, hereinafter called "Fusion", which allows to insert real-time 3D objects in video streams (augmented reality technology).

  Today, in the industry, many products are first designed and modeled on a computer. Before its manufacture, the product is virtual, it is represented by its digital model.

  The present invention relates to a system for displaying a digital mock-up integrated in a real place, in particular an exhibition space, characterized in that it comprises: a first computer equipment comprising a plurality of acquisition cards, in particular of the type SD on a PCI bus, connected to a plurality of cameras, in particular via a composite video link or SVideo, and / or a second piece of computer equipment, connected to the first piece of computer equipment, comprising a high definition acquisition card connected to a high definition camera computer processing means associated with at least one graphics card for associating in real time synthesis images of the digital model with digitized video signals from the plurality of cameras and the high definition camera; visualization means making it possible to display the digital model of the synthesis images associated with the digitized video clips.

  This document describes a technology to visualize this perfectly integrated digital model in a real place, which we will call showroom.

  The innovations brought by the technology allow the following advantages: - N cameras can be used: this allows to visualize the digital model from different angles.

  - Cameras can be static or mobile. They can also have zoom changes (variable focus).

  - The technology is real-time (the cameras film the showroom in real time and the system renders the final image in real time).

  - The digital model can be animated in real time (ex: opening doors on a digital model of vehicle) - The digital model can be moved in the showroom: change of position, rotation etc. ..

  - It is possible to bring real products in the showroom to compare the digital model with respect to these (eg: comparison of a virtual vehicle with real vehicles).

  - Real products can be animated (ex: opening / closing real vehicle doors next to the virtual model).

  - The showroom can also contain natural persons.

  - One can also have reflections from the outside world on the digital model: o Static reflections o Dynamic reflections - The system can be connected to a videoconference rack to transmit images in real time to another site.

  - The cameras can be in high definition (HD standard), which allows to visualize the final result (enhanced model inserted in the real showroom) - View from each camera, the digital model can be partially or completely hidden by the real elements.

System description

  This document describes a typical configuration for the augmented showroom.

  Camera SD Motion Sensor (facnitati f) Motion Sensor (facnitati f) Digital Mock-up Location L _ J Actual Products SD Camera Motion Sensor (optional) Motion Sensor (facnitati f) SD Inputs HD Camera Motion Sensor (facnitati f ) Standard Ethernet PC 2 with MERGER HD Input VGA output Projector 1 Projector 2 Projection screen Explanations on the system The system below describes a typical augmented showroom configuration: The PC1 is equipped with 4 SD acquisition cards on the bus PCI. Four cameras are connected to PCl (for example via a composite video link or SVideo). Possibly, the four cameras or some of the four are equipped with motion sensors, which can handle either static cameras or mobile cameras.

  For example, the motion sensors are of the pan / tilt / zoom type if we are satisfied with rotation and zoom (the cameras are then at three degrees of freedom), we can also have totally mobile cameras using 6-way sensors. degrees of freedom (x, y, z, yaw, pitch, roll).

  PC1 will be able to display the four enriched video streams simultaneously, or it will also be possible to display a single enriched view (at the user's choice).

  The PC2 is equipped with a high definition capture card. At present, only one HD acquisition card can be inserted into the PC (bandwidth limitation on the motherboard), but the system can be generalized to N HD cards in the same PC, and allow the system simultaneous display of N enriched HD streams.

  Preparation before using the system Importing CAD models The digital model is produced using standard modelers on the market.

  The file is converted to the internal format of FUSION (directX format at present). In this case, the CAD models of the real objects that will be present in the showroom, for occlusion management, will also be imported.

  We then store on the hard disk: - 3D textured model (and animated if necessary) of the digital model - 3D model of the real objects present in the showroom. Note:

  - The number of polygons and textures must be compatible with the real-time image generation of the FUSION software.

  - Animations on the models: o Rotation.

o Translation.

o Scaling.

o Bones.

  Environmental Texture Realistic reflections can be applied to the digital model.

  Two possibilities: static reflections (the showroom is considered frozen) or dynamic reflections Static reflections To manage the reflections of the environment on the digital model, a static environment texture is created.

  Two methods are possible: - Model the showroom room (walls / floor / ceiling), then calculate the environment texture using a standard modeler with 180 degree camera settings (spherical environment mapping).

  - We bring in the showroom, where the numerical model will be located, a reflective sphere. We take a picture of this sphere, which will serve as a spherical environment map.

  Dynamic Reflections A video camera is installed where you want to display the digital model. Generally, this camera is fixed to the ground and looks up; it is also equipped with a wide angle lens (fish eye). The video stream is connected to the FUSION PC, it will be used to refresh the environment texture in real time. This allows for example to be reflected by the digital model, and see changes in reflections when moving or that we change the content or position of objects close to the digital model.

  System Calibration The optical distortions and field of view of the N cameras are calculated before the cameras are installed in the showroom.

  N cameras and PCs are installed in the showroom.

  At the place where the numerical model will be displayed, a real object is placed, whose characteristics are the following: - Preferably, the size of the real object is of the order of magnitude of the proportions of the numerical model.

  - The real object has been previously modeled, we have a CAD file that represents it.

  For each camera, we use the localization tool (described in the localization.doc file). We associate the characteristic points of the real object with their equivalent in the CAD model. This allows to obtain, for each camera, its pose (x, y, z, yaw, pitch, roll), in the absolute reference linked to the real object.

  Once the procedure is done, all the cameras are located in the same absolute reference.

  The actual object used for calibration is most often a cube or parallelepiped, preferably in a slightly expandable material.

  FUSION Augmented Reality Software Configuration for Enhanced Showroom Application Resolution - SD cameras are preferably in PAL quality: real-time video acquisitions will be at 768-point resolution by 576 lines.

  - The HD camera has one of the resolutions recommended by the HD standard: for example 1920 by 1080.

  - For optimal quality, the image generation is done at the native resolution of the videoprojector. If needed, the video images will be upscaled to the native resolution of the projector using the high quality texture resizing technology available on the graphics card (bilinear filtering).

  - The layout of the digital model is at the projector's native resolution.

  Features of the FUSION scene of augmented reality Display of the digital model perfectly integrated in the image of each camera. At startup, the software can load N digital mockups. It is then possible to choose sequentially the display of a numerical model among the N. For the current model selected, one can act on the following parameters: - change of material (texture, color ...) - change of position and orientation - launch animations (eg doors that open on a virtual car) Switcher option between the four cameras on the keyboard.

  Case of robotic mobile cameras: Pan / Tilt / Zoom of each camera controllable in real time with the mouse, the keyboard or any other pointing device (ex: joystick).

  Real time adjustment function of the virtual light sources to harmonize with the real lights of the Showroom.

  Display of characters around the digital mockup (billboard management), function of setting the positions of billboards on the floor of the showroom.

  Management of the 3D masks of the real objects to obtain a concealment of the digital mock-up by the 3D objects (by using the model CAD of the real objects occultants the numerical model).

  Acquisition and management of the static or dynamic environment texture of the room at the location where the virtual vehicle will be located.

    Billboards for characters For the display of scale 1 characters around the digital model, the method chosen is as follows: We photograph each person face and back (see photo).

We cut off the character.

  We use the front photo to texture a 3D object that is actually a simple rectangle.

  In real time, this rectangle is always positioned perpendicular to the optical axis of the camera (see photo).

Claims (1)

Claim   1. System for displaying a digital mock-up integrated in a real place, including an exhibition space, characterized in that it comprises: a first computer equipment comprising a plurality of acquisition cards, in particular of SD type on bus PCI, connected to a plurality of cameras, in particular via a composite video link or SVideo, and / or a second computer equipment, connected to the first computer equipment, comprising a high definition acquisition card connected to a high-definition camera, computer processing means associated with at least one graphics card for associating real-time synthesis images of the digital mock-up to digitized video signals from the plurality of cameras and the high-definition camera, - display means to visualize the synthesis images of the digital model associated with the digitized video signals.