ES2332561A1 - Interactive visual immersion system (Machine-translation by Google Translate, not legally binding) - Google Patents

Abstract

This system comprising a display screen (1) connected to a computer system (2) to generate an interactive animation (3) with a user (4), said computer system (2) being connected to a video camera (5) of capturing a scene before said display screen (1) for capturing the image (51) of the user (4) for its interaction with the interactive animation (3), finding a screen (6) illuminated with visible light or light not visible in the background of the scene, which one wishes to capture, and in which the video camera (5) is focused. The video camera (5) is sensitive to the light of said screen (6) to allow the computer system (2) to determine the outline or silhouette (41) of the user (4) in said scene in contrast to the light emitted or reflected by said screen (6). (Machine-translation by Google Translate, not legally binding)

Description

Interactive visual immersion system.

Object of the invention

The present invention relates to a system of interactive visual immersion in which a user can interact with a computer generated image or animation and displayed in a Display or display device.

Background of the invention

Virtual reality systems are known in those that the user, through appropriate tools is immersed or introduced into a dummy or simulated environment that is generated by computer, mainly through audio and video, although it can incorporate, among others, touch sensing devices or manual or body interaction.

So, an example of these reality environments virtual is described in US patent US6409599 in the that an interactive representation enclosure is described in virtual reality for one or more users, using it personal viewers that users wear on their heads and manual portable command devices for the user can exercise limited control over one or more computers, the which recreate a scenario with prerecorded elements and elements generated at the moment. This system can be used both for recreational use as an educational, but the execution of a session requires a previous preparation time for learning by the user and use is limited to users equipped with the aforementioned display and control devices, that is to say that users need to wear specific garments and also manipulate specific devices, which makes system in an expensive system.

Other virtual reality systems or "almost-virtual", as described in the patent American US5534917 mainly comprise a screen of large-scale display connected to a computer system to generate an interactive image or animation with the user, said computer system being equipped with software suitable and connected with a video camera capturing a scene before the aforementioned display screen for capturing the image of the user, users or objects arranged in front of the same for its interaction with the aforementioned image or animation shown, depending on their position and / or movement. In this environment the user or users observe on the screen that incorporated an image or graphic object that corresponds to your image, and that interacts with other elements of the environment according to their movements or actions, without the need to manipulate controls or dispositives. Although an interactive visual immersion system of this type is more limited with respect to the recreation of a realistic simulation that a complete virtual reality system, It has numerous advantages and applications, especially in environments advertising, leisure and information provision. This is so, because it is more robust and less expensive than a system of full virtual reality, with the advantage that if the screen of display is considerably large, allows use more intuitive and with greater participation of several users simultaneously, without requiring any previous preparation users

In this patent US5534917 for the determination of the user's position and their actions, the signal is treated through shape and movement recognition software, according to different techniques, so that through an operation of comparison ("AND" function) the image taken is compared with a map stored in memory to determine if the user or participant is activating areas of interest, for example "buttons simulated "and other elements, which is beneficial because the user or users do not need any previous preparation, can enter or leave the interaction space with the system with total freedom and it is very intuitive. However this software is limited in its possibilities according to the technique used, using The techniques of:

- "Contour extraction". This technique detects and trims the silhouette of the user or user, by means of a Comparison with the background of the image. The user can interact with the system even if it is still, but it presents problems, since if the user can be confused with the fund it may not be detected or poorly detected. This technique requires having previously recorded the background without users, so any change in the background, for example lighting, causes Detection of false objects.

- "Body tracking". This technique is intended for the location of head, eyes, hands, feet, elbows and other parts of the user's body. By itself it can be ineffective if the scene is not contrasted, which is combined with Another technique to improve its functionality.

- "Gesture tracking". This technique has as a basis the recognition of certain gestures of the face, as per example smile, or hand gestures, such as pointing, or gestures of the body, how to jump.

- "Motion detection". This technique It is the most robust and is based on locating where there are movement and its intensity. If a static or little movement, the user arranged in the foreground of the scene it causes areas of the hand or other part of the body with zones Easily detectable for interaction. However if the User is still or moves too slow, it is not detected.

- Other techniques are applicable as the "object tracking" in which certain are recognized and followed objects, "Pattern Tracking", a technique that recognizes patterns and others.

This invention still fulfilling many fundamental characteristics required of this device, it has major limitations such as not using more techniques computer vision and physical interaction to enrich the interaction between the user and the screen or image generated in the same and not being adaptable the equipment and the system to different light environments.

Also US Patent US5913727 describes an interactive game system, more specifically for personal combat games, in which a screen of large visualization and that, in addition to representing the animated image of the fighter emits a selective sweep of light not visible simultaneously. The player willing in front of said screen presents a plurality of sensor devices to said light emissions not visible on your body, so that these sensors connected to the computer that controls the image in the screen is able to determine in real time the position and the player movements to move the fighter represented. To improve the feeling of realism the system is equipped with coordinated three-dimensional vision glasses With the image generated on the screen. This system too require prior preparation, as the user or player must Equip yourself with sensors on your clothes and vision helmets in three dimensions.

In these partial immersion systems or interactive, recognition of the user's position and their action the device makes a comparison between the successive images that are captured, so that the areas of difference indicates areas where the user moves. This It poses a serious problem, since if the user is still or moves very slowly, these zones of difference are not significant enough to provide a trace positive, leading to process errors.

Another disadvantage is that the user has to wear the sensors This is not feasible in a place with a lot of traffic as a Mall. It would slow down and generate rows. Is little natural.

Description of the invention

The interactive visual immersion system, object of this invention, presents some technical peculiarities aimed at improving the operation of screen systems interactive image display and to achieve greater robustness and reliability of operation.

This interactive visual immersion system comprises an interactive display screen, connected to a computer system to generate an interactive image or animation with one or more users arranged in a scene in front of the aforementioned visual display. This system has especially a scene background screen, illuminated with visible light or non-light visible and opposite to the display screen with respect to user, users or interacting objects. Additionally the system comprises a capture camera, focused on the screen background for capturing the scene, and preferably arranged under the display screen, although it could be placed in any other location, as above, below or displaced sideways, for which the use of a image perspective correction software (keystone).

The camera is especially sensitive to this light source or radiation transmitted by the background screen illuminated, allowing the computer system to determine the contour of the user, users or existing objects in that scene by contrast with the light emitted or reflected by said screen of background, similar to a backlight With this system the contrast between the background and the silhouette, either through a light-dark or a color-no color, as happens for example in the audiovisual technique of "Chroma" which consists in replacing one fund with another by computer Basically this technique is to reproduce a solid and uniform color image background and the object or person that you want to insert into the image leaving it to the computer replacing the image with that solid color with another chosen to will and keeping the object or user that overlaps the Image generated from the computer.

This background screen and its arrangement allow a better capture and a visualization of the captured forms that allow an optimized treatment in the computer system, since the camera only needs to recognize at one point two values of lighting, corresponding to the light or radiation received from background, and therefore high intensity, and in the shadow caused by the user filed, of low or no intensity.

The background screen consists of a first embodiment by a backlit panel with one or more visible or non-visible light sources, setting up a field of homogeneous intensity that defines the background of the active scene, in the that the capture camera works. All that part that is outside of said scene area occupied by the screen of fund is discarded directly by the computer system, given its high contrast, no need to filter useful areas As in other systems. This backlight can be achieved using LED or fluorescent diode panels, both visible light as not visible

In a second embodiment, perhaps less advantageous, but not therefore disposable, the background screen is constituted by a panel or a light reflecting surface visible or not visible projected by one or more sources of lighting arranged in front of said screen. This allows the system has a quick installation anywhere, since one or two spotlights and a reflective screen are easy to dispose in a professional way and its transport is fast and comfortable. It is also possible to use contrasting colored screens if the place of installation is subject to the effect of sunlight or sources of variable light, improving the capture performance of the scene, for example using the color green.

The light transmitted or reflected by said background screen can be infrared light, which is not visible to the human eye and therefore the scene is more natural and integrated into the environment, increasing the "magic" effect of the user interactivity.

Likewise, the use with ultraviolet light treatment, especially in environments where interested in creating the background effect "not visible" to the user, but that may be affected by solar radiation or sources of other people's infrared energy. This ultraviolet light can be projected on a phosphorescent background screen, so that screen can be illuminated in a specific color, but do not form visible light cones between the focus and the screen.

For greater explanatory clarity one could say that the three types of presentation, not only possible that there would be more, if not more, feasible that would depend on the light conditions of the installation environment to achieve the cut of the silhouette would be the following:

1.- With infrared lights for environments with low sunlight and lots of ambient light.

2.- With visible lights and white blur screen , for environments with low sunlight and low ambient light.

3.- With visible lights and green or blue diffusing screen ("Chroma") for environments with lots of sunlight and lots of ambient light.

The background screen can present a filter reflective or refractive of light in a bandwidth around a specific wavelength to favor the creation of the fund to capture.

In a case preferred, the computer system that treats the image captured by the video camera, comprises at least two connected computers by appropriate means of communication, such as a network cable with "ethernet" protocol. The video camera is connected to a first computer that using the appropriate software, for example "Openframeworks" performs the capture, analysis and operation filtering of unwanted objects, to obtain, with a large Sharpness, silhouette or silhouettes of users on stage. further you can make distortions corrections to the image of the scene because of the low arrangement of the camera or because the User too close. Once defined the silhouette in the scene captured the image data of the silhouette is sent in a format encapsulated to the second computer. This communication can use free software protocols, such as XML or protocols Owners developed for this purpose.

This second computer is connected to the display screens for image generation or interactive animation to be displayed, taking into account a matrix film or preconfigured graphic environment, for example using "Flash" software, on which the silhouette is mounted specified in the captured image that has been received from the first computer. Thus, the user sees his silhouette captured on the screen of visualization as if it were a mirror, interacting with the others graphic objects displayed, such as icons and similar elements in real time. The interaction in the image shown on the screen of display, occurs based on the "collision" between any point of the silhouette and graphic objects, but also Other aspects such as speed and direction of the movement, the inclination of the silhouette, the position of a part of the body with respect to others, the gestures that you can do so much with his body as with his hands, his voice, etc. So the person can interact with the hair, with your torso, hands, arms and others  body parts trimmed by the silhouette. Since the silhouette is always present, the interaction occurs independently the user is moving or not (the user still interacts being still).

To encourage the feeling of immersion, the display screen is of considerable dimensions, comprising a plurality of forming devices of images for the formation of a large composite image, such as a flat screen videowall, which in a example of embodiment can take the configuration of a panel advertising

The video camera is preferably of the type digital although the use of analog cameras is not disposable, with which in the case of digital cameras, directly supplies the image captured in digital format to the first computer of Image processing of the computer system. This camera it presents a sensitivity to light in a bandwidth corresponding to the bandwidth in which the wavelength of the light coming from the background screen, whether it is visible or not visible to the human eye, increasing the contrast of the capture, and therefore its effectiveness and resolution. In an exemplary embodiment this is achieved by a filter. suitable pass band optic in video camera optics and whose pass band corresponds to the wavelength of the light from the background screen.

These and other characteristics of the claims can be seen in the preferred embodiment that  described below.

Speakers will also be included, for which the software will be prepared to emit sounds also depending on the interaction. For example, the speed of your movement: more faster sharper, slower more serious.

It is planned to show the image of the cropped person (video) and not only his silhouette which is very useful for game themes, in amusement parks, it could be interesting that the user saw his own body integrated in a 3D environment It would be the case of the soccer game in which the user it is integrated into the stage in front of a soccer goal and You must cover the balls with your hands.

Description of the figures

To complement the description that is being performing and in order to facilitate the understanding of characteristics of the invention, is attached herein descriptive a set of drawings in which, with character Illustrative and not limiting, the following has been represented:

- Figure 1 shows a perspective view of an example of realization with a user in the scene.

- Figure 2 shows a block diagram of the system.

- Figure 3 shows a scheme of the capture of the image with a backlit background screen.

- Figure 4 shows a scheme of the capture of the image with a background screen illuminated by spotlights

- Figure 5 shows a representation of a image captured by the camera and the image displayed on the screen of display.

- Figure 6 shows a flow chart of the system operation

Preferred Embodiment of the Invention

As can be seen in the figures referenced, the interactive visual immersion system comprises a display screen (1) connected to a computer system (2), provided with two computers (21, 22), to generate an image or interactive animation (3) with a user (4), users or objects arranged in a scene in front of said display screen (1), the computer system (2) being connected to a video camera (5), which is arranged under the screen of display (1) at a low height, and focused on the scene in front of the display screen (1). In the part after the scene a screen (6) of background, active lighting for capture by said camera video (5), defining the work area, being the image (51) captured and processed by the computer system (2) to visualize the treated silhouette (31) of the user (4) on the display screen (1), together with pre-recorded or generated graphic objects (32) by the computer system (2).

In the example shown in Figure 3, the background screen (6) is backlit by a curtain (61) of LED infrared diodes, arranged behind a panel (62) noticeably translucent, to get the screen (6) of background transmits to the scene an infrared, homogeneous light, intense and cold This is important since both LEDs have been searched as Fluorescent to achieve this goal: heat would result uncomfortable for the user and also dangerous for a structure narrow). With this arrangement, the video camera (5) comprises a optical filter (52) of infrared pass band.

In the example shown in Figure 4, the background display (6) is illuminated frontally, comprising this case an anterior focus (63), with a beam of light projected on said opaque screen (6) to highlight a light color effect over a wavelength using a surface filter (no represented), which in this case is a pigment of a paint.

The display screen (1) is constituted by a "videowall" configured by an array of imaging devices (11), such as screens individual plasma technology in this case, getting the formation of a considerable visual area, suitable for use as interactive ad.

It is an option to offer a second videowall connected to the system, but located elsewhere, for example, to the entrance of a fair to enhance the advertising effect and that a wider audience can also see the commercial.

The computer system (2) comprises two computers (21, 22) connected by means of communication  (23) suitable, such as an Ethernet connection. The first computer (21) of the computer system (2) is connected to the camera video (5) for image processing (51) by "Openframeworks" to perform an appearance correction (24) and distortion and a detection (25) and highlight the contour of the silhouette (41) of the user (4) or users. Once the image is treated in said first computer (21) it is subjected to an encapsulation (26) for transmission to the second computer (22).

The second computer (22) is connected to the display screen (1) and comprises a graphic environment (27) with a matrix movie (28) in "Flash" software in which incorporate the silhouette data (41) received from the first computer (21) to configure the interactive animation (3) to be will display on the display screen (1).

Once nature is sufficiently described of the invention, as well as a preferred embodiment, for the appropriate purposes that the materials, form, size and arrangement of the elements described may be modified, as long as this does not imply an alteration of the essential features of the invention claimed in continuation.

Claims (10)

1. Interactive visual immersion system, of the type comprising a display screen (1) connected to a computer system (2) to generate an interactive image or animation (3) with the user (4), said computer system being (2) equipped with suitable software and connected to a video camera (5) capturing a scene before the aforementioned display screen (1) for capturing the image (51) of the user (4), users or objects arranged in front of said video camera (5) for its interaction with said interactive image or animation (3), depending on its position, gestures and / or movements, characterized in that it comprises a screen (6) illuminated with visible light or non-light visible, arranged at the bottom of the scene, behind the user (4), users or objects to be captured, and on which the video camera (5) is focused, this video camera (5) being sensitive to this source of light or radiation, to allow to the computer system (2) determine the contour or silhouette (41) of the user (4), users or objects existing in said scene by contrast with the light emitted or reflected by said background screen (6). 2. Interactive visual immersion system according to claim 1, characterized in that the background screen (6) comprises a backlit panel (62) with one or more visible or non-visible light sources according to a homogeneous intensity field. 3. Interactive visual immersion system according to claim 1, characterized in that the background screen (6) is constituted by a panel (62) or reflective surface of visible or non-visible light projected by one or more lighting sources arranged in front of said screen (6). 4. Interactive visual immersion system according to claim 1, characterized in that the light transmitted or reflected by said background screen (6) is infrared light. 5. Interactive visual immersion system according to claim 1, characterized in that the light transmitted or reflected by said background screen (6) is ultraviolet light. 6. Interactive visual immersion system according to claim 1, characterized in that the background screen (6) has a reflective or refractive filter of light in a bandwidth around a specific wavelength. 7. Interactive visual immersion system according to claim 1, characterized in that the computer system (2) comprises two computers (21, 22) connected to each other by means of communication (23), the video camera (5) being connected to a first computer (21) for capturing and analyzing the image of the scene and the second computer (22) connected to the display screen (1) for generating the image or interactive animation (3) to be displayed. 8. Interactive visual immersion system according to claim 1, characterized in that the display screen (1) comprises a plurality of imaging devices (11) for the formation of a large composite image. 9. Interactive visual immersion system according to claim 1, characterized in that the video camera (5) has a sensitivity in a bandwidth of the captured light corresponding to the bandwidth of the light coming from the screen (6) of background. 10. Interactive visual immersion system according to claim 9, characterized in that the video camera (5) comprises an optical bandpass filter (52) corresponding to the wavelength of the light coming from the background screen (6) .