<Desc / Clms Page number 1>
PROJECTION DEVICE AND METHOD FOR
PLAYBACK AND CAPTURE OF IMAGE DATA The invention is in the field of rendering image data for a user while simultaneously capturing image data of the user, for example for devices in the field of so-called virtual reality ('VR') or for video conferencing and the like , It relates to a device, an installation and a method according to the preambles of the independent claims.
In virtual reality, an environment is simulated for a user in such a way that he perceives it as his own environment. Locally separated users or objects can virtually stay in one and the same VR environment and even interact with each other. A facility that offers this possibility is called a collaborative VR facility.
In the case of collaborative virtual reality devices as well as in video conferences and the like, there is the problem that image data is reproduced for a user while he is being recorded by cameras at the same time. This inevitably leads to compromises: While the best possible illumination is required for the user's camera recordings, such is disruptive for image reproduction on screens or projectors. Such a compromise can still be accepted in the field of video conferencing
<Desc / Clms Page number 2>
achieve satisfactory results; the demands of 'virtual reality'
But it can never be enough for projects. There, for example, an immersion is required. The user should only perceive the virtual reality.
An active one
Lighting or the visible presence of cameras etc. would interfere.
5 From the publication WO 02/08879 a method and a device are known which allow a user to simultaneously record image data in one
VR environment and a projection of image data into this VR environment. Two environments of the user are provided: in a first environment the image data is played back for the user, this environment is perceived by the user. In a second environment that is not perceived by the user, pictures are taken, possibly with active lighting. The two environments are separated by
Multiplexing, for example by time multiplexing. This can be done by going to
Providing the second environment the view for the user is periodically darkened for 15 seconds.
This method can also have the problem that cameras for taking images remain visible to the user. The cameras should have facial expressions and
To be able to record gestures by the user, consequently there is at least one
Camera directly in the user's field of vision. This can be distracting. In addition, at the point where a camera is located, the projection surface is interrupted
VR environment becomes patchy, immersion drops.
It is therefore an object of the invention to provide a projection device and a method that record data from a VR environment
Allow user or object while still undisturbed playback of image data for the user.
<Desc / Clms Page number 3>
The object is achieved by the invention as defined in the patent claims.
The method of reproducing images as the perceptible environment of a
User and for the simultaneous acquisition of image data of the user is based
5 on the fact that light is projected onto a projection body by projection means. Essentially simultaneously (i.e. simultaneously or quasi-simultaneously), light emitted by the user is at least partially transmitted through the projection body and, viewed by the user, is detected by recording means located behind the projection body. However, the projection body is configured, for example, or is influenced such that the projection body is essentially not transparent to a subset of the light emanating from the projection means.
Here, "transparent" is understood to mean the property that light can penetrate the projection body essentially directly without scattering.
15 Light that is transmitted through the projection body is preferably not perceptible to the user.
Multiplexing creates two environments: one from
User-perceptible, which cannot be recorded by recording means — because of the projection body that is not transparent to the corresponding light — and one that can be recorded by the recordings through the projection body
Environment in which the user or the object can be illuminated, for example.
<Desc / Clms Page number 4>
This can be done by alternately making the projection body transparent or opaque for visible light, and by making light on the
Is irradiated by the user while the projection body is transparent to the user
User is hidden by the user from the view
5 is darkened.
Alternatively, the projection body can only be used for a partial area of the visible
Light spectrum be transparent. To this part of the visible
Light belonging to the light spectrum is hidden from the user.
A device for reproducing images as a user's perceptible environment consequently has at least one in addition to projection means
Projection body with a projection surface on which the images through
Projection means are projectable. According to the invention, the
Projection body means on, a subset of the light striking it in the
To transmit essentially completely, as well as means to make the projection body 15 essentially non-transparent for a further subset of the light impinging on it.
For this purpose, the projection body can be switchable between a transparent state and an opaque state, as a result of which, in the transparent state
Acquisition of image data of the user through the projection body 20 is made possible. The switchable projection body is designed, for example, as switchable glass with at least one glass pane and an active layer, which is active
Layer between the transparent and the opaque state is switchable. The active layer can be arranged between two transparent electrodes
Be a liquid crystal layer. Such glasses are commercially available. The opaque state of a switchable glass allows the projection of image data without the
<Desc / Clms Page number 5>
Make projection diffuse or out of focus. Switchable glasses also have the property of being depolarizing.
On the user side, for example, liquid crystal shutter glasses can be used, even if the projection means emit polarized light. In this case, too, a stereoscopic display is not dependent on the inclination of the head of a user, which is otherwise the case when the shutter glasses are polarization-filtering.
However, it is also possible to use essentially ordinary glass and / or plastic panes for switchable projection surfaces and to use a corresponding shutter system, e.g. B. mechanical shutter. A shutter system can also be designed without additional disks, it is then necessary to pay attention to the inherent stability of the system. An example of a shutter system are lamellar strips of a, u. U. coated, materials that are suitable as a projection surface when closed, opaque. The shutter's field of vision should not be affected by the shutter, e.g. B. by aligning the shutter to the position of the receiving means.
Projectors such as "liquid crystal display" (LCD) and "digital light processing" (DLP) projectors, but also "cathode ray tube" (CRT) projectors, are preferably used to reproduce data. In contrast to the first two, the latter are relatively expensive and complex to adjust an image. In a preferred embodiment, switchable glasses and LCD projectors are used. As mentioned above, polarization effects from the polarizing light-emitting LCD projectors are eliminated by the fact that the projection body has a depolarizing effect due to multiple reflections in the liquid crystal layer of the switchable glass.
<Desc / Clms Page number 6>
The invention also relates to an installation for reproducing images as a user's perceptible environment and for simultaneously capturing image data of the user with a device of the type mentioned above and with recording means for recording image data and fading out means for selectively fading out light for the user. These can have shutter glasses, with additional synchronization means being provided by means of which a transparent / opaque clock of the projection body can be synchronized with an opaque / transparent clock of the shutter glasses. The shutter glasses are at least partially transparent when the projection body is in the opaque state.
Synchronization can be achieved by means of common electronics and / or wireless interfaces, such as infrared interfaces. For this purpose, the individual components have corresponding transmitters and / or receivers.
In this case, the reproduction and the recording of data take place, for example, by alternating and preferably periodically repeating reproduction and recording sequences. If these sequences are repeated at high frequency, the recordings of a user in a VR environment and the quasi-simultaneous reproduction of images are possible without the user being aware of the recording sequences.
The simultaneous recording and reproduction of images also allows different users or objects to stay in the same VR environment at the same time, which is reproduced for both users in separate VR locations. For this purpose, data transmission takes place, for example, by means of high-speed data transmission links which are connected to recording and projection means.
<Desc / Clms Page number 7>
In addition, the installation can have means for active lighting that can be synchronized with the clock mentioned. This serves to create optimal lighting conditions during a recording sequence, for example for the precise recognition of the texture or silhouette of a user. Lighting means, e.g. B. light-emitting diodes, lamps, stroboscopes etc., preferably send white light in order to prevent color-distorted images as far as possible.
As already described per se in WO 02/08879, the shutter glasses can be synchronized with the projection means in such a way that a 3D sensory impression can be generated for the left and right eyes by alternately projecting an image.
Alternatively, 3D perception can also be achieved by frequency multiplexing. Data is then reproduced using light of different wavelengths for each eye of the user. Monochrome 3D perception would be, for example, by superimposing two images, e.g. B. a red and a green image, and the corresponding red-green filters in a masking means of a user possible. Multi-color images can be achieved for example by assigning alternating, preferably as narrow as possible, frequency intervals of light for each eye. A user is then equipped with masking means, for example as glasses with corresponding glasses.
One idea on which an embodiment of the invention is based is that a projection body for a VR device is formed by a switchable glass.
<Desc / Clms Page number 8>
The procedure according to the invention makes it possible for the recording means and preferably also the projection means to not be visible to the user located in the VR environment. As a result, the requirements set out at the beginning can be met. It is therefore possible to optimally capture the user's gestures and facial expressions without disturbing or distracting the user. The entire projection surface is also available for the reproduction of image data without being interrupted, for example, by recording means or by cutouts for recording means.
At the same time, data about the user or the object can be obtained without restriction by means of the recording means.
The device according to the invention and the method are explained in more detail below on the basis of examples, embodiments and schematic figures.
1 shows a schematic top view of a projection device according to the invention, FIG. 2 shows a diagram of components of a projection device according to the invention, FIG. 3 shows the coordination of components of a projection device according to the invention during a recording and playback sequence,
<Desc / Clms Page number 9>
4a-c the coordination of individual components of further embodiments of the projection device according to the invention, with only one projector (FIG. 4a), without
Shutter (Fig. 4b) and with shutter glasses (Fig. 4c),
Fig. 5 and Fig. 6 shows a structure of an electrically switchable glass
5 contacting a switchable glass,
7 shows a section of a voltage characteristic curve of a switchable glass,
8 shows a simplified circuit of a switchable glass,
9 shows a possible functional principle of a vision device in a
Embodiment of the projection device according to the invention with 10 frequency multiplexing, which is used to generate a 3D sensory impression.
1 shows a user 1 in a VR environment. One or more, here three, switchable projection bodies designed as switchable projection walls 2 are arranged essentially at right angles to one another, so that they delimit a VR-15 location for the user. The projection screens separate the user
1 spatially from projection means 3 and recording means 4. At least one projection wall is assigned its own projection means 3 and recording means 4.
Projection means 3 are projectors, e.g. B. LCD projectors. A shutter or an aperture can also be assigned to each projector (not shown), so that the reproduction of images for each projector can be interrupted or regulated individually. The same material as that of the switchable projection screens can be used for the shutters or screens. But it can also
<Desc / Clms Page number 10>
specially made LC screens. The recording means are cameras which are positioned in such a way that they can record a user 1, or possibly also an object, at different angles. This enables the acquisition of 3-dimensional image data for another user who is in a different, spatially separate VR location.
The user 1 can be equipped with suitable switchable shutter glasses. The switching of the shutter glasses is matched to the switching of the projection walls 2 and is preferably controlled by a central control unit. Controllable lighting means are in the
Area of user 1 and, for example, attached above this. They are preferably attached to the upper and lower edges of the projection surfaces. They serve for the most uniform illumination of the user 1 and his to be recorded
Environment, so that the texture and silhouette of the user can be optimally recognized.
The use of shutters, switchable lighting and switchable
Viewing devices allow the environment for the user 1 during the
To design playback so that he only perceives this - an immersion, an immersion of the user 1 in his virtual reality becomes possible. During active lighting, the shutters of the shutter glasses are closed so that the
User 1 is not distracted by the brief lighting.
In Figure 1, an 'open' VR location is shown; H. the projection surfaces 2 are not completely arranged around the user 1. The inventive
However, the procedure also allows projection walls to be attached completely around a user without any projection and / or
Recording means disrupt the user's field of vision or the projection itself. Any lighting can also reach the user through the projection walls, so that the corresponding lighting means are also outside the
Area of the user can be attached.
Because projection and
<Desc / Clms Page number 11>
Illumination means do not have to be attached exactly in front of a user in order to fulfill their task, they can also be removed above a user in his immediate vicinity, for example.
As an alternative to the arrangement shown in the figure, the projection means
5 also project from the inside onto the projection body. You can do this, for example, over each projection body to illuminate the opposite one
Projection body be attached. Any resulting from this arrangement
Image distortions can be corrected mathematically.
FIG. 2 shows preferred components of an installation according to the invention and 10 shows an example of the linkage thereof.
One or more projection means 13 are connected to a data transmission device, e.g. B. a computer 18 connected. The data transmission device is preferably with
Data lines connected. As a result, external data can be fed into the computer 18 via data lines — this can be an entire network — and passed on to the projection means 13 for further use.
Computer 18 preferably controls e.g. B. via the corresponding
Synchronization means, the coordination of different components of the
Projection device. For this, the computer, e.g. B. having appropriate means to be connected via a synchronization path 19a with control units 20,21 20. However, it is also an unsynchronized operation of the whole
Plant possible. The control units 20, 21 are above others
Synchronization links 19b, e.g. B. infrared paths, with specific electronics 22, 23 in connection, which are attached to components or integrated into them.
Other components are, for example, one or more panels 17 or
25 shutters, which are assigned to the individual projection means 13. Furthermore, the
<Desc / Clms Page number 12>
Installation of at least one camera 14, at least one switchable projection screen
12, and lighting means 15. The four last-mentioned components are synchronized by common first electronics 22. A second electronics
23 controls the shutter glasses 16 of a user and is preferably in the form of a chip
5 integrated shutter glasses 16. The two electronics 22, 23 are synchronized with one another. They can also be coordinated with the image clock of the projectors via the infrared interface.
FIG. 3 shows how playback sequences can be coordinated with a recording sequence for one period each with three sequences 30a, 30b, 31. Two 10 sequences correspond to the projection for the left eye 30a and for the right 30b
Eye of a user, a third sequence 31 of the recording sequence. The table shows the state ('transparent' (open), 'black' (closed)) of the individual
Shutter of two projectors, named shutter projector left
Projector) 32a and 'shutter projector right' (shutter right projector) 32b, as well as 15 the shutter of the two glasses, shutter glasses left '(left
Shutterglass) 33b and shutter glasses right 33a.
In addition to this, the state of the illuminating means 34 (“illumination”) with “on” (on) or “off” (off), the state of the switchable projection surface (“screen”) 35 is transparent or “opaque”, and the state of activity of the camera ('camera') 36, 20.
During the first playback sequence for the left eye, the shutter is the left
Projector 32a, and the shutter for the left eye 33a switched to transparent.
The right projector shutter 32b and the shutter for the right eye 33b are closed accordingly, i. H. switched to 'black'. The corresponding reversal applies to the second playback sequence for the right eye.
During the playback sequences, the projection surface 35 is opaque
Illumination 34 off and camera 36 inactive. In the to the playback sequences
<Desc / Clms Page number 13>
subsequent recording sequence, all shutters 32a, b, 33a, b are set to 'black' -d. H. it is not projected, the lighting 34 is switched on and the
Projection surface 35 switched transparent so that the camera through the
Can take pictures through the projection surface ('takes a picture'). To the
5 Recording sequence again connects a first playback sequence.
The actual switching times do not necessarily match the idealized sequences. With this, non-idealities of the individual
Components are compensated. Carrier elements, such as certain LC
Circuits, more time is thus made available.
10 The length of a playback sequence t "is at the same time
Data playback and recording, e.g. approx. 12 ms, that of a recording sequence ta approx.
4 ms. This corresponds to a repetition frequency of approx. 60 Hz, the camera being switched on for practical reasons, for example, only every second, third or even fourth time, since it cannot take more than between 10 and 20 frames per second. The frequency of switching on a camera is with the
Processing speed associated with the amount of data to be processed and may deviate from this information when using newer techniques.
It should be noted here that the shutters before the project funds are by no means a necessary component. It is also conceivable that the 20 projection means themselves are designed in such a way that they only emit light during certain intervals (e.g. with modern digital light processing '(DLP) -
Technology); the projection means themselves would then be synchronized with the other components (shutter glasses, lighting means). It is even conceivable that a single projector sequentially provides the image for the right and left eye.
<Desc / Clms Page number 14>
The entire installation and its control can also be changed and / or simplified, for example in that no separate projection step is used for each eye, the user does not use shutter glasses or in that no shutter is connected upstream of a projector. Examples of these possibilities are shown in FIGS. 4a, 4b and 4c.
4a shows the timing of individual components in accordance with a simplified embodiment of the installation according to the invention with only one projector. Accordingly, there is only one playback sequence 30. The projector shutters can possibly be omitted. The entire control system is therefore very simple. During the playback sequence, the projector 32 projects images onto the opaque projection surface 35. The shutter of the shutter glasses 33 is opened accordingly and the camera 36 is inactive. During the recording sequence, the projector switches off, the projection surface becomes transparent and the shutter glasses 33 become opaque.
4b shows essentially the same situation as shown in FIG. 4a, but entirely without shutter or shutter glasses. Such an embodiment is simple in structure and control. However, it excludes the presence of strong active lighting. In a darkened environment, however, the perception of the projector and / or camera during the very short recording sequences does not have a very disturbing effect on the perception of the user.
This very simple embodiment is also especially for the representation or
Recording objects applicable in a virtual environment.
FIG. 4c shows an embodiment of a projection device according to FIG. 4a.
The same components are used, but the projector 32 is not dimmed during the shooting sequence. Only the shutter glasses 33 des
<Desc / Clms Page number 15>
The user and the projection surface 35 are switched in accordance with the reproduction or recording of data. If lighting means are switched on during the recording sequence, these illuminate the user to be recorded in such a way that the actual projection is outshone. If the image projection is also focused on the very precisely defined area of the projection surface, images are only vaguely perceptible to a user or are perceptible as blurred shades of color and do not or only slightly disturb.
The embodiments, as shown in FIGS. 4a, b and 4c, can also be suitable for evoking a 3D sensation in the user, although only one projection step is used. To do this, both eyes are fed simultaneously, but only the information intended for them. The shutter glasses 33 are provided, for example, with the corresponding frequency filters, which filter different frequencies depending on the eye or are permeable to them. The data for playback may have been prepared accordingly.
The structure of a switchable glass is shown in FIG. Between two transparent supports, e.g. B. glass panes 42, there is a liquid crystal layer 44 (LCD film). Two transparent, conductive layers 43, e.g. B. transparent conducting oxide (TCO) layers, which serve as electrodes for the liquid crystal layer 44, are arranged between the liquid crystal layer 44 and the glass panes 42. Instead of the glass panes, suitable plastic layers or panes are also conceivable, so that a higher breaking strength and possibly a lower weight of a projection screen made therefrom can be achieved. This is particularly advantageous in the case of mobile projection devices.
<Desc / Clms Page number 16>
To stabilize the switchable glasses or the projection walls, these preferably have an integrating frame. This takes into account, for example, the sensitivity of switchable glasses to selective loads. A frame is made, for example, of plastic, which can also be reinforced, e.g. B. with glass or carbon fibers.
FIG. 6 shows an example of contacting a switchable glass 52. Electrical contacts 53 are connected to the glass or the electrodes of the liquid crystal layers on each side of the glass 52. A voltage can be applied to these contacts, so that the switchable glass 52 can be switched between an opaque and a transparent state. LCD films are able to switch between these two states with almost no delay.
Typically, the glass is opaque when no voltage is applied and becomes transparent when a voltage is applied. In order to prevent the liquid crystals from remaining in alignment, the voltage can be adjusted e.g. B. periodically, reversed so that the voltage applied in the time average is 0.
A section of a voltage curve, as it essentially rests on a switchable glass, is shown in FIG. Note the voltage drop across the glass. This should not take place for too short a time. H. the slope v of the voltage curve should not be too steep. This is because contacts and / or electrodes or the conductive layers in switchable glasses can only be loaded to a certain degree. If the current load is too high, the contact electrodes and possibly also the liquid crystal layer are damaged.
This can be prevented by working with a current limit.
FIG. 8 shows a simplified and schematic circuit of a switchable glass. From the applied AC voltage V is a capacitance C
<Desc / Clms Page number 17>
the DC component of the voltage is filtered out. The capacitance C is correspondingly much larger than the intrinsic capacitance Cs of the switchable glass.
FIG. 9 shows a diagram of frequency multiplexing for generating the 3D sensory impression. The left and right sides of FIG. 9, designated L and R, correspond to exemplary frequency spectra of a filter spectacle lens for the left and right eye of the user, respectively. The wavelength X is plotted on the axes, which extends over the wavelength range visible to humans. Each filter side L, R has absorption bands 61, which cover the entire visible spectrum with the other filter half. The absorption bands 61 of a filter half L, R are arranged such that they alternate in the frequency domain. By arranging bands 61 that are as narrow as possible, it can be achieved that color impressions can be represented almost unadulterated.
The individual bands can also be arranged with different densities in different wavelength ranges, for example denser in the green area which is very sensitive to the eye than, for example, in the red area.
As an alternative or in addition to the generation of the 3D sensory impression, the principle of frequency multiplexing can also be used to implement the method according to the invention for the reproduction of images. For example, the projection body can be transmitting for a first subset of the frequency spectrum and opaque for a second subset. For example. filter glasses ensure that the user perceives only those light components that correspond to the second subset of the spectrum, which also prevents perception of components behind the projection body.
Additional filter means can be present and ensure that, for example, lighting flashes are also imperceptible to the user.
<Desc / Clms Page number 18>
Further combinations are conceivable according to the procedure according to the invention.
Thus, the polarization multiplexing described in the publication WO 02/08879 mentioned can also be used for the implementation of the method according to the invention and / or for producing the 3D sensory impression, in which case projection bodies other than the switchable glasses must then be used.