WO1998043414A1

WO1998043414A1 - Virtual studio projection system

Info

Publication number: WO1998043414A1
Application number: PCT/GB1998/000280
Authority: WO
Inventors: Michael Tamir; Avi Sharir
Original assignee: Orad Hi-Tec Systems Limited; Goodman, Christopher
Priority date: 1997-03-25
Filing date: 1998-02-02
Publication date: 1998-10-01
Also published as: GB2323733A; GB9706192D0; AU5872698A; ZA981511B; BR9801870A

Abstract

In a virtual studio the actor on the foreground area is provided with a projected plan of each 3D virtual object to be placed in the foreground area to enable the actor to move within the foreground area without 'colliding' with the virtual objects.

Description

VIRTUAL STUDIO PROJECTION SYSTEM

The present invention relates to a virtual studio projection system and more particularly to a system for providing actors in a virtual studio with an indication of the presence of virtual objects.

In a normal television studio or set there are usually provided several 3D foreground objects around which actors can move. The extent of such objects can be seen by the actors who are therefore able to move round the foreground objects. The background is usually provided by one or more suitably painted backdrops.

In an intermediate arrangement of a virtual set the background can be created and adjusted to camera view point by using a chroma-key panel or sensors to measure the orientation of the camera and focal length of the lens and a suitable background scene can be viewed with foreground objects superimposed on, for example, a video image. In this case real 3D foreground objects such as desks can be positioned in the foreground area and the actor or presenter can be seated at such a desk and can move in front of the desk.

However, if a virtual 3D object is "positioned" in the foreground area the actor cannot see it and in order to move round it the actor must estimate where it is. This is usually done by rehearsal.

If the virtual object is say, for example, a pillar or other stationary object, then the actor can learn its position. In some circumstances the position can be marked on the floor by a small mark which is small enough not to be too visible by the studio cameras. This may be acceptable if the object is small and of a regular shape, in which case the centre can be marked by a small cross but if the object is large or irregular in shape then the actor will not be able to move round it with any confidence. If several actors are involved in a scene then mistakes are usually made and a retake will be necessary if, for example, an actor walks "through" a "solid" virtual object. The situation is further compounded if the virtual object is able to move, as is the case of an animated character or a moving object.

It is an objective of the present invention to provide a means for enabling an actor to know where a virtual 3D object is. Preferably the actor will be provided with an outline of the virtual object continuously throughout a scene.

The present invention therefore provides a virtual studio projection system comprising projector means for projecting on to the foreground area floor of a virtual studio a projection plan profile of virtual 3D objects.

In a preferred embodiment the projection means comprises a video projector situated above the foreground floor area. In a further embodiment one or more such video projectors may be used to cover a large area.

The foreground area may preferably be a chroma-key coloured surface, for example blue.

The video projector may project on to the floor an outline in black and white which will appear to the actors eyes as dark/bright blue if the floor area is painted in plain blue. In this case the actor will see a contrast image of the projection plan outline of the 3D objects.

In accordance with a further embodiment, the TV camera and the video projector are synchronised such that they are not operative together

The TV camera is, for example, arranged to be operative for three quarters (75 %) of a video field by adjusting its electronic shutter and the overhead projector camera for the other quarter (25 %). Preferably, in a practical system, there will be a time delay between switching to ensure no overlap.

Preferably the synchronisation is done by adjusting the electronic shutter of the camera.

Thus, the actor can see the outlines continuously because they will appear 50 times each second. The actor may observe a slight flickering but this should not be intrusive. The outline will be clear to all actors.

If there are several actors and several 3D objects then the use of multiple synchronised video projectors will be beneficial ensuring adequate imaging of the projected plan views even if an actor blocks one of the projectors. Also if the stage foreground area is large, it may be desirable to use several video projectors.

In certain cases objects are not vertically straight. For example, a vase may be curved and an archway will have a curved top. In a preferred embodiment the size of the virtual object at the height of each actor is sampled and this is the projected plan. Thus, for an arch only the support pillars footprints would normally be projected. Embodiments of the present invention will now be described, by way of example with reference to the accompanying drawings in which :

Figure 1 shows schematically in side elevation a virtual studio system illustrating the position of a TV camera and a video projector;

Figure 2 shows schematically in front elevation a virtual studio system showing two video projectors;

Figure 3 shows in block diagrammatic form control circuitry for the arrangement of Figure 2; Figure 4 illustrates in a first embodiment a method for generating the floor plan of a virtual object;

Figure 5 illustrates in a second embodiment a method for generating the floor plan of a virtual object;

Figure 6 illustrates the generation of an orthogonal floor plan; Figure 7 shows block diagrammatically the system of the present invention with 3D animated models; and

Figure 8 is a flow diagram illustrating the operation of the system according to the present invention.

With reference now to Figure 1, a virtual studio 10 is shown schematically and comprises a chroma-key background panel 12 and a foreground floor area 14 which may also be a chroma-key colour. A TV camera 16 is used to photograph foreground objects 18 and electronic processing circuitry 20 is used to incorporate each foreground object 18 into a virtual background adjusted to camera viewpoint. The chroma-key panel may be of the type disclosed in PCT Patent Applications Nos. US95/05324 and GB96/02227 to the present applicant, the descriptions of which are incorporated herein for technical explanation. The invention can also be used for sensor based virtual sets. In the present invention a video projector 30 is mounted in a suitable position above the foreground floor area 14. The video projector 30 is provided with a lens 302 which enables it to project images over the floor area 14 as shown by dotted lines 304,306.

The video projector 30 is preferably linked to the studio electronic apparatus 20 to be able to be synchronised with the gate sync of the camera 16.

If required, as shown in Figure 2, more than one projector 30,32 may be used. This is beneficial to ensure approximately equal lighting for a large floor area which will be covered as shown by dotted lines 304,306,308,310. The two video projectors will be synchronised as indicated in Figure 3.

In Figure 3 the first and second projectors 30,32 are connected to a floor plan rendering machine 40 which in a preferred embodiment is synchronised to the studio apparatus sync 42 from apparatus 20.

The floor plan rendering machine receives signals from a 3D graphics computer 44 which may, for example, be a Silicon Graphics 0₂ model.

The output of the floor plan rendering machine will be explained with reference to Figures 4 to 8 but essentially comprises a pattern showing the orthogonal projected outline of all 3D virtual objects which the computer 44 is programmed to place into the area 14.

The projection can be a black/white pattern. If the floor area is painted blue then the black/white image will show up as dark/bright blue on the floor giving the actor a shaded pattern. If the chroma-keyer has a frequency bandwidth which will accept both shades (see above referenced co-pending applications) then neither shade of blue will show on the combined video output. However, such shades may be difficult for the actors to see especially if the 3D object is moving and the studio lighting is very bright. The actor may have to look carefully and this may detract from the natural movements/expression of the actor.

In the preferred embodiment therefore the pattern is a black and white pattern and the video image projector is a black and white projector. In a preferred example the white projection areas which appear as light blue are where the actors may move, the black areas which appear as dark blue representing the 3D objects.

Referring now to Figures 4 to 6, the floor plan rendering machine

40 generates a pattern which comprises for each 3D virtual object to be inserted into the area 14 an orthogonal plan view of the object at a height H. the height H can be different for each actor.

For the purposes of explanation, two example are given but firstly by reference to figure 6 the orthogonal plan outline is explained. The outline is not a perspective "view" of the object and is shown in dotted lines 324,326. This is generated as shown in figures 4 and 5 in which F represents the correct floor plan for an actor of height H and F¹ for an actor (not shown) of height H¹. As can be seen in Figure 4, the plan F represents the bulged part B of the urn 328, this being the largest orthogonal plan covered by the urn for an actor of height H. Thus, all areas outside F will be white and the are F (in this case a circle will be black). If, however, the actor is taller (or the urn is smaller) the relative height will be H¹ and in this case the actor will have to avoid area F¹ to not collide with the crown or rim C of the urn.

In Figure 5 the importance of this can be seen clearly for an archway 340. the actor can walk through the centre 342 of the archway which will be outlined as white but not through the support pillars shown as F. If the archway has side walls 344,346 then these will also extend die region F longitudinally thereby clearly showing the actor the opening in d e archway.

With reference now to Figure 7, it is possible also to introduce 3D animated characters using an animation controller 70.

In operation the animation controller 70 controls the movements of a 3D character. The character can be moved in accordance with a set computer program or can be moved using sensors attached to a human model.

The 3D character is in known manner displayed as a foreground object by a graphics computer 50. The character can be generated or stored as a 3D model in a store 60.

The 3D character will change in size and shape and this will be computed by the graphics computer 50. The computer can then be used to produce the necessary instruction to the floor plan rendering machine to ensure that the projected plan view of the 3D animated (or moving) character appears in the correct position and shape on the floor area 14.

An exemplary flow diagram is shown in Figure 8. Following start 800 and initial set 802, the basic floor pattern will be projected on to the floor.

With reference to Figure 7, if the pattern is in black and white a studio sync pulse 202 will be provided to the floor plan rendering machine 40. This sync pulse is used to control the ON/OFF times of the video projector 30 (and 32 etc if present). The video projector is switched on a maximum time period not longer than the period during which exposure of the TV camera 16 is off. Thus, if TV camera 16 is exposed for 75 % of a field period then video projector 30 can be operative for 25% . In a practical system the ratios would possible by 70% and 20% with a "dead" changeover period.

In this case the actor will see the projected plan for a period fifty times every second which will appear as a continuous display. The frequency allows the projected plan to follow the movement of animated characters or other 3D objects.

Once the projected plan has been set up and projected on the floor, if the 3D virtual objects are stationary then the plan will remain static. However, if the 3D virtual object changes shape or moves then it will be necessary to change the projection.

The animation controller 70 has the necessary data to prepare the 3D model. The machine may prepare the model in advance using, for example, SOFTIMAGE or Alias 3D model design codes.

The program interrogates the controller 70 to verify if any changes have been made to the 3D model (like motion or shape change (step 804)). The upper and lower clipping plane to floor height is set according to the presenter's height (steps 806,808). If changes are made men diese are incorporated at each change (step 810).

The viewing position for the top view is set (set (812) followed by setting of the orthogonal projection (step 814).

All 3 D model elements are changed to form a 2D black and white image (step 816) and this is transmitted to all video channels (step 818).

The program is then repeated.

Claims

1. A virtual studio including a virtual studio projection system comprising projector means for projecting on to me foreground area floor of a virtual smdio a projection plan profile of a virtual 3D object.

2. A virtual smdio as claimed in claim 1 in which me projection means comprises a video projector situated above the foreground area floor.

3. A virtual studio as claimed in claim 2 in which one or more such video projectors may be used to cover a large area.

4. A virtual smdio as claimed in claim 1 in which the foreground area may preferably be a chroma-key coloured surface.

5. A virtual studio as claimed in claim 1 in which the video projector may project on to the floor an outline in black and white but still within the range of the chroma-keyer.

6. A video studio as claimed in claim 5 in which a TV camera shooting the virtual smdio foreground area and the video projector are synchronised such that d ey are not operative together.

7. A virtual smdio as claimed in claim 6 in which the TV camera is arranged to be operative for less than diree quarters (75 %) of a video field and the overhead projection camera for less than one quarter (25 %).

8. A virtual smdio as claimed in claim 7 in which there is a time delay between switching of the video camera and the overhead projection to ensure no overlap.

9. A virmal smdio as claimed in any one of the preceding claims in which the size of the virmal object at the height of each actor is sampled and this is used to produce the projected plan.

10. A virmal studio including a projection system comprising projector means for projecting on to foreground floor area of the studio a projector plan profile of a virmal 3D object, the projection system being synchronised to electronic shutter means on the camera to project the plan profile only during periods when the camera shutter is closed.