Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
  • H04N13/30—Image reproducers
  • H04N13/363—Image reproducers using image projection screens
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
  • H04N19/597—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding specially adapted for multi-view video sequence encoding
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
  • H04N13/20—Image signal generators
  • H04N13/204—Image signal generators using stereoscopic image cameras
  • H04N13/239—Image signal generators using stereoscopic image cameras using two 2D image sensors having a relative position equal to or related to the interocular distance
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
  • H04N13/30—Image reproducers
  • H04N13/332—Displays for viewing with the aid of special glasses or head-mounted displays [HMD]
  • H04N13/341—Displays for viewing with the aid of special glasses or head-mounted displays [HMD] using temporal multiplexing
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
  • H04N13/30—Image reproducers
  • H04N13/398—Synchronisation thereof; Control thereof
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
  • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
  • H04N13/106—Processing image signals
  • H04N13/111—Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation
  • H04N13/117—Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation the virtual viewpoint locations being selected by the viewers or determined by viewer tracking
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
  • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
  • H04N13/189—Recording image signals; Reproducing recorded image signals
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
  • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
  • H04N13/194—Transmission of image signals
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
  • H04N13/20—Image signal generators
  • H04N13/286—Image signal generators having separate monoscopic and stereoscopic modes
  • H04N13/289—Switching between monoscopic and stereoscopic modes
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
  • H04N13/30—Image reproducers
  • H04N13/332—Displays for viewing with the aid of special glasses or head-mounted displays [HMD]
  • H04N13/334—Displays for viewing with the aid of special glasses or head-mounted displays [HMD] using spectral multiplexing
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
  • H04N13/30—Image reproducers
  • H04N13/332—Displays for viewing with the aid of special glasses or head-mounted displays [HMD]
  • H04N13/337—Displays for viewing with the aid of special glasses or head-mounted displays [HMD] using polarisation multiplexing
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
  • H04N13/30—Image reproducers
  • H04N13/361—Reproducing mixed stereoscopic images; Reproducing mixed monoscopic and stereoscopic images, e.g. a stereoscopic image overlay window on a monoscopic image background
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
  • H04N13/30—Image reproducers
  • H04N13/366—Image reproducers using viewer tracking
  • H04N13/368—Image reproducers using viewer tracking for two or more viewers
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
  • H04N2013/0074—Stereoscopic image analysis
  • H04N2013/0081—Depth or disparity estimation from stereoscopic image signals
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
  • H04N2013/0074—Stereoscopic image analysis
  • H04N2013/0085—Motion estimation from stereoscopic image signals

Definitions

• This invention relates to methods of providing three- dimensional (3D) images using a restricted transmission channel such as conventional television transmission.
• spectacles incorporating electro-optical devices which are alternatively transparent and obscured in synchronism with left and right eye information.
• Such spectacles have been used in conjunction with computer generated images to visualise building designs, for example, three dimensionally. They have not, however, been used hitherto in conjunction with television.
• the present invention seeks to provide a method which uses conventional television transmission to provide 3D images.
• One form of the invention provides images which can be manipulated, and is capable of providing a virtual in-the-round display.
• the present invention in its broadest aspect, provides a method of presenting a 3D image, in which a scene is captured by at least two spatially separated video cameras to provide signals representing at least two viewpoints, said signals are conveyed to a viewing apparatus by standard television broadcast or video recording techniques with frames of alternate viewpoints interleaved to produce time sequential frame displays, and the viewing device is observed by the viewer through eyepieces which are rendered transparent and obscure in synchronism with the frame rate.
• the invention provides a method of providing 3D images of an event in which one or more participants (which may be persons, animals, or inanimate things) are predefined as objects, wherein video data defining each object is created and transmitted prior to the event, and at least one object is defined as a series of salient points and offsets therefrom, and during the event sequential frames are transmitted each of which contains data, which may be coded and embedded digital data, defining the positional co-ordinates of said salient points.
• each frame is partially occupied by said data and partially by conventional video signals.
• Said positional co-ordinates will typically be X, Y, Z co-ordinates obtained from video signals generated by multiple cameras.
• line of sight vectors through salient points are established by video processing software and resolved for the intersection co-ordinates.
• Said offsets are preferably defined as angles or polar co-ordinates from an axis joining two salient points.
• the 3D information is recreated at a virtual venue in the form of a floor or table surface around which viewers sit or stand. Background video scenes may be projected on walls around the viewers. Video may also be used to project faces onto facial surfaces of participants.
• a further aspect of the invention provides a 3D television system in which a scene is imaged by a pair of cameras in binocular fashion to provide left and right signals, said left and right signals are interleaved and transferred to a television set by normal broadcast, cable or record/replay means to produce a picture display consisting of alternate left and right frames, and the picture display is viewed through spectacles of the kind defined above.
• the broadcast signal may suitably include a code at the commencement of each frame defining it as left or right .
• the code may be decoded by a decoder circuit which may be in the form of a set-top box added to a conventional television set, and may include means (such as an infrared link) for transmitting a synchronising signal to the spectacles.
• Fig. 1 is a perspective schematic view of viewers sitting around a blank floor-mounted screen, waiting for an image, generated by a 3D projection system in accordance with the first embodiment of the present invention, to be displayed on the screen;
• Fig. 2 is a perspective schematic view of the viewers sitting around the floor-mounted screen of Fig- 1, with the 3D projection system in accordance with the first embodiment of the present invention displaying a tennis match on the screen;
• Fig. 3 is a front (first direction) view of a footballer being filmed to define the player as an "object" in accordance with the first embodiment of the present invention;
• Fig. 1 is a perspective schematic view of viewers sitting around a blank floor-mounted screen, waiting for an image, generated by a 3D projection system in accordance with the first embodiment of the present invention, to be displayed on the screen;
• Fig. 2 is a perspective schematic view of the viewers sitting around the floor-mounted screen of Fig- 1, with the 3D projection system in accordance with the first embodiment of the present invention displaying
• Fig. 4 is a perspective schematic view of a plurality (in this case two) of stereo cameras capturing the action of a football match which will be transmitted by a transmission means to the 3D projection system of Figs. 1 and 2; and Fig. 5 is a schematic diagram of a 3D projection system m accordance with a second aspect of the present invention.
• a first embodiment allows viewers using 3D visual techniques to view an event such as a sporting event as an instantaneous 3D reproduction m high resolution graphics.
• the events can be viewed or re-viewed from any viewpoint and direction or even from the point of view of any participant m the event.
• part of a football game could be reconstructed from the viewpoint of the referee or the goalkeeper, a record breaking pole vault or high dive could be viewed from the viewpoint of the athlete.
• a real-time computer reconstruction will be under the control of the viewers. It would even be possible to have four stereo viewing images running simultaneously so that viewers could sit m a separate venue and view the event m full 3D from four sides on a single screen on the floor.
• the 3D projection system consists of a high speed computer system which generates/reconstructs the images and uses a downward facing vertically mounted screen projector to display the images on the floor, which can then be viewed using the synchronised left eye/right eye glasses as described above.
• the computer also generates an infrared synchronisation signal, which is received by and infrared sensor provided on the glasses, to keep the glasses in time with the appropriate frame.
• the high speed computer system is capable of producing multiple channel sets of left eye/right eye views.
• Each side sees the event as if they were viewing the event live at the real venue ( Figure 2) .
• Separate left eye/right eye viewing sets will be required for all four directions of view. For example, North side would see Southward view in frames 1 and 4 while the East side would see Westward view in frames 2 and 5 and so on.
• each participant In order to provide this with only conventional television, it is necessary first of all to define each participant as an 'Object'.
• Each player in a football match would be an 'Object' as would the referee, the linesmen and the ball.
• the detailed dimensions and colours of each object would be transmitted before the event to the 3D visualisation receiving computers in digital video form by filming each object from at least 4 directions ( Figure 3) .
• Each object will be stored as a series of salient points and rotational offsets. For example the centre of a knee would be a salient point as would the centre of a heel and all points in the lower leg would be stored as offsets along the line from knee to heel and rotationally at right angles to that line. This definition allows the 'Object' to be reconstructed if only the salient points x, y, z co-ordinates are known.
• the resolution with which viewers will ultimately be able to see the objects will be the resolution created when the object filled the frame which will always be better than large scale TV shots from a distance.
• the fixed background, the pitch, the goals, the corner flags etc will also be defined as 'Objects' but will not require reconstruction from salient points.
• video recognition software can be used in real time to define the X, Y and Z co-ordinates of each salient point on each object.
• the attitude (vertical and horizontal angle) of each camera is entered into the processing software and the offset of any recognised salient point in the image can be resolved into a line of sight vector relative to the attitude of the camera.
• any detail is hidden from all cameras (e.g. a ball m a rugby scrum) it will equally be hidden from viewers and will not reappear until at least two cameras can spot the ball.
• the video recognition software will identify outlines of objects m a plane perpendicular to its line of sight and derive space line equations to the centre points. The intersection of space lines from the multiple cameras determines the 3d position of each salient point ( Figure 4) .
• Motion prediction software will be used to estimate the positions of hidden objects until visual contact is re- established.
• a computer at the real venue will prepare a frame transmission where a relatively small percentage of the frame is used to send the co-ordinates of all the salient points previously defined and the remainder is normal video m split screen mode.
• the data contains coded and embedded digital data.
• One suggestion would be to send a 4 tile split screen image of the real venue crowd for back projection on the virtual venue walls around the viewers.
• the computer which may for example be a Silicon Graphics (TM) workstation running a MUSE operating system
• the whole event can be reconstructed m 3D for the viewers including the images of the moving objects from their salient point co-ordinates. Live video of faces can be projected onto facial surfaces of each participant for further realism.
• the stored data can be subsequently replayed on a conventional wall screen m 3D from any viewpoint or even from the point of view of any object (including the referee, the goalkeeper, the linesman or even the ball) .
• a right eye camera 50 and a left eye camera 52 are physically coupled together to provide right and left video signals which are interleaved to provide alternate frames by firmware 54 and then transmitted by conventional broadcast techniques.
• Each frame is preceded by a code defining it as left or right.
• the code may be a data byte at the leading end of the frame data, and in analogue transmission the code may be a suitable pulse signal or frequency burst signal adjacent in time to the normal frame sync signal .
• the broadcast signal may be received and viewed by a conventional television set 56, with the signal fed in parallel to a set top box 58 which reads the left and right codes and transmits an infra-red signal to viewer spectacles 60, whereby each lens is alternated from opaque to clear in synchronism with the frame signals.

Landscapes

• Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)

Applications Claiming Priority (3)

Publications (1)

Family

ID=10858049

Family Applications (1)

Country Status (5)

Families Citing this family (3)

Family Cites Families (1)

• 1999
  • 1999-07-29 GB GBGB9917658.8A patent/GB9917658D0/en not_active Ceased
• 2000
  • 2000-07-28 WO PCT/GB2000/002922 patent/WO2001010138A1/en not_active Application Discontinuation
  • 2000-07-28 EP EP00948181A patent/EP1201089A1/de not_active Withdrawn
  • 2000-07-28 JP JP2001513905A patent/JP2003522444A/ja active Pending
  • 2000-07-28 AU AU61744/00A patent/AU6174400A/en not_active Abandoned

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events