WO2000039998A2 - Systeme et procede d'enregistrement et de diffusion d'images video en trois dimensions - Google Patents

Systeme et procede d'enregistrement et de diffusion d'images video en trois dimensions Download PDF

Info

Publication number
WO2000039998A2
WO2000039998A2 PCT/US1999/031233 US9931233W WO0039998A2 WO 2000039998 A2 WO2000039998 A2 WO 2000039998A2 US 9931233 W US9931233 W US 9931233W WO 0039998 A2 WO0039998 A2 WO 0039998A2
Authority
WO
WIPO (PCT)
Prior art keywords
field
frame
dimensional
fields
video
Prior art date
Application number
PCT/US1999/031233
Other languages
English (en)
Other versions
WO2000039998A3 (fr
Inventor
Amber Davidson
Robert Boatright
Original Assignee
Chequemate International, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Chequemate International, Inc. filed Critical Chequemate International, Inc.
Priority to AU24870/00A priority Critical patent/AU2487000A/en
Publication of WO2000039998A2 publication Critical patent/WO2000039998A2/fr
Publication of WO2000039998A3 publication Critical patent/WO2000039998A3/fr

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/194—Transmission of image signals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/106—Processing image signals
    • H04N13/161—Encoding, multiplexing or demultiplexing different image signal components
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/106—Processing image signals
    • H04N13/167—Synchronising or controlling image signals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/189—Recording image signals; Reproducing recorded image signals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30—Image reproducers
    • H04N13/332—Displays for viewing with the aid of special glasses or head-mounted displays [HMD]
    • H04N13/341—Displays for viewing with the aid of special glasses or head-mounted displays [HMD] using temporal multiplexing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30—Image reproducers
    • H04N13/398—Synchronisation thereof; Control thereof
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/597—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding specially adapted for multi-view video sequence encoding

Definitions

  • the present invention relates to systems and methods for processing, recording, broadcasting, and displaying video imagery.
  • any approach designed to produce three-dimensional video images relies on the ability to project a different video stream to each eye of the viewer.
  • the video streams contain visual clues that are inte ⁇ reted by the viewer as a three-dimensional image.
  • Many different systems have been developed to present these two video streams to different eyes of an individual. Some systems utilize twin screen displays using passive polarized or differently colored viewing lenses and glasses that are worn by the viewer in order to allow each eye to perceive a different video stream. Other approaches use field or frame multiplexing which utilizes a single display screen that quickly switches between the two video streams. These systems typically have a pair of shuttered glasses that are worn by an individual and the shutters alternately cover one eye and then the other in order to allow each eye to perceive a different video stream. Finally, some systems, such as those commonly used in virtual reality systems, use dual liquid crystal or dual CRT displays that are built into an assembly worn on the viewers head. Other technologies include projection systems and various auto stereoscopic systems that do not require the wearing of glasses.
  • Prior art systems that generate and display three-dimensional video imagery have typically taken one of two approaches.
  • the first approach has been to employ a binocular system, e.g., two lenses or two cameras to produce two channels of visual information.
  • the spacial offset of the two channels creates a parallax effect that mimics the effect created by an individual's eyes.
  • the key factor in producing high quality stereoscopic video that uses two cameras is the maintenance of proper alignment of the two channels of image data.
  • the alignment of the camera lenses must be maintained and the video signals generated by the cameras must maintain a proper temporal alignment as they are processed by system electronics or optics. Misalignment will be perceived as distortion to a viewer.
  • Twin screen viewing systems are known to be particularly prone to misalignment, tend to be bulky and cumbersome, and tend to be rather expensive due to the cost of multiple displays.
  • Single screen solutions which multiplex fields or frames tend to minimize the problems associated with dual display monitors, yet these systems also rely on the accuracy of alignment of the input video data.
  • the second approach taken by various systems has been an attempt to convert an input two-dimensional video signal into a form that is suitable for stereoscopic display. These systems traditionally have split the two-dimensional video signal into two separate channels of visual information and have delayed one channel of video information with respect to the other channel of video information.
  • Systems which synthesize a simulated three-dimensional scene from two-dimensional input data tend to be somewhat less expensive due to the reduced hardware requirements necessary to receive and process two separate channels of information.
  • Such systems may utilize any conventional video source rather than requiring generation of special video produced by a stereoscopic camera system.
  • the reliance on temporal shifting of portions of the data in order to create a simulated three-dimensional scene does not work well for objects that are not moving in the scene.
  • a consumer desires to view three-dimensional video, he must generally either visit a business dedicated to showing three-dimensional video or purchase a three-dimensional decoder box, glasses, and one of the handful of titles converted to three-dimensional.
  • a limited amount of media is currently available in three-dimensional. Accordingly, the opportunities for viewing three-dimensional video are likewise limited.
  • the problems of the prior art have been successfully overcome by the present invention which is directed to systems and methods for synthesizing a simulated three-dimensional video image from a two-dimensional input video signal.
  • the present invention is relatively inexpensive, produces high quality video, and has high user tolerance.
  • the systems of the present invention do not rely on temporal shifting in order to create a simulated three-dimensional scene. However, certain embodiments may use temporal shifting in combination with other processing to produce simulated three-dimensional video from a two-dimensional video source.
  • Traditional video sources such as an NTSC compatible video source is composed of a sequence of frames that are displayed sequentially to a user in order to produce a moving video image.
  • the frame rate for NTSC video is thirty frames per second.
  • Frames are displayed on a display device, such as a monitor or television, by displaying the individual horizontal scan lines of the frame on the display device.
  • televisions have been designed to display the frame by interlacing two different fields. In other words, the television first displays all the odd numbered scan lines and then interlaces the even numbered scan lines in order to display a complete frame.
  • a frame is typically broken down into an even field which contains the even numbered scan lines and an odd field which contains the odd numbered scan lines.
  • the present invention takes a two-dimensional video input signal and digitizes the signal so that it can be digitally processed.
  • the digitized frame is separated into the even field and the odd field.
  • the even field and/or the odd field are then processed through one or more transformations in order to impart characteristics to the field that, when combined with the other field, and properly displayed to a viewer will result in a simulated three-dimensional video stream.
  • the fields are then placed in a digital memory until they are needed for display. When the fields are needed for display, they are extracted from the digital memory and sent to the display device for display to the user.
  • the fields are displayed to the user in such a manner that one field is viewed by one eye and the other field is viewed by the other eye.
  • Many mechanisms may be used to achieve this, including the various prior art mechanisms previously discussed.
  • the system utilizes a pair of shuttered glasses that are synchronized with the display of the different fields so that one eye is shuttered or blocked during the display of one field and then the other eye is shuttered or blocked during the display of the other field.
  • three-dimensional video may be viewed on a conventional display device, such as a conventional television.
  • the mind when receiving signals from the eyes, will interpret the visual clues included in the video stream and will fuse the two fields into a single simulated three-dimensional image.
  • the fields may be spatially offset, temporally offset, or may be transformed with a combination of both temporal and spatial offset. Additionally, a frame may be only partially transformed, such that certain objects within a frame are transformed into three-dimensional, while other objects remain two-dimensional.
  • a system for broadcasting and displaying a three dimensional video stream created from a two dimensional video stream comprises a plurality of video frames intended to be sequentially displayed on a display device, each of said video frames comprising at least a first field and a second field.
  • the system comprises a receiving module configured to receive a frame of said two dimensional video stream and for digitizing said frame so that said frame can be further processed by the system.
  • the system may also comprise a separating module configured to separate said frame into at least a first field and a second field, each of said fields containing a portion of the video data in said frame.
  • a transforming module may also be provided.
  • the transforming module is configured to transform at least one of said first field or said second field using a selected transform that will produce a simulated three-dimensional video frame when said first field and said second field are recombined and displayed on a display device.
  • the system also preferably comprises a recombining module, a broadcasting station, and a decoding module.
  • the recombining module is configured to recombine said first field and said second field for transferring said recombined first field and second field to a display device in order to create said simulated three-dimensional video frame
  • the broadcasting station is preferably configured to broadcast the first and second fields to a plurality of viewers.
  • the decoding module is preferably configured to control said display device so that said first field is viewed by one eye of an individual viewing said display device and said second field is viewed by the other eye of the individual.
  • the recombining module is configured to recombine said first field and said second field without temporally shifting either said first field or said second field.
  • the transforming module may be configured to transform at least one of said first field or said second field with a spatial transformation or alternatively, with a temporal transformation.
  • the selected transform comprises a skew transform that skews one field in the horizontal direction relative to the other field.
  • the selected transform comprises a skew transform that skews one field in the vertical direction relative to the other field.
  • the selected transform may comprise a shift transform that shifts one field in the horizontal direction relative to the other field and may also comprise a shift transform that shifts one field in the vertical direction relative to the other field.
  • the present invention also comprises a method for broadcasting a video stream that is synthesized from two-dimensional video into three-dimensional video.
  • the method comprises the steps of receiving a two-dimensional digitized video stream comprising a plurality of video frames that are intended to be displayed sequentially on a display device, each frame comprising a plurality of fields which together contain all digital video information to be displayed for a frame; generating information adapted to transform at least one of said plurality of fields in a manner that renders the video frames to collectively appear to a viewer to be at least partially three dimensional; and broadcasting the information from a broadcasting station to be received by a viewer station.
  • the method also comprises providing a decoding device at the viewer station; receiving the information broadcast from the broadcasting station into the decoding device; transforming the video frames for transmission in three-dimensional on a television set; and displaying the video frames in three-dimensional on a television set.
  • the information may comprise a spatial transformation of the field and may also comprise a temporal transformation of the field.
  • the method may also comprise receiving and displaying a simulated three- dimensional video frame on a display device disposed at said viewer station by alternating said first field and said second field such that said first field is viewed by one eye of an individual viewing the display device and said second field is viewed by the other eye of the individual.
  • Other alternative steps may include separating said plurality of fields of said two- dimensional digital video frame into at least a first field and a second field; extracting from said video stream a single two-dimensional digital video frame for processing; and separating said plurality of fields of said single two-dimensional digital video frame into at least a first field and a second field.
  • This may be accompanied by spatially transforming at least one of said first field or said second field in order to produce a simulated three-dimensional video frame when said first field and said second field are recombined and viewed on a display device.
  • the method may comprise displaying said first field and said second field without temporally shifting either said first field or said second field in order to create said simulated three-dimensional video frame by displaying said first field and said second field on a display device within a single frame such that said first field is viewed by one eye of an individual viewing the display device and said second field is viewed by the other eye of the individual.
  • the first field and said second field each comprise a plurality of pixels arranged in a matrix having a plurality of rows and columns, and said spatial transformation step skews one field in the vertical direction relative to the other field by performing at least the steps of selecting a total skew value; selecting a starting column of pixels; and for each column after said selected starting column, shifting the column relative to the preceding column in a chosen vertical direction by a predetermined value derived from the total skew value.
  • said transformation step may comprise spatial transformation and further comprise shifting one field in the vertical direction relative to the other field.
  • the spatial transformation step in one embodiment scales one field in the vertical direction relative to the other field.
  • the method may further comprise the step of temporally shifting at least one of said first field or said second field in order to introduce a time delay relative to its original location in said two dimensional video stream.
  • the system comprises the components described above and in addition, a recording station.
  • the three-dimensional recording method in one embodiment comprises the steps of receiving a two-dimensional digitized video stream comprising a plurality of video frames that are intended to be displayed sequentially on a display device, each frame comprising a plurality of fields which together contain all digital video information to be displayed for a frame and extracting from said video stream a single two-dimensional digital video frame for processing.
  • the method also preferably comprises the steps of separating said plurality of fields of said single two-dimensional digital video frame into at least a first field and a second field and spatially transforming at least one of said first field or said second field in order to produce a simulated three-dimensional video frame when said first field and said second field are recombined and viewed on a display device.
  • the method also preferably comprises recording the plurality of fields on a suitable media and displaying said first field and said second field without temporally shifting either said first field or said second field. This is preferably conducted in order to create said simulated three-dimensional video frame by displaying said first field and said second field on a display device within a single frame such that said first field is viewed by one eye of an individual viewing the display device and said second field is viewed by the other eye of the individual.
  • the present invention provides a system and method for selective scaling wherein certain objects within a frame are scaled in time or space to appear in three dimensions while the nonmoving or slower moving objects within a frame are not transformed.
  • a further aspect of the present invention is a system and method for dynamic variance of temporal and/or spatial delay.
  • the temporal or spatial delay is dynamically varied according to what is taking place within the frames. If not much action is occurring, the temporal delay is increased. Conversely, if significant action is occurring, the temporal delay is decreased. Accordingly, for instance, during cut scenes, the transformation is scaled down or eliminated in order to make viewing easier on the eyes.
  • the field that is ahead is preferably frozen until the field that is behind catches up.
  • the spatial delay may also be increased or decreased to enhance 3-D effects according to the amount of action detected as occurring in the frame.
  • Figure 1 is a diagram illustrating the conceptual processing that occurs in one embodiment of the present invention
  • FIG. 2 illustrates the conceptual processing that takes place in another embodiment of the present invention
  • Figures 3 A through 3D illustrate various transformations that may be used to impart visual clues to the synthesized three-dimensional scene
  • Figures 4A through 4D illustrate a specific example using a scaling transformation
  • Figure 5 illustrates temporal transformation
  • FIGS 6 A through 8B illustrate the various circuitry of one embodiment of the present invention.
  • Figure 9 is a schematic block diagram illustrating the components of one embodiment of a system for recording and broadcasting three-dimensional video.
  • Figure 10 is a schematic block diagram illustrating the components of one embodiment of a system for selective scaling of three-dimensional video.
  • Figure 11 is a schematic block diagram illustrating the general components of one embodiment of a system and method for dynamic variance of temporal and/or spatial delay.
  • Figure 12 is one embodiment of a state diagram of a system and method of dynamic variance of temporal and/or spatial delay.
  • Figure 13 is one embodiment of a timing diagram of the system and method of dynamic variance of temporal and/or spatial delay.
  • Figure 14 is a schematic block diagram illustrating one embodiment of the components of a histogram circuit of the system and method of Figure 11.
  • One embodiment of the present invention is directed to systems and methods for synthesizing a three-dimensional video stream from a two-dimensional video source.
  • the video source may be any source of video such as a television signal, the signal from a VCR, DVD, video camera, cable television, satellite TV, or any other source of video. Since the present invention synthesizes a three-dimensional video stream from a two-dimensional video stream no special video input source is required. However, if a video source produces two video channels, each adapted to be viewed by an eye of a user, then the present invention may also be used with appropriate modification. From the discussion below, those skilled in the art will quickly recognize the modifications that should be made.
  • a video signal is comprised of a plurality of frames that are intended to be displayed in a sequential fashion to the user or viewer of a display device in order to provide a moving scene for the viewer.
  • Each frame is analogous to the frame on a movie film in that it is intended to be displayed in its entirety before the next frame is displayed.
  • Traditional display devices such as television sets or monitors, may display these video frames in a variety of ways. Due to limitations imposed by early hardware, televisions display a frame in an interlaced manner. This means that first one sequence of lines is scanned along the monitor and the then another sequence of lines is scanned along the monitor. In this case, a television will scan the odd numbered lines first and then return and scan the even numbered lines.
  • the persistence of the phosphor on the television screen allows the entire frame to be displayed in such a manner that the human eye perceives the entire frame displayed at once even though all lines are not displayed at once.
  • the two different portions of the frame that are displayed in this interlaced manner are generally referred to as fields.
  • the even field contains the even numbered scan lines
  • the odd field contains the odd numbered scan lines. Due to hardware advances, many computer monitors and some television sets are capable of displaying images in a non-interlaced manner where the lines are scanned in order.
  • the even field and odd field are still displayed, only in a progressive manner.
  • the present invention is applicable to either an interlaced scanning or a progressive scanning display. The only difference is the order in which information is displayed.
  • Standard NTSC video has a frame rate of thirty frames per second.
  • the field rate is thus sixty fields per second since each frame has two fields.
  • Other video sources use different frame rates. This, however, is not critical to the invention and the general principles presented herein will work with any video source.
  • an input video stream shown generally as 20 is comprised of a plurality of frames 22 labeled FI through F8.
  • frame 24 is extracted for processing.
  • frame 24 is comprised of a plurality of scan lines.
  • the even scan lines of frame 24 are labeled 26 and the odd scan lines of frame 24 are labeled 28.
  • This is done simply for notational pu ⁇ oses and to illustrate that a frame, such as frame 24, may be divided into a plurality of fields.
  • two fields are illustrated in Figure 1, comprising even scan lines 26 and odd scan lines 28, other delineations may be made. For example, it may be possible to divide the frame into more than two fields.
  • the frame is digitized by encoder 30.
  • Encoder 30 samples the video data of frame 24 and converts it from analog format to a digital format. Encoder 30 may also perform other processing functions relating to color correction/translation, gain adjustments, and so forth. It is necessary that encoder 30 digitize frame 24 with a sufficient number of bits per sample in order to avoid introducing unacceptable distortion into the video signal. In addition, it may be desirable to sample various aspects of the video signal separately. In NTSC video, it may be desirable to sample the luminescence and chrominance of the signal separately. Finally, the sample rate of encoder 30 must be sufficient to avoid introducing aliasing artifacts into the signal.
  • a 13.5 MHZ sample rate using sixteen bits to represent the signal has been found to be sufficient for standard NTSC video.
  • Other video sources may require different sample rates and sample sizes.
  • the digitized frame is illustrated as 32 Digitized frame 32 is processed by modification processing component 34.
  • Modification processing component 34 performs various transformations and other processing on digitized frame 32 in order to introduce visual clues into the frame that, when displayed to a viewer, will cause the frame to be inte ⁇ reted as a three-dimensional image.
  • a wide variety of processing may be utilized in modification processing component 34 to introduce appropriate visual clues.
  • Various transformations and other processing are discussed below. In general, however, modification processing component 34 will prepare the frame to be displayed to a user so that the frame is inte ⁇ reted as a three-dimensional object.
  • the transformations and other processing performed by modification processing component 34 often entail separating frame 32 into two or more components and transforming one component relative to the other.
  • the resultant modified frame is illustrated in Figure 1 as 36.
  • controller 38 stores modified frame 36 in memory 40 until it is needed.
  • modified frame 36 is extracted and sent to the appropriate display device to be displayed. This may require controller 38, or another component, to control the display device or other systems so that the information is displayed appropriately to the viewer. The exact process of extracting the modified frame and displaying it on a display device will be wholly dependent upon the type of display device used.
  • one display system previously described separates the frame into two fields that are multiplexed on a single display device.
  • a pair of shuttered glasses, or other shuttering device is then used so that one field is viewed by one eye while the other eye is covered and then the other field is viewed while the shutter switches. In this manner, one eye is used to view one field and the other eye is used to view the other field.
  • the brain will take the visual clues introduced by modification processing component 34 and fuse the two fields into a single image that is inte ⁇ reted in a three-dimensional manner. Other mechanisms may also be utilized.
  • controller 38 is illustrated as controlling a shuttering device 42 in order to allow images multiplexed on monitor 44 to be viewed appropriately.
  • decoder 46 converts modified frame 36 from a digital form to an analog form appropriate for display on monitor 44. Decoder 46 may also generate various control signals necessary to control monitor 44 in conjunction with shuttering device 42 so that the appropriate eye views the appropriate portion of frame 36. Decoder 46 may also perform any other functions necessary to ensure proper display of frame 36 such as retrieving the data to be displayed in the appropriate order.
  • FIG. 2 a more detailed explanation of one embodiment of the present invention is presented.
  • the embodiment of Figure 2 has many elements in common with the embodiment illustrated in Figure 1. However, a more detailed explanation of certain processing that is performed to modify the frame from two-dimensional to three-dimensional is illustrated.
  • a video frame such as frame 48
  • Encoder 50 represents an example of means for receiving a frame from a two-dimensional video stream and for digitizing the frame so that the frame can be processed. Encoder 50, therefore, digitizes frame 48 among other things.
  • the digitized frame is illustrated in Figure 2 as digitized frame 52. Encoder 50 may also perform other functions as previously described in conjunction with the encoder of Figure 1. Digitized frame 52 is split by splitter 54 into odd field 56 and even field 58.
  • Splitter 54 represents an example of means for separating a frame into a plurality of fields. Odd field 56 and even field 58 are simply representative of the ability to split a frame, such as digitized frame 52, into multiple fields. When interlaced display devices are utilized, it makes sense to split a frame into the even and odd fields that will be displayed on the device. In progressively scanned display devices, even and odd fields may be used, or other criteria may be used to split a frame into multiple fields. For example, at one time it was proposed that an advanced TV standard may use vertical scanning rather than the traditional horizontal scanning. In such a display device, the criteria may be based on a vertical separation rather than the horizontal separation as illustrated in Figure 2.
  • splitter 54 separate frame 52 into at least two fields that will be processed separately.
  • Odd field 56 and even field 58 are processed by modification processing components 60 and 62, respectively.
  • Modification processing component 60 and 62 represent the conceptual processing that occurs to each of the fields separately. In actuality, the fields may be processed by the same component.
  • Modification processing component 60 and 62 represent but one example of means for transforming at least one field using a selected transform. Such a means may be implemented using various types of technologies such as a processor which digitally processes the information or discrete hardware which transforms the information in the field. Examples of one implementation are presented below.
  • modified odd field 64 and modified even field 66 represent the fields that are transformed by modification processing components 60 and 62, respectively. Note that although Figure 2 illustrates modified field 64 and 66, in various embodiments one, the other, or both fields may be modified. The fields may be transformed in any manner that is desirable to introduce appropriate visual clues into the field, as previously explained.
  • transforms that have been found useful to introduce visual clues in order to convert a two-dimensional video stream into a three-dimensional video stream are presented and discussed below.
  • such transforms involve shifting, scaling, or otherwise modifying the information contained in one or both fields.
  • the transforms performed by modification processing components 60 and 62 may be performed either in the horizontal direction, the vertical direction, or both.
  • Modified fields 64 and 66 are then stored by controller 68 in memory 70 until they are needed for display. Once they are needed for display, controller 68 will extract the information in the desired order and transfer the information to decoder 72. If the display requires an interlaced display of one field and then the other, controller 68 will transfer one field and then the other field for appropriate display. If, however, the display is progressively scanned, then controller 68 may supply the information in a different order. Thus, controller 68 represents an example of means for recombining fields and for transferring the recombined fields to a display device. In the alternative, certain of this functionality may be included in decoder 72.
  • Decoder 72 is responsible for taking the information and converting it from a digital form to an analog form in order to allow display of the information. Decoder 72 may also be responsible for generating appropriate control signals that controls the display. In the alternative, controller 68 may also supply certain control signals in order to allow proper display and inte ⁇ retation of the information. As yet another example, a separate device, such as a processor or other device, may be responsible for generating control signals that control the display device so that the information is properly displayed. From the standpoint of the invention, all that is required is that the information be converted from a digital format to a format suitable for use with the display device. Currently, in most cases this will be an analog format, although other display devices may prefer to receive information in a digital format.
  • the display device is then properly controlled so that the information is presented to the viewer in an appropriate fashion so that the scene is inte ⁇ reted as three-dimensional.
  • This may include, for example, multiplexing one field and then the other on the display device while, simultaneously, operating a shuttering device which allows one eye to view one field and the other eye to view the other field.
  • any of the display devices previously discussed may also be used with appropriate control circuitry in order to allow presentation to an individual. In general, however, all these display systems are premised on the fact that one eye views a certain portion of the information and another eye views a different portion of the information. How this is accomplished is simply a matter of choice, given the particular implementation and use of the present invention.
  • a skew transform is presented. This transform skews the data in the horizontal or vertical direction.
  • a field that is to be transformed is illustrated generally as 74. This field has already been digitized and may be represented by a matrix of data points. In Figure 3 this matrix is five columns across by three rows down.
  • the transformations used in the present invention will shift or otherwise modify the data of the field matrix.
  • Typical field matrices are hundreds of columns by hundreds of rows. For example, in NTSC video an even or odd field may contain between eight and nine hundred columns and two to three hundred rows.
  • the skew transform picks a starting row or column and then shifts each succeeding row or column by an amount relative to the column or row that precedes it.
  • each row is shifted by one data point relative to the row above it.
  • the transformed field illustrated generally as 76
  • the transformed field has row 78 being unshifted, row 80 being shifted by one data point, and row 82 being shifted by two data points.
  • the data points of original matrix are thus bounded by dashed lines 84 and takes on a skew shape.
  • the total shift from the beginning row to the ending row is a measure of the amount of skew added to the frame.
  • each row or column in the field matrix is shifted by a set amount.
  • the unshifted field matrix is illustrated as 90, while the shifted field matrix is illustrated as 92.
  • this again places certain data points outside the boundaries of the field matrix.
  • the data points may be wrapped to the beginning of the row and placed in the holes opened up, or the holes that opened up may be filled with a different value and the data points that fall beyond the boundaries of the field matrix may simply be ignored.
  • various schemes may be used to fill the holes such as filling with a fixed data point or using a myriad of inte ⁇ olation schemes.
  • Figures 3C and 3D illustrate various scaling transformations.
  • Figure 3C illustrates a scaling transformation that shrinks the number of data points in the field matrix while
  • Figure 3D illustrates a scaling transformation that increases the number of data points. This would correspond to making something smaller and larger respectively.
  • the unsealed matrix is illustrated as 96 while the scaled field matrix is illustrated by 98.
  • appropriate data points are simply dropped and the remainder of the data points are shifted to eliminate any open space for data points that were dropped.
  • values must be placed in the holes that are opened by the reduced number of data points. Again, such values may be from a fixed value or may be derived through some inte ⁇ olation or other calculation. In one embodiment, the holes are simply filled with black data points.
  • Figure 3D represents a scaling that increases the number of data points in a field matrix.
  • the unsealed field matrix is illustrated by 100 and the scaled field matrix is illustrated by 102.
  • the "holes" open up in the middle of the data points.
  • it is typically adequate to inte ⁇ olate between surrounding data values to arrive at a particular value to put in a particular place.
  • any data points that fall outside the size of the field matrix are simply ignored. This means that the only values that must be inte ⁇ olated and filled are those that lie within the boundaries of the field matrix.
  • transformations may also be utilized. For example, transformations that skew a field matrix from the center outward in two directions may be useful.
  • it may also be possible to transform the values of the data points during the transformation process. In other words, it may be possible to adjust the brightness or other characteristic of a data point during the transformation.
  • FIG. 4A through 4D a specific example is presented in order to illustrate another aspect of the various transformations. It is important to note that when a field is shifted or otherwise transformed, it is possible to pick an alignment point between the transformed field and the other field. For example, it may be desirable to align the fields at the center and then allow the skewing, shifting, scaling, or other transforms to grow outward from the alignment point. In other words, when fields are transformed it is generally necessary to pick an alignment point and then shift the two fields in order to align them to the alignment point. This will determine how the values are then used to fill in the holes that are opened up. As a simple example, consider a skew transform which begins not at the first row as illustrated in Figure 3 A but at the center row.
  • the rows above the center row may then be shifted one direction and the rows below the center row may then be shifted the other direction.
  • a skew transform would be different than a skewed transform which began at the top row and then proceeded downward or a skewed transform that began at the bottom row and then proceeded upward.
  • an untransformed frame 104 is illustrated.
  • This frame comprises six rows, numbered 105 through 110 and seven columns.
  • the rows of the frame are first separated into an even field and an odd field.
  • Odd field 112 contains rows 105, 107, and 109 while even field 114 contains rows 106, 108 and 110.
  • Such a function may be performed, for example, by a splitter or other means for separating a frame into a plurality of fields.
  • Splitter 54 of Figure 2 is but one example.
  • Figure 4D the process of recombining the fields to create a simulated three-dimensional frame is illustrated.
  • the left-hand side of Figure 4D illustrates transformed odd field 116 that has been cropped to the appropriate size.
  • Figure 4D also illustrates even field 114.
  • the frame is reconstructed by interleaving the appropriate rows as indicated on the right-hand side of Figure 4D.
  • the reconstructed frame is illustrated generally as 122.
  • Such a reconstruction may take place, for example, when the fields are displayed on a display device. If the display device is an interlaced display, as for example a conventional television set, then the odd field may be displayed after which the even field is displayed in order to create the synthesized three-dimensional frame.
  • the synthesized three-dimensional frame is referred to as being constructed from a recombining of the various fields of the frame.
  • the reconstructed frame is then illustrated as being displayed on a display device.
  • these two steps may take place virtually simultaneously.
  • one field is displayed after which the other field is displayed.
  • the total display of the two fields represents the reconstructed frame.
  • the total frame is never physically reconstructed except in the mind of the viewer.
  • conceptually the step of creating the synthesized three-dimensional frame by recombining the fields is performed.
  • the examples presented herein should not be construed as limiting the scope of the invention, but the steps should be inte ⁇ reted broadly.
  • the embodiments presented above have processed a frame and then displayed the same frame.
  • the frame rate of the output video stream is equal to the frame rate of the input video stream. Technologies exist, however, that either increase or decrease the output frame rate relative to the input frame rate. It may be desirable to employ such technologies with the present invention.
  • the first approach is simply to send the data of a frame more often. For example, if the output frame rate is doubled, the information of a frame may simple be sent twice.
  • an input video stream comprising a plurality of frames is illustrated generally as 124.
  • a single frame is extracted for processing.
  • This frame is illustrated in Figure 5 as 126.
  • the frame is broken down into a plurality of fields, as for example field 128 and 130. As previously discussed, although two fields are illustrated, the frame may be broken into more than two fields if desired.
  • Modified field 130 is illustrated as field 136.
  • the embodiment illustrated in Figure 5 introduces a temporal shift as illustrated by delay 138.
  • Delay 138 simply holds the transformed field for a length of time and substitutes a transformed field from a previous frame.
  • a field from frame 1 may not be displayed until frames 2 or 3.
  • a delayed field, illustrated in Figure 5 as 140, is combined with field 136 to create frame 142.
  • Frame 142 is then placed in the output video stream 144 for proper display.
  • FIGS 6A through Figures 8B one embodiment of the present invention is presented. These figures represent circuit diagrams with which one of skill in the art is readily familiar. The discussion which follows, therefore, will be limited to a very high level which discusses the functionality inco ⁇ orated into some of the more important functional blocks.
  • the embodiments illustrated in Figures 6A through 8B is designed to operate with a conventional display, such as a television and shuttered glasses which operate to alternatively block one eye and the other so that one field of the frame is seen by one eye and another field of the frame is seen by the other eye.
  • a conventional display such as a television and shuttered glasses which operate to alternatively block one eye and the other so that one field of the frame is seen by one eye and another field of the frame is seen by the other eye.
  • processor 144 is responsible for overall control of the system.
  • processor 144 is responsible for receiving various user input commands, as from a remote control or other input devices in order to allow user input for various parameters of the system.
  • Such inputs may, for example, adjust various parameters in the transforms that are used to produce the synthesized three-dimensional images.
  • Processor 144 will then provide this information to the appropriate components.
  • processor 144 may help perform various transformations that are used in producing the synthesized three-dimensional scenes.
  • Figure 6A also illustrates a schematic representation of shuttered glasses 150, which is discussed in greater detail below.
  • FIG. 6B illustrates a block level connection diagram of video board 146.
  • Video board 146 will be more particularly described in conjunction with Figures 7 A through 71 below.
  • Video board 146 contains all necessary video circuitry to receive a video signal, digitize the video signal, store and receive transformed fields in memory, reconvert transformed fields back to analog signals, and provide the analog signals to the display device.
  • video board 146 may contain a logic to generate control signals that are used to drive shuttered glasses used by this embodiment to produce a synthesized three-dimensional effect when worn by a viewer.
  • Block 148 of Figure 6C contains a schematic representation of the drivers which are used to drive the shuttered glasses.
  • the shuttered glasses are illustrated schematically in Figure 6A by block 150.
  • Figures 6D - 6F contain various types of support circuitry and connectors as for example, power generation and filtering, various ground connectors, voltage converters, and so forth.
  • the support circuitry is labeled generally as 152.
  • Video board 146 comprises decoder 154 ( Figure 7 A), controller 156 ( Figure 7B), memory 158 ( Figures 7C and 7D), and encoder 162 (Figure 7E).
  • Figure 7F an alternate memory configuration is illustrated as block 160.
  • Block 164 of Figure 7G contains various input circuitry that receives video and other data from a variety of sources.
  • Block 165 of Figure 7G illustrates how the pinouts of video board 146 of Figure 6B translate into signals of Figures 7 A through 71.
  • Block 166 of Figures 7H and 71 contains output and other support circuitry.
  • Decoder 154 ( Figure 7A) is responsible for receiving the video signal and for digitizing the video signal.
  • the digitized video signal is stored in memory 158 ( Figures 7C and 7D) under the control of controller 156 ( Figure 7B).
  • Controller 156 is a highly sophisticated controller that basically allows information to be written into memory 158 while, information is being retrieved from memory 158 by encoder 162 ( Figure 7E) for display.
  • the various frames and fields of an input video received by decoder 154 may be identified from the control signals in the video data. The fields may then be separated out for processing and transformation, as previously described.
  • transformations occur in the horizontal direction, then the transformation may be applied line by line as the field is received. If, on the other hand, a transformation occurs in the vertical direction, it may be necessary to receive the entire field before transformation can occur. The exact implementation of the transformations will be dependent upon various design choices that are made for the embodiment.
  • controller 156 of Figure 7B in addition to storing and retrieving information from memory 158, controller 156 also generates the control signals which drive the shuttered glasses. This allows controller 156 to synchronize the shuttering action of the glasses with the display of information that is retrieved from memory 158 and passed to encoder 162 for display on the display device.
  • Encoder 162 ( Figure 7E) takes information retrieved from memory 158 and creates the appropriate analog signals that are then sent to the display device.
  • Alternate memory 160 (Figure 7F), which is more fully illustrated in Figures 8 A and 8B, is an alternate memory configuration using different component parts that may be used in place of memory 158.
  • Figure 8A illustrates the various memory chips used by alternate memory 160.
  • Figure 8B illustrate how the pinouts of Figure 7F translate into the signals of Figures 8 A and 8B in pinout block 161.
  • Figure 8B also illustrates filtering circuitry 163.
  • the present invention produces high-quality, synthesized, three-dimensional video. Because the present invention converts a two-dimensional video source into a synthesized three-dimensional video source, the present invention may be used with any video source.
  • the system will work, for example, with television signals, cable television signals, satellite television signals, video signals produced by laser disks, DVD devices, VCRs, video cameras, and so forth.
  • the use of two-dimensional video as an input source substantially reduces the overall cost of creating three-dimensional video since no specialized equipment must be used to generate an input video source.
  • the present invention retrieves the video source, digitizes it, splits the video frame into a plurality of fields, transforms one or more of the fields, and then reassembles the transformed fields into a synthesized, three-dimensional video stream.
  • the synthesized three-dimensional video stream may be displayed on any appropriate display device.
  • Such display devices include, but are not limited to, multiplexed systems that use a single display to multiplex two video streams and coordinate the multiplexing with a shuttering device such as a pair of shutter glasses worn by a viewer. Additional display options may be multiple display devices which allow each eye to independently view a separate display. Other single or multidisplay devices are also suitable for use with the present invention and have been previously discussed.
  • the three-dimensional conversion system and method mentioned above may also be used to record three- dimensional versions of video works for later viewing.
  • the recorded and converted work may then be viewed with the use of a relatively simple and inexpensive synchronizer which synchronizes the viewing glasses with the three-dimensional video display.
  • FIG. 9 One embodiment of a three-dimensional recording and broadcasting system 200 is shown in Figure 9.
  • the work is first obtained from a two-dimensional source 202.
  • the two-dimensional source 202 is typically as described above, and may be recorded in the NTSC format, and may be on DVD, VHS, Beta, or other video media.
  • the two-dimensional video source 202 is preferably broken into fields 204, 206.
  • the two-dimensional video source 202 is in one embodiment input as a video stream through an encoding and transforming module 210.
  • the encoding and transforming module 210 performs the steps of encoding and transforming the video stream to simulate three- dimensional video to the user in the manner described above.
  • the encoding and transforming module 210 may convert the video stream from the two-dimensional source 202 into a three-dimensional stream 212.
  • the three-dimensional stream 212 contains odd and even fields 204, 206 which are transformed spatially and/or temporally to simulate a three- dimensional frame.
  • the resulting three-dimensional video stream 212 is then recorded onto the appropriate media with a recording module 214.
  • the media may be any suitable video media, including DVD, VHS, and Beta.
  • the three-dimensional video stream 212 is passed from the encoding and transforming module 210 into the recording module 214.
  • the recording module may be a readable DVD, a disk drive device, a VHS recorder, etc.
  • the resultant recording 215 may then be used for later broadcast or private viewing within a user's home.
  • the system 200 may be used for broadcasting three-dimensional video. Accordingly, as seen in Figure 9, a three-dimensional video stream 212 is obtained.
  • the three-dimensional video stream 212 may be obtained by conversion of a two-dimensional video source 202 as described. Alternatively, the three- dimensional video stream 212 may be produced by other transformation techniques, including spatial and temporal displacement of one field relative to the other. Additionally, the video stream may be received as production three-dimensional video shot by dual camera lenses or other known three-dimensional production techniques.
  • the three-dimensional video is transmitted with three-dimensional transformation information for later assembly at the viewer station.
  • This information may be digital video fields for recombining at the viewer station or other suitable information to enable a decoder at the viewer station to assemble the video into three-dimensional video.
  • the three-dimensional video stream 212 may be recorded onto storage media such as the recording 215, or may be converted on-the-fly from a two-dimensional video source 202.
  • the two-dimensional video source 202 may be live or a recording.
  • the video stream 202 is passed through the encoding and transforming module 210 to produce three- dimensional transformation information. As discussed, in one embodiment, this information comprises a resulting three-dimensional video information stream 212.
  • the three-dimensional video information stream 212 may then be recorded and the recording supplied to a transmission station 216.
  • the three-dimensional video stream 212 is transmitted directly to the transmission station 216.
  • the transmission station 216 converts the three-dimensional transformation information into a suitable format for transmission to receiving stations 226.
  • the format is MPEG video.
  • the MPEG video is uplinked 220 to a satellite 222 at a high rate of speed, in one embodiment, 25 megabits per second.
  • the transmission may be by cable, radio magnetic frequencies, etc.
  • the transmission may also be received directly by viewing stations 230.
  • the satellite 222 transmits or broadcasts 224 the video stream 212 to a receiving station 226.
  • the receiving station 226 is a cable company.
  • a satellite dish 228 at the receiving station 226 receives the satellite transmission or broadcast 224.
  • many such local cable companies 226 preferably receive the broadcast 224 at the same time.
  • the receiving stations 226 may be located at varying locations throughout the world. If necessary, the video stream 212 may be passed between several satellites or even several ground links prior to being received by the receiving station 226.
  • the receiving station 226 decodes the video stream from the MPEG or other format in which it was transmitted. The result is a standard video stream 229 containing the three-dimensional transformation information described above.
  • the video stream 229 is then transmitted through communication channels 227 to individual users.
  • the communication channels 227 may be satellite transmission to satellite disks.
  • the communication channels 227 may also be comprise cable transmission, RF transmission, direct links, and the like.
  • the satellite 222 may broadcast 238 the video stream 212 directly to the user stations 230
  • the user stations 230 are provided with receiving stations 240 which may comprise satellite disks 242 for receiving the satellite broadcast 238.
  • the satellite disks may comprise C-band receivers or small dish receivers.
  • the video stream 212 is broadcast in a format 229 receivable by the user stations 230, depending upon the particular medium chosen.
  • the video stream 229 is eventually received by the viewing stations 230.
  • a decoding module is preferably provided for decoding the three- dimensional transformation information.
  • the decoding module may decode the video stream in a manner suitable to the format of the three-dimensional transformation information.
  • the decoding module comprises a synchronizing module 236.
  • the synchronizing module 236 is used read the transformation information in the form of a vertical synchronization signal of each of the odd and even fields 204, 206 of each frame of the video stream 229. Synchronization signals are then transmitted to viewer glasses 244 which alternately shutter the lenses 246 thereof, as described above.
  • the viewer is allowed to view the three-dimensional video 229 from the comfort of his/her home.
  • the encoding and transforming module 210 need not be supplied to each viewing station 230. Instead only the synchronizing module 236 and the viewer glasses 244 are needed at each viewing station 230.
  • the three-dimensional video stream 229 may be of several types which are seamlessly mixed and programmed for the viewer.
  • a separate three-dimensional video channel programming station is used to program up to 24 hours per day of programming most of all of which is in three- dimensional video format. As part of this programming, pre-recorded three-dimensional video may be used.
  • live video which is shot with dual camera or other three-dimensional video generation techniques may also be interspersed therein.
  • live events such as sporting events can be translated on-the-fly by the system 200 of the present invention and immediately transmitted through the above-described channels to the viewing stations 230.
  • the end viewer views each of these different types of three-dimensional video seamlessly through his/her viewing glasses 244 without ever knowing that the video is generated in different manners.
  • the three-dimensional video is in one embodiment transformed with spacial displacement to simulate the three-dimensional effect as described above. Nevertheless, the three-dimensional may also be transformed by temporal displacement to simulate the three- dimensional effect to the viewer. This temporal displacement may be modified on-the-fly according to the amount of motion in the particular scenes being depicted.
  • objects moving fast such as a race car
  • Objects not moving at may be displaced more, such as three frame displacement.
  • Objects moving slow may have an intermediate displacement, for instance, a two frame displacement. This manner of selective scaling will be discussed in greater detail below.
  • the video stream 212 may be transformed within a studio prior to broadcasting and be altered or "sweetened” through computer manipulation.
  • the sweetening may comprise different types of manipulation of the different fields 202, 204 of the three-dimensional video stream 212.
  • video may be added into the stream 212 through animation or inte ⁇ olation.
  • Colors and tones may be changed. For instance, colors which appear to pop out of the screen may be added in instances, and colors which appear to regress into the screen may be added in other instances.
  • Shadowing effects may be added, objects may be added, and the two-dimensional video stream may be passed through the encoding and transforming module 210 in part, in whole, or not at all.
  • the resultant sweetened video stream is then mastered into a recording 215 and distributed for viewing to the end stations 230. This distribution may be through video rental shops, direct cable, or through the broadcast network described above.
  • a further aspect of the present invention is a system and method for selective scaling.
  • selective scaling comprises expanding or reducing objects within a frame while leaving the background the same or scaling the background in the other direction.
  • the objects are chosen based on the pixel movement from frame to frame. By taking the difference in pixels from succeeding frames that change beyond a certain threshold, a shape can be isolated. This shape represents the movement or changes from frame to frame. Complete scene changes would be detected as a massive change and would not be scaled. Slight changes from one frame to the next would also not be scaled.
  • the system 250 comprises a video in line 252 on which digitized video is received into an incoming video sequence storage module 254.
  • the storage module 254 disassembles the video into a plurality of frames and stores those individual frames temporarily on frame buffers 256.
  • the buffers 256 are also used in the embodiments described above for effecting frame delays.
  • the differences in the individual pixels of the different frames stored in the frame buffers 256 is observed and calculated with a pixel difference sequence control.
  • the differences between frames is stored in a pixel difference buffer 262.
  • the outline of a shape of an object that is moving faster or more slowly than the remainder of objects in the frame is determined with a determine shape difference module 266.
  • the shape is scaled with a calculate scaling data module 268.
  • the shape may be scaled to pop out more from the screen, or to regress into the screen.
  • the shape is preferably transformed to have a lesser or greater temporal or spatial displacement from the other objects in the frame. This process is preferably conducted for all frames in which the shape is distinguishable as moving faster or slower than others in the frame.
  • shapes can be made to "pop out" from the screen. In so doing, other objects or background could be scaled in the opposite direction, to further exaggerate movement of the shape.
  • the calculate scaling data module 268 temporarily stores the calculated transformation data in scaling data buffers 270, until the transformation can be completed by successive circuitry.
  • a video control out module 260 coordinates, together with a master logic and timing control 264, the addition of the selectively scaled data into the frames. This is done through a scale shape logic module 272 which provides the scaling data from the buffers 270 to a scaling process 274 which in one embodiment operates similar to the embodiments discussed above, to temporally or spatially transform one field with respect to another, with the exception that the selected moving objects are transformed to a greater or lesser degree as described.
  • a scaling process 274 which in one embodiment operates similar to the embodiments discussed above, to temporally or spatially transform one field with respect to another, with the exception that the selected moving objects are transformed to a greater or lesser degree as described.
  • Only the object could be transformed, and the manner of so doing will be readily apparent from the present discussion and the transformation techniques discussed above.
  • the frames are provided to a video out buffer 276 which provides them to a video out line 278 in a sequence and with a selected timing for recording, broadcasting, and/or viewing.
  • a further aspect of the present invention is a system and method for dynamic variance of temporal and/or spatial delay.
  • the three- dimensional transformations conducted above are dynamically varied according to what is taking place within the frames of the video stream. This is preferably measured by a change of luminescence from one screen to the next.
  • the temporal or spatial transformation is increased.
  • the transformation may be reduced.
  • the transformation is preferably increased.
  • the transformation is preferably substantially scaled down or totally eliminated in order to make viewing easier on the eyes.
  • FIG 11 illustrates one embodiment of a system for effecting dynamic variance of temporal and/or spatial delay.
  • the system 300 of Figure 11 is shown comprising a histogram circuit 304, a processor 320, and a memory controller 324.
  • the processor 320 is the processor U5 of Figure 6A.
  • a series of lines 302 (eight lines in one embodiment) carries a video stream Y-in into the histogram circuit where the video stream is examined for movement.
  • the histogram circuit examines the video stream for activity and communicates with the processor 320 over lines D0-D7 306, Start* 308, RD* 310, FC* 312, Func(0...1) 314, LD* 316, and Clear 318.
  • Lines D0-D7 306 provides the output of the histogram circuit to the processor 320.
  • the remainder of the lines 308, 310, 312, 314, 316, 318 are for control pu ⁇ oses.
  • the histogram circuit 300 samples one of two fields (e.g., the even or odd field) in an area of interest, sums up the luminance values of all pixels in that field for a given number of frames, (in one example, five frames are summed) and compares the sum of luminance values for each new frame to the average value for the previous five frames.
  • a threshold value is set, and if the luminance varies by the threshold value, a signal is sent to the processor indicating that the threshold value has been exceeded. The processor then takes action to phase in or phase out three-dimensional transformation.
  • the histogram circuit 304 merely sums each pixel in each line in an area of interest, then sums the lines in the area of interest and passes the luminance of the most recent frame to the processor 320 on lines 306.
  • the processor 320 compares to new line to an average value of the last five lines to see if the luminance of the most recent frame is within the threshold value of the average luminance for the five previous frames. Again, action is taken according to the variance of luminance in the recent frame.
  • the memory controller 324 which may correspond to controller 38 of Figure 1, controls RAM memory, which may correspond to memory 40 of Figure 1.
  • the RAM memory is provided with frames of a digitized video stream, as discussed above.
  • both transformed versions and original versions of several video frames are stored in the memory.
  • the transforms may be conducted as discussed above, and in one embodiment comprises several types of transformations, including horizontal and vertical scaling, horizontal and vertical shifting, and a temporal time delay.
  • the memory controller 324 receives instructions from the processor 320 regarding which frames of the video stream are to be passed along and viewed by a viewer.
  • the transformed frames of one of the two fields are selected for viewing.
  • the three-dimensional transformation is phased out by selecting the original, untransformed frames to be viewed. This phasing out may be sudden, as when a cut scene occurs. It may also be gradual, by signaling the transformation circuitry to transform the selected field to a lesser degree. It may also be scaled by the particular change in luminance.
  • FIG 12 shown therein is a state chart 330 illustrating the various states of a histogram circuit of one embodiment of the present invention. Timing for the histogram circuit is illustrated in Figure 13, while one embodiment of a histogram circuit 360 suitable for use with the present invention is illustrated in Figure 14.
  • pixel data of a frame of a two-dimensional video stream is sampled during positive clock pulses on a line Y- in(0...7) 362 through a D-type flip-flop 376.
  • the lines 362 carry a luminance value for each pixel.
  • the RAM memory 388 (which preferably comprises SDRAM) is provided with 256 storage locations, one for each possible luminance value. For each pixel within the current line, the luminance of that pixel is sampled and the particular luminance value is used to reference one of the memory locations of the RAM memory 388 according to the luminance values stored therein. Thus, for instance, if a sampled pixel had a luminance value of 155, a one is added to the 155 address location of the RAM memory 388.
  • each line of the area of interest comprises 256 pixels, and there are 256 such lines that are observed.
  • the processor transmits signals to a function decoder 416 to cause a state machine 404 to alter the state to 01 (referenced at 332 in Figure 12).
  • State 01 is the histogram accumulator state. In this state, the pixels of the current line are added to the current pixel total for the current frame. The addition is conducted with a summing node 398 and a 3-to-one input Multiplexor 420.
  • the sum is passed on a line 390 back to the RAM memory 388, and the total sum of luminescence for all previous lines in a frame is read by a plurality of output flip-flops 434 and stored therein.
  • This process continues, summing lines at a 0 0 histogram state 331 and adding the current line to the running total at a 0 1 histogram accumulator state 332 until all lines of a frame have been read and added.
  • the processor signals the function decoder 416 to cause the state control machine 404 to go to state 1 0, the data out state 333. In this state, the sum located in the output flip-flops 434 is passed to the processor 320.
  • the processor compares the newest total luminescence to the average luminescence for the previous five frames to determine if the threshold value has been exceeded. Of course, the comparison could also be done with logic circuitry, such as a comparator.
  • the processor 320 If the threshold has been exceeded, the processor 320 then signals the memory controller 324 to select normal, nontransformed frames for viewing, rather than transformed frames. The viewer thus experiences a reduction or elimination of the three-dimensional effect. Alternatively, of course, if the luminance increases by a certain value, rather than decreases, the three-dimensional transformations may be resumed or increased.
  • Figure 13 is a timing diagram illustrating the timing occurring on various clock, data, and control lines.
  • the timing diagram is broken up into histogram timing, histogram accumulate timing, and data out timing.
  • histogram timing one embodiment of timing 332 for a elk line 364 is shown, together with timing 334 for a start* line 366, timing 336 for the Yin data line 362, and timing 338 on a RAM in line 386.
  • timing 340 for the elk line 364, timing for the start line 366, and timing 334 for an accumulate data operation of the output buffers 434 are shown.
  • timing 346 of a rd* line 368 is shown, as well as the timing 348 of OSEL lines 438, and the timing 350 of the DO lines 440 which are read by the processor 320 on node 458.
  • the elk line 364 is connected to a fixed frequency clock. As discussed, the timing signals on the lines 366 and 368, together with other timing signals are generated by the processor 320 in a manner that will be readily appreciated and understood by one of skill in the art.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)

Abstract

Cette invention concerne des systèmes et des procédés permettant d'obtenir par synthèse des images vidéo 3D à partir d'une source vidéo bidimensionnelle. Une trame provenant d'une source vidéo bidimensionnelle est numérisée et divisée en une pluralité de champs. Chacun de ces champs renferme une partie de l'information contenue dans la trame. On traite ensuite ces champs séparément et on les transforme pour y introduire des indices visuels qui, après assemblage dans d'autres champs, seront perçus visuellement comme une image 3D. La présente invention concerne également des systèmes et des procédés d'enregistrement et de diffusion d'images vidéo 3D après transformation par synthèse d'images vidéo en deux dimensions. De plus, cette invention porte sur un système et un procédé de mise à l'échelle ainsi que sur un système et un procédé de variation dynamique d'écart temporel et spatial.
PCT/US1999/031233 1998-12-30 1999-12-30 Systeme et procede d'enregistrement et de diffusion d'images video en trois dimensions WO2000039998A2 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
AU24870/00A AU2487000A (en) 1998-12-30 1999-12-30 System and method for recording and broadcasting three-dimensional video

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US11426498P 1998-12-30 1998-12-30
US60/114,264 1998-12-30

Publications (2)

Publication Number Publication Date
WO2000039998A2 true WO2000039998A2 (fr) 2000-07-06
WO2000039998A3 WO2000039998A3 (fr) 2000-10-26

Family

ID=22354241

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US1999/031233 WO2000039998A2 (fr) 1998-12-30 1999-12-30 Systeme et procede d'enregistrement et de diffusion d'images video en trois dimensions

Country Status (2)

Country Link
AU (1) AU2487000A (fr)
WO (1) WO2000039998A2 (fr)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2010049820A2 (fr) * 2008-10-30 2010-05-06 Sensio Technologies Inc. Procédé et système d'échelonnage de trames d'images comprimées
GB2490886A (en) * 2011-05-13 2012-11-21 Snell Ltd Method for generating a warning that a stereoscopic (3D) image sequence has been derived from a two-dimensional (2D) image sequence
US8896668B2 (en) 2010-04-05 2014-11-25 Qualcomm Incorporated Combining data from multiple image sensors
US8970672B2 (en) 2010-05-28 2015-03-03 Qualcomm Incorporated Three-dimensional image processing
US9001227B2 (en) 2010-04-05 2015-04-07 Qualcomm Incorporated Combining data from multiple image sensors

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5510832A (en) * 1993-12-01 1996-04-23 Medi-Vision Technologies, Inc. Synthesized stereoscopic imaging system and method
US5850352A (en) * 1995-03-31 1998-12-15 The Regents Of The University Of California Immersive video, including video hypermosaicing to generate from multiple video views of a scene a three-dimensional video mosaic from which diverse virtual video scene images are synthesized, including panoramic, scene interactive and stereoscopic images
US6054969A (en) * 1995-03-08 2000-04-25 U.S. Philips Corporation Three-dimensional image display system

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5510832A (en) * 1993-12-01 1996-04-23 Medi-Vision Technologies, Inc. Synthesized stereoscopic imaging system and method
US6054969A (en) * 1995-03-08 2000-04-25 U.S. Philips Corporation Three-dimensional image display system
US5850352A (en) * 1995-03-31 1998-12-15 The Regents Of The University Of California Immersive video, including video hypermosaicing to generate from multiple video views of a scene a three-dimensional video mosaic from which diverse virtual video scene images are synthesized, including panoramic, scene interactive and stereoscopic images

Cited By (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2010049820A2 (fr) * 2008-10-30 2010-05-06 Sensio Technologies Inc. Procédé et système d'échelonnage de trames d'images comprimées
US20100111195A1 (en) * 2008-10-30 2010-05-06 Sensio Technologies Inc. Method and system for scaling compressed image frames
WO2010049820A3 (fr) * 2008-10-30 2010-08-12 Sensio Technologies Inc. Procédé et système d'échelonnage de trames d'images comprimées
US8254467B2 (en) 2008-10-30 2012-08-28 Sensio Technologies Inc. Method and system for scaling compressed image frames
US8896668B2 (en) 2010-04-05 2014-11-25 Qualcomm Incorporated Combining data from multiple image sensors
US9001227B2 (en) 2010-04-05 2015-04-07 Qualcomm Incorporated Combining data from multiple image sensors
US8970672B2 (en) 2010-05-28 2015-03-03 Qualcomm Incorporated Three-dimensional image processing
GB2490886A (en) * 2011-05-13 2012-11-21 Snell Ltd Method for generating a warning that a stereoscopic (3D) image sequence has been derived from a two-dimensional (2D) image sequence
US9264688B2 (en) 2011-05-13 2016-02-16 Snell Limited Video processing method and apparatus for use with a sequence of stereoscopic images
GB2490886B (en) * 2011-05-13 2017-07-05 Snell Advanced Media Ltd Video processing method and apparatus for use with a sequence of stereoscopic images
US10154240B2 (en) 2011-05-13 2018-12-11 Snell Advanced Media Limited Video processing method and apparatus for use with a sequence of stereoscopic images
US10728511B2 (en) 2011-05-13 2020-07-28 Grass Valley Limited Video processing method and apparatus for use with a sequence of stereoscopic images

Also Published As

Publication number Publication date
WO2000039998A3 (fr) 2000-10-26
AU2487000A (en) 2000-07-31

Similar Documents

Publication Publication Date Title
US11012680B2 (en) Process and system for encoding and playback of stereoscopic video sequences
US5416510A (en) Camera controller for stereoscopic video system
US5193000A (en) Multiplexing technique for stereoscopic video system
WO1998029860A1 (fr) Systeme et procede permettant de synthetiser une video tridimensionnelle a partir d'une source video bidimensionnelle
WO2000039998A2 (fr) Systeme et procede d'enregistrement et de diffusion d'images video en trois dimensions
MXPA99006050A (en) System and method for synthesizing three-dimensional video from a two-dimensional video source
JPH08307905A (ja) 記録装置
JPH08280043A (ja) 立体テレビジョン信号の再生方法およびその装置

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A2

Designated state(s): AE AL AM AT AU AZ BA BB BG BR BY CA CH CN CR CU CZ DE DK DM EE ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MD MG MK MN MW MX NO NZ PL PT RO RU SD SE SG SI SK SL TJ TM TR TT TZ UA UG US UZ VN YU ZA ZW

AL Designated countries for regional patents

Kind code of ref document: A2

Designated state(s): GH GM KE LS MW SD SL SZ TZ UG ZW AM AZ BY KG KZ MD RU TJ TM AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE BF BJ CF CG CI CM GA GN GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
AK Designated states

Kind code of ref document: A3

Designated state(s): AE AL AM AT AU AZ BA BB BG BR BY CA CH CN CR CU CZ DE DK DM EE ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MD MG MK MN MW MX NO NZ PL PT RO RU SD SE SG SI SK SL TJ TM TR TT TZ UA UG US UZ VN YU ZA ZW

AL Designated countries for regional patents

Kind code of ref document: A3

Designated state(s): GH GM KE LS MW SD SL SZ TZ UG ZW AM AZ BY KG KZ MD RU TJ TM AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE BF BJ CF CG CI CM GA GN GW ML MR NE SN TD TG

DFPE Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101)
REG Reference to national code

Ref country code: DE

Ref legal event code: 8642

122 Ep: pct application non-entry in european phase