US8411931B2 - Methods and systems for converting 2D motion pictures for stereoscopic 3D exhibition - Google Patents

Methods and systems for converting 2D motion pictures for stereoscopic 3D exhibition Download PDF

Info

Publication number
US8411931B2
US8411931B2 US12/299,909 US29990907A US8411931B2 US 8411931 B2 US8411931 B2 US 8411931B2 US 29990907 A US29990907 A US 29990907A US 8411931 B2 US8411931 B2 US 8411931B2
Authority
US
United States
Prior art keywords
depth
image
motion
image data
scene
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active, expires
Application number
US12/299,909
Other languages
English (en)
Other versions
US20090116732A1 (en
Inventor
Samuel Zhou
Paul Judkins
Ping Ye
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Imax Corp
Original Assignee
Imax Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Imax Corp filed Critical Imax Corp
Priority to US12/299,909 priority Critical patent/US8411931B2/en
Assigned to IMAX CORPORATION reassignment IMAX CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: JUDKINS, PAUL, YE, PING, ZHOU, SAMUEL
Publication of US20090116732A1 publication Critical patent/US20090116732A1/en
Application granted granted Critical
Publication of US8411931B2 publication Critical patent/US8411931B2/en
Assigned to WELLS FARGO BANK, NATIONAL ASSOCIATION, AS ADMINISTRATIVE AGENT reassignment WELLS FARGO BANK, NATIONAL ASSOCIATION, AS ADMINISTRATIVE AGENT SECURITY INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: IMAX CORPORATION
Active legal-status Critical Current
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/106—Processing image signals
    • H04N13/139—Format conversion, e.g. of frame-rate or size
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators
    • H04N13/261—Image signal generators with monoscopic-to-stereoscopic image conversion

Definitions

  • the present invention relates generally to image processing and more specifically to creating a converted image sequence from an original image sequence.
  • Humans are capable of perceiving depth or distance in a three-dimensional world because they are equipped with binocular vision. Human eyes are separated horizontally by about 2.5 inches, and each eye perceives the world from a slightly different perspective. As a result, images projected onto the retinas of two eyes are slightly different, and such a difference is referred to as binocular disparity.
  • binocular disparity images projected onto the retinas of two eyes are slightly different, and such a difference is referred to as binocular disparity.
  • the brain has the ability to interpret binocular disparity as depth through a process called stereopsis. The ability of the human visual system to perceive depth from binocular disparity is called stereoscopic vision.
  • stereopsis have long been used to record three-dimensional (3D) visual information by producing two stereoscopic 3D images as perceived by human eyes.
  • the stereoscopic 3D image pair would recreate the illusion of depth in the eyes of a viewer.
  • Stereoscopic 3D images are different from volumetric images or three-dimensional computer graphical images in that they only create the illusion of depth through stereoscopic vision while the latter contain true three-dimensional information.
  • One common way of recording stereoscopic 3D images includes using a stereoscopic 3D camera equipped with a pair of horizontally separated lenses with an inter-ocular distance equal or similar to the human eye separation.
  • each camera lens records an image, which by convention are called a left-eye image, or simply a left image, and a right-eye image, or simply a right image.
  • Stereoscopic 3D images can be produced by other types of 3D image capture devices or more recently by computer graphics technology based on the same principle of stereopsis.
  • the illusion of depth is created in the brain when the left image is presented only to the viewer's left eye and the right image is presented only to the right eye.
  • Special stereoscopic 3D display devices are used to ensure each eye only sees a distinct image. Technologies used in those devices include polarizer filters, time-sequential shutter devices, wavelength notch filters, anaglyph filters and lenticular/parallax barrier devices. Despite the technology differences in those stereoscopic 3D display devices, the depth perceived by a viewer is mainly determined by binocular disparity information.
  • the perceived size of an object in stereoscopic 3D images is inversely related to the perceived depth of the object, which means that the object appears small as it moves closer to the viewer.
  • the inter-ocular distance of 3D camera also changes the perceived size of the object in resulting stereoscopic 3D images.
  • Stereoscopic 3D motion pictures are formed by a pair of stereoscopic 3D image sequences produced by stereoscopic 3D motion picture cameras or by computer graphics or a combination of both.
  • 3D is used to mean “stereoscopic 3D,” which should not be confused with the same term used in describing volumetric images or computer graphical images that contain true depth information.
  • disarity is used to mean “binocular disparity.”
  • Producing a 3D motion picture is generally a more costly and more complex process than making a regular two-dimensional (2D) motion picture.
  • a 3D motion picture camera is usually much bulkier and heavier than a regular 2D camera, and it is often more difficult to operate.
  • Special expertise in 3D cinematography is required throughout the entire production process including capturing, video effects (VFX), rendering and editing in order to produce good 3D reality. To this day, there are only a relatively small number of 3D motion picture titles available in comparison with a vast library of 2D motion pictures.
  • An alternative approach of producing 3D motion pictures is to capture images in 2D and digitally convert the resulting footage into 3D images.
  • the basic concept of this approach is that left and right images can be generated from an original 2D image, if appropriate disparity values can be assigned to every pixel of the 2D image.
  • the disparity values of an object can be directly calculated from its depth values. An object closer to the viewer produces a larger disparity value than that resulting from a distant object. The disparity approaches zero when an object moves away towards infinity.
  • correct depth information is needed for the entire image, which can either be computed in some cases, or estimated based on viewer's subjective interpretation of the scene.
  • All depth values assigned to image pixels forms an image referred to as a depth map, and the depth map is called dense if depth values are assigned for all pixels of the image.
  • dense depth maps are collected for all frames in the image sequence, and the resulting image sequence is a depth map sequence.
  • Modeling an object may require that the object first be defined from the rest of the image over every frame.
  • the most common methods for object definition are rotoscoping and matting.
  • a rotoscoping method defines an object by tracing the contour of the object in every frame.
  • a matting method includes extracting object masks based on luminance, color, motion or even sharpness resulting from lens focus. Both rotoscoping and matting methods are usually performed manually using various types of interactive software tools. Although many software tools provide keyframing and motion tracking capability to speed up the operation, object definition remains labor-intensive and time-consuming.
  • a dense depth map sequence can be computed after all objects have been defined for every frame of the image sequence.
  • the disparity values are then calculated directly from depth values and used to generate 3D images.
  • the resulting 3D images inevitably contain “holes” called occlusion regions.
  • An occlusion region is a portion of an object which is occluded by another foreground object. Pixels within an occlusion region have no disparity values because they do not have correspondence in the original 2D images. In general, occlusion regions always accompany depth discontinuity.
  • an occlusion region may be filled with corresponding information about the background object revealed in other image frames. In other cases, the missing information needs to be “faked” or “cloned” in order to fill the holes. Improper occlusion region filling may result in visible artifacts in the 3D images.
  • the size and distribution of occlusion regions in the converted 3D images are determined by the choice of camera parameters used for computing disparity from depth.
  • Key camera parameters typically include camera position, inter-ocular distance and lens focal length. Normally, the camera parameters are selected based on the desired 3D look, but minimizing occlusion regions may also be a factor in consideration.
  • the final 3D images are computed with a selected set of camera parameters and with all occlusion regions filled properly.
  • a full feature motion picture may contain numerous image sequences called scenes and each scene may have up to hundreds of image frames.
  • Methods, processes and systems according to embodiments of the present invention relate to converting conventional 2D motion pictures for 3D cinematic exhibition, known as the IMAX® 2D-to-3D Conversion technology, or otherwise known as the DMR 3D technology. Certain embodiments may be used to convert a 2D motion picture to 3D to meet a day-and-date cinematic release schedule. Some embodiments may be used to convert any 2D image sequences to 3D for any other display applications. For the purpose of computing stereoscopic 3D views of a 2D image sequence available in a form of 2D image data sequence, a set of image elements called image data cues can be collected from the 2D image data sequence. The collected image data cues together with other essential computing information are stored as processing information.
  • a process architectural model provides a systematic process for collecting an adequate set of processing information in order to produce desirable 3D visual quality.
  • the disclosed methods, processes and system are scalable to schedule changes and adaptable to frequent version changes to the 2D image data sequence.
  • the disclosed process architectural model is equally applicable to other motion picture digital re-mastering applications including re-mastering a motion picture to a different frame rate or enhancing the image quality of a motion picture.
  • the process architectural model may include a core conversion block step.
  • a first step in some embodiments of the core conversion block is to collect a set of process information for the intended conversion.
  • a typical set of processing information may include image data cues, editorial information, computing instructions, rendering parameters, as well as other image elements.
  • Some types of processing information are collected at the scene level, while other types are collected at the object level.
  • Various types of image data cues at the object level are collected and a layered structure can be used to facilitate processing objects that have distinct characteristics. Within each layer, image data cues can be collected using a multi-mode computing structure in which various methods are grouped into multiple modes based on the level of automation and versatility.
  • an operation mode analyzer can select the most appropriate methods for processing different types of objects.
  • a similar multi-mode computing structure also can be used at the scene finishing stage.
  • the scene finishing stage may be the last processing stage of the core conversion block.
  • the configuration of the core conversion block may change according to the types of processing information required for the applications.
  • Processing information collected at the core conversion block can be modified and updated until desirable 3D visual quality is achieved.
  • the final set of processing information can be stored in render data records that are sufficient for producing the converted 3D image data sequences.
  • the render data records can be updated following the changes to the 2D image data sequence.
  • the latest version of render data records can be retrieved for computing the final 3D image data sequences in an automated mode or in other modes based on the final version of the 2D image data sequence.
  • Implementation of an architectural model, including the core conversion block may occur in a system.
  • Systems according to various embodiments of the present invention are applicable to other motion picture digital re-mastering applications such as motion picture enhancement or frame rate conversion.
  • FIG. 1 illustrates a flow diagram of a process architectural model for motion picture re-mastering process according to one embodiment of the present invention.
  • FIG. 2 illustrates flow diagram of a change detection and analysis block according to one embodiment of the present invention.
  • FIG. 3 illustrates a motion picture 2D-to-3D conversion system according to one embodiment of the present invention.
  • FIG. 4 illustrates a flow diagram of a core conversion block for motion picture 2D-to-3D conversion according to one embodiment of the present invention.
  • FIG. 5A illustrates an example scene layout including a source 2D image scene according to one embodiment of the present invention.
  • FIG. 5B is an example scene layout including an object layout according to one embodiment of the present invention.
  • FIG. 5C is an example scene layout including a geometry layout according to one embodiment of the present invention.
  • FIG. 6 illustrates a flow diagram of a scene automation analyzer decision-making process according to one embodiment of the present invention.
  • FIG. 7A illustrates a first transparent object according to one embodiment of the present invention.
  • FIG. 7B illustrates a second transparent object according to one embodiment of the present invention.
  • FIG. 8 illustrates a layered reconstruction of missing portion of objects according to one embodiment of the present invention.
  • FIG. 9 illustrates a flow diagram of an image data cues collection process in a single layer of a layer conversion stage according to one embodiment of the present invention.
  • FIG. 10 illustrates a flow diagram of an object mask generation module according to one embodiment of the present invention.
  • FIG. 11A illustrates an example of a source 2D image sequence before assigning a object masks according to one embodiment of the present invention.
  • FIG. 11B illustrates an example of the image of FIG. 11A with assigned and labeled object masks.
  • FIG. 12 illustrates a flow diagram of an object depth modeling module according to one embodiment of the present invention.
  • FIG. 13A illustrates a reconstructed right-eye image with unfilled occlusion regions according to one embodiment of the present invention.
  • FIG. 13B illustrates a finished right-eye image of FIG. 13A with occlusion regions filled according to one embodiment of the present invention.
  • FIG. 14 illustrates a flow diagram of a scene finishing module according to one embodiment of the present invention.
  • FIG. 15 illustrates a flow diagraph of a depth by scaling method according to one embodiment of the present invention.
  • Embodiments of the present invention provide methods and systems for converting 2D motion pictures for stereoscopic 3D exhibitions. Some embodiments may be used to convert a 2D motion picture into a 3D motion picture to be released on the same release date as that for the original 2D motion picture, which is known as a day-and-date release.
  • a production process of a motion picture includes frequent changes to the contents of the motion picture and the contents are not locked until very close to the release date.
  • a color correction process is applied to the images to achieve the final look intended by the filmmakers.
  • the color correction is done photo-chemically through a process known as “color timing”, and more recently it is performed by digital means that provide much finer controls.
  • the finalized digital version of the motion picture is referred to as a digital intermediate (DI).
  • DI digital intermediate
  • all the conversion processing must be performed on the DI within such a time window. Therefore, the conversion process must be scalable and adaptable to a changing production schedule.
  • PI processing information
  • a typical set of PI includes image data cues, editorial information, computing instructions, computing parameters, and other image elements such as VFX elements.
  • Image data cues are image elements and other data that are extracted from the 2D image sequence for the purpose of computing 3D images.
  • Most types of IDC are those directly related to the computing of the missing depth dimension, including shape, color, geometry, depth, occlusion and motion. In general, collecting more accurate and more complete IDC leads to better 3D image quality, but it also consumes more time and cost. The accuracy and completeness of IDC can be described by the level of details.
  • One method is to collect IDC only to a level of details adequate for producing acceptable 3D image quality.
  • the level of details of IDC has an impact on the other types of PI, including scene editorial information and scene geometry layout. Embodiments of the method of determining an adequate level of details is described in the next section as the architectural model.
  • FIG. 1 An embodiment of an architectural model is depicted in FIG. 1 .
  • the architectural model consists of multiple functional blocks including a core conversion block 118 , a verification block 106 , a change detection and analysis block 108 , a render data records block 110 , and a final render block 112 .
  • the architectural model can be applicable to a wide range of motion picture and image sequence conversion processes including 2D-to-3D conversion, frame rate conversion, or image enhancement, or any other conversion processes that contributes to image conversion that facilitates further image enhancement within a projector to produce enhanced images. Embodiments of the 2D-to-3D conversion is discussed below.
  • the core conversion block 118 includes a PI collection block 102 , where PI is collected, and a pre-render block 104 , where initial conversion results are rendered.
  • the PI collection block 102 collects an initial set of from 2D input image data sequences 100 or otherwise known as source image data sequences. As discussed below, various methods can be used to collect different types of IDC from different types of image contents. As shown in FIG. 1 , the initial set of IDC is verified to determine whether the level of details is adequate.
  • the verification process includes the pre-render block 104 and the verification block 106 .
  • the pre-render block 104 computes 3D images from source 2D image data sequences based on the initial set of IDC.
  • the rendered 3D images are visually examined at the verification block 106 by trained personnel using a stereoscopic 3D display system that simulates the viewing environment of a typical 3D cinematic theater. If the quality of the 3D images is acceptable, the initial set of IDC is considered adequate, and then saved by the render data records block 110 together with other types of PI collected including the computing instructions and parameters used by the pretender block 104 .
  • the pre-render block 104 computes new 3D images based on the latest set of IDC.
  • the resulting 3D images are once again visually examined by the verification block 106 . If the results are satisfactory, the latest set of IDC is considered adequate, and is saved as a subset of the latest version of render data records (RDR).
  • the RDR contains IDC and other types of PI, especially the computing instructions and parameters that are sufficient for producing the latest 3D image results.
  • the latest version of RDR replaces the previous RDR version stored by the render data records block 110 . If the resulting 3D images are still deemed unacceptable, the level of details will be raised and more IDC can be collected by the collection module 102 . This process is repeated until the resulting 3D image quality is acceptable by the verification personnel. Once the 3D results are accepted, the latest set of IDC collected is considered adequate, and it is saved together with the latest versions of other types of PI including the latest computing instructions and parameters as the current version of RDR. The saved RDR can be retrieved from the render data records block 110 and used whenever necessary to repeat all the computations required to produce the latest 3D image results.
  • the current version of RDR also contains the latest editorial decision list associated with the latest version of source image data sequences.
  • the current version of RDR may also contain some intermediate results already computed by the pre-render block 104 . Although those intermediate results can be re-computed from the saved RDR, they may be included as part of the current version of RDR to reduce future computations in the final render block 112 since re-computation may require time and increased computing costs.
  • the input image data sequences 100 in FIG. 1 includes original 2D image sequence or the 2D motion picture DI, or otherwise known as source image data sequences.
  • the source image data sequences may be converted to a specified file format or a set of different file formats by the data input block 116 . If there are no changes to the source image data sequences, the converted image data sequences 120 computed by the final render block 112 are final converted 3D image data. For example, all source image data sequences are final in a conversion of a 2D library motion picture to 3D.
  • the architectural model of FIG. 1 includes a change detection and analysis block 108 , which detects changes in the latest version of the source image data sequences by comparing it with the previous version saved by block 108 . If changes are detected, the change detection and analysis block 108 determines how to update the current version of RDR. An embodiment of such a determination is illustrated in FIG. 2 .
  • the latest version of source image data sequences (Version x+1) 200 is compared in step 204 with the previous version of source image data sequences (Version x) 202 .
  • step 210 If changes detected in step 206 , impact from detected changes are analyzed in step 208 and then a decision is made in step 210 on whether the current version of RDR needs to be updated. If the only changes are the results from color-timing, the RDR may not need to be updated.
  • the computing instructions and render parameters may be color independent.
  • Such processing may allow conversion of final color-timed source image data sequences to 3D in a fully automated mode by the final render block 112 in FIG. 1 using the latest version of RDR saved by the render data records block 110 .
  • Such automated rendering may allow the final computing of the converted image data sequences 120 to be performed relatively quickly.
  • the change detection and analysis block 108 can also decide in step 212 if the changes require collecting new PI from the new source image data sequences. In some embodiments, collecting new PI may be avoided. For example, some editorial changes or rendering parameter changes can be handled by directly updating the current version of RDR in step 216 . However, if image contents are changed, collecting new PI may be required in step 214 .
  • Embodiments of the architectural model can allow the conversion process to be adaptable to version changes in the source image data sequences until the image data is finalized.
  • FIG. 3 shows one embodiment of a system implementation of a 2D-to-3D conversion process according to various embodiments of the present invention.
  • the system implementation described in FIG. 3 can also be applied to any other motion picture and image conversion processes such as frame rate conversion, image enhancement, and any other image enhancement conversions or a conversion which facilitates further image enhancement within a projector to produce the enhanced images.
  • the functional blocks of the architectural model in FIG. 1 can be implemented into two subsystems: the front-end subsystem 302 and the back-end subsystem 304 . Those two subsystems are connected through a network connection 322 .
  • the front-end subsystem 302 can be designed for collecting, verifying and updating PI required for producing 3D images. It can provide functions of the PI collection block 102 , the verification block 106 and the change detection and analysis block 108 in FIG. 1 that may require human interactions.
  • the front-end subsystem 302 may include the pre-render block 104 , for example when the front-end subsystem 302 is implemented in a separate location away from the back-end subsystem 304 .
  • the pre-render block 104 shares the same hardware and software with the final render block 112 , implemented in the back-end subsystem 304 .
  • the back-end subsystem 304 can be designed for automated rendering of the final converted image data sequences of a motion picture with high efficiency. It can implement the full functions of the final render block 112 and the render data records block 110 . The functions of the data input block 116 and the data output block 114 also can be implemented at the back-end subsystem 304 .
  • One component of the back-end system 304 is an intelligent controller server 312 , which manages the motion picture conversion process including that of the front-end subsystem 302 . As discussed previously, the back-end subsystem 304 may also provide the function of the pre-render block 104 when it shares the same hardware and software with the final render block 112 .
  • One function of the front-end subsystem 302 is to provide both user-interactive capability and automated rendering capability for collecting an adequate set of PI, especially the IDC, from source image data sequences. As disclosed previously, this function is provided through a recursive process of IDC collection and verification. Collecting various types of IDC required for the conversion is a labor-intensive and time-consuming process because human interactions and human decisions are often needed in the collection process. In some embodiments, an increasing portion of those labor-intensive computations can be automated or semi-automated. Such automated and semi-automated computations may implemented in the pre-render 324 .
  • the front-end subsystem 302 provides an open-platform that supports those commercial software products.
  • the supported software products generally provide render scripting capability.
  • the render scripting capability ensures that all processing steps selected by a user using interactive means can be recorded as render scripts and can be repeated automatically at a render by executing the recorded render scripts.
  • the render scripting capability of a supported software product can execute the render scripts on the parallel and distributed computing platform of the pre-render 324 as well as the automated final render 316 .
  • the PI collection block 102 can also deploy automated and semi-automated custom software applications to improve process efficiency. Those customer software applications are usually designed for efficiently collecting a certain types of IDC under special conditions. Custom software applications deployed in the system of FIG. 3 can support a render scripting capability similar to the commercial software. Some semi-automated custom software applications provide their own user graphical interface (GUI) and some other custom software applications are implemented as a plug-in application of a commercial software product. The automated and semi-automated functions of the custom software applications are executable by the pretender 324 .
  • GUI user graphical interface
  • the front-end subsystem 302 deploys multiple computer workstations 310 ( 1 )-( n ) that provide GUI capability to support user interactivity of both commercial and custom software applications.
  • the workstations 310 ( 1 )-( n ) may also provide render capability to supplement the pre-render 324 .
  • the pre-ender 324 may include at least one computing device.
  • One configuration of the pre-render 324 is a computer cluster equipped with multiple processors that provide parallel and distributed computing capability, as shown in FIG. 3 .
  • the processes of collecting various types of IDC are saved as render scripts, which can be distributed to the multiple processors of the pre-render for fast computing.
  • Each pre-render processor computes a portion of the task independently and in parallel with other processors.
  • a computer cluster is one, nonexclusive way to provide required computing power for IDC collection tasks and for eventual rendering of 3D images based on the collected IDC and PI.
  • the computing capacity of a computer cluster is scalable to meet the changing needs for computing power.
  • the pre-render 324 is controlled and managed by the intelligent controller 312 of the back-end subsystem 304 .
  • the specific computing instructions and parameters executed by the pre-render 324 are saved as RDR in render data records block 110 whose functions are implemented in the central data storage 314 .
  • Some embodiments of the saved RDR are in a form of text-based render scripts files that are supported by both commercial and custom software applications.
  • the saved RDR can be repeatedly executed on the same or a different computer cluster to produce the same results.
  • the front-end subsystem 302 supports at least one 3D projection system 306 for the visual verification of the 3D images rendered by the pre-render 324 .
  • the rendered left-eye and right-eye image data can be streamed by a verification server 308 to be played at a right frame rate by a 3D projector onto a 3D screen.
  • Multiple 3D image projection systems 306 may be required, with each having a separate verification server 308 .
  • each 3D image projection system may provide a viewing experience that is similar, or at least scalable, to the viewing experience in a destined 3D cinematic theater.
  • the image display area of the 3D projection system can be sufficiently large to provide viewers with a field of view similar to what is provided in the 3D cinematic theater.
  • Such a viewing experience is typically delivered by a single 3D image projector that projects a relatively large image onto a screen, or it can be delivered by a pair of image projectors with one projector projecting left-eye images and another projector projecting right-eye images onto a screen.
  • polarizer glasses are used for 3D viewing, the screen preserves polarization.
  • experienced verification personnel are able to evaluate the 3D quality and make decisions based on viewing the 3D images on the screen even though the screen may be smaller in size compared with the screen of a destined 3D cinematic theater.
  • the front-end subsystem 302 also provides computing hardware and software required for detecting changes between two different versions of image data.
  • the supporting software applications may include a GUI to perform visual inspection on the workstations 310 ( 1 )-( n ).
  • the supporting software applications provide a skilled user with sufficient information to make appropriate decisions following the decision process described in FIG. 2 . If direct changes to the RDR are needed, the software applications can provide a GUI that allows editing of RDR,
  • the front-end subsystem 302 can be deployed in a distant physical location from the back-end subsystem 304 using an open platform.
  • the front-end subsystem 302 may also be deployed in multiple physical locations from the back-end subsystem 304 .
  • at least a portion of the user interactive functions of the front-end subsystem 302 can be outsourced to multiple third-party commercial service providers that are equipped with the right types of hardware and software as well as skilled personnel.
  • Technical specifications regarding the types of IDC and PI that need to be collected from images can be provided to the service providers for evaluating outsourced work. Processing tasks executed at remote locations and by service providers can be recorded as render scripts and repeatable at the automated final render 316 .
  • the back-end subsystem 304 includes the intelligent controller server 312 , a central data storage 314 and an automated render system 316 ( 1 )-( m ).
  • the back-end subsystem 304 also provides image data input/output functions, typically provided by data I/O devices 318 such as data tape drives, movable hard disks or optical disk drives.
  • the central data storage 314 provides a data storage capacity sufficient for keeping different versions of source image data sequences, the converted 3D image data sequences and all necessary intermediate results on line for prompt assess.
  • the central data storage 314 also provides the function of render data records block 110 to keep the latest version of RDR.
  • the automated render system is typically implemented as a multi-processor computer cluster 316 ( 1 )-( n ).
  • the intelligent controller 312 provides process control and process tracking functions to the entire motion picture conversion process, including both the front-end subsystem 302 and the back-end subsystems 304 .
  • the intelligent controller 312 maintains a central database that keeps and updates all information about motion picture source images and the entire conversion process, including PI and IDC collected by the front-end subsystem 302 , different versions of RDR, and production management information such as processing change requests, version control and current status in the process. Based on the information, the intelligent controller 312 generates various types of real-time reports on the status of the conversion process.
  • the intelligent controller 312 is responsible for scheduling all rendering jobs of the automated render 316 as well as the pre-render 324 . It can distribute computing jobs to multiple processors based on load balance and job priorities. For a computing job that is distributed to multiple processors, the intelligent controller 312 can assemble the segmented results into a continuous scene. The intelligent controller 312 can also provide integrity checking for occasional missing or incomplete frames and sends re-render requests if errors are found.
  • the intelligent controller 312 can constantly monitor the status of individual processors of the automated render 316 ( 1 )-( n ). If the processor fails, it can raise an alert for repair and can reroute the stalled job to other available processors to continue processing. A diagnostics process ensures to prevent loss of data during the transition. If the intelligent controller 312 experiences a failure, the state of the system before malfunction is preserved. The intelligent controller 312 polls the render processors for their status, finds their current states and resumes the control. Data re-rendering may not be required in the case of a re-start.
  • the intelligent controller 312 also can monitor the operation of the image data input and output devices 318 .
  • the converted image data sequences can be formatted and output onto data apes or movable hard disks or any other data storage devices 318 .
  • the intelligent controller 312 schedules data input and output processes and reports the information back to the central database.
  • the user controls interface 320 allows a skilled user, typically a producer or a technical producer who is responsible for the final look of the resulting 3D images to make a certain types of changes usually for the improvement of the 3D look of the converted images or for the introduction of the latest editorial decisions. Those changes are usually made through direct modifications to the RDR saved at the central data storage 314 without a need for re-collecting IDC.
  • the system implementations illustrated in FIG. 3 can be scalable for expanding process capacity or for easy integration of future improved software and hardware.
  • the back-end subsystem 304 can continue to provide control, tracking and data exchange functions with multiple remote sites through Internet connections or designated network connections 322 .
  • the intelligent controller 312 is capable of controlling and tracking all processes at the remote sites and collecting all PI and RDR from multiple sites.
  • the back-end subsystem 304 is able to convert them into 3D images at the automated render 316 based on all the latest PI and RDR collected from multiple sites.
  • the core conversion block 118 in FIG. 1 can collect a set of PI required for the 2D-to-3D conversion. It can include a PI collection block 102 and a pre-render block 104 . Some types of PI are various types of IDC, which are image elements and other data that are extracted from the 2D image sequence for the purpose of computing 3D images.
  • PI is various types of IDC, which are image elements and other data that are extracted from the 2D image sequence for the purpose of computing 3D images.
  • FIG. 4 One embodiment of the process workflow of the PI collection block 102 for motion picture 2D-to-3D conversion is depicted in FIG. 4 .
  • the process flow diagram in FIG. 4 assumes that the 2D input image data sequences 400 are available in a digital data format. If motion picture images are available in a film format, images on film must be digitized into a digital data format by a film scanner. If the input images are available in a digital data format unsupported by the conversion process, the images must be converted to a
  • Each block in the process workflow of FIG. 4 represents a processing stage in which a specific set of IDC or PI are collected.
  • the first four stages, temporal analysis and enhancement 402 , scene separation 404 , scene layout planning 406 and scene automation analyzer 408 , work at the scene level, and the subsequent layer analyzer 410 , layer conversion 412 and scene compositing 414 stages perform at the object level.
  • a motion picture is divided into scenes, and each scene consists of one or more objects. Objects are image elements separated from each other mainly by depth discontinuity.
  • a layered structure is used to collect IDC describing individual objects.
  • the collection process is back to the scene level.
  • the pre-render block 104 is part of the core conversion block 118 and performs required computing for all stages of FIG. 4 .
  • the functions of the pre-render block 104 are delivered generally by the pre-render 324 .
  • a portion of the pre-render functions may also be provided by the workstations 310 ( 1 )-( n ).
  • the pre-render 324 functions were not explicitly depicted in FIG. 4 , they are inherent in every stage of the IDC collection process.
  • the IDC collected from all stages are saved along with other types of PI information 418 including editorial information, render scripts, computing parameters, intermediate results, etc.
  • a sufficient set of collected PI is saved in a form of the last version of RDR to be used to producing the intended conversion results.
  • An important feature of the present invention is that the RDR should be kept to be color-independent, which means that, if color of the source image data sequences are changed, there is no need to update the current version of RDR.
  • process flow of FIG. 4 is explained in terms of 2D to 3D conversion the same concept of the process flow can apply to any other graphics conversion such as frame rate conversion, image enhancement, any other image enhancement conversions or a conversion that facilitates further image enhancement within a projector to produce the enhanced images.
  • the algorithms within the process flow blocks may be different but the process flow of FIG. 4 can still be utilized.
  • stages are deployed in the core conversion block in FIG. 4 to collect IDC at scene level, including temporal analysis & enhancement 402 , scene separation 404 , scene layout planning 406 , and scene automation analyzer 408 . Each of the stages is discussed below.
  • the source image data sequences are processed by temporal processing methods to enhance image quality.
  • a preferred implementation of the temporal processing process is the DMR (Digital Re-mastering) process by IMAX Corporation and discussed in U.S. patent application Ser. No. 10/474,780.
  • the temporal processing also produces motion cues 424 that can be dense motion vectors that describe the movement of every pixel from one frame to adjacent frames.
  • the motion cues 424 are used as initial motion vectors in the scene automation analyzer stage 408 and the layer conversion stage 412 .
  • the source motion picture image data are divided into scenes, or shots.
  • scene and shots are interchangeable, both describing a sequence of images (or frames or image frames) or image elements of a continuous motion flow resulting from, in some embodiments, a single run of the camera.
  • image “frame” and “image frame” are interchangeable.
  • Scene separation is necessary because many IDC collection methods cannot handle abrupt changes between scenes or shots. If an edit decision list (EDL) is available, it may be used in the scene separation process.
  • EDL edit decision list
  • An EDL is an ordered shot list which records accurate information about every shot in a motion picture including shot length and time codes marking the start and end of each shot.
  • shots representing the same scene can be grouped together so that they can be treated in a similar way throughout the conversion process.
  • the shot grouping is part of the scene separation process, and it can be performed either by human inspection or by an automated method that searches for common characteristics that are shared by those shots.
  • One example is to group all shots that show a “talking head” of the same person in a similar background. Similar methods can be applied to collect IDC from all the shots in the group.
  • the scene separation stage 404 produces a shot list with shot group information 426 .
  • FIGS. 5A-C show an example of scene layout 428 .
  • FIG. 5A shows a 2D source image shot 502 of a baseball game, including a number of objects.
  • FIG. 5B shows a graphical description of the scene in FIG. 5A in an object layout 504 , where objects in the scene are identified.
  • FIG. 5C shows a graphical description of the scene in FIG.
  • Both the object layout 504 and the geometry layout 506 can be produced in a number of formats, including text descriptions, graphical descriptions or a combination of both.
  • objects include the left field player (object # 1 ), the shortstop and the base umpire in the center field (object # 2 , treated as a single object), the second umpire who is closer to the camera (object # 3 ), the player in the right (object # 4 ), the fence and audience seating (object # 5 ) and the baseball field (object # 6 ).
  • the object layout 504 identifies all those six objects without using precise outlines.
  • An object may contain more than one image elements, such as object # 2 that contains two human shapes which are at similar distances from the camera.
  • an object may include multiple disjointed elements sharing similar characteristics, like a flock of flying birds, blowing leaves or snowflakes.
  • An object may contain some undefined parts because it was partially blocked by another object in the foreground, such as object # 2 that is partially blocked by object # 3 in the selected frame.
  • An object may disappear in some frames, such as object # 1 that may move out of the scene in later frames as the camera pans to the right.
  • the background of an image scene is also identified as one or multiple objects which are treated no differently from other objects.
  • the background of the scene can be split into two objects: object # 5 , including the fence and the audience seating and object # 6 , the field.
  • the object layout 504 provides the information necessary for a subsequent layer analyzer stage to produce a layer definition for each layer.
  • objects are assigned with distinctive colors, denoted with hashes, for the purpose of labeling object masks as described in later sections. The colors are selected from a pre-defined color palette in a specified order,
  • the geometry layout 506 describes the scene geometry, which typically includes simple geometry models approximating object shapes, dimensions and spatial locations relative to the camera position in a three-dimensional space.
  • a geometry layout 506 is a graphical illustration.
  • Each object identified in the object layout 504 is modeled with an approximate shape and location in a three-dimensional space which is centered at the camera position.
  • the horizontal axis x extends following the camera baseline in parallel with the horizontal axis of the camera image plane.
  • the camera baseline also defines the positions of the stereoscopic 3D camera for rendering new 3D images in the subsequent layered conversion.
  • the y-axis is usually the vertical axis or in parallel with the y-axis of the camera image plane (they axis of the image plane may not be vertical when the camera is tilted).
  • the geometry layout 506 provides the information necessary for a subsequent layer analyzer stage to produce layer definition.
  • a scene layout 428 is generally planned by a human operator equipped with appropriate software tools.
  • An operator can interpret the geometry of a scene by viewing the 2D images and identify foreground objects from background objects without difficulty.
  • An operator can also produce a more accurate estimation of the scene geometry using a depth by scaling method.
  • the depth by scaling method is also referred to as solid depth modeling.
  • the scene automation analyzer 408 is a processing stage in which special classes of shots that are suitable for automated IDC collection methods are identified.
  • the following scene classes 430 are identified for automated processing with assistance from the motion cues 424 generated from stage 402 :
  • scene class D which include user interactive IDC collection methods.
  • the above three types of scenes can be identified by a skillful human operator or by automated methods. The following automated methods may be used to identify the above three scene classes.
  • the automated methods generally relate to detecting dominant motion.
  • a RANSAC Random Sample Consensus algorithm 604 in FIG. 6 to detect the existence of dominant motion homography.
  • the RANSAC 604 first collects global motion statistics from each pair of image frames based on global homography motion models, and then detects dominant motion from global motion statistics collected from all image frame pairs by maximum votes through a voting process. If a dominant motion homography is found, the algorithm analyzes if it contains a majority of motion vectors.
  • the algorithm assumes that a second dominant motion homography may exist, and it searches for the next dominant motion. This search process is repeated until every dominant motion homography is found.
  • the algorithm 604 then calculates the directions and average absolute magnitudes of the motion vectors belonging to each dominant motion homography.
  • FIG. 6 One embodiment of the scene classification process in the scene automation analyzer stage 408 is described in FIG. 6 .
  • the process starts with analyzing the classified motion homography models 606 produced by the RANSAC algorithm 604 . If only a single dominant motion homography is found and it contains a dominant majority of motion vectors, the image shot is a candidate for either class A or class B.
  • the scene automation analyzer 408 further checks whether a majority of those motion vectors point to the same horizontal or vertical direction. If this is the case, the image shot is classified as scene class A, and the above decision process is referred to as classifier # 1 608 . If it is classified as scene class B, and the decision process is referred to as classifier # 2 610 .
  • the automation analyzer 408 performs a further analysis to determine whether image pixels belonging to one of the dominant motions is in sharp focus while the other pixels are blurry. For example, a check is made to determine if there is a substantial difference in the density of image feature points between those two groups of pixels. Image feature points are calculated by computing the high-order (4 th order) central moment statistics for each pixel within a small neighborhood. The results are scaled to an 8-bit per color integer representation, and a threshold is applied to produce a map of feature points for each frame. The density of feature points between those two groups of pixels can be analyzed. Objects in sharp focus would have a much higher feature density than the blurry background. If one group of pixels has a very higher feature density while the other does not, the scene is classified as scene class C. The above methods are performed in classifier # 3 612 .
  • the automated scene classification can be visually verified by a human operator to ensure the decision is appropriate. If an image shot is considered misclassified, it is by default downgraded to scene class D.
  • the decision-making process of FIG. 6 can be expanded if more scene classes are discovered for automated processing. In such cases, more classifiers can be added.
  • the scene level IDC collected from the above four processing stages are used to facilitate the collection of more detailed and more accurate IDC at the object level.
  • a layered structure is used to describe the complexity of scenes at the object level, allowing IDC to be collected at any levels of details and accuracy.
  • objects assigned to one layer are processed independently from objects assigned to the other layers. Therefore, objects with different characteristics can be processed at different layers using different processes, and objects with similar characteristics can be processed in the same layer with the same process.
  • Various types of IDC collected from different layers are combined later in the scene compositing stage 414 .
  • the layered structure allows various methods to be integrated into the 2D-to-3D conversion process to handle complex objects.
  • objects are assigned to different layers based on characteristics. Multiple criteria are used to group image objects and elements into layers. One of those criteria is the description of object transparency.
  • a transparent object is described by multi-dimensional pixels, which permits an image pixel or a group of pixels to be shared by multiple objects.
  • a multi-dimensional pixel may have multiple pixel values and depth values, each describing a distinctive object that can be seen at the same pixel location in the images.
  • FIGS. 7A-B show examples of transparent objects. In FIG.
  • pixel P(x 1 , y 1 ) 710 is the result of combined contributions from all three transparent objects, and each object may have different pixel values and different depth values at pixel P(x 1 , y 1 ) 710 .
  • the object of tree reflection 706 has a depth value at pixel P(x 1 , y 1 ) 710 corresponding to the distance 712 from the camera to the windshield, while the object of driver 704 has a different depth value at the same pixel corresponding to the distance 714 from the camera to the driver, as shown in FIG. 7B .
  • the color of the tree reflection at pixel P(x 1 , y 1 ) 710 is different from the color of the driver's face.
  • the layered structure provides a method of describing transparent objects.
  • pixel P(x 1 , y 1 ) 710 is defined as a multi-dimensional pixel with three sets of pixel values and depth values: (1) one representing the color and the depth of the windshield glass 702 ; (2) one representing the color and depth of the tree reflection 708 ; and (3) one representing the color and depth of the driver's face 704 .
  • Each object is represented at a distinctive layer in a layered structure, and the multi-dimensional pixel P(x 1 , y 1 ) 710 is split among those three layers with each layer having one set of pixel value and depth value corresponding to the object represented by the layer.
  • the layered structure allows each transparent object be processed independently from other objects.
  • the re-constructed versions of each multi-dimensional pixel at different layers may be mapped to different locations because of difference in depth values.
  • the results from those three layers and those re-constructed versions from different layers are later combined to form 3D images.
  • Such a result is a realistic representation of real world appearances in that pixels representing multiple objects are separated from a different viewing angle.
  • FIGS. 7A-B may not be easily described within a single layer.
  • the same method can be used to describe translucent and reflective image objects such as mist, clouds, fire and reflections.
  • the layered structure provides a method of reconstruction of missing portions of objects.
  • An object may be partially blocked, or occluded, by other objects in the foreground, but the occluded regions may be revealed when the images are reconstructed from a different viewpoint. There is no immediate information in the occluded regions from the original images, but the missing information may be recovered from the other frames of the image sequence.
  • the layered structure allows all frames of an occluded object be stored in a specific layer so that reconstruction is performed within the layer and independently from other layers.
  • the layered structure also allows the re-constructed objects be represented in multiple layers in a similar way as the multi-dimensional pixels so that the occluded regions can be filled when the results from different layers are combined.
  • a frame 802 from a 2D image sequence includes a background object that is a house 806 , occluded by a foreground object that is a tree 804 .
  • the tree object 804 is separated from the image 802 as well as from all the other frames from the image sequence.
  • the image frames containing the house 806 as the remaining background object are saved in one layer, separated from the tree object 804 which is represented in another layer. If the missing part of the background object is revealed in other image frames due to camera motion, the missing pixels can be tracked, transformed and reproduced so that the house 806 can be reconstructed within the layer.
  • Both objects 804 , 806 are computed from a new camera view and the results are combined to form a new image 808 in which the occluded part of the house 806 is filled with reconstructed information.
  • Automated object separation is another method used in some embodiments of the present invention.
  • the layered structure provides a method of separating objects in image shots with a shallow depth of field (DOF).
  • DOF shallow depth of field
  • main objects are in sharp focus while the other objects are blurry because they are outside the DOF.
  • Objects that are in focus can be assigned to a layer separated from the blurry images so that they can be separated automatically, for example, shots classified as scene class C.
  • a particle object can contain many irregular elements, like snow flakes, rain drops, blowing leaves, plankton in the ocean or water bubbles.
  • irregular elements like snow flakes, rain drops, blowing leaves, plankton in the ocean or water bubbles.
  • a large number of similar irregular elements are collectively defined as a single object and represented in a single layer.
  • the geometry of those elements can be defined by a single depth model within the assigned layer and all the elements belonging to the object can be processed simultaneously within the layer.
  • Object size compensation is used in some embodiments of the present invention.
  • Miniaturizing in stereoscopic vision may result from size-distance laws in which the perceived size of an object diminishes when it moves closer to a viewer.
  • the perceived distance of an object is determined by the horizontal disparity of its images in left and right images, which is referred to as parallax.
  • parallax increases, the object is perceived closer to a viewer, but the apparent size of the object is perceived to decrease.
  • Reverse miniaturizing can also occur when the parallax is reduced and the object appears to move away from a viewer but it also appears bigger. The effect of miniaturizing or its reverse is contradictory to real life experience and it may not be acceptable to the audience.
  • One solution is to digitally scale the size of the object in the images before 3D images are computed in order to compensate for miniaturizing or reverse miniaturizing.
  • an object When an object is perceived to be miniaturized, it can be scaled up using an up-sampling method to compensate.
  • the scaling factor is calculated based on the level of miniaturization that can be estimated by applying size-distance laws.
  • the scaling factor can vary from frame to frame if the object changes its distance throughout the image shot.
  • objects that require size compensation are assigned to a separate layer so that they can be scaled in a similar way.
  • Motion classification is also used in some embodiments of the present invention.
  • the layered structure provides a method of processing objects with different motion characteristics. Objects in a scene can be classified into a number of classes based on their motion characteristics. One example is to classify objects into the following five classes:
  • Object Class 1 with dominant camera motion
  • Object Class 2 moving towards or away from camera
  • Object Class 3 moderate motion with no significant motion blur
  • the decisions of object layer assignment and object classifications by the layer analyzer stage 410 are stored by a data record referred to as layer definition.
  • layer definition there is one layer definition for each layer.
  • the layer definition records the objects assigned to the kth layer in a form similar to the scene object layout 504 . It also defines the geometrical relationship between the objects in the kth layer in a form similar to the scene geometry layout 506 .
  • the layer definition also defines the estimated dimension for each object in the layer as well as its depth range in order to avoid potential conflict with other layers.
  • the scene classes and object classes are also recorded in the layer definition.
  • FIG. 9 shows one embodiment of a process flow diagram of a single layer including multiple processing modules.
  • the multiple processing modules can include an object mask generation module 902 , an object depth modeling module 904 , an object reconstruction module 906 , an object depth map generation module 908 , an object scaling module 910 , and a 3D view computing module 912 .
  • One function of the object mask generation module 902 is to produce object masks 920 that describe each object defined in a layer definition information 924 in every frame.
  • An object mask defines a shape, and all pixels included in the object mask have color information.
  • An object mask may contain blank regions without proper color information. For example, an object that is partially occluded by another foreground object so that the occluded part of the object has no color information.
  • the blank regions remaining in the new images are called occlusion regions.
  • the occlusion regions can be filled with proper information in the scene finishing stage 416 , the process is usually tedious and costly.
  • the missing information is revealed from other frames of the image shot as the result of object or camera motion. In those cases, the blank regions can be reconstructed by motion tracking and image registration.
  • the object reconstruction module 906 performs the task of tracking and recovery of the missing information wherever possible. Both object masks and reconstructed objects are types of IDC.
  • IDC Incus of an object.
  • the depth cues are not directly available from 2D images, but they may be estimated in some cases or approximately modeled in other cases.
  • the object depth modeling module 904 produces an object depth model, which is an approximation of the geometrical shape of the object.
  • a depth model is required to match the movement and deformation of the object throughout the image shot.
  • a depth model can be created without precise object contour information, so that the object depth modeling 904 can be carried out in parallel with the process of object mask generation 902 .
  • Each resulting depth model is then matched to the precise object masks of the same object generated from the object mask generation module 902 .
  • Object masks that are paired with depth models are used to produce accurate object depth maps 916 .
  • producing object depth maps 916 can be performed by the object depth map generation module 908 .
  • the left and right images of objects can be computed. However, if object scaling is required, the objects and their corresponding object depth maps 916 must be scaled by the same factor.
  • Object scaling is performed by the object scaling module 910 .
  • the scaled objects and the scaled object depth maps are used to compute the left and right images 918 of the object by the 3D view computing module 912 .
  • the computed 3D images can be verified and, if accepted, the IDC collected, including detailed object masks, detailed object depth maps, reconstructed objects together with render scripts, render parameters and other necessary data from all processing modules, is saved as RDR for the specified layer. Processing module details according to some embodiments of the present invention are disclosed in the following sections.
  • One function of the object mask generation module 902 is to generate object masks that define the contour of each object defined in layer definition information 924 . Because an object is defined for the entire image shot, the precise object masks are created for every image frame of the shot wherever the object is present. If an object consists of multiple disjointed elements, the object masks will contain multiple disconnected segments for all applicable frames. The object masks are usually created at the same image resolution as the original images.
  • FIG. 10 is a process flow diagram of one embodiment of the object mask generation module 902 .
  • a plurality of methods of mask generation may be deployed, each belonging to one of the three operation modes: automated mode 1008 , semi-automated mode 1006 , and manual mode 1004 .
  • An operation mode analyzer 1002 determines an appropriate method for each object by analyzing the motion class of each object and its background objects and other image characteristics. In one embodiment of the present invention, five methods are deployed. These methods include polygon based separation 1010 , color-based separation 1012 , graph-cut based separation 1014 , and shape recovery 1016 , and automated depth recovery 1018 .
  • the operation mode analyzer 1002 may decide to use more than one method and combine the results through mask compositing 1020 .
  • the operation mode analyzer 1002 also determines whether precise object masks are produced only for key frames and whether mask tracking 1022 is needed to produce masks for frames in between key frames.
  • the multi-mode structure of the object mask generation module 902 ensures more automated methods and semi-automated methods can be easily integrated into the present invention to improve process efficiency as technology progresses.
  • the same computing structure is shared by the object depth modeling module as well as the scene finishing stage.
  • the operation mode analyzer 1002 selects an appropriate operation mode for each object. First, it checks the motion class of an object. For a class 1 object, its depth maps may be computed directly by the automated depth recovery method 1018 , implemented in the object depth generation module 908 in FIG. 9 . The class 1 object can bypass the object mask generation module completely and no object masks are created. Such a bypass path 926 .
  • the operation mode analyzer 1002 performs a subsequent analysis to determine if the object is in sharp focus while its background objects are blurry. This is done by determining if there is a substantial difference in feature density between the object and its background. Objects in sharp focus have a much higher feature density than those that are out of focus. If an object is in sharp focus while its background objects are not, the object masks can be generated by the automated shape recovery method 1016 .
  • Semi-automated separation mode 1006 includes methods by which object masks are produced without directly specifying points on the boundaries. Multiple semi-automated separation methods may be implemented. In one embodiment of the present invention, two semi-automated methods are implemented, including a color-based separation method 1012 and a graph-cuts based separation method 1014 .
  • the operation mode analyzer 1002 first computes the color statistics of an object. If the object has a relative uniform range of colors with only small variations and if the object colors are significantly different from background colors, it can be separated by the color-based separation method 1012 .
  • the object can be separated by the graph-cuts based separation method 1014 .
  • a polygon-based separation method 1010 can be used. Although the polygon-based separation method is flexible, it is generally a manual operation.
  • the operation mode analyzer 1002 may also choose to use more than one method for separation and to combine the results from different methods at the mask compositing module 1020 .
  • Another function of the operation mode analyzer 1002 is to analyze the motion fields of an object to determine if key-framing is needed and the exact key frame positions in a shot for each object.
  • Key frames are selected to mark abrupt changes in object motion and the motion between key frames is supposed to be smooth. With key frames defined, precise object masks are produced only at key frames by one or multiple mask separation methods, and the object masks for other frames in between key frames are produced through mask tracking 1022 .
  • shape recovery 1016 is the only automated separation method deployed.
  • shape recovery 1016 object shapes are calculated based on differences in focus. This method applies to a class 0 object that is in sharp focus while its background objects are blurry.
  • the first step of shape recovery 1016 is to compute a feature mask map for an object with surrounding background objects for each frame. The method of computing a feature mask was described previously for identifying scene class C at the scene automation analyzer stage 408 . Because a class 1 object has a much higher feature density, object masks are created by repetitive applications of a morphological reconstruction closing filter followed by a morphological reconstruction opening filter to the feature mask maps. If the resulting object masks are not sufficiently precise, the object should be re-assigned to a semi-automated mode.
  • color based separation 1012 selected color values are relative to the source image data, so that the results are color-independent and repeatable when the source image data has color changes.
  • Color based separation methods 1012 is a gradual refinement process in which users need to refine the selection of color values until precise object masks are generated. Certain embodiments of the color based separation methods 1012 are more suitable for objects with a relative uniform range of colors that are very different from background colors.
  • the graph-cuts based separation method 1014 is a semi-automated method in which users specify “seeds” for both inside and outside the object to be separated. Like a color based separation method, the graph-cuts separation method 1014 is more suitable for objects whose colors are different from background colors.
  • user input is through a GUI environment.
  • users draw a set of free-hand lines, strokes or polygon shapes to mark the interior of an object to be separated and another set of freed-hand lines, strokes or polygon shapes to mark background objects. Those two sets of lines, strokes and shapes form the “seeds” and they are usually marked with different colors.
  • Those “seeds” may be required only at key frames, and the “seeds” needed for the frames in between key frames are interpolated from key frame “seeds.”
  • a graph cuts algorithm will separate the object automatically. If the resulting object masks are not precise enough, a user will add more “seeds” near problematic portions of the masks and repeat the algorithm until required precision is achieved. Certain embodiments of the present invention use an improved graph cuts based separation method. Improved graph cuts based separation methods may be understood with a discussion of original (unimproved) graph cuts algorithms.
  • the original graph-cuts algorithms are designed to minimize an energy function E among all possible labels.
  • the approach is to define two special terminal nodes, one called source and another called sink.
  • a graph-cuts result is a partition of all non-terminal nodes with some connected to the source while others connected to the sink.
  • the best graph-cuts are achieved when the energy function E is minimized.
  • all pixels in an image are defined as a set V with each pixel being regarded as a node in set V.
  • Each pixel forms a pair with a neighbor pixel in a 4-neighbor neighborhood, and all pixel pairs form another set E with each pixel pair as a node in set E.
  • a user defines two sets of “seeds”: one set of seeds representing the object and the other representing the background.
  • the “seeds” do not need to be located on the object contour but they need to cover a representative color range of the object, or the background.
  • image pixels are partitioned into three sets: those belong to the object F, those belong to the background B, and those undefined U. This step is known in the art.
  • the next step is to collect all pixels in set F, perform cluster analysis in set F and divide the set into a number of subclasses. The average color of all subclasses are calculated and used as the representative colors of set F, the object. The same process is repeated to calculate the representative colors of B. Then for each node i in the undefined set U, compute the minimum color difference with the representative colors of F as d(i,F) and the minimum color difference with the representative colors of B as d(i,B). Define a first energy term E 1 as:
  • each node is a pair of pixels (i, j), and a second energy term E 2 is defined in set U as:
  • E 2 ⁇ ( x ⁇ ( i ) , x ⁇ ( j ) ) ⁇ x ⁇ ( i ) - x ⁇ ( j ) ⁇ ⁇ 1 1 + ⁇ C ⁇ ( i ) - C ⁇ ( j ) ⁇ , where ⁇ C(i) ⁇ (j) ⁇ is the absolute color difference of pixels i and j.
  • An energy function E(X) is constructed from E 1 and E 2 as:
  • E ⁇ ( X ) ⁇ i ⁇ V ⁇ E 1 ⁇ ( x ⁇ ( i ) ) + ⁇ ⁇ ⁇ ( i , j ) ⁇ E ⁇ E 2 ⁇ ( x ⁇ ( i ) , x ⁇ ( j ) ) .
  • the original graph-cuts algorithm finds the best partition of pixels (nodes) that minimizes the energy function E(X) and, as a result, separates the object from the backgrounds. If the resulting object masks are not precise enough, more “seeds” can be added near problematic areas, which update the set F and B. The process is repeated until the results are satisfactory.
  • Improved graph-cut based separation methods includes a double-pass object mask generation method based on graph-cuts algorithms.
  • the double-pass object mask generation method reduces the required computing time while maintaining sufficient accuracy.
  • the images are first segmented into regions using a color-based segmentation algorithm, such as, for example, a “Watershed” algorithm or a “Meanshift” algorithm.
  • the segmented regions replace pixels as the nodes that form set V.
  • Each pair of neighboring regions replaces pixel pairs as the nodes that form set E.
  • the average color of each segmented region is defined as the color of the node C(i).
  • the energy functions E 1 , E 2 and E(X) can be defined in the same way, so that the graph-cuts algorithm can be used to separate segmented regions into either the object or the background.
  • the result from the first pass is a rough object mask that needs further refined in the second pass.
  • the second pass refines rough object masks by applying graph-cuts algorithms at pixel level on edge pixels that separate foreground and background regions.
  • the edge pixels are those that are located on or near the edges of rough object masks, and they are redefined as nodes in set U. The remaining pixels will keep their labels from the first pass as nodes either in set F or in set B.
  • the energy function E(X) is defined based on sets F, B and U in the same way as descried previously.
  • the graph-cuts based separation methods 1014 are applied to further partition undefined edge pixels into either the object or the background.
  • the object mask generation module includes the polygon based separation method 1010 as one of the user-assisted methods.
  • Mask compositing 1020 is a process that combines the results from different methods to form a precise object mask. This process can be implemented using software with a GUI environment so that a user can use set logical operations, such as AND and OR, to select the best portions of the mask from a set of masks and combine them into a precise final result.
  • Mask Tracking 1022 can be used if the combined object masks are created at key frames.
  • Mask tracking 1022 predicts object masks for the frames in between two key frames. Because the precise movement of the object from one key frame to the next key frame is known, motion tracking can focus on pixels near the edges of the mask. Unlike most of key frame interpolation methods that require correspondence of mask boundary control points between two key frames, some methods do not require such a correspondence.
  • the following method is used with five different motion models: translation, affine, projective, quadratic and piecewise spline.
  • a narrow band inside the mask boundary is created by subtracting from a smaller shape from the mask by applying a morphological erosion to the mask.
  • the features inside the band are extracted and used for tracking.
  • the feature points inside the band are tracked to the next second frame.
  • the closest motion model among all five-motion models is selected as the motion transformation model.
  • the mask for the second frame is created by applying the selected motion model to all the boundary points of the mask at the key frame.
  • the resulting mask is refined using local warping, in which the boundary of the mask is divided into a number of segments of equal arc length.
  • the tracked features within each segment are used to re-calculate local motion transform model, and the resulting model is subsequently used to refine the boundary of the segment. This process is repeated for all the segments until the entire boundary of the mask of the second frame is refined. Any discontinuity between segments needs to be smoothed.
  • the second frame mask is completed, it is used for creating the mask for the next third frame and the process may be repeated as needed.
  • Another tracking method is used based on global motion.
  • the idea is to compute the global motion of feature points inside the object mask in a key frame and track their movement to subsequent frames towards the next key frame. Any feature point that disappears during the course is considered a bad feature, and any feature that moves to the outside of the mask is also considered a bad feature. After all bad feature points are removed, a set of good feature points is obtained, which is called forward feature points. Repeat the same process from the second key frame backwards to the first key frame and another set of good feature points is obtained, which is called backward feature points.
  • the global motion of each frame is estimated by calculating the center of all good features (both forward and backward feature points) in the frame.
  • the next step is to calculate local motion for each node of the object mask.
  • Nodes of the object mask can be control points created during polygon-based separation mode 1010 , or be calculated from mask boundary using other methods, such as the Douglas-Peucker algorithm.
  • the features that fall into a neighborhood of the node are analyzed and the local global motion is obtained from those features. If there are no features in a local area, the global motion is used for the node. For every frame in between those key frames, the corresponding interpolation coefficients are computed based on local global motion.
  • a new object mask is created based on the interpolation coefficients at each frame in between key frames. The results from this method are substantially more accurate than typical linear interpolation based on node correspondence as implemented in most of commercial software tools today.
  • object masks are labeled with the same color label, shown with hash marks, as defined in the scene layout example shown in FIG. 5B .
  • all object masks of an image shot can be combined in an object mask image sequence in which masks of the same object are marked with a unique color.
  • FIGS. 11A-B the 2D source image example 1102 is the same one as used in FIG. 5A .
  • the color labeled object mask image sequence 1104 in FIG. 11A shows that each object is assigned with the same color label as defined in the object layout 504 in FIG. 5B .
  • the color labeled object mask image sequence is used in the subsequent object depth map generation module 908 for refining object depth maps.
  • one function of the object depth modeling module 904 is to give each pixel of an object a depth value.
  • the main concept is to approximate the three-dimensional shape of an object with a graphical model in real-world dimensions.
  • the depth values can be calculated from the position of a pixel on the surface of the depth model to the position of the camera. In special cases, however, depth values can be directly calculated from scene motion cues in an automated mode.
  • depth modeling method of certain embodiments of the present invention differs from some other known methods of assigning depth values.
  • depth models do not define object contour.
  • the depth model of an object is not required to match object contour precisely.
  • This distinction allows depth modeling to be performed based only on approximate object descriptions defined in the layer definition.
  • the functions of object depth modeling module 904 can be performed independently from the functions of the object mask generation module 902 , with some exceptions, and, can be performed in parallel with object mask generation.
  • FIG. 12 shows a process flow diagram of one embodiment of the object depth modeling module 904 , which has a multi-mode structure similar to that of the object mask generation module 902 .
  • one automated mode 1208 may include automated depth recovery 1220 .
  • the semi-automated mode 1206 may include generic depth modeling 1212 , bump-map depth modeling 1214 , feature-guided depth recovery 1216 , and mask-guided depth recovery 1218 .
  • the manual mode 1204 may include a solid depth modeling method
  • the operation mode analyzer 1202 can select an operation mode for an object based on its motion class. In addition, the operation mode analyzer 1202 also determines if the object is a generic object.
  • a generic object can be represented by a more universal generic depth model.
  • One example of a generic object is a close-up of a person's face while talking, or a “talking head,” which is very commonplace in a motion picture. Because human faces share similar shape, a “talking head” can be approximated by a generic head model that can be used for other human faces.
  • Another example of a generic object is a car model that can be used to approximate other types of cars.
  • One requirement for a generic model is that it can be scaled and animated to follow the motion of a variety of object that it may represent.
  • the operation mode analyzer 1202 assigns the object to the generic depth modeling method 1212 . Otherwise, it analyzes the motion class of an object to determine the method to apply. If the object is a class 1 object, it is assigned to the automated depth recovery method 1220 . If the results from the automated depth recovery 1220 are not satisfactory, the object will be assigned to one of semi-automated depth recovery methods 1206 .
  • the solid depth modeling method 1210 is selected for an object that cannot take advantage of either an automated mode or a semi-automated mode. Based on real-world object dimensions defined in the layer definition, a depth model is created for each object using a modeling software tool, such as Maya produced by Alias.
  • a depth model encloses the object, but does not need to match the precise boundary of the object.
  • depth modeling does not require the creation of full geometrical details.
  • the depth model matches the movement of the object in a scene, which can be achieved by animating the camera or by animating the object or by doing both. The quality of animation is not important. As a result, modeling of depth is a simpler process than conventional solid modeling processes.
  • the estimation of real-world dimensions and depth of objects is performed using a depth by scaling method.
  • depth by scaling is implemented in a software tool that is capable of constructing a virtual three-dimensional environment in which a virtual camera can be placed to capture images of the environment on a 2D image plane from the viewpoint of the camera.
  • the software provides a plane situated within the three-dimensional environment, viewed by the camera, and the 2D image sequence that is to be converted to 3D can be attached, like a virtual billboard, to the image plane.
  • the software is also capable of creating three-dimensional models and placing them in the three-dimensional environment. The movement of the models and the virtual camera can be animated by a user.
  • a virtual camera is a mathematical model of the image formation function of a physical camera.
  • a virtual camera is described by a set of parameters, including lens focal length, field of view and additional lens characteristics.
  • a virtual camera can be implemented by software in a calibrated three-dimensional virtual environment.
  • One embodiment of the depth by scaling method is described in a flow chart in FIG. 15 .
  • Camera parameters 1522 are received and used to calibrate the virtual camera 1502 .
  • the real-world dimension of the object such as layer definition 924 is received.
  • the layer definition 924 may include a person that has a height of 5 feet and 10 inches and an automobile that has a length of 5 meters or a width of 2 meters.
  • a geometrical model that matches at least one real-world dimension of the object is built within the virtual environment at step 1504 .
  • An operator selects a frame of the 2D image sequence at step 1506 that contains the object and places the geometrical model with the object dimension at the position of the object in the frame within the virtual environment at step 1508 .
  • an image of the model through the calibrated virtual camera is computed. The operator can check if the size of the computed model image matches the dimension of the object by comparing the size of the model with the object size in the frame at step 1512 . If the size does not match, the operator then moves the model towards or away from the virtual camera in step 1516 and re-computes the image of the model in step 1510 until it matches the size of the object in the 2D image frame. The resulting location of the model provides a good estimate of the real-world position of the object.
  • the user then computes the depth of the object for the selected frame in step 1518 .
  • This process may be repeated for a sufficient number of key frames in the image sequence by selecting the next frame in step 1517 until the depth of the object is established for the entire sequence and collected at step 1518 .
  • the depth path of the object for the entire sequence is then outputted at step 1520 , which can be used for depth modeling and animation. If a layer contains more than one object, the process described in FIG. 15 can be repeated for every object assigned to the layer until the depth paths of all objects are estimated.
  • the final step of the solid depth modeling is to compute a depth map from the depth model for every frame. Since the depth model does not match the precise object boundary, the resulting depth map sequence is called a rough object depth map sequence or a rough object depth map.
  • the rough depth map can be used as the generic model for the next method, generic depth modeling 1212 .
  • an appropriate generic model is selected from a predefined generic model library or created using the solid depth modeling 1210 process.
  • the generic model is scaled and animated to match the motion of the object.
  • This process is known as “matchmove,” and can be performed through automated, semi-automated or manual methods.
  • matchmove is accomplished using feature matching.
  • a set of features are predefined for each generic model in the library, and those features are selected because they are easily identifiable and traceable from images.
  • typical features used are facial features, such as corner points from eyes, lips, nose, eyebrows and ears.
  • a user identifies those feature points from object images at some key frames, and the user tracks the precise locations of the feature points in every frame with the assistance of software.
  • the precise feature points located in each frame are mapped to the features of the generic model, and a corresponding three-dimensional transformation is calculated. With the three-dimensional transformations as an animation path, the animated generic model becomes the depth model of the generic object.
  • An object depth map can be extracted from a depth model by calculating the Z-depth of every pixel of the depth model for every frame as descried previously.
  • the resulting rough object depth map can be refined by the object depth map generation module 908 .
  • a typical class 1 object usually includes both foreground and background image elements used to in depth recovery.
  • the simplest camera movement is a pan or tilt, which results in direct correspondence between depth and motion.
  • the camera movement is deducted from pixel motion in order to establish depth-motion correspondence.
  • the detection of camera motion starts with feature detection and tracking.
  • Object features are selected automatically from each frame and tracked to neighboring frames.
  • the features are classified based on their motion in a similar way as described in the previous section describing the scene automation analyzer 408 .
  • One difference may be that the scene automation analyzer 408 uses all motion vectors, but only feature motion vectors are used in depth recovery 1220 .
  • a planar-based motion model called homography is used for each motion class.
  • Features classified into the same motion class share the same motion homography model.
  • the camera motion is detected from dominant homography models.
  • the camera motion is then obtained from pixel motion, and the resulting “pure” pixel motion has correspondence with object depth.
  • Depth value of each pixel of an object can directly be computed from such “pure” pixel motion cues.
  • the resulting depth values are filtered using a low-pass filter to remove spatial errors.
  • the temporal errors of the resulting depth values can be reduced through temporal filtering, similar to the method deployed in the temporal analysis & enhancement stage 402 .
  • the depth values that are both spatially and temporally filtered form rough depth maps.
  • the feature-guided depth recovery method 1216 is a semi-automated mode suitable for class 1 objects. This method is usually selected when the automated depth recovery method 1220 fails to deliver satisfactory results. However, the method can also be selected directly by the operation mode analyzer 1202 . This method differs from the automated depth recovery method 1220 in that object features are selected and tracked with user guidance, instead of automated operations.
  • the input from a user improves the accuracy of feature motion tracking. Typical features selected by a user can be, but are not limited to, corner points, edges and lines, which can easily be identified by a user and reliably tracked by software. More accurate feature points result in more accurate motion models and reduction in motion estimation errors.
  • the subsequent steps of camera motion estimation and depth value computation remain the same as described in the previous section.
  • the mask-guided depth recovery method 1218 is another semi-automated mode that is suitable for a subset of class 2 objects consisting of a large number of relatively small elements that move towards or away from the camera. Typical examples are a school of swimming fish, floating planktons, water bubbles and flying debris, all being difficult and tedious to model.
  • This method differs from the automated depth recovery method 1220 in that object masks are provided (as a part of step 926 ) and used as a guide for motion tracking.
  • the object masks can be created by the object mask generation module 902 . With object masks available in every frame, each small element is reliably tracked, and its size is measured.
  • an element is assigned with a single depth value in each frame calculated based on the size and the motion of the object at the frame interval.
  • a large size element is considered closer to the camera than a smaller element, and a fast moving element is also considered closer to the camera than a slow moving element.
  • the assigned depth values are smoothed by a low-pass filter to achieve temporal consistency.
  • the bump-map depth modeling 1214 is another semi-automated mode that can produce depth maps for a certain types of class 1 objects with complex geometrical details. Examples of such objects are trees, plants, coral reef and rocks with pores, all with complex surface details that are difficult and tedious to model.
  • the approximate geometrical shape of such an object is created by a user using simple solid modeling method, and a preliminary depth map is generated from the model.
  • the geometrical details can be added to the preliminary depth map by computing a difference depth map based on color and shading.
  • a bump-map mask is computed from object color and shading, which defines the regions of the object where more depth details are needed.
  • a difference depth map is calculated based on shading so that dark pixels represent deeper depressions while areas with highlights are considered protrusions.
  • a difference depth map is calculated based on color differences. The resulting difference depth map is used to refine the preliminary depth map to produce a more detailed object depth map.
  • a class 1 object may contain a small region whose motion is independent from camera motion.
  • the depth values in such a region cannot be accurately estimated using automated 1208 or semi-automated 1206 depth recovery methods.
  • the small regions may require the use of a different depth modeling mode, such as solid depth modeling 1210 .
  • the resulting two depth maps need to be combined because each is accurate only for part of the object.
  • depth values recovered using automated or semi-automated methods do not match those created by a human operator using a manual method.
  • One stop in depth map compositing 1222 is to scale the depth maps to ensure depth values are consistent.
  • a depth map from an automated or a semi-automated method is converted to a surface by laying a grid mesh on the depth map and identifying key points at the grid lattices that represent abrupt changes in depth values.
  • the resulting mesh surface can be scaled to match the geometry of another depth model. Once all depth models match, different depth maps can be combined into a single depth map based on minimal depth values.
  • the resulting depth maps are referred to as object rough depth maps 1226 .
  • the object rough depth maps 1226 produced from solid depth modeling 1210 , generic depth modeling 1212 and bump-map depth modeling 1214 are usually not accurate to match an object boundary.
  • the scope of an object rough depth map is computed and produced as object rough masks 1228 .
  • the rough masks are labeled to match the precise object masks of the same object. In one embodiment of the present invention, the rough masks use the same color labeling method as described in FIG. 11 . If the depth model encloses the object, the resulting object rough masks 1228 should enclose the precise object masks of the same object.
  • one function of the object depth map generation module 908 is to produce accurate object depth maps 916 .
  • Object depth maps 916 are computed by refining the object rough depth maps 924 using precise object masks 920 to define object boundary. This process can be achieved by “masking” an object rough depth map 924 using precise object masks 920 of the same object. Because both rough and precise object masks share the same color palette, this process becomes a simple color matching in which every pixel of an object rough depth map is matched to the corresponding object mask. If the color of an object rough mask pixel matches the color of the precise object mask, the depth value of this pixel is kept. If the colors do not march, this pixel is removed from the object rough depth map. If there are any pixel of the precise object mask that cannot find a correspondence from the object rough mask, its depth value is copied or interpolated from the nearest pixels in the depth map that have valid depth values.
  • object rough depth maps are available only at key frames, detail object depth maps can be generated for all frames using interpolation. The process can be used when precise object masks are available for all frames. At key frames, detail object depth maps are generated directly from object rough depth maps using the “masking” method. For any in-between frame, a new rough depth map is interpolated from the rough depth maps of adjacent key frames, and then “masked” by the precise object mask of the same frame to produce a new detail object depth map.
  • a dissolve scene is a transition between two different shots whereby the first gradually fades out while the second gradually fades in.
  • the two scenes are treated separately, resulting in two object depth map sequences: one defining the depth of the first scene and the other defining the depth of the second scene.
  • those two object depth maps are dissolved into a single depth map for every frame such that the resulting object depth maps are temporally consistent with the before and after scenes.
  • two different depth dissolving schemes are used.
  • the first scheme is to gradually increase the depth values of both depth map sequences so that both are perceived as moving away from the audience in the first half of the dissolve until they are merged in the distance.
  • the depth values are gradually decreased so that scenes are moving forward until it matches the depth of the second scene.
  • the second scheme is to gradually merge the depth maps linearly in the temporal domain.
  • a foreground object when a foreground object occludes a background object, their object masks overlap each other.
  • the foreground object usually determines the color information in overlap regions unless it is a transparent or a reflective object.
  • part of the background object may have no direct color information. It is possible that the missing part of the background object is revealed in other frames of the sequence when the foreground object moves away.
  • the missing part of the object in an image frame can be reconstructed by tracking the exact motion of the object, registering corresponding features between the current image frame to other frames, and copying corresponding pixels from other frames. With the availability of precise object masks, tracking object motion becomes easier and very reliable.
  • the object reconstruction module 906 fills the missing part of an occluded object with as much information as can be recovered.
  • the process also reduces or completely fills the occlusion regions when objects are computed to new stereoscopic 3D images. For an object that represents image background, the process will reconstruct a “clean plate” or “partially clean plate” if a sufficient number of missing pixels can be recovered from frames.
  • Objects may need to be scaled to compensate for “miniaturizing” or its reverse phenomenon, discussed in earlier sections.
  • the scaling factor for an object is calculated based on perceived size of the object in a theater that is estimated using depth cues from object depth maps.
  • the scaling factor may vary from frame to frame if the object moves towards or away from the camera in the image shot. Both the precise object masks and object depth map of the same object may be scaled by the same scaling factor.
  • the 3D images of an object consist of a left-eye image sequence and a right-eye image sequence of the same object.
  • the left-eye is calculated from the point of view of a left camera
  • the right-eye is calculated from the point of view of a right camera.
  • the left and right cameras are virtual camera models that simulate the performance of a real 3D camera. There are a number of virtual camera models that can be selected, from the simplest pinhole camera model to more complex models with multiple focal nodal points and specific lens characteristics.
  • the 3D virtual camera model includes both left and right cameras spaced with a certain inter-ocular distance with a certain camera convergence setting.
  • Certain embodiments of the present invention provide that the default 3D camera model used for 3D motion pictures to be presented at large-format exhibition venues such as IMAX 3D theaters is modeled from a typical IMAX 15/70 3D camera, such as the IMAX 3D camera, with parallel optical axis and an inter-ocular distance of 2.5′′ in real world coordinates.
  • Certain embodiments of the present invention does not limit the camera viewing positions with which 3D images are computed. If the original camera position is kept as one of the 3D images, the original object images becomes one of the 3D views. Images computed from any new viewing position inevitably contain undefined regions or otherwise known as occlusion regions. If both left-eye and right-eye images are different from the original camera position, both will contain occlusion regions that need to be filled.
  • the default setting may provide that all layers use the same camera views to compute 3D images for all objects, although different camera views can be used for different layers and even for different objects.
  • the left and right images of all objects computed from all layers are combined to form full-frame 3D images based on object depth values.
  • the resulting full-frame 3D images are usually unfinished due to the existence of occlusion regions that need to be filled.
  • Object depth maps from all layers are also combined into full-frame depth maps which describes precise depth values at the scene level.
  • the resulting full-frame depth maps are one of type of IDC that can be saved as RDR. The full depth maps will still be valid if source image data has color changes.
  • Scene finishing 416 is the final stage of some embodiments of the present invention and may be module such as a single processing module.
  • One objective of the scene finishing module 416 is to fill occlusion regions remaining in the newly created 3D images with proper occlusion cues.
  • the occlusion regions in the 3D images are marked with a distinct color that can easily be identified and extracted as separate occlusion masks.
  • FIG. 13A shows an example of a right-eye image 1302 created with occlusion regions marked with hatches.
  • FIG. 14 shows one embodiment of a flow diagram of the scene finishing module 416 . It has a multi-mode structure similar to that of the object mask generation module or the object depth modeling module. At least one of input image data sequences 400 , 3D images with occlusion regions 1424 , and motion cues 424 are received. Similar to the other two modules, an operation mode analyzer 1402 selects appropriate operation methods for each individual occlusion region. Five operational methods are deployed in FIG. 14 , including one manual mode 1404 , three semi-automated modes 1406 , and one automated mode 1408 .
  • the manual mode 1404 may include cloning 1410 .
  • the semi-automated mode 1406 may include texture synthesis 1412 , in-painting 1414 , and feature-guided pixel recovery 1416 .
  • the automated mode 1408 may include a pixel recovery mode 1418 .
  • the operation mode analyzer 1402 selects a set of appropriate methods for each occlusion region by analyzing the motion and local characteristics of the surrounding region. It can be implemented as a user-guided interactive operation. The operation mode analyzer 1402 selects a set of methods based on the following.
  • the automated pixel recovery method 1418 relies on the existence of smooth relative motion between foreground and background objects so that pixels that are visible in other frames can be reliably tracked.
  • the feature-guided pixel recovery method 1416 relies on user-guided feature tracking, so that it is less restrictive than the automated mode and can be applied to a broader range of scenes. Both methods may require the availability of motion cues 424 .
  • Both the texture synthesis method 1412 and the in-painting method 1414 generate “fake” pixels by synthesizing the properties of neighborhood pixels. They are most suitable for those occlusion regions with absolutely no information. Each method is more suitable for a certain types of occlusion regions. For example, texture synthesis 1412 may work better for occlusion regions surrounded by organic texture, while the in-painting method 1414 may be more suitable for occlusion regions surrounded by more structured texture.
  • Cloning 1410 is a manual method for filling occlusion regions where other methods fail to produce satisfactory results, usually for small occlusion regions.
  • the operation mode analyzer 1402 may select more than one method for an occlusion region. In that case, the multiple results are evaluated through a voting process in block 1420 and the best result is temporal filtered in bock 1422 and the output is converted 3D image sequences 1426 .
  • the pixel recovery method 1418 may use similar methods as the object reconstruction 906 except that it works on the scene level with full-frame 3D images. This method is used when occlusion regions can be filled with information revealed from other frames.
  • a search is performed for missing pixels from the frames and the missing frames are copied to a target frame to fill the occlusion region.
  • the number of frames to be searched is considered as a temporal window, and the length of the temporal window increases as the object motion slows down.
  • the matching pixels are analyzed and may be scaled or even warped to match the scale change caused by motion.
  • the scaling factor and/or the warping transforms are determined by a feature tracking method that was disclosed in automated depth recovery.
  • the scaled and/or warped pixels are then copied to the target frame. This process may be repeated for all pixels in an occlusion region in every frame.
  • This method can be similar to automated pixel recovery 1418 except feature tracking can be guided by a trained user, similar to the method used in the feature-guided depth recovery.
  • the features are used for two purposes: (1) to guide the search for matching pixels; and (2) to determine the scaling and warping transforms.
  • the remaining steps in the method may be identical to the automated pixel recovery method 1418 .
  • Texture synthesis 1412 is an approach for synthesizing a new image that looks indistinguishable to a given texture sample.
  • the detailed methods of texture synthesis 1412 are well known to those skilled in the art.
  • the present invention discloses that the methods of texture synthesis 1412 are used to synthesize the texture of regions surrounding an occlusion region in order to fill the occlusion region, resulting no visible artifacts.
  • the set of texture samples, called patches can be selected by a user near the occlusion region. The user can also adjust parameters of a texture synthesis algorithm in order to achieve desirable results. This method works well for regions with natural texture such as trees, leaves and for repetitive textures.
  • Certain embodiments of the present invention also includes a method of maintaining both spatial and temporal consistency of synthesized texture results.
  • an energy function is defined that includes terms to measure local spatial smoothness of each patch with its neighbor areas including the edge pixels around the occlusion borders.
  • the energy function also includes terms to measure temporal smoothness between the current frame and the neighbor frames for each texture patch.
  • the energy function is then minimized to generate the optimal synthesized texture resulting from a combination of synthesized texture patches.
  • This energy function based texture synthesis method improves both spatial and temporal smoothness of synthesized pixels.
  • In-painting 1414 is an alternative method of producing synthesized pixels for missing image regions. It may be particularly suitable for local regions with definite structures. The method is based on the propagating pixel information from the outside regions of an occlusion region. In-painting 1414 extends the structures of outside regions to the interior of the occlusion region based on partial differential equations analysis of local edges and structures. The methods of in-painting 1414 can be used to fill occlusion regions of certain images. A semi-automated implementation of the in-painting method 1414 may be used to maintain temporal consistency. In this method, the occlusion region collected from multiple frames is treated as a volumetric data, and so are the collected regions surrounding the occlusion region.
  • a user defines extension of the structure of outside regions into an occlusion region by drawing some connecting lines and curves for every frame. The iterations are finished until all pixels in the regions are filled. By treating images as volumetric data, the temporal consistency can be maintained.
  • Cloning 1410 The functions of cloning 1410 and brush painting are provided in most graphical processing software tools. Cloning 1410 may be used instead of brush painting because brush painting is not color-independent. In cloning 1410 , pixels to be copied to an occlusion region are selected from other parts of the images. Because cloning can be saved as a vector form in a render script, it can be repeated regardless of source image data color changes.
  • occlusion cues The color information used to fill occlusion regions is referred to as occlusion cues which can be saved as part of RDR.
  • occlusion cues When occlusion cues are created by more than one method, the best result is selected based on certain criteria. One criterion is to maintain temporal consistency, followed by spatial smoothness. This process may be performed by visual inspection by a human operator.
  • FIG. 13B shows an example of finished right-eye image 1304 with occlusion regions filled with proper information.
  • Temporal filtering 1422 is the last step to further reduce temporal inconsistency of the filled occlusion regions.
  • the temporal filter may be similar to those deployed for the temporal analysis and enhancement stage 402 . Applying temporal filtering can improve the quality of the resulting converted 3D image sequences 1426 .

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Processing Or Creating Images (AREA)
  • Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
US12/299,909 2006-06-23 2007-06-25 Methods and systems for converting 2D motion pictures for stereoscopic 3D exhibition Active 2030-07-15 US8411931B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US12/299,909 US8411931B2 (en) 2006-06-23 2007-06-25 Methods and systems for converting 2D motion pictures for stereoscopic 3D exhibition

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US81627206P 2006-06-23 2006-06-23
US12/299,909 US8411931B2 (en) 2006-06-23 2007-06-25 Methods and systems for converting 2D motion pictures for stereoscopic 3D exhibition
PCT/IB2007/001726 WO2007148219A2 (fr) 2006-06-23 2007-06-25 Procédés et systèmes de conversion d'images cinématographiques 2d pour une représentation stéréoscopique 3d

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
PCT/IB2007/001726 A-371-Of-International WO2007148219A2 (fr) 2006-06-23 2007-06-25 Procédés et systèmes de conversion d'images cinématographiques 2d pour une représentation stéréoscopique 3d

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US13/714,503 Division US9282313B2 (en) 2006-06-23 2012-12-14 Methods and systems for converting 2D motion pictures for stereoscopic 3D exhibition

Publications (2)

Publication Number Publication Date
US20090116732A1 US20090116732A1 (en) 2009-05-07
US8411931B2 true US8411931B2 (en) 2013-04-02

Family

ID=38833823

Family Applications (2)

Application Number Title Priority Date Filing Date
US12/299,909 Active 2030-07-15 US8411931B2 (en) 2006-06-23 2007-06-25 Methods and systems for converting 2D motion pictures for stereoscopic 3D exhibition
US13/714,503 Active US9282313B2 (en) 2006-06-23 2012-12-14 Methods and systems for converting 2D motion pictures for stereoscopic 3D exhibition

Family Applications After (1)

Application Number Title Priority Date Filing Date
US13/714,503 Active US9282313B2 (en) 2006-06-23 2012-12-14 Methods and systems for converting 2D motion pictures for stereoscopic 3D exhibition

Country Status (5)

Country Link
US (2) US8411931B2 (fr)
EP (2) EP2160037A3 (fr)
CN (2) CN102685533B (fr)
CA (2) CA2884702C (fr)
WO (1) WO2007148219A2 (fr)

Cited By (62)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100231593A1 (en) * 2006-01-27 2010-09-16 Samuel Zhou Methods and systems for digitally re-mastering of 2d and 3d motion pictures for exhibition with enhanced visual quality
US20110243384A1 (en) * 2010-03-30 2011-10-06 Fujifilm Corporation Image processing apparatus and method and program
US20120075430A1 (en) * 2010-09-27 2012-03-29 Hal Laboratory Inc. Computer-readable storage medium, information processing apparatus, information processing system, and information processing method
US20120188234A1 (en) * 2011-01-20 2012-07-26 University Of Southern California Image processing apparatus and method
US20120218382A1 (en) * 2010-08-02 2012-08-30 Ron Zass Multiclass clustering with side information from multiple sources and the application of converting 2d video to 3d
US20120235988A1 (en) * 2010-09-10 2012-09-20 Dd3D, Inc. Systems and methods for converting two-dimensional images into three-dimensional images
US20130038600A1 (en) * 2011-08-12 2013-02-14 Himax Technologies Limited System and Method of Processing 3D Stereoscopic Image
US20130063549A1 (en) * 2011-09-09 2013-03-14 Lars Schnyder Systems and methods for converting video
US20130069932A1 (en) * 2011-09-15 2013-03-21 Broadcom Corporation Adjustable depth layers for three-dimensional images
US20130187910A1 (en) * 2012-01-25 2013-07-25 Lumenco, Llc Conversion of a digital stereo image into multiple views with parallax for 3d viewing without glasses
US8520975B2 (en) * 2009-10-30 2013-08-27 Adobe Systems Incorporated Methods and apparatus for chatter reduction in video object segmentation using optical flow assisted gaussholding
US20130230136A1 (en) * 2011-08-25 2013-09-05 Toshiba Medical Systems Corporation Medical image display apparatus and x-ray diagnosis apparatus
US20130257851A1 (en) * 2012-04-01 2013-10-03 Chao-Hua Lee Pipeline web-based process for 3d animation
US20130286010A1 (en) * 2011-01-30 2013-10-31 Nokia Corporation Method, Apparatus and Computer Program Product for Three-Dimensional Stereo Display
US20130321408A1 (en) * 2009-09-30 2013-12-05 Disney Enterprises, Inc. Method and system for utilizing pre-existing image layers of a two-dimensional image to create a stereoscopic image
US20140009462A1 (en) * 2012-04-17 2014-01-09 3Dmedia Corporation Systems and methods for improving overall quality of three-dimensional content by altering parallax budget or compensating for moving objects
US8644596B1 (en) * 2012-06-19 2014-02-04 Google Inc. Conversion of monoscopic visual content using image-depth database
US20140035905A1 (en) * 2012-07-31 2014-02-06 Samsung Electronics Co., Ltd. Method for converting 2-dimensional images into 3-dimensional images and display apparatus thereof
US20140118344A1 (en) * 2011-06-10 2014-05-01 Lg Electronics Inc. Method for processing image and apparatus for processing image
US8780183B2 (en) 2010-06-11 2014-07-15 Nintendo Co., Ltd. Computer-readable storage medium, image display apparatus, image display system, and image display method
US20140218358A1 (en) * 2011-12-01 2014-08-07 Lightcraft Technology, Llc Automatic tracking matte system
US8854356B2 (en) 2010-09-28 2014-10-07 Nintendo Co., Ltd. Storage medium having stored therein image processing program, image processing apparatus, image processing system, and image processing method
US20150042772A1 (en) * 2013-08-06 2015-02-12 Samsung Electronics Co., Ltd. Display apparatus and control method for providing a 3d image
US9042636B2 (en) 2009-12-31 2015-05-26 Disney Enterprises, Inc. Apparatus and method for indicating depth of one or more pixels of a stereoscopic 3-D image comprised from a plurality of 2-D layers
US9053562B1 (en) * 2010-06-24 2015-06-09 Gregory S. Rabin Two dimensional to three dimensional moving image converter
US20150170404A1 (en) * 2013-12-16 2015-06-18 Huawei Technologies Co., Ltd. Virtual View Generating Method and Apparatus
US20150178980A1 (en) * 2012-07-20 2015-06-25 Carl Zeiss Ag Multifocal representation device and multifocal representation method for the three-dimensional representation of an object
US9076212B2 (en) 2006-05-19 2015-07-07 The Queen's Medical Center Motion tracking system for real time adaptive imaging and spectroscopy
US20150229903A1 (en) * 2014-02-11 2015-08-13 Electronics And Telecommunications Research Institute Method and apparatus for converting 2d video image into 3d video image
US9111350B1 (en) 2012-02-10 2015-08-18 Google Inc. Conversion of monoscopic visual content to stereoscopic 3D
US9250745B2 (en) 2011-01-18 2016-02-02 Hewlett-Packard Development Company, L.P. Determine the characteristics of an input relative to a projected image
US9282313B2 (en) 2006-06-23 2016-03-08 Imax Corporation Methods and systems for converting 2D motion pictures for stereoscopic 3D exhibition
US9282319B2 (en) 2010-06-02 2016-03-08 Nintendo Co., Ltd. Image display system, image display apparatus, and image display method
WO2016036352A1 (fr) * 2014-09-03 2016-03-10 Hewlett-Packard Development Company, L.P. Présentation d'une image numérique d'un objet
US9305365B2 (en) 2013-01-24 2016-04-05 Kineticor, Inc. Systems, devices, and methods for tracking moving targets
US20160191905A1 (en) * 2008-03-26 2016-06-30 Ricoh Company, Ltd. Adaptive Image Acquisition and Display Using Multi-focal Display
US20160261845A1 (en) * 2015-03-04 2016-09-08 Dolby Laboratories Licensing Corporation Coherent Motion Estimation for Stereoscopic Video
US9471988B2 (en) 2011-11-02 2016-10-18 Google Inc. Depth-map generation for an input image using an example approximate depth-map associated with an example similar image
US9606209B2 (en) 2011-08-26 2017-03-28 Kineticor, Inc. Methods, systems, and devices for intra-scan motion correction
US9661307B1 (en) 2011-11-15 2017-05-23 Google Inc. Depth map generation using motion cues for conversion of monoscopic visual content to stereoscopic 3D
US9674498B1 (en) 2013-03-15 2017-06-06 Google Inc. Detecting suitability for converting monoscopic visual content to stereoscopic 3D
US9717461B2 (en) 2013-01-24 2017-08-01 Kineticor, Inc. Systems, devices, and methods for tracking and compensating for patient motion during a medical imaging scan
US9734589B2 (en) 2014-07-23 2017-08-15 Kineticor, Inc. Systems, devices, and methods for tracking and compensating for patient motion during a medical imaging scan
US9782141B2 (en) 2013-02-01 2017-10-10 Kineticor, Inc. Motion tracking system for real time adaptive motion compensation in biomedical imaging
US9795882B1 (en) 2010-06-24 2017-10-24 Gregory S. Rabin Interactive system and method
US9842875B2 (en) 2013-08-05 2017-12-12 Apple Inc. Image sensor with buried light shield and vertical gate
US9866826B2 (en) 2014-11-25 2018-01-09 Ricoh Company, Ltd. Content-adaptive multi-focal display
US9864205B2 (en) 2014-11-25 2018-01-09 Ricoh Company, Ltd. Multifocal display
US9924240B2 (en) 2015-05-01 2018-03-20 Google Llc Systems and methods for interactive video generation and rendering
US9943247B2 (en) 2015-07-28 2018-04-17 The University Of Hawai'i Systems, devices, and methods for detecting false movements for motion correction during a medical imaging scan
US10004462B2 (en) 2014-03-24 2018-06-26 Kineticor, Inc. Systems, methods, and devices for removing prospective motion correction from medical imaging scans
US10122992B2 (en) 2014-05-22 2018-11-06 Disney Enterprises, Inc. Parallax based monoscopic rendering
US10327708B2 (en) 2013-01-24 2019-06-25 Kineticor, Inc. Systems, devices, and methods for tracking and compensating for patient motion during a medical imaging scan
US10462365B1 (en) * 2013-03-14 2019-10-29 Hrl Laboratories, Llc Low power surveillance system
US20200019861A1 (en) * 2019-08-26 2020-01-16 Lg Electronics Inc. Method for controlling cook based on artificial intelligent and intelligent device
US10716515B2 (en) 2015-11-23 2020-07-21 Kineticor, Inc. Systems, devices, and methods for tracking and compensating for patient motion during a medical imaging scan
RU2752421C1 (ru) * 2018-05-23 2021-07-28 Сцивита Медикал Текнолоджи Ко., Лтд. Способ и устройство обработки изображений и система трехмерной визуализации
US11395947B2 (en) * 2016-01-28 2022-07-26 Nippon Telegraph And Telephone Corporation Virtual environment construction apparatus, video presentation apparatus, model learning apparatus, optimal depth decision apparatus, methods for the same, and program
US11568520B2 (en) * 2017-01-24 2023-01-31 Leica Geosystems Ag Method and device for inpainting of colourised three-dimensional point clouds
US20230128024A1 (en) * 2021-10-26 2023-04-27 Koninklijke Philips N.V. Automated virtual (ar/vr) education content adjustment based on gaze and learner attention tracking
US11670039B2 (en) 2019-03-04 2023-06-06 Dolby Laboratories Licensing Corporation Temporal hole filling for depth image based video rendering
US20230179736A1 (en) * 2021-12-08 2023-06-08 immerVR GmbH Device and method

Families Citing this family (275)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8396328B2 (en) 2001-05-04 2013-03-12 Legend3D, Inc. Minimal artifact image sequence depth enhancement system and method
US7907793B1 (en) 2001-05-04 2011-03-15 Legend Films Inc. Image sequence depth enhancement system and method
US8401336B2 (en) 2001-05-04 2013-03-19 Legend3D, Inc. System and method for rapid image sequence depth enhancement with augmented computer-generated elements
US9286941B2 (en) 2001-05-04 2016-03-15 Legend3D, Inc. Image sequence enhancement and motion picture project management system
CN1650622B (zh) 2002-03-13 2012-09-05 图象公司 用于数字重新灌录或修改电影或其他图像序列数据的系统和方法
KR101183000B1 (ko) 2004-07-30 2012-09-18 익스트림 리얼리티 엘티디. 이미지 프로세싱을 기반으로 한 3ｄ 공간 차원용 시스템 및방법
US8872899B2 (en) * 2004-07-30 2014-10-28 Extreme Reality Ltd. Method circuit and system for human to machine interfacing by hand gestures
US8681100B2 (en) 2004-07-30 2014-03-25 Extreme Realty Ltd. Apparatus system and method for human-machine-interface
US7542034B2 (en) 2004-09-23 2009-06-02 Conversion Works, Inc. System and method for processing video images
JP4282645B2 (ja) * 2005-08-31 2009-06-24 株式会社東芝 撮像装置
WO2011033519A1 (fr) 2009-09-21 2011-03-24 Extreme Reality Ltd. Procédés, circuits, appareils et systèmes pour un interfaçage humain/machine avec un appareil électronique
US9046962B2 (en) 2005-10-31 2015-06-02 Extreme Reality Ltd. Methods, systems, apparatuses, circuits and associated computer executable code for detecting motion, position and/or orientation of objects within a defined spatial region
US20070285554A1 (en) 2005-10-31 2007-12-13 Dor Givon Apparatus method and system for imaging
US8982181B2 (en) * 2006-06-13 2015-03-17 Newbery Revocable Trust Indenture Digital stereo photographic system
JP4818053B2 (ja) * 2006-10-10 2011-11-16 株式会社東芝 高解像度化装置および方法
US8351646B2 (en) * 2006-12-21 2013-01-08 Honda Motor Co., Ltd. Human pose estimation and tracking using label assignment
JP5024067B2 (ja) * 2007-01-19 2012-09-12 コニカミノルタホールディングス株式会社 顔認証システム、方法及びプログラム
US8655052B2 (en) 2007-01-26 2014-02-18 Intellectual Discovery Co., Ltd. Methodology for 3D scene reconstruction from 2D image sequences
US20080226128A1 (en) * 2007-03-12 2008-09-18 Conversion Works, Inc. System and method for using feature tracking techniques for the generation of masks in the conversion of two-dimensional images to three-dimensional images
US8274530B2 (en) 2007-03-12 2012-09-25 Conversion Works, Inc. Systems and methods for filling occluded information for 2-D to 3-D conversion
US20080228449A1 (en) * 2007-03-12 2008-09-18 Conversion Works, Inc. Systems and methods for 2-d to 3-d conversion using depth access segments to define an object
US20110043540A1 (en) * 2007-03-23 2011-02-24 James Arthur Fancher System and method for region classification of 2d images for 2d-to-3d conversion
US8269822B2 (en) * 2007-04-03 2012-09-18 Sony Computer Entertainment America, LLC Display viewing system and methods for optimizing display view based on active tracking
CN101682794B (zh) * 2007-05-11 2012-05-02 皇家飞利浦电子股份有限公司 用于处理深度相关信息的方法、装置和系统
US20080295035A1 (en) * 2007-05-25 2008-11-27 Nokia Corporation Projection of visual elements and graphical elements in a 3D UI
US20090003712A1 (en) * 2007-06-28 2009-01-01 Microsoft Corporation Video Collage Presentation
US8395660B2 (en) * 2007-12-13 2013-03-12 Apple Inc. Three-dimensional movie browser or editor
EP2291825B1 (fr) * 2008-05-28 2013-09-25 Thomson Licensing Système et procédé d extraction de profondeur d images avec prédiction directe et inverse de profondeur
KR100957129B1 (ko) * 2008-06-12 2010-05-11 성영석 영상 변환 방법 및 장치
US20090315981A1 (en) * 2008-06-24 2009-12-24 Samsung Electronics Co., Ltd. Image processing method and apparatus
KR20100002032A (ko) * 2008-06-24 2010-01-06 삼성전자주식회사 영상 생성 방법, 영상 처리 방법, 및 그 장치
TW201005673A (en) * 2008-07-18 2010-02-01 Ind Tech Res Inst Example-based two-dimensional to three-dimensional image conversion method, computer readable medium therefor, and system
JP5347717B2 (ja) * 2008-08-06 2013-11-20 ソニー株式会社 画像処理装置、および画像処理方法、並びにプログラム
US20110163948A1 (en) * 2008-09-04 2011-07-07 Dor Givon Method system and software for providing image sensor based human machine interfacing
JP4636149B2 (ja) * 2008-09-09 2011-02-23 ソニー株式会社 画像データ解析装置、および画像データ解析方法、並びにプログラム
US20100079653A1 (en) * 2008-09-26 2010-04-01 Apple Inc. Portable computing system with a secondary image output
US7881603B2 (en) 2008-09-26 2011-02-01 Apple Inc. Dichroic aperture for electronic imaging device
US8610726B2 (en) * 2008-09-26 2013-12-17 Apple Inc. Computer systems and methods with projected display
EP2180449A1 (fr) * 2008-10-21 2010-04-28 Koninklijke Philips Electronics N.V. Procédé et dispositif pour la fourniture d'un modèle de profondeur stratifié
WO2010050692A2 (fr) 2008-10-28 2010-05-06 Samsung Electronics Co,. Ltd. Procédé et appareil de traitement d'image
US8345956B2 (en) * 2008-11-03 2013-01-01 Microsoft Corporation Converting 2D video into stereo video
KR101547151B1 (ko) * 2008-12-26 2015-08-25 삼성전자주식회사 영상 처리 방법 및 장치
MY152817A (en) * 2009-02-17 2014-11-28 Samsung Electronics Co Ltd Graphic image processing method and apparatus
US8395709B2 (en) * 2009-03-04 2013-03-12 ATI Technology ULC 3D video processing
JP2010245853A (ja) * 2009-04-07 2010-10-28 Hitachi Ltd 動画インデクシング方法及び動画再生装置
CN102450010A (zh) 2009-04-20 2012-05-09 杜比实验室特许公司 定向内插和数据后处理
US8462987B2 (en) * 2009-06-23 2013-06-11 Ut-Battelle, Llc Detecting multiple moving objects in crowded environments with coherent motion regions
CN101945295B (zh) * 2009-07-06 2014-12-24 三星电子株式会社 生成深度图的方法和设备
US8773441B2 (en) * 2009-07-10 2014-07-08 Pixar System and method for conforming an animated camera to an editorial cut
US8547374B1 (en) 2009-07-24 2013-10-01 Lockheed Martin Corporation Detection and reconstruction of 3D objects with passive imaging sensors
WO2011014419A1 (fr) * 2009-07-31 2011-02-03 3Dmedia Corporation Procédés, systèmes et supports de mémorisation lisibles par ordinateur pour création d'images tridimensionnelles (3d) d'une scène
US9380292B2 (en) 2009-07-31 2016-06-28 3Dmedia Corporation Methods, systems, and computer-readable storage media for generating three-dimensional (3D) images of a scene
WO2011014420A1 (fr) * 2009-07-31 2011-02-03 3Dmedia Corporation Procédés, systèmes et supports de mémorisation lisibles par ordinateur pour la sélection de positions de capture d'image dans le but de générer des images tridimensionnelles (3d)
US20110025830A1 (en) 2009-07-31 2011-02-03 3Dmedia Corporation Methods, systems, and computer-readable storage media for generating stereoscopic content via depth map creation
WO2011017310A1 (fr) * 2009-08-04 2011-02-10 Shenzhen Tcl New Technology Ltd. Systèmes et procédés de génération de vidéo tridimensionnelle
WO2011017308A1 (fr) * 2009-08-04 2011-02-10 Shenzhen Tcl New Technology Ltd. Systèmes et procédés de génération de vidéo tridimensionnelle
EP2293586A1 (fr) * 2009-08-04 2011-03-09 Samsung Electronics Co., Ltd. Procédé et système pour transformer un contenu stéréo
US8629899B2 (en) * 2009-08-06 2014-01-14 Qualcomm Incorporated Transforming video data in accordance with human visual system feedback metrics
CN101631256B (zh) * 2009-08-13 2011-02-09 浙江大学 用于三维电视系统中2d视频到3d视频的转换方法
JP5299173B2 (ja) * 2009-08-26 2013-09-25 ソニー株式会社 画像処理装置および画像処理方法、並びにプログラム
US8254760B2 (en) 2009-08-28 2012-08-28 Apple Inc. Pixel analysis and frame alignment for background frames
US9294751B2 (en) 2009-09-09 2016-03-22 Mattel, Inc. Method and system for disparity adjustment during stereoscopic zoom
CN102428501A (zh) * 2009-09-18 2012-04-25 株式会社东芝 图像处理装置
US8878779B2 (en) 2009-09-21 2014-11-04 Extreme Reality Ltd. Methods circuits device systems and associated computer executable code for facilitating interfacing with a computing platform display screen
US8659592B2 (en) * 2009-09-24 2014-02-25 Shenzhen Tcl New Technology Ltd 2D to 3D video conversion
JP2011070415A (ja) * 2009-09-25 2011-04-07 Fujifilm Corp 画像処理装置及び画像処理方法
US9083956B2 (en) * 2009-09-28 2015-07-14 Samsung Electronics Co., Ltd. System and method for creating 3D video
US8619128B2 (en) * 2009-09-30 2013-12-31 Apple Inc. Systems and methods for an imaging system using multiple image sensors
US9030469B2 (en) * 2009-11-18 2015-05-12 Industrial Technology Research Institute Method for generating depth maps from monocular images and systems using the same
GB2475730A (en) * 2009-11-27 2011-06-01 Sony Corp Transformation of occluding objects in 2D to 3D image generation
US8817078B2 (en) * 2009-11-30 2014-08-26 Disney Enterprises, Inc. Augmented reality videogame broadcast programming
KR101633620B1 (ko) * 2010-01-04 2016-06-27 삼성전자 주식회사 영상 기반의 위치 인식을 위한 특징점 등록 장치 및 그 방법
US8885022B2 (en) * 2010-01-04 2014-11-11 Disney Enterprises, Inc. Virtual camera control using motion control systems for augmented reality
US8803951B2 (en) * 2010-01-04 2014-08-12 Disney Enterprises, Inc. Video capture system control using virtual cameras for augmented reality
US8687044B2 (en) * 2010-02-02 2014-04-01 Microsoft Corporation Depth camera compatibility
US8619122B2 (en) * 2010-02-02 2013-12-31 Microsoft Corporation Depth camera compatibility
WO2011097306A1 (fr) * 2010-02-04 2011-08-11 Sony Corporation Conversion d'image 2d en image 3d basée sur un contenu d'image
US20110222757A1 (en) 2010-03-10 2011-09-15 Gbo 3D Technology Pte. Ltd. Systems and methods for 2D image and spatial data capture for 3D stereo imaging
CN102835119B (zh) * 2010-04-01 2016-02-03 英特尔公司 支持自动立体显示器上的实时3d图像渲染的多核心处理器
US11711592B2 (en) 2010-04-06 2023-07-25 Comcast Cable Communications, Llc Distribution of multiple signals of video content independently over a network
US10448083B2 (en) 2010-04-06 2019-10-15 Comcast Cable Communications, Llc Streaming and rendering of 3-dimensional video
US9135514B2 (en) * 2010-05-21 2015-09-15 Qualcomm Incorporated Real time tracking/detection of multiple targets
US8698829B2 (en) 2010-05-28 2014-04-15 Microsoft Corporation Discrete element texture synthesis
WO2012002018A1 (fr) * 2010-06-30 2012-01-05 富士フイルム株式会社 Dispositif de lecture d'image stéréoscopique, son procédé d'ajustement de parallaxe, programme d'ajustement de parallaxe, et dispositif de capture d'image
KR101731343B1 (ko) * 2010-07-14 2017-04-28 엘지전자 주식회사 이동 단말기 및 그 제어방법
US9344701B2 (en) 2010-07-23 2016-05-17 3Dmedia Corporation Methods, systems, and computer-readable storage media for identifying a rough depth map in a scene and for determining a stereo-base distance for three-dimensional (3D) content creation
JP5238767B2 (ja) * 2010-07-26 2013-07-17 株式会社東芝 視差画像生成方法及び装置
US8428342B2 (en) 2010-08-12 2013-04-23 At&T Intellectual Property I, L.P. Apparatus and method for providing three dimensional media content
US8649592B2 (en) 2010-08-30 2014-02-11 University Of Illinois At Urbana-Champaign System for background subtraction with 3D camera
US8885023B2 (en) * 2010-09-01 2014-11-11 Disney Enterprises, Inc. System and method for virtual camera control using motion control systems for augmented three dimensional reality
US8655093B2 (en) * 2010-09-02 2014-02-18 Edge 3 Technologies, Inc. Method and apparatus for performing segmentation of an image
KR20120023431A (ko) * 2010-09-03 2012-03-13 삼성전자주식회사 깊이 조정이 가능한 2차원/3차원 영상 변환 방법 및 그 장치
JP5451894B2 (ja) * 2010-09-22 2014-03-26 富士フイルム株式会社 立体撮像装置およびシェーディング補正方法
WO2012037685A1 (fr) * 2010-09-22 2012-03-29 Berfort Management Inc. Génération de contenu stéréoscopique 3d à partir de contenu vidéo monoscopique
US8538132B2 (en) 2010-09-24 2013-09-17 Apple Inc. Component concentricity
KR20120032321A (ko) * 2010-09-28 2012-04-05 삼성전자주식회사 디스플레이 장치 및 이의 영상 처리 방법
TWI532009B (zh) * 2010-10-14 2016-05-01 華晶科技股份有限公司 產生淺景深影像的方法及裝置
JP5704885B2 (ja) * 2010-10-25 2015-04-22 オリンパスイメージング株式会社 撮影機器、撮影方法及び撮影制御プログラム
US9185388B2 (en) 2010-11-03 2015-11-10 3Dmedia Corporation Methods, systems, and computer program products for creating three-dimensional video sequences
JP5132754B2 (ja) * 2010-11-10 2013-01-30 株式会社東芝 画像処理装置、方法およびそのプログラム
US20120121163A1 (en) * 2010-11-15 2012-05-17 Samsung Electronics Co., Ltd. 3d display apparatus and method for extracting depth of 3d image thereof
US9123115B2 (en) 2010-11-23 2015-09-01 Qualcomm Incorporated Depth estimation based on global motion and optical flow
US9171372B2 (en) * 2010-11-23 2015-10-27 Qualcomm Incorporated Depth estimation based on global motion
KR20120059367A (ko) * 2010-11-30 2012-06-08 삼성전자주식회사 에너지값을 이용한 이미지 처리 장치와 그 이미지 처리 방법 및 디스플레이 방법
US9230315B2 (en) * 2010-12-08 2016-01-05 Thomson Licensing Complexity estimation of a 2D/3D conversion
CN102036090B (zh) * 2010-12-16 2012-05-02 清华大学 一种用于数字电视终端的电视信号转换装置
CN102014293B (zh) * 2010-12-20 2012-08-22 清华大学 平面视频的立体渲染方法
US8773427B2 (en) 2010-12-22 2014-07-08 Sony Corporation Method and apparatus for multiview image generation using depth map information
AU2011348121B2 (en) * 2010-12-22 2016-08-18 Legend 3D, Inc. System and method for minimal iteration workflow for image sequence depth enhancement
CN102567381B (zh) * 2010-12-23 2017-02-08 腾讯科技（深圳）有限公司 一种保存图片信息的方法和装置
US8274552B2 (en) 2010-12-27 2012-09-25 3Dmedia Corporation Primary and auxiliary image capture devices for image processing and related methods
US10200671B2 (en) 2010-12-27 2019-02-05 3Dmedia Corporation Primary and auxiliary image capture devices for image processing and related methods
JP5698529B2 (ja) * 2010-12-29 2015-04-08 任天堂株式会社 表示制御プログラム、表示制御装置、表示制御システム、および表示制御方法
TWI469088B (zh) 2010-12-31 2015-01-11 Ind Tech Res Inst 前景深度地圖產生模組及其方法
US8786673B2 (en) * 2011-01-07 2014-07-22 Cyberlink Corp. Systems and methods for performing video conversion based on non-linear stretch information
US9204123B2 (en) * 2011-01-14 2015-12-01 Comcast Cable Communications, Llc Video content generation
KR20140030138A (ko) 2011-01-23 2014-03-11 익스트림 리얼리티 엘티디. 입체 화상 및 비디오를 생성하는 방법, 시스템, 장치 및 관련 프로세스 로직
US8730232B2 (en) * 2011-02-01 2014-05-20 Legend3D, Inc. Director-style based 2D to 3D movie conversion system and method
US9113130B2 (en) 2012-02-06 2015-08-18 Legend3D, Inc. Multi-stage production pipeline system
US9282321B2 (en) 2011-02-17 2016-03-08 Legend3D, Inc. 3D model multi-reviewer system
US9288476B2 (en) 2011-02-17 2016-03-15 Legend3D, Inc. System and method for real-time depth modification of stereo images of a virtual reality environment
US9407904B2 (en) 2013-05-01 2016-08-02 Legend3D, Inc. Method for creating 3D virtual reality from 2D images
US9241147B2 (en) 2013-05-01 2016-01-19 Legend3D, Inc. External depth map transformation method for conversion of two-dimensional images to stereoscopic images
US9483836B2 (en) 2011-02-28 2016-11-01 Sony Corporation Method and apparatus for real-time conversion of 2-dimensional content to 3-dimensional content
WO2012119188A1 (fr) * 2011-03-04 2012-09-13 Lbt Innovations Limited Procédé d'amélioration des résultats de classification d'un classifieur
JP2012221211A (ja) * 2011-04-08 2012-11-12 Nintendo Co Ltd 情報処理装置、情報処理プログラム、情報処理方法および情報処理システム
US9271035B2 (en) 2011-04-12 2016-02-23 Microsoft Technology Licensing, Llc Detecting key roles and their relationships from video
TW201248552A (en) * 2011-04-13 2012-12-01 Stereo Screen Systems Pty Ltd Method, system and apparatus for producing three dimensional (3D) imagery
CN102186092A (zh) * 2011-04-26 2011-09-14 深圳创维－Rgb电子有限公司 在3d显示模式下实现2d菜单显示的方法、系统及显示器
WO2012169174A1 (fr) * 2011-06-08 2012-12-13 パナソニック株式会社 Dispositif de traitement d'image et procédé de traitement d'image
EP2538388B1 (fr) 2011-06-20 2015-04-01 Alcatel Lucent Procédé et agencement pour la construction de modèles d'image
KR20130008746A (ko) 2011-07-13 2013-01-23 삼성전자주식회사 3ｄ 영상변환장치, 그 구현방법 및 그 저장매체
US9129411B2 (en) * 2011-07-21 2015-09-08 Luca Rossato Upsampling in a tiered signal quality hierarchy
KR101893601B1 (ko) 2011-07-22 2018-08-31 삼성전자 주식회사 디스플레이장치의 입력장치, 디스플레이 시스템 및 그 제어방법
CN102253819B (zh) * 2011-08-08 2013-11-27 深圳超多维光电子有限公司 一种基于浏览器的多媒体显示方法和系统
US8711141B2 (en) * 2011-08-28 2014-04-29 Arcsoft Hangzhou Co., Ltd. 3D image generating method, 3D animation generating method, and both 3D image generating module and 3D animation generating module thereof
KR20130026078A (ko) * 2011-09-05 2013-03-13 삼성전자주식회사 컨텐츠변환장치 및 컨텐츠변환방법
KR101859412B1 (ko) * 2011-09-05 2018-05-18 삼성전자 주식회사 컨텐츠변환장치 및 컨텐츠변환방법
KR101862543B1 (ko) * 2011-09-08 2018-07-06 삼성전자 주식회사 깊이정보 생성장치, 깊이정보 생성방법 및 그 저장매체
KR20130027727A (ko) * 2011-09-08 2013-03-18 삼성전자주식회사 깊이정보 생성장치, 깊이정보 생성방법 및 그 저장매체
US8970666B2 (en) * 2011-09-16 2015-03-03 Disney Enterprises, Inc. Low scale production system and method
US8995755B2 (en) 2011-09-30 2015-03-31 Cyberlink Corp. Two-dimensional to stereoscopic conversion systems and methods
AU2012318854B2 (en) 2011-10-05 2016-01-28 Bitanimate, Inc. Resolution enhanced 3D video rendering systems and methods
KR101841750B1 (ko) * 2011-10-11 2018-03-26 한국전자통신연구원 매칭 정보에 의한 입체 영상 보정 장치 및 그 방법
US9165401B1 (en) 2011-10-24 2015-10-20 Disney Enterprises, Inc. Multi-perspective stereoscopy from light fields
US9113043B1 (en) * 2011-10-24 2015-08-18 Disney Enterprises, Inc. Multi-perspective stereoscopy from light fields
JP5861114B2 (ja) * 2011-11-08 2016-02-16 パナソニックＩｐマネジメント株式会社 画像処理装置、及び画像処理方法
US9672609B1 (en) * 2011-11-11 2017-06-06 Edge 3 Technologies, Inc. Method and apparatus for improved depth-map estimation
JP5510431B2 (ja) * 2011-11-21 2014-06-04 株式会社デンソー 車両用表示装置
US8666119B1 (en) * 2011-11-29 2014-03-04 Lucasfilm Entertainment Company Ltd. Geometry tracking
CN103136781B (zh) 2011-11-30 2016-06-08 国际商业机器公司 用于生成三维虚拟场景的方法和系统
US20130141433A1 (en) * 2011-12-02 2013-06-06 Per Astrand Methods, Systems and Computer Program Products for Creating Three Dimensional Meshes from Two Dimensional Images
US9414048B2 (en) * 2011-12-09 2016-08-09 Microsoft Technology Licensing, Llc Automatic 2D-to-stereoscopic video conversion
US8897542B2 (en) * 2011-12-15 2014-11-25 Sony Corporation Depth map generation based on soft classification
KR101899324B1 (ko) * 2011-12-28 2018-09-18 삼성전자주식회사 3ｄ 입체 영상을 제공하는 디스플레이 장치 및 방법
US9137519B1 (en) * 2012-01-04 2015-09-15 Google Inc. Generation of a stereo video from a mono video
CN102521879A (zh) * 2012-01-06 2012-06-27 肖华 一种2d转3d的方法
JP6024110B2 (ja) * 2012-01-26 2016-11-09 ソニー株式会社 画像処理装置、画像処理方法、プログラム、端末装置及び画像処理システム
KR101953306B1 (ko) * 2012-02-08 2019-03-04 삼성전자주식회사 사용자 인터랙션에 기초하여 3ｄ 영상을 생성하기 위한 방법 및 장치
JP2013172190A (ja) * 2012-02-17 2013-09-02 Sony Corp 画像処理装置、および画像処理方法、並びにプログラム
KR101318552B1 (ko) * 2012-03-12 2013-10-16 가톨릭대학교 산학협력단 3차원 영상에 대한 인지 왜곡을 측정하는 측정 방법
KR101356544B1 (ko) * 2012-03-29 2014-02-19 한국과학기술원 3차원 입체영상 생성 방법 및 장치
KR20150007340A (ko) * 2012-05-22 2015-01-20 보르쥬 컴파운딩 상하이 캄파니 리미티드 낮은 뒤틀림을 갖는 성형 물품
US10134134B2 (en) 2012-05-24 2018-11-20 Qatar Foundation Method and system for creating depth signatures
KR101939628B1 (ko) * 2012-05-30 2019-01-17 삼성전자주식회사 모션 검출 방법 및 모션 검출기
JP5977591B2 (ja) * 2012-06-20 2016-08-24 オリンパス株式会社 画像処理装置及びそれを備えた撮像装置、画像処理方法、並びに画像処理プログラムを記録したコンピュータ読み取り可能な記録媒体
CN102761768A (zh) * 2012-06-28 2012-10-31 中兴通讯股份有限公司 一种实现立体成像的方法及装置
US9251578B2 (en) 2012-07-02 2016-02-02 Sony Corporation System and method for match move quality control
JP5673624B2 (ja) * 2012-07-24 2015-02-18 カシオ計算機株式会社 オブジェクト検索装置、方法、およびプログラム
RU2493602C1 (ru) * 2012-08-10 2013-09-20 Корпорация "САМСУНГ ЭЛЕКТРОНИКС Ко., Лтд." Способ и система для выделения ключевых кадров из видео последовательностей
CN102866887B (zh) * 2012-09-07 2015-03-25 深圳市至高通信技术发展有限公司 三维用户界面的实现方法及装置
US9736456B1 (en) * 2012-09-28 2017-08-15 Pixelworks, Inc. Two dimensional to three dimensional video conversion
JP2014078095A (ja) * 2012-10-10 2014-05-01 Sony Corp 画像処理装置、画像処理方法、及びプログラム
KR101428866B1 (ko) * 2012-10-26 2014-08-12 한국과학기술원 스테레오 영상의 깊이감 조절 장치 및 방법
US9098911B2 (en) * 2012-11-01 2015-08-04 Google Inc. Depth map generation from a monoscopic image based on combined depth cues
JP6102938B2 (ja) * 2012-11-06 2017-03-29 日本電気株式会社 画像処理装置、及び、画像処理方法
GB2499694B8 (en) 2012-11-09 2017-06-07 Sony Computer Entertainment Europe Ltd System and method of image reconstruction
US9007365B2 (en) 2012-11-27 2015-04-14 Legend3D, Inc. Line depth augmentation system and method for conversion of 2D images to 3D images
US9547937B2 (en) 2012-11-30 2017-01-17 Legend3D, Inc. Three-dimensional annotation system and method
US9299152B2 (en) * 2012-12-20 2016-03-29 Hong Kong Applied Science And Technology Research Institute Co., Ltd. Systems and methods for image depth map generation
CN105409215B (zh) 2013-01-11 2018-10-09 华为技术有限公司 深度预测模式选择方法及装置
US20150379720A1 (en) * 2013-01-31 2015-12-31 Threevolution Llc Methods for converting two-dimensional images into three-dimensional images
WO2014130019A1 (fr) 2013-02-20 2014-08-28 Intel Corporation Conversion automatique en temps réel d'images ou d'une vidéo en deux dimensions pour obtenir des images stéréo ou une vidéo en trois dimensions
US10371943B2 (en) * 2013-03-14 2019-08-06 Valve Corporation Method and system to control the focus depth of projected images
US9007404B2 (en) 2013-03-15 2015-04-14 Legend3D, Inc. Tilt-based look around effect image enhancement method
JP6089886B2 (ja) * 2013-03-29 2017-03-08 オムロン株式会社 領域分割方法および検査装置
KR20140122054A (ko) * 2013-04-09 2014-10-17 삼성전자주식회사 2차원 이미지를 3차원 이미지로 변환하는 3차원 이미지 변환 장치 및 그 제어 방법
US9438878B2 (en) 2013-05-01 2016-09-06 Legend3D, Inc. Method of converting 2D video to 3D video using 3D object models
CN103544727B (zh) * 2013-06-26 2017-02-15 Tcl集团股份有限公司 一种基于预测分支的场景渲染优化方法、系统和移动终端
EP2833637A1 (fr) 2013-07-31 2015-02-04 Thomson Licensing Procédé de traitement d'une image en cours d'une séquence d'images, programme informatique correspondant et dispositif de traitement
KR102114346B1 (ko) * 2013-08-30 2020-05-22 삼성전자주식회사 스테레오 컨버전스 제어 방법 및 이를 적용한 스테레오 이미지 프로세서
TWI570666B (zh) * 2013-11-15 2017-02-11 財團法人資訊工業策進會 電子裝置及其影片物體追蹤方法
CN103596014B (zh) * 2013-11-26 2016-12-07 天津大学 多视点分布式视频编码中视点间边信息的生成方法
WO2015089115A1 (fr) * 2013-12-09 2015-06-18 Nant Holdings Ip, Llc Classification d'objets à densité de caractéristiques, systèmes et procédés
US9979952B2 (en) * 2013-12-13 2018-05-22 Htc Corporation Method of creating a parallax video from a still image
US9087405B2 (en) * 2013-12-16 2015-07-21 Google Inc. Depth map generation using bokeh detection
JP6395423B2 (ja) * 2014-04-04 2018-09-26 キヤノン株式会社 画像処理装置、制御方法及びプログラム
CN104469440B (zh) * 2014-04-16 2018-10-30 成都理想境界科技有限公司 视频播放方法、视频播放器及对应的播放设备
GB2526838B (en) * 2014-06-04 2016-06-01 Imagination Tech Ltd Relightable texture for use in rendering an image
JP6490107B2 (ja) * 2014-06-10 2019-03-27 ビッタニメイト インコーポレイテッド 立体視ビデオズーミング
US9591290B2 (en) * 2014-06-10 2017-03-07 Bitanimate, Inc. Stereoscopic video generation
KR101939243B1 (ko) * 2014-06-10 2019-01-16 비트애니메이트 인코포레이티드 입체 깊이 조정 및 초점 조정
US9973745B2 (en) 2014-06-10 2018-05-15 Bitanimate, Inc. Stereoscopic focus point adjustment
US9674500B2 (en) * 2014-06-10 2017-06-06 Bitanimate, Inc. Stereoscopic depth adjustment
US9693042B2 (en) 2014-06-10 2017-06-27 Bitanimate, Inc. Foreground and background detection in a video
CA2961175A1 (fr) 2014-09-03 2016-03-10 Nextvr Inc. Procedes et appareils pour capturer, lire en continu et/ou lire en differe un contenu
KR102219456B1 (ko) * 2014-10-13 2021-02-25 삼성전자 주식회사 컨텐츠를 렌더링하는 방법 및 장치
US10726560B2 (en) 2014-10-31 2020-07-28 Fyusion, Inc. Real-time mobile device capture and generation of art-styled AR/VR content
US10275935B2 (en) 2014-10-31 2019-04-30 Fyusion, Inc. System and method for infinite synthetic image generation from multi-directional structured image array
US10719939B2 (en) 2014-10-31 2020-07-21 Fyusion, Inc. Real-time mobile device capture and generation of AR/VR content
US10726593B2 (en) 2015-09-22 2020-07-28 Fyusion, Inc. Artificially rendering images using viewpoint interpolation and extrapolation
US10262426B2 (en) 2014-10-31 2019-04-16 Fyusion, Inc. System and method for infinite smoothing of image sequences
US10176592B2 (en) 2014-10-31 2019-01-08 Fyusion, Inc. Multi-directional structured image array capture on a 2D graph
US10534525B1 (en) * 2014-12-09 2020-01-14 Amazon Technologies, Inc. Media editing system optimized for distributed computing systems
TWI549095B (zh) * 2014-12-22 2016-09-11 Bin-Feng Sun Real - time map of three - dimensional image generation method
CN104506828B (zh) * 2015-01-13 2017-10-17 中南大学 一种无有效重叠变结构的定点定向视频实时拼接方法
US10007849B2 (en) 2015-05-29 2018-06-26 Accenture Global Solutions Limited Predicting external events from digital video content
US11006095B2 (en) 2015-07-15 2021-05-11 Fyusion, Inc. Drone based capture of a multi-view interactive digital media
US10222932B2 (en) 2015-07-15 2019-03-05 Fyusion, Inc. Virtual reality environment based manipulation of multilayered multi-view interactive digital media representations
US11095869B2 (en) 2015-09-22 2021-08-17 Fyusion, Inc. System and method for generating combined embedded multi-view interactive digital media representations
US10242474B2 (en) 2015-07-15 2019-03-26 Fyusion, Inc. Artificially rendering images using viewpoint interpolation and extrapolation
US10852902B2 (en) 2015-07-15 2020-12-01 Fyusion, Inc. Automatic tagging of objects on a multi-view interactive digital media representation of a dynamic entity
US10750161B2 (en) 2015-07-15 2020-08-18 Fyusion, Inc. Multi-view interactive digital media representation lock screen
US10147211B2 (en) 2015-07-15 2018-12-04 Fyusion, Inc. Artificially rendering images using viewpoint interpolation and extrapolation
JP6564275B2 (ja) * 2015-08-20 2019-08-21 キヤノン株式会社 画像処理装置、及び画像処理方法
US9609307B1 (en) 2015-09-17 2017-03-28 Legend3D, Inc. Method of converting 2D video to 3D video using machine learning
US11783864B2 (en) 2015-09-22 2023-10-10 Fyusion, Inc. Integration of audio into a multi-view interactive digital media representation
US10489970B2 (en) * 2015-11-06 2019-11-26 Microsoft Technology Licensing, Llc 2D image processing for extrusion into 3D objects
US10204447B2 (en) 2015-11-06 2019-02-12 Microsoft Technology Licensing, Llc 2D image processing for extrusion into 3D objects
CN108475330B (zh) * 2015-11-09 2022-04-08 港大科桥有限公司 用于有伪像感知的视图合成的辅助数据
CN105608666A (zh) * 2015-12-25 2016-05-25 普瑞福克斯（北京）数字媒体科技有限公司 一种二维图形生成三维图像的方法及系统
US10252417B2 (en) * 2016-03-02 2019-04-09 Canon Kabushiki Kaisha Information processing apparatus, method of controlling information processing apparatus, and storage medium
CN105913499A (zh) * 2016-04-12 2016-08-31 郭栋 一种立体转制合成的方法及系统
US10134198B2 (en) * 2016-04-19 2018-11-20 Adobe Systems Incorporated Image compensation for an occluding direct-view augmented reality system
US10469821B2 (en) * 2016-06-17 2019-11-05 Altek Semiconductor Corp. Stereo image generating method and electronic apparatus utilizing the method
US10204418B2 (en) * 2016-09-07 2019-02-12 Nokia Technologies Oy Method and apparatus for facilitating stereo vision through the use of multi-layer shifting
CN113873262B (zh) 2016-10-04 2023-03-24 有限公司B1影像技术研究所 图像数据编码/解码方法和装置
US11202017B2 (en) 2016-10-06 2021-12-14 Fyusion, Inc. Live style transfer on a mobile device
US10828114B2 (en) * 2016-10-18 2020-11-10 Synaptive Medical (Barbados) Inc. Methods and systems for providing depth information
CA3041352C (fr) 2016-10-21 2023-12-12 Synaptive Medical (Barbados) Inc. Procedes et systemes pour la fourniture d'informations de profondeur
CN106791770B (zh) * 2016-12-20 2018-08-10 南阳师范学院 一种适用于dibr预处理过程的深度图融合方法
US10437879B2 (en) 2017-01-18 2019-10-08 Fyusion, Inc. Visual search using multi-view interactive digital media representations
CN106910253B (zh) * 2017-02-22 2020-02-18 天津大学 基于不同相机间距的立体图像克隆方法
US20180246635A1 (en) * 2017-02-24 2018-08-30 Microsoft Technology Licensing, Llc Generating user interfaces combining foreground and background of an image with user interface elements
CN106993178A (zh) * 2017-04-05 2017-07-28 上海矽奥微电子有限公司 2d转3d视频图像提升深度图质量的计算方法
CN110494790A (zh) * 2017-04-26 2019-11-22 华为技术有限公司 用于生成3d光场的设备和方法
US10671450B2 (en) 2017-05-02 2020-06-02 Facebook, Inc. Coalescing events framework
US11069019B2 (en) 2017-05-04 2021-07-20 Facebook, Inc. Multi-threaded asynchronous frame processing
US10313651B2 (en) 2017-05-22 2019-06-04 Fyusion, Inc. Snapshots at predefined intervals or angles
US10769438B2 (en) * 2017-05-23 2020-09-08 Samsung Electronics Company, Ltd. Augmented reality
US11069147B2 (en) 2017-06-26 2021-07-20 Fyusion, Inc. Modification of multi-view interactive digital media representation
EP3422711A1 (fr) * 2017-06-29 2019-01-02 Koninklijke Philips N.V. Appareil et procédé de manipulation d'images
DE102017211038A1 (de) * 2017-06-29 2019-01-03 Robert Bosch Gmbh Verfahren zum Einstellen einer Kamera
US10735707B2 (en) * 2017-08-15 2020-08-04 International Business Machines Corporation Generating three-dimensional imagery
JP6998554B2 (ja) * 2017-09-12 2022-01-18 パナソニックＩｐマネジメント株式会社 画像生成装置及び画像生成方法
US11494934B2 (en) * 2018-03-29 2022-11-08 Mitsubishi Electric Corporation Image processing device, image processing method, and monitoring system
WO2019195932A1 (fr) * 2018-04-09 2019-10-17 John Alistair Quee Particules destinées à être utilisées dans une incrustation couleur
US10592747B2 (en) 2018-04-26 2020-03-17 Fyusion, Inc. Method and apparatus for 3-D auto tagging
ES2737452B2 (es) * 2018-07-04 2020-12-10 Guerrero Cristobal Ayala Procedimiento para realizacion de imagenes y peliculas estereoscopias de doble visionado
US20200125651A1 (en) 2018-10-18 2020-04-23 Cae Inc. Loading rendering data for an interactive computer
US10825200B2 (en) * 2019-01-30 2020-11-03 Trivver, Inc. Texture based pixel count determination
US11386636B2 (en) 2019-04-04 2022-07-12 Datalogic Usa, Inc. Image preprocessing for optical character recognition
JP7447417B2 (ja) * 2019-09-27 2024-03-12 ソニーグループ株式会社 画像処理装置、画像処理方法、プログラム
GB2588827B (en) * 2019-11-11 2022-10-12 Advanced Risc Mach Ltd Depth processing
US11625848B2 (en) * 2020-01-30 2023-04-11 Unity Technologies Sf Apparatus for multi-angle screen coverage analysis
BR112022024142A2 (pt) * 2020-05-26 2023-02-14 Unify Medical Sistema e método para gerar imagens tridimensionais
CN112215813B (zh) * 2020-10-12 2024-01-19 中国人民解放军火箭军工程大学 一种基于多特征约束的动态伪装效果评估方法
CN112104856A (zh) * 2020-11-18 2020-12-18 基腾(成都)科技有限公司 低复杂度二维影像视讯转换成三维影像视讯的方法
CN112419385B (zh) * 2021-01-25 2021-04-09 国汽智控(北京)科技有限公司 一种3d深度信息估计方法、装置及计算机设备
CN112967199A (zh) * 2021-03-05 2021-06-15 北京字跳网络技术有限公司 图像处理方法和装置
CN113486803A (zh) * 2021-07-07 2021-10-08 北京沃东天骏信息技术有限公司 视频中嵌入图像的装置
US11551407B1 (en) 2021-09-01 2023-01-10 Design Interactive, Inc. System and method to convert two-dimensional video into three-dimensional extended reality content
JP2023051356A (ja) * 2021-09-30 2023-04-11 キヤノン株式会社 情報処理装置、情報処理方法およびプログラム
CN116977607B (zh) * 2023-07-21 2024-05-07 武汉熠腾科技有限公司 基于像素流的文物模型展示方法及系统
CN118211286A (zh) * 2024-05-16 2024-06-18 成都亿格人居科技有限公司 基于数字场景的设计方案生成方法及系统

Citations (161)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US3740115A (en) 1971-09-02 1973-06-19 Optical Corp Fiberscope viewing system with dynamic image enhancer
US4141624A (en) 1977-11-21 1979-02-27 American Optical Corporation Dynamic image enhancement for fiberscopes
US4154502A (en) 1978-02-01 1979-05-15 American Optical Corporation Dynamic image enhancer for fiberscopes
EP0037722A1 (fr) 1980-04-08 1981-10-14 Technicare Corporation Méthode et dispositif pour la mise en évidence dynamique d'images
GB2081923A (en) 1980-08-15 1982-02-24 American Optical Corp Dynamic image enhancer for fibrescopes
US4359759A (en) 1980-04-21 1982-11-16 Technicare Corporation Bias lighting in a radiographic apparatus and method
GB2162018A (en) 1984-07-17 1986-01-22 Kokusai Denshin Denwa Co Ltd Moving picture frame rate conversion system
US4570308A (en) 1984-03-16 1986-02-18 Weisgerber Robert G Construction for a belt buckle
US4677483A (en) 1985-09-27 1987-06-30 Rca Corporation Interpolation filter with motion compensation
US4794714A (en) 1985-03-07 1989-01-03 Weisgerber Robert G Frame assembly for a display item
GB2210237A (en) 1987-09-18 1989-06-01 Racal Data Communications Inc Constellation multiplexed inband secondary channel for voiceband modem
GB2215936A (en) 1988-03-23 1989-09-27 Benchmark Technologies Video frame and rate conversion
US4889423A (en) 1986-06-26 1989-12-26 Showscan Film Corporation Conversion of high to low frame rate motion picture films
US4925294A (en) 1986-12-17 1990-05-15 Geshwind David M Method to convert two dimensional motion pictures for three-dimensional systems
US5096286A (en) 1990-03-07 1992-03-17 Weisgerber Robert C Method for transitioning between two different frame rates during a showing of a single motion picture film
EP0475499A1 (fr) 1990-09-03 1992-03-18 Koninklijke Philips Electronics N.V. Conversion de fréquence d'image à compensation de mouvement
US5121977A (en) 1990-03-07 1992-06-16 Weisgerber Robert C Apparatus for raising and lowering the lower screen mask in a motion picture theater
US5236302A (en) 1991-10-30 1993-08-17 General Electric Company Turbine disk interstage seal system
US5259040A (en) 1991-10-04 1993-11-02 David Sarnoff Research Center, Inc. Method for determining sensor motion and scene structure and image processing system therefor
US5288210A (en) 1991-10-30 1994-02-22 General Electric Company Turbine disk attachment system
US5325449A (en) 1992-05-15 1994-06-28 David Sarnoff Research Center, Inc. Method for fusing images and apparatus therefor
US5338154A (en) 1993-03-17 1994-08-16 General Electric Company Turbine disk interstage seal axial retaining ring
US5379369A (en) 1991-03-04 1995-01-03 Sharp Kabushiki Kaisha Apparatus for generating stereoscopic image and method therefor
GB2280080A (en) 1993-06-09 1995-01-18 Sony Uk Ltd Video frame rate conversion from higher to lower frame rates
EP0637889A2 (fr) 1993-08-06 1995-02-08 Goldstar Co. Ltd. Dispositif pour convertir le format de trame d'image
US5472313A (en) 1991-10-30 1995-12-05 General Electric Company Turbine disk cooling system
US5510832A (en) 1993-12-01 1996-04-23 Medi-Vision Technologies, Inc. Synthesized stereoscopic imaging system and method
US5541660A (en) 1993-12-24 1996-07-30 Daewoo Electronics Co., Ltd. Systolic realization of motion compensated interpolation filter
EP0742674A2 (fr) 1995-05-08 1996-11-13 Kabushiki Kaisha Toshiba Méthode et système de codage vidéo employant un modèle débit-quantificateur
US5589852A (en) 1989-02-27 1996-12-31 Texas Instruments Incorporated Apparatus and method for image projection with pixel intensity control
US5600731A (en) 1991-05-09 1997-02-04 Eastman Kodak Company Method for temporally adaptive filtering of frames of a noisy image sequence using motion estimation
US5627614A (en) 1995-08-25 1997-05-06 Weisgerber; Robert C. Method for imparting both high-impact cinematic images and conventional cinematic images on the same strip of motion picture film
US5629988A (en) 1993-06-04 1997-05-13 David Sarnoff Research Center, Inc. System and method for electronic image stabilization
US5649032A (en) 1994-11-14 1997-07-15 David Sarnoff Research Center, Inc. System for automatically aligning images to form a mosaic image
US5682437A (en) 1994-09-22 1997-10-28 Sanyo Electric Co., Ltd. Method of converting two-dimensional images into three-dimensional images
US5694491A (en) 1996-03-29 1997-12-02 David Sarnoff Research Center, Inc. Methods and apparatus for assessing the visibility of differences between two image sequences
EP0595895B1 (fr) 1991-07-23 1997-12-29 BRITISH TELECOMMUNICATIONS public limited company Procede et dispositif d'interpolation des trames d'une image mouvante
US5719966A (en) 1996-03-29 1998-02-17 David Sarnoff Research Center, Inc. Apparatus for assessing the visiblity of differences between two image sequences
US5738430A (en) 1996-03-29 1998-04-14 David Sarnoff Research Center, Inc. Method and apparatus for predicting retinal illuminance
US5739844A (en) 1994-02-04 1998-04-14 Sanyo Electric Co. Ltd. Method of converting two-dimensional image into three-dimensional image
US5739894A (en) 1995-08-25 1998-04-14 Weisgerber; Robert C. Method for exhibiting cinematic images photographed at two different frame rates
EP0837601A2 (fr) 1996-10-15 1998-04-22 Hitachi, Ltd. Méthode et circuit de traitement de signal pour une conversion de format d'un signal d'image
US5745213A (en) 1996-05-08 1998-04-28 Goodhill; Dean K. Method of making enhanced resolution motion picture release-print film
US5748199A (en) 1995-12-20 1998-05-05 Synthonics Incorporated Method and apparatus for converting a two dimensional motion picture into a three dimensional motion picture
US5790686A (en) 1995-09-19 1998-08-04 University Of Maryland At College Park DCT-based motion estimation method
US5812245A (en) 1996-03-26 1998-09-22 Goodhill; Dean K. Asynchronous contact printer for motion picture film
US5835627A (en) 1995-05-15 1998-11-10 Higgins; Eric W. System and method for automatically optimizing image quality and processing time
US5841512A (en) 1996-02-27 1998-11-24 Goodhill; Dean Kenneth Methods of previewing and editing motion pictures
US5864366A (en) 1997-02-05 1999-01-26 International Business Machines Corporation System and method for selecting video information with intensity difference
US5909516A (en) 1996-03-29 1999-06-01 Sarnoff Corporation Method and apparatus for decomposing an image stream into units of local contrast
US5946041A (en) 1996-01-31 1999-08-31 Fujitsu Limited Apparatus and method of tracking an image-feature using a block matching algorithm
US5946076A (en) 1996-02-07 1999-08-31 Maxivision Cinema Technology Switchable pulldown film projection system
US5963664A (en) 1995-06-22 1999-10-05 Sarnoff Corporation Method and system for image combination using a parallax-based technique
US5974159A (en) 1996-03-29 1999-10-26 Sarnoff Corporation Method and apparatus for assessing the visibility of differences between two image sequences
US5978029A (en) 1997-10-10 1999-11-02 International Business Machines Corporation Real-time encoding of video sequence employing two encoders and statistical analysis
US5999220A (en) 1997-04-07 1999-12-07 Washino; Kinya Multi-format audio/video production system with frame-rate conversion
US6019473A (en) 1996-02-07 2000-02-01 Maxivision Cinema Technology Switchable format film projection system
US6031564A (en) 1997-07-07 2000-02-29 Reveo, Inc. Method and apparatus for monoscopic to stereoscopic image conversion
EP0665697B1 (fr) 1994-02-01 2000-05-03 SANYO ELECTRIC Co., Ltd. Procédé et appareil pour la conversion d'images bidimensionnelles dans des images tridimensionnelles
GB2343316A (en) 1998-10-26 2000-05-03 Sony Uk Ltd Video interpolation
US6061477A (en) 1996-04-18 2000-05-09 Sarnoff Corporation Quality image warper
US6067125A (en) 1997-05-15 2000-05-23 Minerva Systems Structure and method for film grain noise reduction
US6072907A (en) 1997-05-28 2000-06-06 Xerox Corporation Method and apparatus for enhancing and thresholding images
US6075884A (en) 1996-03-29 2000-06-13 Sarnoff Corporation Method and apparatus for training a neural network to learn and use fidelity metric as a control mechanism
US6108005A (en) 1996-08-30 2000-08-22 Space Corporation Method for producing a synthesized stereoscopic image
US6130717A (en) 1990-10-15 2000-10-10 Canon Kabushiki Kaisha Image pickup apparatus with a lens unit attachment/detachment indication
US6130660A (en) 1993-10-01 2000-10-10 Maxvision Corporation System and method for synthesizing high resolution video
US6137904A (en) 1997-04-04 2000-10-24 Sarnoff Corporation Method and apparatus for assessing the visibility of differences between two signal sequences
US6141459A (en) 1997-09-24 2000-10-31 Sarnoff Corporation Method and apparatus for processing image pyramid borders
US6191809B1 (en) 1998-01-15 2001-02-20 Vista Medical Technologies, Inc. Method and apparatus for aligning stereo images
US6208348B1 (en) 1998-05-27 2001-03-27 In-Three, Inc. System and method for dimensionalization processing of images in consideration of a pedetermined image projection format
US6219462B1 (en) 1997-05-09 2001-04-17 Sarnoff Corporation Method and apparatus for performing global image alignment using any local match measure
US6229570B1 (en) 1998-09-25 2001-05-08 Lucent Technologies Inc. Motion compensation image interpolation—frame rate conversion for HDTV
US6243156B1 (en) 1999-05-03 2001-06-05 Robert C. Weisgerber Method for exhibiting motion picture films at frame rates higher than that in which the films were originally produced
US6266092B1 (en) 1998-05-12 2001-07-24 Genesis Microchip Inc. Method and apparatus for video line multiplication with enhanced sharpness
US6269175B1 (en) 1998-08-28 2001-07-31 Sarnoff Corporation Method and apparatus for enhancing regions of aligned images using flow estimation
US6298090B1 (en) 1998-06-04 2001-10-02 U.S. Philips Corporation System for detecting redundant images in a video sequence by comparing two predetermined threshold values
US20010031003A1 (en) 1999-12-20 2001-10-18 Sawhney Harpreet Singh Tweening-based codec for scaleable encoders and decoders with varying motion computation capability
US20010043310A1 (en) 1996-02-07 2001-11-22 Goodhill Dean K. Switchable format film projection system
US6326999B1 (en) 1994-08-23 2001-12-04 Discovision Associates Data rate conversion
EP1164792A2 (fr) 2000-06-13 2001-12-19 Samsung Electronics Co., Ltd. Convertisseur de format utilisant des vecteurs de mouvement bidirectionnels et méthode correspondante
US6351545B1 (en) 1999-12-14 2002-02-26 Dynapel Systems, Inc. Motion picture enhancing system
US20020035432A1 (en) 2000-06-08 2002-03-21 Boguslaw Kubica Method and system for spatially indexing land
US6370198B1 (en) 1997-04-07 2002-04-09 Kinya Washino Wide-band multi-format audio/video production system with frame-rate conversion
EP0735512B1 (fr) 1995-03-29 2002-06-05 SANYO ELECTRIC Co., Ltd. Méthodes pour la selection de deux trames d'une sequence d'images bidimensionnels, pour le calcul de profondeur relative d'objets d'image
CA2430591A1 (fr) 2000-12-01 2002-06-06 Imax Corporation Techniques et systemes pour la mise au point d'imagerie haute resolution
EP1217839A2 (fr) 2000-12-08 2002-06-26 Matsushita Electric Industrial Co., Ltd. Dispositif de conversion de données, dispositif de codage de données, et dispositif d'enregistrement de données
US6442203B1 (en) 1999-11-05 2002-08-27 Demografx System and method for motion compensation and frame rate conversion
EP1235426A2 (fr) 2001-02-23 2002-08-28 Eastman Kodak Company Procédé de présentation de séquences d'images animées améliorées
EP1237370A2 (fr) 2001-02-23 2002-09-04 Eastman Kodak Company Système d'interpolation des trames d'une image mouvante à vitesse variable
US6450644B1 (en) 2000-01-12 2002-09-17 Maxivision Cinema Technology System and method for registering motion picture film
US6477267B1 (en) 1995-12-22 2002-11-05 Dynamic Digital Depth Research Pty Ltd. Image conversion and encoding techniques
US6487304B1 (en) 1999-06-16 2002-11-26 Microsoft Corporation Multi-view approach to motion and stereo
US6496598B1 (en) * 1997-09-02 2002-12-17 Dynamic Digital Depth Research Pty. Ltd. Image processing method and apparatus
US6515659B1 (en) 1998-05-27 2003-02-04 In-Three, Inc. Method and system for creating realistic smooth three-dimensional depth contours from two-dimensional images
US6522787B1 (en) 1995-07-10 2003-02-18 Sarnoff Corporation Method and system for rendering and combining images to form a synthesized view of a scene containing image information from a second image
US20030036974A1 (en) 1996-12-03 2003-02-20 Richard Allen Apparatus and method for an on demand data delivery system for the preview selection, retrieval and reproduction at a remote location of previously recorded or programmed materials
US20030050940A1 (en) 1999-10-12 2003-03-13 Eric Robinson Automatic backup system
US6549240B1 (en) 1997-09-26 2003-04-15 Sarnoff Corporation Format and frame rate conversion for display of 24Hz source video
US6560281B1 (en) 1998-02-24 2003-05-06 Xerox Corporation Method and apparatus for generating a condensed version of a video sequence including desired affordances
US20030095704A1 (en) 2001-11-22 2003-05-22 Risson Valery J. Determination of an illuminant of digital color image by segmentation and filtering
US6573912B1 (en) 2000-11-07 2003-06-03 Zaxel Systems, Inc. Internet system for virtual telepresence
US6590573B1 (en) 1983-05-09 2003-07-08 David Michael Geshwind Interactive computer system for creating three-dimensional image information and for converting two-dimensional image information for three-dimensional display systems
US20030128871A1 (en) * 2000-04-01 2003-07-10 Rolf-Dieter Naske Methods and systems for 2D/3D image conversion and optimization
US6618439B1 (en) 1999-07-06 2003-09-09 Industrial Technology Research Institute Fast motion-compensated video frame interpolator
US20030169404A1 (en) 2001-07-16 2003-09-11 Weisgerber Robert C. Method for exhibiting high-quality three-dimensional motion from a single strip of motion picture film
US6625333B1 (en) 1999-08-06 2003-09-23 Her Majesty The Queen In Right Of Canada As Represented By The Minister Of Industry Through Communications Research Centre Method for temporal interpolation of an image sequence using object-based image analysis
US20030204384A1 (en) 2002-04-24 2003-10-30 Yuri Owechko High-performance sensor fusion architecture
US20040017510A1 (en) 2002-03-18 2004-01-29 Pioneer Corporation Video signal processing apparatus
US20040022318A1 (en) 2002-05-29 2004-02-05 Diego Garrido Video interpolation coding
EP1397003A1 (fr) 2002-08-07 2004-03-10 Micronas GmbH Méthode pour l'interpolation d'un pixel contenu dans une image intermédiaire d'une suite d'image à l'aide de vecteurs de mouvement
US20040046891A1 (en) 2002-09-10 2004-03-11 Kabushiki Kaisha Toshiba Frame interpolation and apparatus using frame interpolation
US6728317B1 (en) 1996-01-30 2004-04-27 Dolby Laboratories Licensing Corporation Moving image compression quality enhancement using displacement filters with negative lobes
EP1084451B1 (fr) 1998-04-27 2004-06-16 Maxivision Cinema Technology Procede de production d'un film de diffusion d'oeuvres cinematographiques
US20040130680A1 (en) 2002-03-13 2004-07-08 Samuel Zhou Systems and methods for digitally re-mastering or otherwise modifying motion pictures or other image sequences data
US20040136686A1 (en) 2002-11-07 2004-07-15 Seiko Epson Corporation Conversion of frame rate according to image data
EP1455338A1 (fr) 2003-03-06 2004-09-08 Pioneer Corporation Circuit de multiplication de la fréquence de trame
US20040202445A1 (en) 2003-03-20 2004-10-14 Pixar Component color flat panel digital film recorder and method
US20040201777A1 (en) 2003-04-11 2004-10-14 Huaya Microelectronics (Shanghai) Inc. Frame rate conversion of interlaced and progressive video streams
US6806898B1 (en) 2000-03-20 2004-10-19 Microsoft Corp. System and method for automatically adjusting gaze and head orientation for video conferencing
JP2004297719A (ja) 2003-03-28 2004-10-21 Toshiba Corp フレーム補間方法及びこれを用いた画像表示システム
JP2004312669A (ja) 2003-03-25 2004-11-04 Toshiba Corp フレーム補間のための補間画像作成方法及びこれを用いた画像表示システム、補間画像作成装置
US20040223640A1 (en) 2003-05-09 2004-11-11 Bovyrin Alexander V. Stereo matching using segmentation of image columns
US20040227851A1 (en) 2003-05-13 2004-11-18 Samsung Electronics Co., Ltd. Frame interpolating method and apparatus thereof at frame rate conversion
US20040236807A1 (en) 2003-05-23 2004-11-25 Via Technologies, Inc. Method and apparatus for adaptive frame rate conversion
JP2004348117A (ja) 2003-05-24 2004-12-09 Samsung Electronics Co Ltd 映像補間装置および映像補間方法
JP2004357215A (ja) 2003-05-30 2004-12-16 Toshiba Corp フレーム補間方法及び装置並びに画像表示システム
US20040252230A1 (en) 2003-06-13 2004-12-16 Microsoft Corporation Increasing motion smoothness using frame interpolation with motion analysis
US20040252759A1 (en) 2003-06-13 2004-12-16 Microsoft Corporation Quality control in frame interpolation with motion analysis
US20050002456A1 (en) 2003-07-04 2005-01-06 Min Jong-Sul Motion vector detector for frame rate conversion and method thereof
EP1503582A1 (fr) 2003-07-31 2005-02-02 Pioneer Corporation Appareil pour la conversion d'un signal video
US6856314B2 (en) 2002-04-18 2005-02-15 Stmicroelectronics, Inc. Method and system for 3D reconstruction of multiple views with altering search path and occlusion modeling
JP2005057809A (ja) 2004-10-25 2005-03-03 Matsushita Electric Ind Co Ltd シネマ信号作成システムおよび撮像装置
EP1519358A2 (fr) 2003-09-25 2005-03-30 Canon Kabushiki Kaisha Circuit de conversion de la fréquence de trame, méthode de prévision de dépassement pour ce circuit , dispositif de commande d'affichage et dispositif d'affichage avec récepteur vidéo
EP1549054A2 (fr) 2003-12-23 2005-06-29 Genesis Microchip, Inc. Conversion de fréquence de trame à compensation de motion
JP2005204075A (ja) 2004-01-15 2005-07-28 Sony Corp 動きベクトル検出装置とそれを用いたフレーム補間装置、動きベクトル検出方法とそれを用いたフレーム補間方法およびプログラムと記録媒体。
EP1571854A1 (fr) 2002-11-25 2005-09-07 Sanyo Electric Co., Ltd. Procede de creation d'images video stereoscopiques et affichage d'images video stereoscopiques
US6950462B2 (en) 2001-03-08 2005-09-27 Matsushita Electric Industrial Co., Ltd. Video encoding apparatus, video encoding method, and frame rate conversion apparatus
EP1583364A1 (fr) 2004-03-30 2005-10-05 Matsushita Electric Industrial Co., Ltd. Interpolation d'images à mouvements compensés aux contours de l'image pour conversion de fréquence d'image
EP1585326A1 (fr) 2004-03-30 2005-10-12 Matsushita Electric Industrial Co., Ltd. Estimation de vecteur de mouvement aux contours de l'image pour conversion de la fréquence de rafraíchissement
US20050226529A1 (en) 2004-02-19 2005-10-13 Sony Corporation Signal processing apparatus and method, and command-sequence data structure
US20050254011A1 (en) 2004-05-17 2005-11-17 Weisgerber Robert C Method for exhibiting motion picture films at a higher frame rate than that in which they were originally produced
US6968006B1 (en) 2001-06-05 2005-11-22 At&T Corp. Method of content adaptive video decoding
US20050265451A1 (en) 2004-05-04 2005-12-01 Fang Shi Method and apparatus for motion compensated frame rate up conversion for block-based low bit rate video
US6992700B1 (en) 1998-09-08 2006-01-31 Ricoh Company, Ltd. Apparatus for correction based upon detecting a camera shaking
US7006157B2 (en) 2002-02-19 2006-02-28 Samsung Electronics Co., Ltd. Apparatus and method for converting frame rate
US20060120451A1 (en) 2002-01-23 2006-06-08 Nokia Corporation Grouping of image frames in video coding
US7079697B2 (en) 2001-03-19 2006-07-18 Texas Instruments Incorporated Image compression with transform coefficient analysis
US20060165179A1 (en) 2005-01-27 2006-07-27 Technion Research & Development Foundation Ltd. Acquisition of image sequences with enhanced resolution
US7119837B2 (en) 2002-06-28 2006-10-10 Microsoft Corporation Video processing system and method for automatic enhancement of digital video
US20070019883A1 (en) * 2005-07-19 2007-01-25 Wong Earl Q Method for creating a depth map for auto focus using an all-in-focus picture and two-dimensional scale space matching
US7227125B2 (en) 2005-06-29 2007-06-05 Mitsumi Electric Co., Ltd. Encoder device
US7227896B2 (en) 2001-10-04 2007-06-05 Sharp Laboratories Of America, Inc. Method and apparatus for global motion estimation
US20080055490A1 (en) 2004-12-03 2008-03-06 Sony Computer Entertainment Inc. Image Display Device, Image Display Method, and Program
EP1977395A2 (fr) 2006-01-27 2008-10-08 Imax Corporation Procedes et systemes pour le rematriçage numerique de films bi- et tridimensionnels pour une presentation avec une qualite visuelle amelioree
JP4201165B2 (ja) 2001-09-04 2008-12-24 住化エンビロサイエンス株式会社 工業用防カビ組成物
EP2033164A2 (fr) 2006-06-23 2009-03-11 Imax Corporation Procédés et systèmes de conversion d'images cinématographiques 2d pour une représentation stéréoscopique 3d
US7689031B2 (en) * 2002-12-30 2010-03-30 Koninklijke Philips Electronics N.V. Video filtering for stereo images
US20100182406A1 (en) * 2007-07-12 2010-07-22 Benitez Ana B System and method for three-dimensional object reconstruction from two-dimensional images
US20110069152A1 (en) * 2009-09-24 2011-03-24 Shenzhen Tcl New Technology Ltd. 2D to 3D video conversion
US20120056984A1 (en) * 2010-09-03 2012-03-08 Samsung Electronics Co., Ltd. Method and apparatus for converting 2-dimensional image into 3-dimensional image by adjusting depth of the 3-dimensional image

Family Cites Families (28)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5313604A (en) 1990-11-13 1994-05-17 Hewlett-Packard Company Method for locating compressed data in a computed memory back up device including steps of refining estimater location
GB9223114D0 (en) 1992-11-04 1992-12-16 Rank Brimar Ltd Display system
US6327381B1 (en) * 1994-12-29 2001-12-04 Worldscape, Llc Image transformation and synthesis methods
US6075905A (en) 1996-07-17 2000-06-13 Sarnoff Corporation Method and apparatus for mosaic image construction
US6173087B1 (en) 1996-11-13 2001-01-09 Sarnoff Corporation Multi-view image registration with application to mosaicing and lens distortion correction
JP2001526859A (ja) 1997-05-16 2001-12-18 ザ トラスティーズ オブ コロンビア ユニヴァーシティ イン ザ シティ オブ ニューヨーク ワールドワイドウェブ上の圧縮映像の指示及び編集方法及びアーキテクチュア
US6173070B1 (en) 1997-12-30 2001-01-09 Cognex Corporation Machine vision method using search models to find features in three dimensional images
US20050146521A1 (en) 1998-05-27 2005-07-07 Kaye Michael C. Method for creating and presenting an accurate reproduction of three-dimensional images converted from two-dimensional images
US7116323B2 (en) 1998-05-27 2006-10-03 In-Three, Inc. Method of hidden surface reconstruction for creating accurate three-dimensional images converted from two-dimensional images
US7116324B2 (en) 1998-05-27 2006-10-03 In-Three, Inc. Method for minimizing visual artifacts converting two-dimensional motion pictures into three-dimensional motion pictures
US7102633B2 (en) 1998-05-27 2006-09-05 In-Three, Inc. Method for conforming objects to a common depth perspective for converting two-dimensional images into three-dimensional images
US20050231505A1 (en) 1998-05-27 2005-10-20 Kaye Michael C Method for creating artifact free three-dimensional images converted from two-dimensional images
US6295322B1 (en) 1998-07-09 2001-09-25 North Shore Laboratories, Inc. Processing apparatus for synthetically extending the bandwidth of a spatially-sampled video image
WO2000013423A1 (fr) 1998-08-28 2000-03-09 Sarnoff Corporation Procede et dispositif d'imagerie de synthese haute resolution utilisant une camera haute resolution et une camera a resolution plus faible
KR100321897B1 (ko) 1998-12-22 2002-05-13 박남은 입체영상이미지변환방법및그장치
WO2001028238A2 (fr) 1999-10-08 2001-04-19 Sarnoff Corporation Procede et appareil permettant d'ameliorer et d'indexer des signaux video et audio
CA2414723C (fr) 2000-07-03 2012-05-15 Imax Corporation Materiel et techniques destines a accroitre la gamme dynamique d'un systeme de projection
AUPQ921000A0 (en) 2000-08-04 2000-08-31 Martin, Vincent Michael Composite wall panel
US7180663B2 (en) 2002-06-19 2007-02-20 Robert Bruce Collender 3D motion picture theatre
US7034822B2 (en) * 2002-06-19 2006-04-25 Swiss Federal Institute Of Technology Zurich System and method for producing 3D video images
JP2006102484A (ja) 2004-09-07 2006-04-20 Kenshoen:Kk 着物用ハンガー
WO2006078237A1 (fr) 2005-01-15 2006-07-27 In-Three, Inc. Procede pour la conversion d'images bidimensionnelles en images tridimensionnelles utilisant la perspective de profondeur
WO2006078249A1 (fr) 2005-01-21 2006-07-27 In-Three, Inc. Procede pour convertir des images animees bidimensionnelles en images animees tridimensionnelles permettant de minimiser des artefacts visuels
WO2006078250A1 (fr) 2005-01-21 2006-07-27 In-Three, Inc. Procede de reconstruction de surface cachee pour la creation d'images tridimensionnelles precises a partir d'images bidimensionnelles
US7609888B2 (en) * 2005-07-01 2009-10-27 Microsoft Corporation Separating a video object from a background of a video sequence
CN101223778B (zh) * 2005-07-18 2012-11-07 汤姆森许可贸易公司 使用元数据来处理多个视频流的方法和设备
WO2007142649A1 (fr) * 2006-06-09 2007-12-13 Thomson Licensing Procédé et système destinés à une correction de couleur à l'aide d'informations tridimensionnelles
KR100901273B1 (ko) * 2007-12-15 2009-06-09 한국전자통신연구원 렌더링 시스템 및 이를 이용한 데이터 처리 방법

Patent Citations (202)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US3740115A (en) 1971-09-02 1973-06-19 Optical Corp Fiberscope viewing system with dynamic image enhancer
US4141624A (en) 1977-11-21 1979-02-27 American Optical Corporation Dynamic image enhancement for fiberscopes
GB2009959A (en) 1977-11-21 1979-06-20 American Optical Corp Fibrescope dynamic image enhancement
US4154502A (en) 1978-02-01 1979-05-15 American Optical Corporation Dynamic image enhancer for fiberscopes
EP0037722A1 (fr) 1980-04-08 1981-10-14 Technicare Corporation Méthode et dispositif pour la mise en évidence dynamique d'images
US4359759A (en) 1980-04-21 1982-11-16 Technicare Corporation Bias lighting in a radiographic apparatus and method
GB2081923A (en) 1980-08-15 1982-02-24 American Optical Corp Dynamic image enhancer for fibrescopes
US4378952A (en) 1980-08-15 1983-04-05 Warner Lambert Technologies, Inc. Dynamic image enhancer for fiberscopes
US6590573B1 (en) 1983-05-09 2003-07-08 David Michael Geshwind Interactive computer system for creating three-dimensional image information and for converting two-dimensional image information for three-dimensional display systems
US4570308A (en) 1984-03-16 1986-02-18 Weisgerber Robert G Construction for a belt buckle
GB2162018A (en) 1984-07-17 1986-01-22 Kokusai Denshin Denwa Co Ltd Moving picture frame rate conversion system
US4794714A (en) 1985-03-07 1989-01-03 Weisgerber Robert G Frame assembly for a display item
US4677483A (en) 1985-09-27 1987-06-30 Rca Corporation Interpolation filter with motion compensation
US4889423A (en) 1986-06-26 1989-12-26 Showscan Film Corporation Conversion of high to low frame rate motion picture films
US4925294A (en) 1986-12-17 1990-05-15 Geshwind David M Method to convert two dimensional motion pictures for three-dimensional systems
GB2210237A (en) 1987-09-18 1989-06-01 Racal Data Communications Inc Constellation multiplexed inband secondary channel for voiceband modem
GB2215936A (en) 1988-03-23 1989-09-27 Benchmark Technologies Video frame and rate conversion
US5589852A (en) 1989-02-27 1996-12-31 Texas Instruments Incorporated Apparatus and method for image projection with pixel intensity control
US5121977A (en) 1990-03-07 1992-06-16 Weisgerber Robert C Apparatus for raising and lowering the lower screen mask in a motion picture theater
US5096286A (en) 1990-03-07 1992-03-17 Weisgerber Robert C Method for transitioning between two different frame rates during a showing of a single motion picture film
EP0475499A1 (fr) 1990-09-03 1992-03-18 Koninklijke Philips Electronics N.V. Conversion de fréquence d'image à compensation de mouvement
US6130717A (en) 1990-10-15 2000-10-10 Canon Kabushiki Kaisha Image pickup apparatus with a lens unit attachment/detachment indication
US5379369A (en) 1991-03-04 1995-01-03 Sharp Kabushiki Kaisha Apparatus for generating stereoscopic image and method therefor
US5600731A (en) 1991-05-09 1997-02-04 Eastman Kodak Company Method for temporally adaptive filtering of frames of a noisy image sequence using motion estimation
EP0595895B1 (fr) 1991-07-23 1997-12-29 BRITISH TELECOMMUNICATIONS public limited company Procede et dispositif d'interpolation des trames d'une image mouvante
US5259040A (en) 1991-10-04 1993-11-02 David Sarnoff Research Center, Inc. Method for determining sensor motion and scene structure and image processing system therefor
US5288210A (en) 1991-10-30 1994-02-22 General Electric Company Turbine disk attachment system
US5236302A (en) 1991-10-30 1993-08-17 General Electric Company Turbine disk interstage seal system
US5472313A (en) 1991-10-30 1995-12-05 General Electric Company Turbine disk cooling system
US5325449A (en) 1992-05-15 1994-06-28 David Sarnoff Research Center, Inc. Method for fusing images and apparatus therefor
US5488674A (en) 1992-05-15 1996-01-30 David Sarnoff Research Center, Inc. Method for fusing images and apparatus therefor
US5338154A (en) 1993-03-17 1994-08-16 General Electric Company Turbine disk interstage seal axial retaining ring
US5629988A (en) 1993-06-04 1997-05-13 David Sarnoff Research Center, Inc. System and method for electronic image stabilization
GB2280080A (en) 1993-06-09 1995-01-18 Sony Uk Ltd Video frame rate conversion from higher to lower frame rates
EP0637889A2 (fr) 1993-08-06 1995-02-08 Goldstar Co. Ltd. Dispositif pour convertir le format de trame d'image
US6130660A (en) 1993-10-01 2000-10-10 Maxvision Corporation System and method for synthesizing high resolution video
US5510832A (en) 1993-12-01 1996-04-23 Medi-Vision Technologies, Inc. Synthesized stereoscopic imaging system and method
US5541660A (en) 1993-12-24 1996-07-30 Daewoo Electronics Co., Ltd. Systolic realization of motion compensated interpolation filter
EP0665697B1 (fr) 1994-02-01 2000-05-03 SANYO ELECTRIC Co., Ltd. Procédé et appareil pour la conversion d'images bidimensionnelles dans des images tridimensionnelles
US5739844A (en) 1994-02-04 1998-04-14 Sanyo Electric Co. Ltd. Method of converting two-dimensional image into three-dimensional image
US6326999B1 (en) 1994-08-23 2001-12-04 Discovision Associates Data rate conversion
US5682437A (en) 1994-09-22 1997-10-28 Sanyo Electric Co., Ltd. Method of converting two-dimensional images into three-dimensional images
US5649032A (en) 1994-11-14 1997-07-15 David Sarnoff Research Center, Inc. System for automatically aligning images to form a mosaic image
EP0735512B1 (fr) 1995-03-29 2002-06-05 SANYO ELECTRIC Co., Ltd. Méthodes pour la selection de deux trames d'une sequence d'images bidimensionnels, pour le calcul de profondeur relative d'objets d'image
EP0742674A2 (fr) 1995-05-08 1996-11-13 Kabushiki Kaisha Toshiba Méthode et système de codage vidéo employant un modèle débit-quantificateur
US5835627A (en) 1995-05-15 1998-11-10 Higgins; Eric W. System and method for automatically optimizing image quality and processing time
US5963664A (en) 1995-06-22 1999-10-05 Sarnoff Corporation Method and system for image combination using a parallax-based technique
US6522787B1 (en) 1995-07-10 2003-02-18 Sarnoff Corporation Method and system for rendering and combining images to form a synthesized view of a scene containing image information from a second image
US5739894A (en) 1995-08-25 1998-04-14 Weisgerber; Robert C. Method for exhibiting cinematic images photographed at two different frame rates
US5627614A (en) 1995-08-25 1997-05-06 Weisgerber; Robert C. Method for imparting both high-impact cinematic images and conventional cinematic images on the same strip of motion picture film
US5790686A (en) 1995-09-19 1998-08-04 University Of Maryland At College Park DCT-based motion estimation method
US5748199A (en) 1995-12-20 1998-05-05 Synthonics Incorporated Method and apparatus for converting a two dimensional motion picture into a three dimensional motion picture
US6477267B1 (en) 1995-12-22 2002-11-05 Dynamic Digital Depth Research Pty Ltd. Image conversion and encoding techniques
US6728317B1 (en) 1996-01-30 2004-04-27 Dolby Laboratories Licensing Corporation Moving image compression quality enhancement using displacement filters with negative lobes
US5946041A (en) 1996-01-31 1999-08-31 Fujitsu Limited Apparatus and method of tracking an image-feature using a block matching algorithm
EP0879441B1 (fr) 1996-02-07 2004-12-01 Maxivision Cinema Technology Systeme de projection de films a pas d'avancement reglable
US5946076A (en) 1996-02-07 1999-08-31 Maxivision Cinema Technology Switchable pulldown film projection system
US20010043310A1 (en) 1996-02-07 2001-11-22 Goodhill Dean K. Switchable format film projection system
US6019473A (en) 1996-02-07 2000-02-01 Maxivision Cinema Technology Switchable format film projection system
US20030048419A1 (en) 1996-02-07 2003-03-13 Goodhill Dean K. Switchable format film projection system
US5841512A (en) 1996-02-27 1998-11-24 Goodhill; Dean Kenneth Methods of previewing and editing motion pictures
US5812245A (en) 1996-03-26 1998-09-22 Goodhill; Dean K. Asynchronous contact printer for motion picture film
US5909516A (en) 1996-03-29 1999-06-01 Sarnoff Corporation Method and apparatus for decomposing an image stream into units of local contrast
US5738430A (en) 1996-03-29 1998-04-14 David Sarnoff Research Center, Inc. Method and apparatus for predicting retinal illuminance
US6075884A (en) 1996-03-29 2000-06-13 Sarnoff Corporation Method and apparatus for training a neural network to learn and use fidelity metric as a control mechanism
US5974159A (en) 1996-03-29 1999-10-26 Sarnoff Corporation Method and apparatus for assessing the visibility of differences between two image sequences
US5719966A (en) 1996-03-29 1998-02-17 David Sarnoff Research Center, Inc. Apparatus for assessing the visiblity of differences between two image sequences
US5694491A (en) 1996-03-29 1997-12-02 David Sarnoff Research Center, Inc. Methods and apparatus for assessing the visibility of differences between two image sequences
US6061477A (en) 1996-04-18 2000-05-09 Sarnoff Corporation Quality image warper
US20020021412A1 (en) 1996-05-08 2002-02-21 Goodhill Dean K. Method of making motion picture release-print film
US20030043344A1 (en) 1996-05-08 2003-03-06 Goodhill Dean K. Method of making motion picture release-print film
US20050024589A1 (en) 1996-05-08 2005-02-03 Goodhill Dean K. Method of making motion picture release-print film
US20040066491A1 (en) 1996-05-08 2004-04-08 Goodhill Dean K. Method of making motion picture release-print film
US5745213A (en) 1996-05-08 1998-04-28 Goodhill; Dean K. Method of making enhanced resolution motion picture release-print film
EP0920652B1 (fr) 1996-08-23 2003-12-10 Robert C. Weisgerber Procede servant a imprimer sur la meme bande d'un film cinematographique des images animees produisant une forte impression et des images animees classiques
US6108005A (en) 1996-08-30 2000-08-22 Space Corporation Method for producing a synthesized stereoscopic image
EP0837601A2 (fr) 1996-10-15 1998-04-22 Hitachi, Ltd. Méthode et circuit de traitement de signal pour une conversion de format d'un signal d'image
US20030036974A1 (en) 1996-12-03 2003-02-20 Richard Allen Apparatus and method for an on demand data delivery system for the preview selection, retrieval and reproduction at a remote location of previously recorded or programmed materials
US5864366A (en) 1997-02-05 1999-01-26 International Business Machines Corporation System and method for selecting video information with intensity difference
US6137904A (en) 1997-04-04 2000-10-24 Sarnoff Corporation Method and apparatus for assessing the visibility of differences between two signal sequences
US20020136293A1 (en) 1997-04-07 2002-09-26 Kinya Washino Wide-band multi-format audio/video production system with frame-rate conversion
US6370198B1 (en) 1997-04-07 2002-04-09 Kinya Washino Wide-band multi-format audio/video production system with frame-rate conversion
US5999220A (en) 1997-04-07 1999-12-07 Washino; Kinya Multi-format audio/video production system with frame-rate conversion
US6219462B1 (en) 1997-05-09 2001-04-17 Sarnoff Corporation Method and apparatus for performing global image alignment using any local match measure
US6067125A (en) 1997-05-15 2000-05-23 Minerva Systems Structure and method for film grain noise reduction
US6072907A (en) 1997-05-28 2000-06-06 Xerox Corporation Method and apparatus for enhancing and thresholding images
US6215516B1 (en) 1997-07-07 2001-04-10 Reveo, Inc. Method and apparatus for monoscopic to stereoscopic image conversion
US6031564A (en) 1997-07-07 2000-02-29 Reveo, Inc. Method and apparatus for monoscopic to stereoscopic image conversion
EP1002257B1 (fr) 1997-08-07 2004-11-03 Maxivision Cinema Technology Systeme de projection de films de differents formats
US6496598B1 (en) * 1997-09-02 2002-12-17 Dynamic Digital Depth Research Pty. Ltd. Image processing method and apparatus
US6141459A (en) 1997-09-24 2000-10-31 Sarnoff Corporation Method and apparatus for processing image pyramid borders
US6549240B1 (en) 1997-09-26 2003-04-15 Sarnoff Corporation Format and frame rate conversion for display of 24Hz source video
US5978029A (en) 1997-10-10 1999-11-02 International Business Machines Corporation Real-time encoding of video sequence employing two encoders and statistical analysis
US6191809B1 (en) 1998-01-15 2001-02-20 Vista Medical Technologies, Inc. Method and apparatus for aligning stereo images
US6560281B1 (en) 1998-02-24 2003-05-06 Xerox Corporation Method and apparatus for generating a condensed version of a video sequence including desired affordances
EP1084451B1 (fr) 1998-04-27 2004-06-16 Maxivision Cinema Technology Procede de production d'un film de diffusion d'oeuvres cinematographiques
US6266092B1 (en) 1998-05-12 2001-07-24 Genesis Microchip Inc. Method and apparatus for video line multiplication with enhanced sharpness
US6208348B1 (en) 1998-05-27 2001-03-27 In-Three, Inc. System and method for dimensionalization processing of images in consideration of a pedetermined image projection format
US6686926B1 (en) 1998-05-27 2004-02-03 In-Three, Inc. Image processing system and method for converting two-dimensional images into three-dimensional images
US6515659B1 (en) 1998-05-27 2003-02-04 In-Three, Inc. Method and system for creating realistic smooth three-dimensional depth contours from two-dimensional images
US6298090B1 (en) 1998-06-04 2001-10-02 U.S. Philips Corporation System for detecting redundant images in a video sequence by comparing two predetermined threshold values
US6269175B1 (en) 1998-08-28 2001-07-31 Sarnoff Corporation Method and apparatus for enhancing regions of aligned images using flow estimation
US6992700B1 (en) 1998-09-08 2006-01-31 Ricoh Company, Ltd. Apparatus for correction based upon detecting a camera shaking
US6229570B1 (en) 1998-09-25 2001-05-08 Lucent Technologies Inc. Motion compensation image interpolation—frame rate conversion for HDTV
GB2343316A (en) 1998-10-26 2000-05-03 Sony Uk Ltd Video interpolation
US6243156B1 (en) 1999-05-03 2001-06-05 Robert C. Weisgerber Method for exhibiting motion picture films at frame rates higher than that in which the films were originally produced
US6487304B1 (en) 1999-06-16 2002-11-26 Microsoft Corporation Multi-view approach to motion and stereo
US6618439B1 (en) 1999-07-06 2003-09-09 Industrial Technology Research Institute Fast motion-compensated video frame interpolator
CA2279797C (fr) 1999-08-06 2010-01-05 Demin Wang Methode d'interpolation temporelle d'une sequence d'images au utilisant une analyse d'image fondee sur les objets
US6625333B1 (en) 1999-08-06 2003-09-23 Her Majesty The Queen In Right Of Canada As Represented By The Minister Of Industry Through Communications Research Centre Method for temporal interpolation of an image sequence using object-based image analysis
US20030050940A1 (en) 1999-10-12 2003-03-13 Eric Robinson Automatic backup system
EP1234447A4 (fr) 1999-11-05 2008-03-12 Dolby Lab Licensing Corp Systeme et procede de compensation du mouvement et de conversion de la frequence des trames
US6442203B1 (en) 1999-11-05 2002-08-27 Demografx System and method for motion compensation and frame rate conversion
US6351545B1 (en) 1999-12-14 2002-02-26 Dynapel Systems, Inc. Motion picture enhancing system
US20010031003A1 (en) 1999-12-20 2001-10-18 Sawhney Harpreet Singh Tweening-based codec for scaleable encoders and decoders with varying motion computation capability
US20050036113A1 (en) 2000-01-12 2005-02-17 Goodhill Dean K. System and method for registering motion picture film
EP1247138B1 (fr) 2000-01-12 2004-10-13 Maxivision Cinema Technology Systeme et procede d'enregistrement d'un film cinematographique
US6450644B1 (en) 2000-01-12 2002-09-17 Maxivision Cinema Technology System and method for registering motion picture film
US20040057015A1 (en) 2000-01-12 2004-03-25 Goodhill Dean K. System and method for registering motion picture film
US20030011746A1 (en) 2000-01-12 2003-01-16 Goodhill Dean K. System and method for registering motion picture film
US6806898B1 (en) 2000-03-20 2004-10-19 Microsoft Corp. System and method for automatically adjusting gaze and head orientation for video conferencing
US7254265B2 (en) * 2000-04-01 2007-08-07 Newsight Corporation Methods and systems for 2D/3D image conversion and optimization
US20030128871A1 (en) * 2000-04-01 2003-07-10 Rolf-Dieter Naske Methods and systems for 2D/3D image conversion and optimization
US20020035432A1 (en) 2000-06-08 2002-03-21 Boguslaw Kubica Method and system for spatially indexing land
US6900846B2 (en) 2000-06-13 2005-05-31 Samsung Electronics Co., Ltd. Format converter using bi-directional motion vector and method thereof
EP1164792A2 (fr) 2000-06-13 2001-12-19 Samsung Electronics Co., Ltd. Convertisseur de format utilisant des vecteurs de mouvement bidirectionnels et méthode correspondante
US6573912B1 (en) 2000-11-07 2003-06-03 Zaxel Systems, Inc. Internet system for virtual telepresence
EP1354292A1 (fr) 2000-12-01 2003-10-22 Imax Corporation Techniques et systemes pour la mise au point d'imagerie haute resolution
CA2430591A1 (fr) 2000-12-01 2002-06-06 Imax Corporation Techniques et systemes pour la mise au point d'imagerie haute resolution
US7260274B2 (en) 2000-12-01 2007-08-21 Imax Corporation Techniques and systems for developing high-resolution imagery
EP1217839A2 (fr) 2000-12-08 2002-06-26 Matsushita Electric Industrial Co., Ltd. Dispositif de conversion de données, dispositif de codage de données, et dispositif d'enregistrement de données
US20030016750A1 (en) 2001-02-23 2003-01-23 Eastman Kodak Company Frame-interpolated variable-rate motion imaging system
US20020149696A1 (en) 2001-02-23 2002-10-17 Eastman Kodak Company Method for presenting improved motion image sequences
EP1235426A2 (fr) 2001-02-23 2002-08-28 Eastman Kodak Company Procédé de présentation de séquences d'images animées améliorées
EP1237370A2 (fr) 2001-02-23 2002-09-04 Eastman Kodak Company Système d'interpolation des trames d'une image mouvante à vitesse variable
US6950462B2 (en) 2001-03-08 2005-09-27 Matsushita Electric Industrial Co., Ltd. Video encoding apparatus, video encoding method, and frame rate conversion apparatus
US7079697B2 (en) 2001-03-19 2006-07-18 Texas Instruments Incorporated Image compression with transform coefficient analysis
US6968006B1 (en) 2001-06-05 2005-11-22 At&T Corp. Method of content adaptive video decoding
US20030169404A1 (en) 2001-07-16 2003-09-11 Weisgerber Robert C. Method for exhibiting high-quality three-dimensional motion from a single strip of motion picture film
JP4201165B2 (ja) 2001-09-04 2008-12-24 住化エンビロサイエンス株式会社 工業用防カビ組成物
US7227896B2 (en) 2001-10-04 2007-06-05 Sharp Laboratories Of America, Inc. Method and apparatus for global motion estimation
US20030095704A1 (en) 2001-11-22 2003-05-22 Risson Valery J. Determination of an illuminant of digital color image by segmentation and filtering
US20060120451A1 (en) 2002-01-23 2006-06-08 Nokia Corporation Grouping of image frames in video coding
US7006157B2 (en) 2002-02-19 2006-02-28 Samsung Electronics Co., Ltd. Apparatus and method for converting frame rate
US20040130680A1 (en) 2002-03-13 2004-07-08 Samuel Zhou Systems and methods for digitally re-mastering or otherwise modifying motion pictures or other image sequences data
EP1483909A1 (fr) 2002-03-13 2004-12-08 Imax Corporation Systemes et procedes permettant de remasterisation ou de modification numerique de films cinematographiques ou autres donnees de sequences d'images
US7856055B2 (en) 2002-03-13 2010-12-21 Imax Corporation Systems and methods for digitally re-mastering or otherwise modifying motion pictures or other image sequences data
US20040017510A1 (en) 2002-03-18 2004-01-29 Pioneer Corporation Video signal processing apparatus
US6856314B2 (en) 2002-04-18 2005-02-15 Stmicroelectronics, Inc. Method and system for 3D reconstruction of multiple views with altering search path and occlusion modeling
US20030204384A1 (en) 2002-04-24 2003-10-30 Yuri Owechko High-performance sensor fusion architecture
US20040022318A1 (en) 2002-05-29 2004-02-05 Diego Garrido Video interpolation coding
US7119837B2 (en) 2002-06-28 2006-10-10 Microsoft Corporation Video processing system and method for automatic enhancement of digital video
EP1397003A1 (fr) 2002-08-07 2004-03-10 Micronas GmbH Méthode pour l'interpolation d'un pixel contenu dans une image intermédiaire d'une suite d'image à l'aide de vecteurs de mouvement
US20040046891A1 (en) 2002-09-10 2004-03-11 Kabushiki Kaisha Toshiba Frame interpolation and apparatus using frame interpolation
US20040136686A1 (en) 2002-11-07 2004-07-15 Seiko Epson Corporation Conversion of frame rate according to image data
EP1571854A1 (fr) 2002-11-25 2005-09-07 Sanyo Electric Co., Ltd. Procede de creation d'images video stereoscopiques et affichage d'images video stereoscopiques
US7689031B2 (en) * 2002-12-30 2010-03-30 Koninklijke Philips Electronics N.V. Video filtering for stereo images
EP1455338A1 (fr) 2003-03-06 2004-09-08 Pioneer Corporation Circuit de multiplication de la fréquence de trame
JP2004274219A (ja) 2003-03-06 2004-09-30 Pioneer Electronic Corp 映像信号のフレームレート変換装置
US20040202445A1 (en) 2003-03-20 2004-10-14 Pixar Component color flat panel digital film recorder and method
JP2004312669A (ja) 2003-03-25 2004-11-04 Toshiba Corp フレーム補間のための補間画像作成方法及びこれを用いた画像表示システム、補間画像作成装置
US20040246374A1 (en) 2003-03-28 2004-12-09 Nao Mishima Method of generating frame interpolation image and an apparatus therefor
JP2004297719A (ja) 2003-03-28 2004-10-21 Toshiba Corp フレーム補間方法及びこれを用いた画像表示システム
US20040201777A1 (en) 2003-04-11 2004-10-14 Huaya Microelectronics (Shanghai) Inc. Frame rate conversion of interlaced and progressive video streams
US20040223640A1 (en) 2003-05-09 2004-11-11 Bovyrin Alexander V. Stereo matching using segmentation of image columns
JP2004343715A (ja) 2003-05-13 2004-12-02 Samsung Electronics Co Ltd フレームレート変換時のフレーム補間方法およびその装置
US20040227851A1 (en) 2003-05-13 2004-11-18 Samsung Electronics Co., Ltd. Frame interpolating method and apparatus thereof at frame rate conversion
US20040236807A1 (en) 2003-05-23 2004-11-25 Via Technologies, Inc. Method and apparatus for adaptive frame rate conversion
JP2004348117A (ja) 2003-05-24 2004-12-09 Samsung Electronics Co Ltd 映像補間装置および映像補間方法
JP2004357215A (ja) 2003-05-30 2004-12-16 Toshiba Corp フレーム補間方法及び装置並びに画像表示システム
US20050053291A1 (en) 2003-05-30 2005-03-10 Nao Mishima Frame interpolation method and apparatus, and image display system
US20040252230A1 (en) 2003-06-13 2004-12-16 Microsoft Corporation Increasing motion smoothness using frame interpolation with motion analysis
US20040252759A1 (en) 2003-06-13 2004-12-16 Microsoft Corporation Quality control in frame interpolation with motion analysis
US20050002456A1 (en) 2003-07-04 2005-01-06 Min Jong-Sul Motion vector detector for frame rate conversion and method thereof
EP1503582A1 (fr) 2003-07-31 2005-02-02 Pioneer Corporation Appareil pour la conversion d'un signal video
JP2005051389A (ja) 2003-07-31 2005-02-24 Pioneer Electronic Corp 映像信号変換装置
EP1519358A2 (fr) 2003-09-25 2005-03-30 Canon Kabushiki Kaisha Circuit de conversion de la fréquence de trame, méthode de prévision de dépassement pour ce circuit , dispositif de commande d'affichage et dispositif d'affichage avec récepteur vidéo
JP2005124167A (ja) 2003-09-25 2005-05-12 Canon Inc フレームレート変換装置、それに用いられる追い越し予測方法、表示制御装置及び映像受信表示装置
US20050105001A1 (en) 2003-09-25 2005-05-19 Canon Kabushiki Kaisha Frame rate conversion device, overtaking prediction method for use in the same, display control device and video receiving display device
EP1549054A2 (fr) 2003-12-23 2005-06-29 Genesis Microchip, Inc. Conversion de fréquence de trame à compensation de motion
JP2005210697A (ja) 2003-12-23 2005-08-04 Genesis Microchip Inc 動き補償フレームレート変換
JP2005204075A (ja) 2004-01-15 2005-07-28 Sony Corp 動きベクトル検出装置とそれを用いたフレーム補間装置、動きベクトル検出方法とそれを用いたフレーム補間方法およびプログラムと記録媒体。
US20050226529A1 (en) 2004-02-19 2005-10-13 Sony Corporation Signal processing apparatus and method, and command-sequence data structure
EP1585326A1 (fr) 2004-03-30 2005-10-12 Matsushita Electric Industrial Co., Ltd. Estimation de vecteur de mouvement aux contours de l'image pour conversion de la fréquence de rafraíchissement
EP1583364A1 (fr) 2004-03-30 2005-10-05 Matsushita Electric Industrial Co., Ltd. Interpolation d'images à mouvements compensés aux contours de l'image pour conversion de fréquence d'image
US20050265451A1 (en) 2004-05-04 2005-12-01 Fang Shi Method and apparatus for motion compensated frame rate up conversion for block-based low bit rate video
US20050254011A1 (en) 2004-05-17 2005-11-17 Weisgerber Robert C Method for exhibiting motion picture films at a higher frame rate than that in which they were originally produced
JP2005057809A (ja) 2004-10-25 2005-03-03 Matsushita Electric Ind Co Ltd シネマ信号作成システムおよび撮像装置
US20080055490A1 (en) 2004-12-03 2008-03-06 Sony Computer Entertainment Inc. Image Display Device, Image Display Method, and Program
US20060165179A1 (en) 2005-01-27 2006-07-27 Technion Research & Development Foundation Ltd. Acquisition of image sequences with enhanced resolution
US7227125B2 (en) 2005-06-29 2007-06-05 Mitsumi Electric Co., Ltd. Encoder device
US20070019883A1 (en) * 2005-07-19 2007-01-25 Wong Earl Q Method for creating a depth map for auto focus using an all-in-focus picture and two-dimensional scale space matching
CN101375315A (zh) 2006-01-27 2009-02-25 图象公司 数字重制2d和3d运动画面以呈现提高的视觉质量的方法和系统
US20100231593A1 (en) 2006-01-27 2010-09-16 Samuel Zhou Methods and systems for digitally re-mastering of 2d and 3d motion pictures for exhibition with enhanced visual quality
EP1977395A2 (fr) 2006-01-27 2008-10-08 Imax Corporation Procedes et systemes pour le rematriçage numerique de films bi- et tridimensionnels pour une presentation avec une qualite visuelle amelioree
CN101479765A (zh) 2006-06-23 2009-07-08 图象公司 对2d电影进行转换用于立体3d显示的方法和系统
US20090116732A1 (en) 2006-06-23 2009-05-07 Samuel Zhou Methods and systems for converting 2d motion pictures for stereoscopic 3d exhibition
EP2160037A2 (fr) 2006-06-23 2010-03-03 Imax Corporation Procédés et systèmes de conversion d'images cinématographiques 2D pour une représentation stéréoscopique 3D
EP2033164A2 (fr) 2006-06-23 2009-03-11 Imax Corporation Procédés et systèmes de conversion d'images cinématographiques 2d pour une représentation stéréoscopique 3d
US20100182406A1 (en) * 2007-07-12 2010-07-22 Benitez Ana B System and method for three-dimensional object reconstruction from two-dimensional images
US20110069152A1 (en) * 2009-09-24 2011-03-24 Shenzhen Tcl New Technology Ltd. 2D to 3D video conversion
US20120056984A1 (en) * 2010-09-03 2012-03-08 Samsung Electronics Co., Ltd. Method and apparatus for converting 2-dimensional image into 3-dimensional image by adjusting depth of the 3-dimensional image

Non-Patent Citations (110)

* Cited by examiner, † Cited by third party
Title
Ahmadi, et al., "Improved Algorithms for Phase Prediction and Frame Interpolation in Low Bit Rate Sinusoidal Coders", Conference Record of Thirty-Second Asilomar Conference on Signals, Systems and Computers, 1998,vol. 1, pp. 362-366.
Al-Mualla, et al., "Motion Field Interpolation for Frame Rate Conversion", Proceedings of the IEEE International Symposium on Circuits and Systems, 2003, vol. 2, pp. 652-655.
Bagnara, et al., "Frame Interpolation in Low Bit Rate Coding by 3D Motion Estimation", Image and Multidimensional Digital Signal Processing, 1998, pp. 127-130.
Baker, et al., "A Layered Approach to Stereo Reconstruction," Proceedings of Computer Vision and Pattern Recognition Conference, Jun. 1998, pp. 434-441.
Baroncini, et al. "The Image Resolution of 35mm Cinema Film in Theatrical Presentation," SMPTE Imaging Journal, vol. 113, No. 2&3, Feb./Mar. 2004, pp. 60-66.
Bergen, et al., "Hierarchical model-based motion estimation", In Proceedings 2nd European Conference on Computer vision, 1992, pp. 237-252.
Biswas, et al., "A Novel Motion Estimation Algorithm Using Phase Plane Correlation for Frame Rate Conversion", Conference Record of the Thirty-Sixth Asilomar Conference on Signals Systems and Computers, 2002, vol. 1, pp. 492-496.
Borcsok, et al. "Generation of 3D Image Sequences from Mixed 2D and 3D Image Sources," SCI 2001, The 5th Multi-Conference on Systemics, Cybernetics and Informatics, Jul. 22-25, Florida, USA, 2001.
Burt, et al., "The Laplacian Pyramid as a Compact Image Code," IEEE Transactions on Communications, vol. 31 (4), pp. 532-540 (1983).
Canadian Application No. 2,430,591, Notice of Allowance mailed on Jan. 6, 2011, 1 Page.
Canadian Application No. 2,430,591, Office Action mailed on Apr. 12, 2010, 3 Pages.
Canadian Application No. 2,478,671, Office Action mailed on Apr. 29, 2009, 2 Pages.
Canadian Application No. 2,478,671, Office Action mailed on Dec. 6, 2007, 2 Pages.
Canadian Application No. 2,478,671, Office Action mailed on Jul. 20, 2010, 2 Pages.
Castagno, et al., "A Method for Motion Adaptive Frame Rate Up-Conversion", IEEE Transactions on Circuits and Systems for Video Technology, Oct. 1996, vol. 6, No. 5, pp. 436-446.
Chen, et al., "A New Frame Interpolation Scheme for Talking Head Sequences", Proceedings of International Conference on Image Processing, 1995, vol. 2, pp. 591-594.
Chinese Application No. 03810057.6, Office Action mailed on Jun. 6, 2008, 11 Pages.
Chinese Application No. 03810057.6, Office Action mailed on Nov. 14, 2008, 6 Pages.
Chinese Application No. 200780003734.3, Office Action mailed on Aug. 6, 2010, 15 Pages of Office Action and 2 Pages of English Translation.
Chinese Application No. 200780023673.7, Office Action mailed on Apr. 1, 2011, 3 Pages.
Dane, et al., "Encoder-Assisted Adaptive Video Frame Interpolation", IEEE International Conference on Acoustics, Speech and Signal Processing, 2005, vol. 2, pp. 349-352.
Dane, et al., "Motion Vector Processing for Frame Rate Up Conversion", IEEE International Conference on Acoustics, Speech and Signal Processing, 2004, vol. 3, pp. 309-312.
Debevec, et al., "Modeling and Rendering Architecture from Photographs: A Hybrid Geometry- and Image-based Approach," Proceedings of SIGGRAPH '96, New Orleans, Louisiana, Aug. 4-9, 1996, pp. 11-20.
Dunkley, "A New 3-D from 2-D Virtual Display Process," Proceedings of SPIE, vol. 1915, Stereo Displays and Applications IV, San Jose, Feb. 1, 1993, San Jose, CA, pp. 132-140.
Ernst, "Motion Compensated Interpolation for Advanced Standards Conversion and Noise Reduction," Signal Processing of HDTV, III, pp. 319-328 (Sep. 4, 1991).
European Application No. 01 989 802.2-2218, Communication pursuant to Article 94(3) EPC mailed on Sep. 9, 2010, 4 pages.
European Application No. 01989802.2, European Search Report mailed on Mar. 23, 2009, 4 Pages.
European Application No. 01989802.2, Office Action mailed on Sep. 2, 2010, 4 Pages.
European Application No. 09153591.4, European Search Report mailed on Oct. 14, 2010, 8 Pages.
European Patent Application No. 07789439.2 , Article 94(3) EPC Communication mailed Mar. 30, 2012 ( 5 pages).
European Patent Application No. 07789439.2, Extended European Search Report, mailed Oct. 14, 2010 (6 pages).
European Patent Application No. 09153591.4, Extended European Search Report, mailed Oct. 14, 2010 (8 pages).
Feldman, et al., "Interactive 2D to 3D stereoscopic image synthesis," Proceedings of SPIE vol. 5664 Stereoscopic Displays and Virtual Reality Systems XII, San Jose, CA, Jan. 17-19, 2005.
Garcia, "Approaches to Stereoscopic Video Based on Spatio-Temporal Interpolation," Proceedings of SPIE, vol. 2653, Stereoscopic Displays and Virtual Reality Systems III, Jan. 30-Feb. 1, 1996, San Jose, CA, pp. 85-95.
Ha, et al., "Motion Compensated Frame Interpolation by New Block-Based Motion Estimation Algorithm", IEEE Transactions on Consumer Electronics, May 2004, vol. 50, No. 2, pp. 752-759.
Ha, et al., "Motion Compensated Frame Rate Conversion by Overlapped Block-Based Motion Estimation Algorithm", IEEE International Symposium on Consumer Electronics, 2004, pp. 345-350.
Hanna, "Direct multi-resolution estimation of ego-motion and structure from motion". In Proceedings IEEE Workshop on Visual Motion, Nassau Inn, Princeton, New Jersey, Oct. 7-9, 1991, pp. 156-162.
Hanna, et al., "Combining stereo and motion analysis for direct estimation of scene structure". In Proceedings IEEE International Conference on Computer Vision, Berlin, Germany, pp. 357-365, May 11-14, 1993.
Harman, "An Architecture for Digital 3D Broadcasting," Proceedings of SPIE, vol. 3639, Stereoscopic Displays and Virtual Reality Systems VI, Jan. 25-28, 1999, San Jose, CA, pp. 254-259.
Harman, "Home Based 3D Entertainment-An Overview," Proceedings of IEEE International Conference on Image Processing, Sep. 10-13, 2000, Vancouver, Canada, pp. 1-4.
Harman, et al., "Rapid 2D-to-3D Conversion," Proceedings of SPIE, vol. 4660, Stereoscopic Displays and Virtual Reality Systems IX, Jan. 21-24, 2002, San Jose, CA, pp. 78-86.
Healey, et al., "Global Color Constancy: Recognition of objects by use of illumination invariant properties of color distributions," Journal of the Optical Society of America A, 11(11):3003-3010, Nov. 1994.
Hilman, et al., "Using Motion-Compensated Frame-Rate Conversion for the Correction of 3:2 Pull down Artifacts in Video Sequences", IEEE Transactions on Circuits and Systems for Video Technology, Sep. 2000,vol. 10, No. 6, pp. 869-877.
Hoberman, "Depth Painting: the Interactive Transformation of Existing Images into Stereoscopic 3D," Proceedings of SPIE, vol. 2177, Stereoscopic Displays and Virtual Reality Systems, Feb. 8-10, 1994, San Jose, CA, pp. 78-85.
Hodges, "Tutorial: Time-Multiplexed Stereoscopic Computer Graphics," IEEE Computer Graphics and Applications, Mar. 1992, pp. 20-20.
Horry, et al. "Tour Into the Picture: Using a Spidery Mesh Interface to Make Animation from a Single Image," Proceedings of Siggraph '97, Aug. 3-8, 1997, LA, CA, pp. 225-232.
Indian Application No. 1303/KOLNP/2004, Office Action mailed on Sep. 26, 2006, 4 pages.
International Preliminary Examination Report for PCT/IB2003/00919, dated Mar. 24, 2004.
International Preliminary Examination report for PCT/IB2007/000188, Jul. 29, 2008.
International Preliminary Examination report for PCT/IB2007/001726, dated Jan. 6, 2009.
International Preliminary Examination report for PCT/US2001/44995, dated Aug. 4, 2002.
International Search Report for PCT/IB2003/00919, dated Dec. 8, 2003.
International Search Report for PCT/IB2007/000188, dated Jun. 26, 2007.
International Search Report for PCT/IB2007/001726, dated Feb. 12, 2008.
International Search Report for PCT/US2001/44995, dated Apr. 4, 2002.
Japanese Patent Application No. 03810057.6 , Decision Reversing Rejection of Application mailed Dec. 21, 2011 ( 35 pages).
Jeon, et al., "Coarse-To-Fine Frame Interpolation for Frame Up-Conversion Using Pyramid Structure", IEEE Transactions on Consumer Electronics, Aug. 2003, vol. 49, No. 3, pp. 499-508.
Karim, et al., "Low Rate Video Frame Interpolation-Challenges and Solutions", IEEE International Conference on Acoustic, Speech and Signal Processing, vol. 3, pp. 117-120.
Karim, et al., "Multiresolution Motion Estimation for Low-Rate Video Frame Interpolation", EURASIP Journal on Applied Signal Processing, 2004, No. 11, pp. 1708-1720.
Kennel, "Digital Film Scanning and Recording: The Technology and Practice," SMPTE Journal, pp. 174-181 (1994).
Kim, et al., "A New Video Interpolation Technique Based on Motion-Adaptive Subsampling", Digest of Technical Papers, International Conference on Consumer electronics, 1999, pp. 370-371.
Kim, et al., "A New Video Interpolation Technique Based on Motion-Adaptive Subsampling", IEEE Transactions on Consumer Electronics, Aug. 1999, vol. 45, No. 3, pp. 782-787.
Kim, et al., "Stereoscopic Conversion of Monoscopic Video by the Transformation of Vertical-to-horizontal Disparity," Proceedings of SPIE, vol. 3295, Stereo Displays and Virtual Reality Systems V, San Jose, CA, Jan. 26-29, 1998, pp. 65-75.
Kim, et al., "Synthesis of a High-resolution 3D-stereoscopic Image Pair from a High Resolution Monoscopic Image and a Low-resolution Depth Map," Proceedings of SPIE, vol. 3295, Stereo Displays and Virtual Reality Systems V, San Jose, CA, Jan. 26-29, 1998, pp. 76-86.
Krishnamurthy, et al., "Frame Interpolation and Bidirectional Prediction of Video Using Compactly encoded Optical-Flow fields and Label Fields", IEEE Transactions on Circuits and Systems for Video Technology, Aug. 1999, vol. 9, No. 5, pp. 713-726.
Kumar, et al., "3D Manipulation of Motion Imagery," Image Processing, 2000. Proceedings. 2000 International Conference on Sep. 10-13, 2000, Piscataway, NJ, USA, IEEE, pp. 17-20, Sarnoff Corporation.
Kumar, et al., "Direct multi-resolution estimation of ego-motion and structure from motion". In Proceedings IEEE Workshop on Visual Motion, Nassau Inn, Princeton, New Jersey, Oct. 7-9, 1991, pp. 156-162.
Kumar, et al., "Representation of scenes from collection of images," In Proc. IEEE Wkshp. on Representation of Visual Scenes, Cambridge, MA 1995.
Kuo, et al., "Motion-Compensated Frame Interpolation Scheme for H.263 Codec", Proceeding of the IEEE International Symposium on Circuits and Systems VLSI, 1999, vol. 4, pp. 491-494.
Lagendijk, et al., "Motion Compensated Frame Rate Conversion of Motion Pictures", IEEE International Conference on Acoustics, Speech and Signal Processing, 1992, vol. 3, pp. 453-456.
Lee, et al., "Hierarchical Motion Compensated Frame Rate Up-Conversion Based on the Gaussian/Laplacian Pyramid", Digest of Technical Papers, International Conference on Consumer Electronics, 2003, pp. 350-351.
Levinson, et al., "Development of an Autostereoscopic Monitor and 2D to 3D Conversion for Medical and Surgical Uses; Requirements, Clinical Trials and Degree of Acceptance," Proceedings of SPIE, vol. 3639, Stereoscopic Displays and Virtual Reality Systems VI, Jan. 25-28, 1999, San Jose, CA, pp. 36-43.
Liu, et al., "Frame Interpolation Scheme Using Inertia Motion Prediction", Signal Processing: Image Communication, Mar. 2003, vol. 18, No. 3, pp. 221-229.
Matsumoto, et al., "Conversion System of Monocular Image Sequence to Stereo using Motion Parallax," Proceedings of SPIE, vol. 3012, Stereo Displays and Virtual Reality Systems IV, San Jose, Feb. 11-14, 1997, pp. 108-115.
McAllister, "Stereo Pairs from Linear Morphing," Proceedings of SPIE, vol. 3295, Stereoscopic Displays and Virtual Reality Systems V, Jan. 26-29, 1998, San Jose, CA, pp. 46-52.
Mishima, et al., "Novel Frame Interpolation Method for Hold-Type Displays", International Conference on Image Processing, 2004, vol. 3, pp. 1473-1476.
Morton, et al., "Assessing the Quality of Motion Picture Systems from Scene-to-Digital Data," SMPTE Journal, vol. 111, No. 2, Feb./Mar. 2002, pp. 85-96.
Murata, et al., "A Real-Time 2-D to 3-D Image Conversion Technique Using Computed Image Depth," SID Symposium Digest of Technical Papers, May 1998-vol. 29, Issue 1, pp. 919-923.
Okino, et al., "New Television with 2D/3D image conversion technologies," Proceedings of SPIE, vol. 2653, Stereoscopic Displays and Virtual Reality Systems III, Jan. 30-Feb. 1, 1996, San Jose, CA, pp. 96-105.
Panabaker, et al. "IMAX HD-A High Definition Motion Picture System," 135th SMPTE Technical Conference, Oct. 29-Nov. 2, 1993, Los Angeles, CA.
R. Kumar, et al., "Direct recovery of shape from multiple views: A parallax based approach". In International Conference on Pattern Recognition, pp. 685-688 (1994).
Rotem, et al., "Automatic video-to-stereoscopic-video conversion," Proceedings of SPIE vol. 5664 Stereoscopic Displays and Virtual Reality Systems XII, San Jose, CA, Jan. 17-19, 2005.
Sawhney, "3D Geometry from Planar Parallax," In Proc. IEEE Computer Vision and Pattern Recognition Conference, Seattle, WA, Jun. 21-23, 1994.
Sawhney, "Hybrid Stereo Camera: An IBR Approach for Synthesis of Very High Resolution Stereoscopic Image Sequences," Computer Graphics. Siggraph 2001. Conference Proceedings, Los Angeles, CA, Aug. 12-17, 2001, pp. 451-460.
Scharstein, "stereo vision for view synthesis," IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'96), pp. 852-858, San Francisco, CA, Jun. 1996.
Sezan, et al., "Frame Rate Conversion of Motion Picture Films for Progressive-Scan HDTV", Signal Processing VI-Theories and Applications, Sixth European Signal Processing Conference, 1992, vol. 3, pp. 1279-1282.
Shao, et al., "Automatic 2D to 3D Footage Conversion for 3D Glasses-free Display Systems," ICCV 2003 demonstrations, Oct. 13-16, 2003.
Shin, et al., "High-Resolution Image Sequence Interpolation", Proceedings for IEEE TENCON. IEEE Region 10 Annual Conference on Speech and Image Technologies for Computing and Telecommunications, 1997, vol. 2, pp. 781-784.
Skoneczny, et al., "Classical and Neural methods of Image Sequence", Proceedings of the SPIE, The International Society for Optical Engineering, 2001, vol. 4535, pp. 191-204.
Slater, et al., "The Illumination-Invariant Recognition of 3-D Objects Using Local Color Invariants," IEEE Transactions on Pattern Analysis and Machine Intelligence, 18(2):206-210, Feb. 1996.
Stiller, et al., "Estimating Motion in Image Sequences," IEEE Signal Processing Magazine, Jul. 1999, pp. 70-91.
Supplementary European Search Report for Application No. 01989802.2, dated Mar. 23, 2009.
Szeliski, "Scene Reconstruction from Multiple Cameras," Proceedings of IEEE International Conference on Image Processing, Sep. 10-13, 2000, Vancouver, Canada, pp. 13-16.
Szostakowski, et al., "Missing frames interpolation by artificial neural networks", Engineering Benefits from Neural Networks. Proceedings of the International conference EANN '98, 1998, 2 pages.
Tam, et al., "Stereoscopic Image Rendering Based on Depth Maps Created from Blur and Edge Information," Proceedings of SPIE vol. 5664 Stereoscopic Displays and Virtual Reality Systems XII, San Jose, CA, Jan. 17-19, 2005.
U.S. Appl. No. 10/433,338, Notice of Allowance mailed on May 4, 2007, 7 Pages.
U.S. Appl. No. 10/433,338, Office Action mailed Oct. 5, 2006, 11 Pages.
U.S. Appl. No. 10/474,780, Notice of Allowance mailed on Feb. 4, 2010, 13 Pages.
U.S. Appl. No. 10/474,780, Office Action mailed on Apr. 16, 2008, 17 Pages.
U.S. Appl. No. 10/474,780, Office Action mailed on Apr. 5, 2007, 15 Pages.
U.S. Appl. No. 10/474,780, Office Action mailed on Jan. 6, 2009, 18 Pages.
U.S. Appl. No. 10/474,780, Office Action mailed on Jun. 25, 2009, 17 Pages.
Valencia, et al., "Synthesizing stereo 3D views from focus cues in monoscopic 2D images," Proceedings of SPIE, vol. 5006, Stereoscopic Displays and Virtual Reality Systems X, Jan. 21-23, 2003, Santa Clara, CA.
Wang, et al., "A Fast Block-Based Motion Compensation Video Frame Interpolation", Conference Record of the Thirty-Eighth Asilomar Conference on Signals, Systems and Computers, 2004, vol. 2, pp. 1740-1743.
Wang, et al., "Comparison of Motion-Compensated Algorithms for Frame Interpolation", Optical Engineering, Feb. 2003, vol. 42, No. 2, pp. 586-590.
Wang, et al., "On Motion-Compensated Algorithms for Frame Interpolation", Proceedings of the Fourth IASTED International Conference on Signal and Image Processing, 2002, pp. 419-423.
Wittebrood, et al., "Tackling Occlusion in Scan Rate Conversion Systems", Digest of Technical Papers, International Conference on Consumer Electronics, 2003, pp. 344-345.
Yoon, et al., "Frame Interpolation Using Transmitted Block-Based Motion Vectors", Proceedings International Conference on Image Processing, 2001, vol. 3, pp. 856-859.
Zhai, et al., "A Low Complexity Motion Compensated Frame Interpolation Method", IEEE International Symposium on Circuits and Systems, 2005, vol. 5, pp. 4927-4930.
Zhao, et al., "A New Algorithm for Motion-Compensated Frame Interpolation", IEEE International Symposium on Circuits and Systems, May 1993, vol. 1, pp. 9-12.

Cited By (100)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100231593A1 (en) * 2006-01-27 2010-09-16 Samuel Zhou Methods and systems for digitally re-mastering of 2d and 3d motion pictures for exhibition with enhanced visual quality
US8842730B2 (en) 2006-01-27 2014-09-23 Imax Corporation Methods and systems for digitally re-mastering of 2D and 3D motion pictures for exhibition with enhanced visual quality
US10869611B2 (en) 2006-05-19 2020-12-22 The Queen's Medical Center Motion tracking system for real time adaptive imaging and spectroscopy
US9076212B2 (en) 2006-05-19 2015-07-07 The Queen's Medical Center Motion tracking system for real time adaptive imaging and spectroscopy
US9138175B2 (en) 2006-05-19 2015-09-22 The Queen's Medical Center Motion tracking system for real time adaptive imaging and spectroscopy
US9867549B2 (en) 2006-05-19 2018-01-16 The Queen's Medical Center Motion tracking system for real time adaptive imaging and spectroscopy
US9282313B2 (en) 2006-06-23 2016-03-08 Imax Corporation Methods and systems for converting 2D motion pictures for stereoscopic 3D exhibition
US20160191905A1 (en) * 2008-03-26 2016-06-30 Ricoh Company, Ltd. Adaptive Image Acquisition and Display Using Multi-focal Display
US9865043B2 (en) * 2008-03-26 2018-01-09 Ricoh Company, Ltd. Adaptive image acquisition and display using multi-focal display
US9342914B2 (en) * 2009-09-30 2016-05-17 Disney Enterprises, Inc. Method and system for utilizing pre-existing image layers of a two-dimensional image to create a stereoscopic image
US20130321408A1 (en) * 2009-09-30 2013-12-05 Disney Enterprises, Inc. Method and system for utilizing pre-existing image layers of a two-dimensional image to create a stereoscopic image
US8971584B2 (en) 2009-10-30 2015-03-03 Adobe Systems Incorporated Methods and apparatus for chatter reduction in video object segmentation using a variable bandwidth search region
US8520975B2 (en) * 2009-10-30 2013-08-27 Adobe Systems Incorporated Methods and apparatus for chatter reduction in video object segmentation using optical flow assisted gaussholding
US9042636B2 (en) 2009-12-31 2015-05-26 Disney Enterprises, Inc. Apparatus and method for indicating depth of one or more pixels of a stereoscopic 3-D image comprised from a plurality of 2-D layers
US8849012B2 (en) * 2010-03-30 2014-09-30 Fujifilm Corporation Image processing apparatus and method and computer readable medium having a program for processing stereoscopic image
US20110243384A1 (en) * 2010-03-30 2011-10-06 Fujifilm Corporation Image processing apparatus and method and program
US9282319B2 (en) 2010-06-02 2016-03-08 Nintendo Co., Ltd. Image display system, image display apparatus, and image display method
US8780183B2 (en) 2010-06-11 2014-07-15 Nintendo Co., Ltd. Computer-readable storage medium, image display apparatus, image display system, and image display method
US10015473B2 (en) 2010-06-11 2018-07-03 Nintendo Co., Ltd. Computer-readable storage medium, image display apparatus, image display system, and image display method
US9795882B1 (en) 2010-06-24 2017-10-24 Gregory S. Rabin Interactive system and method
US9053562B1 (en) * 2010-06-24 2015-06-09 Gregory S. Rabin Two dimensional to three dimensional moving image converter
US20120218382A1 (en) * 2010-08-02 2012-08-30 Ron Zass Multiclass clustering with side information from multiple sources and the application of converting 2d video to 3d
US9485497B2 (en) * 2010-09-10 2016-11-01 Reald Inc. Systems and methods for converting two-dimensional images into three-dimensional images
US20120235988A1 (en) * 2010-09-10 2012-09-20 Dd3D, Inc. Systems and methods for converting two-dimensional images into three-dimensional images
US9278281B2 (en) * 2010-09-27 2016-03-08 Nintendo Co., Ltd. Computer-readable storage medium, information processing apparatus, information processing system, and information processing method
US20120075430A1 (en) * 2010-09-27 2012-03-29 Hal Laboratory Inc. Computer-readable storage medium, information processing apparatus, information processing system, and information processing method
US8854356B2 (en) 2010-09-28 2014-10-07 Nintendo Co., Ltd. Storage medium having stored therein image processing program, image processing apparatus, image processing system, and image processing method
US9753585B2 (en) 2011-01-18 2017-09-05 Hewlett-Packard Development Company, L.P. Determine a position of an interaction area
US9250745B2 (en) 2011-01-18 2016-02-02 Hewlett-Packard Development Company, L.P. Determine the characteristics of an input relative to a projected image
US20120188234A1 (en) * 2011-01-20 2012-07-26 University Of Southern California Image processing apparatus and method
US20130286010A1 (en) * 2011-01-30 2013-10-31 Nokia Corporation Method, Apparatus and Computer Program Product for Three-Dimensional Stereo Display
US9269177B2 (en) * 2011-06-10 2016-02-23 Lg Electronics Inc. Method for processing image and apparatus for processing image
US20140118344A1 (en) * 2011-06-10 2014-05-01 Lg Electronics Inc. Method for processing image and apparatus for processing image
US20130038600A1 (en) * 2011-08-12 2013-02-14 Himax Technologies Limited System and Method of Processing 3D Stereoscopic Image
US8817073B2 (en) * 2011-08-12 2014-08-26 Himax Technologies Limited System and method of processing 3D stereoscopic image
US9202301B2 (en) * 2011-08-25 2015-12-01 Kabushiki Kaisha Toshiba Medical image display apparatus and X-ray diagnosis apparatus
US20130230136A1 (en) * 2011-08-25 2013-09-05 Toshiba Medical Systems Corporation Medical image display apparatus and x-ray diagnosis apparatus
US9606209B2 (en) 2011-08-26 2017-03-28 Kineticor, Inc. Methods, systems, and devices for intra-scan motion correction
US10663553B2 (en) 2011-08-26 2020-05-26 Kineticor, Inc. Methods, systems, and devices for intra-scan motion correction
US20130063549A1 (en) * 2011-09-09 2013-03-14 Lars Schnyder Systems and methods for converting video
US8928729B2 (en) * 2011-09-09 2015-01-06 Disney Enterprises, Inc. Systems and methods for converting video
US9100642B2 (en) * 2011-09-15 2015-08-04 Broadcom Corporation Adjustable depth layers for three-dimensional images
US20130069932A1 (en) * 2011-09-15 2013-03-21 Broadcom Corporation Adjustable depth layers for three-dimensional images
US9471988B2 (en) 2011-11-02 2016-10-18 Google Inc. Depth-map generation for an input image using an example approximate depth-map associated with an example similar image
US10194137B1 (en) 2011-11-02 2019-01-29 Google Llc Depth-map generation for an input image using an example approximate depth-map associated with an example similar image
US9661307B1 (en) 2011-11-15 2017-05-23 Google Inc. Depth map generation using motion cues for conversion of monoscopic visual content to stereoscopic 3D
US20140218358A1 (en) * 2011-12-01 2014-08-07 Lightcraft Technology, Llc Automatic tracking matte system
US9014507B2 (en) 2011-12-01 2015-04-21 Lightcraft Technology Llc Automatic tracking matte system
US9786253B2 (en) * 2012-01-25 2017-10-10 Lumenco, Llc Conversion of a digital stereo image into multiple views with parallax for 3D viewing without glasses
US20130187910A1 (en) * 2012-01-25 2013-07-25 Lumenco, Llc Conversion of a digital stereo image into multiple views with parallax for 3d viewing without glasses
US9111350B1 (en) 2012-02-10 2015-08-18 Google Inc. Conversion of monoscopic visual content to stereoscopic 3D
US9769460B1 (en) 2012-02-10 2017-09-19 Google Inc. Conversion of monoscopic visual content to stereoscopic 3D
US20130257851A1 (en) * 2012-04-01 2013-10-03 Chao-Hua Lee Pipeline web-based process for 3d animation
US20140009462A1 (en) * 2012-04-17 2014-01-09 3Dmedia Corporation Systems and methods for improving overall quality of three-dimensional content by altering parallax budget or compensating for moving objects
US8644596B1 (en) * 2012-06-19 2014-02-04 Google Inc. Conversion of monoscopic visual content using image-depth database
US8953874B2 (en) 2012-06-19 2015-02-10 Google Inc. Conversion of monoscopic visual content using image-depth database
US9582927B2 (en) * 2012-07-20 2017-02-28 Carl Zeiss Ag Multifocal representation device and multifocal representation method for the three-dimensional representation of an object
US20150178980A1 (en) * 2012-07-20 2015-06-25 Carl Zeiss Ag Multifocal representation device and multifocal representation method for the three-dimensional representation of an object
US20140035905A1 (en) * 2012-07-31 2014-02-06 Samsung Electronics Co., Ltd. Method for converting 2-dimensional images into 3-dimensional images and display apparatus thereof
US9779502B1 (en) 2013-01-24 2017-10-03 Kineticor, Inc. Systems, devices, and methods for tracking moving targets
US9717461B2 (en) 2013-01-24 2017-08-01 Kineticor, Inc. Systems, devices, and methods for tracking and compensating for patient motion during a medical imaging scan
US10327708B2 (en) 2013-01-24 2019-06-25 Kineticor, Inc. Systems, devices, and methods for tracking and compensating for patient motion during a medical imaging scan
US9607377B2 (en) 2013-01-24 2017-03-28 Kineticor, Inc. Systems, devices, and methods for tracking moving targets
US10339654B2 (en) 2013-01-24 2019-07-02 Kineticor, Inc. Systems, devices, and methods for tracking moving targets
US9305365B2 (en) 2013-01-24 2016-04-05 Kineticor, Inc. Systems, devices, and methods for tracking moving targets
US9782141B2 (en) 2013-02-01 2017-10-10 Kineticor, Inc. Motion tracking system for real time adaptive motion compensation in biomedical imaging
US10653381B2 (en) 2013-02-01 2020-05-19 Kineticor, Inc. Motion tracking system for real time adaptive motion compensation in biomedical imaging
US10462365B1 (en) * 2013-03-14 2019-10-29 Hrl Laboratories, Llc Low power surveillance system
US9674498B1 (en) 2013-03-15 2017-06-06 Google Inc. Detecting suitability for converting monoscopic visual content to stereoscopic 3D
US9842875B2 (en) 2013-08-05 2017-12-12 Apple Inc. Image sensor with buried light shield and vertical gate
US20150042772A1 (en) * 2013-08-06 2015-02-12 Samsung Electronics Co., Ltd. Display apparatus and control method for providing a 3d image
US9519996B2 (en) * 2013-12-16 2016-12-13 Huawei Technologies Co., Ltd. Virtual view generating method and apparatus
US20150170404A1 (en) * 2013-12-16 2015-06-18 Huawei Technologies Co., Ltd. Virtual View Generating Method and Apparatus
US20150229903A1 (en) * 2014-02-11 2015-08-13 Electronics And Telecommunications Research Institute Method and apparatus for converting 2d video image into 3d video image
US10004462B2 (en) 2014-03-24 2018-06-26 Kineticor, Inc. Systems, methods, and devices for removing prospective motion correction from medical imaging scans
US10652522B2 (en) 2014-05-22 2020-05-12 Disney Enterprises, Inc. Varying display content based on viewpoint
US10122992B2 (en) 2014-05-22 2018-11-06 Disney Enterprises, Inc. Parallax based monoscopic rendering
US11100636B2 (en) 2014-07-23 2021-08-24 Kineticor, Inc. Systems, devices, and methods for tracking and compensating for patient motion during a medical imaging scan
US9734589B2 (en) 2014-07-23 2017-08-15 Kineticor, Inc. Systems, devices, and methods for tracking and compensating for patient motion during a medical imaging scan
US10438349B2 (en) 2014-07-23 2019-10-08 Kineticor, Inc. Systems, devices, and methods for tracking and compensating for patient motion during a medical imaging scan
WO2016036352A1 (fr) * 2014-09-03 2016-03-10 Hewlett-Packard Development Company, L.P. Présentation d'une image numérique d'un objet
US10725586B2 (en) 2014-09-03 2020-07-28 Hewlett-Packard Development Company, L.P. Presentation of a digital image of an object
US10168833B2 (en) 2014-09-03 2019-01-01 Hewlett-Packard Development Company, L.P. Presentation of a digital image of an object
US9866826B2 (en) 2014-11-25 2018-01-09 Ricoh Company, Ltd. Content-adaptive multi-focal display
US9864205B2 (en) 2014-11-25 2018-01-09 Ricoh Company, Ltd. Multifocal display
US20160261845A1 (en) * 2015-03-04 2016-09-08 Dolby Laboratories Licensing Corporation Coherent Motion Estimation for Stereoscopic Video
US10200666B2 (en) * 2015-03-04 2019-02-05 Dolby Laboratories Licensing Corporation Coherent motion estimation for stereoscopic video
US9924240B2 (en) 2015-05-01 2018-03-20 Google Llc Systems and methods for interactive video generation and rendering
US9943247B2 (en) 2015-07-28 2018-04-17 The University Of Hawai'i Systems, devices, and methods for detecting false movements for motion correction during a medical imaging scan
US10660541B2 (en) 2015-07-28 2020-05-26 The University Of Hawai'i Systems, devices, and methods for detecting false movements for motion correction during a medical imaging scan
US10716515B2 (en) 2015-11-23 2020-07-21 Kineticor, Inc. Systems, devices, and methods for tracking and compensating for patient motion during a medical imaging scan
US11395947B2 (en) * 2016-01-28 2022-07-26 Nippon Telegraph And Telephone Corporation Virtual environment construction apparatus, video presentation apparatus, model learning apparatus, optimal depth decision apparatus, methods for the same, and program
US11998819B2 (en) 2016-01-28 2024-06-04 Nippon Telegraph And Telephone Corporation Virtual environment construction apparatus, video presentation apparatus, model learning apparatus, optimal depth decision apparatus, methods for the same, and program
US11568520B2 (en) * 2017-01-24 2023-01-31 Leica Geosystems Ag Method and device for inpainting of colourised three-dimensional point clouds
RU2752421C1 (ru) * 2018-05-23 2021-07-28 Сцивита Медикал Текнолоджи Ко., Лтд. Способ и устройство обработки изображений и система трехмерной визуализации
US11670039B2 (en) 2019-03-04 2023-06-06 Dolby Laboratories Licensing Corporation Temporal hole filling for depth image based video rendering
US20200019861A1 (en) * 2019-08-26 2020-01-16 Lg Electronics Inc. Method for controlling cook based on artificial intelligent and intelligent device
US11687776B2 (en) * 2019-08-26 2023-06-27 Lg Electronics Inc. Method for controlling cook based on artificial intelligent and intelligent device
US20230128024A1 (en) * 2021-10-26 2023-04-27 Koninklijke Philips N.V. Automated virtual (ar/vr) education content adjustment based on gaze and learner attention tracking
US20230179736A1 (en) * 2021-12-08 2023-06-08 immerVR GmbH Device and method

Also Published As

Publication number Publication date
CN101479765A (zh) 2009-07-08
EP2160037A3 (fr) 2010-11-17
US20130169749A1 (en) 2013-07-04
EP2033164A2 (fr) 2009-03-11
CN102685533A (zh) 2012-09-19
US9282313B2 (en) 2016-03-08
US20090116732A1 (en) 2009-05-07
CA2884702C (fr) 2018-06-05
WO2007148219A2 (fr) 2007-12-27
CN102685533B (zh) 2015-03-18
EP2160037A2 (fr) 2010-03-03
WO2007148219A3 (fr) 2008-04-10
EP2033164A4 (fr) 2010-11-17
CN101479765B (zh) 2012-05-23
CA2653815C (fr) 2016-10-04
CA2653815A1 (fr) 2007-12-27
EP2033164B1 (fr) 2015-10-07
CA2884702A1 (fr) 2007-12-27

Similar Documents

Publication Publication Date Title
US8411931B2 (en) Methods and systems for converting 2D motion pictures for stereoscopic 3D exhibition
US8791941B2 (en) Systems and methods for 2-D to 3-D image conversion using mask to model, or model to mask, conversion
US9094675B2 (en) Processing image data from multiple cameras for motion pictures
CA2636858C (fr) Procedes et systemes pour le rematricage numerique de films bi- et tridimensionnels pour une presentation avec une qualite visuelle amelioree
US20080225045A1 (en) Systems and methods for 2-d to 3-d image conversion using mask to model, or model to mask, conversion
US20080225042A1 (en) Systems and methods for allowing a user to dynamically manipulate stereoscopic parameters
US20080228449A1 (en) Systems and methods for 2-d to 3-d conversion using depth access segments to define an object
US20080226181A1 (en) Systems and methods for depth peeling using stereoscopic variables during the rendering of 2-d to 3-d images
US20080226160A1 (en) Systems and methods for filling light in frames during 2-d to 3-d image conversion
US20080226128A1 (en) System and method for using feature tracking techniques for the generation of masks in the conversion of two-dimensional images to three-dimensional images
Schnyder et al. 2D to 3D conversion of sports content using panoramas
WO2014121108A1 (fr) Procédés pour convertir des images bidimensionnelles en images tridimensionnelles
Ceulemans et al. Robust multiview synthesis for wide-baseline camera arrays
EP1668919A1 (fr) Imagerie stereoscopique
Zhu Perceiver–A Five-view Stereo System for High-quality Disparity Generation and its Application in Video Post-Production
Lucas et al. 2D–3D Conversion
Nazzar Automated detection of defects in 3D movies
Vazquez et al. 2D‐to‐3D Video Conversion: Overview and Perspectives
WO2008112786A2 (fr) Systèmes et procédé pour générer une géométrie en 3d utilisant des points à partir de séquences d'image

Legal Events

Date Code Title Description
AS Assignment

Owner name: IMAX CORPORATION, CANADA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:ZHOU, SAMUEL;JUDKINS, PAUL;YE, PING;REEL/FRAME:022097/0284;SIGNING DATES FROM 20081125 TO 20081126

Owner name: IMAX CORPORATION, CANADA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:ZHOU, SAMUEL;JUDKINS, PAUL;YE, PING;SIGNING DATES FROM 20081125 TO 20081126;REEL/FRAME:022097/0284

FEPP Fee payment procedure

Free format text: PAYOR NUMBER ASSIGNED (ORIGINAL EVENT CODE: ASPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

STCF Information on status: patent grant

Free format text: PATENTED CASE

CC Certificate of correction
FPAY Fee payment

Year of fee payment: 4

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 8TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1552); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

Year of fee payment: 8

AS Assignment

Owner name: WELLS FARGO BANK, NATIONAL ASSOCIATION, AS ADMINISTRATIVE AGENT, COLORADO

Free format text: SECURITY INTEREST;ASSIGNOR:IMAX CORPORATION;REEL/FRAME:059771/0488

Effective date: 20220428