WO2011127434A2 - Apparatus and method for capturing images - Google Patents

Apparatus and method for capturing images Download PDF

Info

Publication number
WO2011127434A2
WO2011127434A2 PCT/US2011/031830 US2011031830W WO2011127434A2 WO 2011127434 A2 WO2011127434 A2 WO 2011127434A2 US 2011031830 W US2011031830 W US 2011031830W WO 2011127434 A2 WO2011127434 A2 WO 2011127434A2
Authority
WO
WIPO (PCT)
Prior art keywords
camera
view
cameras
base
field
Prior art date
Application number
PCT/US2011/031830
Other languages
French (fr)
Other versions
WO2011127434A3 (en
Inventor
Sankar Jayaram
Brett Buchholtz
Charles Dechenne
John Harrison
Original Assignee
3D-4U, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 3D-4U, Inc. filed Critical 3D-4U, Inc.
Priority to CN201180026684.7A priority Critical patent/CN103026700B/en
Priority to EP11766841.8A priority patent/EP2556654B1/en
Publication of WO2011127434A2 publication Critical patent/WO2011127434A2/en
Publication of WO2011127434A3 publication Critical patent/WO2011127434A3/en

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60Control of cameras or camera modules
    • H04N23/698Control of cameras or camera modules for achieving an enlarged field of view, e.g. panoramic image capture
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20Image signal generators
    • H04N13/204Image signal generators using stereoscopic image cameras
    • H04N13/243Image signal generators using stereoscopic image cameras using three or more 2D image sensors
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20Image signal generators
    • H04N13/204Image signal generators using stereoscopic image cameras
    • H04N13/239Image signal generators using stereoscopic image cameras using two 2D image sensors having a relative position equal to or related to the interocular distance
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/50Constructional details
    • H04N23/54Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/50Constructional details
    • H04N23/55Optical parts specially adapted for electronic image sensors; Mounting thereof
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/58Means for changing the camera field of view without moving the camera body, e.g. nutating or panning of optics or image sensors
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/90Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Definitions

  • the present invention pertains to systems and methods for capturing detectable phenomena, such as images, sounds, or any measurable physical phenomena. More particularly, the present invention relates to the capture of images using an array of detectors over ranges that exceed the range for a single detector, such as monoscopic detectors, as well as for stereoscopic images captured with an array of detectors over ranges that exceed the range for a single pair of stereoscopic detectors.
  • One technique involves rotating a camera about a central axis while capturing overlapping images that are spliced together using software.
  • Another technique involves capturing a series of overlapping images, then splicing together adjacent overlapping images in order to produce an image that is wider than an image captured by a single camera.
  • a stereoscopic pair of cameras has been used to capture a stereoscopic field of view.
  • problems with capturing a field of view that is greater than the field of view for a stereoscopic pair of cameras.
  • wide angle cameras such as cameras with a 1 80 degree field of view, adjacent left and rig ht cameras can interfere with each other.
  • An array of monoscopic detectors as well as an array of stereoscopic pairs of detectors, are provided to capture information from a surrounding environment, such as monoscopic images or stereoscopic images, and audio inputs from ranges exceeding that for a si ngle detector or stereoscopic pair of detectors.
  • a surrounding environment such as monoscopic images or stereoscopic images
  • audio inputs from ranges exceeding that for a si ngle detector or stereoscopic pair of detectors.
  • i mage inputs stereoscopic pai rs of cameras are provided in an array.
  • pairs of separated , stereoscopic di rectional microphones are provided in an array with a microphone substituting for each camera.
  • an apparatus for capturi ng i mages includi ng a base, and i mage capture adjustment mechanism, a first camera, and a second camera.
  • the base is constructed and arranged to support an alignable array of cameras.
  • the i mage captu re adjustment mechanism is disposed relative to the base for adjusting an i mage capture li ne of sig ht for a camera relative to the base.
  • the fi rst camera is carried by the base, operably coupled with the image capture adjustment mechanism , and has an image capture device.
  • the first camera has a line of sight defi ning a fi rst field of view adjustable with the image capture adjustment mechanism relative to the base.
  • the second camera is carried by the base and has an image captu re device.
  • the second camera has a line of sight defining a second field of view extendi ng beyond a range of the field of view for the first camera in order to produce a field of view that is greater than the field of view provided by the first camera.
  • an apparatus for captu ri ng digital images including a base, a first digital camera, a second digital camera, and an i mage captu re adjustment mechanism.
  • the base is constructed and arranged to support a plurality of cameras for physically alig ning images captured by adjacent cameras relative to one another.
  • the fi rst digital camera is supported by the base and has an i mage capture device.
  • the first camera has a line of sight defini ng a first field of view.
  • the second digital camera is carried by the base and has an image capture device.
  • the second camera has a line of sig ht defi ning a second field of view extendi ng beyond a range of the field of view for the first camera i n order to produce an adjacent field of view that extends beyond the field of view provided by the first camera.
  • the image capture adjustment mechanism is disposed relative to the base and is operatively coupled with the first digital camera for physically adjusting an i mage capture line of sight for the first digital camera relative to the base and the second digital camera to align an adjacent field of view for the first digital camera relative to the second digital camera.
  • a method for captu ri ng i mages including : providing a fi rst camera and a second camera carried by a base to have a line of sight defi ning a respective field of view, the second camera havi ng a field of view at least in part adjacent to the field of view for the first camera, the first camera carried by the base for adjustable positioning of the field of view relative to the field of view for the second camera; axially aligni ng the first camera relative to the second camera to render colli near an i mage segment within a field of view for the fi rst camera relative to a corresponding image segment within the field of view for the second camera; and angularly aligni ng the first camera relative to the second camera to render the image segment withi n the field of view for the first camera angularly aligned relative to the i mage seg ment i n the field of view for the second camera.
  • a stereoscopic camera system provides an apparatus for capturi ng a stereoscopic field of view.
  • the stereoscopic camera system i ncludes a support structu re, a plurality of pairs of stereoscopic cameras, a plurality of camera mounting platforms, and a plurality of articulating support structures.
  • Each of the plurality of pairs of stereoscopic cameras includes a left camera and a right camera.
  • Each of the plurality of camera mounting platforms supports at least one of a left camera and a right camera of a specific pair of stereoscopic cameras.
  • Each of the plurality of articulating support structures is configured to adjustably position a respective camera mount platform relative to the base to axially and angularly align two adjacent left cameras and two adjacent right cameras within adjacent pairs of stereoscopic left and right cameras having adjacent fields of view.
  • a left and right eye camera pair as described in this invention also includes cameras that include special lenses to collect stereoscopic left/right eye images using a single camera.
  • Fig. 1 is perspective view of an environment including a basketball court being captured with a 180 degree stereoscopic camera system, or apparatus, according to one embodiment.
  • Fig. 2 is an enlarged perspective view of the stereoscopic camera system of Fig. 1 shown positioned at one end of a courtside scorer's table.
  • Fig. 3 is a plan view of the basketball court of Fig. 1 showing additional and/or optional placements for additional stereoscopic camera systems about the environment.
  • Fig. 4 is a perspective view from above of a second 180 degree stereoscopic camera system, similar to that depicted in Figs. 1-3, but self-supported atop a post and base.
  • Fig. 5 is a perspective view from above of a third stereoscopic camera system , si milar to that depicted i n Fig. 4, but with a 360 degree stereoscopic camera array self-supported atop a tripod .
  • Fig. 6 is an enlarged perspective view from above of the 1 80 degree stereoscopic camera system of Fig . 4.
  • Fig. 7 is an enlarged perspective view of the stereoscopic camera system of Fig. 6 with portions removed .
  • Fig. 8 is an enlarged perspective view from above of the 360 degree stereoscopic camera system of Fig . 5.
  • Fig. 9 is an enlarged perspective view of the stereoscopic camera system of Fig. 9 with portions removed .
  • Fig. 1 0 is a simplified plan view from above of a pai r of 1 80 degree fisheye cameras configured to capture stereoscopic i mages and illustrati ng interference and camera spacing complexities.
  • Fig. 1 1 is a si mplified plan view from above of three adjacent sets of stereoscopic cameras from the stereoscopic camera system of Figs 4 and 6-7 or Figs. 5 and 8-9, as well as the stereoscopic camera system of Figs. 1 -3, but where each camera is suspended beneath the base plate, and illustrating lens fields of view for two adjacent right cameras.
  • Fig. 1 2 is simplified plan view correspondi ng with the view in Fig. 1 1 , but taken further away to illustrate overlapping lens fields of view for the two adjacent right cameras.
  • Fig. 1 3 is a si mplified schematic plan view of a basketball cou rt showi ng overlapping fields of view for two adjacent rig ht cameras for the stereoscopic camera system of Figs. 1 -3.
  • Fig. 1 4 is an enlarged perspective view of the base plate for the stereoscopic camera system of Fig . 6 (as well as Figs. 1 -3, if flipped upside down) and showing one camera and articulating support structure in exploded view that provides an image capture adjustment mechanism.
  • Fig. 15 is another enlarged perspective view of the base plate of Fig. 14 and showing two adjacent pairs of stereoscopic cameras, each mounted to the base plate with a dedicated articulating support structure.
  • Fig. 16 is a further enlarged breakaway perspective view of the camera, base plate and articulating support structure of Fig. 15.
  • Fig. 17 is another embodiment of an articulating support structure suitable for use with the stereoscopic cameras systems of Figs. 1-9 and 11-16, providing another embodiment for an image capture adjustment mechanism.
  • Fig. 18 is an exploded perspective view of the articulating support structure of Fig. 17.
  • Fig. 19 is a pedestal for mounting the stereoscopic camera systems of Figs.4 and 5 atop a post and a tripod, respectively.
  • Fig. 20 is vertical sectional view of the pedestal taken along line 20-20 of Fig. 19.
  • Fig.21 is a perspective view from above of a fourth stereoscopic camera system, similar to that depicted in Fig. 5, but with a 180 degree stereoscopic camera array using a housing with discrete planar glass windows for each set of left and right stereoscopic cameras.
  • Fig. 22 is an enlarged perspective view of the stereoscopic camera system of Fig. 21 with housing portions removed in order to illustrate camera system components within the housing.
  • Fig. 23 is a further enlarged breakaway perspective view of the camera, base plate and articulating support structure of Figs. 21 and 22, providing even another embodiment for an image capture adjustment mechanism.
  • Fig. 24 is a simplified diagrammatic view of even another image capture adjustment mechanism providing a camera lens that is repositionable within a plane perpendicular to the viewing direction.
  • Fig. 25 is a simplified diagrammatic view of yet another image capture adjustment mechanism providing an image capture device (CCD) that is repositionable within a plane of the image capture device.
  • CCD image capture device
  • Fig. 26 is a simplified diagrammatic view of yet even another image capture adjustment mechanism providing a repositionable mirror between a camera lens and an image capture device (CCD).
  • Fig. 27 is a logic flow diagram illustrating one method for physically aligning adjacent image capture devices provided on a support base.
  • Fig. 28 is a logic flow diagram illustrating another method for physically aligning a plurality of adjacent stereoscopic pairs of left and right cameras provided on a support base.
  • Embodiments of the present invention disclose an apparatus and method for capturing information from a surrounding environment using an array of detectors.
  • monoscopic and stereoscopic images such as still images and video images or frames are captured using an array of cameras.
  • directional audio inputs are captured with an array of directional microphones.
  • arrays of detectors can be used to capture infrared, ultrasonic, sonic, subsonic, ultraviolet, or electromagnetic events or signals.
  • the term “field of view” is understood to mean the angular or linear or areal or volumetric extent of the observed world that is seen or viewed by an image capture device at any given moment.
  • the term “adjustable line of sight” is understood to encompass one or both of translation or rotation, including modifying the inter-camera spacing between adjacent cameras in a pair.
  • image capture adjustment mechanism is understood to mean any structural mechanism capable of physically aligning or moving (angularly or linearly) the position of an image of a target object on an image capture device.
  • Figures 1-3 are illustrations of an apparatus and method for capturing and aligning stereoscopic camera images from an array of multiple pairs of stereoscopic cameras provided by a stereoscopic camera system 10 within an environment 12, such as a basketball court.
  • camera system 10 of Figures 1-2 is essentially the same as camera system 110 of Figures 4, 6-7 and 10-15, except that stereoscopic camera system 10 is mounted up-side-down in the application of Figures 1-3 and the housing side plate 46 and back plate 48 of system 110 are extended downwardly in height by two inches sufficient to provide a support base that extends below connectors 40 so that system 10 can be supported atop a scorer's table 14, as shown in Figures 1 and 2.
  • the right-side-up configuration of system 110 depicted in Figures 4 and 6-7 uses a pedestal 238, post 36 and base plate 38 to support system 110 with cameras at a desired viewing height, typically corresponding with eye level height for a typical observer present within an environment under normal observation conditions.
  • a desired viewing height typically corresponding with eye level height for a typical observer present within an environment under normal observation conditions.
  • An observer can either be sitting or standing, depending on the particular environment and event being observed.
  • the viewing height can be some other desirable height simulating either an elevated or a lowered condition, such as simulating an extra-tall person's view, birds-eye view or a floor-level view.
  • cameras can be placed in adjacent vertical rows to provide an elevational field of view with an angle between vertical pairs, such as in a spherical housing with cameras pointing radially outwardly in all directions.
  • a vertical configuration can be implemented by stacking a plurality of horizontal base plates one atop another, with each base plate having an array of physically alignable cameras.
  • an array of cameras can be mounted individually onto a vertically extending base plate (in contrast with a horizontally extending base plate as depicted in Fig. 9), or onto a baseplate having an angular configuration somewhere between horizontal and vertical.
  • stereoscopic camera system 10 is supported atop scorer's table 14 by a bottom edge of housing 16 (see Fig. 2).
  • stereoscopic camera system 10 is supported at one end of scorer's table 14, where table 14 is position at court-side, typically at center court.
  • An array of stereoscopic pairs of video cameras, such as left video camera 32 and right video camera 34 of pair 20, are supported within housing 16 in order to capture stereoscopic video images across a range of different fields of view, for given video camera fields of view.
  • Each video camera 34 captures images through a port, or aperture 18 in housing 16.
  • Figure 3 illustrates additional locations for stereoscopic camera system 10 alongside a basketball court environment 12.
  • camera system 10 is provided in multiple locations, and stereoscopic image capture and retrieval can be provided to a viewer remote in space and/or time from each of the camera systems 10. In one case, a viewer can select from which camera system 10 they are going to view stereoscopic video images.
  • camera system 110 of Figures 4 and 6-7, as well as camera system 210 of Figures 5 and 8-9 can also be used at any of the locations of camera system 10 in Figure 3.
  • system 10 can be place in other environments, such as football fields, hockey rinks, museums, or any location where an individual may desire observing the real environment as a virtual environment displaced in either time or place, or both. It is also understood that cameras can generate pictures or video. In a museum application, system 10 can be moved within a museum to known locations in order to take pictures at many discrete locations in order to capture images from all angles within the museum, and a user can then visit the museum by retrieving the images at a remote location.
  • FIGs 4 and 6-7 illustrate an embodiment where cameras, such as cameras 32 and 34 of stereoscopic set 20 (see Fig.4) are provided along a bottom edge of housing 116. In such case, electrical connectors 40 extend through a top of housing 116. Each camera 32 and 34 receives video images through a respective aperture 118.
  • camera system 110 is mounted atop a post 36 of desired height. Post 36 is mounted atop a base 38 to provide stability.
  • a directional microphone can be substituted for each camera.
  • housing 116 includes a base, or support plate 42, a top plate 44, a semi-circular vertical plate 46 and a back plate 48.
  • Each camera 32 and 34 is coupled to a data and power connector 40 that extends through an aperture 50 in top plate 44.
  • Threaded cap-head bolts 52 couple together top plate 44, side plates 46 and 48, and bottom plate 42.
  • An L-shaped bracket 62 with a bore is mounted between each cap screw 52 that mounts through bottom plate 42, providing a shelf for supporting a bottom edge of side plate 46.
  • housing 116 of stereoscopic camera system 110 are removed in order to observe a plurality of pairs 20-25 of stereoscopic left cameras 32 and right cameras 34.
  • Each camera is mounted to base plate 42 with an image capture adjustment mechanism in the form of an articulating support structure, or base frame 56 that can be used to align the camera relative to the base plate 42.
  • a directional microphone can be mounted to each base frame, substituting for each camera 32, 34 in pairs 20- 25.
  • an image capture device similar to cameras 32 and 34, can be provided for capturing electromagnetic radiation in one or more of any of the spectrums, including infrared and/or optical (visible) portions of the electromagnetic spectrum.
  • an image capture device can be provided for capturing ultrasonic images using an array of ultrasonic image capture devices.
  • each post 54 has a threaded bore at each end sized to receive a respective threaded cap-screw, or fastener 52 (see Fig. 6).
  • Base plate 42 and top plate 44 each have corresponding holes that align with each post 54 to receive a fastener 52, where the enlarged head of fastener 52 secures respective plates 42 and 44 together against side plate 46 and back plate 48.
  • Back plate 48 includes a ventilation port 58 in which a cooling fan is mounted (not shown) and a switch port in which a power switch (not shown) is mounted for turning the fan on and off.
  • An array of four bores 61 are configured in bottom plate 42 to align with corresponding bores 242 in pedestal 238 (see Fig. 18) so each receives a bolt and nut (not shown) to secure system 116 atop pedestal 238.
  • Pedestal 238 is then mounted atop post 36 (see Fig. 4) or atop a tripod, such as tripod 236 shown in Figure 5 using a threaded mounting aperture 245 (see Fig. 19).
  • bore 60 aligns with bore 243 of pedestal 238 (of Fig. 18) to enable communications cables and power cables for cameras 32 and 34 to exit housing 116.
  • the construction of camera system 110 of Figure 7 can be constructed to have a monoscopic array of detectors, such as video cameras.
  • a monoscopic array of detectors such as video cameras.
  • Such a construction would generate aligned 180 degree or 360 degree monoscopic output, or video, where the field of view of adjacent cameras can be physically aligned using individual base plates and image captu re adjustment mechanisms and methods.
  • each left camera can be turned off and the output from each rig ht camera can be captured i n order to achieve panoramic 1 80 degree or 360 degree video imagi ng (or each right camera can be turned off) .
  • the resulting array of cameras would not be arranged concentrically about a poi nt.
  • the cameras can be arranged concentrically about a point, or they can be arranged so that they are not concentric.
  • an array of monoscopic cameras can be provided , each on an adjustable base plate with an i mage capture adjustment mechanism .
  • a mi rrored adapter can be mounted onto each monoscopic camera lens in order to generate alternating left and right offset stereoscopic video i mages.
  • Such a mi rrored stereoscopic 3D camera adapter is provided by a NuView SX2000 video lens adapter sold by Mi ndlfux, Jasandre Pty. Ltd. , P.O. Box 494, Roseville, NSW 2069 Australia, which generates 3- D field sequential video, under a patent pending process entitled STE R EO-OPTIX!.
  • Figu res 5 and 8-9 illustrate another embodiment for a stereoscopic camera system 21 0 having a full 360 degree array of pairs 20 of stereoscopic left and rig ht cameras 32 and 34 supported withi n housi ng 21 6.
  • Housi ng 21 6 mounts via pedestal 238 atop a tripod, or support structure 236, as shown in Figure 5.
  • threaded cap screws 52 affix together a top plate 244, a side plate 246, and a bottom plate 242 for housing 21 6 of camera system 21 0.
  • Left and rig ht video cameras 32 and 34 form a stereoscopic pai r 220, with a lens from each camera extending th roug h a respective aperture 1 8 in side plate 246.
  • An L-shaped bracket 62 with a bore is mou nted between each cap screw 52 that mounts th rough bottom plate 242, providing a shelf for supporting a bottom edge of side plate 246.
  • stereoscopic camera system 210 includes a complete circumferential array of stereoscopic pairs 220-231 of left cameras 32 and right cameras 34 used to capture dynamic (or static) landscape images within an environment surrounding system 210.
  • a circumferential array of posts 54, threaded cap screws 52 and L- brackets 62 are used to hold together housing 216 (of Fig. 9), in a manner similar to that shown for system 110 in Figure 7.
  • Each camera 32, 34 is mounted in an adjustable manner onto base plate 242 via an adjustable, articulating support structure 56. Bores 61 and port 60 facilitate mounting of system 210 onto pedestal 238 (of Fig 18) in a manner similar to that shown for system 110 (of Fig.7).
  • Figure 10 is a simplified plan view from above of a pair of 180 degree left and right fisheye cameras 1032 and 1034 configured to capture stereoscopic images.
  • an interspace distance, t between left camera 1032 and right camera 1034 is shown.
  • interspace distance, t 0° provides a maximum lateral spacing between cameras 1032 and 1034.
  • angle, ⁇ increases, interspace distance, t, diminishes, which also diminishes stereoscopic perspective.
  • a lens on one of camera 1032 and 1034 interferes with another of lens on another of camera 1032 and 1034, which interferes with image capture. Accordingly, there is a limit on the angular field of view that is practical when capturing images from a wide field of view, thereby necessitating a solution provided by use of multiple cameras in stereoscopic configurations, as shown in the various embodiments depicted herein.
  • a field of view for an array of six pairs of stereoscopic cameras in Figure 7 will have a trimmed field of view for each camera of 30° (each camera has an untrimmed field of view of approximately 41 °).
  • This configuration provides a highly preferred physical alignment with minimal image distortion.
  • a field of view for an array of five pairs of stereoscopic cameras (covering 180 degree range) will have a trimmed field of view for each camera of 36° (approximately 45° untrimmed).
  • This configuration provides a preferred physical alignment with slightly greater image distortion than for a six pair array.
  • a field of view for an array of four pairs of stereoscopic cameras (covering 180 degree range) will have a trimmed field of view for each camera of 45° (approximately 60° untrimmed).
  • This configuration provides a moderately preferred physical alignment with slightly greater image distortion than for a six pair array.
  • a field of view for an array of three pairs of stereoscopic cameras (covering 180 degree range) will have a trimmed field of view for each camera of 60° (approximately 75° untrimmed).
  • This configuration provides a reasonably preferred physical alignment with slightly greater image distortion than for a six pair array.
  • a field of view for an array of two pairs of stereoscopic cameras (covering 180 degree range) will have a trimmed field of view for each camera of 90° (approximately 110° untrimmed).
  • This configuration provides a somewhat preferred physical alignment with slightly greater image distortion than for a six pair array.
  • the difference between untrimmed field of view and trimmed field of view is the extent of overlap between adjacent cameras on either side of a specific camera, after alignment of the adjacent cameras.
  • Figures 11 and 12 illustrate in plan view one exemplary set of adjacent right cameras 34 from adjacent pairs of stereoscopic pairs 20, 21 of cameras 32, 34 on stereoscopic camera system 110. More particularly, simplified representations of fields of view for right camera 34 of pair 20 and right camera 34 of pair 21 are shown. It is understood that a similar representation exists for left cameras 32 in any adjacent pair, as well as for any adjacent right cameras 34.
  • Each camera 34 has a lens angle. Adjacent cameras 34 have lens angles that form a convergence angle. As shown in this embodiment, there is a 30 degree angle between the central line of sight (view direction) from lenses 64 on adjacent pairs of stereoscopic cameras.
  • Figure 12 illustrates an overlap region 66 that occurs at a specific distance away from system 110.
  • Figure 13 further illustrates adjacent overlapping fields of view for right cameras (R1) and (R2), along with overlap region 66 within a basketball court environment.
  • Output from respective cameras is processed using seaming and warping techniques to join together the output of adjacent right cameras and adjacent left cameras in order to produce a field of view that is greater than that provided by a single camera, or a single pair of stereoscopic cameras.
  • cameras 32 and 34 are aligned in order to significantly reduce post-processing in order to achieve alignment of captured images between left and right cameras in a pair, as well as between right cameras in adjacent pairs and left cameras in adjacent pairs.
  • the articulating support structure 56 (of Fig.7) enables this alignment between cameras prior to capturing images, as will be discussed below in greater detail with reference to Figures 14-16.
  • Figure 14 illustrates assembly of a typical camera 32 onto base plate 42 with articulating support structure 56.
  • camera 32 can be adjusted relative to plate 42 to induce (relative to lens 64) pitch 101 and roll 103 (see Fig. 15).
  • yaw 102 can also be induced.
  • bearing 70 can be made out of an elastic material, such as a plastic, which can be slightly compressed, enabling vertical adjustment by tightening and loosening all of the fasteners.
  • Figu re 1 5 depicts two adjacent stereoscopic pairs 20 and 21 of left cameras 32 and rig ht cameras 34, each mounted atop a dedicated articulati ng support structure 56.
  • system 1 0 of Figures 1 -3 merely supports these pairs i n an up-side-down configuration in order to produce a more compact housi ng when the housing is supported on a table, as the individual lenses are spaced further apart from the table top su rface which mig ht otherwise interfere with i mage capture.
  • Figu re 1 6 illustrates construction of articulati ng support structure 56. More particularly, structu re 56 includes a support plate, or frame 68 that has a semi-spherical seat, or recess 80 on a central bottom surface for receiving a spherical bearing 70 (either plastic or hardened steel) . Camera 32 is rigidly secured onto a top surface of plate 68 with threaded fasteners 84 that pass through bores 78 in plate 68 and i nto threaded bores (not shown) in the bottom of camera 32.
  • Threaded cap screws 74 (with fine th reads) pass th roug h bores 76 i n base plate 42, through coil steel springs 72, and i nto th readed bores 86 i n plate 68.
  • Ball beari ng 70 also seats i n another semi- spherical seat 82 in a top surface of plate 42.
  • Beari ng 70 is larger i n diameter than the combined depths of seats 80 and 82 so that plate 68 is spaced from plate 42 in assembly.
  • Figure 17 illustrates another embodiment for an image capture adjustment mechanism, or articulating support structure 156 that further enables adjustment to rotate camera 32 in order to induce yaw 102. More particularly, a turret plate 170 is captured for constrained rotation between an upper plate 168 and a lower plate 169. A radial arm 171 on plate 170 is constrained within a slot 172 to enable rotational adjustment of turret plate 170 relative to plates 168 and 169. As shown in Figure 17, a set screw 173 is used to fix positioning of turret plate 170 and camera 32 relative to plates 168 and 169. More particularly, threaded fasteners 84 pass through bores 179 in plate 170 and into complementary threaded bores (not shown) in the bottom of camera 32.
  • Camera 34 is similarly mounted. Threaded recessed head screws 184 pass through bores 182 in plate 169 and into complementary threaded bores 180 in plate 168 to hold together plates 168-170. Threaded fasteners 74 pass through the base plate (not shown), springs 72, and into complementary threaded bores 182. This traps bearing 70 in a manner similar to that shown in Figure 16 and adjustment of screws 74 enables pitch and roll adjustment of camera 32, while turret 170 enables yaw adjustment.
  • Figure 19 illustrates construction details of pedestal 238. A top ring 239 and a bottom plate 240 are held together with four posts 241 using threaded fasteners 244.
  • Central bore 243 enables passage of communications and power cables passing to the cameras on the system mounted atop pedestal 239 via fasteners passing through bores 242.
  • a threaded recess 245 is used to affix plate 240 atop a post or tripod using a complementary threaded fastener.
  • cameras 32 and 34 are each an IDS GigE Model Number UI-5649 HE-C, a high-performance GigE camera with large functional range.
  • Image data from an Aptina CMOS sensor in 1.3 Megapixel resolution (1280 x1024 pixels) is output with up to 12 bits per channel.
  • An internal FPGA with 64 MB image memory offers additional features and ensures fast and reliable data transfer.
  • the UI-5649 HE-C comes with a multi-l/O-interface including 4 digital in-/outputs and an RS232 interface.
  • An optional camera is the IDS GigE Model Number Ul- 5640HE. Both cameras are available in the United States at IDS Imaging Development Systems, Inc., 400 West Cummings Park, Suite 3400, Woburn, MA 01801. Such cameras capture still or video images, where video images are time displaced still images.
  • FIGs 21-23 illustrate another embodiment for a stereoscopic camera system 310 having a 180 degree array of pairs 320 of stereoscopic left and right cameras 332 and 334 supported within housing 316.
  • Housing 316 mounts via pedestal atop a tripod, or support structure, similar to support structure provided by post 36 and base plate 38 shown in Figure 5.
  • threaded cap screws 352 affix together a top plate 344, a side plate 346, and a bottom plate 342 for housing 316 of camera system 310.
  • Left and right video cameras 332 and 334 form a stereoscopic pair 320, with a lens from each camera extending through a respective aperture 318 in side plate 346.
  • Housing 316 is formed by welding together a plurality of individual vertical face plates and a back plate 348 atop a bottom plate 342.
  • a plurality of apertures, or windows 318 are provided in vertical faces of housing 316 for each pair 320-325 (see Fig. 22) of stereoscopic cameras.
  • An aluminum bezel plate 382 is used to retain a rectangular glass plate 380 over each window 318, preferably with one or more rubber gaskets (not shown) provided between glass 380 and housing 316 and/or bezel 382.
  • Figure 22 illustrates internal components mounted atop bottom plate 342 and within housing 316 of camera system 310. More particularly, pairs 320-325 of stereoscopic cameras, such as cameras 332 and 334, are mounted atop plate 342, each with an image capture adjustment mechanism that enables physical alignment between cameras. More particularly, a camera power fuse board 360, a power supply 362, and a sound fuse board 364 are affixed to a back in ner surface of housing 31 6. A fiber cable multiplexer, or fiber switch, 368 is affixed atop bottom plate 342. An array of seven individual Ethernet-fiber media/mode converters 366 are mounted atop multiplexer switch 368.
  • Each converter 366 has respective ports for receivi ng two RJ45 Ethernet con nectors from each pai r of cameras and a fiber optic connector for sending a fiber optic output signal to fiber multiplexer, or switch 368 where signals are combi ned i n order to generate a si ngle output signal delivered on a fiber to video and audio production facility.
  • One suitable converter 366 is an I E-Multiway, 1 0/1 00/1 000 Mbps Ethernet Media/Mode converter, sold by I MC Networks, of 1 9772 Pauling, Foothills Collins , CA 9261 0.
  • first camera 332 and second camera 334 are carried by the base 342 for adjustable positioni ng using an image captu re adjustment mechanism.
  • Adjacent left cameras, as well as adjacent rig ht cameras, have at least i n part adjacent fields of view.
  • Adjacent first and second right cameras (as well as adjacent fi rst and second left cameras) are axially aligned to render collinear an image seg ment within a field of view for the second camera relative to an i mage seg ment within the field of view for the first camera.
  • Adjacent first and second right cameras are angularly aligned to render the image segment withi n the field of view for the second camera angularly aligned relative to the i mage segment in the field of view for the first camera.
  • each camera 332 is supported atop bottom plate 342 in a manner that enables physical adjustment of camera 332 (and lens 364) so as to alig n the resulting field of view relative to adjacent and related cameras (see Fig . 22) .
  • adjacent left and right cameras 332 and 334 within a pai r 321 of stereoscopic cameras can be physically adjusted to alig n respective fields of view, thereby reducing or eli minating any need to perform alignment and/or adjustment using software techniques.
  • software tech niques are not ideally suited for live broadcast, as they require computational power, which can introduce significant ti me delays.
  • nearest-neighbor left cameras and nearest-neighbor rig ht cameras can also be physically adjusted in order to align respective fields of view.
  • Figu re 23 illustrates one construction for the image capture adjustment mechanism used i n Figure 22 comprising a camera mou nting platform , or plate 368, and an articulating support structure comprisi ng a spherical beari ng 70 seated between two semi-spherical seats 380 and 382 i n plates 368 and 342, respectively.
  • Threaded cap screws 374 pass through a respective hard plastic bushing 375, aperture 376 i n plate 342, hard plastic bushing 377, coil steel spri ng 372, and i nto threaded bore 378 in plate 368.
  • th readed fasteners 384 pass through bores 386 i n plate 368 and i nto respective threaded bores (not shown) i n a bottom plate of each camera 332 i n order to rigidly secure camera 332 atop plate 368.
  • Camera 332 and plate 368 can be adjusted i n pitch and roll by adjusti ng threaded position of each screw 374 so as to d rop and tilt, respectively, lens 364.
  • Adjacent cameras 332 and 334 are carried by a com mon support base 342 (see Fig . 22) .
  • bushings 375 and 377 each have an in ner bore sized slig htly smaller than an outer thread diameter on each screw 375.
  • Screw 374 self-taps i nto bushing 375 upon threaded insertion.
  • Such construction enables the removal of one machine screw 374 for maintenance or replacement, after which such screw can be rei nserted and specific nu mber of tu rns can be applied to the screw equal to the number needed for removal in order to achieve a close approxi mation to the original position.
  • Figu re 24 is a si mplified diagrammatic view illustrati ng layout of even another i mage captu re adjustment mechanism 456 providi ng a camera lens 462 that is repositionable withi n a plane perpendicular to an image viewing direction . More particularly, lens 462 is rigidly supported by a first linear rack 474 and rotary gear 474, and a second linear rack 476 and rotary gear 478. Accordi ng to such construction , gear 474 is mounted i n axially slidable relation on a central shaft 473 in order to accommodate linear motion generated by rack 476 and gear (or pinion) 478.
  • an image 466 can be aligned or repositioned relative to an i mage capture device, or CC D 464.
  • An object 460 is captu red within a field of view 468 via lens 462 and is projected with an image range, or region of interest 470 as an i mage 466 on CC D 464 at a focal length 482.
  • lens 462 has peripheral edge portions that are affixed to racks 472 and 476.
  • a peripheral edge portion of lens 462 is affixed to a carrier plate, and such plate is rigidly affixed to racks 472 and 476.
  • Figu re 25 is a simplified diagrammatic view illustrating yet even another i mage capture adjustment mechanism 556 providing an i mage captu re device (CC D) 564 that is repositionable withi n a plane of the i mage capture device 564. More particularly, an object 560 is captu red within a field of view 568 via a lens 562 and is projected with an i mage range 570 as an image 566 on CC D 564 at a focal length 582.
  • CC D 564 is rigidly supported by a first linear rack 574 and rotary gear 574, and a second linear rack 576 and rotary gear 578.
  • Figu re 26 is a simplified diag ram matic view illustrating yet another i mage capture adjustment mechanism 656 providi ng a repositionable mirror 658 between a camera lens 662 and an i mage capture device (CCD) 664. More particularly, an object 660 is captured within a field of view 668 via lens 662, is reflected by mirror 658, and is projected with an image range 670 as an image 666 on CCD 664 at a focal length 682.
  • Mirror 658 is rigidly supported by a first linear rack 674 and a second linear rack 676.
  • First linear rack 674 interacts with rotary gear 674
  • second linear rack 676 interacts with rotary gear 678.
  • gear 678 is mounted in axially slidable relation along a guide slot 680 in order to accommodate linear motion generated by rack 676 and gear (or pinion) 678.
  • Figure 27 forms a process flow diagram showing the logic processing for capturing images with one method using the apparatus depicted in Figures 1-26. More particularly, Figure 27 illustrates logic processing used to capture images that are aligned.
  • a logic flow diagram illustrates the steps implemented by any one of the camera systems of Figures 1-26 when performing a physical alignment between cameras in the system.
  • Step "S1" a camera system (monoscopic or stereoscopic) provides a first camera and a second camera carried by a base to have a line of sight defining a respective field of view.
  • the second camera has a field of view at least in part adjacent to the field of view for the first camera, the first camera carried by the base for adjustable positioning of the field of view relative to the field of view for the second camera.
  • Step "S2" the system axially aligns the first camera relative to the second camera to render collinear an image segment within a field of view for the first camera relative to a corresponding image segment withi n the field of view for the second camera.
  • Step "S3" the process proceeds to Step "S3".
  • Step "S3" the system angularly aligns the first camera relative to the second camera to render the image seg ment within the field of view for the first camera angularly alig ned relative to the image seg ment in the field of view for the second camera.
  • the process either ends, or proceeds to success adjacent third , fourth , etc. camera for axial aligni ng and angularly aligni ng such cameras.
  • a logic flow diagram illustrates the steps i mplemented by any one of the camera systems of Figures 1 -26 when performi ng a physical align ment between cameras in the system.
  • Step "SS1” a stereoscopic camera system provides a first stereoscopic pai r of left and right cameras and a second stereoscopic pair of left and rig ht cameras carried by the base. After performing Step “SS1 ", the process proceeds to Step "SS2".
  • Step "SS2" the system axially and angularly aligns an image seg ment withi n a field of view for a fi rst left camera with a corresponding i mage segment within a field of view for a second left camera.
  • Step "SS3" the process proceeds to Step "SS3".
  • Step "SS3" the system axially and angularly aligns an image seg ment within a field of view for the fi rst left camera with a corresponding image segment within a field of view for the fi rst right camera. After performing Step "SS4", the process proceeds to Step "SS4".
  • Step "SS4" the system axially and angularly aligns an image seg ment withi n the field of view for the first rig ht camera with a corresponding image segment within the field of view for the second right camera.
  • Step "SS2" the process either ends or proceeds back to Step "SS2" and repeats for successive adjacent pairs of stereoscopic cameras.
  • This computer will display the left and rig ht cameras i n two panoramic views overlaid (either simultaneously or alternatively in rapid succession) on top of each other in order to evaluate and adjust offsets for stereoscopic viewing.
  • U ncap Camera R3 (preferably, on a middle or central pair) .
  • Usi ng a reference (envi ron mental or something designed) adjust the R3 camera until you are satisfied with the camera level .
  • U ncap Camera L3. Adjust L3 u ntil you are satisfied that L3 and R3 make a viable stereoscopic pair (P3) . Test this with the 3 D glasses to confirm the quality of the alig nment. Adjust either camera as needed .
  • a first right camera and an adjacent second right camera are aligned.
  • a first right camera and a first left camera are also aligned.
  • a first left camera and a second left camera are aligned relative to each other. This process is repeated for successive adjacent cameras until all pairs of cameras within a stereoscopic camera system have been aligned. Alignment occurs by aligning a scan line from a captured image from each adjacent camera both in horizontal alignment and angular alignment, such as by adjusting pitch and roll of one camera relative to the other camera.
  • the images/frames captured using the camera system described in this invention are processed using several different techniques based on the application and end user needs.
  • the images are cropped at the two edges and warped along a straight line for fast processing and immediate (real-time or near real-time) consumption by a viewer.
  • each scan line (or a group of scan lines) is processed individually using pattern recognition techniques to determine a non-linear scan line merging/warping.
  • This second method may be suitable for parallel processi ng using multiple computers to reduce processi ng time.
  • the processed i mage will appear seamless to the user as the user pans from one camera pair of i mages to the next.
  • a viewing system will be provided to the end user for viewing the data captured and processed as described above.
  • the viewing system will allow the user to control the view direction withi n the total field of view captured by the enti re camera set. For example, if the viewer's viewing device (stereoscopic headset, stereoscopic screen - with or without glasses, etc.) provides a field of view or 45 degrees, the user can pan left/rig ht (or any other direction depending on the data captured and processed by the system described above) u ntil the li mits of the data set are reached .
  • the user will also be provided with a focus angle control that will allow the user to adjust the view angle between the left and rig ht images by rotating the panoramic left and right i mages using different angles.
  • a stereoscopic image processing system having : processi ng ci rcuitry for retrieving the images and processing the i mages ; memory for storing the images ; and an i mage adjustment mechanism for adjusti ng the left and right eye image sets independently of each set to mini mize the visibility of seams between i mages taken from different cameras.
  • a stereoscopic image processing system having : processi ng ci rcuitry for retrieving the images and processing the i mages ; memory for storing the images ; and an i mage adjustment mechanism for adjusting each left and rig ht eye i mage pai r and combini ng with the adjustment in the previous paragraph to mini mize the visibility of seams between stereoscopic i mages taken from different cameras.
  • a stereoscopic i mage presentation system is provided havi ng : a visual output device configured to output a left stereoscopic image and a rig ht stereoscopic i mage to a respective left eye and a respective right eye of a viewer; processing circuitry for retrievi ng the i mages and presenting the i mages ; memory for storing the images ; and a user interface communicating with the processing circuitry and configured to adjust the viewing direction for the viewer by selection of a portion of the processed or unprocessed i mage data corresponding to the actual field of view of the display system realized by the user.
  • a stereoscopic camera system is provided i ncludi ng a support structu re ; a plurality of pairs of stereoscopic cameras comprisi ng a left camera and a right camera; a plurality of camera mounti ng platforms each supporting at least one of a left camera and a rig ht camera of a specific pair of stereoscopic cameras ; and a plurality of articulati ng support structures each configured to adjustably position a respective camera mou nt platform relative to the base to axially and angularly align two adjacent left cameras and two adjacent rig ht cameras withi n adjacent pairs of stereoscopic left and right cameras havi ng adjacent fields of view.
  • each left camera and right camera withi n a pair of stereoscopic cameras is supported by a u nique camera mounting platform and a u nique articulating support structure for alig nment of the respective camera relative to the support structu re.
  • each articulati ng support structure enables adjustable positioni ng of a respective camera mou nt platform along at least two degrees of freedom.
  • each articulati ng support structure enables pitch adjustment of a respective camera field of view.
  • each articulati ng support structure enables yaw adjustment of a respective camera field of view. Additionally, in one case each articulati ng support structure enables roll adjustment of a respective camera field of view.
  • each articulati ng support structure enables vertical displacement adjustment of a respective camera field of view.
  • each articulati ng support structure comprises a pair of plates each with a central socket, a spherical bearing disposed within each socket, with the pair of plates in opposed relation , and a plurality of th readed fasteners extendi ng between the pai r of plates to pivotally adjust one plate relative to the other plate about the spherical bearing and sockets.
  • one plate i n cludes a cylindrical turntable havi ng camera fastener mounts for receivi ng a camera, the turntable mounted to the one plate and the another plate i ncludes fastener mou nts for affixi ng the another plate to the support structu re.
  • an apparatus for capturi ng stereoscopic i mages including a support structure, a fi rst pai r of stereoscopic cameras, and a second pair of stereoscopic cameras.
  • the fi rst pair of stereoscopic cameras includes a fi rst left camera and a fi rst rig ht camera mou nted to the base and configured to simulate human depth perception and having a first field of view for the fi rst left camera and the first right camera.
  • the second pair of stereoscopic cameras includes a second left camera and a second right camera mou nted to the base and configu red to si mulate hu man depth perception and having a second field of view for the second left camera and the second rig ht camera.
  • the first field of view for each of the first left camera and the first right camera extends beyond a range of the second field of view for each of the second left camera and the second right camera.
  • a stereoscopic image presentation system is provided i ncludi ng a visual output device, processing ci rcuitry, memory, and a user interface.
  • the visual output device is configured to output a left stereoscopic image and a right stereoscopic image to a respective left eye and a respective right eye of a viewer.
  • the processing circuitry is operative for retrieving the images and presenting the images.
  • the memory is operative for storing the images.
  • the user interface communicates with the processing circuitry and is configured to adjust a lateral or rotational offset between the left stereoscopic image and the right stereoscopic image to produce an adjustable stereoscopic convergence angle for the viewer.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Stereoscopic And Panoramic Photography (AREA)
  • Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
  • Studio Devices (AREA)
  • Accessories Of Cameras (AREA)
  • Image Input (AREA)

Abstract

An apparatus is provided for capturing images including a base, and image capture adjustment mechanism, a first camera, and a second camera. The base is constructed and arranged to support an alignable array of cameras. The image capture adjustment mechanism is disposed relative to the base for adjusting an image capture line of sight for a camera relative to the base. The first camera is carried by the base, operably coupled with the image capture adjustment mechanism, and has an image capture device. The first camera has a line of sight defining a first field of view adjustable with the image capture adjustment mechanism relative to the base. The second camera is carried by the base and has an image capture device. The second camera has a line of sight defining a second field of view extending beyond a range of the field of view for the first camera in order to produce a field of view that is greater than the field of view provided by the first camera. A method is also provided.

Description

Apparatus and Method for Capturing Images
CROSS REFERENCE TO RELATED APPLICATIONS This application claims priority to U.S. Provisional Patent
Application Serial No. 61/322,714 which was filed on April 9, 2010, and U.S Patent Application No. 13/071,353 which was filed on March 24, 2011, the entirety of each of which are incorporated by reference herein. TECHNICAL FIELD
The present invention pertains to systems and methods for capturing detectable phenomena, such as images, sounds, or any measurable physical phenomena. More particularly, the present invention relates to the capture of images using an array of detectors over ranges that exceed the range for a single detector, such as monoscopic detectors, as well as for stereoscopic images captured with an array of detectors over ranges that exceed the range for a single pair of stereoscopic detectors.
BACKGROUND OF THE INVENTION
Techniques are known for capturing panoramic images. One technique involves rotating a camera about a central axis while capturing overlapping images that are spliced together using software. Another technique involves capturing a series of overlapping images, then splicing together adjacent overlapping images in order to produce an image that is wider than an image captured by a single camera.
A stereoscopic pair of cameras has been used to capture a stereoscopic field of view. However, there exist problems with capturing a field of view that is greater than the field of view for a stereoscopic pair of cameras. Furthermore, if wide angle cameras are used , such as cameras with a 1 80 degree field of view, adjacent left and rig ht cameras can interfere with each other.
SU M MARY OF TH E I NVENTION
An array of monoscopic detectors, as well as an array of stereoscopic pairs of detectors, are provided to capture information from a surrounding environment, such as monoscopic images or stereoscopic images, and audio inputs from ranges exceeding that for a si ngle detector or stereoscopic pair of detectors. For the case of i mage inputs, stereoscopic pai rs of cameras are provided in an array. For the case of audio inputs, pairs of separated , stereoscopic di rectional microphones are provided in an array with a microphone substituting for each camera.
Accordi ng to one aspect, an apparatus is provided for capturi ng i mages includi ng a base, and i mage capture adjustment mechanism, a first camera, and a second camera. The base is constructed and arranged to support an alignable array of cameras. The i mage captu re adjustment mechanism is disposed relative to the base for adjusting an i mage capture li ne of sig ht for a camera relative to the base. The fi rst camera is carried by the base, operably coupled with the image capture adjustment mechanism , and has an image capture device. The first camera has a line of sight defi ning a fi rst field of view adjustable with the image capture adjustment mechanism relative to the base. The second camera is carried by the base and has an image captu re device. The second camera has a line of sight defining a second field of view extendi ng beyond a range of the field of view for the first camera in order to produce a field of view that is greater than the field of view provided by the first camera.
Accordi ng to another aspect, an apparatus is provided for captu ri ng digital images including a base, a first digital camera, a second digital camera, and an i mage captu re adjustment mechanism. The base is constructed and arranged to support a plurality of cameras for physically alig ning images captured by adjacent cameras relative to one another. The fi rst digital camera is supported by the base and has an i mage capture device. The first camera has a line of sight defini ng a first field of view. The second digital camera is carried by the base and has an image capture device. The second camera has a line of sig ht defi ning a second field of view extendi ng beyond a range of the field of view for the first camera i n order to produce an adjacent field of view that extends beyond the field of view provided by the first camera. The image capture adjustment mechanism is disposed relative to the base and is operatively coupled with the first digital camera for physically adjusting an i mage capture line of sight for the first digital camera relative to the base and the second digital camera to align an adjacent field of view for the first digital camera relative to the second digital camera. Accordi ng to yet another aspect, a method is provided for captu ri ng i mages, including : providing a fi rst camera and a second camera carried by a base to have a line of sight defi ning a respective field of view, the second camera havi ng a field of view at least in part adjacent to the field of view for the first camera, the first camera carried by the base for adjustable positioning of the field of view relative to the field of view for the second camera; axially aligni ng the first camera relative to the second camera to render colli near an i mage segment within a field of view for the fi rst camera relative to a corresponding image segment within the field of view for the second camera; and angularly aligni ng the first camera relative to the second camera to render the image segment withi n the field of view for the first camera angularly aligned relative to the i mage seg ment i n the field of view for the second camera.
Accordi ng to even another aspect, a stereoscopic camera system provides an apparatus for capturi ng a stereoscopic field of view. The stereoscopic camera system i ncludes a support structu re, a plurality of pairs of stereoscopic cameras, a plurality of camera mounting platforms, and a plurality of articulating support structures. Each of the plurality of pairs of stereoscopic cameras includes a left camera and a right camera. Each of the plurality of camera mounting platforms supports at least one of a left camera and a right camera of a specific pair of stereoscopic cameras. Each of the plurality of articulating support structures is configured to adjustably position a respective camera mount platform relative to the base to axially and angularly align two adjacent left cameras and two adjacent right cameras within adjacent pairs of stereoscopic left and right cameras having adjacent fields of view. A left and right eye camera pair as described in this invention also includes cameras that include special lenses to collect stereoscopic left/right eye images using a single camera.
BRIEF DESCRIPTION OF THE DRAWINGS
Preferred embodiments of the disclosure are described below with reference to the following accompanying drawings.
Fig. 1 is perspective view of an environment including a basketball court being captured with a 180 degree stereoscopic camera system, or apparatus, according to one embodiment. Fig. 2 is an enlarged perspective view of the stereoscopic camera system of Fig. 1 shown positioned at one end of a courtside scorer's table.
Fig. 3 is a plan view of the basketball court of Fig. 1 showing additional and/or optional placements for additional stereoscopic camera systems about the environment.
Fig. 4 is a perspective view from above of a second 180 degree stereoscopic camera system, similar to that depicted in Figs. 1-3, but self-supported atop a post and base. Fig. 5 is a perspective view from above of a third stereoscopic camera system , si milar to that depicted i n Fig. 4, but with a 360 degree stereoscopic camera array self-supported atop a tripod .
Fig. 6 is an enlarged perspective view from above of the 1 80 degree stereoscopic camera system of Fig . 4.
Fig. 7 is an enlarged perspective view of the stereoscopic camera system of Fig. 6 with portions removed .
Fig. 8 is an enlarged perspective view from above of the 360 degree stereoscopic camera system of Fig . 5. Fig. 9 is an enlarged perspective view of the stereoscopic camera system of Fig. 9 with portions removed .
Fig. 1 0 is a simplified plan view from above of a pai r of 1 80 degree fisheye cameras configured to capture stereoscopic i mages and illustrati ng interference and camera spacing complexities. Fig. 1 1 is a si mplified plan view from above of three adjacent sets of stereoscopic cameras from the stereoscopic camera system of Figs 4 and 6-7 or Figs. 5 and 8-9, as well as the stereoscopic camera system of Figs. 1 -3, but where each camera is suspended beneath the base plate, and illustrating lens fields of view for two adjacent right cameras.
Fig. 1 2 is simplified plan view correspondi ng with the view in Fig. 1 1 , but taken further away to illustrate overlapping lens fields of view for the two adjacent right cameras.
Fig. 1 3 is a si mplified schematic plan view of a basketball cou rt showi ng overlapping fields of view for two adjacent rig ht cameras for the stereoscopic camera system of Figs. 1 -3.
Fig. 1 4 is an enlarged perspective view of the base plate for the stereoscopic camera system of Fig . 6 (as well as Figs. 1 -3, if flipped upside down) and showing one camera and articulating support structure in exploded view that provides an image capture adjustment mechanism.
Fig. 15 is another enlarged perspective view of the base plate of Fig. 14 and showing two adjacent pairs of stereoscopic cameras, each mounted to the base plate with a dedicated articulating support structure.
Fig. 16 is a further enlarged breakaway perspective view of the camera, base plate and articulating support structure of Fig. 15. Fig. 17 is another embodiment of an articulating support structure suitable for use with the stereoscopic cameras systems of Figs. 1-9 and 11-16, providing another embodiment for an image capture adjustment mechanism.
Fig. 18 is an exploded perspective view of the articulating support structure of Fig. 17.
Fig. 19 is a pedestal for mounting the stereoscopic camera systems of Figs.4 and 5 atop a post and a tripod, respectively.
Fig. 20 is vertical sectional view of the pedestal taken along line 20-20 of Fig. 19. Fig.21 is a perspective view from above of a fourth stereoscopic camera system, similar to that depicted in Fig. 5, but with a 180 degree stereoscopic camera array using a housing with discrete planar glass windows for each set of left and right stereoscopic cameras. Fig. 22 is an enlarged perspective view of the stereoscopic camera system of Fig. 21 with housing portions removed in order to illustrate camera system components within the housing. Fig. 23 is a further enlarged breakaway perspective view of the camera, base plate and articulating support structure of Figs. 21 and 22, providing even another embodiment for an image capture adjustment mechanism. Fig. 24 is a simplified diagrammatic view of even another image capture adjustment mechanism providing a camera lens that is repositionable within a plane perpendicular to the viewing direction.
Fig. 25 is a simplified diagrammatic view of yet another image capture adjustment mechanism providing an image capture device (CCD) that is repositionable within a plane of the image capture device.
Fig. 26 is a simplified diagrammatic view of yet even another image capture adjustment mechanism providing a repositionable mirror between a camera lens and an image capture device (CCD). Fig. 27 is a logic flow diagram illustrating one method for physically aligning adjacent image capture devices provided on a support base.
Fig. 28 is a logic flow diagram illustrating another method for physically aligning a plurality of adjacent stereoscopic pairs of left and right cameras provided on a support base.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
This disclosure is submitted in furtherance of the constitutional purposes of the U.S. Patent Laws "to promote the progress of science and useful arts" (Article 1, Section 8).
Embodiments of the present invention disclose an apparatus and method for capturing information from a surrounding environment using an array of detectors. According to one construction, monoscopic and stereoscopic images, such as still images and video images or frames are captured using an array of cameras. According to other constructions, directional audio inputs are captured with an array of directional microphones. Further optionally, arrays of detectors can be used to capture infrared, ultrasonic, sonic, subsonic, ultraviolet, or electromagnetic events or signals.
Various embodiments described herein are described with reference to figures. However, certain embodiments may be practiced without one or more of these specific details, or in combination with other known methods and configurations. In the following description, numerous specific details are set forth, such as specific configurations and methods, etc., in order to provide a thorough understanding of the present invention. In other instances, well-known construction techniques and methods have not been described in particular detail in order to not unnecessarily obscure the present invention. Reference throughout this specification to "one embodiment" or "an embodiment" means that a particular feature, configuration, composition, or characteristic described in connection with the embodiment is included in at least one embodiment of the invention. Thus, the appearances of the phrase "in one embodiment" or "an embodiment" in various places throughout this specification are not necessarily referring to the same embodiment of the invention. Furthermore, the particular features, configurations, compositions, or characteristics may be combined in any suitable manner in one or more embodiments.
As used herein, the term "field of view" is understood to mean the angular or linear or areal or volumetric extent of the observed world that is seen or viewed by an image capture device at any given moment. As used herein, the term "adjustable line of sight" is understood to encompass one or both of translation or rotation, including modifying the inter-camera spacing between adjacent cameras in a pair.
As used herein, the term "image capture adjustment mechanism" is understood to mean any structural mechanism capable of physically aligning or moving (angularly or linearly) the position of an image of a target object on an image capture device.
Figures 1-3 are illustrations of an apparatus and method for capturing and aligning stereoscopic camera images from an array of multiple pairs of stereoscopic cameras provided by a stereoscopic camera system 10 within an environment 12, such as a basketball court. For purposes of disclosure, it is understood that camera system 10 of Figures 1-2 is essentially the same as camera system 110 of Figures 4, 6-7 and 10-15, except that stereoscopic camera system 10 is mounted up-side-down in the application of Figures 1-3 and the housing side plate 46 and back plate 48 of system 110 are extended downwardly in height by two inches sufficient to provide a support base that extends below connectors 40 so that system 10 can be supported atop a scorer's table 14, as shown in Figures 1 and 2. In contrast, the right-side-up configuration of system 110 depicted in Figures 4 and 6-7 uses a pedestal 238, post 36 and base plate 38 to support system 110 with cameras at a desired viewing height, typically corresponding with eye level height for a typical observer present within an environment under normal observation conditions. An observer can either be sitting or standing, depending on the particular environment and event being observed. Further optionally, the viewing height can be some other desirable height simulating either an elevated or a lowered condition, such as simulating an extra-tall person's view, birds-eye view or a floor-level view. Even further optionally, cameras can be placed in adjacent vertical rows to provide an elevational field of view with an angle between vertical pairs, such as in a spherical housing with cameras pointing radially outwardly in all directions. Such a vertical configuration can be implemented by stacking a plurality of horizontal base plates one atop another, with each base plate having an array of physically alignable cameras. Further optionally, an array of cameras can be mounted individually onto a vertically extending base plate (in contrast with a horizontally extending base plate as depicted in Fig. 9), or onto a baseplate having an angular configuration somewhere between horizontal and vertical.
As shown in Figure 1-2, stereoscopic camera system 10 is supported atop scorer's table 14 by a bottom edge of housing 16 (see Fig. 2). In one case, stereoscopic camera system 10 is supported at one end of scorer's table 14, where table 14 is position at court-side, typically at center court. An array of stereoscopic pairs of video cameras, such as left video camera 32 and right video camera 34 of pair 20, are supported within housing 16 in order to capture stereoscopic video images across a range of different fields of view, for given video camera fields of view. Each video camera 34 captures images through a port, or aperture 18 in housing 16.
Figure 3 illustrates additional locations for stereoscopic camera system 10 alongside a basketball court environment 12. In this embodiment, camera system 10 is provided in multiple locations, and stereoscopic image capture and retrieval can be provided to a viewer remote in space and/or time from each of the camera systems 10. In one case, a viewer can select from which camera system 10 they are going to view stereoscopic video images. It is understood that camera system 110 of Figures 4 and 6-7, as well as camera system 210 of Figures 5 and 8-9 can also be used at any of the locations of camera system 10 in Figure 3. Although environment 12 is shown as a basketball court, it is understood that system 10 can be place in other environments, such as football fields, hockey rinks, museums, or any location where an individual may desire observing the real environment as a virtual environment displaced in either time or place, or both. It is also understood that cameras can generate pictures or video. In a museum application, system 10 can be moved within a museum to known locations in order to take pictures at many discrete locations in order to capture images from all angles within the museum, and a user can then visit the museum by retrieving the images at a remote location. Furthermore, the inter-camera spacing between stereoscopic pairs of cameras can be changed from a typical eye spacing for a human, say 2.5 inches, to a greater (or lesser) distance, say for a tyrannosaurus rex, thereby providing a lateral translating adjustable line of sight. Figures 4 and 6-7 illustrate an embodiment where cameras, such as cameras 32 and 34 of stereoscopic set 20 (see Fig.4) are provided along a bottom edge of housing 116. In such case, electrical connectors 40 extend through a top of housing 116. Each camera 32 and 34 receives video images through a respective aperture 118. As shown in Figure 4, camera system 110 is mounted atop a post 36 of desired height. Post 36 is mounted atop a base 38 to provide stability. Optionally, a directional microphone can be substituted for each camera.
As shown in Figure 6, housing 116 includes a base, or support plate 42, a top plate 44, a semi-circular vertical plate 46 and a back plate 48. Each camera 32 and 34 is coupled to a data and power connector 40 that extends through an aperture 50 in top plate 44. Threaded cap-head bolts 52 couple together top plate 44, side plates 46 and 48, and bottom plate 42. An L-shaped bracket 62 with a bore is mounted between each cap screw 52 that mounts through bottom plate 42, providing a shelf for supporting a bottom edge of side plate 46.
As shown in Figure 7, portions of housing 116 of stereoscopic camera system 110 are removed in order to observe a plurality of pairs 20-25 of stereoscopic left cameras 32 and right cameras 34. Each camera is mounted to base plate 42 with an image capture adjustment mechanism in the form of an articulating support structure, or base frame 56 that can be used to align the camera relative to the base plate 42. Optionally, a directional microphone can be mounted to each base frame, substituting for each camera 32, 34 in pairs 20- 25. Optionally, an image capture device, similar to cameras 32 and 34, can be provided for capturing electromagnetic radiation in one or more of any of the spectrums, including infrared and/or optical (visible) portions of the electromagnetic spectrum. Further optionally, an image capture device can be provided for capturing ultrasonic images using an array of ultrasonic image capture devices. As shown in Figure 7, each post 54 has a threaded bore at each end sized to receive a respective threaded cap-screw, or fastener 52 (see Fig. 6). Base plate 42 and top plate 44 (see Fig. 6) each have corresponding holes that align with each post 54 to receive a fastener 52, where the enlarged head of fastener 52 secures respective plates 42 and 44 together against side plate 46 and back plate 48. Back plate 48 includes a ventilation port 58 in which a cooling fan is mounted (not shown) and a switch port in which a power switch (not shown) is mounted for turning the fan on and off. An array of four bores 61 are configured in bottom plate 42 to align with corresponding bores 242 in pedestal 238 (see Fig. 18) so each receives a bolt and nut (not shown) to secure system 116 atop pedestal 238. Pedestal 238 is then mounted atop post 36 (see Fig. 4) or atop a tripod, such as tripod 236 shown in Figure 5 using a threaded mounting aperture 245 (see Fig. 19). When assembled together, bore 60 aligns with bore 243 of pedestal 238 (of Fig. 18) to enable communications cables and power cables for cameras 32 and 34 to exit housing 116.
Optionally, the construction of camera system 110 of Figure 7 (or optional configurations depicted in Figures 5 and 8-9) can be constructed to have a monoscopic array of detectors, such as video cameras. Such a construction would generate aligned 180 degree or 360 degree monoscopic output, or video, where the field of view of adjacent cameras can be physically aligned using individual base plates and image captu re adjustment mechanisms and methods. As a further option , each left camera can be turned off and the output from each rig ht camera can be captured i n order to achieve panoramic 1 80 degree or 360 degree video imagi ng (or each right camera can be turned off) . According to this optional i mplementation, the resulting array of cameras would not be arranged concentrically about a poi nt. For the case where only a monoscopic array of adjustable cameras are provided on base plates, the cameras can be arranged concentrically about a point, or they can be arranged so that they are not concentric.
Accordi ng to a further optional construction, an array of monoscopic cameras can be provided , each on an adjustable base plate with an i mage capture adjustment mechanism . A mi rrored adapter can be mounted onto each monoscopic camera lens in order to generate alternating left and right offset stereoscopic video i mages. Such a mi rrored stereoscopic 3D camera adapter is provided by a NuView SX2000 video lens adapter sold by Mi ndlfux, Jasandre Pty. Ltd. , P.O. Box 494, Roseville, NSW 2069 Australia, which generates 3- D field sequential video, under a patent pending process entitled STE R EO-OPTIX!.
Figu res 5 and 8-9 illustrate another embodiment for a stereoscopic camera system 21 0 having a full 360 degree array of pairs 20 of stereoscopic left and rig ht cameras 32 and 34 supported withi n housi ng 21 6. Housi ng 21 6 mounts via pedestal 238 atop a tripod, or support structure 236, as shown in Figure 5. As shown i n Figure 8, threaded cap screws 52 affix together a top plate 244, a side plate 246, and a bottom plate 242 for housing 21 6 of camera system 21 0. Left and rig ht video cameras 32 and 34 form a stereoscopic pai r 220, with a lens from each camera extending th roug h a respective aperture 1 8 in side plate 246. An L-shaped bracket 62 with a bore is mou nted between each cap screw 52 that mounts th rough bottom plate 242, providing a shelf for supporting a bottom edge of side plate 246.
As shown in Figure 9, stereoscopic camera system 210 includes a complete circumferential array of stereoscopic pairs 220-231 of left cameras 32 and right cameras 34 used to capture dynamic (or static) landscape images within an environment surrounding system 210. A circumferential array of posts 54, threaded cap screws 52 and L- brackets 62 are used to hold together housing 216 (of Fig. 9), in a manner similar to that shown for system 110 in Figure 7. Each camera 32, 34 is mounted in an adjustable manner onto base plate 242 via an adjustable, articulating support structure 56. Bores 61 and port 60 facilitate mounting of system 210 onto pedestal 238 (of Fig 18) in a manner similar to that shown for system 110 (of Fig.7).
Figure 10 is a simplified plan view from above of a pair of 180 degree left and right fisheye cameras 1032 and 1034 configured to capture stereoscopic images. As shown in Figure 10, an interspace distance, t, between left camera 1032 and right camera 1034 is shown. For the case where images are captured directly in front on cameras 1032 and 1034, interspace distance, t, provides a maximum lateral spacing between cameras 1032 and 1034. However, such interspace distance diminishes for viewing angles Θ = 45° and Θ = 85°. As the angle, Θ, increases, interspace distance, t, diminishes, which also diminishes stereoscopic perspective. Additionally, as angle, Θ, increases, a lens on one of camera 1032 and 1034 interferes with another of lens on another of camera 1032 and 1034, which interferes with image capture. Accordingly, there is a limit on the angular field of view that is practical when capturing images from a wide field of view, thereby necessitating a solution provided by use of multiple cameras in stereoscopic configurations, as shown in the various embodiments depicted herein.
For the case where t= 2.5", a field of view for an array of six pairs of stereoscopic cameras in Figure 7 will have a trimmed field of view for each camera of 30° (each camera has an untrimmed field of view of approximately 41 °). This configuration provides a highly preferred physical alignment with minimal image distortion. For the same case where t= 2.5", a field of view for an array of five pairs of stereoscopic cameras (covering 180 degree range) will have a trimmed field of view for each camera of 36° (approximately 45° untrimmed). This configuration provides a preferred physical alignment with slightly greater image distortion than for a six pair array. For the same case where t= 2.5", a field of view for an array of four pairs of stereoscopic cameras (covering 180 degree range) will have a trimmed field of view for each camera of 45° (approximately 60° untrimmed). This configuration provides a moderately preferred physical alignment with slightly greater image distortion than for a six pair array. For the same case where t= 2.5", a field of view for an array of three pairs of stereoscopic cameras (covering 180 degree range) will have a trimmed field of view for each camera of 60° (approximately 75° untrimmed). This configuration provides a reasonably preferred physical alignment with slightly greater image distortion than for a six pair array. For the same case where t= 2.5", a field of view for an array of two pairs of stereoscopic cameras (covering 180 degree range) will have a trimmed field of view for each camera of 90° (approximately 110° untrimmed). This configuration provides a somewhat preferred physical alignment with slightly greater image distortion than for a six pair array. The difference between untrimmed field of view and trimmed field of view is the extent of overlap between adjacent cameras on either side of a specific camera, after alignment of the adjacent cameras.
Figures 11 and 12 illustrate in plan view one exemplary set of adjacent right cameras 34 from adjacent pairs of stereoscopic pairs 20, 21 of cameras 32, 34 on stereoscopic camera system 110. More particularly, simplified representations of fields of view for right camera 34 of pair 20 and right camera 34 of pair 21 are shown. It is understood that a similar representation exists for left cameras 32 in any adjacent pair, as well as for any adjacent right cameras 34.
Each camera 34 has a lens angle. Adjacent cameras 34 have lens angles that form a convergence angle. As shown in this embodiment, there is a 30 degree angle between the central line of sight (view direction) from lenses 64 on adjacent pairs of stereoscopic cameras. Figure 12 illustrates an overlap region 66 that occurs at a specific distance away from system 110. Figure 13 further illustrates adjacent overlapping fields of view for right cameras (R1) and (R2), along with overlap region 66 within a basketball court environment. Output from respective cameras is processed using seaming and warping techniques to join together the output of adjacent right cameras and adjacent left cameras in order to produce a field of view that is greater than that provided by a single camera, or a single pair of stereoscopic cameras. However, in order to reduce computer processing, either in real time or after image capture (video or still), cameras 32 and 34 (see Fig. 11) are aligned in order to significantly reduce post-processing in order to achieve alignment of captured images between left and right cameras in a pair, as well as between right cameras in adjacent pairs and left cameras in adjacent pairs. The articulating support structure 56 (of Fig.7) enables this alignment between cameras prior to capturing images, as will be discussed below in greater detail with reference to Figures 14-16.
Figure 14 illustrates assembly of a typical camera 32 onto base plate 42 with articulating support structure 56. By adjusting the threaded engagement of individual fasteners 74, camera 32 can be adjusted relative to plate 42 to induce (relative to lens 64) pitch 101 and roll 103 (see Fig. 15). According to an alternative construction depicted in Figure 17, yaw 102 can also be induced. Furthermore, bearing 70 can be made out of an elastic material, such as a plastic, which can be slightly compressed, enabling vertical adjustment by tightening and loosening all of the fasteners. Figu re 1 5 depicts two adjacent stereoscopic pairs 20 and 21 of left cameras 32 and rig ht cameras 34, each mounted atop a dedicated articulati ng support structure 56. It is u nderstood that system 1 0 of Figures 1 -3 merely supports these pairs i n an up-side-down configuration in order to produce a more compact housi ng when the housing is supported on a table, as the individual lenses are spaced further apart from the table top su rface which mig ht otherwise interfere with i mage capture.
Figu re 1 6 illustrates construction of articulati ng support structure 56. More particularly, structu re 56 includes a support plate, or frame 68 that has a semi-spherical seat, or recess 80 on a central bottom surface for receiving a spherical bearing 70 (either plastic or hardened steel) . Camera 32 is rigidly secured onto a top surface of plate 68 with threaded fasteners 84 that pass through bores 78 in plate 68 and i nto threaded bores (not shown) in the bottom of camera 32. Threaded cap screws 74 (with fine th reads) pass th roug h bores 76 i n base plate 42, through coil steel springs 72, and i nto th readed bores 86 i n plate 68. Ball beari ng 70 also seats i n another semi- spherical seat 82 in a top surface of plate 42. Beari ng 70 is larger i n diameter than the combined depths of seats 80 and 82 so that plate 68 is spaced from plate 42 in assembly. By tig htening the front pai r of fasteners 74 and loosening the rear pair of fasteners, camera 32 can be pitched forward, thereby enabling relative adjustment of the field of view compared to an adjacent camera. Likewise, tig htening of a left pair of fasteners 74 and loosening of a rig ht pair of fasteners 74 will cause camera 32 to roll left. It is understood that total compression is still maintai ned against the beari ng 70 after this adjustment is made. I n this manner, a horizontal alig nment and an angular align ment between adjacent cameras can be performed . By placing a horizontal object in front of an adjacent pai r of cameras (two lefts, two rights, or a left and a rig ht) , scan lines can be adjusted to be both parallel and in align ment horizontally between adjacent cameras using articulating support structure 56. Figure 17 illustrates another embodiment for an image capture adjustment mechanism, or articulating support structure 156 that further enables adjustment to rotate camera 32 in order to induce yaw 102. More particularly, a turret plate 170 is captured for constrained rotation between an upper plate 168 and a lower plate 169. A radial arm 171 on plate 170 is constrained within a slot 172 to enable rotational adjustment of turret plate 170 relative to plates 168 and 169. As shown in Figure 17, a set screw 173 is used to fix positioning of turret plate 170 and camera 32 relative to plates 168 and 169. More particularly, threaded fasteners 84 pass through bores 179 in plate 170 and into complementary threaded bores (not shown) in the bottom of camera 32. Camera 34 is similarly mounted. Threaded recessed head screws 184 pass through bores 182 in plate 169 and into complementary threaded bores 180 in plate 168 to hold together plates 168-170. Threaded fasteners 74 pass through the base plate (not shown), springs 72, and into complementary threaded bores 182. This traps bearing 70 in a manner similar to that shown in Figure 16 and adjustment of screws 74 enables pitch and roll adjustment of camera 32, while turret 170 enables yaw adjustment. Figure 19 illustrates construction details of pedestal 238. A top ring 239 and a bottom plate 240 are held together with four posts 241 using threaded fasteners 244. Central bore 243 enables passage of communications and power cables passing to the cameras on the system mounted atop pedestal 239 via fasteners passing through bores 242. As shown in Figure 20, a threaded recess 245 is used to affix plate 240 atop a post or tripod using a complementary threaded fastener.
According to one construction, cameras 32 and 34 are each an IDS GigE Model Number UI-5649 HE-C, a high-performance GigE camera with large functional range. Image data from an Aptina CMOS sensor in 1.3 Megapixel resolution (1280 x1024 pixels) is output with up to 12 bits per channel. An internal FPGA with 64 MB image memory offers additional features and ensures fast and reliable data transfer. Besides a lockable GigE port, the UI-5649 HE-C comes with a multi-l/O-interface including 4 digital in-/outputs and an RS232 interface. An optional camera is the IDS GigE Model Number Ul- 5640HE. Both cameras are available in the United States at IDS Imaging Development Systems, Inc., 400 West Cummings Park, Suite 3400, Woburn, MA 01801. Such cameras capture still or video images, where video images are time displaced still images.
Figures 21-23 illustrate another embodiment for a stereoscopic camera system 310 having a 180 degree array of pairs 320 of stereoscopic left and right cameras 332 and 334 supported within housing 316. Housing 316 mounts via pedestal atop a tripod, or support structure, similar to support structure provided by post 36 and base plate 38 shown in Figure 5. As shown in Figure 21, threaded cap screws 352 affix together a top plate 344, a side plate 346, and a bottom plate 342 for housing 316 of camera system 310. Left and right video cameras 332 and 334 form a stereoscopic pair 320, with a lens from each camera extending through a respective aperture 318 in side plate 346. Housing 316 is formed by welding together a plurality of individual vertical face plates and a back plate 348 atop a bottom plate 342. A plurality of apertures, or windows 318 are provided in vertical faces of housing 316 for each pair 320-325 (see Fig. 22) of stereoscopic cameras. An aluminum bezel plate 382 is used to retain a rectangular glass plate 380 over each window 318, preferably with one or more rubber gaskets (not shown) provided between glass 380 and housing 316 and/or bezel 382.
Figure 22 illustrates internal components mounted atop bottom plate 342 and within housing 316 of camera system 310. More particularly, pairs 320-325 of stereoscopic cameras, such as cameras 332 and 334, are mounted atop plate 342, each with an image capture adjustment mechanism that enables physical alignment between cameras. More particularly, a camera power fuse board 360, a power supply 362, and a sound fuse board 364 are affixed to a back in ner surface of housing 31 6. A fiber cable multiplexer, or fiber switch, 368 is affixed atop bottom plate 342. An array of seven individual Ethernet-fiber media/mode converters 366 are mounted atop multiplexer switch 368. Each converter 366 has respective ports for receivi ng two RJ45 Ethernet con nectors from each pai r of cameras and a fiber optic connector for sending a fiber optic output signal to fiber multiplexer, or switch 368 where signals are combi ned i n order to generate a si ngle output signal delivered on a fiber to video and audio production facility. One suitable converter 366 is an I E-Multiway, 1 0/1 00/1 000 Mbps Ethernet Media/Mode converter, sold by I MC Networks, of 1 9772 Pauling, Foothills Ranch , CA 9261 0.
As shown i n Figu re 22, first camera 332 and second camera 334 are carried by the base 342 for adjustable positioni ng using an image captu re adjustment mechanism. Adjacent left cameras, as well as adjacent rig ht cameras, have at least i n part adjacent fields of view. Adjacent first and second right cameras (as well as adjacent fi rst and second left cameras) are axially aligned to render collinear an image seg ment within a field of view for the second camera relative to an i mage seg ment within the field of view for the first camera. Adjacent first and second right cameras (as well as adjacent first and second left cameras) are angularly aligned to render the image segment withi n the field of view for the second camera angularly aligned relative to the i mage segment in the field of view for the first camera. As shown in Figure 23, each camera 332 is supported atop bottom plate 342 in a manner that enables physical adjustment of camera 332 (and lens 364) so as to alig n the resulting field of view relative to adjacent and related cameras (see Fig . 22) . More particularly, adjacent left and right cameras 332 and 334 within a pai r 321 of stereoscopic cameras can be physically adjusted to alig n respective fields of view, thereby reducing or eli minating any need to perform alignment and/or adjustment using software techniques. I n many cases software tech niques are not ideally suited for live broadcast, as they require computational power, which can introduce significant ti me delays. Furthermore, nearest-neighbor left cameras and nearest-neighbor rig ht cameras can also be physically adjusted in order to align respective fields of view.
Figu re 23 illustrates one construction for the image capture adjustment mechanism used i n Figure 22 comprising a camera mou nting platform , or plate 368, and an articulating support structure comprisi ng a spherical beari ng 70 seated between two semi-spherical seats 380 and 382 i n plates 368 and 342, respectively. Threaded cap screws 374 pass through a respective hard plastic bushing 375, aperture 376 i n plate 342, hard plastic bushing 377, coil steel spri ng 372, and i nto threaded bore 378 in plate 368. Four th readed fasteners 384 pass through bores 386 i n plate 368 and i nto respective threaded bores (not shown) i n a bottom plate of each camera 332 i n order to rigidly secure camera 332 atop plate 368. Camera 332 and plate 368 can be adjusted i n pitch and roll by adjusti ng threaded position of each screw 374 so as to d rop and tilt, respectively, lens 364. Adjacent cameras 332 and 334 are carried by a com mon support base 342 (see Fig . 22) .
Accordi ng to one construction , bushings 375 and 377 each have an in ner bore sized slig htly smaller than an outer thread diameter on each screw 375. Screw 374 self-taps i nto bushing 375 upon threaded insertion. Such construction enables the removal of one machine screw 374 for maintenance or replacement, after which such screw can be rei nserted and specific nu mber of tu rns can be applied to the screw equal to the number needed for removal in order to achieve a close approxi mation to the original position. Fu rthermore, such construction has been fou nd to resist or eli mi nate any tendency for plate 368 to tilt relative to plate 342 in the event load is applied to camera 332, such as during mai ntenance or shipping. Essentially, threads within bushings 375 and 377 resist or prevent any stroki ng of machine screws 374 relative to plate 342.
Figu re 24 is a si mplified diagrammatic view illustrati ng layout of even another i mage captu re adjustment mechanism 456 providi ng a camera lens 462 that is repositionable withi n a plane perpendicular to an image viewing direction . More particularly, lens 462 is rigidly supported by a first linear rack 474 and rotary gear 474, and a second linear rack 476 and rotary gear 478. Accordi ng to such construction , gear 474 is mounted i n axially slidable relation on a central shaft 473 in order to accommodate linear motion generated by rack 476 and gear (or pinion) 478. I n this manner, an image 466 can be aligned or repositioned relative to an i mage capture device, or CC D 464. An object 460 is captu red within a field of view 468 via lens 462 and is projected with an image range, or region of interest 470 as an i mage 466 on CC D 464 at a focal length 482. It is u nderstood that lens 462 has peripheral edge portions that are affixed to racks 472 and 476. Optionally, a peripheral edge portion of lens 462 is affixed to a carrier plate, and such plate is rigidly affixed to racks 472 and 476.
Figu re 25 is a simplified diagrammatic view illustrating yet even another i mage capture adjustment mechanism 556 providing an i mage captu re device (CC D) 564 that is repositionable withi n a plane of the i mage capture device 564. More particularly, an object 560 is captu red within a field of view 568 via a lens 562 and is projected with an i mage range 570 as an image 566 on CC D 564 at a focal length 582. CC D 564 is rigidly supported by a first linear rack 574 and rotary gear 574, and a second linear rack 576 and rotary gear 578. According to such construction gear, 574 is mounted i n axially slidable relation on a central shaft 573 in order to accommodate li near motion generated by rack 576 and gear (or pi nion) 578. Figu re 26 is a simplified diag ram matic view illustrating yet another i mage capture adjustment mechanism 656 providi ng a repositionable mirror 658 between a camera lens 662 and an i mage capture device (CCD) 664. More particularly, an object 660 is captured within a field of view 668 via lens 662, is reflected by mirror 658, and is projected with an image range 670 as an image 666 on CCD 664 at a focal length 682. Mirror 658 is rigidly supported by a first linear rack 674 and a second linear rack 676. First linear rack 674 interacts with rotary gear 674, and second linear rack 676 interacts with rotary gear 678. According to such construction, gear 678 is mounted in axially slidable relation along a guide slot 680 in order to accommodate linear motion generated by rack 676 and gear (or pinion) 678.
In order to better understand embodiments of the method, detailed examples are presented below for capturing images with respect to Figures 27 and 28.
Figure 27 forms a process flow diagram showing the logic processing for capturing images with one method using the apparatus depicted in Figures 1-26. More particularly, Figure 27 illustrates logic processing used to capture images that are aligned.
As shown in Figure 27, a logic flow diagram illustrates the steps implemented by any one of the camera systems of Figures 1-26 when performing a physical alignment between cameras in the system.
In Step "S1", a camera system (monoscopic or stereoscopic) provides a first camera and a second camera carried by a base to have a line of sight defining a respective field of view. The second camera has a field of view at least in part adjacent to the field of view for the first camera, the first camera carried by the base for adjustable positioning of the field of view relative to the field of view for the second camera. After performing Step "S1", the process proceeds to Step "S2".
In Step "S2", the system axially aligns the first camera relative to the second camera to render collinear an image segment within a field of view for the first camera relative to a corresponding image segment withi n the field of view for the second camera. After performi ng Step "S2", the process proceeds to Step "S3".
I n Step "S3", the system angularly aligns the first camera relative to the second camera to render the image seg ment within the field of view for the first camera angularly alig ned relative to the image seg ment in the field of view for the second camera. After performi ng Step "S3", the process either ends, or proceeds to success adjacent third , fourth , etc. camera for axial aligni ng and angularly aligni ng such cameras. As shown in Figure 28, a logic flow diagram illustrates the steps i mplemented by any one of the camera systems of Figures 1 -26 when performi ng a physical align ment between cameras in the system.
I n Step "SS1 ", a stereoscopic camera system provides a first stereoscopic pai r of left and right cameras and a second stereoscopic pair of left and rig ht cameras carried by the base. After performing Step "SS1 ", the process proceeds to Step "SS2".
I n Step "SS2", the system axially and angularly aligns an image seg ment withi n a field of view for a fi rst left camera with a corresponding i mage segment within a field of view for a second left camera. After performi ng Ste ; "SS2", the process proceeds to Step "SS3".
I n Step "SS3", the system axially and angularly aligns an image seg ment within a field of view for the fi rst left camera with a corresponding image segment within a field of view for the fi rst right camera. After performing Step "SS4", the process proceeds to Step "SS4".
I n Step "SS4", the system axially and angularly aligns an image seg ment withi n the field of view for the first rig ht camera with a corresponding image segment within the field of view for the second right camera. After performi ng Step "SS4", the process either ends or proceeds back to Step "SS2" and repeats for successive adjacent pairs of stereoscopic cameras.
Following is one procedure for aligni ng cameras 32 and 34 on the systems depicted above. For pu rposes of this procedure, camera pairs R1 - R6 correspond with pairs 20-25 (see Fig. 7) , respectively.
1 ) Mou nt each camera to the left and right "Camera Adjustment Plates" by screwi ng directly into the camera.
2) Mount each Camera Adjustment Plate to the Mounti ng Plate using screws (tensioned by spri ngs) . The plate will sit on top of a bearing for a central rotation point.
3) Power up all the cameras, start captu ri ng the video signals and transferring the video to the display computer. This computer will display the left and rig ht cameras i n two panoramic views overlaid (either simultaneously or alternatively in rapid succession) on top of each other in order to evaluate and adjust offsets for stereoscopic viewing.
4) Balance auto white, auto exposure, and colors for each camera and across all cameras.
5) Adjust the focus of each lens u ntil you are satisfied. Set the screws on the lens so that the focus and aperture will not be changed accidentally. Put the lens caps back on the cameras.
6) U ncap Camera R3 (preferably, on a middle or central pair) . Usi ng a reference (envi ron mental or something designed) adjust the R3 camera until you are satisfied with the camera level . 7) U ncap Camera L3. Adjust L3 u ntil you are satisfied that L3 and R3 make a viable stereoscopic pair (P3) . Test this with the 3 D glasses to confirm the quality of the alig nment. Adjust either camera as needed . 8) Cap Camera L3 and uncap Camera R2. Adjust camera R2 so that R2 and R3 are approximately aligned.
9) Adjust the cropping on the left edge of the R2 image so that the bottom of the image seam is aligned with R3. Add any warping to R2 necessary to align the upper portion of the image seam. This image should now appear to be seamless if you disregard any color/brightness differences. Alternatively, instead of adjusting cropping only at the bottom and top, the cropping can be adjusted in any of a number of positions from bottom to top in order to obtain the desired quality, limited only by making the adjustment at every single scan line (which is limited by resolution of the charge coupled device).
10) Cap Camera R2 and R3. Uncap Camera L3 and L2. Adjust the cropping on the left edge of the L2 image so that the bottom of the image seam is aligned with L3. Add any warping to L2 necessary to align the upper portion of the image seam for multiple points. This image should now appear to be seamless if you disregard any color/brightness differences.
11) Uncap Camera R2 and R3. Confirm that L2 and R2 make a viable stereoscopic pair (P2). Test this with the 3D glasses (or view it in 3D) in order to confirm the quality of alignment. Adjust cameras as needed confirming seam quality and stereoscopic quality as you go.
12) Repeat Steps 9 and 10 for P1, P4, P5, and P6.
13) Uncap all lenses and access overall quality of stereo and seams. Continue if satisfied. 14) Cap all Left Cameras. Turn off auto white balance and auto exposure. Using a color chart adjust the color and brightness of each right camera so that you have a universal appearance of color and brightness. 15) Choose the right camera which has the most exceptional color and brightness. Cap every other Right Camera.
16) Uncap all Left Cameras. Turn off auto white balance and auto exposure. Adjust the color and brightness of the Left Camera to match the corresponding Right Camera which is still showing.
17) Cap the remaining Right Camera. Using a color chart, adjust the color and brightness of each left camera so that you have a universal appearance of color and brightness.
18) Uncap all the cameras. Confirm that the color and brightness of the cameras is universal. Adjust where necessary.
19) Save configuration.
According to one method, a first right camera and an adjacent second right camera are aligned. A first right camera and a first left camera are also aligned. Furthermore, a first left camera and a second left camera are aligned relative to each other. This process is repeated for successive adjacent cameras until all pairs of cameras within a stereoscopic camera system have been aligned. Alignment occurs by aligning a scan line from a captured image from each adjacent camera both in horizontal alignment and angular alignment, such as by adjusting pitch and roll of one camera relative to the other camera.
The images/frames captured using the camera system described in this invention are processed using several different techniques based on the application and end user needs. In one method, the images are cropped at the two edges and warped along a straight line for fast processing and immediate (real-time or near real-time) consumption by a viewer. In a second method, each scan line (or a group of scan lines) is processed individually using pattern recognition techniques to determine a non-linear scan line merging/warping. This second method may be suitable for parallel processi ng using multiple computers to reduce processi ng time. The processed i mage will appear seamless to the user as the user pans from one camera pair of i mages to the next.
A viewing system will be provided to the end user for viewing the data captured and processed as described above. The viewing system will allow the user to control the view direction withi n the total field of view captured by the enti re camera set. For example, if the viewer's viewing device (stereoscopic headset, stereoscopic screen - with or without glasses, etc.) provides a field of view or 45 degrees, the user can pan left/rig ht (or any other direction depending on the data captured and processed by the system described above) u ntil the li mits of the data set are reached . The user will also be provided with a focus angle control that will allow the user to adjust the view angle between the left and rig ht images by rotating the panoramic left and right i mages using different angles.
A stereoscopic image processing system is provided having : processi ng ci rcuitry for retrieving the images and processing the i mages ; memory for storing the images ; and an i mage adjustment mechanism for adjusti ng the left and right eye image sets independently of each set to mini mize the visibility of seams between i mages taken from different cameras.
A stereoscopic image processing system is provided having : processi ng ci rcuitry for retrieving the images and processing the i mages ; memory for storing the images ; and an i mage adjustment mechanism for adjusting each left and rig ht eye i mage pai r and combini ng with the adjustment in the previous paragraph to mini mize the visibility of seams between stereoscopic i mages taken from different cameras.
A stereoscopic i mage presentation system is provided havi ng : a visual output device configured to output a left stereoscopic image and a rig ht stereoscopic i mage to a respective left eye and a respective right eye of a viewer; processing circuitry for retrievi ng the i mages and presenting the i mages ; memory for storing the images ; and a user interface communicating with the processing circuitry and configured to adjust the viewing direction for the viewer by selection of a portion of the processed or unprocessed i mage data corresponding to the actual field of view of the display system realized by the user.
A stereoscopic camera system is provided i ncludi ng a support structu re ; a plurality of pairs of stereoscopic cameras comprisi ng a left camera and a right camera; a plurality of camera mounti ng platforms each supporting at least one of a left camera and a rig ht camera of a specific pair of stereoscopic cameras ; and a plurality of articulati ng support structures each configured to adjustably position a respective camera mou nt platform relative to the base to axially and angularly align two adjacent left cameras and two adjacent rig ht cameras withi n adjacent pairs of stereoscopic left and right cameras havi ng adjacent fields of view.
Additionally, i n one case each left camera and right camera withi n a pair of stereoscopic cameras is supported by a u nique camera mounting platform and a u nique articulating support structure for alig nment of the respective camera relative to the support structu re.
Furthermore, in one case each articulati ng support structure enables adjustable positioni ng of a respective camera mou nt platform along at least two degrees of freedom.
Additionally, in one case each articulati ng support structure enables pitch adjustment of a respective camera field of view.
Furthermore, in one case each articulati ng support structure enables yaw adjustment of a respective camera field of view. Additionally, in one case each articulati ng support structure enables roll adjustment of a respective camera field of view.
Furthermore, in one case each articulati ng support structure enables vertical displacement adjustment of a respective camera field of view.
Additionally, in one case each articulati ng support structure comprises a pair of plates each with a central socket, a spherical bearing disposed within each socket, with the pair of plates in opposed relation , and a plurality of th readed fasteners extendi ng between the pai r of plates to pivotally adjust one plate relative to the other plate about the spherical bearing and sockets.
Furthermore, in one case one plate i ncludes a cylindrical turntable havi ng camera fastener mounts for receivi ng a camera, the turntable mounted to the one plate and the another plate i ncludes fastener mou nts for affixi ng the another plate to the support structu re.
Even furthermore, an apparatus is provided for capturi ng stereoscopic i mages including a support structure, a fi rst pai r of stereoscopic cameras, and a second pair of stereoscopic cameras. The fi rst pair of stereoscopic cameras includes a fi rst left camera and a fi rst rig ht camera mou nted to the base and configured to simulate human depth perception and having a first field of view for the fi rst left camera and the first right camera. The second pair of stereoscopic cameras includes a second left camera and a second right camera mou nted to the base and configu red to si mulate hu man depth perception and having a second field of view for the second left camera and the second rig ht camera. The first field of view for each of the first left camera and the first right camera extends beyond a range of the second field of view for each of the second left camera and the second right camera. Fu rthermore, a stereoscopic image presentation system is provided i ncludi ng a visual output device, processing ci rcuitry, memory, and a user interface. The visual output device is configured to output a left stereoscopic image and a right stereoscopic image to a respective left eye and a respective right eye of a viewer. The processing circuitry is operative for retrieving the images and presenting the images. The memory is operative for storing the images. The user interface communicates with the processing circuitry and is configured to adjust a lateral or rotational offset between the left stereoscopic image and the right stereoscopic image to produce an adjustable stereoscopic convergence angle for the viewer.

Claims

CLAIMS The invention claimed is:
1. An apparatus for capturing images, comprising:
a base constructed and arranged to support an alignable array of cameras;
an image capture adjustment mechanism disposed relative to the base for adjusting an image capture line of sight for a camera relative to the base;
a first camera carried by the base, operably coupled with the image capture adjustment mechanism, and having an image capture device, the first camera having a line of sight defining a first field of view adjustable with the image capture adjustment mechanism relative to the base; and
a second camera, carried by the base and having an image capture device, the second camera having a line of sight defining a second field of view extending beyond a range of the field of view for the first camera in order to produce a field of view that is greater than the field of view provided by the first camera.
2. The apparatus of claim 1, wherein the image capture adjustment mechanism comprises a first image capture adjustment mechanism, and further comprising a second image capture
adjustment mechanism disposed relative to the base for adjusting an image capture line of sight for the second camera relative to the base.
3. The apparatus of claim 1 , wherein the i mage capture adjustment mechanism comprises a camera mou nting platform affixed to the first camera and an articulating support structure operatively coupled between the camera mounti ng platform and the base and configured to adjustably position the camera mounti ng platform and the fi rst camera relative to the base to axially and angularly alig n the field of view of the first camera relative to the field of view of the second camera.
4. The apparatus of claim 1 , wherein the i mage capture adjustment mechanism comprises an adjustable support base for the i mage captu re device of the first camera.
5. The apparatus of claim 1 , wherein the i mage capture adjustment mechanism comprises an adjustable camera lens support base.
6. The apparatus of claim 1 , wherein the i mage capture adjustment mechanism comprises an adjustable mi rror i nterposed between the image captu re device and a lens of the camera.
7. The apparatus of claim 1 , further comprisi ng a third camera stereoscopically paired adjacent the fi rst camera and a fourth camera stereoscopically paired adjacent the second camera.
8. The apparatus of claim 7, further comprisi ng a second i mage captu re adjustment mechanism communicating with the third camera, and a third i mage capture adjustment mechanism
com mu nicating with the fourth camera.
9. The apparatus of claim 8, wherein each of the first, second and third i mage capture adjustment mechanisms comprises a camera mounti ng platform and an articulati ng support structure configured to adjustably position a respective one of the camera mou nting platforms relative to the base to axially and angularly align the field of view of a respective one of the cameras relative to a field of view of another of the cameras.
1 0. The apparatus of claim 8, wherein the camera mou nting platform comprises a first support plate affixed to the camera, the base comprises a second support plate and the articulati ng support structu re comprises the fi rst support plate having a first seat, the second support plate having an opposed, second seat, a spherical ball bearing interposed between the first seat and the second seat, and a plurality of threaded fasteners extending between the fi rst support plate and the second support plate, distributed about the ball bearing and adjustable to tilt the first support plate relative to the second support plate.
1 1 . The apparatus of claim 1 , wherein the base comprises a u nitary base plate and the articulating support structure comprises a plurality of individual, articulating support structures each i nterposed between the base plate and a respective one of the cameras.
1 2. The apparatus of claim 1 , wherein the first camera and the second camera each comprise a video camera configured to captu re a sequential array of images over ti me.
1 3. The apparatus of claim 1 , wherein the i mage capture adjustment mechanism is mounted between the base and the first digital camera.
1 4. An apparatus for capturing digital images, comprisi ng : a base constructed and arranged to support a plurality of cameras for physically alig ning i mages captu red by adjacent cameras relative to one another;
a first digital camera supported by the base and havi ng an i mage captu re device, the first camera having a line of sight defini ng a first field of view;
a second digital camera, carried by the base and havi ng an image captu re device, the second camera havi ng a line of sig ht defining a second field of view extending beyond a range of the field of view for the first camera in order to produce an adjacent field of view that extends beyond the field of view provided by the fi rst camera; and
an image captu re adjustment mechanism disposed relative to the base and operatively coupled with the first digital camera for physically adjusting an i mage capture line of sig ht for the first digital camera relative to the base and the second digital camera to align an adjacent field of view for the first digital camera relative to the second digital camera.
1 5. The apparatus of claim 1 4, wherei n the fi rst camera comprises a left camera and the second camera comprises a right camera of a stereoscopic pai r of left and rig ht cameras.
1 6. The apparatus of claim 1 5, wherei n the stereoscopic pair of left and right cameras is a first stereoscopic pai r, and further comprisi ng a third camera and a fourth camera comprisi ng a left camera and a right camera of a second stereoscopic pai r of left and right cameras provided adjacent the first stereoscopic pai r of left and right cameras.
17. The apparatus of claim 16, wherein an image capture adjustment mechanism is provided between the base and each of a respective one of the left and right cameras in the first stereoscopic pair and the second stereoscopic pair.
18. The apparatus of claim 17, wherein each of the image capture adjustment mechanisms comprises an articulating frame operatively coupled between one of the cameras and the base.
19. A method for capturing images, comprising:
providing a first camera and a second camera carried by a base to have a line of sight defining a respective field of view, the second camera having a field of view at least in part adjacent to the field of view for the first camera, the first camera carried by the base for adjustable positioning of the field of view relative to the field of view for the second camera;
axially aligning the first camera relative to the second camera to render collinear an image segment within a field of view for the first camera relative to a corresponding image segment within the field of view for the second camera; and
angularly aligning the first camera relative to the second camera to render the image segment within the field of view for the first camera angularly aligned relative to the image segment in the field of view for the second camera.
20. The method of claim 19, further comprising providing adjacent pairs of stereoscopic left and right cameras, wherein the first camera comprises one of the left and right cameras of the first pair and the second camera comprises one of the left and right cameras of the second pair, wherein axially aligning comprises axially aligning the one of the left and right cameras of the first pair and the second pair, and angularly aligning comprises angularly aligning the one of the left and right cameras of the first pair and the second pair.
21. The method of claim 20, further comprising axially aligning another of the left and right cameras of the first pair and the second camera.
22. The method of claim 20, further comprising angularly aligning another of the left and right cameras of the first pair and the second pair.
23. The method of claim 20, further comprising axially aligning the left camera and the right camera within the first pair and angularly aligning the left camera and the right camera within the first pair.
24. The method of claim 20, further comprising providing at least three pairs of stereoscopic left and right cameras, axially and angularly aligning left cameras within adjacent pairs of stereoscopic cameras, and axially and angularly aligning right cameras within adjacent pairs of stereoscopic cameras.
25. The method of claim 19, wherein axial aligning comprises aligning a scan line from a captured image from each adjacent camera in horizontal alignment, and angular alignment comprises angularly aligning the scan line from a captured image from each adjacent camera in angular alignment, such as by adjusting pitch and roll of one camera relative to the other camera.
26. The method of claim 19, the first camera and the second camera each comprise a digital video camera, and further comprising, after axially aligning and angularly aligning the first camera and the second camera, simultaneously capturing images over time with the first camera and the second camera.
27. The method of claim 26, wherein the first camera and the second camera each comprise an image capture device, and further comprising, after axially aligning and angularly aligning the first camera and the second camera, simultaneously capturing optical images with the first camera and the second camera.
PCT/US2011/031830 2010-04-09 2011-04-08 Apparatus and method for capturing images WO2011127434A2 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201180026684.7A CN103026700B (en) 2010-04-09 2011-04-08 The apparatus and method of capture image
EP11766841.8A EP2556654B1 (en) 2010-04-09 2011-04-08 Apparatus and method for capturing images

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US32271410P 2010-04-09 2010-04-09
US61/322,714 2010-04-09
US13/071,353 US8896671B2 (en) 2010-04-09 2011-03-24 Apparatus and method for capturing images
US13/071,353 2011-03-24

Publications (2)

Publication Number Publication Date
WO2011127434A2 true WO2011127434A2 (en) 2011-10-13
WO2011127434A3 WO2011127434A3 (en) 2012-03-08

Family

ID=44760651

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2011/031830 WO2011127434A2 (en) 2010-04-09 2011-04-08 Apparatus and method for capturing images

Country Status (4)

Country Link
US (2) US8896671B2 (en)
EP (1) EP2556654B1 (en)
CN (1) CN103026700B (en)
WO (1) WO2011127434A2 (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104023255A (en) * 2014-06-23 2014-09-03 北京视博云科技有限公司 Fitness system based on cloud platform and video acquiring device thereof
US8896671B2 (en) 2010-04-09 2014-11-25 3D-4U, Inc. Apparatus and method for capturing images
WO2016200646A1 (en) * 2015-06-10 2016-12-15 Otoy, Inc. Adaptable camera array structures
DE102017123261A1 (en) 2017-10-06 2019-04-11 Soccerwatch.Tv Gmbh Panoramic camera for video recording of playing fields and a method for operating this panoramic camera

Families Citing this family (68)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9479768B2 (en) * 2009-06-09 2016-10-25 Bartholomew Garibaldi Yukich Systems and methods for creating three-dimensional image media
KR101182549B1 (en) * 2010-12-16 2012-09-12 엘지이노텍 주식회사 3d stereoscopic camera module
US20140327733A1 (en) * 2012-03-20 2014-11-06 David Wagreich Image monitoring and display from unmanned vehicle
US9350954B2 (en) * 2012-03-20 2016-05-24 Crane-Cohasset Holdings, Llc Image monitoring and display from unmanned vehicle
US11228733B2 (en) * 2012-07-11 2022-01-18 Cyclops Technology Group, Llc Surveillance system and associated methods of use
DE102012106834A1 (en) * 2012-07-27 2014-01-30 Conti Temic Microelectronic Gmbh Method for aligning two image recording elements of a stereo camera system
US10298815B1 (en) * 2012-10-18 2019-05-21 Altia Systems, Inc. Chassis and mounting arrangement for a panoramic camera
US9338370B2 (en) * 2012-11-05 2016-05-10 Honeywell International Inc. Visual system having multiple cameras
CA2890174A1 (en) * 2012-11-05 2014-05-08 360 Heros, Inc. 360 degree camera mount and related photographic and video system
US9497380B1 (en) * 2013-02-15 2016-11-15 Red.Com, Inc. Dense field imaging
US20140267598A1 (en) * 2013-03-14 2014-09-18 360Brandvision, Inc. Apparatus and method for holographic poster display
US10296281B2 (en) 2013-11-05 2019-05-21 LiveStage, Inc. Handheld multi vantage point player
US20180227572A1 (en) * 2013-11-05 2018-08-09 Livestage Inc. Venue specific multi point image capture
US20180227501A1 (en) * 2013-11-05 2018-08-09 LiveStage, Inc. Multiple vantage point viewing platform and user interface
CA2933704A1 (en) * 2013-12-13 2015-06-18 8702209 Canada Inc. Systems and methods for producing panoramic and stereoscopic videos
US20160118036A1 (en) 2014-10-23 2016-04-28 Elwha Llc Systems and methods for positioning a user of a hands-free intercommunication system
US20150334346A1 (en) * 2014-05-16 2015-11-19 Elwha Llc Systems and methods for automatically connecting a user of a hands-free intercommunication system
US9779593B2 (en) 2014-08-15 2017-10-03 Elwha Llc Systems and methods for positioning a user of a hands-free intercommunication system
CN104702907B (en) * 2014-03-20 2019-05-10 杭州海康威视数字技术股份有限公司 A kind of method and system of video-splicing
US9196039B2 (en) 2014-04-01 2015-11-24 Gopro, Inc. Image sensor read window adjustment for multi-camera array tolerance
AU2015260904B2 (en) * 2014-05-13 2018-01-18 Eco Net Ltd. Digital sports fishing
WO2015179574A1 (en) * 2014-05-20 2015-11-26 Nextvr Inc. Methods and apparatus including or for use with one or more cameras
CN104079917A (en) * 2014-07-14 2014-10-01 中国地质大学(武汉) 360-degree panorama stereoscopic camera
US9602702B1 (en) * 2014-09-08 2017-03-21 Sulaiman S. Albadran Video camera with multiple data input
US9208668B1 (en) * 2014-09-16 2015-12-08 Geovision Inc. Camera system with a full view monitoring function
US10257494B2 (en) * 2014-09-22 2019-04-09 Samsung Electronics Co., Ltd. Reconstruction of three-dimensional video
US11205305B2 (en) 2014-09-22 2021-12-21 Samsung Electronics Company, Ltd. Presentation of three-dimensional video
US10531071B2 (en) * 2015-01-21 2020-01-07 Nextvr Inc. Methods and apparatus for environmental measurements and/or stereoscopic image capture
US20160253839A1 (en) * 2015-03-01 2016-09-01 Nextvr Inc. Methods and apparatus for making environmental measurements and/or using such measurements in 3d image rendering
US10126634B1 (en) * 2015-03-18 2018-11-13 Davo Scheich Variable radius camera mount
US10725631B2 (en) 2015-04-30 2020-07-28 Pixia Corp. Systems and methods of selecting a view from a plurality of cameras
US20160349600A1 (en) * 2015-05-26 2016-12-01 Gopro, Inc. Multi Camera Mount
EP3304195A1 (en) * 2015-05-27 2018-04-11 Google LLC Camera rig and stereoscopic image capture
USD856394S1 (en) 2015-05-27 2019-08-13 Google Llc Video camera rig
US9817299B2 (en) * 2015-07-23 2017-11-14 Admiral Video, LLC High-definition pylon-mounted camera assembly with mounting structure
US10504499B2 (en) * 2015-08-12 2019-12-10 George A. Economou Extracting features from auditory observations with active or passive assistance of shape-based auditory modification apparatus
US20170078653A1 (en) * 2015-09-15 2017-03-16 Qualcomm Incorporated Systems and methods for producing a surround view
UA114549C2 (en) * 2015-09-16 2017-06-26 Сергій Іванович Мірошниченко MULTI-SENSOR DEVICE MAKERS ON THE BASE OF OPTOELECTRONIC CONVERTERS
US10057487B1 (en) * 2016-03-25 2018-08-21 Scott Zhihao Chen Panoramic imaging systems based on normal-lens cameras
US10230904B2 (en) 2016-04-06 2019-03-12 Facebook, Inc. Three-dimensional, 360-degree virtual reality camera system
CN106060523B (en) * 2016-06-29 2019-06-04 北京奇虎科技有限公司 Panoramic stereo image acquisition, display methods and corresponding device
USD830445S1 (en) * 2016-06-30 2018-10-09 Facebook, Inc. Panoramic virtual reality camera
USD830444S1 (en) 2016-06-30 2018-10-09 Facebook, Inc. Panoramic virtual reality camera
USD837275S1 (en) * 2016-06-30 2019-01-01 Facebook, Inc. Panoramic virtual reality camera assembly
KR102629569B1 (en) * 2016-08-17 2024-01-29 삼성전자주식회사 electronic device including camera
USD824990S1 (en) 2016-08-30 2018-08-07 Freedom360 Llc Camera mount
US10025165B2 (en) 2016-08-30 2018-07-17 Freedom360 Llc Camera mount
US10893212B2 (en) * 2017-01-26 2021-01-12 Sony Corporation Spherically-arranged imaging array having pairs of cameras with overlapping fields of view
CN106878599B (en) * 2017-03-16 2022-09-06 芜湖星途机器人科技有限公司 Camera protection cover for robot
US20180324389A1 (en) * 2017-05-02 2018-11-08 Frederick Rommel Cooke Surveillance Camera Platform
US9921464B1 (en) * 2017-05-03 2018-03-20 Seung Kwon Choi Gimbal for 360-degree video and picture shooting
US11049218B2 (en) 2017-08-11 2021-06-29 Samsung Electronics Company, Ltd. Seamless image stitching
CN107395985B (en) * 2017-08-28 2019-11-05 京东方科技集团股份有限公司 A kind of mobile terminal image composition method, device and mobile terminal
CA2993561C (en) * 2018-01-31 2020-06-30 Synaptive Medical (Barbados) Inc. System for three-dimensional visualization
KR102177401B1 (en) * 2018-02-02 2020-11-11 재단법인 다차원 스마트 아이티 융합시스템 연구단 A noiseless omnidirectional camera device
CN108234955A (en) * 2018-02-02 2018-06-29 江西辰锐信息科技有限公司 Rod piece-based panoramic video monitoring device and method
FR3082690B1 (en) * 2018-06-15 2021-05-21 Safran Electronics & Defense PROXIMAL WATCH DEVICE
US20200019045A1 (en) * 2018-07-12 2020-01-16 Nextvr Inc. Stereoscopic camera system
CN110853073A (en) * 2018-07-25 2020-02-28 北京三星通信技术研究有限公司 Method, device, equipment and system for determining attention point and information processing method
WO2020069422A1 (en) * 2018-09-28 2020-04-02 Brainbaby Inc. Self-contained low profile cross shooting video apparatus
CN111327838B (en) * 2018-12-13 2021-09-24 杭州海康威视数字技术股份有限公司 Video camera
US10460473B1 (en) * 2018-12-14 2019-10-29 Zoox, Inc. Camera calibration system
US11012601B1 (en) 2019-09-23 2021-05-18 Amazon Technologies, Inc. Dual camera module systems
DE202020105395U1 (en) * 2020-08-04 2020-12-10 Polymath.Vision UG (haftungsbeschränkt) Communication robot
CN112128565B (en) * 2020-09-27 2021-12-24 深圳市走去科技集团有限公司 Real-time two-way adjusting device based on VR technique
DE102020131571A1 (en) * 2020-11-27 2022-06-02 SPORTTOTAL TECHNOLOGY GmbH panoramic camera
CN112594490B (en) * 2020-12-09 2022-03-25 长沙超创电子科技有限公司 Camera array structure and adjusting device for panoramic intelligent monitoring and early warning
US11635167B1 (en) 2021-09-09 2023-04-25 Amazon Technologies, Inc. Quick-connect camera mounts with multiple degrees of freedom

Family Cites Families (44)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US3269291A (en) 1961-06-29 1966-08-30 Globe Ind Inc Stereoscopic camera
US4418993A (en) 1981-05-07 1983-12-06 Stereographics Corp. Stereoscopic zoom lens system for three-dimensional motion pictures and television
GB2256567B (en) 1991-06-05 1995-01-11 Sony Broadcast & Communication Modelling system for imaging three-dimensional models
JP2698251B2 (en) 1991-10-22 1998-01-19 シャープ株式会社 Assembly structure
US5422653A (en) 1993-01-07 1995-06-06 Maguire, Jr.; Francis J. Passive virtual reality
US6690338B1 (en) 1993-08-23 2004-02-10 Francis J. Maguire, Jr. Apparatus and method for providing images of real and virtual objects in a head mounted display
US5614941A (en) 1993-11-24 1997-03-25 Hines; Stephen P. Multi-image autostereoscopic imaging system
DE59506365D1 (en) 1994-01-28 1999-08-12 Cfb Gmbh DEVICE AND METHOD FOR PRODUCING AN IMAGE SEQUENCE
US5850352A (en) 1995-03-31 1998-12-15 The Regents Of The University Of California Immersive video, including video hypermosaicing to generate from multiple video views of a scene a three-dimensional video mosaic from which diverse virtual video scene images are synthesized, including panoramic, scene interactive and stereoscopic images
US5729471A (en) 1995-03-31 1998-03-17 The Regents Of The University Of California Machine dynamic selection of one video camera/image of a scene from multiple video cameras/images of the scene in accordance with a particular perspective on the scene, an object in the scene, or an event in the scene
US5694257A (en) 1996-05-06 1997-12-02 New Focus, Inc. Rotary beamsplitter prism mount
US6084979A (en) 1996-06-20 2000-07-04 Carnegie Mellon University Method for creating virtual reality
NL1006407C2 (en) * 1997-06-26 1998-12-29 Hollandse Signaalapparaten Bv System for visualizing an environment.
US6348918B1 (en) 1998-03-20 2002-02-19 Microsoft Corporation Stereo reconstruction employing a layered approach
US6864910B1 (en) 1999-06-30 2005-03-08 Canon Kabushiki Kaisha Optical apparatus
US7015954B1 (en) * 1999-08-09 2006-03-21 Fuji Xerox Co., Ltd. Automatic video system using multiple cameras
AU2599701A (en) 1999-12-28 2001-07-09 Vrex, Inc. A 3d camera
US6375370B1 (en) * 2000-02-10 2002-04-23 Cam Guard Systems, Inc. Temporary surveillance system
US6788333B1 (en) 2000-07-07 2004-09-07 Microsoft Corporation Panoramic video
US6535242B1 (en) 2000-10-24 2003-03-18 Gary Steven Strumolo System and method for acquiring and displaying vehicular information
US20020082466A1 (en) * 2000-12-22 2002-06-27 Jeongho Han Laser surgical system with light source and video scope
US7126630B1 (en) 2001-02-09 2006-10-24 Kujin Lee Method and apparatus for omni-directional image and 3-dimensional data acquisition with data annotation and dynamic range extension method
US6947059B2 (en) * 2001-08-10 2005-09-20 Micoy Corporation Stereoscopic panoramic image capture device
US7096428B2 (en) 2001-09-28 2006-08-22 Fuji Xerox Co., Ltd. Systems and methods for providing a spatially indexed panoramic video
US6590723B1 (en) 2001-11-08 2003-07-08 Sd Instruments Optical instrument mount
FR2836215B1 (en) 2002-02-21 2004-11-05 Yodea SYSTEM AND METHOD FOR THREE-DIMENSIONAL MODELING AND RENDERING OF AN OBJECT
US7224382B2 (en) 2002-04-12 2007-05-29 Image Masters, Inc. Immersive imaging system
US7463280B2 (en) 2003-06-03 2008-12-09 Steuart Iii Leonard P Digital 3D/360 degree camera system
KR100545903B1 (en) 2003-11-12 2006-01-31 김배훈 Camera Device For 360-degree Panorama Shot and Method
US7570301B2 (en) * 2004-06-03 2009-08-04 Electronic Security Products, Inc. Device, system and method of mounting audio/video capturing equipment
US20060072005A1 (en) 2004-10-06 2006-04-06 Thomas-Wayne Patty J Method and apparatus for 3-D electron holographic visual and audio scene propagation in a video or cinematic arena, digitally processed, auto language tracking
US20070024701A1 (en) 2005-04-07 2007-02-01 Prechtl Eric F Stereoscopic wide field of view imaging system
US20060228018A1 (en) * 2005-04-12 2006-10-12 Gil Abramovich Reconfigurable machine vision system
CN101366269B (en) * 2006-01-20 2010-11-03 索尼爱立信移动通讯股份有限公司 Camera for electronic device
US7822338B2 (en) 2006-01-20 2010-10-26 Sony Ericsson Mobile Communications Ab Camera for electronic device
JP4396674B2 (en) 2006-08-11 2010-01-13 船井電機株式会社 Panorama imaging device
US8619127B2 (en) 2007-05-10 2013-12-31 Assad F. Mora Stereoscopic three dimensional visualization system and method of use
JP5067885B2 (en) * 2007-09-14 2012-11-07 株式会社リコー Image input device and personal authentication device
US20090128621A1 (en) * 2007-11-21 2009-05-21 Charles Gregory Passmore System and/or method for automated stereoscopic alignment of images
US8032296B2 (en) * 2008-04-30 2011-10-04 Verizon Patent And Licensing Inc. Method and system for providing video mapping and travel planning services
US7848635B2 (en) 2008-05-28 2010-12-07 Creat3 Inc. Modular stereoscopic rig
JP5428210B2 (en) * 2008-06-11 2014-02-26 ソニー株式会社 Information processing apparatus, imaging system, recording control method, and program
TWI405457B (en) * 2008-12-18 2013-08-11 Ind Tech Res Inst Multi-target tracking system, method and smart node using active camera handoff
US8896671B2 (en) 2010-04-09 2014-11-25 3D-4U, Inc. Apparatus and method for capturing images

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See references of EP2556654A4 *

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8896671B2 (en) 2010-04-09 2014-11-25 3D-4U, Inc. Apparatus and method for capturing images
CN104023255A (en) * 2014-06-23 2014-09-03 北京视博云科技有限公司 Fitness system based on cloud platform and video acquiring device thereof
WO2016200646A1 (en) * 2015-06-10 2016-12-15 Otoy, Inc. Adaptable camera array structures
DE102017123261A1 (en) 2017-10-06 2019-04-11 Soccerwatch.Tv Gmbh Panoramic camera for video recording of playing fields and a method for operating this panoramic camera
WO2019068836A1 (en) 2017-10-06 2019-04-11 Soccerwatch.Tv Gmbh Panoramic camera for video-recording playing fields and a method for operating said panoramic camera

Also Published As

Publication number Publication date
CN103026700A (en) 2013-04-03
CN103026700B (en) 2016-10-26
WO2011127434A3 (en) 2012-03-08
US20150035951A1 (en) 2015-02-05
US10009541B2 (en) 2018-06-26
US20110249100A1 (en) 2011-10-13
EP2556654A2 (en) 2013-02-13
EP2556654A4 (en) 2014-01-01
US8896671B2 (en) 2014-11-25
EP2556654B1 (en) 2018-08-29

Similar Documents

Publication Publication Date Title
US10009541B2 (en) Apparatus and method for capturing images
CN102959964B (en) System and method for multi-viewpoint video capture
US7429997B2 (en) System and method for spherical stereoscopic photographing
CN100524017C (en) Foveated panoramic camera system
US7224382B2 (en) Immersive imaging system
US20080298674A1 (en) Stereoscopic Panoramic imaging system
US10154194B2 (en) Video capturing and formatting system
US9479768B2 (en) Systems and methods for creating three-dimensional image media
US7486324B2 (en) Presenting panoramic images with geometric transformation
WO2007055943A3 (en) Multi-user stereoscopic 3-d panoramic vision system and method
WO2010118674A1 (en) Image acquisition equipment
US20100045773A1 (en) Panoramic adapter system and method with spherical field-of-view coverage
US20040263611A1 (en) Omni-directional camera design for video conferencing
US5619255A (en) Wide-screen video system
US20080030573A1 (en) Volumetric panoramic sensor systems
CN1531826A (en) Method and apparatus for omni-directional image and 3-dimensional data acquisition with data annotation and dynamic range extension method
CN102209254A (en) One-dimensional integrated imaging method and device
WO2013052089A1 (en) Stereoscopic three-dimensional camera rigs
US20170322396A1 (en) Compact camera design to adjust parallax effects
AU1074199A (en) Method and system for the creation and interactive viewing of totally immersive stereoscopic images
Baker et al. Capture considerations for multiview panoramic cameras
Baker et al. Capture and display for live immersive 3D entertainment
CN108848368A (en) A kind of 3D image pickup method of VR video
CN116112802A (en) Concentric splicing method and device
GB2528462A (en) A camera mounting apparatus, a camera assembly, and a method of mounting cameras

Legal Events

Date Code Title Description
WWE Wipo information: entry into national phase

Ref document number: 201180026684.7

Country of ref document: CN

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 11766841

Country of ref document: EP

Kind code of ref document: A2

NENP Non-entry into the national phase

Ref country code: DE

WWE Wipo information: entry into national phase

Ref document number: 2011766841

Country of ref document: EP