WO2011143622A2

WO2011143622A2 - Underwater acquisition of imagery for 3d mapping environments

Info

Publication number: WO2011143622A2
Application number: PCT/US2011/036531
Authority: WO
Inventors: Justin Baird; Lucas Tomkins
Original assignee: Google Inc.
Priority date: 2010-05-13
Filing date: 2011-05-13
Publication date: 2011-11-17
Also published as: WO2011143622A8; WO2011143622A3

Abstract

A panoramic imagery acquisition system and method is disclosed, which facilitates capture of panoramic images underwater. A panoramic camera system can be deployed underwater, and stabilized to reduce effects of movement of a seagoing vehicle. In addition to panoramic images, location data can be collected using, for example, a Global Positioning System (GPS) system. A GPS antenna can be located above water to improve location accuracy. An offset between the panoramic camera system and GPS system can be determined, and incorporated into the location data associated with captured images. Additional data can be acquired and associated with the panoramic images, including topographical data acquired using LIDAR.

Description

UNDERWATER ACQUISITION OF PANORAMIC IMAGERY FOR 3D

MAPPING ENVIRONMENTS

FIELD

[0001] This disclosure generally relates to capturing imagery and applications thereof.

BACKGROUND

[0002] Panoramic image capture can be used in a geographic information system (GIS) associating images with location data. For example, a land-based vehicle can use a camera system to capture panoramic images. The land-based vehicle can use a Global Positioning System (GPS) device to determine location data. However, land-based vehicles and cameras do not work in underwater environments. Land-based vehicles and above ground cameras do not even typically encounter various difficulties associated with water-based vehicles and underwater photography. Furthermore, GPS signals typically do not penetrate water.

BRIEF SUMMARY

[0003] Embodiments relate to capturing underwater panoramic imagery, and applications thereof. In an embodiment, a panoramic imagery acquisition method includes capturing a panoramic image underwater using a panoramic camera system, and associating the image with location data.

[0004] A panoramic imagery acquisition system includes a panoramic camera system to capture a panoramic image underwater, and a module to associate the image with location data.

[0005] Further embodiments, features, and advantages of the present invention, as well as the structure and operation of the various embodiments of the present invention, are described in detail below with reference to the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS/FIGURES

[0006] Embodiments are described with reference to the accompanying drawings. In the drawings, like reference numbers may indicate identical or functionally similar elements. [0007] FIG. 1 is a diagram of an exemplary imagery acquisition system, in accordance with an embodiment.

[0008] FIG. 2 is a diagram illustrating an example panoramic camera system, in accordance with an embodiment.

[0009] FIG. 3 is a diagram of an exemplary imagery acquisition system, in accordance with an embodiment.

[0010] FIG. 4 is a diagram of an exemplary imagery acquisition system, in accordance with an embodiment.

[0011] FIG. 5 A is a diagram of an exemplary imagery acquisition system, in accordance with an embodiment.

[0012] FIG. 5B is a top view of an exemplary imagery acquisition system, in accordance with an embodiment.

[0013] FIG. 5C is a diagram of an exemplary imagery acquisition system, in accordance with an embodiment.

[0014] FIG. 6A is a diagram of an exemplary imagery acquisition system, in accordance with an embodiment.

[0015] FIG. 6B is a diagram of a panoramic camera system, housing, and sealed cable, in accordance with an embodiment.

[0016] FIG. 7 is a diagram of an exemplary imagery acquisition system, in accordance with an embodiment.

[0017] FIG. 8 is a diagram of an exemplary imagery acquisition system, in accordance with an embodiment.

[0018] Embodiments of the present invention are described with reference to the accompanying drawings. The drawing in which an element first appears is typically indicated by the leftmost digit or digits in the corresponding reference number. The drawings are not necessarily drawn to scale.

DETAILED DESCRIPTION

[0019] Embodiments of the present invention relate to underwater panoramic imagery acquisition systems and methods, and applications thereof. In the detailed description of the invention herein, references to "one embodiment", "an embodiment", "an example embodiment", etc., indicate that the embodiment described may include a particular feature, structure, or characteristic, but every embodiment may not necessarily include the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same embodiment. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is submitted that it is within the knowledge of one skilled in the art to effect such feature, structure, or characteristic in connection with other embodiments whether or not explicitly described.

[0020] "Panoramic camera system" as used herein refers to a camera system capable of capturing panoramic images. Examples include, but are not limited to, a panoramic camera system having multiple optical elements arranged to view or scan a panoramic field of view and one or more detector elements to detect images obtained by the optical elements. For example, optical elements may include one or more lenses, fiber optics, or other imagery for obtaining an image. Detector elements may include imaging arrays, detectors, arrays of detectors, and non-imaging devices.

[0021] FIG. 1 is a diagram of an exemplary underwater panoramic imagery acquisition system, in accordance with an embodiment. The imagery acquisition system includes a panoramic camera system 100 and a control module 150 mounted to a boat 1 12 in water 1 10.

[0022] Underwater panoramic camera system 100 enables acquisition of one or more images. The images can represent scenes in a field of view of the panoramic camera system 100. For example, panoramic camera system 100 includes one or more lenses 102 arranged to capture images in a panoramic field of view. In one example, not intended to be limiting, four lenses 102 are visible in the view illustrated in FIG. 1.

[0023] Images can be communicated from panoramic camera system 100 to control module 150 using various technologies, including wireless transmission, optical fibers, and wires. The embodiment of FIG. 1 illustrates the use of wires 104. Images can be stored onboard for later download once ashore. Embodiments can also download/upload or transmit images from panoramic camera system 100 to a remote storage device (not illustrated). For example, images can be transmitted to a remote storage device using cellular telephone networks, wireless local area networks (WLAN), satellite networks, or other wireless communication links using corresponding onboard equipment interfacing with the panoramic camera system 100. Images can be transmitted to the control module 150 while also being transmitted to the remote storage device. [0024] Panoramic camera system 100 can include a waterproof enclosure 106. Enclosure

106 can be designed to be waterproof to various depths in water 1 10. Embodiments can provide an enclosure 106 spaced from the one or more lenses 102. Various features of the panoramic camera system 100 can compensate for use of the enclosure. For example, the panoramic camera system 100 can compensate for an optical interface between water 1 10 and the enclosure 106, as well as an interface between the enclosure 106 and air within the enclosure 106. Additionally, embodiments can provide a waterproof enclosure 106 that incorporates the one or more lenses 102 in walls of the enclosure itself. This can make the device even more compact, reduce weight, and reduce distortion. In an example, panoramic camera system 100 can incorporate enclosure 106 as a housing for the panoramic camera system 100.

[0025] Lighting system 108 can be used to provide illumination. The lighting system 108 is optional. For example, lighting system 108 can be used to explore underwater environments at night, in poor lighting conditions, or other situations (e.g., lower depths) where scenery otherwise lacks sufficient illumination for image capture by panoramic camera system 100. In an embodiment, lighting system 108 includes two sets of four floodlights arranged radially at 90 degree angles to each other. Additional types of lights and arrangements are possible, including varying angles, elevations, numbers, overlap, and mounting arrangements of light sources. Lighting system 108 can be incorporated into other embodiments of stabilizers, camera systems, and vehicles. Embodiments can including lighting system 108 mounted above and/or below panoramic camera system 100. In embodiments, lighting system 108 can be mounted to boat 1 12, support mount 128, stabilizer 120, extendable arm 124, panoramic camera system 100, enclosure 106, and/or other mounting locations.

[0026] Stabilizer 120 and extendable arm 124 can position panoramic camera system 100 and lighting system 108 underwater, below the surface of the water 1 10. Stabilizer 120 can compensate for movements of support mount 128 with respect to panoramic camera system 100. For example, stabilizer 120 can move to compensate for roll, pitch, and yaw movements of support mount 128 relative to panoramic camera system 100. Such movements of the support mount 128 can be caused by wave motion or currents of the water 1 10 acting on the boat 1 12. Movements of support mount 128 can also be caused by actions initiated by the boat 1 12, e.g., course adjustments. Such stabilization is especially important when capturing high-resolution panoramic images of underwater scenes as boat 1 12 moves through water 1 10.

[0027] Stabilizer 120 can include joints, flexible support mounts, support arms, springs, actuators, motors, servos, dampers, counterweights, and other mechanisms for providing movement compensation, positional control, and other stabilizing effects. Stabilization can be passive and/or active.

[0028] In the embodiment illustrated in FIG. 1, stabilizer 120 is coupled to extendable arm 124. In one example, extendable arm 124 can be implemented using a telescoping mechanism. Accordingly, a depth of panoramic camera system 100 in the water 1 10 can be adjusted by telescoping the extendable arm 124. Furthermore, telescoping of the extendable arm 124 can provide motion compensation benefits, e.g., can compensate for wave motion of the boat along the axis of the telescoping mechanism. Embodiments can include a non-telescoping arm 124, such that stabilizer 120 can reposition arm 124 vertically.

[0029] In one example, wires 104 can extend through a waterproof path in arm 124 from panoramic camera system 100 into the air. The wires 104 emerge from the top of the arm 124. Cable management member 130, which can be mounted to support mount 128, secures the wires 104. In an example, cable management member 130 is a ring having a diameter large enough to accommodate wires 104. Cable management member 130 can also be incorporated along stabilizer 120 and arm 124. For example, cable management member 130 can position the wires 104 within stabilizer 120 for added protection of the wires 104. Accordingly, the wires 104 are kept out of the water and safely maintained during motion compensation and depth adjustments. Wires 104 can be bundled together into a wire loom using various techniques such as cable ties, sleeves, and other techniques.

[0030] Support mount 128 can provide a secure support for stabilizer 120 and wires 104

(e.g., via cable management member 130). Support mount 128 can be universally adjustable to mount to varying structures of the boat 1 12. Support mount 128 can include conforming structures, such as rubber or foam pads, to conform to varying structures upon which support mount 128 is mounted.

[0031] A Global Positioning System (GPS) device 140 can provide location data to be associated with images acquired by panoramic camera system 100. Embodiments can also use stored location data corresponding to a predetermined path along which the panoramic camera system 100 is guided. The location data and other data can be stored as metadata with the panoramic images, as the images are captured. Embodiments can also maintain time logs of the images, other information, and the location data. The time logs enable synchronizing and associating the images and other information with the location data. GPS device 140 is positioned above the surface of the water 1 10, enabling receipt of GPS signals uninterrupted by the water 1 10. Embodiments can position the GPS device 140 elsewhere, including positioning the GPS device 140 below the surface of the water 110. For example, embodiments can position GPS device 140 proximate to panoramic camera system 100, e.g., within waterproof enclosure 106. Embodiments can use a GPS antenna positioned above the surface of the water 1 10 and coupled to GPS device 140.

[0032] Embodiments can monitor and track the position of the panoramic camera system

100 relative to GPS device 140. In the embodiment illustrated in FIG. 1, the panoramic camera system 100 is located at a positional offset from the GPS device 140. The imagery acquisition system can include sensors to detect relative movements and positions of the stabilizer 120 and extendable arm 124. The relative movements and positions can then be used to calculate the relative offset between the panoramic camera system 100 and GPS device 140.

[0033] For example, the stabilizer 120 can position the extendable arm 124 at a distance of 2 meters from the GPS device 140. The extendable arm 124 can position the panoramic camera system 100 at a depth of 3 meters below the surface of the water 110. The sensors can account for these offsets, compensating for adjustments in positions and movement. The offset can be negligible with respect to the location data. The offset can be used to update GPS information (e.g., longitude, latitude, and altitude/depth) associated with images from the panoramic camera system 100. The offset can be used to update stored location data associated with predetermined routes. The offset can be dynamically adjusted to account for repositioning of the GPS device 140 and panoramic camera system 100.

[0034] Control module 150 is coupled to the GPS device 140 and panoramic camera system 100. Control module 150 can associate location data, including location data from GPS device 140, with images acquired by panoramic camera system 100. Control module 150 can be implemented on a computer with a processor, local memory, a display, and one or more input devices such as a keyboard, a touchscreen, or a mouse. Control module 150 can be a specialized computing device such as, for example, a mobile device. Control module 150 can be implemented using a computer capable of collecting images from panoramic camera system 100 and associating the images with location data.

[0035] Control module 150 can also include storage 152, power management module 154 and ancillary equipment 156. In embodiments, power management module 154 can provide power to the panoramic camera system 100 and/or the lighting system 108. The imagery acquisition system can include a motorized stabilizer 120 and a motorized extendable arm 124. Such motorized features can be powered, controlled, and positioned by control module 150. For example, accelerometer, depth, and compass sensors can be interfaced with and/or implemented in the stabilizer 120, panoramic camera system 100, and/or control module 150. For example, such sensors can be implemented in ancillary equipment 156 of control module 150. Control module 150 can interface with on-board systems of the boat 1 12, for example, navigational systems. An embodiment can use active feedback from such sensors to compensate for movements and mechanically decouple the panoramic camera system 100 from undesired movements of support mount 128.

[0036] Control module 150 can retrieve images from the panoramic camera system 100, for example, in the form of panoramic images or in the form of panoramic image tiles. In another embodiment, the images can be associated with additional information related to capturing panoramic scenery, e.g., Light Detection And Ranging (LIDAR), spectroscopy, bathymetry, topography, and other information. As the panoramic camera system 100 captures images, control module 150 can associate the images with updated and/or additional information.

[0037] The panoramic camera system 100 can be submerged in the water 1 10 to acquire images underwater. The stabilizer 120 and arm 124 mechanically decouple the panoramic camera system 100 from roll, tilt, and other undesirable movements of the boat 1 12. The control module 150 associates location and other information with the images.

[0038] FIG. 2 is a diagram illustrating an example panoramic camera system 200, in accordance with an embodiment. Panoramic camera system 200, as an example, comprises a rosetta arrangement of image capture apparatuses. For example, an arrangement of eight image capture apparatuses enables a full 360° of subject image matter to be captured. Each image capture apparatus of FIG. 2 can correspond to, for example a lens 102 as illustrated in FIG. 1.

[0039] As panoramic camera system 200 travels in a forward scan direction (indicated as upwards in FIG. 2), a panoramic image can be captured including the subject matter on the sides shown as 21 1 A and 21 IB, as well as the subject matter in front, 211C, and behind, 21 ID. In this manner, system 500 can capture high resolution, rectilinear, 360° subject matter images according to an embodiment of the present invention. For example, the panoramic camera system 200 can be guided along reefs, coral atolls, or other underwater ecosystems, to acquire images and associated information, while maintaining alignment of the acquired images and information.

[0040] Panoramic camera system 200 can comprise four fixed focal length image capture apparatuses 202A, 202B, 202C, and 202D, and four variable focal length image capture apparatuses 203A, 203B, 203C, and 203D. As illustrated in this embodiment, a fixed focal length image capture apparatus is adjoined on each side by a variable focal length image capture apparatus. Thus, the focal lengths of the adjoining image capture apparatuses are approximately equal at the adjoining sides in this embodiment. In other words, the variable focal length image capture apparatus has a leading edge and a trailing edge. The focal length at the leading edge is approximately equal to the focal length of the first adjoining image capture apparatus. The focal length of the trailing edge approximately equal to the focal length of the second image capture apparatus.

[0041] While the embodiment of FIG. 2 includes fixed focal length image capture apparatuses that are adjacent to variable length image capture apparatuses, other embodiments can use various types of image capture apparatuses. For example, embodiments include variable length image capture apparatuses adjacent to other variable length image capture apparatuses, such that the focal lengths of the adjoining image capture apparatuses are approximately equal at the adjoining sides.

[0042] In embodiments, LIDAR, spectroscopy, bathymetry, topography, and other information can be acquired using additional data acquisition systems. In an embodiment, a LIDAR system can be implemented using four lasers mounted above and/or below the panoramic camera system 200, arranged to acquire underwater information including measurements that can be used to render dimensional and other information about the areas surrounding the LIDAR system. In embodiments, the LIDAR system can obtain measurements of underwater topology, even in shallow areas as a useful and practical alternative or addition to bathymetry methods.

[0043] Topological information, acquired using LIDAR or other technology, can be used to enhance captured images that are associated with the topological information. For example, panoramic images can be associated with a 3D mapping environment, including derived underwater shallow topography. Detailed underwater terrain views can be provided based on panoramic images and/or topological information for use with mapping and other data tools.

[0044] Additional types of information can be captured using LIDAR or other technology. In an embodiment, spectroscopy can be used to reveal a frequency spectrum of returning laser pulses associated with a LIDAR or other system. In embodiments, single channel fast Fourier analysis and other techniques can be used to provide additional data and statistics. Datasets can be associated with acquired images and/or topography. In embodiments, statistics on biodiversity can be derived, for example, from laser spectra associated with LIDAR or other technologies. In embodiments, biodiversity datasets can be derived based on analysis of organic material reflecting differently than rock.

[0045] FIG. 3 is a diagram of an exemplary imagery acquisition system, in accordance with an embodiment, including a panoramic camera system 300. Panoramic camera system 300 includes one or more lenses 302 connected via wires 304. The panoramic camera system 300 is enclosed in waterproof enclosure 306. Panoramic camera system 300 is supported under the water 310 by stabilizer 320. Stabilizer 320 is connected to support mount 328 mounted to boat 312. Cable management members 330 route wires 304 to control module 350. In an embodiment, control module 350 can include storage 352, power management module 354, and ancillary equipment 356. Control module 350 is coupled to a GPS device 340.

[0046] In the embodiment of FIG. 3, the wires 304 are enclosed in a protective sheath

326. The protective sheath 326 can be waterproof, and can provide reinforcing protection to the wires 304. Multiple cable management members 330 can be used to manage the wires 304. Embodiments can arrange the wires 304 within the stabilizer 320 for added protection and cable management. [0047] The stabilizer 320 includes an upper arm 322 and arm 324 for providing stability compensation and decoupling of the panoramic camera system 300 from movement of the support mount 328. Although illustrated as fixed-length, arm 324 can be extendable. The stabilizer 320 also includes a joint 323, which enables additional stability and positional control and flexibility. Stabilizer 320 can include springs, weights, and other mechanisms for providing stability.

[0048] FIG. 4 is a diagram of an exemplary imagery acquisition system in accordance with an embodiment, including panoramic camera system 400 mounted to submersible vehicle 412, submerged below the surface of the water 410. Panoramic camera system 400 includes one or more lenses 402 connected via wires 404. The panoramic camera system 400 is enclosed in waterproof enclosure 406. Panoramic camera system 400 is supported by stabilizer 420. Stabilizer 420 is connected to submersible vehicle 412. Wires 404 are routed to control module 450 through stabilizer 420. In an embodiment, control module 450 can include storage 452, power management module 454, and ancillary equipment 456.

[0049] In the embodiment of FIG. 4, a GPS float 442 is used to support GPS device 440 at the surface of the water 410 to obtain a GPS signal. GPS device 440 is coupled to control module 450 via GPS tether 444. GPS tether 444 enables the submersible vehicle 412 to dive to various depths while maintaining a GPS signal. Embodiments can use wireless communication between GPS device 440 and control module 450. Embodiments can enable the GPS device 440 to separately store GPS data associated with a timing log. The GPS data can be coordinated with associated timing information of captured panoramic images and other acquired data. Additionally, embodiments can use an extendible stabilizer 420 to position panoramic camera system 400.

[0050] In embodiments, GPS device 440 can be located at the control module 450 and coupled to a GPS antenna positioned in GPS float 442. In an embodiment, the GPS antenna can be supported above the surface of the water 410 by a mast. Embodiments can also arrange the GPS device 440 in a waterproof enclosure located on a top surface of the submersible vehicle 412, enabling the GPS device 440 to obtain a GPS signal while submersible vehicle 412 travels at or just below the surface of the water 410.

[0051] Embodiments can omit the GPS device 440, and use stored location data corresponding to a predetermined route. For example, submersible device 412 can be a sea sled remotely navigated or piloted underwater by a diver, following a predetermined route. Embodiments can include a sea sled with a GPS device and GPS antenna floating at the surface of the water 410 or supported by a mast. Embodiments can be autonomously navigated based on various criteria, including behavioral rules (e.g., remain 3 meters back from a coral reef or shore; cover a given stretch of water, etc.).

[0052] FIG. 5A is a diagram of an exemplary imagery acquisition system, in accordance with an embodiment, including a submersible stabilizer 560. Panoramic camera system 500 is coupled to submersible stabilizer 560 under the water 510. Panoramic camera system 500 includes one or more lenses 502. The panoramic camera system 500 can be enclosed in waterproof enclosure 506. In embodiments, enclosure 506 can include a balancer 572 and can be coupled to submersible stabilizer 560 by enclosure couplings 562. Wires 504 couple panoramic camera system 500 to control module 550. In an embodiment, control module 550 can include storage 552, power management module 554, and ancillary equipment 556. Control module 550 is coupled to a GPS device 540.

[0053] Cable management member 530 routes wires 504. In embodiments, management member 530 can include a telescoping pole that suspends the wires 504 out of the water 510 away from the boat 512. In the embodiment illustrated in FIG. 5 A, cable management member 530 can maintain the wires 504 away from outboard motors, propellers, propulsion systems, or other protrusions of boat 512. Cable management member 530 is mounted to boat 512 by support mount 528.

[0054] In the illustrated embodiment, submersible stabilizer 560 can be towed behind boat 512 such that the submersible stabilizer 560 is decoupled from a mechanical connection to the boat 512. The submersible stabilizer 560 can use one or more stabilizing fins 564 to help achieve stability and an orientation parallel to the water surface during movement through the water 510.

[0055] In an embodiment, the submersible stabilizer 560 includes air bladder 566 or other buoyancy control system. Air bladder 566 can be coupled to air line 568, which can be coupled to air source 570. Air source 570 can add and release air pressure to/from air line 568, to obtain an appropriate buoyancy for positioning submersible stabilizer 560 to acquire images and other data. Other gases and/or fluids can be used to adjust the buoyancy of the submersible stabilizer 560. Additionally, an angle of the stabilizing fins 564 can be adjusted to control the submersible stabilizer 560. [0056] Wires 504 and air line 568 can be bundled together. In embodiments, a tow cable can be used for tensile loads associated with moving submersible stabilizer 560 through the water 510. The wires 504 and air line 568 can be coupled to the cable using various bundling techniques.

[0057] FIG. 5B is a top view of an exemplary imagery acquisition system, in accordance with an embodiment. Submersible stabilizer 560 includes one or more stabilizing fins 564 and air bladder 566. The stabilizing fin 564 is illustrated as having a streamlined profile that can benefit movement of submersible stabilizer 560 through the water 510.

[0058] FIG. 5C is a diagram of an exemplary imagery acquisition system, in accordance with an embodiment. Panoramic camera system 500 includes one or more lenses 502. The panoramic camera system 500 can be enclosed in waterproof enclosure 506. In embodiments, enclosure 506 includes balancer 572 and is coupled to submersible stabilizer 560 by enclosure couplings 562. Wires 504 are coupled to panoramic camera system 500, and air line 568 is coupled to air bladder 566. Air bladder 566 is coupled to stabilizing fin 564.

[0059] The balancer 572 can mechanically decouple the panoramic camera system 500 from the enclosure 506, further stabilizing the panoramic camera system 500 for improved acquisition of panoramic images and other data. The balancer 572 can act as a balance scale, allowing panoramic camera system 500 to align with respect to a gravitational pull. FIG. 5C illustrates the panoramic camera system 500 displaced within the enclosure 506 to remain level.

[0060] In an embodiment, the balancer 572 is a balancing roller. A cylindrical roller can be mounted stationary with respect to the enclosure 506. The cylindrical roller can be mounted perpendicular to a direction the panoramic camera system 500 is moved through the water 510. Embodiments of balancer 572 can include other fulcrums such as a knife edge. Embodiments can also balance the panoramic camera system 500 by suspending it like a pendulum using a pivot joint, tether, or other pivoting coupling. Fulcrums can include spherical bearings, points, or other mechanisms to accommodate omni directional balancing stability of the panoramic camera system 500 with respect to the enclosure 506.

[0061] As illustrated in FIG. 5C, submersible stabilizer 560 is tilted slightly downwards.

Submersible stabilizer 560 can tilt due to a towing force and corresponding movement of submersible stabilizer 560 through the water 510. Accordingly, submersible stabilizer 560 can use stabilizing fin 564 to interact with water movement and dive to a lower depth under the water 510. Air bladder 566 can be used to counteract the diving force and/or tilt when an appropriate depth is reached. Weights/sinkers can also be used to provide a diving and/or tilting force, in addition to the weight of the imagery acquisition system.

[0062] In the illustrated embodiment, enclosure 506 is coupled to the submersible stabilizer 560 by enclosure couplings 562. Accordingly, the enclosure 506 tilts along with the submersible stabilizer 560. The enclosure 506 isolates the panoramic camera system 500 from forces caused by water pressure as a result of movement through the water. Accordingly, the balancer 572 can allow the panoramic camera system 502 to passively align itself under the force of gravity. Additionally, the balancer 572 can allow the panoramic camera system 502 to resist jerks, accelerations, or other forces introduced by the towing cable.

[0063] Embodiments can use active or passive stability on the panoramic camera system

500 with respect to the enclosure 506. For example, the imagery acquisition system can use a tilt sensor or other sensors and a motorized balancer 572 to stabilize the panoramic camera system 500 with respect to the enclosure 506.

[0064] Various features from various embodiments can be combined. For example, the embodiments of FIGS. 4 and 5 can include an articulated stabilizer as in FIGS. 1 and 3 to stabilize the panoramic camera system and enclosure. The balancer 572 and stabilizing fins of FIGS. 5A-5C can be used on the embodiments of FIGS. 1, 3, and 4. Accordingly, the panoramic camera system can remain steady as it moves along a scan path underwater. This allows high-resolution images, as well as additional data, to be captured as a sea-based vehicle moves along a path over fairly large distances efficiently and with minimal downtime.

[0065] FIG. 6A is a diagram of an exemplary imagery acquisition system, in accordance with an embodiment, including panoramic camera system 600 below the surface of the water 610. For example, panoramic camera system 600 may be coupled via wires 604 to a control module (not shown). The panoramic camera system 600 can be enclosed in waterproof enclosure 606. In some embodiments, panoramic camera system 600 may include one or more optical elements, such as lenses 602, for obtaining images over a panoramic field of view and associated detector elements. [0066] In one example, panoramic camera system 600 is mounted on a side of an underwater propulsion unit 660. A diver 668 can grip propulsion unit 660 and swim freely underwater while panoramic imagery is captured. A diver 668 can hold on to and guide underwater propulsion unit 660 while panoramic camera system 600 captures imagery for panoramic images.

[0067] In the embodiment of FIG. 6A, a GPS float 642 is used to support GPS device 640 at the surface of the water 610 to obtain a GPS signal. GPS device 640 is coupled to camera system 600 via GPS tether 644. Camera system 600 may be coupled to a control module. Embodiments can use wireless communication between GPS device 640 and a control module. In some embodiments, camera system 600 includes a sealed multiconductor cable for data transmission. In some embodiments, the sealed multiconductor cable is a sealed Ethernet cable.

[0068] A length of tether 644 can vary. For example, in one embodiment, a length of tether 644 is 3-5 meters. In another embodiment, a length of tether 644 is less than 3 meters or greater than 5 meters. Tether 644 may provide power and data transfer to camera system 600. For example, tether 644 may allow camera system 600 to be used indefinitely for imagery acquisition.

[0069] FIG. 6B is a diagram of panoramic camera system 600, housing 680, and sealed cable 607, in accordance with an embodiment. In one example not intended to be limiting, camera system 600 is approximately spherical. A spherical section of panoramic camera system 600 can vary. In one embodiment, a spherical section of panoramic camera system 600 is approximately 20 centimeters in diameter. In another embodiment, the spherical section of panoramic camera system 600 is less than or greater than 20 centimeters in diameter.

[0070] In one embodiment, the panoramic camera system is 12.5 centimeters x 12.5 centimeters x 34 centimeters long. Housing 680 may include heat sinks. In one embodiment, the heat sinks dissipate approximately 25 watts into the water.

[0071] Sealed cable 607 can be coupled to a control module and panoramic camera system 600. Sealed cable 607 can be underwater or above a surface of water. In one embodiment, sealed cable 607 connects to supporting equipment. For example, wires can be housed in sealed cable 607, and couple panoramic camera system 600 to a control module. [0072] In another embodiment, sealed cable 607 is not connected to supporting equipment. For example, panoramic camera system 600 can run independently. In one embodiment, panoramic camera system 600 may be a standalone version that uses battery power and local storage.

[0073] Deep sea exploration may be difficult if, for example, the GPS tether 644 in FIG.

6A is too long. For instance, GPS tether 644 may get caught in or tangled around an object, or a water depth may be too deep for a GPS signal. In some embodiments, a panoramic camera system is not tethered to a separate device. For example, the panoramic camera system can acquire imagery independent from a separate connection.

[0074] FIG. 7 is a diagram of an exemplary imagery acquisition system, in accordance with an embodiment, including panoramic camera system 700 submerged below the water. In some embodiments, panoramic camera system 700 may include one or more optical elements, such as lenses 702, for obtaining images over a panoramic field of view and associated detector elements. The panoramic camera system 700 is enclosed in waterproof enclosure 706.

[0075] The embodiment of FIG. 7 may be used in deep sea exploration, and can be used to capture images underwater, for example, at or near a bottom of an ocean floor (e.g., shipwrecks). A diver 768 can swim underwater and capture panoramic images using panoramic camera system 700. Images captured by panoramic camera system 700 can be associated with location data. In the embodiment of FIG. 7, a distance or a direction of an object is determined underwater. Location data can be acquired from, for example, positioning devices 774 and 782, which are devices that detect position.

[0076] Positioning devices 774 and 782 can be used as reference points (e.g., relative to a predetermined location). For example, positioning device 774 can be mounted on a pole 720 and the pole 720 can be placed at a first predetermined location. The first predetermined location may be an absolute reference point. Positioning device 782 can be mounted on a pole 728 and the pole 728 can be placed at a second predetermined location. The second predetermined location may be an absolute reference point. The first location may be a first distance (e.g., 100 meters) in a first direction away from a site. The second location may be a second distance (e.g., 100 meters) in a second direction away from the site. Images taken by panoramic camera system 700 can be taken relative to absolute reference points established by positioning devices 774 and 782. [0077] In one example, a location of panoramic camera system 700 can be detected based on sonar (sound navigation and ranging) and locations of the reference points. Sonar is a technique that uses sound propagation to detect, navigate to, and communicate with objects. Sonar may be used as a means of acoustic location. Acoustic location is a science of using sound to determine a distance or a direction of an object.

[0078] Active sonar or passive sonar may be used to obtain reference or location data.

Active sonar may emit pulses of sound waves through a medium (e.g., water), and receive waves reflected by objects. Active acoustic location involves creating sound to produce an echo, which is then analyzed to determine a location of an object.

[0079] Passive sonar may include listening to sound waves generated by objects. Passive acoustic location involves detecting sound or vibration created by a target object, which is then analyzed to determine a location of the target object. Acoustic mirrors and dishes, when using microphones, may support passive acoustic localization. Further, using speakers can be used to carry out active localization. Typically, more than one device is used, and a location can be triangulated between devices.

[0080] Signals from panoramic camera system 700 or positioning devices 774 and 782 can be used to obtain location data. In some embodiments, positioning devices 774 and 782 can provide location data to be associated with images acquired by panoramic camera system 700. In one example, positioning devices 774 and 782 include a detector that can detect signals from panoramic camera system 700. The signals can be used to determine a location of camera system 700 in reference to positioning devices 774 and 782. For example, positioning device 774 can detect a position of panoramic camera system 700 in reference to positioning device 774 and positioning device 782. The detected location can be associated with images captured by panoramic camera system 700. Positioning devices 774 and 782 may use sonar. A positioning device may also associate captured panoramic images with corresponding location data.

[0081] In some embodiments, panoramic camera system 700 can provide location data to be associated with images acquired by panoramic camera system 700. In one example, panoramic camera system 700 includes a detector that can detect signals from positioning devices 774 and 782. The signals can be used to determine a location of camera system 700 in reference to positioning devices 774 and 782. The detected location can be associated with images captured by panoramic camera system 700. Panoramic camera system 700 may include detectors that use sonar. Panoramic camera system 700 may also associate captured panoramic images with corresponding location data.

[0082] Systems which employ underwater acoustic energy for observation or communication may be called sonar systems. The term sonar may also be used for the equipment used to generate and/or receive the sound waves. The frequencies used in sonar systems vary from infrasonic to ultrasonic, and include relatively low to relatively high frequencies.

[0083] In one embodiment, the embodiment of FIG. 7 is a low frequency system that supports referencing. In another embodiment, the embodiment of FIG. 7 is a high frequency system that supports referencing. The high frequency system can be used to acquire more accurate positioning within the field. The choice of frequency to use can vary depending upon a particular application such as depth of water, cost, or accuracy desired, as would be appreciated by a person skilled in the relevant art.

[0084] In some embodiments, positioning devices 774 and 782 are coupled to a control module. The control module can be coupled to panoramic camera system 700. Embodiments can use wireless communication between positioning devices 774 and 782 and the control module. Location data of captured imagery can be sent to a remote location. For example, location data of captured panoramic images can be stored and processed on board to get the location data.

[0085] In some embodiments, more than two positioning devices may be used. A number of positioning devices used may depend on such factors as a desired area to cover to capture panoramic imagery and associated it with corresponding location data and a strength of a signal transmission/reception.

[0086] FIG. 8 is a diagram of an exemplary imagery acquisition system in accordance with an embodiment, including panoramic camera system 800 mounted to a submersible vehicle 812, submerged below a surface of water. In one embodiment, panoramic camera system 800 is housed within submersible vehicle 812. Submersible vehicle 812 may be remotely controlled.

[0087] In some embodiments, panoramic camera system 800 may include one or more optical elements such as ienses 802 for obtaining images over a panoramic field of view and associated detector elements. The panoramic camera system 800 is enclosed in waterproof enclosure 806. In one embodiment, submersible vehicle 812 includes a control module. Wires 804 can be routed to the control module. In another embodiment, positioning devices 774 and 782 and a control module use wireless communication. In some embodiments, submersible vehicle 812 is a REMUS (Remote Environmental Monitoring Units) vehicle.

[0088] While various embodiments of the present invention have been described above, it should be understood that they have been presented by way of example, and not limitation. It will be apparent to persons skilled in the relevant art(s) that various changes can be made therein without departing from the scope of the invention. Furthermore, it should be appreciated that the detailed description of the present invention provided herein, and not the summary and abstract sections, is intended to be used to interpret the claims. The summary and abstract sections may set forth one or more but not all exemplary embodiments of the present invention as contemplated by the inventors.

[0089] The foregoing description of the specific embodiments will so fully reveal the general nature of the invention that others can, by applying knowledge within the skill of the art, readily modify and/or adapt for various applications such specific embodiments, without undue experimentation, without departing from the general concept of the present invention. Therefore, such adaptations and modifications are intended to be within the meaning and range of equivalents of the disclosed embodiments, based on the teaching and guidance presented herein. It is to be understood that the phraseology or terminology herein is for the purpose of description and not of limitation, such that the terminology or phraseology of the present specification is to be interpreted by the skilled artisan in light of the teachings and guidance.

Claims

WHAT IS CLAIMED IS:

1. A panoramic imagery acquisition method comprising:

capturing a panoramic image underwater using a panoramic camera system; and associating the captured panoramic image with corresponding location data.

2. The method of claim 1, further comprising stabilizing the panoramic camera system to mechanically decouple movement of the panoramic camera system from movement of a support mount.

3. The method of claim 2, wherein the stabilizing comprises using active feedback from sensors to compensate from movement of the support mount.

4. The method of claim 1, further comprising telescoping an extendable arm to mechanically decouple movement of the panoramic camera system from movement of a support mount.

5. The method of claim 1, further comprising a waterproof case, wherein the panoramic camera system is enclosed in the waterproof case.

6. The method of claim 1, further comprising:

maintaining time logs of captured panoramic images and location data, wherein the time logs enable synchronizing the captured panoramic images with the location data.

7. The method of claim 1 , further comprising illuminating scenery for the panoramic camera system using a lighting system.

8. The method of claim 1, further comprising:

capturing topographic data underwater using a LIDAR system; and associating the captured panoramic image with corresponding topographic data.

9. The method of claim 8, further comprising:

capturing reflected laser spectra underwater; and deriving biodiversity data based on the reflected laser spectra.

10. The method of claim 1, further comprising capturing the location data using a GPS system.

1 1. The method of claim 1, further comprising stabilizing the panoramic camera system using a stabilizing fin.

12. The method of claim 1, further comprising capturing the location data using a sonar system.

13. A panoramic imagery acquisition system comprising:

a panoramic camera system to capture a panoramic image underwater; and a controller operable to associate the captured panoramic image with corresponding location data.

14. The system of claim 13, further comprising a stabilizer to mechanically decouple movement of the panoramic camera system from movement of a support mount.

15. The system of claim 13, further comprising a telescoping unit to mechanically decouple movement of the panoramic camera system from movement of a support mount.

16. The system of claim 13, further comprising a waterproof case to enclose the panoramic camera system.

17. The system of claim 13, further comprising a balancer to mechanically decouple the panoramic camera system from the waterproof case.

1-8. The system of claim 13, further comprising a lighting system to illuminate scenery for the panoramic camera system.

19. The system of claim 13, further comprising: a LIDAR system to capture topographic data underwater; and

a controller operable to associate the captured panoramic image with corresponding topographic data.

20. The system of claim 19, further comprising a receiver to capture reflected laser spectra underwater to derive biodiversity data based on the reflected laser spectra.

21. The system of claim 13, further comprising a GPS system to capture the location data.

22. The system of claim 13, further comprising sensors to account for offsets between a location of the panoramic camera system underwater and the GPS system, wherein the offset is incorporated into the location data associated with the captured panoramic image.

23. The system of claim 13, further comprising a stabilizing fin to stabilize the panoramic camera system.

24. The system of claim 13, further comprising a positioning system that uses sonar to capture the location data.

25. The system of claim 13, wherein the panoramic camera system is mounted to a remotely controlled submersible unit.

26. The system of claim 13, wherein the panoramic camera system comprises one or more lenses arranged to capture images in a panoramic field of view.

27. The system of claim 13, further comprising:

a submersible vehicle, wherein the panoramic camera system is mounted to the submersible vehicle underwater; and

a GPS device coupled to a control module via a tether, the control module being housed in the submersible vehicle, and the tether enabling the submersible vehicle to dive to various depths while maintaining a GPS signal.

28. The system of claim 13, wherein the GPS device is located at the control module and further coupled to a GPS antenna positioned in a GPS float.