WO2008079862A1

WO2008079862A1 - A wide-angle, high-resolution imaging system

Info

Publication number: WO2008079862A1
Application number: PCT/US2007/088089
Authority: WO
Inventors: Michael Mojaver; Steven Branson
Original assignee: Tempest Microsystems
Priority date: 2006-12-20
Filing date: 2007-12-19
Publication date: 2008-07-03

Abstract

In one aspect, the present invention provides an optical imaging system that includes a wide-angle imaging device (e.g., a fisheye camera) and a high-resolution camera having pan, tilt and zoom capabilities that are mounted to single enclosure such that neither camera occludes the field-of-view accessible to the other.

Description

A WIDE-ANGLE, HIGH-RESOLUTION IMAGING SYSTEM

Background The present invention relates generally to imaging systems, and more particularly, to imaging systems that employ wide-angle as well as high-resolution cameras.

The use of surveillance and monitoring systems has rapidly increased over the past decade. Many such systems rely on a variety of imaging devices to monitor a field-of-view. Although a variety of such systems are known, there is a still a need for enhanced imaging systems that allow efficient monitoring of a scene.

Summary

In one aspect, the present invention provides an optical imaging system that includes a wide-angle imaging device and a camera having pan, tilt and zoom capabilities (herein referred to as the "PTZ camera"), which are mounted in a single enclosure such that the camera's pan and tilt can be adjusted to view any portion of the field of view accessible to the wide-angle imaging device (herein also referred to as the wide-angle imaging subsystem). Further, the zoom capability of the PTZ camera can be utilized to obtain an enlarged image of any desired portion of that field of view.

In a related aspect, the wide-angle imaging device includes a wide-angle lens, such as a fish-eye lens, that is optically coupled to an image capture device, e.g., a CCD array, to obtain the wide-angle image of a field of view. By way of example, the wide-angle lens can be a fish-eye lens, optically coupled to an imaging device, so as to generate a hemispherical image (i.e., one characterized by azimuthal angles spanning

180 degrees and polar angles spanning 360 degrees) of a scene of interest.

In another aspect, in the above imaging system, the enclosure can include a circular track onto which the PTZ camera can be movably mounted so as to obtain images of a scene without interfering with the wide-angle imaging device's field of view. By way of example, the enclosure can include a dome-like structure at the center of which the wide-angle lens is mounted with the circular track forming the perimeter of a cross-sectional segment of the dome. In this manner, the wide-angle imaging system can obtain images of a field of view, e.g., one subtended by a solid angle defined by the wide-angle lens, while the PTZ camera can be moved along the track to obtain images of different portions of that field of view and/or portions lying outside that field of view. In a related aspect, the wide-angle imaging subsystem and the PTZ camera are positioned relative to one another so as to ensure that the PTZ camera does not obstruct any portion of the wide-angle imaging subsystem's field of view, and/or the wide- angle imaging subsystem (e.g., a wide-angle camera) does not occlude any portion of a field of view accessible by the PTZ camera (e.g., a field of view accessible via full range of motion (tilt and pan) of the PTZ camera). Such positioning of the wide-angle imaging subsystem and the PTZ camera can be achieved, e.g., via their radial and/or axial offset relative to one another.

In another aspect, the imaging system includes a processing unit that can receive image signals from both the wide-angle imaging subsystem as well as the PTZ camera and process those images. Further, the processing unit can control the pan, tilt and/or zoom of the PTZ camera, e.g., in real-time. In many embodiments, the processing unit can compress and record video images obtained by the wide-angle imaging device and/or the PTZ camera. Moreover, the processing unit can include instructions for applying computer vision algorithms to images acquired by the wide- angle imaging system and/or the PTZ camera.

In a related aspect, the processing unit utilizes one or more images generated by the wide-angle imaging subsystem to control the movement of the PTZ camera (pan, tilt and/or zoom), e.g., in real-time. For example, the processor can employ image processing software to identify an image of a particular object in the wide-angle image, and then instruct the PTZ camera to acquire a high resolution image of that object.

In other aspects, an imaging system is disclosed that includes an enclosure and a wide-angle camera that is mounted to the enclosure so as to provide a wide-angle image of a scene. The imaging system further includes a second camera mounted to the same enclosure so as to be movable about two substantially orthogonal directions for acquiring images corresponding to different portions of a field-of-view accessible to the wide-angle camera. The two cameras are offset relative to one another such that the field-of-view accessible to the wide-angle camera remains unoccluded by the second camera and/or the field-of-view accessible to the second camera remain unoccluded by the wide-angle camera. In other words, preferably, neither camera causes the other to have a blind spot. In a related aspect, the enclosure includes a track, which at least partially surrounds the wide-angle camera and to which the second camera is coupled. In some embodiments the track is rotatable, e.g., about an axis perpendicular to a plane defined by the track, to allow rotating the second camera about the wide-angle camera. In some other embodiments, the track is stationary and the second camera is movable around the track.

In another aspect, the enclosure comprises a rotatable dome (e.g., a dome that is capable of rotating 360 degrees) to which the PTZ camera is coupled. By way of example, the dome can be driven by a motorized gear disposed beneath the dome. A motorized zoom lens associated with the PTZ camera can be mounted to the side (hemispheric part) of the dome along a pivoting axis and be driven, e.g., by another gear and motor. In some cases, a flexible wire can connect the zoom lens enclosure to the bottom of the dome. The dome can include a disk at a bottom thereof, where the disk has opening (e.g., at its center) through which the wide-angle camera (e.g., a fisheye camera) can be inserted into the dome. A tilt mechanism can be mounted on an axis that is away from the center and is attached to one side of the dome.

Further understanding of the invention can be obtained by reference to the following detailed description in conjunction with the associated drawings, which are described briefly below.

Brief Description of the Drawings

FIGURE IA is a schematic view of an imaging system according to one embodiment of the invention having a wide-angle as well as a PTZ camera,

FIGURE IB is a schematic bottom view of the system shown in FIGURE IA,

FIGURE 1C is another schematic perspective view of the system shown in FIGURES IA and IB, depicting the movement of the PTZ camera about the wide- angle camera, FIGURE 2 schematically depicts that the wide-angle camera in an optical imaging system according to some embodiments of the invention can include a fish- eye lens and an image capture device,

FIGURE 3 schematically illustrates some components of an imaging system according to an embodiment of the invention,

FIGURE 4 schematically depicts an imaging system in accordance with some embodiments of the invention in which a microcontroller is programmed to apply vision algorithms to acquired images, and

FIGURE 5 schematically depicts a plane of a fish-eye image.

Detailed Description

With reference to FIGURES IA, IB, 1C, and 2, an imaging system 10 in accordance with an exemplary embodiment of the invention includes a wide-angle imaging camera 12 comprising a fish-eye lens 14 (or other wide-angle lens) coupled to an image acquisition device 16 (e.g., a CCD array) and a pan/tilt/zoom (PTZ) camera 18 (in some embodiments, the camera 18 may not have a zoom capability). The wide- angle camera and the PTZ camera are integrated within a single enclosure 20 so as to obtain a high-resolution hemispherical image with no blind spots. In this exemplary embodiment, the enclosure 20 is in the form of a dome-like structure, to which the wide-angle imaging camera is mounted such that it has a full hemispherical view without being occluded by any part of the enclosure. More specifically, in this embodiment, the wide-angle imaging camera is mounted to the enclosure along an axial direction A that extends vertically from the apex of the structure. The enclosure 20 further includes a circular track 22 to which the PTZ camera can be mounted (in some other embodiments, a rotatable disk, rather than a circular track, is employed or the entire dome is adapted for rotation). The track is rotatable about the axial direction

A (i.e., it is rotatable about the wide-angle camera 12) such that the PTZ camera can pan around the wide-angle camera. Although in this embodiment, the track 22 spans 360 degrees around the wide-angle camera, in other embodiments, it can surround the camera along an arc that provides a circular span that is less than 360 degrees. Various mounting devices known in the art can be employed to couple the PTZ camera to the track. In this embodiment, the PTZ camera 18 is axially offset (i.e., along the axial direction A) relative to the wide-angle camera such that it would not occlude the wide- angle camera's field of view. By way of example, in embodiments in which the wide- angle camera includes a fish-eye lens, this axial offset can be selected so as to ensure that the fish-eye lens has a full hemispherical field of view without interference from the PTZ camera (without being occluded by the PTZ camera).

Not only can the PTZ camera be rotated around the wide-angle camera, but it can also be tilted about an axis B (which in this embodiment is substantially orthogonal to the axis A) to view different portions of the field of view that is accessible to the wide-angle camera. More particularly, in this exemplary embodiment, the PTZ camera is pivotably mounted to a bracket 24, which is in turn coupled to the track 22, such that the PTZ camera can be tilted about the axis B at any desired tilt angle in a range greater than zero to less than or equal to about 90 degrees. The ability of the PTZ camera to rotate and tilt about the axes A and B, respectively; that is, its ability to rotate about two orthogonal directions, allows it to acquire an image of any portion of a scene that is within the field of view of the fish-eye lens.

The PTZ camera can also include a zoom lens system that allows zooming in on desired locations within a field of view, e.g., within the hemispherical field of view of a fish-eye lens of the wide-angle camera.

As shown schematically in FIGURE 3, the exemplary imaging system can also include a processing unit 26 that is in communication with the wide-angle camera and the PTZ camera, e.g., via a communications interface 28 (e.g., an Ethernet interface), to process video sequences that it receives from both cameras. In addition, the processing unit can control the pan, tilt and/or zoom of the PTZ camera, e.g., in realtime. In some embodiments, the processing unit can compress and/or record, e.g., simultaneously, video images it receives from one or both cameras. In some embodiments, the processing unit can apply computer vision algorithms to the imaging data it receives from one or both of the cameras, as well as record, and in some cases compress, semantic information obtained by those algorithms. While in some embodiments, the processing unit can be integrated within the enclosure 20, in other embodiments it can be disposed in a separate enclosure. In addition, a variety of Communications protocols known in the art, including wireless protocols, can be employed to transfer data and/or commands between the cameras 12 and 18 and the processing unit 26.

In some embodiments, the processing unit 26 can control the movement of the PTZ camera based on images acquired by the wide-angle camera. By way of example, the processing unit can include image-processing software that can identify an object of interest in a wide-angle image. The processing unit can then instruct the PTZ camera to focus on that object to obtain a high resolution image thereof.

As shown schematically in FIGURE 4, in some embodiments, a microcontroller 30 onto which one or more desired vision algorithms are programmed is mounted within the enclosure 20, and is in communication with one or both cameras. By way of example, the micro-controller can be used to autonomously control the PTZ camera, generate semantic information, and compress the video streams in real-time. This information can be made available on request, e.g., through an Ethernet communication port mounted to the camera enclosure.

The wide-angle camera and the PTZ camera can be implemented based on a number of different architectures. For example, as discussed above (See Figure 2), the wide-angle camera can include a fish-eye lens in communication with an image- capture device, e.g., a CCD array. In addition, the wide-angle camera can include its dedicated processor that can be utilized to process the captured images. By way of example, U.S. Patent No. 6,833,843 entitled "Panoramic Imaging and Display System with Canonical Magnifier," which is herein incorporated by reference, teaches a wide- angle imaging system that is suitable for use in the practice of the invention.

The imaging system 10 can be employed in a variety of applications, e.g., intelligent tracking and surveillance. For example, the enclosure can be mounted on the ceiling of a large room or an external wall surface of a building. The fisheye camera, e.g., a megapixel fisheye camera, can then be used to observe a continuous

360-degree video stream of the entire room at all times. Further, computer vision algorithms known in the art can be applied to the images acquired by the fisheye camera to detect moving objects and/or observe patterns of motion. The PTZ camera can be used to zoom in on a particular location within the hemispherical field of view to acquire a high-resolution image of that location, which can be used, e.g., for object or facial recognition. For example, analysis of the fisheye image can indicate motion within a certain portion of the wide-angle image, the processor can then direct the PTZ to view that portion of the image so as to obtain a high-resolution image that can provide more detailed information regarding the detected motion.

Appendix A below provides further details regarding various aspects of the invention. Those having ordinary skill in the art will appreciate that various changes can be made to the above embodiments without departing from the scope of the invention. For example, in the wide-angle imaging system, rather than a fisheye lens, other types of wide-angle lenses can be employed.

Appendix A

In many embodiments, the PTZ camera includes a zoom lens, which can rotate along two orthogonal axes (pan and tilt) using a combination of gears and levers. The zoom lens can rotate (pan) 360° within the image plane by rotating on a circular disk within the dome. In addition, it can rotate (tilt) 90° orthogonal to the image plane. The combined rotations about the two axes allow the PTZ camera to point in any direction within a hemisphere of view.

Pan: 0° <= θ <= 360° Tilt: 0° <= β <= 90°

The wide-angle (e.g., fisheye) camera remains static and covers the same field of view as the PTZ camera. With reference to FIGURE 5, a point (P) in the fisheye image can be correlated to a pan/tilt orientation (θ,β) using the following equations:

Q = θ_f β = (d/r) (90°)

where θf is the angle of point P counterclockwise from the x-axis in the fisheye image, d is the distance in pixels of P from the center of the fisheye image, and r is the distance in pixels of the radius of the fisheye image.

In this way, an object detected within the fisheye image can be easily used to specify a desired orientation of the PTZ camera.

Claims

Claims:

1. An imaging system comprising an enclosure, a first wide-angle camera mounted to said enclosure so as to provide a wide- angle image of a scene, a second camera mounted to said enclosure so as to be movable about two substantially orthogonal directions for acquiring images corresponding to different portions of a field-of-view accessible to said wide-angle camera, wherein said cameras are offset relative to one another such that the field-of- view accessible to the wide-angle camera remains unoccluded by said second camera.

2. The imaging system of claim 1, wherein said second camera comprises a high- resolution camera.

3. The imaging system of claim 1, wherein said enclosure further comprises a track at least partially surrounding said wide-angle camera to which said second camera is mounted.

4. The imaging system of claim 3, wherein said track is rotatable about said wide- angle camera.

5. The imaging system of claim 1, wherein said wide-angle camera comprises a fish-eye lens

6. The imaging system of claim 1, further comprising a processor in communication with said first and second cameras, said processor utilizing one or more images generated by said first camera to control said second camera in real-time.

7. The imaging system of claim 6, wherein said first processor controls movement of said second camera along at least one of said orthogonal directions based on said images generated by the first camera.

8. An imaging system, comprising a dome-like structure having a circular track along a perimeter of a cross- sectional segment thereof, a wide-angle camera mounted to said dome so as to provide a wide-angle image of a field of view, and a PTZ camera movably mounted to said circular track so as to be capable of obtaining images of different portions of said field of view.

9. The imaging system of claim 8, wherein said wide-angle camera and said PTZ camera are positioned relative to one another such that neither camera would obstruct a field of view of the other.

10. The imaging system of claim 9, wherein said PTZ camera is positioned outside a solid angle representing a field of view of said wide-angle camera.

11. The imaging system of claim 8, wherein said wide-angle camera is disposed along an axial direction extending from an apex of said dome-like structure.

12. The imaging system of claim 8, further comprising a processing unit adapted to receive image signals from said wide-angle and said PTZ cameras.

13. The imaging system of claim 12, wherein said processing unit is adapted to apply one or more control signals to said PTZ camera to instruct the camera to obtain a high-resolution image of a portion of an image obtained by said wide-angle camera.