US20140009571A1 - Wide Field of View Reflector and Method of Designing and Making Same - Google Patents

Wide Field of View Reflector and Method of Designing and Making Same Download PDF

Info

Publication number
US20140009571A1
US20140009571A1 US13/925,728 US201313925728A US2014009571A1 US 20140009571 A1 US20140009571 A1 US 20140009571A1 US 201313925728 A US201313925728 A US 201313925728A US 2014009571 A1 US2014009571 A1 US 2014009571A1
Authority
US
United States
Prior art keywords
scene
sub
freeform
sensor
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US13/925,728
Inventor
Zheng Jason Geng
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Geng Zheng Jason
Original Assignee
Zheng Jason Geng
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority to US86712906P priority Critical
Priority to US11/984,702 priority patent/US8471892B2/en
Application filed by Zheng Jason Geng filed Critical Zheng Jason Geng
Priority to US13/925,728 priority patent/US20140009571A1/en
Publication of US20140009571A1 publication Critical patent/US20140009571A1/en
Application status is Abandoned legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/222Studio circuitry; Studio devices; Studio equipment ; Cameras comprising an electronic image sensor, e.g. digital cameras, video cameras, TV cameras, video cameras, camcorders, webcams, camera modules for embedding in other devices, e.g. mobile phones, computers or vehicles
    • H04N5/225Television cameras ; Cameras comprising an electronic image sensor, e.g. digital cameras, video cameras, camcorders, webcams, camera modules specially adapted for being embedded in other devices, e.g. mobile phones, computers or vehicles
    • H04N5/232Devices for controlling television cameras, e.g. remote control ; Control of cameras comprising an electronic image sensor
    • H04N5/23238Control of image capture or reproduction to achieve a very large field of view, e.g. panorama
    • GPHYSICS
    • G08SIGNALLING
    • G08BSIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B13/00Burglar, theft or intruder alarms
    • G08B13/18Actuation by interference with heat, light or radiation of shorter wavelength; Actuation by intruding sources of heat, light or radiation of shorter wavelength
    • G08B13/189Actuation by interference with heat, light or radiation of shorter wavelength; Actuation by intruding sources of heat, light or radiation of shorter wavelength using passive radiation detection systems
    • G08B13/194Actuation by interference with heat, light or radiation of shorter wavelength; Actuation by intruding sources of heat, light or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
    • G08B13/196Actuation by interference with heat, light or radiation of shorter wavelength; Actuation by intruding sources of heat, light or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
    • G08B13/19617Surveillance camera constructional details
    • G08B13/19626Surveillance camera constructional details optical details, e.g. lenses, mirrors, multiple lenses
    • G08B13/19628Surveillance camera constructional details optical details, e.g. lenses, mirrors, multiple lenses of wide angled cameras and camera groups, e.g. omni-directional cameras, fish eye, single units having multiple cameras achieving a wide angle view
    • GPHYSICS
    • G08SIGNALLING
    • G08BSIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B13/00Burglar, theft or intruder alarms
    • G08B13/18Actuation by interference with heat, light or radiation of shorter wavelength; Actuation by intruding sources of heat, light or radiation of shorter wavelength
    • G08B13/189Actuation by interference with heat, light or radiation of shorter wavelength; Actuation by intruding sources of heat, light or radiation of shorter wavelength using passive radiation detection systems
    • G08B13/194Actuation by interference with heat, light or radiation of shorter wavelength; Actuation by intruding sources of heat, light or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
    • G08B13/196Actuation by interference with heat, light or radiation of shorter wavelength; Actuation by intruding sources of heat, light or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
    • G08B13/19639Details of the system layout
    • G08B13/19641Multiple cameras having overlapping views on a single scene
    • G08B13/19643Multiple cameras having overlapping views on a single scene wherein the cameras play different roles, e.g. different resolution, different camera type, master-slave camera
    • GPHYSICS
    • G08SIGNALLING
    • G08BSIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B13/00Burglar, theft or intruder alarms
    • G08B13/18Actuation by interference with heat, light or radiation of shorter wavelength; Actuation by intruding sources of heat, light or radiation of shorter wavelength
    • G08B13/189Actuation by interference with heat, light or radiation of shorter wavelength; Actuation by intruding sources of heat, light or radiation of shorter wavelength using passive radiation detection systems
    • G08B13/194Actuation by interference with heat, light or radiation of shorter wavelength; Actuation by intruding sources of heat, light or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
    • G08B13/196Actuation by interference with heat, light or radiation of shorter wavelength; Actuation by intruding sources of heat, light or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
    • G08B13/19697Arrangements wherein non-video detectors generate an alarm themselves
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00Television systems
    • H04N7/18Closed circuit television systems, i.e. systems in which the signal is not broadcast
    • H04N7/183Closed circuit television systems, i.e. systems in which the signal is not broadcast for receiving images from a single remote source

Abstract

A system and method for designing and using freeform reflectors to collect images of a wide angle field-of-view scene is provided. A freeform reflector may enable a wide angle field-of-view to be collected in an unwarped and unwrapped manner such that computer processing may be eliminated. Furthermore, the use of a freeform reflector allows for larger areas of an image sensor chip to be used, thereby providing higher resolution images. Because freeform reflectors may be configured to map a scene onto the image sensor chip in a scalar and mathematically correct manner, output images may be directly displayed from the image sensor chip. Wide angle field-of-view imaging systems, such as surveillance, alarm, and projector system, may utilize freeform reflectors as provided herein.

Description

    CROSS-REFERENCE TO RELATED APPLICATION
  • This applications claims priority to co-pending U.S. Provisional Patent Application Ser. No. 60/867,129, filed on Nov. 22, 2006, the entire contents of which are incorporated herein by reference.
  • BACKGROUND
  • In the 19th century, numerous still-picture cameras were created for taking panoramic still pictures through split-rotating mechanisms. Straight forward, yet tedious, procedures were used to first capture multiple still pictures of a surrounding scene and then stitch the pictures together as a seamless panoramic image. Even today, these types of cameras utilize mechanical moving parts and need tedious manual procedures to process multiple pictures in a panoramic view. Ultimately, these types of cameras are inherently awkward and cannot be used to generate field-of-view real-time video. The term “wide angle field-of-view” may vary for different applications. For example, for automobile rearview mirrors, a wide angle field-of-view may range between 30 degrees and 90 degrees. However, for other applications, such as surveillance systems, wide angle field-of-view may range between 100 degrees and 360 degrees. It should be understood that different applications may have generally understood fields-of-view that have industry accepted ranges, where a wide angle field-of-view is considered above a certain field-of-view value or within a range of values of fields-of-view.
  • With rapid advances in high-resolution charge-coupled device or complementary metal oxide semiconductor (CCD/CMOS) video sensor technology in recent years, much research went into developing video cameras and techniques that can simultaneously provide a 360° field-of-view. Resulting from the research and development of high-resolution CCD-CMOS video sensor technology, most 360° video cameras use conventional optics with fisheye lenses or omni-directional mirrors to obtain the 360° field-of-view. FIG. 1 shows a conventional wide angle video surveillance system 100 for capturing a scene up to a 360° field-of-view. Two alternative and different input elements are typically used for capturing a wide-angle field-of-view image of a scene, where the two different input elements include as omni-directional mirror capturing system 102 a and a fisheye lens image capturing system 102 b. These two different wide angle field-of-view image capturing systems 102 a and 102 b use different techniques for viewing a scene, but produce similar images. Each of the wide angle field-of-view image capturing systems 102 a and 102 b use CCD/CMOS image sensors 104 a and 104 b to collect video images of the scene. However, the wide angle field-of-view sensing system 102 a uses an omni-directional lens 106 to reflect images of the scene onto the CCD/CMOS image sensor, while the wide angle field-of-view imaging system 102 b uses a fisheye lens 108 to collect images of the scene and focus the images onto the CCD/CMOS image sensor 104 b. Although each of these two systems 102 a and 102 b use different image capturing techniques (i.e., reflective omni-directional mirror versus fisheye lens), a resulting image of each of the two techniques is captured onto a CCD/CMOS image sensor chip that is typically rectangular.
  • A typically CCD/CMOS image sensor chip is formed of a pixel array having a 4:3 ratio of width to height. As shown, a CCD/CMOS image sensor chip 110 is formed of pixels, and an image is imaged onto the CCD/CMOS image sensor chip 110 in an image area 112 that is circular in nature. An image 114 within the image area 112 is shown to be highly distorted, circular, and not suitable for direct viewing. In addition, the image area 112 utilizes only approximately 58% of the pixels available on the CCD/CMOS image sensor chip 110, which means that approximately 42% of the pixels on the CCD/CMOS image sensor chip 110 are not used. This low rate of pixel utilization significantly deteriorates resolution of a final video output due to an entire image being compressed into a small area of pixels.
  • FIG. 2 is an illustration of an exemplary image sensor chip 200 that shows an image area 202 imaged onto the image sensor chip 200 as a result of a traditional fisheye lens or parabolic mirror that generates circular images. Because the image is circular, it uses a relatively small portion of effective pixels on the CCD/CMOS sensor chip 200. Because only a small portion of the pixels are used, resolution is compromised during video processing to unwrap the circular image in the image area 202 and enlarge the image to be displayed on a video display or storage on a storage medium, such as a computer disc or memory.
  • Continuing with FIG. 1, to facilitate visualization of the image 114 that is imaged onto the CCD/CMOS image sensor chip 110, a computing system 116 is typically used to digitally resample and unwrap the image 114 displayed in a circular image in the image area 112 to produce rectangular image in which the abscissa represents azimuth and ordinate elevation. An unwrapped and unwarped image 118 is shown to be output from the computing system 116. The computing system 116 uses complex software to perform the unwarping, unwrapping, and other correction of the image. However, because the digital unwarping requires external computation resources, a wide angle field-of-view system is increased in cost, complexity, and size. Because of these and other issues, wide angle field-of-view video cameras have not yet found wide-spread applications.
  • Fisheye lenses and reflective omni-directional mirrors are often used for wide angle field-of-view imaging systems because these optical elements are rotationally symmetric. Although a fisheye lens is conceptually relatively easy to build, fisheye lenses generally include multiple lenses. These multiple lenses or multistage lenses may introduce string optical aberrations, which in turn, need additional lenses to correct. As a result, fisheye lenses and other rotationally symmetric optical elements generally require more complex optics for an entire system, are bulky in size and weight, and expensive in cost. In general, wide angle field-of-view systems that use rotationally symmetric optical elements use computing systems to alter a sensed image to be properly viewed by a person (e.g., without being wrapped or warped). In addition to the problem of low pixel utilization of a CCD/CMOS sensor chip, fisheye lenses also suffer severe axial unevenness of pixel resolution (i.e., a peripheral edge has much lower pixel resolution than in the center area). For most surveillance systems, the ability to detect targets in peripheral areas may be as or more important than in a central area of an image since better early warning procedures may be taken if a target can be detected at a far distance.
  • Omni-directional reflective mirrors are generally used with wide angle video cameras based on Catadioterics framework. One advantage of using a mirror instead of a lens is that a mirror may have a simple structure and much less color aberration than a lees. Hyperbolic mirrors and other omni-directional shaped mirrors are utilized with wide angle video cameras. However, as previously described, the use of such mirrors results in circular images on image sensor chips, thereby having reduced resolution due to using a portion of an image sensor chip and requiring a computing system to unwarp and unwrap an image having a wide angle field-of-view.
  • SUMMARY
  • To overcome the problem of wide angle field-of-view video imaging having distorted and low resolution images, and requiring computers to unwarp and unwrap images generated from the use of omni-directional reflector lenses and fisheye lenses, the principles of the present invention provide for using freeform reflectors to collect images of a wide angle field-of-view scene. A freeform reflector may enable a wide angle field-of-view to be collected in an unwarped and unwrapped manner such that computer processing may be eliminated. Furthermore, the use of a freeform reflector allows for larger areas of an image sensor chip to be used, thereby providing higher resolution images. Because freeform reflectors may be configured to map a scene onto the image sensor chip in a scalar and mathematically correct manner, the principles of the present invention may be used to directly output images from the image sensor chip to an output device, such as an electronic display.
  • One embodiment of a reflector may include a freeform surface being reflective and having a shape defined by a mapping to reflect and map an electromagnetic signal onto a sensor defined by pixels arranged in a geometric shape. The reflective surface may be an optical reflective surface and the electromagnetic signal may be an optical image having a field-of-view ranging up to 360°. The mapping may cause the electromagnetic signal to be redistributed when imaged on the sensor. The redistribution may minimize or eliminate distortion of a scene being imaged on the sensor. The redistribution of the electromagnetic signal may implement a certain mathematical function such that the electromagnetic signal mapped onto the sensor is a corrected signal that does not need to be corrected digitally or otherwise.
  • One embodiment of a method of collecting an electromagnetic signal may include acquiring an electromagnetic signal representing a predefined field-of-view of a scene. The acquired electromagnetic signal may be reflected from a freeform surface configured by a predefined mapping function to map the acquired electromagnetic signal onto pixels of the sensor. The reflected electromagnetic signal may be sensed as mapped from the predefined field-of-view of the scene by the pixels of the sensor. The reflected electromagnetic signal may be an optical signal. The pixels of the sensor may have a predetermined shape that is used in defining the predefined mapping function. In one embodiment, the field-of-view may a wide angle field-of view that ranges up to 360°. In reflecting the acquired electromagnetic signals, an optical processing of the electromagnetic signal is caused to occur prior to the reflected electromagnetic signal being sensed.
  • One embodiment for designing a freeform reflector may include defining a scene having a predetermined wide angle field-of-view. Sensor parameters of a sensor having a pixel configuration arranged to sense an electromagnetic signal may be determined. A mapping relationship between the scene and sensor parameters may be defined. A mapping function that causes the scene to be reflected onto the sensor pixels based on the defined mapping relationship between the scene and the sensor parameters may be generated, where the mapping function defines a freeform reflector. The freeform reflector may be used for wide angle field-of-view video generation without having to use digital signal processing to unwarp and unwrap an image of the wide angle field-of-view. The scene may be defined to have a field-of-view ranging up to 360°.
  • Another embodiment of designing a mirror may include defining a scene having a predetermined field-of-view. First and second subscenes of the scene may be defined. Sensor parameters of an optical sensor having a pixel array configuration may be determined. First and second sub-arrays of the optical pixel array may be defined, where the second sub-array may be mutually exclusive of the first sub-array. A first mapping relationship between the first sub-scene and first sub-array may be defined, and a second mapping relationship between the second sub-scene and second sub-array may be defined. A first mapping function may be generated that, when used to define a first portion of a freeform optical reflective surface, causes the first sub-scene to be reflected from the first portion of the freeform optical reflective surface onto the first sub-array. A second mapping function may be generated that, when used to define a second portion of a freeform optical reflective surface, causes the second sub-scene to be reflected from the second portion of the freeform optical reflective surface onto the second sub-array. Through the mapping functions, the mirror may be defined.
  • One embodiment of a video surveillance system may include an optical sensor configured to sense optical signals of a scene through a wide angle field-of-view and to generate electrical signals in response to sensing the optical signals. A freeform mirror may be configured to reflect the optical signals onto the optical sensor. An output device may be responsive to the electrical signals generated by the optical sensor to generate an output signal. In one embodiment, the output device is an electronic display. Alternatively, the output device may be a storage medium. The freeform mirror may be configured to be positioned in a location to reflect a panoramic view of a scene onto the optical sensor. The freeform mirror may be configured to be positioned in a location to reflect a hemispheric view of the scene onto the optical sensor. Alternatively, the freeform mirror may be configured to be positioned in a location to reflect a panoramic view of the scene onto the optical sensor. The electrical signal generated by the optical sensor may produce an unwarped image without image processing performed thereon. In one embodiment of an application, the wide angle field-of-view may be greater than approximately 120°.
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • Illustrative embodiments of the present invention are described in detail below with reference to the attached drawing figures, which are incorporated by reference herein and wherein:
  • FIG. 1 is an illustration of a conventional image capturing system for capturing wide angle field-of-view images of a scene using fisheye optics or an omni-directional mirror;
  • FIG. 2 is an illustration of a conventional image projected onto an image sensor chip;
  • FIG. 3 is an illustration of an exemplary image capturing system using a freeform reflector for capturing panoramic or hemispherical scene images;
  • FIG. 4 is an of an exemplary coordinate system showing scene images being collected on an image sensor chip by being reflected from a freeform reflector;
  • FIG. 5 is a flowchart of an exemplary process for designing a freeform reflector in accordance with the principles of the present invention;
  • FIGS. 6A-6D are illustration of an exemplary coordinate system of a panoramic imaging system showing an exemplary mapping for imaging onto an image sensor using a freeform reflector (FIG. 6C);
  • FIGS. 7A-7D are illustrations of an alternative embodiment of an exemplary coordinate system of panoramic imaging system showing another exemplary snapping for imaging onto an image sensor using a freeform reflector (FIG. 7C);
  • FIGS. 8A and 8B are illustration of another alternative embodiment of an exemplary coordinate system of a panoramic imaging system showing another exemplary mapping for imaging onto an image sensor using a freeform reflector;
  • FIGS. 9 and 9B are illustration of another exemplary embodiment of a panoramic imaging system that uses a freeform reflector;
  • FIG. 10 is an illustration of an exemplary embodiment of a hemispherical imaging system;
  • FIG. 11 is an illustration of another exemplary embodiment of a panoramic imaging system configured as a surveillance system;
  • FIG. 12 is an illustration of an exemplary hemispherical imaging system configured to operate as a surveillance system;
  • FIG. 13A is an illustration of an exemplary embodiment for using a freeform reflector for protecting usages on a circular screen;
  • FIG. 13B is a block diagram of an exemplary projection system that utilizes a freeform reflector;
  • FIG. 14 is a flowchart of an exemplary process for collecting an electromagnetic signal;
  • FIG. 15 is a flowchart of an exemplary process for designing a freeform reflector; and
  • FIG. 16 is a flowchart of an exemplary process for designing a mirror.
  • DETAILED DESCRIPTION OF THE DRAWINGS
  • FIG. 3 is an illustration of an exemplary image capturing system 300 using a freeform reflector 302 for capturing panoramic or hemispherical images of a scene. The image capturing system 300 uses a freeform reflector 302, which is a rotationally non-symmetric reflector (e.g., mirror), to map a wide angle field-of-view scene onto an image sensor system 304. The image capturing system 300 may be configured to substantially fully utilize resolution of the image sensor system 304. The image sensor system 304 may include an image sensor chip or image sensor, such as a CCD or CMOS image sensor chip, that is of any configuration. For example, the image sensor may be configured as a rectangle having a 4:3 ratio of width to height. The freeform reflector 302 may reflect a wide angle field-of-view onto the image sensor chip of the image sensor system 304 such that an image of the scene covers substantially the entire image sensor chip, thereby utilizing substantially full resolution of the image sensor chip. In addition, the freeform reflector 302 may have a geometric configuration that performs a mathematical function to map the scene onto the image sensor chip such that no digital processing is needed to correct, unwarp, or unwrap the image of the wide angle field-of-view scene, as shown in output image 306. The output image 306 includes an upper half 306 a and a lower half 306 b, where the upper half 306 a is an image of the scene between 0° and 180°, and the lower half 306 b is an image of the scene between 100° and 360°. Because no or minimal image processing is needed once the image is collected by the image sensor 304, the image capturing system 300 may be reduced in cost and size. In one embodiment, minimal image processing may include color correction or color enhancement, such as generating a visual image of an image in a non-visible portion of the electromagnetic spectrum (e.g., infrared or ultraviolet). In another embodiment, because the freeform reflector 302 is defined by a mathematical mapping function to map from one space to another, no image processing, either using software or hardware, to perform a mapping function is utilized.
  • FIG. 4 is an illustration of an exemplary coordinate system 400 showing a scene image point s being mapped on an image sensor chip 402 by being reflected from a freeform reflector 404. An arbitrary scene-to-image mapping relationship may be formed for any point s in a scene onto a corresponding point q on an image plane of the image sensor 402 via a reflecting point r on the freeform reflector 404. A ray 406 a may be incident to reflection point r and reflect a ray 406 b toward a focal point p, which causes the ray 406 b to be incident to the image sensor 402 at image point q. In accordance with the principle of the present invention, the reflector 404 may be designed by defining a mapping relationship between the scene and image sensor 402 to define a reflector surface profile. In doing so, a surface of the freeform reflector 404 may be defined by vector manipulation of the incident ray 406 a and reflected ray 406 b to compute a normal vector n. In other words, a composite of normal vectors across reflection points may be used to define a surface of the freeform reflector 404. In one embodiment, vector manipulation to generate a normal vector n includes performing a cross-product. In the case of a 360° video camera design, a scene-to-image mapping relationship can be defined so as to map an entire panoramic or hemispherical scene image onto an image plane of the image sensor 402 according to a prescribed order. A prescribed order defines locations at which points of a scene are mapped onto the image sensor 402. For example, a scene may start at angle 0°, which may correspond to a top left pixel on the image sensor 402. At 1° within the scene, a corresponding pixel location may be X pixels to the right of the top left pixel of the image sensor 402. As the scene progresses to 180°, a corresponding location on the image sensor may be at a top right pixel of the image sensor 402. By defining a mapping from the scene onto the image sensor 402, a determination of a reflector 404 may be made in a precise manner, thereby defining a relationship in mathematical terms that causes the scene to be reflected onto the image sensor 402 in a predetermined manner or order.
  • In general a theoretical closed-form solution for N(r) with an arbitrary defined mapping relationship is difficult. With a probability of 1, there are no theoretical closed-form solutions to a general design problem. Instead of operating in a universal reflector design theory platform, a computational method for designing a freeform reflector that is able to produce accurate reflector surface geometry to realize arbitrary and desirable imager-to-scene mapping relationships is described herein with regard to FIG. 5.
  • FIG. 5 is a flowchart of an exemplary process 500 for designing a freeform reflector in accordance with the principles of the present invention. The process 500 starts at step 502. At step 504, an image sensor model is defined. The image sensor model may be defined to be perspective, orthographic, or any other image model as understood in the art. At step 506, a mapping relationship between an imager (i.e., image sensor) and scene may be defined. At step 508, a starting point (x0, y0, z0) may be selected for performing a mapping calculation. The starring point (x0, y0, z0) is defined as a point on a reflector surface r. The point (x0, y0, z0) is typically determined by system design parameters, such as stand-off distance, mirror size, or any other system design parameter. One technique for defining surface shape of a reflector is to determine a surface normal at any point on the reflector surface.
  • At step 510, an imager-to-scene mapping may be used to determine a focal surface normal at current (x, y, z) point. In accordance with the principles of the present invention, the program may calculate the surface normal at the current surface point (x, y, z) based on a pre-defined imager-to-scene mapping and the reflection law of optics, where an incident angle equals a reflection angle on a reflection surface, such as a mirror. The normal vector is given by:
  • N ( r ) = q - r + s - r = q - r q - r + s - r s - r = [ n x n y n z ] T
  • where point q on the image plane corresponds to point s in the scene via reflecting point r on the reflector surface.
  • At step 512, a “pixel ray” (i.e., an electromagnetic signal, such as a light ray, that is mapped onto a pixel of a sensor) passing through a next pixel on the image sensor chip may be calculated. The location of a current surface point r(x0, y0, z0) and local surface normal N(x0, y0, z0) determines a local surface. The process 500 propagates from “current surface point” to a “next surface point” by solving for an intersection point of “current surface” with the pixel ray. To obtain the next surface point r(x0, y0, z0), the process 500 calculates the pixel ray based on projection geometry and the pixel location on the image sensor chip:
  • Q ( r ) = [ Qx Qx Qx - F ] T [ Qx Qx Qx - f ] T = [ q x q y q z ] T
  • At step 514, an intersection point (x, y, z) between the pixel ray and local surface may be calculated. In determining the intersection point (x, y, z), a linear equation may be computed to locate the coordinates of the next intersection point r:
  • Ar = B , where A = [ n x n y n z 1 / q x - 1 / q y 0 0 1 / q y - 1 / q z ] , B = [ n x R x + n y r y + n z R z Q x / q x - Q y / q y Q y / q y - Q z / q z ]
  • At step 516, a decision is made to determine whether all the pixels have been mapped with scene locations to define an accurate surface geometry of a reflector that implements the prescribed imager-to-scene mapping. If all the pixels have not been completed, then the process returns to step 510. Alternatively, if the pixels have all been mapped, such that the entire surface of the reflector has been defined, then the process continues at step 518, where a collection of the surface points (x, y, z) forms the reflector surface geometry. In one embodiment, the normal vectors from each of the surface points (x, y, z) that define the reflective surface are collected for use in manufacturing the reflective surface. The process SOD stops at step 520.
  • The process 500 is exemplary and may be used for determining surface points for any size or shape of freeform reflector. For example, one embodiment of a freeform reflector may include a mirror having miniature (e.g., nanometer) dimensions for use in cell phone cameras, for example. Another embodiment of a freeform reflector may have larger (e.g., centimeter) dimensions for use in a surveillance camera. It should be understood that the principles of the present invention provide for an infinite number of sizes and shapes use in different applications. In addition, the surface points defined by local surface normal vectors may be used to generate a mold to produce reflectors having a surface geometry the same or substantially the same as defined by the surface points. In one embodiment, a diamond turning or molding technique as understood in the art may be used in constructing a freeform reflector. Other techniques for making a freeform mirror may be utilized.
  • Although the description presented herein refer to the “imager-to-scene” map that can be implemented by the freeform reflector being from the scene field-of-view to an imager, the reflector and its design methodology can implement any general optical mapping relationship between the output of an upstream optical system, such as an objective lens module, and the downstream optical system, such as another lens module, for performing image aberration correction. The reflector can serve as a component in a complex system, such as a video system. Many modifications and variations will be apparent to one skilled in the art.
  • FIGS. 6A-6D are illustrations of an exemplary coordinate system 600 of a panoramic imaging system showing an exemplary mapping for imaging onto an image sensor using a freeform reflector. A 360° panoramic cylinder 602 representative of a scene may be mapped onto an image sensor 604. The mapping may be performed by using a perspective model and projection geometry. The projection geometry is based on reflecting a source point from a freeform mirror 606 at a reflection point r through the image sensor 604 at an image point q to a focal point p. In one embodiment, the imager-scene mapping relationship maps the entire 360° panoramic scene onto two rectangular bands on the imager 604, where each rectangular band has a 180° scene coverage (i.e., 180° around the panoramic cylinder 602).
  • Mapping and order of the scene on the panoramic cylinder 602 is shown between FIGS. 6A and 6B. A scene path 608 having a positive β=30° extending from a +x-axis to −x-axis along an upper edge of the panoramic cylinder 602 is reflected from the freeform mirror 606 and imaged onto the image sensor 604 along an image path 608′. As further shown, scene path 610 that extends from scene path 608 to a β+−40° is reflected from the freeform mirror 606 and projected onto the image sensor 604 along image path 610′. Scene path 612 that extends along a lower edge of the panoramic cylinder 602 having a β=−40° is reflected from freeform mirror 606 to image sensor 604 along image path 612′. Scene path 614, which exceeds from scene path 612 back to scene path 608 is reflected from freeform mirror 606 and projected onto the image sensor 604 along image path 614′. The projections of the scene path 608-614 onto the image sensor 604 having image paths 608′-614′ show the mapping and order of the different areas of the panoramic cylinder 602 that are imaged by the image sensor 604.
  • The freeform reflector 606 may have its surface defined by mapping source points along the panoramic cylinder 602 onto the image sensor 604, as previously described. That is, the source points are reflected from the freeform reflector 606 at a reflection point r to a mapped image point q and a normal vector n is computed by a cross-product calculation of the source s to reflection point r vector and reflection point r to image point q vector, as understood in the art. The normal points are used to define the surface of the freeform reflector 606. It should be understood that the surface of the freeform reflector 606 is a function of how a designer desires to map the scene onto the image sensor.
  • FIG. 6C is an illustration of an exemplary freeform reflector 616. The complex shape of the reflective surface of freeform reflector 616 is a function of normal vectors resulting from the mapping of the panoramic cylinder 602 onto the image sensor 604 of FIGS. 6A and 6B. As shown, the freeform reflector 616 is curvilinear, which may include linear and nonlinear regions. The image sensor 604 that receives the mapped images of the scene from the freeform reflector 616 may convert the images into electrical signals that may be displayed directly or indirectly by an electronic display. Such a display of an image of a scene is shown in FIG. 3, where the scene is mapped between 0°and 180° on a top portion 306 a and 180°to 360° on a lower portion 306 b.
  • FIG. 6D is an illustration of an exemplary imaging system 618 that includes the freeform reflector 616 and imaging sensor system 620. The imaging system 618 captures an image of a scene represented by a panoramic cylinder 622. In this case, the scene includes the words, “Front” and “Back” with smiley faces before each word. In reflecting the scene from the panoramic cylinder 622 to the image sensor system 620, the freeform reflector 616 causes the word “Front” to be mapped upside down onto a top portion of an image sensor and word “Back” to be mapped right side up onto a bottom portion of an image sensor, such that the words as imaged onto the image sensor may be displayed on an electronic display 624, a top portion 624 a and bottom portion 624 b displaying the words “Front” and “Back,” respectively.
  • FIGS. 7A-7D are illustrations of an alternative embodiment of an exemplary coordinate system of a panoramic imaging system showing another exemplary mapping for imaging onto an Image sensor using a freeform reflector (FIG. 7C). A panoramic cylinder 702 and image sensor 704 are shown to be the same or similar of those of FIG. 6A. In addition, scene paths 708-712 are the same or similar to those of FIG. 6A. The difference from FIGS. 6A and 6B is the mapping from the scene paths 708-714 onto the image sensor 704 with image paths 708′-714′. As shown, scene path 708 is mapped in this embodiment onto the image sensor 794 on a positive right edge of the image sensor 704 and ordered from a +x-axis side to a −x-axis side (as compared to scene path 608 being mapped onto the image sensor 604 at or near the x-axis). The other image paths 710′, 712′, and 714′ are also mapped to different positions on the image sensor system 704 and ordered in different directions. Resulting from the different mapping and ordering from the panoramic cylinder 702 onto the image sensor 704 is a freeform reflector 706 being different from the freeform reflector 616 shown in FIG. 6C.
  • FIG. 7C is an illustration of an exemplary freeform reflector 716 that is defined by normal vectors resulting from a mapping of the scene on the panoramic cylinder 702 onto the image sensor system 704 of FIGS. 7A and 7B. The freeform reflector 716 provides a different mapping and order for the scene being imaged onto the image sensor 704 than freeform reflector 616 of FIG. 6C.
  • FIG. 7D is an exemplary scene represented by panoramic cylinder 718 being mapped by the freeform reflector 716 of FIG. 7C onto an image sensor system 720. The scene has the words “Front” and “Back” printed thereon with two smiley faces prior to each of the words. A resulting display 722 includes a top portion 724 a and bottom portion 724 b that has the words “Front” and “Back” written from left to right and both in a right-side up manner with the smiley faces being to the left of the words. By comparing FIGS. 6D and 7D, it should be recognized that the different shapes of the freeform reflective surfaces 616 and 716 cause the images of the scenes mapped and projected onto the image sensors 604 and 704, respectively, to be different. The ability to have mapping flexibility through use of the principles of the present invention to be used in a wide variety of applications.
  • FIGS. 8A and 8B are illustrations of another alternative embodiment of an exemplary coordinate system of a panoramic imaging system 800 showing another exemplary mapping for imaging onto an image sensor 804 using a freeform reflector 806. A scene may be mapped from a panoramic cylinder 802 to the image sensor 804 to define the freeform reflector 806. In mapping the scent to the image sensor 804, scene paths 808, 810, 812, and 814 may be mapped to image paths 808′, 810′, 812′, and 814′, as described with regard to FIGS. 6A-6B and 7A-7B. In this embodiment, the paths and ordering are different from those shown in FIGS. 6B and 7B, thereby resulting in a different freeform reflector shape.
  • FIGS. 9A and 9B are illustrations of another exemplary embodiment of a panoramic imaging system that uses a freeform reflector. The panoramic imaging system 900 may include a freeform reflector 902 and image sensor 904. The image sensor 904 may be a CCD, CMOS, infrared (IR), or any other electromagnetic wavelength sensor. In one embodiment, sensor primary optics 906, which may include a first lens 906 a and second lens 906 b may be disposed in optical relation with the freeform reflector to cause electromagnetic signals (e.g., light beams) to be illuminated on the image sensor 904. The sensor primary optics 906 may result in a columnized and aberration corrected light being projected from the freeform reflector 902 onto the image sensor 904. It should be understood that a wide range of optical designs may be utilized in accordance with the principles of the present invention.
  • However, the use of primary optics does not affect the ability for the freeform reflector 902 to form an image of a scene on the image sensor 904 that substantially utilizes all or most of the pixels of the image sensor 904, thereby providing good resolution and substantially eliminating the need for post processing from the image sensor 904 to produce a scene image 908 having one or more portions of the scene being unwrapped and unwarped such that a viewer may readily understand the scene captured by the image sensor. As shown, the scene image includes a first portion 908 a and a second portion 908 b, where the first portion ranges from 0° to 180° and the second portion 908 b ranges from 180° to 360° of the hemispherical scene.
  • Because the freeform mirror 902 enables the image sensing system 900 to operate without any moving pasts, the imaging sensor system 900 may operate to capture both still and video images. In other words, since no moving parts are needed for the image sensing system 900, there is no need to take multiple images in different directions and then stitch them together using computer software. Furthermore, as shown in the scene image 908, image resolution is substantially evenly distributed over the entire 360° of the scene image 908 because the scene image 908 is mapped onto a large portion of the image sensor 904. Because the image resolution is evenly distributed throughout the scene image 908, a surveillance system using such a configuration may facilitate advanced image processing algorithms, such as target detection, for monitoring images within the scene and intelligent transportation systems. The advanced image processing algorithms may use the captured image without or with minimal correction (e.g., color correction). The advanced image processing algorithms may be used to perform a variety of different functions depending on the application, such as determining whether an object or person enters or moves within an image for a surveillance application. Furthermore, because no resampling on an original image on the image sensor 904 is performed, artifacts are not produced by post-processing software and image quality is preserved.
  • FIG. 9B shows the imaging system 900 with an image sensor system 910 having a housing 912 that the freeform reflector 902, image sensor 904, and sensor primary optics 906 of FIG. 9A may be encapsulated. An electronic display may be connected to or in communication with the image sensor system 910 for display of scenes captured by the image sensor 904.
  • FIG. 10 is illustration of an exemplary embodiment of a hemispherical imaging system. The image sensor system 1000 may be configured for acquiring a 360° hemispherical scene, where the entire 360° hemispherical scene is mapped onto a predetermined shaped sensor, such as a rectangular or otherwise configured image sensor. Rectangular shaped image sensors are common and typically used for imaging applications. However, other predetermined shaped image sensors may be utilized in accordance with the principles of the present invention. As previously described, the mapping of the 360° hemispherical scene may be the same or similar to mapping of the panoramic scene, where the scene is mapped into two 180 ° bands to form a full screen image. An image system 1002 that receives images from a freeform reflector 1004 may output an unwarped and unwrapped data stream of undistorted hemispherical video or photographs. Using such a configuration, no host computer or unwarping software is needed. However, in the case of the video being used for motion detection or other surveillance functionality, a computing system may be utilized for such purposes. However, the original video captured by the image processing system 1000 may be viewed with substantially no distortion or image processing. In a surveillance or motion detection system, a box 1006 or other graphical representation (e.g., arrow or other symbol or indicia) may be displayed on an output image 1008 to identify motion in the scene. In addition, words, such as “Motion Detected Person,” may be displayed on the output image 1008.
  • FIG. 11 is an illustration of another exemplary embodiment of a panoramic imaging system 1100 configured as a surveillance system. The image capturing system 1100 may be configured as a “smart alarm” device for using in homes, offices, warehouses, and cars, for example. As described with regard to previous figures, a freeform reflector (not shown) may be used in an image sensor system 1102 to capture images of a scene in a panoramic view about the image sensor system 1102. The scene to be captured may be defined by the designer of the image sensor system 1102 to define the freeform mirror to be utilized for mapping the panoramic scene onto an image sensor chip within the image sensor system 1102. A digital signal processor 1104 may receive inputs from other alarms (e.g., door alarms, fire alarms, etc.) and process image signals 1106 that are generated by the image sensor system 1102 of the scene. The digital signal processor 1104 may utilize motion and intrusion detection algorithms, as understood in the art, to facilitate automatic detection and tracing of any suspicious moving object or stationary object in the areas that no one is supposed to enter.
  • If any suspicious activity is detected by the image capturing system 1100 that raises a security concern, an alert message or notification, which may be communicated in data packets 1108, may be communicated using a radio 1110, such, as a general packet radio service (GPRS), to communicate the data packets 1108 via a wireless antenna 1112 to a communications network, such as a cellular communications network. The alert message or messages may include one or more photographs or videos captured during a detected suspicious activity text message, telephone call, or any other type of message. The alert message may be communicated to an owner's telephone number, email address, police, neighbors, or any other party that the owner desires to notify of a suspicious activity occurring within view of the image sensor system 1102. In one embodiment, a log of alert messages, photographs, videos, or other detection information may be created in a database or other data repository locally or on a network such as the Internet.
  • Although shown as a wireless system, it should be understood that the principles of the present invention may also use wired communications systems, such as Ethernet or other wide area network communications systems. In one embodiment, a receiver of the message from the imaging system 1100 may utilize a mobile telephone, handheld wireless device, or any other computing system to control electronics or objects within the image sensing system 1100 to obtain additional details, images, or other information about the scene being captured. In one embodiment, a speaker and microphone (not shown) may be in communication with the digital signal processor 1104 to enable an owner or monitor to receive a message and communicate with whoever is near the device using a telephone, such as a mobile telephone or other computing system. In one embodiment, the GPRS 1110 may communicate voice over IP data packets and be in communication with the local wireless access points for communication over the internet or public switched telephone network. Because of the 360° imaging of the image sensor system 1102, false alarms may be reduced by enabling the receiver of a photograph to accurately determine whether a person or other object has entered into the field of view. It should be understood that a wide range of embodiments, communications, possibilities, and alerts may be provided through use of the principles of the present invention.
  • FIG. 12 is an illustration of an exemplary hemispherical imaging system configured to operate as a surveillance system. The imaging system 1200 may include three components, a 360° imaging sensor 1202, high speed pan/tilt/zoom (PTZ) camera 1204, and embedded DSP hardware 1206 for performing image processing, target detection, target tracking, video analysis, alarm processing, network Internet Protocol (IP) server, and other functionality. The imaging sensor 1202 may be located within or extending from a dome 1208. The imaging sensor 1202 may be include a freeform reflector (not shown) that is configured to view a hemispherical scene and image the scene onto an image sensor that is rectangular in shape in a similar manner as described hereinabove such that no image processing may be needed to unwrap or unwarp the image, thereby enabling processing to be dedicated to monitoring movement and other surveillance functionality. By utilizing a 360° image sensor 1202, the full 360° scene may be continuously monitored, thereby having no “blind spots” due to a rotating camera, which is typically used in conventional domed imaging systems.
  • In one embodiment, the high speed PTZ camera 1204 may include pan/tilt/zoom capabilities to enable agile target search and track operation that does not require human operator intervention. The use of the high speed PTZ camera 1204 enables a second video stream to be viewed by a person to more closely monitor images within the 360° image scene, which is continuously monitored through use of the imaging sensor 1202. The image sensing system 1200 may also use a wireless communication radio to communicate data packets or otherwise with a communications system. Furthermore, the DSP 1206 may generate video images using any type of video protocol, including NTSC, PAL, MPEG3 or any other video format as understood in the art. Furthermore, the DSP 1206 may generate alerts and other messages in response to detecting movement or objects within the scene.
  • FIG. 13A is an illustration of an exemplary embodiment of a projection system 1300 for using a freeform reflector 1306 for protecting images on a circular screen 1302. It should be understood that other shaped screens may be utilized in accordance with the principles of the present invention. The circular screen 1302 may receive images projected by a slide or video projector 1304 with perspective or orthographic optics onto the freeform reflector 1306. It should be understood that other optics may be utilized in the projector 1304. The circular screen 1302 may be configured to display images on the inside of the circular screen where the circular screen is large enough for people to view the images displayed on the circular screen 1302 or be configured to enable the images projected onto the circular screen 1302 on the outside of the circular screen, as in a rear view projection screen.
  • FIG. 13B is a block diagram of an exemplary projection system 1308 that utilizes a freeform reflector 1310. The projection system 1308 may include an upstream optical module 1312 and a downstream optical module 1314 with respect to the freeform reflector 1310. The upstream and downstream optical modules 1312 may include optical components, such as lenses, mirrors, or other optical components that are used in the projection system 1308. The upstream optical module 1312 may project electromagnetic signals 1316 a onto the freeform reflector 1310, which, in turn, reflects the electromagnetic signals 1316 a as electromagnetic signals 1316 b to the downstream optical module 1314. Opposite to an image collection system, projection system 1308 uses the freeform reflector 1310 to map from a source of the electromagnetic signals 1316 a at the upstream optical module 1312 to a wide angle field-of-view image scene (e.g., 360 degree projection screen) downstream of the downstream optical module 1314. The freeform reflector 1310 may define a mathematical function such that the image projected onto the wide angle field-of-view image scene is mapped from a rectangular coordinate system to a cylinder coordinate system and corrected for the coordinate system transformation.
  • FIG. 14 is a flowchart of an exemplary process 1400 for collecting an electromagnetic signal. The process 1400 starts at step 1402, where an electromagnetic signal representing a predefined field-of-view may be acquired. In one embodiment, the predefined field-of-view is a wide angle field-of-view. As previously described, a wide angle field-of-view may vary by application (e.g., surveillance, rearview mirror, etc.). The electromagnetic signal may be an optical signal, infrared signal, or any other signal at a different wavelength. At step 1404, the acquired electromagnetic signal may be reflected from a freeform surface configured by a predefined mapping function to map the acquired electromagnetic signal onto pixels of a sensor. The predefined mapping function may be one that maps a scene in a certain field-of-view (e.g., 360° field-of-view) having angular elevation angles between two angles, such as β=30° to β=−4°, for example. At step 1406, the reflected electromagnetic signal of the scene as mapped from the predefined field-of-view to the sensor may be sensed. When sending the reflected electromagnetic signal, no computation may be needed to post process the sensed signals as a mathematical function integrated into the preformed surface may cause the electromagnetic signal to be in a form that a human can interpret without computer correction.
  • FIG. 15 is a flowchart of an exemplary process 1500 for designing a freeform reflector. The process 1500 may start at step 1502, where a scene having a predetermined wide angle field-of-view may be defined. At step 1504, sensor parameters of a sensor having a pixel configuration arranged to sense an electromagnetic signal may be determined. In determining the sensor parameters, length and width of a sensor or of a number of pixels of the sensor may be determined such that a mapping relationship between the scene and sensor parameters may be defined at step 1506. The mapping relationship may be determined by the scene parameters, such as field of view, angle and height and size of a sensor, thereby enabling a designer to generate a mapping function that causes the scene to be reflected onto the sensor pixels based on the defined mapping relationship between the scene and sensor parameters at step 1508. The mapping function as generated may define a freeform reflector that may be used in mapping the scene onto the sensor.
  • FIG. 16 is a flowchart of an exemplary process 1600 for designing a mirror. The process starts at step 1602, where a scene having a predetermined field-of-view may be defined. At step 1604, a first sub-scene of the scene may be defined. At step 1606, a second sub-scene of the scene may be defined. At step 1608, sensor parameters of an optical sensor having a pixel array configuration may be determined. At step 1610, a first sub-array of the optical pixel array may be defined. At step 1612, a second sub-array of the optical pixel array of the pixel array may be defined. The first and second sub-arrays of the optical pixel array may be an upper half and a lower half of the pixel array, such that the first and second sub-scenes of the scene may be imaged onto the two different sub-arrays, thereby eliminating the need to correct for images displayed on the pixel array.
  • At step 1614, a first mapping relationship between the first sub-scene in the first sub-array may be defined. At step 1616, a second mapping relationship between the second sub-scene and the second sub-array may be defined. At step 1618, a first mapping function that, when used to define a first portion of a freeform optical reflective surface, causes the first sub-scene to be reflected from the first portion of the freeform optical reflective surface onto the first sub-array may be generated. At step 1620, a second mapping function that, when used to define a second portion of the freeform optical reflective surface, causes the second sub-scene to be reflected from the second portion of the freeform optical reflective surface onto the second sub-array may be generated. By generating the first and second mapping functions, the entire scene may be imaged onto different portions of the optical sensor so that no image correction is needed thereafter.
  • While much of the previous description focuses on imaging formation optics, it should be understood that the principles of the present invention provide for reversing the direction of the optical path such that substantially the same reflector and design methodology can be applied to design and implement illumination and/or image display systems. Many modifications and variations will be apparent to those of ordinary skill in the art.
  • The above description has been presented for purposes of illustration and description, and is not intended to be exhaustive or limited to the illustrative embodiments in the form disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art.

Claims (17)

1-5. (canceled)
6. A method for collecting an electromagnetic signal, said method comprising:
acquiring an electromagnetic signal representing a predefined field-of-view of a scene;
reflecting the acquired electromagnetic signal from a freeform swiace configured by a predefined mapping function to map the acquired electromagnetic signal onto pixels of a sensor; and
sensing the reflected electromagnetic signal as mapped from the predefined field-of-view of the scene by the pixels of the sensor.
7. The method according to claim 6, wherein reflecting the acquired electromagnetic signal includes reflecting an optical signal.
8. The method according to claim 6, wherein sensing the reflected electromagnetic signal includes sensing the electromagnetic signal using a pixel configuration having a predetermined shape that is used in defining the predefined mapping function.
9. The method according to claim 6, wherein acquiring an electromagnetic signal of a scene from a predefined field-of-view is from a field-of-view ranging up to 360 degrees.
10. The method according to claim 6, wherein reflecting causes an optical processing the electromagnetic signal to occur prior to sensing the reflected electromagnetic signal.
11. A method for designing a freeform reflector, said method comprising:
defining a scene having a predetermined wide angle field-of-view;
determining sensor parameters of a sensor having a pixel configuration arranged to sense an electromagnetic signal;
defining a mapping relationship between the scene and sensor parameters; and
generating a mapping function that causes the scene to be reflected onto the sensor pixels based on the defined mapping relationship between the scene and sensor parameters, the mapping function defining a freeform reflector.
12. The method according to claim 11, wherein defining the mapping relationship between the scene and sensor parameters includes defining the mapping relationship such that at least 70 percent of the pixels of the sensor receives the electromagnetic signal when reflected from the freeform reflector.
13. The method according to claim 11, further comprising producing a freeform reflector surface by forming a plurality of individual pixel dispersed projections, wherein each pixel projection is individually determined to correspond to a previously calculated scene map and wherein a collection of all projection data points form geometry of the freeform reflector surface.
14. The method according to claim 11, wherein generating the mapping function includes:
establishing a first point in the scene to map onto a first predetermined pixel of the sensor;
forming a first vector from the established point to a reflection location;
forming a second vector from the reflection location to the predetermined pixel of the sensor;
determining a local surface normal vector extending the reflection location, the local surface normal vector defining a surface of the freeform reflector; and
repeating establishing, forming, forming and determining for multiple points in the scene to map onto other predetermined pixels of the sensor.
15. The method according to claim 11, wherein generating a mapping function includes generating a mapping function that lUlwarp5 arrangement of the electromagnetic signal in response to reflecting from the freeform reflector to generate a substantially undistorted image.
16. A method of designing a mirror, said method comprising:
defining a scene having a predetermined field-of-view;
defining a first sub-scene of the scene;
defining a second sub-scene of the scene;
determining sensor parameters of an optical sensor having a pixel array configuration, the pixel array configured to sense optical signals;
defining a first sub-array of the optical pixel array;
defining a second sub-array of the optical pixel array, the second sub-array being mutually exclusive of the first sub-array;
defining a first mapping relationship between the first sub-scene and the first sub-array;
defining a second mapping relationship between the second sub-scene and the second sub-array;
generating a first mapping function that, when used to define a first portion of a freeform optical reflective surface, causes the first sub-scene to be reflected from the first portion of the freeform optical reflective surface onto the first sub-array; and
generating a second mapping function that, when used to define a second portion of the freeform optical reflective surface, causes the second sub-scene to be reflected from the second portion of the freeform optical reflective surface onto the second sub-array.
17. The method according to claim 16, wherein defining the first and second sub-array includes defining each of the first and second sub-array with different definitions of areas of the optical pixel array.
18. The method according to claim 16, wherein generating the first mapping function includes generating an order of mapping corresponding the first sub-scene with the first sub-array.
19. The method according to claim 16, wherein generating the first and second mapping functions includes generating first and second mapping functions for sensing panoramic views of the first and second sub-scenes using the first and second sub-arrays, respectively.
20. The method according to claim 16, wherein generating the first and second mapping functions includes generating first and second mapping functions for sensing hemispherical views of the first and second sub-scenes using the first and second sub-arrays, respectively.
21-25. (canceled)
US13/925,728 2006-11-23 2013-06-24 Wide Field of View Reflector and Method of Designing and Making Same Abandoned US20140009571A1 (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
US86712906P true 2006-11-23 2006-11-23
US11/984,702 US8471892B2 (en) 2006-11-23 2007-11-21 Wide field-of-view reflector and method of designing and making same
US13/925,728 US20140009571A1 (en) 2006-11-23 2013-06-24 Wide Field of View Reflector and Method of Designing and Making Same

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
US13/925,728 US20140009571A1 (en) 2006-11-23 2013-06-24 Wide Field of View Reflector and Method of Designing and Making Same

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
US11/984,702 Division US8471892B2 (en) 2006-11-23 2007-11-21 Wide field-of-view reflector and method of designing and making same

Publications (1)

Publication Number Publication Date
US20140009571A1 true US20140009571A1 (en) 2014-01-09

Family

ID=39469369

Family Applications (2)

Application Number Title Priority Date Filing Date
US11/984,702 Expired - Fee Related US8471892B2 (en) 2006-11-23 2007-11-21 Wide field-of-view reflector and method of designing and making same
US13/925,728 Abandoned US20140009571A1 (en) 2006-11-23 2013-06-24 Wide Field of View Reflector and Method of Designing and Making Same

Family Applications Before (1)

Application Number Title Priority Date Filing Date
US11/984,702 Expired - Fee Related US8471892B2 (en) 2006-11-23 2007-11-21 Wide field-of-view reflector and method of designing and making same

Country Status (2)

Country Link
US (2) US8471892B2 (en)
WO (1) WO2008066742A1 (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8798451B1 (en) * 2013-06-15 2014-08-05 Gyeongil Kweon Methods of obtaining panoramic images using rotationally symmetric wide-angle lenses and devices thereof
WO2016065130A1 (en) * 2014-10-24 2016-04-28 Magik Eye Inc. Distance sensor
WO2018106670A1 (en) * 2016-12-07 2018-06-14 Magik Eye Inc. Distance sensor projecting parallel patterns
US10228243B2 (en) 2015-05-10 2019-03-12 Magik Eye Inc. Distance sensor with parallel projection beams
US10337860B2 (en) 2017-12-05 2019-07-02 Magik Eye Inc. Distance sensor including adjustable focus imaging sensor

Families Citing this family (36)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
IL177987D0 (en) * 2006-09-10 2007-07-04 Wave Group Ltd Vision ball - a self contained compact & portable omni - directional monitoring and automatic alarm video device
US20080200224A1 (en) 2007-02-20 2008-08-21 Gametank Inc. Instrument Game System and Method
US8907193B2 (en) 2007-02-20 2014-12-09 Ubisoft Entertainment Instrument game system and method
US20080239109A1 (en) * 2007-03-31 2008-10-02 David John Jennings Method and apparatus for correcting image distortion
US9092951B2 (en) * 2008-10-01 2015-07-28 Ncr Corporation Surveillance camera assembly for a checkout system
EP2187622A1 (en) 2008-11-12 2010-05-19 Axis AB Camera assembly
WO2010059994A2 (en) * 2008-11-21 2010-05-27 Poptank Studios, Inc. Interactive guitar game designed for learning to play the guitar
KR20100062575A (en) * 2008-12-02 2010-06-10 삼성테크윈 주식회사 Method to control monitoring camera and control apparatus using the same
JP5453953B2 (en) * 2009-06-24 2014-03-26 ソニー株式会社 Moving mechanism control apparatus, the movable mechanism unit control method, program
JP2011009929A (en) * 2009-06-24 2011-01-13 Sony Corp Movable-mechanical-section controlling device, method of controlling movable mechanical section, and program
CN102577347B (en) * 2009-06-29 2015-09-23 博世安防系统有限公司 Spherical camera surveillance system and method for comprehensive situational awareness and intelligent automatic inspection of
MY147904A (en) * 2009-07-29 2013-01-31 Mimos Bhd Optical device and method therefor
US20110134245A1 (en) * 2009-12-07 2011-06-09 Irvine Sensors Corporation Compact intelligent surveillance system comprising intent recognition
DE102010008688A1 (en) * 2010-02-19 2011-08-25 Z-Laser Optoelektronik GmbH, 79100 Optical sensor system for detecting e.g. hand in detection area that contains circular saw blade of desk circular saw, has free molded surfaces formed such that measuring points illuminated by light beams are imaged in sensing element
KR20110099845A (en) * 2010-03-03 2011-09-09 삼성전자주식회사 Apparatus and method for omnidirectional caller detection in video call system
JP5742210B2 (en) * 2010-12-22 2015-07-01 ソニー株式会社 Imaging device, an image processing apparatus, image processing method, and program
DE102013211342A1 (en) * 2013-06-18 2014-12-18 Siemens Aktiengesellschaft Photo-based 3D surface inspection system
US9863767B2 (en) * 2013-06-27 2018-01-09 Panasonic Intellectual Property Corporation Of America Motion sensor device having plurality of light sources
JP6270813B2 (en) * 2013-07-01 2018-01-31 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカPanasonic Intellectual Property Corporation of America Motion sensor apparatus having a plurality of light sources
US20150009327A1 (en) * 2013-07-02 2015-01-08 Verizon Patent And Licensing Inc. Image capture device for moving vehicles
JP5866499B2 (en) * 2014-02-24 2016-02-17 パナソニックIpマネジメント株式会社 Control method of a security camera system and a monitoring camera system
CN104898274B (en) * 2014-03-05 2018-01-30 清华大学 Applied to a free surface of the optical system design method
US20150373322A1 (en) * 2014-06-20 2015-12-24 Qualcomm Incorporated Automatic multiple depth cameras synchronization using time sharing
US9883101B1 (en) * 2014-07-23 2018-01-30 Hoyos Integrity Corporation Providing a real-time via a wireless communication channel associated with a panoramic video capture device
DE102014214750B3 (en) * 2014-07-28 2015-06-11 Reimar Lenz Imaging system with rapidly vibrating global shutter CMOS sensor
US9866734B2 (en) * 2014-08-26 2018-01-09 Dolby Laboratories Licensing Corporation Scene-change detection using video stream pairs
US9810887B1 (en) * 2014-09-05 2017-11-07 Hoyos Integrity Corporation Overhang enclosure of a panoramic optical device to eliminate double reflection
TWI500904B (en) * 2014-09-18 2015-09-21 Vivotek Inc Stereo camera and automatic range finding method for measuring a distance between stereo camera and reference plane
CN105635635A (en) 2014-11-19 2016-06-01 杜比实验室特许公司 Adjustment for space consistency in video conference system
US10003813B2 (en) * 2015-06-25 2018-06-19 Samsung Electronics Co., Ltd. Method and system for decoding by enabling optimal picture buffer management
US20170048455A1 (en) * 2015-08-11 2017-02-16 Douglas Arnold Brand Optic for enabling instantaneously 360° Degree Panoramic Video of the Surroundings
US9888174B2 (en) * 2015-10-15 2018-02-06 Microsoft Technology Licensing, Llc Omnidirectional camera with movement detection
US10277858B2 (en) 2015-10-29 2019-04-30 Microsoft Technology Licensing, Llc Tracking object of interest in an omnidirectional video
US20170148152A1 (en) * 2015-11-25 2017-05-25 General Electric Company Systems and Methods for Monitoring Component Strain
US9547883B1 (en) 2016-08-19 2017-01-17 Intelligent Security Systems Corporation Systems and methods for dewarping images
US9609197B1 (en) 2016-08-19 2017-03-28 Intelligent Security Systems Corporation Systems and methods for dewarping images

Citations (23)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5760826A (en) * 1996-05-10 1998-06-02 The Trustees Of Columbia University Omnidirectional imaging apparatus
US5790181A (en) * 1993-08-25 1998-08-04 Australian National University Panoramic surveillance system
WO2000023842A1 (en) * 1998-10-21 2000-04-27 Aleadesign S.R.L. Anamorphic technique for filming and projecting images of 360 degree
US6130783A (en) * 1998-05-14 2000-10-10 Sharp Kabushiki Kaisha Omnidirectional visual sensor having a plurality of mirrors with surfaces of revolution
US6226035B1 (en) * 1998-03-04 2001-05-01 Cyclo Vision Technologies, Inc. Adjustable imaging system with wide angle capability
US20010010555A1 (en) * 1996-06-24 2001-08-02 Edward Driscoll Jr Panoramic camera
US20010015751A1 (en) * 1998-06-16 2001-08-23 Genex Technologies, Inc. Method and apparatus for omnidirectional imaging
US6333826B1 (en) * 1997-04-16 2001-12-25 Jeffrey R. Charles Omniramic optical system having central coverage means which is associated with a camera, projector, or similar article
US6332687B1 (en) * 2000-11-02 2001-12-25 The United States Of America As Represented By The Secretary Of The Air Force Plunger controlled, near-parabolic optical membrane mirror
CA2324802A1 (en) * 2000-10-31 2002-04-30 Vr Interactive International, Inc. Panoramic imaging apparatus and method
US6412961B1 (en) * 2000-05-30 2002-07-02 Robert Andrew Hicks Rectifying mirror
US20020159166A1 (en) * 2001-02-24 2002-10-31 Herman Herman Panoramic mirror and system for producing enhanced panoramic images
EP1267313A2 (en) * 2001-06-12 2002-12-18 Sharp Corporation Image surveillance apparatus, image surveillance method, and image surveillance processing program
FR2830128A1 (en) * 2001-09-26 2003-03-28 Egg Solution Sa Photoelectric sensor with sensing elements of pixels laid out in concentric circles, and device comprising such sensor for acquisition of panoramic images
US20030081952A1 (en) * 2001-06-19 2003-05-01 Geng Z. Jason Method and apparatus for omnidirectional three dimensional imaging
JP2004170607A (en) * 2002-11-19 2004-06-17 Matsushita Electric Ind Co Ltd Mirror element and its manufacturing method
US20050086032A1 (en) * 2003-07-28 2005-04-21 Light Prescriptions Innovators, Llc Three-dimensional simultaneous multiple-surface method and free-form illumination-optics designed therefrom
US20050094884A1 (en) * 2003-09-22 2005-05-05 Kddi Corporation Adaptable shape image encoding apparatus and decoding apparatus
US6897883B1 (en) * 2000-05-23 2005-05-24 Sharp Kabushiki Kaisha Omniazimuthal visual system
WO2006047488A2 (en) * 2004-10-25 2006-05-04 The Trustees Of Columbia University In The City Of New York Systems and methods for displaying three-dimensional images
US7123777B2 (en) * 2001-09-27 2006-10-17 Eyesee360, Inc. System and method for panoramic imaging
US7227705B2 (en) * 2004-10-01 2007-06-05 Diehl Bgt Defence Gmbh & Co., Kg Wide-angle optical system
US7952606B2 (en) * 2005-05-26 2011-05-31 Korea Advanced Institute Of Science And Technology Apparatus for providing omnidirectional stereo image with single camera

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5774569A (en) * 1994-07-25 1998-06-30 Waldenmaier; H. Eugene W. Surveillance system
US5990934A (en) * 1995-04-28 1999-11-23 Lucent Technologies, Inc. Method and system for panoramic viewing
US6118474A (en) * 1996-05-10 2000-09-12 The Trustees Of Columbia University In The City Of New York Omnidirectional imaging apparatus
US6611282B1 (en) * 1999-01-04 2003-08-26 Remote Reality Super wide-angle panoramic imaging apparatus
AU4328501A (en) * 2000-03-02 2001-09-12 Donnelly Corp Video mirror systems incorporating an accessory module
US7377671B2 (en) * 2003-02-04 2008-05-27 Light Prescriptions Innovators, Llc Etendue-squeezing illumination optics
US7411729B2 (en) * 2004-08-12 2008-08-12 Olympus Corporation Optical filter, method of manufacturing optical filter, optical system, and imaging apparatus

Patent Citations (25)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5790181A (en) * 1993-08-25 1998-08-04 Australian National University Panoramic surveillance system
US5760826A (en) * 1996-05-10 1998-06-02 The Trustees Of Columbia University Omnidirectional imaging apparatus
US20010010555A1 (en) * 1996-06-24 2001-08-02 Edward Driscoll Jr Panoramic camera
US6333826B1 (en) * 1997-04-16 2001-12-25 Jeffrey R. Charles Omniramic optical system having central coverage means which is associated with a camera, projector, or similar article
US6226035B1 (en) * 1998-03-04 2001-05-01 Cyclo Vision Technologies, Inc. Adjustable imaging system with wide angle capability
US6130783A (en) * 1998-05-14 2000-10-10 Sharp Kabushiki Kaisha Omnidirectional visual sensor having a plurality of mirrors with surfaces of revolution
US20010015751A1 (en) * 1998-06-16 2001-08-23 Genex Technologies, Inc. Method and apparatus for omnidirectional imaging
WO2000023842A1 (en) * 1998-10-21 2000-04-27 Aleadesign S.R.L. Anamorphic technique for filming and projecting images of 360 degree
US6897883B1 (en) * 2000-05-23 2005-05-24 Sharp Kabushiki Kaisha Omniazimuthal visual system
US6412961B1 (en) * 2000-05-30 2002-07-02 Robert Andrew Hicks Rectifying mirror
CA2324802A1 (en) * 2000-10-31 2002-04-30 Vr Interactive International, Inc. Panoramic imaging apparatus and method
US6332687B1 (en) * 2000-11-02 2001-12-25 The United States Of America As Represented By The Secretary Of The Air Force Plunger controlled, near-parabolic optical membrane mirror
US20020159166A1 (en) * 2001-02-24 2002-10-31 Herman Herman Panoramic mirror and system for producing enhanced panoramic images
US6856472B2 (en) * 2001-02-24 2005-02-15 Eyesee360, Inc. Panoramic mirror and system for producing enhanced panoramic images
EP1267313A2 (en) * 2001-06-12 2002-12-18 Sharp Corporation Image surveillance apparatus, image surveillance method, and image surveillance processing program
US20030081952A1 (en) * 2001-06-19 2003-05-01 Geng Z. Jason Method and apparatus for omnidirectional three dimensional imaging
FR2830128A1 (en) * 2001-09-26 2003-03-28 Egg Solution Sa Photoelectric sensor with sensing elements of pixels laid out in concentric circles, and device comprising such sensor for acquisition of panoramic images
US7123777B2 (en) * 2001-09-27 2006-10-17 Eyesee360, Inc. System and method for panoramic imaging
JP2004170607A (en) * 2002-11-19 2004-06-17 Matsushita Electric Ind Co Ltd Mirror element and its manufacturing method
US20050086032A1 (en) * 2003-07-28 2005-04-21 Light Prescriptions Innovators, Llc Three-dimensional simultaneous multiple-surface method and free-form illumination-optics designed therefrom
US20050094884A1 (en) * 2003-09-22 2005-05-05 Kddi Corporation Adaptable shape image encoding apparatus and decoding apparatus
US7227705B2 (en) * 2004-10-01 2007-06-05 Diehl Bgt Defence Gmbh & Co., Kg Wide-angle optical system
WO2006047488A2 (en) * 2004-10-25 2006-05-04 The Trustees Of Columbia University In The City Of New York Systems and methods for displaying three-dimensional images
US7614748B2 (en) * 2004-10-25 2009-11-10 The Trustees Of Columbia University In The City Of New York Systems and methods for displaying three-dimensional images
US7952606B2 (en) * 2005-05-26 2011-05-31 Korea Advanced Institute Of Science And Technology Apparatus for providing omnidirectional stereo image with single camera

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
Anonymous, Research Diclosure RD 435140A, 7/2000. *

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8798451B1 (en) * 2013-06-15 2014-08-05 Gyeongil Kweon Methods of obtaining panoramic images using rotationally symmetric wide-angle lenses and devices thereof
WO2016065130A1 (en) * 2014-10-24 2016-04-28 Magik Eye Inc. Distance sensor
US10268906B2 (en) 2014-10-24 2019-04-23 Magik Eye Inc. Distance sensor with directional projection beams
US10228243B2 (en) 2015-05-10 2019-03-12 Magik Eye Inc. Distance sensor with parallel projection beams
WO2018106670A1 (en) * 2016-12-07 2018-06-14 Magik Eye Inc. Distance sensor projecting parallel patterns
US10337860B2 (en) 2017-12-05 2019-07-02 Magik Eye Inc. Distance sensor including adjustable focus imaging sensor

Also Published As

Publication number Publication date
US8471892B2 (en) 2013-06-25
WO2008066742A1 (en) 2008-06-05
US20080122922A1 (en) 2008-05-29

Similar Documents

Publication Publication Date Title
EP1341383B1 (en) Composite camera system, zoom camera image display control method, zoom camera control method, control program, and computer readable recording medium
US8381985B2 (en) Camera applications in a handheld device
US7450165B2 (en) Multiple-view processing in wide-angle video camera
US5760826A (en) Omnidirectional imaging apparatus
CN1985266B (en) Panoramic vision system and method
EP1765014B1 (en) Surveillance camera apparatus and surveillance camera system
CN103402052B (en) The method of using a rotational symmetry angle lens to obtain a panoramic image and the linearity and apparatus
RU2201607C2 (en) Omnidirectional facility to form images
US20040027451A1 (en) Immersive imaging system
US20040061774A1 (en) Digital imaging system using overlapping images to formulate a seamless composite image and implemented using either a digital imaging sensor array
US20050029458A1 (en) System and a method for a smart surveillance system
JP3290993B2 (en) Method and apparatus for creating a spherical image
EP0908053B1 (en) Panoramic camera
US20090034086A1 (en) Panoramic three-dimensional adapter for an optical instrument and a combination of such an adapter and such an optical instrument
US20050036036A1 (en) Camera control apparatus and method
US7893971B2 (en) Light source estimation device that captures light source images when it is determined that the imaging device is not being used by the cameraman
US8792002B2 (en) System for extending a field-of-view of an image acquisition device
US6304285B1 (en) Method and apparatus for omnidirectional imaging
US7479979B2 (en) Omnidirectional monitoring control system, omnidirectional monitoring control method, omnidirectional monitoring control program, and computer readable recording medium
JP3996805B2 (en) Monitoring camera apparatus, the mask method of surveillance camera system apparatus and an imaging screen
US20060023105A1 (en) Panoramic video system with real-time distortion-free imaging
EP2375755B1 (en) Apparatus for detecting direction of image pickup device and moving body comprising same
Zhang Camera calibration
US9215358B2 (en) Omni-directional intelligent autotour and situational aware dome surveillance camera system and method
JP4048511B2 (en) Fisheye lens camera device and the image distortion correction method

Legal Events

Date Code Title Description
STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION