US7893985B1 - Wide angle electronic camera with improved peripheral vision - Google Patents

Wide angle electronic camera with improved peripheral vision Download PDF

Info

Publication number
US7893985B1
US7893985B1 US11/081,080 US8108005A US7893985B1 US 7893985 B1 US7893985 B1 US 7893985B1 US 8108005 A US8108005 A US 8108005A US 7893985 B1 US7893985 B1 US 7893985B1
Authority
US
United States
Prior art keywords
image
distortion
sensor
wide
circular
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related, expires
Application number
US11/081,080
Inventor
Yavuz Ahiska
Mark Kenneth Davey
Michael Benjamin Kirsch
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Grandeye Ltd
Original Assignee
Grandeye Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Grandeye Ltd filed Critical Grandeye Ltd
Priority to US11/081,080 priority Critical patent/US7893985B1/en
Assigned to GRANDEYE, LTD. reassignment GRANDEYE, LTD. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: AHISKA, YAVUZ, DAVEY, MARK KENNETH, KIRSCH, MICHAEL BENJAMIN
Application granted granted Critical
Publication of US7893985B1 publication Critical patent/US7893985B1/en
Expired - Fee Related legal-status Critical Current
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/58Means for changing the camera field of view without moving the camera body, e.g. nutating or panning of optics or image sensors

Definitions

  • the present inventions relate generally to wide angle optical systems, and more specifically to a video or still camera system with various image processing that improves peripheral resolution.
  • wide angle imaging systems including “fish eye” lenses or other distorted view imaging systems.
  • wide angle imaging has many applications in security and surveillance, such as close circuit surveillance of property or other monitoring.
  • the video camera When the video camera is used with a conventional fish eye lens, e.g., one with a field of view of 180 degrees, the image output by the camera (and projected onto an electronic imaging device) is distorted, usually circular.
  • the field of view of a camera with a fish eye lens is illustrated in FIG. 1 .
  • FIG. 1 shows a camera 100 with wide angle lens system (such as a 180 degrees fisheye lens).
  • Camera 100 view is broken down into two types of region, including main areas of interest 102 and central region 104 .
  • main areas of interest 102 are typically warped and occupy the periphery of any optical image output by camera 100 .
  • This distortion is typically circular for a circular imaging system, but can be of other shapes, depending on the lens system implemented.
  • Cameras such as that depicted in FIG. 1 often output an optical signal which is projected or otherwise input to an electronic sensor, such as (but not limited to) a charge-coupled display (hereinafter “CCD”) or complementary metal-oxide semiconductor (hereinafter “CMOS”) sensor system.
  • CCD charge-coupled display
  • CMOS complementary metal-oxide semiconductor
  • Sensors used in a video camera are generally rectangular, so part of the sensor's photo-sensitive area is wasted since it is not illuminated by the circular image projected onto its surface.
  • the optical signal may be projected so as to cover most or all of the sensor area, but this results in lost information from the optical signal. These cases are illustrated in FIGS. 2A and 2B .
  • FIG. 2A shows a system wherein an optical signal is projected onto an image sensor.
  • most of the area of the sensor 200 is occupied by the lens image circle 202 (though some of the image sensor 200 is not occupied and is thus wasted sensor region 208 ).
  • some of the lens image 202 is lost, as depicted by lost image region 210 .
  • Central region 206 is also shown, and typically comprises the least distorted region of the output optical signal.
  • Main region of interest 204 typically includes more distortion, depending on the lens system used to gather the image.
  • FIG. 2B shows another situation, wherein the entire lens image circle 202 is projected within the boundaries of image sensor 200 .
  • This situation results in a greater wasted sensor region 208 and also requires the region of interest 204 and central region 206 to occupy a smaller area of the sensor 200 , meaning the optical image covers fewer detectors or pixels, and thus has lower resolution.
  • the peripheral region of the field of view is the desired area for surveillance applications that require visibility of a panorama. This corresponds, for example, with regions of interest 204 .
  • the region of interest may be on the periphery of the total captured image, and because the image was captured with a lens system that introduces greater distortion on the periphery (such as the case with a fisheye lens system or other wide-angle lens system), the region of interest may include high amounts of distortion while covering a relatively small area of the available image sensor region. This results in an inefficient use of the sensor region, lower resolution in the areas of interest, and higher resolution in areas that are of lower interest.
  • the present invention in one example embodiment, provides a system and method for improving the peripheral imaging of a distorted optical image.
  • one embodiment relates to the design of a video or still camera with a novel optical system and an electronic image processing system.
  • the optical system captures the image and distorts it such that regions of interest are expanded as they are projected onto a sensor.
  • the electronic image processor then eliminates the distortion by applying a substantially opposite distortion compared with the distortion introduced by some or all of the optical system.
  • the present innovations are applied to increase resolution of regions of interest, and/or to improve the fit of a non-rectangular image to a rectangular image sensor, for example.
  • these innovations as described below, can be used in other shaped optical and sensor implementations.
  • the increase in resolution provides capability to magnify (by increasing resolution of) one or more regions of interest, while trading off resolution in one or more regions outside the region(s) of interest.
  • FIG. 1 shows a camera with 180 degree viewing angle.
  • FIGS. 2A and 2B show the projection of circular images onto square sensors.
  • FIG. 3 shows a circular image without peripheral magnification.
  • FIG. 4 shows magnification of peripheral regions of an image consistent with a preferred embodiment of the present innovations.
  • FIG. 5 shows an example view of regular patterned object with conventional circular image fish eye lens.
  • FIG. 6 shows an example of deformation showing magnification of peripheral regions consistent with a preferred embodiment of the present innovations.
  • FIG. 7 shows ellipsoidal distortion consistent with a preferred embodiment of the present innovations.
  • FIG. 8 shows more radical distortion consistent with a preferred embodiment of the present innovations.
  • FIG. 9 shows a system consistent with implementing preferred embodiments of the present innovations.
  • FIG. 10 shows a flowchart with process steps consistent with implementing preferred embodiments of the present innovations.
  • the present innovations include a system for capturing an image, introducing predetermined first distortion into the captured image so as to alter the shape and/or magnify one or more regions of interest when projected onto a sensor, and a system for introducing second distortion to cancel the first distortion.
  • the present innovations include an optical imaging system for capturing wide angle images and introducing predetermined first distortion into the captured image so as to alter the shape and/or magnify one or more regions of interest when projected onto a sensor, and an electronic system for introducing second distortion to cancel the first distortion.
  • the present innovations include a novel optical lens and/or mirror system it introduce optical distortion to a region or regions of interest within a wide field of view in order to increase resolution and/or to reshape the image.
  • the resolution of fine detail within the region or regions of interest is increased, since the distortion causes the optical image to impinge upon a larger area of an electronic imaging device, such as a CCD or CMOS sensor.
  • the overall field of view is substantially maintained.
  • the geometric distortion introduced by the novel optical system is preferably later removed or transformed to a different set of distorted but human viewable image forms by electronic processing of the electronic image, using methods such as those disclosed in U.S. provisional patent application No.
  • a plurality of transformations may be applied to a part or total distorted image produced by the optical system to produce a plurality of human viewable image forms. Such transformations may include perspective corrected projections, circular or cylindrical projections for a given field of view. The plurality of images may be produced in multi-window image format. It is noted that other means of distortion and distortion transformation are also applicable and within the contemplation of the present innovations.
  • symmetric distortion such as radial distortion
  • asymmetric distortion such as ellipsoidal or other asymmetric distortion.
  • the types of distortion described herein are only examples, and serve to describe the broader concepts of the present innovations.
  • FIGS. 3 and 4 show how radial distortion affects a circular image.
  • FIG. 3 shows a circular image, such as that which might be captured by a fish eye lens.
  • Region 302 is a peripheral region, and in this example is the region of interest.
  • Region 304 is the central region. The length of the arrows indicates the degree to which the region of interest 302 covers, for example, a sensor region 300 on which the optical image is projected.
  • Radial distortion, as depicted here can be introduced, for example, by a parabolic mirror. Other means of introducing symmetric distortion are also within the contemplation of the present innovations.
  • FIG. 4 shows the same circular image with symmetric, in this case radial, distortion added.
  • the radial distortion magnifies the region of interest 402 while demagnifying the central region 404 .
  • the image of FIG. 4 effectively magnifies the region of interest 402 relative to that same region in FIG. 3 , namely region 302 .
  • Region 402 covers a larger area of a sensor region 400 , and thus has greater resolution.
  • resolution in central region 302 , or 402 has been decreased.
  • the region of interest is magnified or increased in resolution, while the central region has been demagnified or decreased in resolution. It is noted that, in this example, the total field of view is substantially maintained.
  • FIG. 5 shows an image projected onto the electronic image sensor viewing a regular patterned scene by a conventional fish eye lens, which emphasizes the central portion of the scene.
  • the novel optical system described herein introduces a projection distortion illustrated in FIG. 6 which magnifies the required main area of interest (in this example, the periphery). By projecting the main area of interest onto a larger proportion of the electronic image sensor, a higher resolution image of the main area of interest is obtained.
  • This type of distortion spreads the region of interest over a greater number of sensor photo-sites, which results in greater accuracy of the captured image where required, and lower accuracy of the captured image in areas of lesser interest.
  • periphery is used as an example area of interest in this example description, other areas could also be used.
  • the novel optical system is preferably designed to project a non-circular image onto the electronic imaging sensor in order to illuminate a larger part, or substantially all, of the area of the sensor which would otherwise be unilluminated by a conventional circular image, and/or to capture part of the image which would be projected outside the region of the sensor.
  • the innovative system projects a non-circular image onto the rectangular area of the electronic sensor or imaging device.
  • the projection can be ellipsoidal or similarly shaped as illustrated in FIG. 7 .
  • image sensor 700 is rectangular, and what was previously (prior to introducing distortion) a circular image is now distorted into an ellipsoidal image projection 702 .
  • Main area of interest 704 is likewise distorted, being part of the same image. It is noted that, in this example, the region of interest 704 can be magnified (as previously described) if the system is designed to perform such a function.
  • the present innovations include both types of distortion, implemented singly or in combination. It is useful to combine both types of distortion to result in increases of the total image resolution and greater increase in resolution of regions of interest. In preferred embodiments of both cases, it is noted, distortion introduced (preferably by the optical system) is corrected (preferably by electronic image processing).
  • FIG. 8 shows even greater amounts of asymmetric distortion.
  • image sensor 800 is nearly completely covered by the extremely distorted lens image projection 802 , including area of interest 804 .
  • FIG. 9 shows an example system consistent with implementing a preferred embodiment of the present innovations.
  • a system 902 preferably an optical system
  • the resulting electronic image is then transferred to section 906 (preferably electronic image processing circuitry, but which could be implemented as complex optical processing system) which transforms the image in such a way as to reduce or substantially eliminate the distortion introduced by optical system 902 .
  • section 906 preferably electronic image processing circuitry, but which could be implemented as complex optical processing system
  • Wide angle camera image processing circuitry should normally include other capabilities, such as conventional capabilities commonly found in digital photography and video cameras.
  • an image correction pipeline can include debayering, exposure correction, gamma correction, white balance, color correction, sharpening, image filtering, and other manipulation known in the art.
  • the system can also include remote (or local) inputs to select fields of view, for example.
  • gamma correction and some image filtering may be performed after image dewarping.
  • FIG. 10 shows a simple process flow consistent with implementing a preferred embodiment of the present innovations.
  • the image is received at the optical system (step 1002 ).
  • the image is distorted by the optical system (step 1004 ), resulting in an altered image.
  • the image is distorted so as to increase resolution of regions of interest, to modify the overall shape of the image to more closely correspond to the shape of the sensor, or both.
  • the image is projected onto the sensor and converted into an electronic image (step 1006 ).
  • This electronic image is conveyed to image processing circuitry (step 1008 ) which corrects the distortion added by the optical system (step 1010 ).
  • the image processing circuitry is also capable of other functions, such as debayering, color correction, exposure correction, sharpening, filtering, etc.
  • the final image can be output and made available as an analog signal or a digital signal.
  • Other types of correction can be accomplished at this stage as well, such as perspective correction and projections such as cylindrical projections, spherical projections, etc.
  • the US patent to Ford Oxaal, U.S. Pat. No. 6,323,862 is informative in this area, and is hereby incorporated by reference.
  • a wide-angle video camera capable of high resolution peripheral vision comprising a) a wide-angle optical system which performs radial distortion to magnify the peripheral area of the view, b) an image sensor and circuitry to convert the output of said lens to electrical signals suitable for use by an image processing circuit, c) said image processing circuit capable of correcting optical distortions introduced by said wide angle optical system.
  • a wide-angle video camera capable of high resolution peripheral vision comprising: a) a wide-angle optical system that projects an image onto an image sensor and circuitry to convert the output of said lens to electrical signals suitable for use by an image processing circuit, b) said optical system to introduce asymmetric distortion to form a non-circular image on said sensor, c) said image processing circuit capable of transforming including but not limited to perspective correcting optical distortions introduced by said wide angle optical system.
  • a method of improving peripheral resolution of a captured image comprising the steps of: receiving an image at a wide angle image capture system, which introduces a first distortion to the image; simultaneously introducing a second distortion to the image; conveying the image to an image sensor, which outputs a signal; conveying the signal to image processing circuitry which substantially transforms the distortions to a human viewable form.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Studio Devices (AREA)

Abstract

The present inventions provide a system and method for a wide angle camera capable of high resolution peripheral vision. A wide angle lens system distorts the image and conveys it to an image sensor, which converts the image to an electronic signal and conveys it to image processing circuitry. The image processing circuitry corrects the distortion or transforms it to human recognizable image forms. In preferred embodiments, the distortion provides greater resolution of regions of interest and more efficient use of the area of the image sensor.

Description

CROSS REFERENCE TO RELATED APPLICATIONS
This application claims priority from provisional patent application 60/553,186, titled “Wide Angle Electronic Camera with Improved Peripheral Vision,” filed on Mar. 15, 2004, which is hereby incorporated by reference.
Other applications which relate to the present application, and which are hereby incorporated by reference include:
    • U.S. Application Ser. No. 60/467,808, entitled “Wide-Angle Motion Camera with Electronic PTZ.”
    • U.S. Application Ser. No. 60/467,705, entitled “Correction of Optical Distortion by Image Processing.”
    • U.S. Application Ser. No. 60/467,588, entitled “Multiple View Processing in Wide-Angle Video Camera.”
    • U.S. Application Ser. No. 60/467,643, entitled “Multiple Object Processing in Wide-Angle Video Camera.”
    • U.S. Application Ser. No. 60/467,644, entitled “Method of Simultaneously Displaying Multiple View for Video Surveillance.”
FIELD OF THE INVENTION
The present inventions relate generally to wide angle optical systems, and more specifically to a video or still camera system with various image processing that improves peripheral resolution.
BACKGROUND AND SUMMARY OF THE INVENTION
There are many applications for wide angle imaging systems, including “fish eye” lenses or other distorted view imaging systems. For example, wide angle imaging has many applications in security and surveillance, such as close circuit surveillance of property or other monitoring.
When the video camera is used with a conventional fish eye lens, e.g., one with a field of view of 180 degrees, the image output by the camera (and projected onto an electronic imaging device) is distorted, usually circular. The field of view of a camera with a fish eye lens is illustrated in FIG. 1.
FIG. 1 shows a camera 100 with wide angle lens system (such as a 180 degrees fisheye lens). Camera 100 view is broken down into two types of region, including main areas of interest 102 and central region 104. Because of the distortion introduced by the wide angle lens system of camera 100, the main areas of interest 102 are typically warped and occupy the periphery of any optical image output by camera 100. This distortion is typically circular for a circular imaging system, but can be of other shapes, depending on the lens system implemented.
Cameras such as that depicted in FIG. 1 often output an optical signal which is projected or otherwise input to an electronic sensor, such as (but not limited to) a charge-coupled display (hereinafter “CCD”) or complementary metal-oxide semiconductor (hereinafter “CMOS”) sensor system. Sensors used in a video camera are generally rectangular, so part of the sensor's photo-sensitive area is wasted since it is not illuminated by the circular image projected onto its surface. Alternatively, depending on the projection scheme used, the optical signal may be projected so as to cover most or all of the sensor area, but this results in lost information from the optical signal. These cases are illustrated in FIGS. 2A and 2B.
FIG. 2A shows a system wherein an optical signal is projected onto an image sensor. In this example, most of the area of the sensor 200 is occupied by the lens image circle 202 (though some of the image sensor 200 is not occupied and is thus wasted sensor region 208). In this example, in order to maximize the main region of interest 204 on the sensor 200, some of the lens image 202 is lost, as depicted by lost image region 210. Central region 206 is also shown, and typically comprises the least distorted region of the output optical signal. Main region of interest 204 typically includes more distortion, depending on the lens system used to gather the image.
FIG. 2B shows another situation, wherein the entire lens image circle 202 is projected within the boundaries of image sensor 200. This situation results in a greater wasted sensor region 208 and also requires the region of interest 204 and central region 206 to occupy a smaller area of the sensor 200, meaning the optical image covers fewer detectors or pixels, and thus has lower resolution.
Often, the peripheral region of the field of view is the desired area for surveillance applications that require visibility of a panorama. This corresponds, for example, with regions of interest 204. Because the region of interest may be on the periphery of the total captured image, and because the image was captured with a lens system that introduces greater distortion on the periphery (such as the case with a fisheye lens system or other wide-angle lens system), the region of interest may include high amounts of distortion while covering a relatively small area of the available image sensor region. This results in an inefficient use of the sensor region, lower resolution in the areas of interest, and higher resolution in areas that are of lower interest.
Wide Angle Electronic Camera with Improved Peripheral Vision
The present invention, in one example embodiment, provides a system and method for improving the peripheral imaging of a distorted optical image.
For example, one embodiment relates to the design of a video or still camera with a novel optical system and an electronic image processing system. The optical system captures the image and distorts it such that regions of interest are expanded as they are projected onto a sensor. The electronic image processor then eliminates the distortion by applying a substantially opposite distortion compared with the distortion introduced by some or all of the optical system.
In preferred embodiments, the present innovations are applied to increase resolution of regions of interest, and/or to improve the fit of a non-rectangular image to a rectangular image sensor, for example. Of course, these innovations, as described below, can be used in other shaped optical and sensor implementations.
The increase in resolution provides capability to magnify (by increasing resolution of) one or more regions of interest, while trading off resolution in one or more regions outside the region(s) of interest.
The disclosed innovations, in various embodiments, provide one or more of at least the following advantages:
    • improved resolution in regions of interest;
    • magnification capability for regions of interest beyond normal magnification capability;
    • more efficient use of sensor area;
    • capture of all or a greater percentage of a whole image while improving resolution in a specific region or regions.
BRIEF DESCRIPTION OF THE DRAWINGS
The disclosed inventions will be described with reference to the accompanying drawings, which show important sample embodiments of the invention and which are incorporated in the specification hereof by reference, wherein:
FIG. 1 shows a camera with 180 degree viewing angle.
FIGS. 2A and 2B show the projection of circular images onto square sensors.
FIG. 3 shows a circular image without peripheral magnification.
FIG. 4 shows magnification of peripheral regions of an image consistent with a preferred embodiment of the present innovations.
FIG. 5 shows an example view of regular patterned object with conventional circular image fish eye lens.
FIG. 6 shows an example of deformation showing magnification of peripheral regions consistent with a preferred embodiment of the present innovations.
FIG. 7 shows ellipsoidal distortion consistent with a preferred embodiment of the present innovations.
FIG. 8 shows more radical distortion consistent with a preferred embodiment of the present innovations.
FIG. 9 shows a system consistent with implementing preferred embodiments of the present innovations.
FIG. 10 shows a flowchart with process steps consistent with implementing preferred embodiments of the present innovations.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
The numerous innovative teachings of the present application will be described with particular reference to the presently preferred embodiment (by way of example, and not of limitation).
In various embodiments, the present innovations include a system for capturing an image, introducing predetermined first distortion into the captured image so as to alter the shape and/or magnify one or more regions of interest when projected onto a sensor, and a system for introducing second distortion to cancel the first distortion.
In preferred embodiments, the present innovations include an optical imaging system for capturing wide angle images and introducing predetermined first distortion into the captured image so as to alter the shape and/or magnify one or more regions of interest when projected onto a sensor, and an electronic system for introducing second distortion to cancel the first distortion.
In one example embodiment, the present innovations include a novel optical lens and/or mirror system it introduce optical distortion to a region or regions of interest within a wide field of view in order to increase resolution and/or to reshape the image. The resolution of fine detail within the region or regions of interest is increased, since the distortion causes the optical image to impinge upon a larger area of an electronic imaging device, such as a CCD or CMOS sensor. In preferred embodiments, the overall field of view is substantially maintained. The geometric distortion introduced by the novel optical system is preferably later removed or transformed to a different set of distorted but human viewable image forms by electronic processing of the electronic image, using methods such as those disclosed in U.S. provisional patent application No. 60/467,705, “Correction of Optical Distortion by Image Processing,” or using transformation methods such as those disclosed in U.S. Pat. No. 6,323,862 by Ford Oxaal, which are hereby incorporated by reference. A plurality of transformations may be applied to a part or total distorted image produced by the optical system to produce a plurality of human viewable image forms. Such transformations may include perspective corrected projections, circular or cylindrical projections for a given field of view. The plurality of images may be produced in multi-window image format. It is noted that other means of distortion and distortion transformation are also applicable and within the contemplation of the present innovations.
In preferred embodiments of the present inventions, there are two categories of distortion that are introduced, either singly or in combination. They are symmetric distortion, such as radial distortion, and asymmetric distortion, such as ellipsoidal or other asymmetric distortion. The types of distortion described herein are only examples, and serve to describe the broader concepts of the present innovations.
FIGS. 3 and 4 show how radial distortion affects a circular image. FIG. 3 shows a circular image, such as that which might be captured by a fish eye lens. Region 302 is a peripheral region, and in this example is the region of interest. Region 304 is the central region. The length of the arrows indicates the degree to which the region of interest 302 covers, for example, a sensor region 300 on which the optical image is projected. Radial distortion, as depicted here, can be introduced, for example, by a parabolic mirror. Other means of introducing symmetric distortion are also within the contemplation of the present innovations.
FIG. 4 shows the same circular image with symmetric, in this case radial, distortion added. In this example, the radial distortion magnifies the region of interest 402 while demagnifying the central region 404.
In these examples, the image of FIG. 4 effectively magnifies the region of interest 402 relative to that same region in FIG. 3, namely region 302. Region 402 covers a larger area of a sensor region 400, and thus has greater resolution. Meanwhile, resolution in central region 302, or 402, has been decreased. Thus, by introducing radial distortion, the region of interest is magnified or increased in resolution, while the central region has been demagnified or decreased in resolution. It is noted that, in this example, the total field of view is substantially maintained.
FIG. 5 shows an image projected onto the electronic image sensor viewing a regular patterned scene by a conventional fish eye lens, which emphasizes the central portion of the scene. The novel optical system described herein introduces a projection distortion illustrated in FIG. 6 which magnifies the required main area of interest (in this example, the periphery). By projecting the main area of interest onto a larger proportion of the electronic image sensor, a higher resolution image of the main area of interest is obtained.
This type of distortion spreads the region of interest over a greater number of sensor photo-sites, which results in greater accuracy of the captured image where required, and lower accuracy of the captured image in areas of lesser interest. Though the periphery is used as an example area of interest in this example description, other areas could also be used.
Asymmetric distortion can also be introduced into the captured image. In such a case, the novel optical system is preferably designed to project a non-circular image onto the electronic imaging sensor in order to illuminate a larger part, or substantially all, of the area of the sensor which would otherwise be unilluminated by a conventional circular image, and/or to capture part of the image which would be projected outside the region of the sensor.
For example, referring back to FIG. 2A, it is noted that part of the projected image is lost because it falls outside the area of the electronic sensor 200. Alternately, in FIG. 2B, though all the image is within the bounds of the sensor 200, there are large unused regions 208 of the sensor 200. Neither case is optimal, and both cases are effectively addressed by implementing the present innovations.
By implementing asymmetric distortion via the optical system of the present innovations, the innovative system projects a non-circular image onto the rectangular area of the electronic sensor or imaging device. The projection can be ellipsoidal or similarly shaped as illustrated in FIG. 7. In this figure, image sensor 700 is rectangular, and what was previously (prior to introducing distortion) a circular image is now distorted into an ellipsoidal image projection 702. Main area of interest 704 is likewise distorted, being part of the same image. It is noted that, in this example, the region of interest 704 can be magnified (as previously described) if the system is designed to perform such a function. The present innovations include both types of distortion, implemented singly or in combination. It is useful to combine both types of distortion to result in increases of the total image resolution and greater increase in resolution of regions of interest. In preferred embodiments of both cases, it is noted, distortion introduced (preferably by the optical system) is corrected (preferably by electronic image processing).
FIG. 8 shows even greater amounts of asymmetric distortion. In this example, image sensor 800 is nearly completely covered by the extremely distorted lens image projection 802, including area of interest 804.
FIG. 9 shows an example system consistent with implementing a preferred embodiment of the present innovations. In this example system 900, a system 902 (preferably an optical system) captures and distorts an image, and the distorted image is projected on (or otherwise sent to) an electronic sensor, such as a CCD or CMOS sensor 904. The resulting electronic image is then transferred to section 906 (preferably electronic image processing circuitry, but which could be implemented as complex optical processing system) which transforms the image in such a way as to reduce or substantially eliminate the distortion introduced by optical system 902.
Wide angle camera image processing circuitry should normally include other capabilities, such as conventional capabilities commonly found in digital photography and video cameras. For example, an image correction pipeline can include debayering, exposure correction, gamma correction, white balance, color correction, sharpening, image filtering, and other manipulation known in the art. the system can also include remote (or local) inputs to select fields of view, for example. In other implementations, such as gamma correction and some image filtering, may be performed after image dewarping.
FIG. 10 shows a simple process flow consistent with implementing a preferred embodiment of the present innovations. First the image is received at the optical system (step 1002). The image is distorted by the optical system (step 1004), resulting in an altered image. In preferred embodiments, the image is distorted so as to increase resolution of regions of interest, to modify the overall shape of the image to more closely correspond to the shape of the sensor, or both. Next, the image is projected onto the sensor and converted into an electronic image (step 1006). This electronic image is conveyed to image processing circuitry (step 1008) which corrects the distortion added by the optical system (step 1010). In preferred embodiments, the image processing circuitry is also capable of other functions, such as debayering, color correction, exposure correction, sharpening, filtering, etc. The final image can be output and made available as an analog signal or a digital signal. Other types of correction can be accomplished at this stage as well, such as perspective correction and projections such as cylindrical projections, spherical projections, etc. The US patent to Ford Oxaal, U.S. Pat. No. 6,323,862 is informative in this area, and is hereby incorporated by reference.
According to a disclosed class of innovative embodiments, there is provided: A wide-angle video camera capable of high resolution peripheral vision comprising a) a wide-angle optical system which performs radial distortion to magnify the peripheral area of the view, b) an image sensor and circuitry to convert the output of said lens to electrical signals suitable for use by an image processing circuit, c) said image processing circuit capable of correcting optical distortions introduced by said wide angle optical system.
According to a disclosed class of innovative embodiments, there is provided: A wide-angle video camera capable of high resolution peripheral vision comprising: a) a wide-angle optical system that projects an image onto an image sensor and circuitry to convert the output of said lens to electrical signals suitable for use by an image processing circuit, b) said optical system to introduce asymmetric distortion to form a non-circular image on said sensor, c) said image processing circuit capable of transforming including but not limited to perspective correcting optical distortions introduced by said wide angle optical system.
According to a disclosed class of innovative embodiments, there is provided: A method of improving peripheral resolution of a captured image, comprising the steps of: receiving an image at a wide angle image capture system, which introduces a first distortion to the image; simultaneously introducing a second distortion to the image; conveying the image to an image sensor, which outputs a signal; conveying the signal to image processing circuitry which substantially transforms the distortions to a human viewable form.
Modifications and Variations
As will be recognized by those skilled in the art, the innovative concepts described in the present application can be modified and varied over a tremendous range of applications, and accordingly the scope of patented subject matter is not limited by any of the specific exemplary teachings given.
Additional general background, which helps to show variations and implementations, may be found in the following publications, all of which are hereby incorporated by reference:
None of the description in the present application should be read as implying that any particular element, step, or function is an essential element which must be included in the claim scope: THE SCOPE OF PATENTED SUBJECT MATTER IS DEFINED ONLY BY THE ALLOWED CLAIMS. Moreover, none of these claims are intended to invoke paragraph six of 35 USC section 112 unless the exact words “means for” are followed by a participle.

Claims (20)

1. A wide-angle video camera capable of high resolution peripheral vision comprising:
a) a wide-angle optical system which performs radial distortion to provide a distorted non-rectilinear image of a view, in which a peripheral annulus of the image is magnified, and a central area of the image is demagnified, as compared with a fisheye or similar lens with similar field of view and image size;
b) an image sensor which receives said image and converts it to electrical signals suitable for use by an image processing circuit;
c) said image processing circuit transforming the optical distortions introduced by said wide-angle optical system to correct the optical distortions and accordingly output a human viewable image with greater resolution in peripheral regions of interest.
2. The device of claim 1 in which a conventional lens and a specially manufactured parabolic mirror is used to achieve said radial distortion.
3. The device of claim 1 in which said wide-angle camera is a still picture camera.
4. The device of claim 1, wherein said distortion distorts the projected image into substantially the same shape as a non-circular image sensor.
5. The device of claim 1, wherein the distortion modifies the projected image to substantially fit within a defined border of a non-circular image sensor.
6. A wide-angle video camera capable of high resolution peripheral vision comprising:
a) a wide-angle optical system that projects a distorted non-rectilinear image onto a non-circular image sensor, which converts said non-circular image into electrical signals suitable for use by an image processing circuit;
b) said optical system introducing radially asymmetric distortion to project a non-circular image onto said non-circular sensor, said distortion modifying the image of a view to thereby improve the fit of the projected image to the sensor;
c) said image processing circuit transforming optical distortions introduced by said wide angle optical system output a human recognizable image with a greater proportion of the sensor area utilized than if a circular image were projected.
7. The device of claim 6 in which a conventional lens and a specially manufactured parabolic mirror is used to achieve said radial distortion.
8. The device of claim 6 in which said wide-angle camera is a still picture camera.
9. The device of clam 6, further comprising: a wide-angle optical system which performs combination of radial and asymmetric distortion to magnify the peripheral area of the view.
10. The device of claim 9 in which a combination of a conventional lens and specially manufactured parabolic mirrors is used to achieve said distortion.
11. The device of claim 9 in which said wide-angle camera is a still camera.
12. A method of improving peripheral resolution of a captured image, comprising the steps of:
receiving an image at a wide-angle image capture system, which introduces a first distortion to the image so as to project a substantially hemispherical field of view onto a non-circular image sensor;
simultaneously introducing a second distortion to the image, said second distortion modifying the image projection onto the image sensor to increase total image resolution with a greater increase in image resolution for at least one selected region of interest, and projecting a greater proportion of said image corresponding to the said selected region of interest onto a greater portion of area on said sensor;
conveying the image to the image sensor, which outputs a signal;
conveying the signal to image processing circuitry which transforms both the first and second distortions into a human recognizable image output.
13. The method of claim 12, wherein the wide-angle image capture system is an optical system including a special form of fisheye lens.
14. The method of claim 12, wherein the second distortion is a radial distortion to magnify one or more regions of interest.
15. The method of claim 14, wherein the one or more regions of interest comprise a peripheral view.
16. The method of claim 12, wherein the second distortion is an asymmetric distortion to project a non-circular image.
17. The method of claim 12, wherein the second distortion includes both a radial distortion to magnify one of more regions of interest and an asymmetric distortion to project a non-circular image.
18. The method of claim 17, wherein the one or more regions of interest comprise a peripheral region.
19. The method of claim 12, wherein the first distortion warps the substantially hemisphere view into a non-circular projection onto the non-circular image sensor.
20. The method of claim 12, wherein the second distortion warps the view into a non-circular projection corresponding to the shape of the sensor to better fit upon the non-circular image sensor.
US11/081,080 2004-03-15 2005-03-15 Wide angle electronic camera with improved peripheral vision Expired - Fee Related US7893985B1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US11/081,080 US7893985B1 (en) 2004-03-15 2005-03-15 Wide angle electronic camera with improved peripheral vision

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US55318604P 2004-03-15 2004-03-15
US11/081,080 US7893985B1 (en) 2004-03-15 2005-03-15 Wide angle electronic camera with improved peripheral vision

Publications (1)

Publication Number Publication Date
US7893985B1 true US7893985B1 (en) 2011-02-22

Family

ID=43597124

Family Applications (1)

Application Number Title Priority Date Filing Date
US11/081,080 Expired - Fee Related US7893985B1 (en) 2004-03-15 2005-03-15 Wide angle electronic camera with improved peripheral vision

Country Status (1)

Country Link
US (1) US7893985B1 (en)

Cited By (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080239109A1 (en) * 2007-03-31 2008-10-02 David John Jennings Method and apparatus for correcting image distortion
US20100020202A1 (en) * 2006-04-13 2010-01-28 Opt Corporation Camera apparatus, and image processing apparatus and image processing method
US20120002096A1 (en) * 2010-07-05 2012-01-05 Samsung Electronics Co., Ltd. Imaging device including a plurality of imaging units
US8139896B1 (en) 2005-03-28 2012-03-20 Grandeye, Ltd. Tracking moving objects accurately on a wide-angle video
US8264524B1 (en) * 2008-09-17 2012-09-11 Grandeye Limited System for streaming multiple regions deriving from a wide-angle camera
WO2013002758A1 (en) 2011-06-27 2013-01-03 Oncam Global, Inc. Method and systems for providing video data streams to multiple users
WO2013192400A1 (en) 2012-06-22 2013-12-27 Oncam Global, Inc. Method and systems for providing video data streams to multiple users
US20150350621A1 (en) * 2012-12-27 2015-12-03 Panasonic Intellectual Property Management Co., Ltd. Sound processing system and sound processing method
JP2017134394A (en) * 2016-01-21 2017-08-03 パナソニックIpマネジメント株式会社 Lens system, and camera system having the same
US20170358061A1 (en) * 2016-06-10 2017-12-14 Hyundai Motor Company Image processing apparatus and method for performing preprocessing to obtain image with improved sharpness
WO2018167581A1 (en) * 2017-03-15 2018-09-20 Corephotonics Ltd. Camera with panoramic scanning range
US10909384B2 (en) 2015-07-14 2021-02-02 Panasonic Intellectual Property Management Co., Ltd. Monitoring system and monitoring method
US11012621B2 (en) 2019-05-24 2021-05-18 Panasonic Intellectual Property Management Co., Ltd. Imaging device having capability of increasing resolution of a predetermined imaging area using a free-form lens
US11153481B2 (en) * 2019-03-15 2021-10-19 STX Financing, LLC Capturing and transforming wide-angle video information
US11153482B2 (en) * 2018-04-27 2021-10-19 Cubic Corporation Optimizing the content of a digital omnidirectional image
US20240005461A1 (en) * 2022-07-04 2024-01-04 Harman Becker Automotive Systems Gmbh Driver assistance system
US11936975B2 (en) 2021-05-12 2024-03-19 Nio Technology (Anhui) Co., Ltd. Combined computer vision and human vision camera system
EP4408006A1 (en) * 2023-01-27 2024-07-31 Canon Kabushiki Kaisha Image processing system, movable apparatus, image processing method, and storage medium

Citations (32)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US3725563A (en) 1971-12-23 1973-04-03 Singer Co Method of perspective transformation in scanned raster visual display
US4667236A (en) 1985-04-26 1987-05-19 Digital Services Corporation Television perspective effects system
US4728839A (en) 1987-02-24 1988-03-01 Remote Technology Corporation Motorized pan/tilt head for remote control
US4763280A (en) 1985-04-29 1988-08-09 Evans & Sutherland Computer Corp. Curvilinear dynamic image generation system
US4821209A (en) 1986-01-21 1989-04-11 International Business Machines Corporation Data transformation and clipping in a graphics display system
US5027287A (en) 1988-06-07 1991-06-25 Thomson Video Equipement Device for the digital processing of images to obtain special geometrical effects
US5185667A (en) 1991-05-13 1993-02-09 Telerobotics International, Inc. Omniview motionless camera orientation system
US5321776A (en) 1992-02-26 1994-06-14 General Electric Company Data compression system including successive approximation quantizer
US5359363A (en) 1991-05-13 1994-10-25 Telerobotics International, Inc. Omniview motionless camera surveillance system
US5396284A (en) 1993-08-20 1995-03-07 Burle Technologies, Inc. Motion detection system
US5434617A (en) 1993-01-29 1995-07-18 Bell Communications Research, Inc. Automatic tracking camera control system
US5495292A (en) 1993-09-03 1996-02-27 Gte Laboratories Incorporated Inter-frame wavelet transform coder for color video compression
US5666157A (en) 1995-01-03 1997-09-09 Arc Incorporated Abnormality detection and surveillance system
US5684937A (en) 1992-12-14 1997-11-04 Oxaal; Ford Method and apparatus for performing perspective transformation on visible stimuli
US6049281A (en) 1998-09-29 2000-04-11 Osterweil; Josef Method and apparatus for monitoring movements of an individual
US6147709A (en) 1997-04-07 2000-11-14 Interactive Pictures Corporation Method and apparatus for inserting a high resolution image into a low resolution interactive image to produce a realistic immersive experience
WO2001005991A1 (en) 1999-07-19 2001-01-25 Medigene Aktiengesellschaft Scleroprotein of an adeno-associated virus with modified chromatographic properties, the production thereof and use of the same
US6215519B1 (en) 1998-03-04 2001-04-10 The Trustees Of Columbia University In The City Of New York Combined wide angle and narrow angle imaging system and method for surveillance and monitoring
US6243099B1 (en) 1996-11-14 2001-06-05 Ford Oxaal Method for interactive viewing full-surround image data and apparatus therefor
US6344852B1 (en) 1999-03-17 2002-02-05 Nvidia Corporation Optimized system and method for binning of graphics data
WO2002062056A1 (en) 2001-01-29 2002-08-08 Koninklijke Philips Electronics N.V. Camera system and method for operating same
US6509926B1 (en) 2000-02-17 2003-01-21 Sensormatic Electronics Corporation Surveillance apparatus for camera surveillance system
US6545810B1 (en) * 1997-03-06 2003-04-08 Olympus Optical Co., Ltd. Image pickup optical system and image pickup apparatus using the same
US20030128756A1 (en) 2001-12-28 2003-07-10 Nokia Corporation Method and apparatus for selecting macroblock quantization parameters in a video encoder
EP1341383A2 (en) 2002-02-28 2003-09-03 Sharp Kabushiki Kaisha Composite camera system, zoom camera image display control method, zoom camera control method, control program, and computer readable recording medium
US6724421B1 (en) 1994-11-22 2004-04-20 Sensormatic Electronics Corporation Video surveillance system with pilot and slave cameras
US6757434B2 (en) 2002-11-12 2004-06-29 Nokia Corporation Region-of-interest tracking method and device for wavelet-based video coding
US20050122400A1 (en) * 2003-08-13 2005-06-09 Topcon Corporation Photographic apparatus with function of image correction and method thereof
US20050174437A1 (en) * 2004-02-10 2005-08-11 Fujitsu Limited Distortion correction circuit
US20050259118A1 (en) * 2001-12-03 2005-11-24 Michael Mojaver Panoramic imaging and display system with canonical magnifier
WO2006070483A1 (en) 2004-12-28 2006-07-06 Nippon Beet Sugar Manufacturing Co., Ltd. Process for producing difructose dianhydride iii crystal
WO2009039083A1 (en) 2007-09-18 2009-03-26 Georgia-Pacific Consumer Products Lp Resilient, water dispersible polyurethane foams and products incorporating same

Patent Citations (33)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US3725563A (en) 1971-12-23 1973-04-03 Singer Co Method of perspective transformation in scanned raster visual display
US4667236A (en) 1985-04-26 1987-05-19 Digital Services Corporation Television perspective effects system
US4763280A (en) 1985-04-29 1988-08-09 Evans & Sutherland Computer Corp. Curvilinear dynamic image generation system
US4821209A (en) 1986-01-21 1989-04-11 International Business Machines Corporation Data transformation and clipping in a graphics display system
US4728839A (en) 1987-02-24 1988-03-01 Remote Technology Corporation Motorized pan/tilt head for remote control
US5027287A (en) 1988-06-07 1991-06-25 Thomson Video Equipement Device for the digital processing of images to obtain special geometrical effects
US5185667A (en) 1991-05-13 1993-02-09 Telerobotics International, Inc. Omniview motionless camera orientation system
US5359363A (en) 1991-05-13 1994-10-25 Telerobotics International, Inc. Omniview motionless camera surveillance system
US5321776A (en) 1992-02-26 1994-06-14 General Electric Company Data compression system including successive approximation quantizer
US5684937A (en) 1992-12-14 1997-11-04 Oxaal; Ford Method and apparatus for performing perspective transformation on visible stimuli
US5434617A (en) 1993-01-29 1995-07-18 Bell Communications Research, Inc. Automatic tracking camera control system
US5396284A (en) 1993-08-20 1995-03-07 Burle Technologies, Inc. Motion detection system
US5495292A (en) 1993-09-03 1996-02-27 Gte Laboratories Incorporated Inter-frame wavelet transform coder for color video compression
US6724421B1 (en) 1994-11-22 2004-04-20 Sensormatic Electronics Corporation Video surveillance system with pilot and slave cameras
US5666157A (en) 1995-01-03 1997-09-09 Arc Incorporated Abnormality detection and surveillance system
US6243099B1 (en) 1996-11-14 2001-06-05 Ford Oxaal Method for interactive viewing full-surround image data and apparatus therefor
US6545810B1 (en) * 1997-03-06 2003-04-08 Olympus Optical Co., Ltd. Image pickup optical system and image pickup apparatus using the same
US6147709A (en) 1997-04-07 2000-11-14 Interactive Pictures Corporation Method and apparatus for inserting a high resolution image into a low resolution interactive image to produce a realistic immersive experience
US6215519B1 (en) 1998-03-04 2001-04-10 The Trustees Of Columbia University In The City Of New York Combined wide angle and narrow angle imaging system and method for surveillance and monitoring
US6049281A (en) 1998-09-29 2000-04-11 Osterweil; Josef Method and apparatus for monitoring movements of an individual
US6344852B1 (en) 1999-03-17 2002-02-05 Nvidia Corporation Optimized system and method for binning of graphics data
WO2001005991A1 (en) 1999-07-19 2001-01-25 Medigene Aktiengesellschaft Scleroprotein of an adeno-associated virus with modified chromatographic properties, the production thereof and use of the same
US6509926B1 (en) 2000-02-17 2003-01-21 Sensormatic Electronics Corporation Surveillance apparatus for camera surveillance system
WO2002062056A1 (en) 2001-01-29 2002-08-08 Koninklijke Philips Electronics N.V. Camera system and method for operating same
US20050259118A1 (en) * 2001-12-03 2005-11-24 Michael Mojaver Panoramic imaging and display system with canonical magnifier
US20030128756A1 (en) 2001-12-28 2003-07-10 Nokia Corporation Method and apparatus for selecting macroblock quantization parameters in a video encoder
US6763068B2 (en) 2001-12-28 2004-07-13 Nokia Corporation Method and apparatus for selecting macroblock quantization parameters in a video encoder
EP1341383A2 (en) 2002-02-28 2003-09-03 Sharp Kabushiki Kaisha Composite camera system, zoom camera image display control method, zoom camera control method, control program, and computer readable recording medium
US6757434B2 (en) 2002-11-12 2004-06-29 Nokia Corporation Region-of-interest tracking method and device for wavelet-based video coding
US20050122400A1 (en) * 2003-08-13 2005-06-09 Topcon Corporation Photographic apparatus with function of image correction and method thereof
US20050174437A1 (en) * 2004-02-10 2005-08-11 Fujitsu Limited Distortion correction circuit
WO2006070483A1 (en) 2004-12-28 2006-07-06 Nippon Beet Sugar Manufacturing Co., Ltd. Process for producing difructose dianhydride iii crystal
WO2009039083A1 (en) 2007-09-18 2009-03-26 Georgia-Pacific Consumer Products Lp Resilient, water dispersible polyurethane foams and products incorporating same

Non-Patent Citations (7)

* Cited by examiner, † Cited by third party
Title
"A System for Video Surveillance and Monitoring," in Proc. American Nuclear Society (ANS) Eighth International Topical Meeting on Robotics and Remote Systems, Pittsburgh, PA, Apr. 25-29, 1999 by Collins, Lipton and Kanade.
A M. Bagci, Y. Yardimci, A. E. Cetin, "Moving Object Detection Using Adaptive Subband Decomposition and Franctional Lower-Order Statistics in Video Sequences," Signal Processing, 82 (12): 1941-1947, Dec. 2002.
Aube, 12th International Conference on Automatic Fire Detection, 2001.
C. Stauffer, W. Grimson, "Adaptive Background Mixture Models for Real-Time Tracking." Proc. IEEE CS Conf. On Computer Vision and Pattern Recognition, vol. 2, 1999, pp. 246-252.
Comaniciu, D., Ramesh, V., and Meer, P., "Real-Time Tracking of Non-Rigid Objects Using Mean-shift," IEEE Computer Vision and Pattern Recognition, vol. 1 II, 2000, pp. 142-149.
X. Zhou, R. Collins, T. Kanade, and P. Metes, "A Master-Slave System to Acquire Biometric Imagery of Humans at Distance", ACM International Workshop on Video Surveillance, Nov. 2003.
Y. Yardimci, I. Yilmaz, A. E. Cetin, "Correlation Tracking Based on Wavelet Comain Information," Proceedings of SPIE vol. 5204, San Diego, Aug. 5-7, 2003.

Cited By (23)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8139896B1 (en) 2005-03-28 2012-03-20 Grandeye, Ltd. Tracking moving objects accurately on a wide-angle video
US20100020202A1 (en) * 2006-04-13 2010-01-28 Opt Corporation Camera apparatus, and image processing apparatus and image processing method
US20080239109A1 (en) * 2007-03-31 2008-10-02 David John Jennings Method and apparatus for correcting image distortion
US8264524B1 (en) * 2008-09-17 2012-09-11 Grandeye Limited System for streaming multiple regions deriving from a wide-angle camera
US20120002096A1 (en) * 2010-07-05 2012-01-05 Samsung Electronics Co., Ltd. Imaging device including a plurality of imaging units
US8531580B2 (en) * 2010-07-05 2013-09-10 Samsung Electronics Co., Ltd. Imaging device including a plurality of imaging units
WO2013002758A1 (en) 2011-06-27 2013-01-03 Oncam Global, Inc. Method and systems for providing video data streams to multiple users
WO2013192400A1 (en) 2012-06-22 2013-12-27 Oncam Global, Inc. Method and systems for providing video data streams to multiple users
US9826211B2 (en) * 2012-12-27 2017-11-21 Panasonic Intellectual Property Management Co., Ltd. Sound processing system and processing method that emphasize sound from position designated in displayed video image
US10244219B2 (en) 2012-12-27 2019-03-26 Panasonic Intellectual Property Management Co., Ltd. Sound processing system and sound processing method that emphasize sound from position designated in displayed video image
US20150350621A1 (en) * 2012-12-27 2015-12-03 Panasonic Intellectual Property Management Co., Ltd. Sound processing system and sound processing method
US10536681B2 (en) 2012-12-27 2020-01-14 Panasonic Intellectual Property Management Co., Ltd. Sound processing system and sound processing method that emphasize sound from position designated in displayed video image
US10909384B2 (en) 2015-07-14 2021-02-02 Panasonic Intellectual Property Management Co., Ltd. Monitoring system and monitoring method
JP2017134394A (en) * 2016-01-21 2017-08-03 パナソニックIpマネジメント株式会社 Lens system, and camera system having the same
US10007972B2 (en) * 2016-06-10 2018-06-26 Hyundai Motor Company Image processing apparatus and method for performing preprocessing to obtain image with improved sharpness
US20170358061A1 (en) * 2016-06-10 2017-12-14 Hyundai Motor Company Image processing apparatus and method for performing preprocessing to obtain image with improved sharpness
WO2018167581A1 (en) * 2017-03-15 2018-09-20 Corephotonics Ltd. Camera with panoramic scanning range
US11153482B2 (en) * 2018-04-27 2021-10-19 Cubic Corporation Optimizing the content of a digital omnidirectional image
US11153481B2 (en) * 2019-03-15 2021-10-19 STX Financing, LLC Capturing and transforming wide-angle video information
US11012621B2 (en) 2019-05-24 2021-05-18 Panasonic Intellectual Property Management Co., Ltd. Imaging device having capability of increasing resolution of a predetermined imaging area using a free-form lens
US11936975B2 (en) 2021-05-12 2024-03-19 Nio Technology (Anhui) Co., Ltd. Combined computer vision and human vision camera system
US20240005461A1 (en) * 2022-07-04 2024-01-04 Harman Becker Automotive Systems Gmbh Driver assistance system
EP4408006A1 (en) * 2023-01-27 2024-07-31 Canon Kabushiki Kaisha Image processing system, movable apparatus, image processing method, and storage medium

Similar Documents

Publication Publication Date Title
US7893985B1 (en) Wide angle electronic camera with improved peripheral vision
US5739852A (en) Electronic imaging system and sensor for use therefor with a nonlinear distribution of imaging elements
US11962906B2 (en) Image processing systems for correcting processed images using image sensors
US9635273B2 (en) Multi-imager video camera with frame-by-frame view switching
US7227573B2 (en) Apparatus and method for improved-resolution digital zoom in an electronic imaging device
US9077916B2 (en) Improving the depth of field in an imaging system
US7098949B2 (en) Apparatus and method for improved-resolution digital zoom in a portable electronic imaging device
JP5247702B2 (en) Image system with improved image quality and associated method
US10521891B2 (en) Image processing apparatus, system, image processing method, and non-transitory recording medium
KR101583646B1 (en) Method and apparatus for generating omnidirectional plane image
US20080158372A1 (en) Anti-aliasing in an imaging device using an image stabilization system
JP6119235B2 (en) Imaging control apparatus, imaging system, imaging control method, and program
CN111954893A (en) Constant resolution continuous hybrid zoom system
US20100020202A1 (en) Camera apparatus, and image processing apparatus and image processing method
JP7383911B2 (en) Imaging system, image processing device, imaging device and program
JP3262775B2 (en) Digital zoom-out processing device
US10922787B2 (en) Imaging apparatus and method for controlling imaging apparatus
JPH06292052A (en) Still picture image pickup device
WO1996008105A1 (en) Method for creating image data
JP2000341568A (en) Device, system and method for image pickup
JP3888847B2 (en) Wide-field imaging device
JP3805631B2 (en) Multifocal omnidirectional imaging device
CA2324802A1 (en) Panoramic imaging apparatus and method
WO2022226701A1 (en) Image processing method, processing apparatus, electronic device, and storage medium
JP4114633B2 (en) Image processing apparatus and method

Legal Events

Date Code Title Description
STCF Information on status: patent grant

Free format text: PATENTED CASE

FPAY Fee payment

Year of fee payment: 4

FEPP Fee payment procedure

Free format text: MAINTENANCE FEE REMINDER MAILED (ORIGINAL EVENT CODE: REM.); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY

FEPP Fee payment procedure

Free format text: 7.5 YR SURCHARGE - LATE PMT W/IN 6 MO, SMALL ENTITY (ORIGINAL EVENT CODE: M2555); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 8TH YR, SMALL ENTITY (ORIGINAL EVENT CODE: M2552); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY

Year of fee payment: 8

FEPP Fee payment procedure

Free format text: MAINTENANCE FEE REMINDER MAILED (ORIGINAL EVENT CODE: REM.); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY

LAPS Lapse for failure to pay maintenance fees

Free format text: PATENT EXPIRED FOR FAILURE TO PAY MAINTENANCE FEES (ORIGINAL EVENT CODE: EXP.); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY

STCH Information on status: patent discontinuation

Free format text: PATENT EXPIRED DUE TO NONPAYMENT OF MAINTENANCE FEES UNDER 37 CFR 1.362

FP Lapsed due to failure to pay maintenance fee

Effective date: 20230222