EP1478966B1 - Traitement optimise d'image pour systemes d'imagerie codes en front d'onde - Google Patents

Traitement optimise d'image pour systemes d'imagerie codes en front d'onde Download PDF

Info

Publication number
EP1478966B1
EP1478966B1 EP03711338A EP03711338A EP1478966B1 EP 1478966 B1 EP1478966 B1 EP 1478966B1 EP 03711338 A EP03711338 A EP 03711338A EP 03711338 A EP03711338 A EP 03711338A EP 1478966 B1 EP1478966 B1 EP 1478966B1
Authority
EP
European Patent Office
Prior art keywords
wavefront
image
processing
filter kernel
optical
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Lifetime
Application number
EP03711338A
Other languages
German (de)
English (en)
Other versions
EP1478966A1 (fr
Inventor
Gregory Edward Johnson
Edward R. Dowski
Ashley K. Macon
Hans B. Wach
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Omnivision CDM Optics Inc
Original Assignee
CDM Optics Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by CDM Optics Inc filed Critical CDM Optics Inc
Priority to EP07019276A priority Critical patent/EP1923731B1/fr
Publication of EP1478966A1 publication Critical patent/EP1478966A1/fr
Application granted granted Critical
Publication of EP1478966B1 publication Critical patent/EP1478966B1/fr
Anticipated expiration legal-status Critical
Expired - Lifetime legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/0025—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00 for optical correction, e.g. distorsion, aberration
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/42—Diffraction optics, i.e. systems including a diffractive element being designed for providing a diffractive effect
    • G02B27/46—Systems using spatial filters
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/70—Denoising; Smoothing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/73—Deblurring; Sharpening
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image

Definitions

  • Wavefront coding is a set of specialized aspheric optics, detection and signal processing.
  • the signal processing is dictated by the aspheric optics such that the spatial blur caused by wavefront coding is removed.
  • the aspheric optics "code" the wavefront so that the sampled image is relatively insensitive to misfocus-related aberrations.
  • the intermediate image from the coded wavefront i.e., the image sampled by the detector
  • Signal processing processes data from the detector to remove the spatial blur caused by the aspheric optics, to "decode" wavefront coding.
  • FIG. 1 shows one prior art wavefront coded optical imaging system 100.
  • System 100 includes optics 101 and a wavefront coded aspheric optical element 110 that cooperate to form an intermediate image at a detector 120.
  • Element 110 operates to code the wavefront within system 100.
  • a data stream 125 from detector 120 is processed by image processing section 140 to decode the wavefront and produce a final image 141.
  • a digital filter (a "filter kernel") 130 is implemented by image processing section 140.
  • Filter kernel 130 consists of filter taps, or "weights,” that are real or integer values with a maximum dynamic range due to the limits of processing hardware and software, for example 128-bit extended floating-point arithmetic, real number mathematics, and scaling/truncation of integer values. Scaling may for example include general series of partial products computations and accumulations.
  • optical element 110 and detector 120 do not necessarily match the available processing capability of image processing section 140; accordingly, image degradation may be noted within final image 141. If optical element 110 and detector 120 are matched to the available processing capability, it may require a very complex and costly hardware implementation in terms of detector capability, optical design of element 110, and/or computer processing architectures associated with image processing section 140.
  • wavefront coding imaging systems are disclosed with jointly optimized aspheric optics and electronics.
  • the optics and electronics are jointly optimized for desired optical and/or mechanical characteristics, such as to control aberrations, minimize physical lens count, loosen mechanical tolerances, etc.
  • the optics and electronics are jointly optimized with targeted emphasis on electronic parameters, so as to reduce the amount of silicon and/or memory required in hardware processing; this serves to optimize image quality in the presence of non-ideal detectors and/or to optimize image quality with a fixed hardware processing solution (e.g., a low cost solution).
  • An emphasis on optimizing electronic processing is for example important in imaging applications that involve high unit quantities and dedicated hardware processing. Examples of such applications include miniature cameras for cell phones, video conferencing, and personal communication devices. By jointly optimizing the optics and mechanics with the electronic parameters, high quality imaging systems are made that are inexpensive in terms of physical components, assembly, and image processing.
  • wavefront coded optical imaging systems are disclosed with optimized image processing.
  • the image processing is optimized for hardware implementations in which digital filters have filter tap values that are a specialized, reduced set of integers. These integers in turn affect the processing hardware to reduce complexity, size and/or cost. Examples of such reduced set filter tap values include "power of two" values, sums and differences of power of two values, and filters where differences of adjacent filter values are power of two. The restrictions associated with these example reduced sets operate to reduce the number of multiplications (or generalized partial products summations) associated with image processing.
  • the filter tap values may associate with the spatial location of the filter kernel.
  • the dynamic range of the filter tap values may also be a function of the spatial position relative to positioning of the optical elements and wavefront coding element. In one example, values near the geometric center of the filter kernel have a larger dynamic range and values near the edge of the filter kernel have a smaller dynamic range, useful when the wavefront coding response tends to converge toward small values as distance from the optical axis increases.
  • the filter tap values may also be a function of multi-color imaging channel characteristics since different wavelength bands respond differently when processed by a detector or a human eye.
  • ⁇ bi], and the cosinusoidal forms p(r,theta) Sum ai r ⁇ bi * cos(ci * theta + phii).
  • an image processing method including the steps of: wavefront coding a wavefront that forms an optical image; converting the optical image to a data stream; and processing the data stream with a reduced set filter kernel to reverse effects of wavefront coding and generate a final image.
  • the step of processing may include the step of utilizing a filter kernel that is complimentary to an MTF of the optical image.
  • the steps of wavefront coding, converting and processing may occur such that the MTF is spatially correlated to mathematical processing of the data stream with the reduced set filter kernel.
  • the method includes the step of formulating the reduced set filter kernel to an MTF of the optical image resulting from phase modification of the wavefront through constant path profile optics.
  • the method may include the step of processing the data with a reduced set filter kernel consisting of a plurality or regions wherein at least one of the regions has zero values.
  • the step of wavefront coding may include the step of wavefront coding the wavefront such that a PSF of the optical image spatially correlates to the regions of the reduced set filter kernel, wherein a substantial amount of information of the PSF is within the final image.
  • the method includes the step of modifying one of the wavefront coding, converting and processing steps and then optimizing and repeating (in a design loop) one other of the wavefront coding, converting and processing steps.
  • the step of processing may include utilizing a reduced set filter kernel with a weighted matrix.
  • an image processing method including the steps of: wavefront coding a wavefront that forms an optical image; converting the optical image to a data stream; and processing the data stream with a color-specific filter kernel to reverse effects of wavefront coding and generate a final image.
  • an image processing method including the steps of: wavefront coding a wavefront that forms an optical image; converting the optical image to a data stream; colorspace converting the datastream; separating spatial information and color information of the colorspace converted datastream into one or more separate channels; deblurring one or both of the spatial information and the color information; recombining the channels to recombine deblurred spatial information with deblurred color information; and colorspace converting the recombined deblurred spatial and color information to generate an output image.
  • the method may include a first step of filtering noise from the data stream.
  • the method generates an MTF of the optical image such that spatial correlation exists with the first step of filtering noise.
  • a second step of filtering noise may occur by processing the recombined deblurred spatial and color information. This second step of filtering noise may include the step of utilizing a complement of an MTF of the optical image.
  • an optical imaging system for generating an optical image.
  • a wavefront coding element codes a wavefront forming the optical image.
  • a detector converts the optical image to a data stream.
  • An image processor processes the data stream with a reduced set filter kernel to reverse effects of wavefront coding and generate a final image.
  • the filter kernel may be spatially complimentary to a PSF of the optical image.
  • a spatial frequency domain version of the filter kernel may be complimentary to a MTF of the optical image.
  • an electronic device including a camera having (a) a wavefront coding element that phase modifies a wavefront that forms an optical image within the camera, (b) a detector for converting the optical image to a data stream, and (c) an image processor for processing the data stream with a reduced set filter kernel to reverse effects of wavefront coding and generate a final image.
  • the electronic device is for example one of a cell phone and teleconferencing apparatus.
  • an image processing method includes the steps of: wavefront coding a wavefront with constant path profile optics that forms an optical image; converting the optical image to a data stream; and processing the data stream to reverse effects of wavefront coding and generate a final image.
  • FIG. 2 shows one wavefront coded optical imaging system 200 with optimized image processing.
  • System 200 includes optics 201 and a wavefront coded aspheric optical element 210 that cooperate to form an intermediate image at a detector 220.
  • Element 210 operates to code the wavefront within system 200; by way of example, element 210 is a phase mask that modifies phase of the wavefront.
  • a data stream 225 from detector 220 is processed by image processing section 240 to decode the wavefront and produce a final image 241.
  • System 200 has a filter kernel 230 with a reduced set of filter kernel values that optimally match the particular image processing implementation of image processing section 240.
  • Filter kernel 230 may be constructed and arranged with the reduced set of kernel values such that a final image 241 of system 200 is substantially equivalent to final image 141 of system 100.
  • certain examples of the reduced set of kernel values (i.e., tap or weight values) suitable for use with filter kernel 230 include: (i) powers of two, (ii) sums and differences of powers of two, (iii) a difference between adjacent taps restricted to a power of two, and (iv) taps with a range of values specified by spatial location or color of the image being processed.
  • FIG. 3 shows another wavefront coded optical imaging system 300 with optimized image processing.
  • System 300 includes optics 301 and a wavefront coded aspheric optical element 310 that cooperate to form an intermediate image at a detector 320.
  • Element 310 operates to code the wavefront within system 300.
  • a data stream 325 from detector 320 is processed by image processing section 340 to decode the wavefront and produce a final image 341.
  • Image processing section 340 processes data stream 325 using a filter kernel 330.
  • Detector 320, data stream 325, filter kernel 330 and image processing section 340 are constructed are arranged to optimize system 300 in terms of complexity, size, and/or cost.
  • Final image 341 may be substantially equivalent to final image 141, FIG. 1.
  • detector 320 and data stream 325 may include diagonal readouts for latter diagonal reconstruction, and/or multi-channel readouts for color filter array detectors.
  • Diagonal reconstruction is useful when performing processing on color filter arrays since many color interpolation algorithms operate on diagonal components in the image.
  • Color-specific filter kernels and processing offer reduced cost and improved performance as opposed to processing all colors equally.
  • Filter kernel 330 may be the same as filter kernel 230 of FIG. 2; though those skilled in the art appreciate that filter kernel 230 may include effects of color and directional (e.g., diagonal) convolution, such as described below.
  • FIG. 4 shows another wavefront coded optical imaging system 400 with optimized image processing.
  • System 400 includes optics 401 and a wavefront coded aspheric optical element 410 that cooperate to form an intermediate image at a detector 420.
  • Element 410 operates to code the wavefront within system 400.
  • a data stream 425 from detector 420 is processed by image processing section 440 to decode the wavefront and produce a final image 441.
  • Image processing section 440 processes data stream 425 using a filter kernel 430.
  • Image processing section 440 may be optimized such that final image 441 is substantially equivalent to final image 141, FIG. 1.
  • Optics 401 and image processing section 440 are jointly optimized for alternative reconstruction algorithms, such as diagonal processing or processing with a reduced set of kernel values, that facilitate higher performance at lower cost, size, etc.
  • FIG. 5 demonstrates one embodiment of a filter kernel 530 and an image processing section 540; kernel 530 and section 540 may for example be used within certain implementations of systems 200-400 above (e.g., to operate as filter kernel 430 and image processing section 440 of FIG. 4).
  • a detector 520 captures an intermediate image of the optical system, similar to detector 220, 320, 420 above; data stream 525 is input to image processing section 540, similar to data stream 225, 325, 425 above.
  • Filter kernel 530 is a reduced set filter kernel with values restricted to absolute values that are a power of two (including zero), such as ⁇ -2 N ,...,-4, -2, 1, 0, 1 , 2, 4, ..., 2 N ⁇ .
  • image processing section 540 implements the multiplication for filter kernel 530 through only shifting, as illustrated; only this operation is used since filter kernel 530 is limited to powers of two.
  • filter kernel 530 may be represented by the exponent or equivalently the shift for that coefficient.
  • digital filtering is a sum of products for both linear and non-linear filtering.
  • filter kernel 530 is applied to the intermediate blurred image sampled by detector 520.
  • filter kernel 530 and the intermediate blurred image as separate two-dimensional objects, with filter kernel 530 centered on a particular image pixel output from detector 520.
  • a product of the associated filter kernel value and image pixel is made; these products are then summed.
  • this sum is the final filtered image pixel.
  • filter kernel 530 is then similarly centered over each other pixel in the entire image to produce a like value for each pixel.
  • each filtered image pixel has N 2 products and N 2 -1 sums of these products.
  • Wavefront coded optical imaging systems 200, 300, 400 may thus be optimized with respective optics and filter kernels matched to particular hardware implementations of the image processing section so as to reduce the number of multipliers, sums and related implementation costs. Note that a kernel value of 0 requires only storage, so controlling sparseness within a kernel also reduces implementation cost.
  • FIG. 6A and FIG. 6B show, respectively, a schematic diagram of a general set filter kernel and a schematic diagram of a reduced set filter kernel (based on power of two values).
  • the general set filter kernel is not limited to coefficients that are powers of two; it thus requires a generalized arithmetic logic unit (ALU 550) to perform the product of each filter coefficient (i.e., the kernel value) and the image pixel, as shown.
  • the generalized ALU 550 performs an extended series of intermediate logic (known as partial products 552) and accumulations of the intermediate results (at accumulators 554) to form the final product (pixel out).
  • partial products 552 an extended series of intermediate logic
  • accumulations of the intermediate results at accumulators 554
  • the arithmetic logic used to form the product may be a shifter 560.
  • the amount of bits shifted for each image pixel is determined by the exponent of the power of two filter value. If for example the reduced set filter kernel consists of a sum of two power of two kernel values, then only two shifters and one adder are used in the arithmetic logic. Accumulations of all shifters 560 result in the final product (pixel out).
  • the implementation of FIG. 6B is thus simplified as compared to the generalized ALU 550 of FIG. 6A since the shifting and adding logic is predefined as a shift-add sequence, and not as a generalized partial products summation.
  • the arithmetic logic of FIG. 6B may favorably compare to other multiplication or generalized scaling implementations.
  • One example is integer-based look-up-table (LUT) multiplication.
  • Short bit-depth ALUs often implement scalar operands within the LUT to improve speed and simplify instruction sets since a series of partial product summations may consume more time and register space as compared to LUT resources.
  • the LUT multiplier thus allocates resources for generalized scaling operations.
  • the arithmetic logic of FIG. 6B employs a reduced set of scaling operations through bit-shifting logic using power of two coefficients.
  • the arithmetic logic of FIG. 6B is therefore many times smaller and faster than the generalized ALU 550 needed to process generalized filter kernel values.
  • FIG. 7A, FIG. 7B and FIG. 7C show further detail of the arithmetic logic in FIG. 6A and FIG. 6B, to compare generalized and specialized arithmetic logical functions used at each tap in a linear filter:
  • FIG. 7A schematically illustrates generalized ALU 550;
  • FIG. 7B and FIG 7C schematically illustrate alternate configurations of specialized ALU 560.
  • the generalized filter set kernel can have all values between 1 and 128.
  • the reduced set filter kernel can only have the values in this range that are a power of two, i.e.: 1, 2, 4, 8, 16, 32, 64, and 128.
  • ALU 560A has just one shift. Even a more sophisticated implementation of generalized ALU 550, such as a vector machine or pipelined device, would likely require more resources than such such a single shift.
  • FIG. 7C Another specialized ALU 560B of FIG. 7C depicts a shift-subtract sequence for a known non-power of two coefficient.
  • the coefficient is 127 for a pixel value p
  • FIG. 8 describes one wavefront coded optical imaging system 800 in terms of a one reduced set filter kernel.
  • System 800 includes optics 801 and a wavefront coded aspheric optical element 810 that cooperate to form an intermediate image at a detector 820.
  • Element 810 operates to code the wavefront within system 800.
  • a data stream 825 from detector 820 is processed by image processing section 840 to decode the wavefront and produce a final image 841.
  • Image processing section 840 processes data steam 825 using a filter kernel 830.
  • the hardware implementation of image processing section 840 may be optimized to optics 801, wavefront coding aspheric optical element 810 and detector 820.
  • Regions A, B, C, and D of filter kernel 830 illustrate the reduced geometric set.
  • values of filter kernel 830 are restricted to zero value in regions A and D, to a moderate range of values (or dynamic range) in region B, and to a large range of values in region C. All particular kernel values within the prescribed ranges are efficiently implemented from power of two values, sums and differences of power of two values, etc.
  • Wavefront coded optical imaging system 800 is thus optimized to the reduced geometric set of filter kernels implemented in hardware (e.g., via filter kernel 830 and image processing section 840).
  • Reduced geometric est kernels are particularly important in systems with specialized hardware processors are used with a variety of different wavefront coded optics and detectors, or used with optics that change characteristics, like wavefront coded zoom optics.
  • optics 801 and optical element 810 adjust the depth of field and also adjust scaling within the kernel size. The prescribed dynamic range is therefore achieved by positioning the coefficient values of filter kernel 830 in appropriate spatial location.
  • image processing section 840 is optimized for such optics 801 and element 810 given the constraints of the largest kernel. While different filter kernels have different values within the geometric regions A-D, the arrangement of values is optimized with respect to the capabilities of image processing section 840.
  • the rotation of optical element 810 e.g., by 45 degrees
  • FIG. 9A shows one wavefront coded optical imaging system 900A that is optimized for its detector and data stream readout.
  • System 900A includes optics 901 A and a wavefront coded aspheric optical element 910A that cooperate to form an intermediate image at a detector 920A.
  • Element 910A operates to code the wavefront within system 900A.
  • a data stream 925A from detector 920A is processed by image processing section 940A tc decode the wavefront and produce a final image 941 A.
  • Image processing section 940A processes data stream 925A using filter kernel 930A.
  • detector 920A and data stream 925A are not read out in typical row and column format; rather, the output format is for example a diagonal readout.
  • image processing section 940A does not utilize rectilinear processing (e.g., rectangularly separable or rank-N processing) since the image format is not row and column based.
  • optics 901A and wavefront coded aspheric optical element 910A are arranged such that the resulting image of a point object (or its point spread function) contains substantially all information along diagonals in a manner equivalent to the data stream diagonals.
  • the corresponding filter kernel 930A and image processing section 940A then optimally act on image data 925A in a diagonal format, as shown; and either, or both, diagonals can be processed.
  • the diagonals are remapped to row-column order and operated on in a rectilinear fashion, and then remapped again in the diagonally ordered output sequence.
  • optics 901A and optical element 910A are optimized with image processing section 940A to provide information in a geometrical orientation that reduces cost.
  • image processing section 940A One example is an optical element 910A that is rotated in a way that corresponds to the diagonal angle of data stream readout 925A.
  • FIG. 9B shows one wavefront coded optical imaging system 900B that is optimized for detector and data stream readout.
  • System 900B includes optics 901B and a wavefront coded aspheric optical element 910B that cooperate to form an intermediate image at a detector 920B.
  • Element 910B operates to code the wavefront within system 900B.
  • a data stream 925B from detector 920B is processed by image processing section 940B to decode the wavefront and produce a final image 941 A.
  • Image processing section 940B processes data stream 925B utilizing a filter kernel 930B.
  • Detector 920B and data stream 925B are not of typical rectilinear design and are not read out in typical row and column format.
  • the output format is, for example, an annular or radial-based readout from a log-polar detector.
  • image processing section 940B once again does not utilize rectilinear processing (e.g., rectangularly separable or rank-N processing) since the image format is not row and column based.
  • optics 901B and wavefront coded aspheric optical element 910B are arranged such that the resulting image of a point object (or its point spread function) contains substantially all information along annular or radial regions in a manner equivalent to data stream 925B.
  • the corresponding filter kernel 930B and image processing section 940B then optimally act on image data 925B in a circular format, as shown.
  • the concentric rings are remapped to row-column order and operated on in a rectilinear fashion, and then remapped again in the circular ordered output sequence.
  • optics 90 1 B and element 910B may be optimized with image processing section 940B to provide information in a geometrical orientation that reduces cost.
  • FIG. 10 describes one wavefront coded optical imaging system 1000 optimized for color imaging.
  • System 1000 includes optics 1001 and a wavefront coded aspheric optical element 1010 that cooperate to form an intermediate image at a detector 1020.
  • Element 1010 operates to code the wavefront within system 1000.
  • a data stream 1025 from detector 1020 is processed by image processing section 1040 to decode the wavefront and produce a final image 1041.
  • Image processing section 1040 utilizes a color-specific kernel filter 1030 in processing data stream 1025.
  • Optics 1001, aspheric optical element 1010, detector 1020, data stream 1025, filter kernel 1030, and image processing section 1040 cooperate such that each color channel has separate characteristics in producing final output image 1041.
  • wavefront coded optical imaging system 1000 takes advantage of the fact that a human eye "sees" each color differently.
  • the human eye is most sensitive to green illumination and is least sensitive to blue illumination. Accordingly, the spatial resolution, spatial bandwidth, and dynamic range associated with filter kernel 1030 and image processing section 1040 in the blue channel can be much less than for the green channel - without degrading the perceived image of image 1041 when viewed by a human.
  • the opto-mechanical and opto-electrical implementation of system 1000 is further optimized as compared to treating all color channels equivalently.
  • FIG. 11A through 11D show related embodiments of a wavefront coded optical imaging system 1100 that is optimized with specialized filtering (i.e., within an image processing section 1140).
  • Each system 1100A-1100D includes optics 1101 and a wavefront coded aspheric optical element 1110 that cooperate to form an intermediate image at a detector 1120.
  • Element 1110 operates to code the wavefront within system 1100.
  • a data stream 1125 from detector 1120 is processed by image processing section 1140 to decode the wavefront and produce a final image 1141.
  • Image processing section 1140 processes data stream 1125 with a filter kernel 1130.
  • Optics 1101, aspheric optical element 1110, detector 1120 and data stream 1125 cooperate such that a filter kernel 1130 utilizes specialized filtering and image processing section 1140 utilizes specialized taps 1145, providing desired cost savings and/or design characteristics.
  • FIG. 11A-11D illustrate four separate optimizations that may take into account any given coefficient, or sub-group of coefficients, to best implement a particular configuration. Any combination or combinations of the embodiments in FIG. 11 A through 11D can be implemented with an efficient image processing section.
  • FIG. 11 A specifically shows one embodiment of a gradient filtering operation.
  • Optics 1101A, aspheric optical element 1110A, detector 1120A and data stream 1125A cooperate such that a filter kernel 1130A utilizes gradient filtering and image processing section 1140A utilizes specialized differential taps 1145A.
  • the reduced set filter kernel of filter kernel 1130A is such that the difference between adjacent coefficients supports efficient implementation.
  • filter kernel 1130A and image processing section 1140A may be configured such that the difference between the first and second tap, between the second and third tap, between the third and forth tap, and so on, are all powers of two (or another efficiently implemented value or set of values).
  • each difference of the filter kernel 1130A is a reduced set of power of two values
  • the geometry of each implemented filter tap is a differential tap 1145A.
  • Differential filter tap 1145A is an efficient implementation because only a single summation is required for any coefficient value in kernel 1130A.
  • prior art linear filtering such as employed by FIR tap delay lines
  • only the original image pixel value is propagated to the next tap while the previous scaled value is not available to the next stage.
  • Recovery of the previously scaled value 1146A in the differential tap 1145A allows a single addition or subtraction to generate the next value. Since the prior result 1146A is propagated to the next tap, a savings is further realized in the size and cost of the hardware implementation of image processor section 1140A.
  • a differential tap 1145A is not limited to a single addition (or subtraction), rather those skilled in the art can recognize that any combination of additions and subtractions can be used in conjunction with carrying forward the previous result 1146A - it is the carrying-forward action 1146A that provides the ability to create an optimal solution, as opposed to the particular chosen design of the differential tap.
  • FIG. 11 B shows an embodiment of a scale-filtering operation.
  • Optics 1101B, aspheric optical element 1110B, detector 1120B and data stream 1125B cooperate such that a filter kernel 1130B utilizes scaled filtering and image processing section 1140B utilizes specialized scaling taps 1145B.
  • the reduced set filter kernel of filter kernel 1130B is such that the scale factor between adjacent coefficients supports efficient implementation.
  • filter kernel 1130B and image processing section 1140B may be configured such that the scale factor between the first and second tap, between the second and third tap, between the third and forth tap, and so on, are all factors of powers of two (or another efficiently implemented value or set of values).
  • each scale factor of the adjacent filter kernel coefficient is a reduced set of power of two values, for example, then the geometry of each implemented filter tap is a scaling tap 1145B and can be implemented with a simple shift on a binary computer (other scale factors may be efficiently implemented by other devices in a more efficient manner than powers of 2).
  • scaling filter tap 1145B is an efficient implementation because only the scaled value of an image pixel at any one tap is passed to both the next tap and the kernel accumulator.
  • prior art linear filtering such as employed by FIR tap delay lines, the original image pixel value has to propagate to the next tap while the scaled value is also propagated to the accumulator. Storage of two values (the delayed pixel value plus the scaled pixel value) requires two registers.
  • FIG. 11C shows an embodiment of a distributive-arithmetic property filtering operation.
  • Optics 1101C, aspheric optical element 1110C, detector 1120C and data stream 1125C cooperate such that a filter kernel 1130C utilizes distributive property of arithmetic filtering and image processing section 1140C utilizes specialized distributive arithmetic taps 1145C.
  • the tap 1145C performs accumulation of scaled values.
  • the reduced set filter kernel of filter kernel 1130C is such that the distribution of all coefficients supports efficient implementation.
  • filter kernel 1130C and image processing section 1140C may be configured such that there are only two scale factors or multipliers that are implemented, providing five distinct coefficients available for reconstruction (5 by considering positive and negative values of the two coefficients, plus the zero coefficient).
  • Scaling filter tap 1145C supports efficient implementation because a single tap can be "overclocked" to service many pixels. Certain implementations may prefer to overclock a multiplier as in 1145C, or some may to prefer to overclock an accumulator as in 1145D, FIG. 11D. In prior art linear filtering, such as employed by FIR tap delay lines, a separate and distinct operator was required for each pixel and coefficient.
  • FIG. 11D shows an embodiment of a distributive-arithmetic property filtering operation.
  • Optics 1101D, aspheric optical element 1110D, detector 1120D and data stream 1125D cooperate such that a filter kernel 1130D utilizes distributive-arithmetic filtering and image processing section 1140D utilizes specialized distributive-arithmetic taps 1145D.
  • the tap 1145D performs scaling of accumulated values.
  • the reduced set filter kernel of filter kernel 1130D is such that the distribution of all coefficients supports efficient implementation.
  • filter kernel 1130D and image processing section 1140D may be configured such that there are only two scale factors or multipliers that are implemented, providing 5 distinct coefficients available for reconstruction (5 by considering positive and negative values of the two coefficients, plus the zero coefficient).
  • scaling filter tap 1145D supports efficient implementation because a single tap can be "overclocked" to service many pixels. Certain implementations may to prefer to overclock an accumulator as in 1145D.
  • linear filtering such as employed by FIR tap delay lines, a separate and distinct operator was required for each pixel and coefficient.
  • One method of designing certain of the wavefront coded aspheric optical elements utilizes a plurality of weighted matrices that target image processing.
  • Other system design goals such as image quality, sensitivity to aberrations, etc., may also have associated weight matrices.
  • the weighted matrices may be optimally joined to achieve the selected goals. Since the optics and the signal processing can be jointly optimized, many types of solutions are reachable that are otherwise difficult or impossible to recognize if, for example, only one of the optics or signal processing is optimized.
  • one weighted matrix for a generalized integer filter (e.g., utilized by logic of.FIG. 6A) has values of zero for all possible integer values.
  • a weighted matrix for a power of two reduced set filter kernel (e.g., utilized by logic of FIG. 6B) has values of zero only for integers that are a power of two.
  • the integer zero has a zero weight value since this value is trivial to implement.
  • Other integers have higher weight values.
  • weight values are for integers that are not a power of two, but are above zero, the optimization between optics, filter kernel, image processing, and final image may occur to trade cost to performance by setting appropriate weight values above zero.
  • the general implementation e.g., FIG. 6A
  • requires three summations, i.e., 7 4 + 2 + 1.
  • the weighted matrix can also be used to optimize a gradient filter (e.g., filter 1130A, FIG. 11A) or a scaling filter (e.g., filter 1130B, FIG. 11 B).
  • Adjacent integer values may be given a weight based on their difference or gradient, or scale factor. In one example, every pair of integer values is assigned a zero value if their difference or scale is a power of two or otherwise a large value.
  • the pairing of coefficients also depends on the readout of pixels since the readout mode determines the adjacency rules for neighboring pixels and since both the readout and kernel orientation are included in the weighted matrix.
  • Other gradient kernel filters may assign small values when the difference of the integers is a sum of powers of two, and so on.
  • FIG. 12 illustratively shows a final image 1201 processed by system 100 of FIG. 1 and a final image 1202 processed by system 200 of FIG. 2.
  • both systems 100, 200 have the same optics 101, 201 and detector 120, 200, respectively, but have different filter kernels and image processing sections (implementing respective one.dimensional kernel filters within rectangularly separable processing).
  • Image 1201 was formed with a filter kernel 130 that is a generalized integer filter.
  • Image 1202 was formed with a filter kernel 230 that is a reduced set filter kernal utilizing power of two values. Both images 1201, 1202 are similar, but image 1202 has improved contrast, as shown.
  • reduced set filter kernel 230 has a smaller and less costly implementation then the generalized integer filter of filter kernel 130.
  • FIG. 13 illustratively shows a final image 1301 processed by system 100 of FIG. 1 and a final image 1302 processed by system 200 of FIG. 2.
  • both systems 100, 200 have equivalent optics and detectors 101, 201 and 120,220, respectively, but have different filter kernels and image processing sections (implementing respective one dimensional kernel filters within rectangularly separable processing).
  • System 200 employs a kernel filter 1130A, FIG. 11A, as kernel filter 230, FIG. 2.
  • System 100 employs a generalized integer filter as kernel filter 130; thus image 1301 was formed with the generalized integer filter.
  • Both images 1301, 1302 are similar, except that image 1302 has lower contrast. Since contrast is also one possible imaging goal, the reduced contrast can be set to meet this goal.
  • the reduced set gradient filter kernel used to process image 1302 can be smaller and less costly to implement then the generalized integer filter.
  • FIG. 14 shows the frequency response for the three digital filters used in FIG. 12 and FIG. 13.
  • Each filter has a slightly different frequency response and a different implementation. Certain imaging applications prefer one frequency response while another may prefer a different frequency response.
  • the particular reduced set power of two frequency response (e.g., employed in filter kernel 230, FIG. 2) has the highest contrast.
  • the generalized integer frequency response has the next highest contrast, and the reduced set gradient power of two filter kernel 1130 has the lowest frequency response in this example. This need not be the general case. Accordingly, depending on the application, the appropriate response is chosen; if the filter kernal also employs a reduced set of filter kernel values, then the implementation may also be made with reduced cost and complexity.
  • FIG. 15 thus illustrates one other optical imaging system with optimized color image processing.
  • System 1500 includes optics 1501 and a wavefront coded aspheric optical element 1510 that cooperate to form an intermediate image at a detector 1520.
  • Element 1510 operates to code the wavefront within system 1500.
  • a data stream 1525 from detector 1520 is then processed by a series of processing blocks 1522, 1524, 1530, 1540, 1552, 1554, 1560 to decode the wavefront and produce a final image 1570.
  • Block 1522 and block 1524 operate to preprocess data stream 1525 for noise reduction.
  • FPN block 1522 corrects for fixed pattern noise (e.g., pixel gain and bias, and nonlinearity in response) of detector 1510; pre-filtering block 1524 utilizes knowledge of wavefront coded optical element 1510 to further reduce noise from data stream 1525.
  • Color conversion block 1530 converts RGB color components (from data stream 1525) to a new colorspace.
  • Blur & filtering block 1540 removes blur from the new colorspace images by filtering one or more of the new colorspace channels.
  • Block 1552 and block 1554 operate to post-process data from block 1540, for further noise reduction.
  • SC Block 1552 filters noise within each single channel of data using knowledge of digital filtering within block 1540;
  • MC Block 1554 filters noise from multiple channels of data using knowledge of digital filtering within block 1540.
  • another color conversion block converts the colorspace image components back to RGB color components.
  • FIG. 16A schematically illustrates one color processing system 1600 that produces a final three-color image 1660 from a color filter array detector 1602.
  • System 1600 employs optics 1601 (with one or more wavefront coded optical elements or surfaces) to code the wavefront of system 1600 and to produce an intermediate optical image at detector 1602. Wavefront coding of optics 1601 thus forms a blurred image on detector 1602.
  • This intermediate image is then processed by NRP and colorspace conversion block 1620.
  • the noise reduction processing of block 1620 for example functions to remove detector non-linearity and additive noise; while the colorspace conversion of block 1620 functions to remove spatial correlation between composite images to reduce the amount of silicon and/or memory required for blur removal processing (in blocks 1642, 1644).
  • Data from block 1620 is also split into two channels: a spatial image channel 1632 and one or more color channels 1634.
  • Spatial image channel 1632 has more spatial detail than color channels 1634. Accordingly, the dominant spatial channel 1632 has the majority of blur removal within process block 1642.
  • the color channels have substantially less blur removal processing within process block 1644.
  • channels 1632 and 1634 are again combined and are processed within noise reduction processing and colorspace conversion block 1650. This further removes image noise accentuated by blur removal; and colorspace conversion transforms image into RGB for final image 1660.
  • FIG. 17 through FIG. 23 further illustrate colorspace conversion and blur removal processes for a particular embodiment of system 1600.
  • Detector 1602 is a CCD detector with a Bayer color filter array.
  • FIG. 17 shows red, green and blue component images 1700, 1702 and 1704, respectively, of an actual object imaged through system 1600 but without wavefront coding in optics 1601 (i.e., the wavefront coded optics are not present with optics 1601).
  • images 1700, 1702, 1704 it is apparent that each color image has a high degree of spatial similarity to each other image, and that many parts of the image are blurred.
  • the blurred images are not correctable without wavefront coding, as described below.
  • FIG. 18 shows raw red, green and blue component images 1800, 1802 and 1804, respectively, of the same object imaged through system 1600 with wavefront coded optics 1601.
  • Images 1800, 1802 and 1804 represent images derived from data stream 1625 and prior to processing by block 1620.
  • Each component image 1800, 1802, 1804 is again fairly similar to each other image; and each image is blurred due to aspheric optics 1601.
  • FIG. 19 shows the color component images of FIG 18 after blur removal processing of block 1642; image 1900 is image 1800 after block 1642; image 1902 is image 1802 after block 1644; and image 1904 is image 1804 after block 1644.
  • processing block 1644 performed no processing. Again, processing blocks 1620 and 1650 were not used.
  • Each component image 1900, 1902 and 1904 is sharp and clear; and the combination of all three component images results in a sharp and clear three color image.
  • FIG. 20 shows component images 2000, 2002, 2004 after colorspace conversion (block 1620) from RGB to YIQ colorspace, before processing of spatial and color channels 1632 and 1634.
  • Component images 2000, 2002, 2004 thus represent component images of FIG. 18 after processing by block 1620, on channel 1632 and channel 1634.
  • Y channel image 2000 is similar to the red and green images 1800, 1802 of FIG. 18.
  • the I and Q channel images 2002, 2004 are however much different from all channels of FIG. 18.
  • the YIQ colorspace conversion (block 1620) resulted in the Y channel component image 2000 containing the majority of the spatial information.
  • the I channel component image 2002 contains much less spatial information than does the Y channel, although some spatial information is visible. Notice the differences in the intensity scales to the right of each image.
  • the Q channel component image 2004 has essentially no spatial information at the intensity scale shown for these images.
  • FIG. 21 shows the component YIQ wavefront coded images 2100, 2102, 2104 after blur removal process 1642 of the spatial channel 1632.
  • the I and Q channel images 2002, 2004 were not filtered in block 1644, thereby saving processing and memory space; accordingly, images 2102, 2104 are identical to images 2002, 2004.
  • images 2002, 2004 could be filtered in block 1644 with low resolution and small bit depth filters.
  • the final three color image 1660 is sharp and clear with much less processing then required to produce the final image based on figure 19.
  • the YIQ colorspace is one of many linear and non-linear colorspace conversions available.
  • One method for performing colorspace conversion is to select a single space to represent the entire image.
  • Wavefront coded optics and processors can cooperate such that an optimal, dynamic, colorspace conversion can be achieved that reduces the amount of signal processing and improves image quality.
  • wavefront coded system 1600 may employ a colorspace process block 1620 that varies spatially across the entire image. Such spatial variation may be denoted as Dynamic Color Space (DCS).
  • DCS Dynamic Color Space
  • This spatially-varying colorspace conversion 1620 performs a single transformation on a region, where the region is dynamic and not constrained as enclosed.
  • an image with blue sky in the upper half and sandy beach in the lower half is well suited to the following split colorspace: a "blue-spatial-information-preserving" colorspace conversion for the upper half of the image and a "speckle-brown-information-preserving" colorspace transformation for the lower half.
  • Such spatially varying regions can be defined as an ordered set of rectilinear blocks, or as randomly assigned pixels, or as dynamically allocated contours that are defined in an optimal fashion as each scene changes.
  • Dynamic block allocation methods in conversion process 1620 can be sensitive to the total computational power and desired throughput and accuracy tradeoffs desired in the imaging system, and can be guided or limited by knowledge of the spatial correlating effects of optics 1601.
  • DCS can provide combined noise reduction, color-plane array interpolation, and reduced-cost image deblurring.
  • the DCS algorithm is appropriate for a variety of processing approaches within software and hardware based systems.
  • One version of DCS implements dynamic linear colorspace conversion.
  • Other versions of DCS implement dynamic non-linear colorspaces.
  • DCS may be combined or preceded by non-linear colorspace conversions or other transformations, such as HSV, depending on processing system resources and application goal.
  • FIG. 16B fits entirely within the block 1620 in FIG. 16A.
  • the Bayer sensor 1602 and the output image 1632 are numbered identically to FIG 16A to highlight the connection. Since DCS is a reversible transform, those skilled in the art recognize that an inverse, either exact or approximate, DCS process occurs entirely within block 1650 of FIG. 16A.
  • the determination of the principle components (or principal color or colors) from the estimated correlation of colors in multi-channel images of an arbitrary region of pixels with an arbitrary number of color channels (more than one) determines the optimal (in the least-squares sense) colorspace transformation for that particular region.
  • DCS allows blocks 1620 (and 1650 via the inverse DCS transformation) in FIG 16A to be optimized jointly with all other wavefront coding system parameters.
  • a color channel matrix is formed by collecting the sampled color channels into columns of a matrix 1603A.
  • Various schemes can be used to provide estimates of image color channels at each desired location. Not all pixel locations need to be estimated for the entire region.
  • Figure 16B shows a zero-order hold method that essentially copies nearby red and blue pixel values to every green pixel location, and generates 8 rows of data in the 8x3 matrix, from 16 pixels in the 4x4 region.
  • a principle component decomposition of the correlation estimate 1603B of the color channel matrix 1603A provides the colorspace transformation 1603C. This is only one such example stacking of values to form a color channel matrix for multi-channel color systems; those skilled in the art appreciate that other arrangements are possible without departing from the scope hereof.
  • An estimate of the principle components 1603C of the correlation matrix 1603B forms the optimal colorspace transformation matrix for this particular region and channel selection.
  • Obtaining the spatial intensity image 1632 in FIG. 16A is achieved by applying the first principle component to the color channel matrix 1603A in FIG 16B, forming the transformed spatial image 1632 in FIG. 16A.
  • the deblurring function 1642 in FIG. 16A then operates on the spatial image 1632, providing image reconstruction and simultaneous Bayer-pattern interpolation.
  • Returning to the original colorspace then occurs in block 1650 by re-transforming the reconstructed spatial channel using a form of an inverse of the color channel transformation matrices 1603C determined for each region, producing the final image 1660.
  • Color channel images 1634 may also be extracted from DCS by using the secondary and tertiary principle components in this example.
  • each block or region that was transformed contains it's own unique mapping matrix but all regions share a common spatial image 1642.
  • the optimal transformation for a given region does not achieve complete transformation of all spatial information to the spatial image 1642.
  • deblurring of the color channels may be made by blur removal process block 1644.
  • Final image assembly is then performed by "coloring" the dynamic regions using respective matrices of colorspace inversion 1650.
  • FIG. 22 shows the component images of FIG. 18 after colorspace conversion with the DCS algorithm. After conversion with the dynamically changing colorspace, essentially all spatial information is contained in channel 1 (2200). The other two color channels 2 (2202), 3 (2204) have essentially no spatial information relative to the shown intensity scale.
  • FIG. 23 shows these component images after filtering to remove image blur. Only channel 1 (2300) was filtered in this example, again due to low resolution in channels 2 (2302), 3 (2303). After conversion from DCS to RGB (process block 1650), the final image is again sharp and clear, and with fewer color artifacts as compared to the final image of FIG. 21 (and with less stringent processing and memory requirements).
  • FIG. 24 shows one such electronic device 2400 to illustrate such advantages.
  • Device 2400 has an optical lens 2402 that forms an image from object space 2404 onto a digital detector 2406 (e.g., a 3-color CMOS array), as shown.
  • Detector 2406 converts the image into a data stream 2408.
  • a microprocessor 2410 processes data stream to generate a final image 2412.
  • Optical lens 2402 is also wavefront coded; one surface 2414 of lens 2402 is for example a wavefront coded aspheric optical element. Accordingly, optical lens 2402 and element 2404 may function as the optics and wavefront coded elements discussed hereinabove (e.g., optics 201, element 210 of FIG. 2).
  • Microprocessor 2410 is for example a processing section discussed hereinabove (e.g., image processing section 240, FIG. 2).
  • Microprocessor 2410 employs a filter kernel in processing data stream 2408, thereby decoding the wavefront due to phase manipulation by lens 2402 and generating a crisp image 2412.
  • Image 2412 may for example be displayed on an LCD display 2415 or other display screen.
  • optical lens 2402 and post-processing by microprocessor 2410 are for example constructed and arranged to reduce mis-focus related aberrations such as defocus, field curvature, or manufacturing and assembly related mis-focus. Accordingly, electronic device 2400 may employ a single (plastic) optical element 2402 without the need for other complex optical elements.
  • microprocessor 2410 may operate with reduced processor load and memory requirement.
  • microprocessor 2410 employs image processing section 540 and kernel 530, FIG. 5, to reduce such processor load and memory requirement.
  • microprocessor 2410 employs logic architectures of FIG. 6B and/or FIG. 7 to reduce such processor load and memory requirement.
  • microprocessor 2410 may employ image processing section 840 and kernel 830 to achieve certain other advantages, as discussed in FIG. 8.
  • microprocessor 2410 may for example employ processing techniques disclosed in connection with FIG.
  • electronic device 2400 may employ opto-electronic components that implement imaging architectures of FIG. 9A or FIG. 9B, or one of FIG. 11 A-11 D.
  • electronic device 2400 may be constructed with a less expensive processor or with less memory.
  • memory savings may similarly translate to other memory devices or cards 2416, resulting in further savings.
  • electronic device 2400 may further include other electronics 2420 to incorporate other desired operation and functionality, for example cell phone functionality.
  • FIG. 25 illustrating components in a wavefront coded imaging system.
  • Component optics 2501 is for example optics 201 FIG. 2.
  • Phase mask 2410 is for example wavefront coded aspheric optical element 210, FIG. 2.
  • Detector component 2520 is for example detector 220, FIG. 2, while kernel filter 2530 is for example filter kernel 230, FIG. 2.
  • Image processing section 2540 is for example section 240, FIG. 2.
  • FIG. 25 illustrates one three-component optimization 2570 utilizing a reduced set filter kernel 2530 (e.g., FIG. 8); optimization 2570 thus ties together (in design) wavefront coded optics 2510, filter kernel 2530 and image processing section 2540.
  • a standard detector 2520 and optics 2501 is not necessarily optimized with optimization 2570, if desired.
  • one four-component optimization 2572 utilizes a detector 2520 with a particular readout format (e.g., as in FIG. 9A, 9B). In optimization 2572, therefore, wavefront coded optics 2510, filter kernel 2530 and image processing section 2540 are also tied together (in design).
  • optimizing color images e.g., as in FIG.
  • one two-component optimization 2574 can include a color-specific detector 2520 tied together (in design) with image processing section 2540.
  • optics 2501 may be optimized with any of optimizations 2570, 2572, 2574 to achieve imaging characteristics that support other optimizations, for example.
  • optics and optical elements used in the above-described imaging systems are also optimized. Such form is for example to provide low variability to misfocus-like aberrations, high MTFs, and low noise gain values.
  • these optics and optical elements may make up, or be part of, imaging systems such as microscopes, endoscopes, telescopes, machine vision systems, miniature cameras, and cell phone cameras, video cameras, digital cameras, barcode scanners, biometrics systems, etc.
  • ⁇ bi], and the cosinusoidal forms p(r,theta) Sum ai r ⁇ bi.* cos(ci * theta + phii).
  • the optics also may include specialized contour surfaces (sometimes denoted herein as "constant profile path optics") that provide phase variation within the wavefront coded optical element (e.g., element 210, FIG. 2).
  • the wavefront coded optical element and optics may be combined as a single optical element or system (e.g., utilizing lenses and/or mirrors).
  • the optics produce an MTF within the optical imaging system with a frequency domain that is complimentary to the kernel filter; for example, high power within the MTF implies low power in filter kernel.
  • the MTF and filter kernel may provide a degree of spatial correlation (e.g., between the filter kernel and wavefront coded aspheric optical element) such as described herein.
  • these wavefront coded imaging systems have non-traditional aspheric optics and image processing.
  • One possible goal for such systems is to produce a final image that is substantially insensitive to misfocus-like aberrations.
  • This insensitivity supports (a) a large depth of field or depth of focus, (b) tolerance to fabrication and/or assembly errors that for example create misfocus-like aberrations, and/or (c) optically-generated misfocus-like aberrations (e.g., spherical aberrations, astigmatism, petzval curvature, chromatic aberration, temperature-related misfocus).
  • Internal optics within such imaging systems form images with specialized blur that may also be insensitive to these misfocus-like aberrations.
  • image processing may operate to remove the blur associated with the intermediate image to produce a final image that is sharp and clear, and with, a high signal to noise ratio (SNR).
  • SNR signal to noise ratio
  • the design process to enable such wavefront coded imaging systems may be such that the optical system is insensitive to misfocus-like aberrations, the size and shape of the point spread function (PSF) is compact and constant as a function wavelength, field angle, object position, etc, and the resulting MTF has high values.
  • PSF point spread function
  • the following constant profile path optics provide efficient designs for use in such systems.
  • Certain constant profile path optics are based on parametrically efficient descriptions (e.g., low number of parameters) of optical form; such form may for example support high performance optical imaging systems with selected imaging and/or cost characteristics.
  • these parametrically efficient forms are not required. That is, the surface height of each part of the aspheric surface can be designated as an independent variable used in design and optimization; however the number of variables to optimize this general case is extremely large and impractical. Constant profile path optics thus facilitates a powerful and general optical form for use in design and optimization.
  • constant profile path optics include aspheric optical elements where the surface height is defined along paths and where the functional form, or profile, of the surface is the same along normalized version of the paths. The actual surface height varies from path to path, but the functional form or profile along each path does not.
  • Such surface profiles on an optical element operate to modify phase of a wavefront within the optical system; image effects caused by these phase changes is then reversed in image processing, e.g., through operation of the filter kernel.
  • the resulting MTF at the intermediate image correlates to the functional and spatial characteristics of the filter kernel used in image processing.
  • the constant profile path optics and the filter kernel may therefore be complimentary to one another to provide the desired imaging characteristics.
  • FIG. 26 Four examples 2600A-2600D of constant profile path optics are shown in FIG. 26.
  • the paths within profile 2600A are along square contours of the optics; for this optic form, the normalized surface height is the same over normalized versions of the square contours.
  • the paths of profile 2600B are pentagon-shaped.; the functional form of the surface height along a normalized version of each pentagon is the same.
  • the paths of profiles 2600C and 2600D are cross- and star-shaped, respectively. Similar to profiles 2600A, 2600B, such functional forms have common normalized surface heights along common paths.
  • FIG. 27 show other variations 2700A, 2700B in constant profile path optics.
  • Profile 2700B represents wavefront coded optics where the paths traces can have closed contours, such as shown by region #1.
  • the paths in profile 2700A on the other hand trace open contours.
  • region #1 contains a set of related paths.
  • the functional form of the surface height along a normalized version of each path in region #1 is the same. The same is true for regions 2, 3, 4, and 5.
  • the actual surface height along each path within each region can be different.
  • the functional forms over different regions can be related or not.
  • the optics of profile 2700B shows a combination of paths that are open and closed; the paths of region #1 trace closed contours while the paths of the other four regions trace open contours. Accordingly, the functional form of the surface height along each path may be the same for each region, yet the actual surface height can change within the region.
  • one of several variations of the path profiles shown in FIG. 26 and FIG. 27 may include non-straight line paths. That is, straight line paths such as shown in profiles of FIG. 26 and FIG. 27 may take other forms wherein the paths are composed of curved segments.
  • the pentagon shaped form of profile 2700A split into straight line regions may instead be of circular form with separate regions defining arcs as the paths.
  • the type of path traced in constant profile path optics can also change over different regions, as in FIG. 28.
  • the paths in the outer region of profile 2800A trace closed square contours, while its inner region paths trace closed circular contours. These inner and outer regions can be reversed, as shown by profile 2800B. Only some of the paths in the outer region need to be closed, as shown in 2800B.
  • FIG. 29 shows two profiles 2900A, 2900B where at least one region of the optics is not altered with constant profile path optics.
  • the paths form contours only in the outer region of the optics; the inner region has no paths and thus has no specialized surface shape.
  • the optics of profiles 2900A, 2900B may be thought of as the optics associated with profiles 2600A, 2600B, respectively, but with an amplitude of zero applied to the paths in the inner region.
  • the parameters that describe the specialized surfaces of FIG. 26-FIG. 29 may be considered as a composition of two components: 1) the parameters that describe the profile of the optical surface height along the paths of the particular optic and 2) the parameters that describe the surface height across the paths.
  • the second set of components may for example describe an amplitude change for each path in a given region. If the amplitude of the set of paths in a region is set to zero, then optics as in FIG. 29 may result.
  • Parameters 'a' and 'b' define the characteristics of the particular surface.
  • the contributions from C( a ) are constant over all paths in a region.
  • the contributions from D( b ) change for each path in a region.
  • Parameters C( a ) may define the surface along each path and the overall surface modulated between or across the paths in a region of D( b ).
  • the optical surface of a constant profile path optics may also be separable in terms along the paths and across the paths for the particular optic.
  • Each path can be modified by a gain or constant term without altering the functional form of the surface along each path.
  • the PathNumber parameter is a sequence of values that describes the set of paths in a region. The number of paths in a region may be large.
  • the overall surface is then mathematically composed of the product of the "along the paths" surface description and the "across the paths" surface description.
  • Constant power path optics are particularly effective forms for the wavefront coded optics.
  • the zeroth order term ao describes the amount of thickness
  • the first order term a 1 x describes the amount of tilt
  • the second order term a 2 x 2 describes the amount of optical power. If higher order terms are used, then the amount of optical power, or second derivative, can change along the contour. Higher order terms can also more accurately model optical power since a second order polynomial is an approximation to a sphere.
  • the tilt parameter is non-zero
  • optical surfaces with discontinuities may be described. Given the high degree of accuracy that free-form surfaces can be fabricated today, surfaces with discontinuities need not be avoided.
  • the across-the-paths term D is chosen so that the central region of the optical surface is flat (or nearly flat). Often the slope of D can become quite large near the edge of the surface.
  • the central region may also process optical imaging rays that need not change in order to control misfocus aberrations, as rays outside of the central region typically cause the misfocus effects within imaging systems.
  • FIG. 30 shows one surface profile 3000, its in-focus MTF 3002, and along the path surface form C( a ) 3004 and across the path amplitude D( b ) 3006.
  • a third order polynomial has been used for across the path amplitude 3006 and a second order polynomial has been used for the along the path surface form 3004.
  • This type of optics has an approximately constant amount of optical power along the normalized square path contours.
  • the surface profile 3000 and in-focus MTF 3002 have been drawn via contours of constant surface and MTF height. Notice that the constant height contours vary from roughly circular to rectangular along the diameter of the surface. The resulting in-focus MTF has non-circular contours that roughly follow a "+" shape.
  • the functional form in waves along the paths and across the path amplitude is shown in the bottom graphs 3004, 3006. About one wave of optical power is used along the sides of the paths (as shown by the minimum of 2.8 and a max of 3.8 waves) with an amplitude variation between -0.1 and -0.4 across the paths.
  • FIG. 31 Another example of constant profile path optics is shown in FIG. 31.
  • the polynomial form S(R,theta, a , b ) C( a )D( b ) is used, with C( a ) being a second order constant optical power polynomial and D( b ) being a third order polynomial.
  • the surface profile 3100 and MTF 3102 are very different from those of FIG. 30.
  • the MTF 3102 for this example is more compact then MTF 3002 of FIG. 30, with asymmetry also less pronounced.
  • About six waves of optical power are used along the sides of the path contours as shown in 3104, with an increasing amplitude used across the paths having a maximum value of about 4.
  • FIG. 32 shows another example that follows the form of FIG. 30, FIG. 31 with a second order constant optical power polynomial describing the functional form along the paths and a third order polynomial describing the across the path amplitude; except the form of the amplitude function changes to allow a flat center region.
  • the surface profile 3200 has a four-sided nature with a flat center, while its resulting MTF 3202 is roughly a four sided pyramid.
  • Such an MTF shape 3200 is suitable for non-uniform sampling, such as found on the green channel of three color Bayer color filter array detectors (see FIG. 16-23). Less then one wave of optical power is used along the paths 3204 with an amplitude variation from 0 to -9 used across the paths 3206.
  • FIG. 33 shows one example of a constant profile path optics with pentagon-shaped paths as in profile 2700 A, Fig 27.
  • the functional form of the surface is the same for each straight line segment within the five pentagon path regions.
  • a second order polynomial describes the surface along the paths (3304) while a third order polynomial describes the amplitude across the paths (3306).
  • the optical surface (profile 3300) has ten unequally shaped lobes and a fairly flat central region.
  • the corresponding MTF 3302 has near-circular form about the center of the 2D MTF plane. About 1.5 waves of optical power is used along the paths 3304 with an amplitude of zero to 18 used across the paths 3306.
  • FIG. 34 shows the sampled PSFs from the example of FIG. 32 without wavefront coding, and for a range of defocus values.
  • the small squares in each mesh drawing of each PSF represent individual pixel samples (output from the detector).
  • the physical parameters used to generate these PSFs are: 10 micron illumination wavelength; grayscale pixels with 25.4 micron a centers and 100% fill factor, working F-number of the optics is 1.24; misfocus aberration coefficient W 20 varies from 0 to 2 waves. Accordingly, it is apparent that without wavefront coding, the sampled PSFs have an unavoidable large change in size due to misfocus effects.
  • FIG. 35 similarly shows the sampled PSFs from the example of FIG. 32 but with wavefront coding. Notice that the sampled PSFs (for the range of defocus values) have a sharp spike and a broad pedestal that decreases with the distance from the sharp spike. Each of the PSFs is similar and essentially independent of misfocus. The small squares in the mesh drawing again represent individual pixels.
  • FIG. 36 and FIG. 37 show and compare cross sections through sampled PSFs from FIG. 34 and FIG. 35.
  • the PSFs of FIG. 36 and FIG. 37 are shown with five misfocus values evenly spaced between 0 and 2 waves.
  • the sampled PSFs in FIG. 36 are normalized to constant volume, while the PSFs in FIG. 37 are normalized for a constant peak value.
  • FIGS. 36 and 37 thus illustrate changes in the sampled PSFs of the system without wavefront coding and the lack of change in the sampled PSFs with wavefront coding.
  • FIG. 38 shows an example of one 2D digital filter (plotted as an image and based on zero misfocus PSF) that may be used to remove the wavefront coding blur from the sampled PSFs of FIG. 35.
  • This digital filter is for example used within image processing (e.g., within section 240, FIG. 2) as a filter kernel. Notice that this kernel has high values only near the center of the filter and decreasing values related to the distance from the center.
  • the filter, and the sampled-PSFs of FIG 35, are spatially compact.
  • FIG. 40 illustrates an amount of power contained in a geometric concept known as "rank" for the in-focus sampled PSF and the digital filter of FIG. 35 and FIG. 38, respectively.
  • This separable nature allows rectangularly separable filtering that is computationally efficient.
  • One drawback of rectangularly separable optics is that the PSF can spatially shift with misfocus.
  • Constant profile path optics may also produce PSFs, MTFs and corresponding digital filters that approximate low rank, to further facilitate efficient processing.
  • the top plot of FIG. 39 shows that there are only two geometric ranks that have appreciable value for this particular sampled PSF; this system is thus approximated by a rank two system.
  • the corresponding digital filter for this example (the lower plot of FIG. 39) is also low rank and can be made at least as low as the sampled PSF.
  • the filtering of this example PSF may be accomplished with computationally efficient separable filtering.
  • FIG. 41 shows the corresponding MTFs before and after filtering and in the spatial frequency domain.
  • the MTFs of the identical physical system with no wavefront coding are shown with misfocus values from 0 to 2 waves in five steps.
  • the MTFs of the system with no wavefront coding is seen to drastically change with misfocus.
  • the MTFs of the system with the constant profile path optics before filtering has essentially no change with misfocus.
  • the filtered MTFs result from filtering by the 2D digital filter of FIG. 38.; such MTFs have high values and degrade only for the largest misfocus value.
  • other forms of constant profile path optics may control the MTF profile over larger amounts of misfocus by utilizing more processing capability and/or lower MTFs before filtering.
  • the mathematical form of the contour optics of FIG. 32 may be as follows.
  • C x 0.4661 - 0.7013 ⁇ x ⁇ ⁇ 2 , x ⁇ 1
  • the form along the path contours is given by an even second-order polynomial; and the amplitude across the paths is given by a third order polynomial that changes form so that the central region is relatively flat.
  • Using higher order polynomials for along the paths and for the across the path amplitude supports higher performance then shown in these examples.
  • Constant profile path optics may be designed with specialized techniques that allow optimization with non-ideal optics that can have large amounts of aberrations, vignetting, and loose tolerances. These techniques allow optical system design that is not practical by traditional analytical methods.
  • FIG. 42 illustrates a design process 4200 that illustrates certain of these techniques.
  • process 4200 optics are modified (in design) and the loop may repeat, as shown.
  • process 4200 includes step 4202 to design constant profile path optics.
  • the effects of the lens aberrations can be added (step 4204) with information about the digital detector (step 4206) in order to accurately simulate the sampled PSF and MTF.
  • These lens aberrations are in general a function of field angle, wavelength, object position and zoom position; the aberrations can also be a function of the particular surface form of the constant profile path optic (step 4202).
  • One aberration considered in design process step 4204 is vignetting.
  • vignetting is often used in design of traditional optics to trade off light gathering for sharpness
  • vignetting within wavefront coding optics can lead to a poor match between optical design and the actual optical system.
  • the optical surfaces and aberrations that lead to sampled PSFs can be either simulated through ray-based methods or Fourier Transform methods depending on the speed of the lens being designed and the spatial resolution of the digital detector. Both of these general types of PSF simulation methods are well known to those skilled in the art of optical simulation.
  • a digital filter is used (step 4208) to remove wavefront coding blur.
  • This digital filter can be general and calculated for each iteration in design process 4200 or can be fixed or limited in form.
  • An example of a limited form digital filter is a rectangularly separable filter. If the digital filter is limited in this way, the design of the constant profile path optics may be optimized for minimum rank PSFs and MTFs since the rank of the separable filter is 1.
  • Other limited digital filters are filters where costs are assigned to particular filter values and sequences of filter values in order to optimize the implementation (e.g., hardware) of the image processing section. These costs may be reduced or increased as part of design process 4200.
  • limited digital filters are those that have particular power spectrum characteristics. Since the additive noise is modified by the power spectrum of the digital filter, controlling the characteristics of the filter controls the characteristics of the additive noise after digital filtering. Noise Reduction techniques can be optimized jointly with the characteristics of the noise by limited the digital filter.
  • a quality assessment is performed (step 4210).
  • Assessment 4210 is typically system and application specific but may include (a) the quality of the filtered PSFs/MTFs both within and outside of the design range and/or (b) the characteristics of the digital filter (and/or its implementation and/or noise effects).
  • the quality assessment is then used with non-linear optimization to modify the optics and repeat the iteration through process 4200.
  • the modified optics can include particular surfaces that contain the wavefront coding as well as other surfaces, and/or thickness and distances of elements within the optical system. The parameters of the functional form along the paths and across the path amplitude can be similarly optimized.

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Optics & Photonics (AREA)
  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)
  • Studio Devices (AREA)
  • Color Television Image Signal Generators (AREA)
  • Picture Signal Circuits (AREA)

Claims (39)

  1. Procédé de traitement d'image, comportant les étapes de :
    codage de front d'onde d'un front d'onde qui forme une image optique ;
    conversion de l'image optique en un flux de données ; et
    traitement du flux de données à l'aide d'un noyau de filtre à ensemble réduit pour inverser les effets de codage de front d'onde et générer une image finale.
  2. Procédé de traitement d'image selon la revendication 1, dans lequel l'étape de traitement comporte l'étape d'utilisation d'un noyau de filtre qui est complémentaire d'un spectre de fréquence spatiale de l'image optique.
  3. Procédé de traitement d'image selon la revendication 2, dans lequel les étapes de codage de front d'onde, de conversion et de traitement ont lieu de sorte que le spectre de fréquence spatiale est spatialement corrélé à un traitement mathématique du flux de données avec le noyau de filtre à ensemble réduit.
  4. Procédé selon la revendication 1, dans lequel l'étape de codage de front d'onde comporte l'étape d'utilisation d'un élément optique ayant subi un codage de front d'onde asphérique.
  5. Procédé selon la revendication 1, dans lequel l'étape de codage de front d'onde comporte un codage de front d'onde du front d'onde à l'aide d'optiques à trajet de profil constant.
  6. Procédé selon la revendication 5, comportant de plus l'étape de formulation du noyau de filtre à ensemble réduit en un spectre de fréquence spatiale de l'image optique résultant d'une modification de phase du front d'onde par l'intermédiaire des optiques à trajet de profil constant.
  7. Procédé selon la revendication 1, dans lequel l'étape de traitement comporte l'étape de traitement de l'image, pour chaque pixel, à l'aide d'une logique de prise de filtre constituée d'un déphaseur.
  8. Procédé selon la revendication 1, dans lequel l'étape de traitement comporte l'étape de traitement de l'image, pour chaque pixel, à l'aide d'une logique de prise de filtre constituée d'un déphaseur et d'un additionneur.
  9. Procédé selon la revendication 1, dans lequel l'étape de traitement comporte l'étape de traitement du flux de données à l'aide d'un noyau de filtre à ensemble réduit constitué d'un ou de plusieurs parmi (a) des coefficients de puissance deux, (b) des sommes de coefficients de puissance deux, et (c) des différences de coefficients de puissance deux.
  10. Procédé selon la revendication 1, dans lequel l'étape de traitement comporte l'étape de traitement du flux de données à l'aide d'un noyau de filtre à ensemble réduit constitué d'une pluralité de régions dans laquelle au moins l'une des régions a des valeurs nulles.
  11. Procédé selon la revendication 10, dans lequel l'étape de codage de front d'onde comporte l'étape de codage de front d'onde du front d'onde de sorte qu'une fonction du dispersion des points (PSF) est corrélée spatialement aux régions du noyau de filtre à ensemble réduit, dans lequel une quantité importante d'informations de la PSF se trouve dans l'image finale.
  12. Procédé selon la revendication 1, dans lequel l'étape de conversion de l'image optique en un flux de données comporte l'émission de données à partir d'un détecteur dans un format non-rectiligne.
  13. Procédé selon la revendication 12, dans lequel le format comporte l'une parmi une acquisition en diagonale et une acquisition circulaire.
  14. Procédé selon la revendication 12, dans lequel l'étape de traitement comporte le traitement du flux de données à l'aide d'un noyau de filtre à ensemble réduit adapté au format non-rectiligne.
  15. Procédé selon la revendication 14, comportant de plus l'étape de remappage de données dans un format rectiligne.
  16. Procédé selon la revendication 12, dans lequel l'étape de codage de front d'onde comporte le codage de front d'onde du front d'onde de sorte que l'image optique dans le détecteur coïncide sensiblement avec le format non-rectiligne du détecteur.
  17. Procédé selon la revendication 1, dans lequel l'étape de traitement comporte les étapes d'utilisation d'un noyau de filtre à gradient d'ensemble réduit et une série de prises différentielles à décalage unique.
  18. Procédé selon la revendication 17, dans lequel l'étape de traitement comporte l'étape de traitement d'une seule somme pour toute valeur de coefficient du noyau de filtre.
  19. Procédé selon la revendication 17, dans lequel l'étape de codage de front d'onde comporte l'étape de codage de front d'onde du front d'onde de sorte qu'une fonction PSF contient sensiblement des informations de l'image optique dans un motif spatial correspondant au noyau de filtre.
  20. Procédé selon la revendication 1, dans lequel l'étape de traitement comporte les étapes d'utilisation d'un noyau de filtre de mise à l'échelle à jeu réduit et d'une série de prises de mise à l'échelle.
  21. Procédé selon la revendication 20, dans lequel l'étape de traitement comporte l'étape de traitement à l'aide d'une seule somme pour toute valeur de coefficient du noyau de filtre.
  22. Procédé selon la revendication 20, dans lequel l'étape de codage de front d'onde comporte l'étape de codage de front d'onde du front d'onde de sorte qu'une fonction PSF contient sensiblement des informations dans un motif spatial correspondant au noyau de filtre.
  23. Procédé selon la revendication 1, dans lequel l'étape de traitement comporte les étapes d'utilisation d'un noyau de filtre distributif à ensemble réduit et une série de prises d'échelle-ajout de propriété distributives.
  24. Procédé selon la revendication 23, dans lequel l'étape de traitement comporte l'étape de traitement d'une seule somme pour toute valeur de coefficient du noyau de filtre.
  25. Procédé selon la revendication 23, dans lequel l'étape de codage de front d'onde comporte l'étape de codage de front d'onde du front d'onde de sorte qu'une fonction PSF contient sensiblement des informations de l'image optique dans un motif spatial correspondant au noyau de filtre.
  26. Procédé selon la revendication 1, dans lequel l'étape de traitement comporte les étapes d'utilisation d'un noyau de filtre distributif à ensemble réduit et d'une série de prises d'ajout-échelle de propriété distributives.
  27. Procédé selon la revendication 26, dans lequel l'étape de traitement comporte l'étape de traitement à l'aide d'une seule somme pour toute valeur de coefficient du noyau de filtre.
  28. Procédé selon la revendication 26, dans lequel l'étape de codage de front d'onde comporte l'étape de codage de front d'onde du front d'onde de sorte qu'une fonction PSF contient sensiblement des informations de l'image optique dans un motif spatial correspondant au noyau de filtre.
  29. Procédé selon la revendication 1, comportant de plus l'étape de sélection d'une réponse en fréquence voulue de l'image finale en sélectionnant le noyau de filtre à ensemble réduit.
  30. Procédé selon la revendication 1, comportant de plus l'étape de modification de l'une des étapes de codage de front d'onde, de conversion et de traitement et ensuite d'optimisation et de répétition d'une autre étape des étapes de codage de front d'onde, de conversion et de traitement.
  31. Procédé selon la revendication 30, dans lequel l'étape de traitement comporte l'utilisation d'un noyau de filtre à ensemble réduit comportant une matrice de poids.
  32. Système d'imagerie optique pour générer une image optique, comportant :
    un élément de codage de front d'onde qui code un front d'onde formant l'image optique ;
    un détecteur pour convertir l'image optique en un flux de données ; et
    un processeur d'image pour traiter le flux de données à l'aide d'un noyau de filtre à ensemble réduit pour inverser les effets de codage de front d'onde et générer une image finale.
  33. Système d'imagerie optique selon la revendication 32, dans lequel le noyau de filtre est spatialement complémentaire d'une fonction PSF.
  34. Système d'imagerie optique selon la revendication 32, dans lequel une version en domaine de fréquence spatiale du noyau de filtre est complémentaire d'un spectre de fréquence spatiale de l'image optique.
  35. Système d'imagerie optique selon la revendication 32, dans lequel l'élément de codage de front d'onde, le détecteur et le processeur d'image coopèrent de sorte que l'image optique est spatialement corrélée à un traitement mathématique du flux de données avec le noyau de filtre à ensemble réduit.
  36. Système d'imagerie optique selon la revendication 32, comportant des moyens pour exécuter les étapes d'au moins l'une des revendications 4 à 30.
  37. Système d'imagerie optique selon la revendication 32, dans lequel l'élément de codage de front d'onde comporte des optiques à trajet de profil constant.
  38. Système d'imagerie optique selon la revendication 37, dans lequel le noyau de filtre à ensemble réduit est complémentaire d'une fonction PSF résultant d'une modification de phase du front d'onde par l'intermédiaire des optiques à trajet de profil constant.
  39. Système d'imagerie optique selon la revendication 32, dans lequel le noyau de filtre à ensemble réduit comporte une matrice de poids.
EP03711338A 2002-02-27 2003-02-27 Traitement optimise d'image pour systemes d'imagerie codes en front d'onde Expired - Lifetime EP1478966B1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP07019276A EP1923731B1 (fr) 2002-02-27 2003-02-27 Traitement d'image optimisé pour systèmes d'imagerie codés du front d'onde

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US36014702P 2002-02-27 2002-02-27
US360147P 2002-02-27
PCT/US2003/006289 WO2003073153A1 (fr) 2002-02-27 2003-02-27 Traitement optimise d'image pour systemes d'imagerie codes en front d'onde

Related Child Applications (1)

Application Number Title Priority Date Filing Date
EP07019276A Division EP1923731B1 (fr) 2002-02-27 2003-02-27 Traitement d'image optimisé pour systèmes d'imagerie codés du front d'onde

Publications (2)

Publication Number Publication Date
EP1478966A1 EP1478966A1 (fr) 2004-11-24
EP1478966B1 true EP1478966B1 (fr) 2007-11-14

Family

ID=27766194

Family Applications (1)

Application Number Title Priority Date Filing Date
EP03711338A Expired - Lifetime EP1478966B1 (fr) 2002-02-27 2003-02-27 Traitement optimise d'image pour systemes d'imagerie codes en front d'onde

Country Status (8)

Country Link
US (4) US7379613B2 (fr)
EP (1) EP1478966B1 (fr)
JP (2) JP2005519361A (fr)
CN (2) CN101118317B (fr)
AT (1) ATE378619T1 (fr)
AU (1) AU2003213651A1 (fr)
DE (1) DE60317472T2 (fr)
WO (1) WO2003073153A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TWI480583B (zh) * 2011-05-31 2015-04-11 Omnivision Tech Inc 以色相關波前編碼延伸透鏡系統景深的系統及其方法

Families Citing this family (132)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020195548A1 (en) 2001-06-06 2002-12-26 Dowski Edward Raymond Wavefront coding interference contrast imaging systems
US6842297B2 (en) * 2001-08-31 2005-01-11 Cdm Optics, Inc. Wavefront coding optics
US8717456B2 (en) 2002-02-27 2014-05-06 Omnivision Technologies, Inc. Optical imaging systems and methods utilizing nonlinear and/or spatially varying image processing
EP1672912B1 (fr) 2003-01-16 2012-03-14 DigitalOptics Corporation International Procédé de fabrication d'un système optique comprenant un processeur électronique pour l'amélioration d'image
US7627193B2 (en) * 2003-01-16 2009-12-01 Tessera International, Inc. Camera with image enhancement functions
US8294999B2 (en) 2003-01-16 2012-10-23 DigitalOptics Corporation International Optics for an extended depth of field
US7773316B2 (en) * 2003-01-16 2010-08-10 Tessera International, Inc. Optics for an extended depth of field
US20070236573A1 (en) * 2006-03-31 2007-10-11 D-Blur Technologies Ltd. Combined design of optical and image processing elements
US7180673B2 (en) 2003-03-28 2007-02-20 Cdm Optics, Inc. Mechanically-adjustable optical phase filters for modifying depth of field, aberration-tolerance, anti-aliasing in optical systems
JP4565192B2 (ja) * 2003-03-31 2010-10-20 オムニビジョン テクノロジーズ， インコーポレイテッド 画像システムにおける収差を生じる影響を最小化するための、システムおよび方法
US7088419B2 (en) 2003-05-30 2006-08-08 Cdm Optics, Inc. Lithographic systems and methods with extended depth of focus
US7606417B2 (en) 2004-08-16 2009-10-20 Fotonation Vision Limited Foreground/background segmentation in digital images with differential exposure calculations
US7680342B2 (en) * 2004-08-16 2010-03-16 Fotonation Vision Limited Indoor/outdoor classification in digital images
US7876974B2 (en) * 2003-08-29 2011-01-25 Vladimir Brajovic Method for improving digital images and an image sensor for sensing the same
US8254714B2 (en) * 2003-09-16 2012-08-28 Wake Forest University Methods and systems for designing electromagnetic wave filters and electromagnetic wave filters designed using same
WO2005054927A2 (fr) * 2003-12-01 2005-06-16 Cdm Optics, Inc. Systeme et procede pour optimiser des modeles de systemes optiques et numeriques
US7944467B2 (en) * 2003-12-01 2011-05-17 Omnivision Technologies, Inc. Task-based imaging systems
US20050147313A1 (en) * 2003-12-29 2005-07-07 Dimitry Gorinevsky Image deblurring with a systolic array processor
US7412110B1 (en) 2004-01-28 2008-08-12 Adobe Systems Incorporated Using forward and backward kernels to filter images
US7359576B1 (en) 2004-02-27 2008-04-15 Adobe Systems Incorporated Using difference kernels for image filtering
US7216811B2 (en) * 2004-04-16 2007-05-15 Microscan Systems Incorporated Barcode scanner with linear automatic gain control (AGC), modulation transfer function detector, and selectable noise filter
EP1787463A1 (fr) * 2004-09-09 2007-05-23 Nokia Corporation Procede pour creer une image couleur, dispositif d'imagerie et module d'imagerie
WO2007011375A1 (fr) * 2004-09-13 2007-01-25 Cdm Optics, Inc. Dispositifs de capture d'images d'iris et systemes associes
EP1793587B8 (fr) * 2004-09-14 2012-12-26 Fujitsu Ltd. Processeur d'image, méthode de traitement d'image et programme de traitement d'image
US7566004B2 (en) * 2004-10-29 2009-07-28 Symbol Technologies Inc. Method and apparatus for extending the range of a product authentication device
FR2881011B1 (fr) * 2005-01-19 2007-06-29 Dxo Labs Sa Procede de realisation d'un appareil de capture et/ou restitution d'images et appareil obtenu par ce procede
KR100960577B1 (ko) 2005-02-08 2010-06-03 오블롱 인더스트리즈, 인크 제스처 기반의 제어 시스템을 위한 시스템 및 방법
WO2006102201A1 (fr) * 2005-03-18 2006-09-28 Cdm Optics, Inc. Systemes d'imagerie a modulateurs spatiaux de lumiere pixelises
US20060269150A1 (en) * 2005-05-25 2006-11-30 Omnivision Technologies, Inc. Multi-matrix depth of field image sensor
US7616841B2 (en) 2005-06-17 2009-11-10 Ricoh Co., Ltd. End-to-end design of electro-optic imaging systems
JP2007047228A (ja) * 2005-08-05 2007-02-22 Olympus Corp 結像光学装置、及び光学ユニット
KR20080035689A (ko) * 2005-08-11 2008-04-23 글로벌 바이오닉 옵틱스 피티와이 엘티디 광학 렌즈 시스템
WO2008008084A2 (fr) 2005-09-19 2008-01-17 Cdm Optics, Inc. Systèmes d'imagerie basés sur des tâches
US20070081224A1 (en) * 2005-10-07 2007-04-12 Robinson M D Joint optics and image processing adjustment of electro-optic imaging systems
US20070093993A1 (en) 2005-10-20 2007-04-26 Stork David G End-to-end design of electro-optic imaging systems using backwards ray tracing from the detector to the source
US7692696B2 (en) * 2005-12-27 2010-04-06 Fotonation Vision Limited Digital image acquisition system with portrait mode
US8370383B2 (en) 2006-02-08 2013-02-05 Oblong Industries, Inc. Multi-process interactive systems and methods
US9823747B2 (en) 2006-02-08 2017-11-21 Oblong Industries, Inc. Spatial, multi-modal control device for use with spatial operating system
US9075441B2 (en) * 2006-02-08 2015-07-07 Oblong Industries, Inc. Gesture based control using three-dimensional information extracted over an extended depth of field
US9910497B2 (en) * 2006-02-08 2018-03-06 Oblong Industries, Inc. Gestural control of autonomous and semi-autonomous systems
US8407725B2 (en) 2007-04-24 2013-03-26 Oblong Industries, Inc. Proteins, pools, and slawx in processing environments
US8531396B2 (en) 2006-02-08 2013-09-10 Oblong Industries, Inc. Control system for navigating a principal dimension of a data space
US8537111B2 (en) 2006-02-08 2013-09-17 Oblong Industries, Inc. Control system for navigating a principal dimension of a data space
IES20060558A2 (en) * 2006-02-14 2006-11-01 Fotonation Vision Ltd Image blurring
US7469071B2 (en) 2006-02-14 2008-12-23 Fotonation Vision Limited Image blurring
CN101449193B (zh) * 2006-03-06 2011-05-11 全视Cdm光学有限公司 具有波前编码的变焦透镜系统
US20070236574A1 (en) * 2006-03-31 2007-10-11 D-Blur Technologies Ltd. Digital filtering with noise gain limit
US20070239417A1 (en) * 2006-03-31 2007-10-11 D-Blur Technologies Ltd. Camera performance simulation
US7911501B2 (en) * 2006-04-03 2011-03-22 Omnivision Technologies, Inc. Optical imaging systems and methods utilizing nonlinear and/or spatially varying image processing
US8514303B2 (en) * 2006-04-03 2013-08-20 Omnivision Technologies, Inc. Advanced imaging systems and methods utilizing nonlinear and/or spatially varying image processing
CN101473439B (zh) * 2006-04-17 2013-03-27 全视技术有限公司 阵列成像系统及相关方法
WO2008020899A2 (fr) * 2006-04-17 2008-02-21 Cdm Optics, Inc. Systèmes d'imagerie en réseau et procédés associés
IES20060564A2 (en) 2006-05-03 2006-11-01 Fotonation Vision Ltd Improved foreground / background separation
EP2016454A1 (fr) * 2006-05-09 2009-01-21 Omnivision Cdm Optics, Inc. Système d'imagerie dans l'infrarouge lointain tolérant aux aberrations
US7692709B2 (en) * 2006-05-12 2010-04-06 Ricoh Co., Ltd. End-to-end design of electro-optic imaging systems with adjustable optical cutoff frequency
US7889264B2 (en) * 2006-05-12 2011-02-15 Ricoh Co., Ltd. End-to-end design of superresolution electro-optic imaging systems
DE602007005481D1 (de) * 2006-05-23 2010-05-06 Omnivision Cdm Optics Inc Sättigungsoptik
US7924341B2 (en) * 2006-06-05 2011-04-12 Ricoh Co., Ltd. Optical subsystem with descriptors of its image quality
US8036481B2 (en) * 2006-07-14 2011-10-11 Eastman Kodak Company Image processing apparatus and image restoration method and program
JP4293225B2 (ja) * 2006-10-31 2009-07-08 セイコーエプソン株式会社 画像処理回路
WO2008149363A2 (fr) * 2007-06-05 2008-12-11 Dblur Technologies Ltd. Transformations non lineaires de rehaussement d'images
US7852572B2 (en) * 2007-06-25 2010-12-14 Ricoh Co., Ltd. Compact super wide-angle imaging system
WO2009037367A1 (fr) * 2007-09-17 2009-03-26 Indra Sistemas, S.A. Codage d'image dans des systèmes optiques faisant intervenir la coma
US8077401B2 (en) * 2007-10-03 2011-12-13 Ricoh Co., Ltd. Catadioptric imaging system
US20100295973A1 (en) * 2007-11-06 2010-11-25 Tessera North America, Inc. Determinate and indeterminate optical systems
US8897595B2 (en) 2008-03-26 2014-11-25 Ricoh Co., Ltd. Adaptive image acquisition for multiframe reconstruction
US9866826B2 (en) 2014-11-25 2018-01-09 Ricoh Company, Ltd. Content-adaptive multi-focal display
US9865043B2 (en) 2008-03-26 2018-01-09 Ricoh Company, Ltd. Adaptive image acquisition and display using multi-focal display
US9740293B2 (en) 2009-04-02 2017-08-22 Oblong Industries, Inc. Operating environment with gestural control and multiple client devices, displays, and users
US9740922B2 (en) 2008-04-24 2017-08-22 Oblong Industries, Inc. Adaptive tracking system for spatial input devices
US9684380B2 (en) 2009-04-02 2017-06-20 Oblong Industries, Inc. Operating environment with gestural control and multiple client devices, displays, and users
US10642364B2 (en) 2009-04-02 2020-05-05 Oblong Industries, Inc. Processing tracking and recognition data in gestural recognition systems
US8723795B2 (en) 2008-04-24 2014-05-13 Oblong Industries, Inc. Detecting, representing, and interpreting three-space input: gestural continuum subsuming freespace, proximal, and surface-contact modes
US9495013B2 (en) 2008-04-24 2016-11-15 Oblong Industries, Inc. Multi-modal gestural interface
US9952673B2 (en) 2009-04-02 2018-04-24 Oblong Industries, Inc. Operating environment comprising multiple client devices, multiple displays, multiple users, and gestural control
US8135233B2 (en) * 2008-05-22 2012-03-13 Aptina Imaging Corporation Method and apparatus for the restoration of degraded multi-channel images
US20110110586A1 (en) * 2008-05-29 2011-05-12 Colour Code Technologies, Co., Ltd Information code
US7948550B2 (en) * 2008-06-27 2011-05-24 Ricoh Co., Ltd. Electro-optic imaging system with aberrated triplet lens compensated by digital image processing
US8248684B2 (en) * 2008-08-26 2012-08-21 Ricoh Co., Ltd. Control of adaptive optics based on post-processing metrics
JP5806615B2 (ja) * 2008-09-03 2015-11-10 オブロング・インダストリーズ・インコーポレーテッド データ空間の主要次元をナビゲートするための制御システム
JP4618355B2 (ja) * 2008-09-25 2011-01-26 ソニー株式会社 画像処理装置及び画像処理方法
US20100123009A1 (en) * 2008-11-20 2010-05-20 Datalogic Scanning Inc. High-resolution interpolation for color-imager-based optical code readers
US8948513B2 (en) * 2009-01-27 2015-02-03 Apple Inc. Blurring based content recognizer
EP2396744B1 (fr) * 2009-02-11 2016-06-01 Datalogic ADC, Inc. Formation de l'image d'un code optique haute résolution à l'aide d'un dispositif de formation d'image de couleur
US8800874B2 (en) * 2009-02-20 2014-08-12 Datalogic ADC, Inc. Systems and methods of optical code reading using a color imager
US8928892B2 (en) * 2009-03-04 2015-01-06 Elie Meimoun Wavefront analysis inspection apparatus and method
CN101846798B (zh) * 2009-03-24 2013-02-27 财团法人工业技术研究院 景物深度信息的取得方法与装置
US9317128B2 (en) 2009-04-02 2016-04-19 Oblong Industries, Inc. Remote devices used in a markerless installation of a spatial operating environment incorporating gestural control
US10824238B2 (en) 2009-04-02 2020-11-03 Oblong Industries, Inc. Operating environment with gestural control and multiple client devices, displays, and users
US20100271536A1 (en) * 2009-04-27 2010-10-28 Digital Imaging Systems Gmbh Blended autofocus using mechanical and softlens technologies
US8121439B2 (en) * 2009-05-22 2012-02-21 Ricoh Co., Ltd. End-to-end design of electro-optic imaging systems using the nonequidistant discrete Fourier transform
CN102549478B (zh) 2009-08-14 2016-02-24 爱克透镜国际公司 带有同时变量的像差校正的光学器件
US20110054872A1 (en) * 2009-08-31 2011-03-03 Aptina Imaging Corporation Optical simulator using parallel computations
US9971807B2 (en) 2009-10-14 2018-05-15 Oblong Industries, Inc. Multi-process interactive systems and methods
US9933852B2 (en) 2009-10-14 2018-04-03 Oblong Industries, Inc. Multi-process interactive systems and methods
US8130229B2 (en) 2009-11-17 2012-03-06 Analog Devices, Inc. Methods and apparatus for image processing at pixel rate
CN102804030A (zh) 2010-02-17 2012-11-28 爱克透镜国际公司 可调的手性眼用透镜
CN102845052B (zh) * 2010-04-21 2015-06-24 富士通株式会社 摄像装置以及摄像方法
US8416334B2 (en) 2010-04-27 2013-04-09 Fm-Assets Pty Ltd. Thick single-lens extended depth-of-field imaging systems
US8477195B2 (en) 2010-06-21 2013-07-02 Omnivision Technologies, Inc. Optical alignment structures and associated methods
US8923546B2 (en) 2010-07-02 2014-12-30 Digimarc Corporation Assessment of camera phone distortion for digital watermarking
US8457393B2 (en) 2010-07-14 2013-06-04 Omnivision Technologies, Inc. Cross-color image processing systems and methods for sharpness enhancement
US10032254B2 (en) * 2010-09-28 2018-07-24 MAX-PLANCK-Gesellschaft zur Förderung der Wissenschaften e.V. Method and device for recovering a digital image from a sequence of observed digital images
US8905314B2 (en) 2010-09-30 2014-12-09 Apple Inc. Barcode recognition using data-driven classifier
US8523075B2 (en) 2010-09-30 2013-09-03 Apple Inc. Barcode recognition using data-driven classifier
US8687040B2 (en) 2010-11-01 2014-04-01 Omnivision Technologies, Inc. Optical device with electrically variable extended depth of field
US8949078B2 (en) 2011-03-04 2015-02-03 Ricoh Co., Ltd. Filter modules for aperture-coded, multiplexed imaging systems
CN102735609A (zh) * 2011-03-31 2012-10-17 西门子公司 一种体液成像系统、方法及景深扩展成像装置
JP2012237693A (ja) * 2011-05-13 2012-12-06 Sony Corp 画像処理装置、画像処理方法及び画像処理プログラム
US9124797B2 (en) 2011-06-28 2015-09-01 Microsoft Technology Licensing, Llc Image enhancement via lens simulation
US8729653B2 (en) 2011-10-26 2014-05-20 Omnivision Technologies, Inc. Integrated die-level cameras and methods of manufacturing the same
US9432642B2 (en) 2011-12-12 2016-08-30 Omnivision Technologies, Inc. Imaging system and method having extended depth of field
US9137526B2 (en) * 2012-05-07 2015-09-15 Microsoft Technology Licensing, Llc Image enhancement via calibrated lens simulation
US9013590B2 (en) * 2012-12-13 2015-04-21 Raytheon Company Pixel multiplication using code spread functions
US9219866B2 (en) 2013-01-07 2015-12-22 Ricoh Co., Ltd. Dynamic adjustment of multimode lightfield imaging system using exposure condition and filter position
WO2014209431A1 (fr) * 2013-06-27 2014-12-31 Fusao Ishii Affichage portable
KR102103984B1 (ko) * 2013-07-15 2020-04-23 삼성전자주식회사 깊이 영상 처리 방법 및 장치
US9030580B2 (en) 2013-09-28 2015-05-12 Ricoh Company, Ltd. Color filter modules for plenoptic XYZ imaging systems
US9990046B2 (en) 2014-03-17 2018-06-05 Oblong Industries, Inc. Visual collaboration interface
US9589175B1 (en) * 2014-09-30 2017-03-07 Amazon Technologies, Inc. Analyzing integral images with respect to Haar features
US9864205B2 (en) 2014-11-25 2018-01-09 Ricoh Company, Ltd. Multifocal display
US9342894B1 (en) * 2015-05-01 2016-05-17 Amazon Technologies, Inc. Converting real-type numbers to integer-type numbers for scaling images
US10529302B2 (en) 2016-07-07 2020-01-07 Oblong Industries, Inc. Spatially mediated augmentations of and interactions among distinct devices and applications via extended pixel manifold
DE102016013472A1 (de) 2016-11-11 2017-05-18 Daimler Ag Abbildungsvorrichtung zur Erzeugung eines Bildes für einen Kraftwagen sowie Verfahren zum Erzeugen eines Bildes einer Abbildungsvorrichtung
JP2018091805A (ja) * 2016-12-07 2018-06-14 三星電子株式会社Ｓａｍｓｕｎｇ Ｅｌｅｃｔｒｏｎｉｃｓ Ｃｏ．，Ｌｔｄ． 光学検査装置の評価方法
CN108132530B (zh) * 2017-03-03 2022-01-25 中国北方车辆研究所 一种基于像差平衡和控制的大景深光学方法及其系统
KR102521656B1 (ko) 2018-01-03 2023-04-13 삼성전자주식회사 객체를 인식하는 방법 및 장치
US11138502B2 (en) * 2018-05-01 2021-10-05 International Business Machines Corporation Foiling neuromorphic hardware limitations by reciprocally scaling connection weights and input values to neurons of neural networks
KR102661983B1 (ko) 2018-08-08 2024-05-02 삼성전자주식회사 이미지의 인식된 장면에 기반하여 이미지를 처리하는 방법 및 이를 위한 전자 장치
CN113191972B (zh) * 2021-04-27 2023-04-14 西安交通大学 一种轻量真实图像去噪的神经网络设计及训练方法
US11330145B1 (en) 2021-06-10 2022-05-10 Bank Of America Corporation Image processing edge device for document noise removal
CN113267909B (zh) * 2021-07-19 2021-10-08 军事科学院系统工程研究院网络信息研究所 基于波前畸变补偿的防窥显示方法

Family Cites Families (28)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US3820878A (en) * 1971-04-01 1974-06-28 Technical Operations Inc Spectral zonal information storage and retrieval
NL8800199A (nl) * 1987-02-09 1988-09-01 Gen Signal Corp Digitale vitale snelheidsdecodeur.
US4837451A (en) * 1987-08-26 1989-06-06 The Boeing Company Ring array imaging system
KR920007919B1 (ko) * 1988-06-27 1992-09-19 미쯔비시 덴끼 가부시기가이샤 텔레비젼 전화기
JP2574563B2 (ja) * 1990-09-03 1997-01-22 松下電器産業株式会社 フィルタ
US5168375A (en) * 1991-09-18 1992-12-01 Polaroid Corporation Image reconstruction by use of discrete cosine and related transforms
US5179273A (en) * 1992-05-29 1993-01-12 Eastman Kodak Company Method for testing or controlling a performance of an adaptive optic
JPH0630393A (ja) * 1992-07-10 1994-02-04 Casio Comput Co Ltd Ｑｍｆ装置
JP3489796B2 (ja) * 1994-01-14 2004-01-26 株式会社リコー 画像信号処理装置
US5710839A (en) * 1994-04-20 1998-01-20 Eastman Kodak Company Method and apparatus for obscuring features of an image
US20020118457A1 (en) * 2000-12-22 2002-08-29 Dowski Edward Raymond Wavefront coded imaging systems
WO1996024085A1 (fr) 1995-02-03 1996-08-08 The Regents Of The University Of Colorado Systemes optiques a profondeur de champ etendue
JPH08241068A (ja) * 1995-03-03 1996-09-17 Matsushita Electric Ind Co Ltd 情報記録媒体およびビットマップデータ復号化装置とビットマップデータ復号化方法
JPH08313823A (ja) * 1995-05-15 1996-11-29 Olympus Optical Co Ltd 内視鏡画像処理装置
US5751340A (en) * 1996-08-21 1998-05-12 Karl Storz Gmbh & Co. Method and apparatus for reducing the inherently dark grid pattern from the video display of images from fiber optic bundles
US6240219B1 (en) * 1996-12-11 2001-05-29 Itt Industries Inc. Apparatus and method for providing optical sensors with super resolution
JP3519239B2 (ja) * 1997-04-15 2004-04-12 富士電機システムズ株式会社 対象物の回転角度検出方法
JP4215844B2 (ja) * 1997-11-05 2009-01-28 日本テキサス・インスツルメンツ株式会社 半導体記憶装置
JP2000005127A (ja) * 1998-01-23 2000-01-11 Olympus Optical Co Ltd 内視鏡システム
KR20010043223A (ko) * 1998-05-01 2001-05-25 유니버시티 테크놀러지 코포레이션 확장된 필드 깊이를 가진 광학 시스템
JP2000098301A (ja) * 1998-09-21 2000-04-07 Olympus Optical Co Ltd 拡大被写界深度光学系
JP2000275582A (ja) * 1999-03-24 2000-10-06 Olympus Optical Co Ltd 被写界深度拡大システム
DE60114498T2 (de) * 2000-04-11 2006-08-03 Hoya Corp. Darstellung der Leistung eines Brillenglases durch eine Indexanzeige in visuell verständlicher Weise
JP3919069B2 (ja) * 2000-04-11 2007-05-23 Ｈｏｙａ株式会社 眼鏡レンズ性能表示方法及び装置
JP4474748B2 (ja) * 2000-07-14 2010-06-09 ソニー株式会社 信号処理装置及び方法、映像信号記録装置、並びに映像信号再生装置
JP4389258B2 (ja) * 2000-08-02 2009-12-24 横河電機株式会社 光学観察装置
US6525302B2 (en) * 2001-06-06 2003-02-25 The Regents Of The University Of Colorado Wavefront coding phase contrast imaging systems
US6927922B2 (en) * 2001-12-18 2005-08-09 The University Of Rochester Imaging using a multifocal aspheric lens to obtain extended depth of field

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TWI480583B (zh) * 2011-05-31 2015-04-11 Omnivision Tech Inc 以色相關波前編碼延伸透鏡系統景深的系統及其方法

Also Published As

Publication number Publication date
CN100338499C (zh) 2007-09-19
DE60317472T2 (de) 2008-09-04
US7995853B2 (en) 2011-08-09
CN101118317A (zh) 2008-02-06
CN101118317B (zh) 2010-11-03
US20080044103A1 (en) 2008-02-21
WO2003073153A1 (fr) 2003-09-04
EP1478966A1 (fr) 2004-11-24
JP2010183638A (ja) 2010-08-19
US8111937B2 (en) 2012-02-07
CN1650216A (zh) 2005-08-03
JP2005519361A (ja) 2005-06-30
DE60317472D1 (de) 2007-12-27
US20080107354A1 (en) 2008-05-08
US20080131023A1 (en) 2008-06-05
US7379613B2 (en) 2008-05-27
ATE378619T1 (de) 2007-11-15
AU2003213651A1 (en) 2003-09-09
JP5318027B2 (ja) 2013-10-16
US20030169944A1 (en) 2003-09-11

Similar Documents

Publication Publication Date Title
EP1478966B1 (fr) Traitement optimise d'image pour systemes d'imagerie codes en front d'onde
KR100647402B1 (ko) 이미지 센서의 화질개선장치 및 그 방법
US8971622B2 (en) Image processing apparatus, image capture apparatus, and image processing method
EP2715447B1 (fr) Système et procédé destinés à étendre la profondeur de champ dans un système de lentille à l'aide d'un codage de front d'onde dépendant de la couleur
Yasuma et al. Generalized assorted pixel camera: postcapture control of resolution, dynamic range, and spectrum
US7889264B2 (en) End-to-end design of superresolution electro-optic imaging systems
US7627193B2 (en) Camera with image enhancement functions
Guichard et al. Extended depth-of-field using sharpness transport across color channels
JP2008516299A (ja) 撮像装置及び画像改質処理方法
JP2005519361A5 (fr)
Akpinar et al. Learning wavefront coding for extended depth of field imaging
WO2007072477A2 (fr) Amelioration d'image au moyen de deconvolution materielle
RU2595759C2 (ru) Способ и устройство для захвата изображения и одновременного извлечения глубины
US11677918B2 (en) Method and apparatus for color space conversion
EP3910385A1 (fr) Capteur d'images
Tisse et al. Extended depth-of-field (EDoF) using sharpness transport across colour channels
EP1923731B1 (fr) Traitement d'image optimisé pour systèmes d'imagerie codés du front d'onde
JP7414745B2 (ja) 学習データの製造方法、学習方法、学習データ製造装置、学習装置、およびプログラム
US20240135508A1 (en) Image processing method, image processing apparatus, image processing system, imaging apparatus, and storage medium
Eboli Hybrid Non-blind Image Deblurring for Real Scenarios
WO2023150671A1 (fr) Procédés, systèmes et dispositifs pour augmenter la résolution et la plage dynamique d'image
JP2018037791A (ja) 撮像装置、画像処理装置、画像処理方法および画像処理プログラム
Feng et al. Low-complexity digital filter geometry for spherical coded imaging systems

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20040908

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LI LU MC NL PT SE SI SK TR

AX Request for extension of the european patent

Extension state: AL LT LV MK RO

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LI LU MC NL PT SE SI SK TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REF Corresponds to:

Ref document number: 60317472

Country of ref document: DE

Date of ref document: 20071227

Kind code of ref document: P

REG Reference to a national code

Ref country code: SE

Ref legal event code: TRGR

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20071114

Ref country code: LI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20071114

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20080225

Ref country code: CH

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20071114

NLV1 Nl: lapsed or annulled due to failure to fulfill the requirements of art. 29p and 29m of the patents act
PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20071114

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20080214

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20071114

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20071114

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20071114

ET Fr: translation filed
PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20071114

Ref country code: BE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20071114

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20080414

26N No opposition filed

Effective date: 20080815

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20080228

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20080227

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20080215

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20071114

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20071114

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20080227

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20080515

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20071114

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20080229

REG Reference to a national code

Ref country code: DE

Ref legal event code: R082

Ref document number: 60317472

Country of ref document: DE

Representative=s name: KUDLEK & GRUNERT PATENTANWAELTE PARTNERSCHAFT, DE

REG Reference to a national code

Ref country code: GB

Ref legal event code: 732E

Free format text: REGISTERED BETWEEN 20121101 AND 20121107

REG Reference to a national code

Ref country code: DE

Ref legal event code: R082

Ref document number: 60317472

Country of ref document: DE

Representative=s name: KUDLEK & GRUNERT PATENTANWAELTE PARTNERSCHAFT, DE

Effective date: 20121022

Ref country code: DE

Ref legal event code: R081

Ref document number: 60317472

Country of ref document: DE

Owner name: OMNIVISION TECHNOLOGIES, INC., SANTA CLARA, US

Free format text: FORMER OWNER: CDM OPTICS, INC., BOULDER, COL., US

Effective date: 20121022

Ref country code: DE

Ref legal event code: R081

Ref document number: 60317472

Country of ref document: DE

Owner name: OMNIVISION TECHNOLOGIES, INC., US

Free format text: FORMER OWNER: CDM OPTICS, INC., BOULDER, US

Effective date: 20121022

REG Reference to a national code

Ref country code: FR

Ref legal event code: TP

Owner name: OMNIVISION TECHNOLOGIES, INC., US

Effective date: 20130408

Ref country code: FR

Ref legal event code: CD

Owner name: OMNIVISION TECHNOLOGIES, INC., US

Effective date: 20130408

Ref country code: FR

Ref legal event code: CA

Effective date: 20130408

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 14

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 15

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 16

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20200115

Year of fee payment: 18

Ref country code: SE

Payment date: 20200210

Year of fee payment: 18

Ref country code: FI

Payment date: 20200128

Year of fee payment: 18

Ref country code: GB

Payment date: 20200130

Year of fee payment: 18

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20200124

Year of fee payment: 18

REG Reference to a national code

Ref country code: DE

Ref legal event code: R082

Ref document number: 60317472

Country of ref document: DE

Representative=s name: DEHNSGERMANY PARTNERSCHAFT VON PATENTANWAELTEN, DE

REG Reference to a national code

Ref country code: DE

Ref legal event code: R119

Ref document number: 60317472

Country of ref document: DE

REG Reference to a national code

Ref country code: SE

Ref legal event code: EUG

REG Reference to a national code

Ref country code: FI

Ref legal event code: MAE

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20210227

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210227

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210228

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210901

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210227

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210228