Classifications

• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T3/00—Geometric image transformations in the plane of the image
  • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
  • G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
• • G—PHYSICS
  • G02—OPTICS
  • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
  • G02B26/00—Optical devices or arrangements for the control of light using movable or deformable optical elements
  • G02B26/08—Optical devices or arrangements for the control of light using movable or deformable optical elements for controlling the direction of light
  • G02B26/10—Scanning systems
  • G02B26/106—Scanning systems having diffraction gratings as scanning elements, e.g. holographic scanners
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
  • H04N1/04—Scanning arrangements, i.e. arrangements for the displacement of active reading or reproducing elements relative to the original or reproducing medium, or vice versa
  • H04N1/19—Scanning arrangements, i.e. arrangements for the displacement of active reading or reproducing elements relative to the original or reproducing medium, or vice versa using multi-element arrays
  • H04N1/195—Scanning arrangements, i.e. arrangements for the displacement of active reading or reproducing elements relative to the original or reproducing medium, or vice versa using multi-element arrays the array comprising a two-dimensional array or a combination of two-dimensional arrays
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
  • H04N1/04—Scanning arrangements, i.e. arrangements for the displacement of active reading or reproducing elements relative to the original or reproducing medium, or vice versa
  • H04N1/19—Scanning arrangements, i.e. arrangements for the displacement of active reading or reproducing elements relative to the original or reproducing medium, or vice versa using multi-element arrays
  • H04N1/195—Scanning arrangements, i.e. arrangements for the displacement of active reading or reproducing elements relative to the original or reproducing medium, or vice versa using multi-element arrays the array comprising a two-dimensional array or a combination of two-dimensional arrays
  • H04N1/19589—Optical means, e.g. an optical fibre bundle, for mapping the whole or a part of a scanned image onto the array
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
  • H04N1/387—Composing, repositioning or otherwise geometrically modifying originals
  • H04N1/3876—Recombination of partial images to recreate the original image
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
  • H04N23/58—Means for changing the camera field of view without moving the camera body, e.g. nutating or panning of optics or image sensors
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N3/00—Scanning details of television systems; Combination thereof with generation of supply voltages
  • H04N3/02—Scanning details of television systems; Combination thereof with generation of supply voltages by optical-mechanical means only
  • H04N3/08—Scanning details of television systems; Combination thereof with generation of supply voltages by optical-mechanical means only having a moving reflector
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N5/00—Details of television systems
  • H04N5/222—Studio circuitry; Studio devices; Studio equipment
  • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
  • H04N5/2628—Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N2201/00—Indexing scheme relating to scanning, transmission or reproduction of documents or the like, and to details thereof
  • H04N2201/04—Scanning arrangements
  • H04N2201/0402—Arrangements not specific to a particular one of the scanning methods covered by groups H04N1/04 - H04N1/207
  • H04N2201/0426—Scanning an image in a series of contiguous zones

Definitions

• the present invention relates to an imaging system for selectively providing either a low resolution real-time image or a high resolution scanned image. More particularly, the invention relates to a camera-like imaging system in which high resolution digital capture of an image is provided by combining a series of sequentially obtained sub-images.
• An alternative method of increasing resolution uses just one image sensor, but uses that image sensor to capture only a portion (sub-image) of the desired image.
• a series of sub-images called “tiles” are sequentially captured and stored in the memory of an image processor.
• the image processor combines ("stitches") the tiles together to form a composite ("mosaic") image.
• Tiling systems present a number of problems involving the capture of the sub-images.
• the sub-images should fit together tightly, avoiding both overlap of tiles or gaps between tiles.
• a high degree of repeatability in the tile capture process is desirable, because repeatable (characterizable) errors can be corrected. Because of the complexity involved in acquiring the tiles, known tiling systems have been limited to highly sophisticated and costly applications, such as satellite imaging of the earth.
• the present invention uses tiling to provide a low cost scanner for commercial applications.
• a low cost tiling system incorporated as part of a scanning camera, could find application, for example, as an aid to the visually impaired, as the imaging component of a face-up copying system for books, as an x-ray film scanner, as a variable resolution desktop scanner, as a microfilm copying system or as a video teleconferencing input device.
• This invention provides an image input device for providing a composite image of an object composed of a plurality of sub-images of the object.
• the system comprises an image sensor for receiving a sub-image, a lens for focusing the sub-image on the image sensor, an optical deflecting means positioned between the lens and the object, the deflecting means having a plurality of optical elements, each such element for deflecting one of a plurality of sub-images to the lens, and control means for causing the optical deflection means to sequence through the sub-images comprising the composite image to form a composite image.
• FIG. 1 is a schematic illustration of a first embodiment of the invention
• FIG. 2A is a bottom view of the disk of holographic optical elements of the first embodiment of the invention.
• FIG. 2B is a bottom view of an alternate embodiment of the disk of FIG. 2A;
• FIG. 3A is an illustration of a tiling pattern used in the first embodiment of the invention
• FIG. 3B is an illustration of an embodiment of the invention including a focal correction lens
• FIG. 3C is an illustration of an embodiment of the invention including a motor-driven focus adjustment
• FIG. 4 is a schematic illustration of a second embodiment of the invention.
• FIG. 5 is a schematic illustration of a third embodiment of the invention
• FIG. 6 is a schematic illustration of a fourth embodiment of the invention
• FIG. 7 is a schematic illustration of a fifth embodiment of the invention.
• FIG. 8 is a schematic embodiment of a linear slide of optical deflectors.
• the imaging system of the present invention provides both a real-time (television-frame rate) camera and a high resolution scanning camera.
• the camera uses an area array charge coupled device ("CCD") sensor to provide the real-time imaging capability.
• CCD area array charge coupled device
• the high resolution scanning mode of the camera is implemented by stepping the point of gaze of the CCD sensor over an object image, such as an 8.5 by 12.5 inch document, so that the document image is acquired as a series of tiles.
• a 32-element disk functions as an optical deflector. Step-wise rotation of the optical deflection disk steps the point of gaze of the CCD array over an object plane so that 32 image "tiles" are acquired.
• the tiles are "stitched" together in the scanner's memory to form a complete composite image.
• a sampling resolution of 300 dots per inch can be achieved over an 8.5 by 12.5 inch area, with a time to scan from 2 to 4 seconds, based on photometric calculations for the optics and CCD, and including the predicted settling time for disk motion.
• a first preferred embodiment of the imaging system of the present invention as shown in FIG. 1, comprises an image sensor 10, a lens 20, an optical deflection means 30, a positioning means 50 having a shaft 52, and an image processor 70.
• Object plane 25 which is not a part of the invention, indicates the location of the image to be scanned.
• Image sensor 10 receives an optical image and provides an electrical representation of the image as an output.
• Image sensors are well known in the art.
• An illuminator or monochromatic light source 15 may be used to improve the signal-to-noise ratio of image sensor 10.
• Image sensor 10 is preferably an area array CCD, such as the TC245 available from Texas Instruments Incorporated, Dallas, Texas.
• the TC245 has 365,420 pixels arranged as 484 lines of 755 pixels.
• the TC245 generates an NTSC interlaced video format; therefore, two fields, one comprising odd-numbered scan lines and one comprising even-numbered scan lines, comprise a frame. Each of the two fields is transferred at a 60 Hz rate; therefore, the frame transfer rate is 30 Hz.
• the output frames from image sensor 10 are coupled to sensor interface 72, which is shown in FIG. 1 as part of image processor 70.
• Sensor interface 72 is preferably a TCK 244/245 CCD image sensor evaluation kit available from Texas Instruments, or an integrated circuit chip set, also available from Texas Instruments, for driving the TC245.
• Sensor interface 72 receives frame information from image sensor 10 and passes it to frame grabber 74.
• Frame grabber 74 is preferably model number SVM-BW-1MB-20-14.3 available from EPIX, Inc., Northbrook, Illinois.
• Frame grabber 74 stores and formats the frame information from sensor interface 72, and provides the frame information to personal computer 76. Although it is shown outside personal computer 76, frame grabber 74 is available as a circuit card which plugs into personal computer 76.
• Personal computer 76 which is an IBM PC in the first preferred embodiment, executes known software algorithms to combine the frames into a composite image stored in memory 90, which is shown as part of personal computer 76 in the first preferred embodiment.
• image sensor 10 sensor interface 72
• frame grabber 74 frame grabber 74
• personal computer 76 without departing from the scope of the present invention.
• Deflection means 30 comprises a moveable array of fixed optical deflection elements 35 for dividing the object image into a plurality of image tiles.
• deflection means 30 comprises a disk 32, rotatable about its central axis, having 32 holographic optical elements 201-232 disposed about its circumference as shown in FIG. 2A.
• element 35 represents one of a plurality of elements.
• the deflection means of the present invention is not limited to 32 elements. More or fewer elements may be selected depending on the application.
• Disk 32 serves as an optical deflector, with each element 201-232 along its circumference corresponding to a tile position.
• Positioning means 50 causes disk 32 to rotate, thereby bringing each of the holographic optical elements 201-232 sequentially into the optical path to deflect its corresponding sub-image to image sensor 10.
• Holographic optical elements 201- 232 are manufactured, according to known methods, to incorporate microscopic ridges to serve as a diffraction grating on a portion of disk 32, which is preferably a molded plastic material.
• Imaging lens 20 is provided in the optical path between deflection element 35 and image sensor 10 to project the object onto image sensor 10.
• Deflection element 35 is positioned in front of lens 20 (between lens 20 and the object to be imaged on object plane 25) to avoid off-axis image distortion.
• the angle subtended by the image from deflection element 35 is much smaller than the angle subtended by the image on object plane 25.
• Positioning lens 20 in front of deflection element 35 is undesirable, because lens 20 would then have to subtend a much larger angle. A larger angle would require lens 20 to be a much more costly wide-angle lens, and such a lens would be more prone to aberrations.
• disk 32 is mounted on shaft 52 of a stepping motor, which serves as positioning means 50.
• Disk 32 should preferably be constructed of a thin, lightweight material such as plastic in order to minimize its settling time. For each tile to be acquired, the disk is positioned and allowed to settle. Alignment of disk 32 is not critical on any axis. Moreover, disk 32 has a low mass. From these considerations it follows that positioning will be possible within one frame time of image sensor 10 per tile, and that disk 32 may be supported by shaft 52 alone.
• Positioning means 50 is electrically coupled to control means 80. In response to signals from control means 80, positioning means 50 positions disk 32 in one of the 32 angular positions occupied by holographic optical elements 201-232 shown in FIG. 2. Each position corresponds to one setting of deflection means 30. Because alignment of holographic element 35 is not critical, positioning means 50 may comprise an open-loop positioning mechanism. An open-loop system is inherently faster than a closed-loop system, because the closed-loop system operates by making a series of adjustments to achieve precise positioning, whereas the open-loop system seeks its desired position in just one step.
• Disk 37 shown in FIG. 2B, comprises a disk encircled by a band of holographic elements 251-282 disposed along its circumference. Because elements
• positioning means 50 can comprise a continually rotating motor instead of a stepping motor.
• a further benefit of the deflection continuum provided by elements 251-282 is that minor defects in the holographic material average out. An image is acquired when the boundaries of an element, such as element 251, are in alignment with a desired tile.
• Each element 251-282 corresponds to a frame transfer from image sensor 10. Accordingly, disk 37 should rotate at a speed that allows image sensor 10 to transfer both fields comprising a frame at a single deflection setting.
• Control means 80 may be a conventional stand ⁇ alone computer or microprocessor, or may be part of image processor 70 as shown in FIG. 1. Control means 80 causes positioning means 50 to stop deflection means 30 at known positions or settings. Control means 80 further directs memory 90 to store the images received from image sensor 10 at appropriate times, allowing for the settling time of deflection means 30.
• lens 20 would be 15 inches (26.67 cm.) from an object on object plane 25, and would have a minimum depth of focus of 10 mm. Based on the pixel dimensions of the TC245 image sensor and the desired
• lens 20 have a 40 mm focal length and f#10, 4.5 mm aperture, working distance of 380 mm, and depth of field of 14 mm. Assuming 800 lux room level illumination and 50% page reflectivity yields sensor illumination of 1.0 lux. This corresponds to 4 times the dark current of the TC245 sensor when operated at television frame rates (16.6 ms integration time per frame).
• An illuminator 15 can improve the signal-to-noise ratio for sensor 10.
• a near-IR filter may be included in the optical path to compensate for the chromaticity of holographic elements 201-232.
• FIG. 3A illustrates the tiling scheme adopted in the first preferred embodiment of the invention.
• Tiles 301-332 correspond to the 32 tiling elements 201- 232 shown in FIG. 2A.
• Tiles 301-332 comprise the object, for example a page of printed text, located on object plane 25.
• a 4X8 array of tiles is shown, based on the layout of the TC245 sensor, with the horizontal scanning axis of the sensor aligned with the short dimension of the page.
• FIG. 3A shows the optical path of tile 312 from object plane 25, through holographic optical element 35 and lens 20, to image sensor 10.
• Line 350 indicates the uncorrected focal surface and its relation to object plane 25. It is to be understood that line 350 is a cross-section of the focal surface, which is spherical.
• uncorrected focal surface 350 curves through object plan 25. If optimum focus is obtained at a point falling on tile 310, then tile 301 will be out of focus.
• Lens 22, shown in FIG. 3B, corrects the disparity between object plane 25 and uncorrected focal surface 350.
• Lens 22 is mounted on the upper surface of disk 32, above representative holographic optical element 35. It is to be understood that a lens corresponding to lens 22 would be provided, as needed, for each optical element 201-232, and each such lens 22 would correct for the particular focal length disparity of each corresponding optical element 201-232. Some tiles intersect the focal surface and therefore, require no correction. Lens 22 will slightly alter the size of its corresponding tile as seen by image sensor 10.
• holographic optical elements 201-232 are diffractive optics, the diffraction rulings can be arranged to include both a lens and a prism by appropriate micromachining or molding operations. Therefore, holographic optical elements 201-232 can incorporate lens 22, instead of adding a plano-convex lens to the upper surface of disk 32.
• Element 233 is an optically neutral element (such as a hole) that allows light to pass through undeflected.
• additional known optical elements can be inserted into the optical path between disk 32 and lens 20 to implement fully a zoom function and also to provide an extra motor-driven focus adjustment 24, shown in FIG. 3C.
• Focus adjustment 24 brings into optimum focus objects on object plane 25 that stand higher than the depth of field or which are not flat, such as books or household objects. Both focus adjustment 24 and optical zoom may be implemented using standard optical techniques.
• FIG. 4 A second preferred embodiment of the system of the present invention is shown in FIG. 4.
• mirror 36 serves as the optical deflection element.
• Wedge support 34 is used to affix mirror 36 to disk 32. Only one mirror 36 is depicted in FIG. 4; however, it is to be understood that a plurality of mirrors are disposed around the circumference of disk 32, similar to the holographic optical elements encircling disk 32 in FIG. 2. Each mirror 36 is affixed to disk 32 at a different angle, so that a two dimensional tiling pattern is obtained as in the first embodiment.
• positioning means 50 preferably uses a closed-loop system to avoid the image distortion problem with mirrors.
• the closed-loop system is slower, but more precise, than an open-loop system, because the closed-loop positioning means hunts for the proper position until it is successful.
• FIG. 5 A third preferred embodiment of the system of the present invention is shown in FIG. 5.
• a plurality of offset plano-convex lens pairs 44, 45 are disposed around the circumference of transparent disk 42.
• Lens 20 is not required in this embodiment, because the offset lens pair 44, 45 can adequately focus the sub-image onto image sensor 10.
• FIG. 6 A fourth preferred embodiment is shown in FIG. 6, wherein wedge prisms 47 act as optical deflection means, disposed around the circumference of transparent disk 42.
• prism 47 represents one of a plurality of optical deflectors.
• a correction lens, such as lens 22 shown in FIG. 3B, may also be used in conjunction with prism 47.
• deflection means 30 comprises a pair of mirror galvanometers 53, 55.
• Mirror galvanometers are available from General Scanning, Inc., Watertown, Massachusetts.
• Mirror galvanometer 55 comprises mirror 38, shaft 56 and motor 58.
• Mirror 38 rotates on shaft 56 of motor 58, in response to signals from control 80.
• Mirror 37 rotates on axis 54, which is the shaft of the motor (not shown) of mirror galvanometer 53.
• Mirrors 38 and 37 have mutually perpendicular axes of rotation, an X-axis and a Y-axis, thereby allowing scanning of object plane 25 in two directions.
• Each tile setting for mirror galvanometers 53, 55 comprises an X- coordinate, corresponding to rotation about axis 56 and a Y-coordinate corresponding to rotation about axis 54.
• X- coordinate corresponding to rotation about axis 56
• Y-coordinate corresponding to rotation about axis 54.
• circular, cylindrical or spherical coordinates could be substituted for the X-Y coordinate system, and that a single gimbel-mounted mirror could provide a scanning means as well.
• Other mirror arrangements can also be employed, provided that a suitable drive motor is used that can position the mirrors with a high degree of repeatabi1ity.
• mirror galvanometers 53, 55 provide a variable number of scanning settings. Therefore, mirror galvanometers 53, 55 provide a variable scanning means, capable of scanning either a discrete series or a continuum of sub-images. Prisms are a preferred deflection element, because they are inexpensive, relatively position insensitive and easily manufactured in arrays, with each array in a manufacturing run exhibiting the same characteristics as the other arrays in the batch.
• element 231 in each array will exhibit the same characteristics. Therefore, any minor optical deviation in element 231 can be corrected either in software or by additional optical elements.
• Holographic optical elements also provide highly consistent characteristics within a manufacturing batch; however, holographic optical elements are more highly chromatic than prisms. Prisms have far less chromaticity than holographic optical elements for a given amount of deflection. Accordingly, prisms are preferred deflection elements in applications where chromaticity is a problem.
• a monochromatic light source 15, shown in FIG. 1A can be used with holographic optical elements.
• disk 32 could easily be replaced by an array of image deflectors disposed on linear slide 33 as shown in FIG. 8, or on a loop or a cylinder.
• image deflectors other than mirrors, prisms, offset lens pairs, mirror galvanometers and holographic optical elements can be used without departing from the spirit of the invention.
• the scanning camera of the present invention can obtain a pre-scan image in the time required by one extra step of the deflection means, by adding an additional optical focusing element that, in conjunction with motor-driven focus adjustment 24, focuses the entire object image onto image sensor 10.
• the low resolution grey-scale image so obtained can be analyzed in software by an area-histogram to produce a thresholding map. This map may then be used dynamically to vary the binarizing threshold applied to incoming pixels during high resolution scanning. This permits the acquisition of accurate text images over tinted backgrounds, even on the multicolor pages commonly found in periodicals.
• the real-time imaging capacity inherent in the scanning camera of the present invention permits the simplified provision of essential features in image scanning applications which do not need real-time display capability.

Landscapes

• Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• Optics & Photonics (AREA)
• Theoretical Computer Science (AREA)
• Facsimile Scanning Arrangements (AREA)
• Transforming Light Signals Into Electric Signals (AREA)
• Facsimile Heads (AREA)
• Image Input (AREA)

Applications Claiming Priority (3)

Publications (2)

Family

ID=25231168

Family Applications (1)

Country Status (4)

Families Citing this family (8)

Citations (4)

Family Cites Families (9)

• 1993
  • 1993-01-12 WO PCT/US1993/000294 patent/WO1993014594A1/en not_active Application Discontinuation
  • 1993-01-12 EP EP93903458A patent/EP0622001A1/de not_active Withdrawn
  • 1993-01-12 AU AU34714/93A patent/AU663760B2/en not_active Ceased
  • 1993-01-12 JP JP5512639A patent/JPH07503113A/ja active Pending

Patent Citations (4)

Non-Patent Citations (2)

Also Published As

Similar Documents

Legal Events