Classifications

• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06K—GRAPHICAL DATA READING; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
  • G06K7/00—Methods or arrangements for sensing record carriers, e.g. for reading patterns
  • G06K7/10—Methods or arrangements for sensing record carriers, e.g. for reading patterns by electromagnetic radiation, e.g. optical sensing; by corpuscular radiation
  • G06K7/10544—Methods or arrangements for sensing record carriers, e.g. for reading patterns by electromagnetic radiation, e.g. optical sensing; by corpuscular radiation by scanning of the records by radiation in the optical part of the electromagnetic spectrum
  • G06K7/10821—Methods or arrangements for sensing record carriers, e.g. for reading patterns by electromagnetic radiation, e.g. optical sensing; by corpuscular radiation by scanning of the records by radiation in the optical part of the electromagnetic spectrum further details of bar or optical code scanning devices
  • G06K7/1092—Methods or arrangements for sensing record carriers, e.g. for reading patterns by electromagnetic radiation, e.g. optical sensing; by corpuscular radiation by scanning of the records by radiation in the optical part of the electromagnetic spectrum further details of bar or optical code scanning devices sensing by means of TV-scanning

Definitions

• This invention relates in general to information gathering means for locating and identifying marked or labelled objects during storage and transport. It relates to apparatus for electronic image acquisition, computer vision, and image analysis. More specifically the invention provides high-resolution image acquisition means adapted for use in a variable and uncontrolled environment and may be applied to the reliable recovery of inventory data from one or more labels included within each image.
• the timber industry like many others, has a particular need for the convenient and preferably automatic recording of individual commodity items (logs and processed timber in this case), noting in particular the location of each item at a recorded time. Often the recording act must be carried out in outdoors environments during transportation, processing, and storage.
• Logs may be initially identified at a weighing station by ownership, location, handler, type, or quality. Many other items of goods apart from logs, even containers of goods may similarly need to be identified and located from time to time and at one place or another. This information should help with the management of stocks and resources (particularly important with the "just-in-time" management practice), in optimising the flow of transport and movement of goods, and in maintaining a watch over property. Information may be used to categorise shipments, to prepare processing stations, or to pinpoint the whereabouts of each of many logs during movements. Accurate identification can aid in efficient operation and production management and can help reduce stock loss.
• a conventional solid-state video camera according to NTSC, PAL, or CCIR standards may record a total of from 0.06 to 0.3 million pixels. Although these cameras can be used on single labels virtually filling the image area, their resolution is insufficient to adequately resolve the patterns printed on each of a group of many labels in the one image. In order to gather information speedily from an accumulation of labelled items there is a need for a higher resolution than can be provided by conventional video cameras. Similar considerations apply if a label occupies only a small part of an image. CCD chips having a higher pixel count - up to or over 4096 x 4096 pixels in X and Y directions are known, and a 6K x 6K array would have over 100 times as many pixels to be read as are in a standard video camera.
• a further need is for a high-resolution surveillance camera, preferably having good performance under low lighting conditions, optionally equipped with means for remote operation.
• An identification system should be modular so that it can be reconfigured from standard building blocks or modules for different applications or for different operating modes.
• the invention provides a frame grabber capable of receiving a first digital signal representing an image, comprising means to compress the first digital signal by at least 4: 1 in real time to produce a compressed digital image having a reduced number of bytes compared to the total number of bytes of the first digital signal, and buffer means to store at least one compressed digital image.
• the invention provides a frame grabber wherein said means to compress the digital signal is capable of compressing the digital signal in the range of 20:1 to 30:1.
• the invention provides a frame grabber wherein said buffer means is capable of storing at least two images.
• the invention provides a frame grabber wherein said means to compress the digital signal has at least two input channels capable of processing the information in parallel.
• the invention provides a frame grabber wherein said means to compress the digital signal comprises a JPEG processor.
• the invention provides image capture means for a camera comprising at least one solid-state CCD light-sensitive array capable of being positioned at an image plane of a camera, at least one analogue output channel from said CCD array, one or more analogue to digital converter(s) for converting analogue signals from the or each analogue output channel to one or more digital signals, and a frame grabber connected to the output(s) of the analogue to digital converters).
• the invention provides image capture means wherein the solid-state CCD light-sensitive array is an X-Y array having at least 2048 pixels along each edge (i.e. a total of 2 ⁇ pixels)
• the invention provides image capture means wherein the solid-state CCD light-sensitive array is an X-Y array having at least 4096 pixels along each edge (i.e. a total of 2 u pixels).
• the invention provides image capture means further comprising means to communicate the one or more compressed digital image(s) to an external image processing unit or other receiver of image data.
• the invention provides a camera for recording labelled items comprising image capture means and optics including image focusing means capable of focusing an image on the CCD array, aperture control means, and shutter means.
• the invention provides a camera wherein the camera is mounted on a self-powered mobile platform so that the camera can be positioned relative to one or more labelled items to initiate a data capture process.
• the invention provides a camera wherein means are provided to laterally move the camera along a guide rail, and to tilt and pan the camera when at selected positions in order to repeat the process of capturing an image from more than one viewpoint.
• Fig 1 This is a diagram showing a vehicle with a modular camera and recording system mounted on the front, image interpretation devices within the vehicle which is equipped with a radio link, and a pile of logs with identification labels on the ends in position for recording the encoded labels displayed on the ends of the logs.
• Fig 2 This block diagram shows data and signal flow between major components of the modular recording system of Fig 1.
• FIG. 1 This diagram shows a front view of a modified vehicle which has the camera and ancillary equipment mounted in front.
• Fig 4 This diagram shows a side view of the vehicle with the attached camera and ancillary equipment.
• Fig 5 This diagram depicts details of the camera carriage and the guide rail with attached motor and sensors, in a side view.
• Fig 6 This diagram shows details from the top of the camera support mounted on the guide rail.
• Fig 7 This diagram shows details from the front of the camera support mounted on the guide rail. This diagram shows details of a spring-loaded CCD mount support allowing a CCD to be used within a photographic camera body.
• This diagram shows details of the camera back replacement which locates the CCD at the film plane of a standard photographic camera.
• This diagram shows the operator's console which includes rangefinder and positioning information.
• This diagram shows in plan view a preferred hydraulic powering system for use on the camera housing to control flaps, filters, and the like.
• This diagram shows a side view a preferred hydraulic powering system for use on the camera housing to control flaps, filters, and the like.
• This diagram shows the front view of a preferred hydraulic powering system for use on the camera housing to control flaps, filters, and the like.
• This diagram shows a portable version of the camera system optionally attached either to a portable computer carried in a backpack, or attached from time to time to a docking station. The operator is shown taking a picture of a pile of logs.
• This diagram shows a fixed version of the camera system attached to an adjacent image processing computer, both mounted on a mast or pole.
• the camera is aimed at the rear of a trailer loaded with labelled logs.
• this invention offers the particular advantages of being able to remotely record and decode one or more labels on merchandise being transported or in storage.
• Previous image recorders are restricted (by resolution) to considering single labels when under relatively ideal viewing conditions.
• This invention uses a high-resolution electronic image capture camera to generate a detailed digital version of an image including possibly many labels.
• the image so captured may be scoured by an image analysis computer hunting for labels of a type adapted for computerised recognition, that decodes and presents the information within each label.
• Each label is preferably unique to that particular item of merchandise.
• the camera may in fact be a back adapted to fit an existing camera.
• the invention may be regarded as a kind of CCD array/frame grabber/image storage combination, for use at the film plane, which compresses the image data as soon as the image is taken, so that its digital representation takes up considerably less storage space than if it was not compressed.
• a particular application for which this invention has been developed is that of monitoring the flow of timber; principally whole logs, from the forest to the final destination. In recent times the value of a single log of Pinus radiata has risen to be of the order of NZ $1000. There is a need for a device capable of reliably and economically showing that a certain labelled log was at a certain place at a certain time.
• One embodiment of this invention is intended to be moved about a dockside area, for example, capturing images of labels on bundles of logs stacked on cradles ready for shipping, extracting the data within the labels and forwarding it to an administrative computer facility which keeps track of the progress of the goods in question.
• This invention has increasing relevance as the value of the items it can record rises, as the volume of material in transport rises, and as the "just-in-time" approach to the movement and delivery of goods becomes more widespread.
• the modularity of this invention resides in the manner in which a functional assembly can be put together using components of different capacities yet of similar function. Thus it may be vehicle-mounted (Fig 1), mounted on a recording mast within a goods yard, (Fig 15), or carried by an operator (Fig 14).
• a preferred identification label which generally comprises an encoded or text-character label having a distinctive outer frame, which helps to make the label recognisable against a background of any realistic degree of image complexity.
• the preferred image locating means of this invention comprises solid-state, charged-coupled devices (CCD) as image capture devices, preferably with automatic control of orientation, focus, light levels, and viewing quality.
• CCD solid-state, charged-coupled devices
• the label identification application uses image analysis software that has been developed to search out and decode any labels that may occur within an image.
• a preferred CCD array has 4096 x 4096 pixels although the apparatus can function with a lesser or a greater resolution, and 2048 x 2048 pixel cameras have been used. Trials have been conducted with scanning linear arrays (eg 3084 x 1) but XY arrays (eg 4096 x 4096 or greater) are preferred. 6144 x 6144 pixel arrays have recently been announced.
• a 6144 x 6144 array digitised to 8 bits per pixels then compressed by 40:1 requires just under one Mb of storage instead of the 37 Mb as first extracted from the CCD array, and a 2048 x 2048 array image can be compressed to 100K from the original 4 Mb. (These compressed sizes are approximate; they depend somewhat on picture content and on the control information stored with the image so that it can be used in decompression.
• Fig 16 illustrates a compact camera having several data reduction circuits included within its back. While this camera has been developed for use with the scan vehicle, (see below) it may be used in other types of mounts.
• a scan system will be mounted on a small vehicle 101, 301, 401 (as shown in Figs 1, 3, and 4) and is for use in identifying objects during handling, transport, or in storage, such as the ends of rods, pipes, bottles or logs 102 which have no normal single positional orientation about their axis of rotation.
• FIG. 14 Another embodiment 1400 of the modular scan system is shown in fig 14. It will preferably be "person-mounted" and battery-powered in the same manner as a news video cameraman - with a hand-held camera 1402 and viewer 1403.
• a cable 1406 may connect via a plug 1407 into a backpack 1404 containing preferably miniaturised electronic equipment 1405 to transmit label contents to a remote station.
• the cable 1406 and the plug 1407 may connect the camera to a docking station for downloading the stored information onto a local-area network; preferably fibre-optic based or alternatively using copper wires or radio.
• This version might be used to take stock of stacks of timber in storage in a yard.
• the camera employs a high-resolution CCD array as described elsewhere in this specification.
• a further embodiment 1500 can be mounted on a mast 1505, or wooden pole or on a building or some other fixed structure.
• a camera 1502 is connected by cable to a control box 1503 containing preferably miniaturised electronic equipment to compress, store, and transmit label contents from an aerial 1504 to a remote station. Both the camera equipment and the data compression apparatus may be incorporated in the same box. Alternatively, fibre-optic cable may be used to transmit information.
• This embodiment may be used to automatically record logs 1501 loaded on a transporter, forced to stop with the labelled area in position at a predetermined spot (see below - "Example application for automatic single-scan system").
• the camera employs a high-resolution CCD array as described elsewhere in this specification.
• An application (Fig 1) for such a mobile scan vehicle is in the timber export industry; in the dockside capture of pictures of labeled logs 102 for later automatic recognition of the logs from the labels in the pictures.
• the advantages of this invention are: simultaneous capture - from a safe distance - of the data pertaining to an entire cradle of logs, and high reading accuracy, substantially independent of the ambient light and other conditions.
• the resulting information defines the presence of an identified log at an identified site at an identified time. (Global positioning satellite receivers or the like may be used in conjunction with the recording apparatus in order to indicate position and time).
• a nine character identification is assigned to the log.
• a computer generates two copies of a (for example) 18 cm diameter or diagonal plastic label incorporating the assigned identification, preferably in a form providing duplication and redundancy in case of partial defacement of the label, and preferably in a form providing a distinctive border for computer recognition of the label.
• the plastic sheets are then stapled or otherwise affixed to the opposite ends of the log.
• the usual procedure is for a grab to pick a load of logs and carry them across to a cradle where a crane is then used to load the cradle load onto a ship for shipment.
• a bunting machine bumps the ends of the logs to align them for lifting with the crane.
• the scan vehicle is driven to the scanning position facing one end of the cradle of logs and takes preferably three overlapping pictures of the log ends, including the labels identifying the cradle.
• the three pictures are obtained, in the preferred embodiment, by moving the camera to each of three assigned positions along the guide rail (106, 302, 402, 502, 602, 702). Each position is predetermined by placement of a fixed "target" (519, 313, 314) holding a pattern of metal plates in alignment with the path of an array of three proximity detectors 518 on the moving platform (536, 520, 620, 720) supporting the camera, and control means (within the box 311/411) follows a sequence in order to capture sets of images.
• the log-holding cradles will preferably have permanent labels attached which uniquely identify the cradle and the side of the cradle.
• the scan vehicle may carry Global Positioning Satellite receivers to automatically record position and time along with the pictorial data.
• Pictures (which may initially be up to 16 Mb in size) are preferably compressed as part of the capture process (see later).
• Image analysis preferably occurs on board the vehicle or other image capture device (as indicated in Fig 2, for the sake of immediately indicating the occurrence of unreadable captured images, to avoid unnecessary volumes of data storage or data transmission) but may be arranged to occur at a base station or a base computer facility after transmission by (for example) radio.
• the vehicle then moves around to the other side and takes a similar picture of the log ends on the other side of the cradle, in order to provide further duplication of data, in case particular log ends are visible at only one end, or in case only one end of one or more logs received or retained an at least partly intact label.
• the cradle of logs is then strapped and lifted onto the ship.
• the scan vehicle may thereupon move about the dockside area to record from another cradle which has been loaded and bunted in the meantime.
• a record which includes information that certain logs were situated at a certain site at a certain time. This is communicated to a base station for storage, later viewing, and analysis to identify the logs in the cradle and correlate that information with (for example) ownership, weight/volume, or type.
• Fig 1 Multiple vehicles of the type shown in Fig 1 can be used together on the wharf and each vehicle can handle scanning for multiple cradles depending on the timing of the operations.
• Another application for such a scan system is the capture of pictures of labeled logs at or soon after their initial collation in a marshalling yard, for later automatic recognition of the logs from the labels in the pictures.
• a nine character identification is assigned to the log.
• a computer generates two copies of a 18 cm diameter plastic label incorporating the assigned identification.
• the plastic sheets are then stapled to the opposite ends of the log.
• Fig 15 illustrates the recording arrangement that may be used in such an area.
• Mechanical devices about the recoiding area will preferably positively force the truck into proper alignment at a fixed position with the fixed-mount scan system.
• the scan system then causes the light levels to be at an appropriate level by energising floodlights (like those as 303, 304) and automatically takes a picture of the visible ends of the logs.
• the image is then compressed and transmitted by radio or by fibre-optic or high-speed data cable along with the truck number to a base station for storage, optional later viewing of the image, and analysis to identify the logs on the truck.
• An application for the portable scan version of this system is the capture of labels when the use of vehicles is inconvenient or not feasible.
• labelled warehouse goods are stacked in bins along a wall with labels visible from a distance, or a stack of logs may lie within a yard.
• the operator takes the portable camera 1402 (as per Fig 14) and takes a picture of at least some of the merchandise, adjusting zoom and focus and camera light controls in accordance with readings given in the viewfinder 1403.
• the picture is automatically compressed and stored in the portable computer 1405.
• a sequence of such pictures is taken which encompass all visible labels of the total of the merchandise.
• the camera system is then taken back to a docking station in an office or tally shed and the camera pictures transferred to an analysis computer for label recognition and transfer of label numbers to an inventory tracking system. (Alternatively a radio link may be used to send compressed pictures during the procedure, so enabling the operator to be quickly alerted of any inadequacy in the images.
• Fig 3 shows a front view of a vehicle 301 equipped for log recording.
• the structure bolted to the front of the vehicle may be regarded as (a) support means for a camera guide or traversing rail 302, (b) the camera assembly 310 and (c) support means for ancillary devices used to enhance the captured image (such as floodlights (303, 304) preferably ultrasonic range finders for automatic focus purposes (307, 308) and a TV camera 309).
• the scan or image-capture camera (105, 310, 410, 515, 902) and guide rail (106, 302, 402, 502, 602, 702 will be mounted (as by bolts 434) on the front of the vehicle 401 along with dual rangefinders (107-108, 305-306) and floodlights (109-110, 303-304).
• Protective bars (not shown) will provide crash protection for the scan equipment.
• Power 436 and signal lines 435 (Fig 4) will attach the camera and electronics to the vehicle power (430) and computer equipment (431, 432, 433) inside the vehicle.
• a TV camera (111, 309) mounted in front and below the image-capture camera will be connected to a monitor (112, 321, 421) inside the vehicle positioned next to the driver alongside the operator console (112, 320, 420, 1010) used to control camera capture actions and control the onboard computers (431, 432, 433).
• a motor-driven alternator and inverter (114, 430) may be attached to the vehicle to provide additional power for the camera system.
• An antenna (113, 324, 424) will be mounted on or under the vehicle roof to relay data between the mobile scan vehicle and a base station.
• a TV monitor 320 Inside the vehicle there is a TV monitor 320, a control console 321, a computer 323, 432, 433, such as a 386 PC clone, and interfacing boards which are adapted to function with the camera control circuitry in a box at 311. (See also the block diagram of Fig 2.)
• the camera 310 is caused to take preferably three pictures of the labelled goods (e. g. logs) from both ends and the centre of the traversing rail 302.
• the rail is preferably a square section rail mounted on a horizontal axis, with its walls at 45 degrees to the vertical.
• the camera is supported by running wheels (503, 504, 603, 703, 704) within a frame 536 and provided with compression springs 530 to bear against the rail.
• Preferably brushes or the like are provided to keep the running surfaces clean during use in dusty or muddy conditions.
• a second frame 520 can swivel (about bearings 531 under control of a swivel motor 532) so as to cause the camera to face slightly towards the centre line of the vehicle 101 when at either end of the guide rail.
• the camera 310 is also provided with means for tilting on its optical axis.
• a preferred mechanism to control this traversing movement comprises (a) an array of proximity sensors 518, (b) three inset plates 519, placed at each end (313, 314) and at the midpoint of the rail (obscured in Fig 3). Each inset plate has a distinct arrangement of metal strips so that the proximity sensors can determine camera position. Two limit switches (315, 316, 533, 733) are provided.
• a motor 535 drives, a long toothed belt (317, 534, 634, 734) via a reduction gear, attached to the camera platform, in order to cause traversing motion.
• motion of the belt is detected by means of an attached toothed wheel and a proximity sensor capable of detecting passage of a tooth.
• the toothed wheel is remote from the driving mechanism so as to better sense actual belt motion.
• the motor section is controlled from within one enclosure; this being made responsible for the camera's position along the rail, range flnder flaps, and various motors and pneumatic actuators within the camera assembly.
• this controller also manages the digitisation of the rangefinder data.
• it also includes a digital output to drive various illuminating floodlights.
• a fibre-optic link is used to convey compressed pixel data from the camera unit to the PC computer, and a CAN bus (or the like) link (a two-wire omnidirectional serial bus using open drain wired-or drivers similar to but more powerful in terms of speed, addressing, and instances than the I 2 C bus, interfaced through microprocessors of the 80XX51 family) is used for all other data.
• the CAN bus may be conventionally driven from a board within the PC, with CAN addresses mapped so that various remote boards appear to the PC to be at separately addressable I/O ports.
• Each module is preferably given a unique computer-readable serial number so that the controlling computer can test whether any module of a given system has been changed, indicating that calibration may be in jeopardy.
• this serial number is embodied in a one-time-programmable memory chip as a 24-bit number.
• the motor enclosure contains a 256 byte EEprom (e.g. type Philips PCF8574) within a totally sealed box.
• This chip includes a serial number and calibration information for the rail.
• the camera identifier module it would include the standard settings for that camera head.
• Fig 5 also shows the interior of the camera housing 510.
• TM semiconductor-vapor deposition
• TM semiconductor-vapor deposition
• TM semiconductor-vapor deposition
• our invention high resolution camera with inbuilt data compression
• the modified "Pentax" camera 515 is shrouded by housing 510 from water, foreign matter, and from stray light. There is a motor-driven front flap which swings up when an image is about to be captured. Neutral-density filters 512, 513 may similarly be swung into or out of position in order to reduce the intensity of strong light. Circuit boards 508, 516, and 517 correspond to "CCD control board", “S&H, A->D board", "PC Header + JPEG Compression” in Fig 2.
• a fibre-optic cable is used to convey image data as photon streams to the "PC peripheral" board incorporated in or adjacent to the image-processing computer.
• Optical coupling has been found to minimise induced noise, yet has a high signal-carrying capacity.
• a hydraulic or pneumatic version of this camera housing is described later.
• Fig 16 shows an alternative sealed camera module, intended to be opened up for service in only a controlled laboratory environment.
• Various electronic modules are now included in it, to cause CCD chip readout to occur and then to accept the analogue "video" signals from the CCD chip, digitise, compensate, and compress the signals and transmit them - preferably as serial data in a fibre-optic cable - ready for passage to a base station or other image analysis device.
• Fig 16, 1600 is the entire assembly
• 1601 is the camera lens
• 1603 is the image plane formed by the lens of an object at the predetermined distance
• 1604 is the environmentally secure case
• 1607 is a CCD array
• 1608 is a circuit board carrying sample and hold chips and also serving as a base for the four
• 1606 is a light-tight and electrically shielded barrier separating various circuit boards 1611 (and the like - such as the video compression board(s)) from the sensitive areas around the CCD chip
• 1605 is a series of connectors used for a CAN bus used for control (including adjustment, calibration, and test purposes) and also power connections
• 1610 is a fibre-optics connector used to transmit the compressed picture data from this assembly at high speed.
• high-speed electronic circuitry radiates much more electromagnetic interference than low-speed ones and CCD chip input amplifiers which work at millivolt levels yet at a wide bandwidth are likely to pick up this interference.
• the preferred boards provide for remote adjustment of all parameters such as CCD voltage levels. They preferably include a 32K byte RAM, for which one use is in accepting various JPEG tables from the host computer at startup time, then passing them to the JPEG board (with the help of a local microcontroller) in response to a single instruction.
• JPEG board with the help of a local microcontroller
• the film back is here replaced with a spring-loaded assembly extending from the rear of the camera body that holds the active area of the CCD chip in the focal plane of the lens.
• the camera includes at least one solid-state CCD light-sensitive array.
• the camera may have a shutter and this may be used either or both to protect the CCD array from inadvertent strong lights (such as the sun) and to define the actual exposure, although for the latter purpose the actual exposure can also be determined electrically by means of bias voltages applied to the chip.
• Typical CCD chips require a number of electrical inputs; some are steady and are used to provide suitable biases across semiconductor junctions for light (i.e. photoelectron) storage, and others may from time to time oscillate between states so as to transfer the resulting charges out from each pixel and into an output connector.
• High-resolution chips commonly provide typically two or four output connectors connected to different areas of the image-sensitive area, so that several sectors of the image may be read out at the same time.
• the (or each) output signal which is an analogue or "video" signal representing the charges in each pixel read out in a sequence, is preferably buffered and is then fed to a device which converts it to a digital format.
• Preferred devices are known as flash A to D converters because they offer a suitably high conversion rate, which is preferably at a rate such that each pixel of the CCD is also digitally represented as one byte (or other word).
• the parallel streams of analogue information leaving the CCD array are first "dissected" to extract the video information from DC offsets and pulses, then digitised as parallel streams by converters clocked to work in sequence, the digitised pixel values are passed to individual inputs of a JPEG board, where each is handled separately by a process including a DCT algorithm, the output of which is quantised and Hoffman-compressed, and combined into one stream at the output of the preferred JPEG hardware.
• This preferred hardware is provided with several inputs - originally intended for use as colour inputs (RGB or CMYK) and we find it convenient to tie the separately digitised camera outputs because compressing several streams of information in parallel, as it flows out of the CCD chip, saves a great deal of time. Examples of effects that may require compensation measures to be applied include
• Compensatory values may be stored in a non-volatile memory beside the camera and read back at the time of compression, or they may be stored in the host computer and subtracted prior to computation, though this approach may lead to extra work for the compression utility and it may reduce the working range of the camera (256 levels, without gamma correction).
• Thermal cooling or temperature control for example using a Peltier thermo-electric device, may be used as an option, especially in hot environments. Although there is little need in this application to cool the CCD to the astronomers preferred values of -60 degrees C or below, to nullify dark currents, it may be useful to maintain it at a known and substantially constant temperature.
• One preferred method for compensating for the baseline drift is by dynamic baseline correction. This method holds compensatory values in a lookup table (2 x 8K byte RAM or ROM or EEProm) which is addressed simultaneously with the column addresses for the CCD readout. Memory data is passed through a digital-to-analogue converter and summed with the video data in an operational amplifier so that after compensation the video signal has a substantially constant baseline.
• both the CCD video amplifier/buffer chips and the clock chips should be close to the actual CCD.
• Sample construction practices are the use of surface-mounted devices on printed-circuit boards which include an internal ground plane, good power supply decoupling, and folding the boards so as to pack the integrated circuits into a small space.
• the raw stream of digitised data is preferably compressed so that the total number of pixels required to adequately represent the image is significantly less that the original number of pixels.
• This compression hardware, used with a 4096 x 4096 pixel array reduces the initial 16 megabytes stream of data to typically about 600 Kbytes depending on image content. It is then relatively simple to handle the reliable transfer of 600K bytes over a distance in about 6 seconds; a preferred time between shots.
• JPEG Joint Photographic Expert Group
• JPEG Still Picture Compression Algorithm October 15th 1991, available from C-Cube Microsystems marketing Department, 1778 McCarthy Boulevard Milpitas CA 95035, USA.
• the memory unit can store several compressed images and acts as a kind of pre-transmission buffer which can hold a number of images.
• a suitable memory or buffer is made up of dynamic random-access memory (D-RAM) in one or four megabyte modules, as are used in personal computers; and 4 Mb of storage can store at least five or six typical compressed images.
• D-RAM dynamic random-access memory
• Use of memory modules avoids disadvantages of storage on magnetic media such as physical size, slow data transfer speed, inherently lower reliability, lower capacity, and higher power consumption.
• Hash memory is a useful alternative to D-RAM.
• a further application for a memory unit may be at the communications end of the processing chain, because the preferred use of a handshake protocol requires that the sending of data may be delayed momentarily or for some time, until conditions in the transmission medium or at the receiving station are suitable for transmission.
• the camera may also include a suitable communications adapter, capable of bidirectional communications in the chosen medium.
• a suitable communications adapter capable of bidirectional communications in the chosen medium.
• One preferred medium is fibre-optic cable because of the high speeds (i,e, wide bandwidth) yet excellent electrical isolation that may be achieved with this medium.
• one of our proposed modes of use involves a portable (backpack) type of camera with storage for about six captured and compressed images, and a docking station at which the stored images may be dumped into a tap made into a fibre-optic "bus" running the length of a wharf or dock.
• This docking station may also include means for recharging the batteries of the portable camera.
• Alternative data transmission means include radio (preferably at a high frequency as the link may be required to have a bandwidth of the order of 1 MHz), and : 'Wavelan" (TM) is one example of a product adapted to transmit digital data over moderate distances, or infra-red beam communications, or conventional conducting cables. If direct connection is used, interference due to earth loops and the like (such as from a high-speed computer) should be minimised.
• radio preferably at a high frequency as the link may be required to have a bandwidth of the order of 1 MHz
• TM 'Wavelan
• TM is one example of a product adapted to transmit digital data over moderate distances, or infra-red beam communications, or conventional conducting cables. If direct connection is used, interference due to earth loops and the like (such as from a high-speed computer) should be minimised.
• the entire assembly is contained within an environmentally resistant housing, and preferably this housing is compact.
• environmental hazards include mechanical damage through collision or abrasion, heat damage through solar or floodlight radiation, and electrical damage through accidental contacts or electromagnetic radiation. Additional protection to shield the sensor itself from excessive light is also preferred.
• the speed of the process of communicating an originally 16 million pixel image from the CCD chip within the camera through the JPEG compression device and into its internal storage means as an about 600K version of the image is at the present time about 6 seconds although further circuit improvements may render this time much shorter.
• the JPEG processor operates in real time; that is, there is no perceptible time delay imposed on the transfer of information from the CCD chip into memory as a result of compression.
• the invention has a number of advantages over the usual frame grabber which is generally an accessory board plugged into a bus within a computer that creates a mapped representation of an image in digital form in a memory plane, the advantages including:
• subsequent data analysis may be faster and require fewer computer resources if applied to a more compact data structure (the compressed version of the image).
• the invention as a combination of a machine- vision camera and compression unit, it has further advantages including that it brings high-resolution digital image capture into almost the real-time environment; that is, it provides for the repetitive capture and handling of high-resolution images at rates compatible with a number of industrial and transport procedures. For example it converts a 4096 x 4096 pixel camera image into transmittable data in about 6 seconds, or at least it is ready to take the next picture in about 6 seconds.
• test exposures for example to determine light levels, focus, or subject matter .
• Figs 6 and 7 illustrate the attachment of the camera platform 730 to the guide rail (602, 702).
• the use of a toothed belt 634, 734 provides a positive drive to the camera platform.
• Figs 8 and 9 illustrate means to adapt the CCD chip 909 to a conventional camera 901.
• the CCD chip is mounted on a metal plate 904, which is pressed into contact with the film guides 903 by two or more spring-loaded legs 800.
• Each leg includes a compression spring 802 which presses a telescoping peg 801 from within the hollow cylindrical shaft 803.
• Peg 804 serves to limit the maximum movement of the parts 803 and 801.
• pegs 800 are installed in the camera as at 905.
• Wires 907 from the CCD chip socket are taken to connectors 906.
• Optional CCD chip cooling means may be installed; for example a miniature fan or a Peltier cooling module may be placed behind the chip.
• Fig 2 illustrates in block form the path of the signals representing an image.
• the CCD 4K 2 (4096 x 4096) block represents the camera chip, mounted within a camera body as in Fig 9. These devices require a number of steady voltages or sequenced arrays of pulses in order to clear the array, to retrieve image signals, or to put the array into a light-sensitive mode.
• the CCD control board generates the required voltages.
• In order to convert the analogue levels into computer-compatible form two analogue-to-digital converters are provided, as there are two output lines from the preferred type of chip.
• Image compression preferably according to the JPEG format but optionally in another format is preferred in order to reduce redundancy and volume in the potentially large image.
• the signal is passed, preferably along a fibre-optics link 435 (Fig 4) to a mini-computer, preferably of the PC type, via a peripheral or board adapted to receive fibre-optics data.
• a mini-computer preferably of the PC type
• the image data is optionally stored and optionally transmitted by radio or possibly a cellular telephone modem to a remote base station.
• Fig 16 illustrates a physical arrangement for placing such boards in close proximity to a CCD camera chip.
• the operator's control display is illustrated at 1010 in Fig 10.
• 1011 is a LCD display indicating the rangefinder readings
• 1012 is a bar LED display to indicate computer progress
• 1013 is a steering guide indicating off-centre states
• 1015, 1016, and 1017 are pushbuttons to start lining up, to capture an image, or to abort an operation.
• An optional key switch 1014 is also provided.
• the operator's console makes use of one or two integrated circuits linked by the I 2 C bus to a microprocessor.
• One of these chips is capable of either generating an informative video overlay, electrically mixed on the TV monitor which forms the rangefinder, and the other (or both in a different embodiment) can activate lamps under button switches and detect any switch depression, in order to lead the operator to a suitable action.
• FIG. 11100 shows a top view of a camera case, with an inspection aperture 1104, an outer flap 1102, an inner flap 1106, a pneumatic/hydraulic cylinder 1103 to drive the outer flap, another pneumatic/hydraulic cylinder 1105 to drive the inner flap, fluid reservoirs 1101, and linking pipes.
• 1107 is a sequencing control valve.
• 1200 shows the case from one side.
• 1202 is the pneumatic/hydraulic cylinder to drive the inner flap 1205
• 1201 indicates the position of the inspection flap
• 1211 is a cylinder to move the neutral density filter 1206
• 1210 is an upper limit switch for the movement of 1211
• 1207 is a lower limit switch.
• 1212 indicates two fluid reservoirs. The arc indicates the travel of the protective outer flap 1204.
• 1300 is the front view of the hydraulic camera assembly.
• 1302 is the position of the internal cylinder for operating the neutral density filter.
• 1301 is the end of the outer flap cylinder (1202).
• 1303 is the end of the inner flap cylinder.
• 1304 is a fluid reservoir.
• the following example shows a typical interactive instruction sequence for control instructions for the log identification when a dual-scan vehicle is available.
• Scan unit continually measures distance of two rangefinders from cradle and reports results on the operator display.
• a TV display is also operational.
• the driver maneuvers the vehicle using rangefinder measurements and the TV image to align the assembly on the front of the vehicle perpendicular to the cradle of log ends and at substantially the required distance.
• the computer opens the cover flap(s) over the scan module, measures light, adjusts the camera lens aperture and if necessary turns off or on the lights, or inserts or removes a neutral density filter, to ensure that the light level as seen at the CCD is appropriate.
• These exposure adjustment means may be automatic.
• the camera is also moved to one end of the guide rail.
• the computer takes the first picture.
• the computer then closes the flaps and begins moving the camera to the central position on the guide rail while the image is being compressed and sent to the computer in the vehicle.
• the computer begins to transmit the image or extracted data over radio back to a base station, optionally also storing it in case of poor reception.
• the camera flaps are opened and the second picture is taken from the central position in the same way as the first picture.
• the third picture is taken from the third guide rail position the same way as previously.
• the driver (who may have aborted the sequence by way of the Abort button 1017) is notified of capture completion by the operator console lights going off.
• the last picture is compressed and information is transmitted to the base station while the camera is returned to the central guide rail position and also while the driver drives on to the next cradle position.
• Steps 1 to 14 are repeated for both sides of each cradle and for each new cradle full of logs.
• a variation of the above involves the use of a backpack camera, which is carried about the wharf by a person who photographs first one end of the cradled logs and then the other, preferably with three photographs of each end, and then returns to a docking station in a tally hut to download the compressed images to an analysis station while the logs from the cradle are being slung and then hoisted into the ship.
• a fibre-optic cable run along a slit cut into the wharf and brought out to a docking point inside each tally hut.
• the design of the self-contained camera in the form described above having as its output an image in compressed format for relaying by optical fibre to a nearby radio transmitter may be altered for certain applications by including the radio transmitter within the self-contained camera.
• a block diagram of this embodiment is shown in Fig 17.
• Surveillance of remote areas with a high-resolution camera e.g. 16 million pixels resolution
• the camera may survey remote airfields to record the identification markings on aircraft, where the high sensitivity of this type of camera allows it to work under conditions of low light intensity.
• the compression of the video signal by a built-in compression algorithm in hardware or software is an advantage, to help overcome the volume of traffic to be transmitted.
• the wide bandwidth of the compressed signal suggests use of UHF radio links in order to accommodate the bandwidth and in order to carry UHF links over a distance a satellite link may be preferred if there is no nearby microwave relaying means.
• use of extreme forms of compression may compress a high-resolution picture to such an extent that it can be transmitted on a short-wave radio link or an ordinary telephone line. Fractal compression as it is presently applied tends to be rather a lengthy process for the materials-handling applications that form the bulk of this specification, but in other applications a processing time of some minutes may be acceptable if a transmission duration of perhaps an hour is to be avoided.
• 1700 indicates the entire module which is preferably provided in a weathertight and damage-resistant enclosure.
• 1701 is a lens, which for this purpose may be motorised and indeed the entire module may be provided with a pan and tilt mechanism, driven locally or remotely - though the high resolution of the preferred CCD devices 1702 may render movement, and the use of lenses of longer focal lengths relatively unnecessary.
• the module may be provided with means to check a newly acquired image against an older image, and then automatically (a) transmit the new image (or just the altered part or parts of it) to a remote station, and/or (b) close in on the altered parts and send a more detailed image of the altered parts.
• the data compression module 1703 is preferably a board implementing the JPEG compression procedure, though other compression modules (if any) are alternatives. Recent developments in video-on-demand (VOD) via telephone lines may provide further alternative effective data compression systems.
• the transmitter module 1704 is conveniently a UHF transmitter though other radio frequencies and even microwaves may be used.
• 1705 is an aerial here shown as a directional Yagi type with a fixed mounting though if a low-orbiting satellite is used as a relay station a more omnidirectional aerial may be preferred.
• 1706 represents a storage battery (preferably including power management) and 1707 represents for example a solar cell array to maintain a charge in the battery.
• the design of the camera may be varied in a number of ways depending on the requirements of the particular application.
• the limiting resolution of the system may be varied to suit the application; for example by using CCD arrays of between 512 x 512 and 4096 x 4096 pixels (the range of sizes commercially available and under test at the time of preparing this specification) or a greater number.
• Range finding features may be removed if not required by the application environment.
• Internal light control may be removed if not required by the application environment.
• the scan system design may be varied in a number of ways depending on the requirements of the particular application.
• Standard video cameras and signals may be used instead of the preferred high-resolution CCD devices and resultant readout signals.
• the CCD resolution may be reduced by defocusing, image averaging, combining pixels, or other means.
• Light control may be removed if not required by the application environment
• flashlights may be used instead of floodlights. Typical exposure durations are of the order of 2-3 mS.
• the response of the camera may be restricted to one or more specified spectral bandwidths, to take advantage of object reflectivities, lens performance, or the like.
• Range finding may be eliminated if not required by the application environment.
• the sensitivity of the CCD devices allows small lens apertures to be used, although diffraction limiting of allowable apertures should be considered. For this reason a neutral-density filter is a preferred method for reducing the incoming light as it permits the lens aperture to remain wider.
• Identification of containers for groups of items may use the same system used for the items being scanned and be scanned at the same time.
• identification of containers may be done by another means or not at all depending on the application requirements.
• Controlled cover flaps for the scan units may be eliminated if the unit is not to be used in adverse environmental conditions.
• More cameras may be used to simultaneously scan larger areas.
• Rotational alignment of cameras may be eliminated if not required by the application.
• Identification of location of the container for groups of items may be predetermined and stored with the container identification so that only the container identification is required to locate the container.
• Scan units may be scaled up or down as required to accommodate the containers and items being recorded.
• the guide rail may be eliminated if not required.
• the bumper support frame may be shaped differently if required by the application.
• the modular unit may be attached to different kinds of platforms such as poles, boats, planes, cars, and robot arms.
• this device is by no means limited to felled timber or even to labelled merchandise in various containers. It could be used to monitor rolling stock on a railway. It could be used for rolling inventory in a warehouse or for monitoring the usage of supermarket trolleys in a supermarket Advances in the automated recognition of the human face, for example, may allow an automated roll call to be made at a school.

Landscapes

• Physics & Mathematics (AREA)
• Electromagnetism (AREA)
• Engineering & Computer Science (AREA)
• Health & Medical Sciences (AREA)
• General Health & Medical Sciences (AREA)
• Toxicology (AREA)
• Artificial Intelligence (AREA)
• Computer Vision & Pattern Recognition (AREA)
• General Physics & Mathematics (AREA)
• Theoretical Computer Science (AREA)
• Closed-Circuit Television Systems (AREA)

Applications Claiming Priority (7)

Publications (2)

Family

ID=27353697

Family Applications (1)

Country Status (3)

Families Citing this family (12)

Citations (5)

Family Cites Families (5)

• 1994
  • 1994-08-31 AU AU76669/94A patent/AU7666994A/en not_active Abandoned
  • 1994-08-31 WO PCT/NZ1994/000090 patent/WO1995007000A1/en not_active Application Discontinuation
  • 1994-08-31 EP EP94927108A patent/EP0677227A4/de not_active Withdrawn

Patent Citations (5)

Non-Patent Citations (4)

Also Published As

Similar Documents

Legal Events