WO2001068326A2 - Camera intelligente - Google Patents

Camera intelligente Download PDF

Info

Publication number
WO2001068326A2
WO2001068326A2 PCT/US2001/007586 US0107586W WO0168326A2 WO 2001068326 A2 WO2001068326 A2 WO 2001068326A2 US 0107586 W US0107586 W US 0107586W WO 0168326 A2 WO0168326 A2 WO 0168326A2
Authority
WO
WIPO (PCT)
Prior art keywords
smart camera
camera module
host computer
accordance
actuator
Prior art date
Application number
PCT/US2001/007586
Other languages
English (en)
Other versions
WO2001068326A3 (fr
Inventor
Edison T. Hudson
James Mccormick
Ronald G. Genise
Jerome Dahl
Original Assignee
Meta Controls, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Meta Controls, Inc. filed Critical Meta Controls, Inc.
Priority to AU2001243537A priority Critical patent/AU2001243537A1/en
Publication of WO2001068326A2 publication Critical patent/WO2001068326A2/fr
Publication of WO2001068326A3 publication Critical patent/WO2001068326A3/fr

Links

Classifications

    • H—ELECTRICITY
    • H05—ELECTRIC TECHNIQUES NOT OTHERWISE PROVIDED FOR
    • H05K—PRINTED CIRCUITS; CASINGS OR CONSTRUCTIONAL DETAILS OF ELECTRIC APPARATUS; MANUFACTURE OF ASSEMBLAGES OF ELECTRICAL COMPONENTS
    • H05K13/00—Apparatus or processes specially adapted for manufacturing or adjusting assemblages of electric components
    • H05K13/08—Monitoring manufacture of assemblages
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Programme-controlled manipulators
    • B25J9/16—Programme controls
    • B25J9/1694—Programme controls characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
    • B25J9/1697—Vision controlled systems
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00—Program-control systems
    • G05B2219/20—Pc systems
    • G05B2219/21—Pc I-O input output
    • G05B2219/21053—Each unit, module has unique identification code, set during manufacturing, fMAC address
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00—Program-control systems
    • G05B2219/20—Pc systems
    • G05B2219/23—Pc programming
    • G05B2219/23298—Remote load of program, through internet
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00—Program-control systems
    • G05B2219/20—Pc systems
    • G05B2219/25—Pc structure of the system
    • G05B2219/25072—Initialise each module during start up
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00—Program-control systems
    • G05B2219/20—Pc systems
    • G05B2219/25—Pc structure of the system
    • G05B2219/25166—USB, firewire, ieee-1394
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00—Program-control systems
    • G05B2219/20—Pc systems
    • G05B2219/26—Pc applications
    • G05B2219/2651—Camera, photo
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00—Program-control systems
    • G05B2219/30—Nc systems
    • G05B2219/34—Director, elements to supervisory
    • G05B2219/34012—Smart, intelligent I-O coprocessor, programmable sensor interface
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00—Program-control systems
    • G05B2219/30—Nc systems
    • G05B2219/37—Measurements
    • G05B2219/37494—Intelligent sensor, data handling incorporated in sensor
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00—Program-control systems
    • G05B2219/30—Nc systems
    • G05B2219/37—Measurements
    • G05B2219/37555—Camera detects orientation, position workpiece, points of workpiece
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00—Program-control systems
    • G05B2219/30—Nc systems
    • G05B2219/39—Robotics, robotics to robotics hand
    • G05B2219/39391—Visual servoing, track end effector with camera image feedback

Definitions

  • the present invention relates to machine control systems.
  • the invention has broad applicability to machine systems requiring controllers which control actuators and/or monitor sensors. More particularly, the present invention is directed to electronic cameras. Still more particularly, the present invention is directed to machine vision systems employing electronic video cameras.
  • Machine control systems are well known in the art. Such systems include, for example, systems for controlling robotic assembly equipment such as pick and place (or placement) machines.
  • a placement machine is a robotic instrument for picking up electronic and similar parts from component feeders and placing them at their assigned locations on a printed circuit board (PCB). Once all parts are placed, the PCB is placed in a reflow oven and solder paste disposed on the PCB melts forming permanent electrical connections between pads on the PCB and electrical contacts, leads or "pins" on the electrical components.
  • PCB printed circuit board
  • a smart camera system provides focused images to an operator at a host computer by processing digital images at the imaging location prior to sending them to the host computer.
  • the smart camera has a resident digital signal processor for preprocessing digital images prior to transmitting the images to the host.
  • the preprocessing includes image feature extraction and filtering, convolution and deconvolution methods, correction of parallax and perspective image error and image compression. Compression of the digital images in the smart camera at the imaging location permits the transmission of very high resolution color or high resolution grayscale images at real-time frame rates such as 30 frames per second over a high speed serial bus to a host computer or to any other node on the network, including any remote address on the Internet.
  • FIG. 1 is a schematic diagram of a machine control hardware architecture as applied to a placement machine in accordance with a specific embodiment of the present invention.
  • FIG. 2 is plan view of components of a hardware module including a common communications device and a unique function device in accordance with a specific embodiment of the present invention.
  • FIG. 3 is a system block diagram of a module in accordance with a specific embodiment of the present invention.
  • FIG. 4 is a schematic diagram of a distributed machine control system in accordance with a specific embodiment of the present invention.
  • FIG. 5 is a block diagram of a camera-based imaging system in accordance with the prior art.
  • FIG. 6 is a block diagram of a smart camera system in accordance with a specific embodiment of the present invention. DETAILED DESCRIPTION
  • the components, process steps, and/or data structures may be implemented using various types of operating systems, computing platforms, computer programs, and/or general purpose machines.
  • devices of a less general purpose nature such as hardwired devices, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), or the like, may also be used without departing from the scope and spirit of the inventive concepts disclosed herein.
  • a hardware architecture links control modules to a host computer in a high-speed serial bus network to achieve efficient distributed machine control.
  • Each control module includes a communications unit and a function unit that are coupled to each other as a part of the control module. All communications units are similarly constructed with each having a unique identification set by a component thereof and provide communications between control modules and the host computer.
  • the function unit of each control module is distinctly configured for executing functions specific to the particular control module to which it is attached.
  • the high-speed serial bus provides deterministic synchronization of data transfers between control modules and the host computer using either an isochronous mode and protocol or an asynchronous mode with a fixed real-time clock issuing data requests at fixed time intervals.
  • Each control module maintains data blocks containing the desired control status from the host computer and the current state of all variables related to the particular control module. These data blocks are available to the host computer and each control module at fixed time intervals without the need for data requests. This permits near real-time intervention or change of state within the controlled machine processes.
  • FIG. 1 A specific embodiment of the machine control architecture of the present invention is illustrated in FIG. 1 as applied to a placement machine 100.
  • the placement machine 100 has a pick-up head 102 transportable in X, Y, Z and T (rotational) directions which picks up components 104 (with a vacuum pick-up, gripper pick-up, or similar device) from component feeders 106 and transports them for placement onto a target substrate 108 such as a PCB.
  • the components 104 in accordance with this example are typically electrical, electro-mechanical or electro-optic components and require highly accurate placement onto the target substrate 108 due to typically densely packed input/output (I/O) connections.
  • I/O input/output
  • the placement machine 100 has an imaging system 110 of some kind which observes the components 104 and the target substrate 108 in order to measure, register and align under-side contact and edge features of the components to corresponding target substrate features in order to achieve accurate placement.
  • Placement machine 100 usually includes a number of control modules 112 for driving motors (also referred to as actuators) and sending and receiving digital and analog data.
  • peripherals of placement machine 100 such as the imaging system 110, camera lighting (not shown), pick-up head 102 and vacuum generators (for use with vacuum pick-ups) may be wired to and controlled by specific data I/O lines of control modules 112.
  • Real-time processing of images captured by the imaging system 110 permits calculation of coordinate feature locations for components 104 and target substrates 108 and corresponding control of the pick-up head 102 motion to achieve proper registration and alignment between the component 104 and target substrate 108.
  • Control modules 112 are preferably positioned on or about the placement machine 100 at critical process locations such as motor control axes 109 and the imaging system 110 so as to minimize cabling.
  • the control modules 112 are connected in a data communications network over a high-speed serial data bus 114 to each other and to a host computer 116.
  • the data communications network may be a high speed network such as those defined in the IEEE Standard for a High Performance Serial Bus — Amendment 1 (IEEE Std. 1394a-2000 (Amendment to IEEE Std 1394-1995)) (Published by the Institute of Electrical and Electronics Engineers, Inc., 3 Park Avenue, New York, NY 10016-5997 on June 30, 2000) (hereinafter referred to as the "IEEE 1394 bus”), and the like.
  • Each control module 112 includes a common communications device 200 which may be a single circuit board as illustrated in FIG. 2 that provides computing power, volatile and non-volatile data storage, and one or more communication ports 202 supporting the high speed serial bus 114.
  • Each communications device 200 on each control module 112 preferably has the same or a similar physical configuration.
  • the communications device 200 on each control module 112 is coupled to a function device 204 which may be a single circuit board using a standardized electrical and data interface 206 which performs signal conditioning, buffering, and power amplification functions.
  • Each function device 204 is distinctly configured to execute functions specific to the particular control module 112 to which it is coupled.
  • control module 112 The combination of the common communications device 200 with a unique function device 204 constitutes a control module 112 that can perform a variety of functions. These functions include closed loop motion control of various types of motors using digital encoder, analog or video sensor feedback in combination with digital algorithms such as PJ-D (proportional, integral, derivative) control providing realtime position, velocity and torque control and motor phase commutation. Machine vision functions using digital image acquisition and transmission without the use of specialized host resident hardware such as frame grabbers is of particular benefit in the placement machine 100 application.
  • PJ-D proportional, integral, derivative
  • control module 112 Other functional capabilities of the control module 112 include but are not limited to logic control with electrical opto-isolation for input or output signals, analog to digital and digital to analog conversion of voltage, current, or resistance from sensors or other electromagnetic sources, communication bridging and translation to and from other serial (or other) buses such as CANBUS, USB, RS232/422, and Ethernet, power control and amplification using pulse width modulation methods to create variable output levels of power for process functions such as lighting and heating, and real-time process variable control using any of the above digital or analog inputs or outputs.
  • logic control with electrical opto-isolation for input or output signals analog to digital and digital to analog conversion of voltage, current, or resistance from sensors or other electromagnetic sources
  • communication bridging and translation to and from other serial (or other) buses such as CANBUS, USB, RS232/422, and Ethernet
  • power control and amplification using pulse width modulation methods to create variable output levels of power for process functions such as lighting and heating
  • real-time process variable control using any of the
  • control module 112 The distinct configuration and corresponding functional capability of any one control module 112 is uniquely identified by a coded serial number that is hardware embedded and non -programmable in each module 112.
  • Control modules 112 of the same type i.e. modules having identical function devices 204 in addition to common communications devices 200
  • Each control module 112 transmits its coded serial number to the "root node" (which is usually host computer 116) upon a start-up initialization procedure causing the host computer 116 to assign a distinct network address to each control module 112.
  • the host computer 116 then downloads appropriate control firmware that it has stored in an archive to the common communications device 200 in each control module 112 and thus configures the functionality and performance characteristics of each control module 112 as needed based on the application requirements for each control module 112.
  • the capabilities of the host computer 116 controlled architecture combined with the "plug and work" nature of the control modules 112 and high-speed serial bus 114 permits a rapid configuration of many functions.
  • the high speed serial bus 114 linking the control modules 112 and host computer 116 has a high data transfer rate that typically ranges from about 100 megabits per second to about 3 gigabits per second (referred to herein as "high-speed").
  • the serial bus provides a mechanism for time deterministic data transfers such as the isochronous mode of an IEEE 1394 bus.
  • IEEE 1394 bus isochronous mode all real-time control system variables such as PID parameters, motion information and the state of digital and analog I/O are transferred from the host computer 116 to the control module 112 and vice versa at a fixed time interval using the isochronous protocol.
  • sensor data captured by sensors coupled to digital or analog inputs of the control module, can be transferred to the host computer for use in calculations such as PID servo loops, machine vision algorithms, and the like.
  • this can also be accomplished using an asynchronous mode of communication by establishing a fixed real-time clock that issues a read request to each control module 112 at a fixed time interval. Immediately following such a request, the control module 112 issues a write request to load its control variable data to the host computer 116.
  • the synchronous communication between the host computer 116 and physically remote control modules 112 over the high speed serial bus 114 occurs in a memory mapped fashion which emulates the remote control modules 112 being actually resident in host computer 116.
  • Each control module 112 maintains an in-page data block containing a complete set or state table of the desired control state commands from the host computer 116 and an out-page data block containing the current state of all control variables within that particular control module 112.
  • the host computer 116 maintains in-page and out-page data blocks for each control module 112 attached to the network.
  • the time synchronized transmission mode of the hardware architecture guarantees that the state of each control module 112 is known within a deterministic time domain.
  • control variables are available to host computer 116 at a repeatable interval without requiring a specific request for such information.
  • This permits near realtime intervention or change-of-state of any control variable for any control module 112 without the need to interrupt the normal stream of events at the control modules 112 or in the host computer 116.
  • the updating or painting of the state of the in-page and out-page data blocks takes place at the lowest level in host computer 116 so as to have minimal impact on higher level functions.
  • SERCOS the well-known SErial Real-time Communication System
  • an open interface specification designed for high-speed serial communication of standardized closed-loop data, or other known control protocols, no command interpretation or parsing and no polling is required. The updates are simply accomplished automatically and repeatedly at fixed time intervals.
  • asynchronous and isochronous communication modes provided by the high-speed serial bus system, such as in the IEEE 1394 bus, permit a "peer-to- peer" communication. This means that no interaction with the host computer 116 is required to set-up or transfer data to any valid control module 112 within the network. These data transfers do not require permission from, redirection from or routing to the host computer 116.
  • a motor control axis can receive position commands via direct state table or in-page data block updates from another control module 112 performing a sensor function with no host computer 116 interaction.
  • the hardware architecture provides for data error checking with at least two error checks occurring at each transfer of data between a control module 112 and host computer 116.
  • error checks include a checksum on data integrity of the in-page data blocks and the out-page data blocks and a recirculating message sequence number for message order integrity between control modules 112 and the host computer 116.
  • FIG. 3 is a system block diagram of a module in accordance with a presently preferred specific embodiment of the present invention.
  • Module 300 includes a base processor/communications unit 302 and a function unit 304.
  • Base processor/communications unit 302 may be present in each module present in a particular machine control system.
  • Serial number chip 306 provides a unique serial number to processor/communications device 302.
  • Digital signal processor (DSP) 308 may be a model TMS320F240 available from Texas Instruments.
  • Flash memory 310 provides a bootstrap loadable program for initially configuring DSP 308 upon power-up.
  • DSP 308 is generally programmed by the instructions in flash memory 310 to request download of software from a host computer after power-up and bootstrap load is completed. Once the software is downloaded from the host it is stored in volatile program memory 312.
  • DSP 308 communicates off-module via a conventional PHY (physical layer device) and LLC (link layer) 314 with serial ports 316a, 316b and 316c that may be IEEE 1394 bus serial ports and which together form a three-port IEEE 1394 bus hub of which some of the serial ports may be 4-wire (unpowered) ports and others may be 6-wire (powered) ports under the IEEE 1394 standard, as desired.
  • function unit 304 which in this embodiment is disposed on a separate PCB and connected to the PCB upon which processor/communications unit 302 is disposed via a multi- connector block connector that carries various signals and voltage levels.
  • Function unit 304 may contain circuitry and ports supporting such communications activities as Analog Input (318), Analog Output (320), Digital Input (322), Digital Output (324), and various types of motor controls (326).
  • DSP 308 may communicate with function unit 304 via a programmable logic device 328 in a conventional manner.
  • motor control signals are preferably routed via error loop current block 330. If a low current signal routed through various components of the machine control system and through the "IN” and "OUT" ports of error loop current block 330 is interrupted, the signal(s) controlling the motorized components will cease to pass to the motorized components even if the DSP 308 has crashed or locked up.
  • Error loop current block 330 operates, for example, by providing current to the gate of a switch transistor which allows motor control signals to pass from the switch transistor's source to drain.
  • Other electrically controlled switch technologies e.g., relays, opto-isolators, etc.
  • This feature is provided as a safety feature to provide a positive mechanism for motor shutdown in the case of an emergency or detected anomaly.
  • Power block 332 receives power from an external source on lines "+" and "GND”. Chassis ground input 334 is preferably isolated from "GND" input.
  • FIG. 4 is a schematic diagram of a distributed machine control system in accordance with a specific embodiment of the present invention.
  • FIG. 4 illustrates a typical application for the distributed machine control system of the present invention.
  • a six axis motion system 400 under the control of a host computer 402 includes a number of actuators 404 (X-axis), 406 (Y-axis), 408a and 408b (Z-axis), and 410a and 410b (T- axis).
  • Cameras 412a and 412b are IEEE 1394-type cameras and are present to provide machine vision control. Machine vision algorithms and servo-loops are preferably implemented by a corresponding control module (e.g., control module 414b processes machine vision for camera 412a).
  • control modules 414a, 414b and 414c provide control to the various actuators and cameras.
  • Control modules 414a, 414b and 414c may be coupled to one another using an IEEE 1394 bus 416 including flexible serial data cables and may be connected in daisy-chain fashion or in a tree fashion.
  • imaging system 110 is modified to include a smart camera or image sensor which observes the components 104 and the target substrate 108 in order to measure, register and align under-side contact and edge features of the component to corresponding target substrate features.
  • Real-time image processing of images captured by the smart camera with a digital signal processor located at the smart camera permits calculation of coordinate feature locations for components 104 and target substrates 108 and corresponding control of the pick-up head 102 motion to achieve proper registration and alignment between the component 104 and target substrate 108.
  • the realtime image processing at the smart camera location permits the transmission of very high resolution color or high resolution grayscale images at real-time frame rates such as 30 frames per second over a high speed serial bus 114 to a host computer 116 or to any other node on the network, including any remote address on the internet.
  • the smart camera system of the present invention has fully digital camera system architecture which permits the use of a variety of different formats of image sensors coupled with a high speed digital signal processor for real-time image enhancement, compression, or feature extraction and a high speed serial communication bus for communicating the processed or raw images to a network host computer.
  • the smart camera system architecture improves image signal to noise ratios by providing a very short signal path length (i.e. only a few millimeters) for the raw image data signal to travel from the imaging chip to a high speed analog to digital converter where it is converted into quantized raw image data and collated in a temporary storage buffer.
  • the buffer can then directly route the quantized raw image data for transmission over a high speed, isochronous serial bus such as an IEEE 1394 bus or equivalent.
  • the quantized raw image data can be taken from the buffer by an embedded digital signal processor (DSP) for image processing.
  • DSP can interpret, filter, compress or otherwise modify the data in whatever manner selected by a user at a host computer.
  • the DSP has a program and data storage that can be loaded with a library of firmware capable of performing a wide array of image processing computations on the image data.
  • These computations include but are not limited to mathematical operations such as addition, subtraction and multiplication by a constant or by another image or partial image, compression including MPEG, JPEG, and GIF at various compression factors, convolution processes of various data array sizes, deconvolution processes such as wavefront coding and digital focusing, edge detection, filtering and enhancement, centroid location, region connectivity, hole finding, region statistics, intensity normalization, white balancing, saturation and hue adjustment, color interpolation from weighted color filter tables matching an image array filter pattern, feature extraction such as corner, arc, and line segment formation, image rotation, zooming functions, scaling functions, run length encoding, chain coding, morphological erosion, dilation, normalized correlation to stored templates, frame to frame motion detection, motion control vector offset calculations, user defined algorithmic sequences and the like.
  • Compression of the digital images in the smart camera at the imaging location by the embedded digital signal processor provides an enabling advantage for very high resolution images such as those with 4,000 x 4000 pixels.
  • the compression enables the transmission of very high resolution color or high resolution grayscale images at real- time frame rates such as 30 frames per second over a high speed serial bus such as the IEEE 1394 bus to a host computer or to any other node on the network, including any remote address on the internet.
  • a high speed serial bus such as the IEEE 1394 bus
  • the embedded digital signal processor can implement the digital deconvolution so that the image transmitted to the host computer can be immediately displayed and viewed by a human operator in real time.
  • Additional advantages of the smart camera system of the present invention include compressing live, high resolution video images with greater than 1000 x 800 pixels containing 24 bits of color and intensity data per pixel generated at rates exceeding 20 frames per second, such that remote observation of the compressed images is enabled over networks of normally insufficient bandwidth, such as dial-up internet networks. Further, on-command compression may be suspended for one or more frames so that full resolution, non-compressed, single frames may be transmitted over the same network at less than real-time rates.
  • a prior art vision system 500 is illustrated in FIG. 5.
  • An analog image sensor 502 obtains the image and transmits it to a host computer 504 over a cable or transmission line (e.g., of the RS-170 type) 506.
  • the image data transmitted over cable 506 may be either in analog or digital format.
  • the maximum resolution of the image and the frame rate are limited by the bandwidth of cable 506.
  • the finite bandwidth of cable 506 causes jitter and delay in the image which, in turn, lowers the quality and timeliness of the image.
  • the host computer 504 includes a frame grabber 508 which includes an FEP block 510 for performing conventional analog/digital conversion, gain control and balance. It also includes a frame memory 512 and a CPU 514 for handling basic tasks.
  • the host computer 504 also includes a PC CPU 516 and PC Memory 518.
  • the PC CPU 516 of the host computer 504 runs software programs that process the image data as desired.
  • the host computer 504 uses the processed image data along with other sensor data as a part of the machine operation in controlling actuators.
  • Smart camera 600 moves the processing operation from a separate host computer physically distanced from the imaging sensor into the camera unit itself, thus eliminating the need for a host computer to process images. This has the advantage of eliminating the cable and its bandwidth limitations and allows processing of higher resolution images with higher frame rates. The jitter and delay problem is thus eliminated.
  • smart camera 600 includes the essential parts of block 300 of FIG. 3 as shown.
  • the DSP 308 receives data from FEP block 614 that handles analog/digital conversion, gain and balance for image information received from image sensor 616.
  • the smart camera can receive commands and control information from other control nodes in a system without the need to work through a host computer.
  • the smart camera module can also obtain sensor data directly from other sensors.
  • the smart camera can process image data in real time, and, along with other sensor data, can send commands and control information directly to any actuators on the serial bus without a need to involve any host computer.
  • a component may trigger a simple sensor input 618 such as a photoelectric switch as it enters the optical field of view of the image sensor.
  • a simple sensor input 618 such as a photoelectric switch
  • the closing of the switch can be used by directly linking the DSP 308 or camera timing trigger so as to cause an immediate commencement of image capture.
  • the DSP 308 or camera timing hardware may provide an output signal 620 that causes a light source such as a strobe to flash to provide an exact stop motion image.
  • a light source such as a strobe to flash
  • the latency involved in using the network for camera control is reduced by the hardware inputs or outputs so as to substantially overcome any latency and thereby synchronize external events.
  • the synchronizing signals can further be used by DSP 308 to capture the exact clock time or motion position (encoder value) at the instant that such lines are activated.

Landscapes

  • Engineering & Computer Science (AREA)
  • Operations Research (AREA)
  • Manufacturing & Machinery (AREA)
  • Microelectronics & Electronic Packaging (AREA)
  • Robotics (AREA)
  • Mechanical Engineering (AREA)
  • Studio Devices (AREA)

Abstract

Un système de caméra intelligente fournit des images mises au point à un opérateur sur un ordinateur hôte grâce au traitement des images numériques à l'emplacement d'images avant l'envoi de ces images à l'ordinateur hôte. La caméra intelligente comporte un processeur de signaux numériques destiné à prétraiter les images numériques avant de transmettre les images à l'hôte. Le prétraitement comprend une fonction d'extraction et de filtrage d'images et des procédés de convolution et de déconvolution, la correction du parallaxe et des erreurs de perspective dans les images ainsi que la compression d'images. La compression des images numériques par la caméra intelligente à l'emplacement d'images permet de transmettre des images en couleur ou en nuances de gris à résolution très élevée avec des débits de temps réel tels que 30 trames par seconde, à travers un bus série à vitesse élevée, à destination d'un ordinateur hôte ou de n'importe quel noeud de réseau qui comprend n'importe quelle adresse sur l'Internet.
PCT/US2001/007586 2000-03-10 2001-03-09 Camera intelligente WO2001068326A2 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
AU2001243537A AU2001243537A1 (en) 2000-03-10 2001-03-09 Smart camera

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US18856200P 2000-03-10 2000-03-10
US60/188,562 2000-03-10

Publications (2)

Publication Number Publication Date
WO2001068326A2 true WO2001068326A2 (fr) 2001-09-20
WO2001068326A3 WO2001068326A3 (fr) 2002-05-30

Family

ID=22693664

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2001/007586 WO2001068326A2 (fr) 2000-03-10 2001-03-09 Camera intelligente

Country Status (2)

Country Link
AU (1) AU2001243537A1 (fr)
WO (1) WO2001068326A2 (fr)

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2002065107A2 (fr) * 2001-02-09 2002-08-22 Wintriss Engineering Corporation Systeme d'inspection de bandes
US6867717B1 (en) 2002-04-04 2005-03-15 Dalsa, Inc. Digital encoder and method of encoding high dynamic range video images
EP1734425A2 (fr) * 2005-06-10 2006-12-20 Fanuc Ltd Commande pour robot
WO2007118097A1 (fr) 2006-04-03 2007-10-18 Omnivision Cdm Optics, Inc. systèmes et méthodes d'imagerie optique utilisant UN traitement d'image non linéaire et/ou spatialement variable
EP1868313A2 (fr) * 2006-06-14 2007-12-19 Siemens Aktiengesellschaft Procédé destiné à la transmission de données entre un dispositif de commande à tête dýimplantation de composants, automate dýimplantation, tête dýimplantation de composants, dispositif de transmission du côté émetteur et système pourvu du système de transmission des côtés émetteur et récepteur
WO2009074708A1 (fr) * 2007-10-11 2009-06-18 Euroelektro International Oy Utilisation d'une caméra intelligente pour contrôler une commande à courant alternatif industrielle
EP2293236A1 (fr) * 2002-12-18 2011-03-09 Snap-On Technologies, Inc. Carte de camera à calcul par gradient
US8514303B2 (en) 2006-04-03 2013-08-20 Omnivision Technologies, Inc. Advanced imaging systems and methods utilizing nonlinear and/or spatially varying image processing
US8717456B2 (en) 2002-02-27 2014-05-06 Omnivision Technologies, Inc. Optical imaging systems and methods utilizing nonlinear and/or spatially varying image processing
CN111328257A (zh) * 2020-03-11 2020-06-23 广东省电信规划设计院有限公司 一种上下位机的数据同步方法及装置

Citations (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4613942A (en) * 1982-02-19 1986-09-23 Chen Richard M Orientation and control system for robots
US4675993A (en) * 1984-12-26 1987-06-30 Tdk Corporation Electronic component mounting system having a vacuum suction head with a magnetic sensor for locating an electronic component with a magnet mark
WO1989004578A1 (fr) * 1987-11-10 1989-05-18 Echelon Systems Reseau et cellule intelligente assurant la detection, les communications bidirectionnelles et la gestion
US4890241A (en) * 1987-10-26 1989-12-26 Megamation Incorporated Robotic system
GB2257538A (en) * 1991-06-06 1993-01-13 Hitachi Ltd Air conditioning control.
DE29615180U1 (de) * 1996-08-21 1996-10-31 Siemens AG, 80333 München Funktionsüberwachung für eine Näherungsschaltereinrichtung zur Beeinflussung einer Programmsteuereinrichtung
EP0777263A2 (fr) * 1995-11-30 1997-06-04 Lucent Technologies Inc. Méthode et dispositif d'alignement et de soudage
DE19642737A1 (de) * 1996-04-23 1997-10-30 Mitsubishi Electric Corp Bordsteuersystem zum Steuern von Einrichtungen, die in einem Motorfahrzeug installiert sind, und Verfahren zum Neuschreiben eines Steuerprogramms und von Variablen dafür
US5821994A (en) * 1994-05-06 1998-10-13 Asahi Kogaku Kogyo Kabushiki Kaisha Image pickup and data processing apparatus having a plurality of special-purpose detachable substrates
US5903662A (en) * 1995-06-30 1999-05-11 Dci Systems, Inc. Automated system for placement of components
EP0932103A1 (fr) * 1998-01-27 1999-07-28 Deutsche Thomson-Brandt Gmbh Méthode et dispositif pour transférer des données bidirectionelles entre un bus IEEE 1394 et un appareil
WO1999048276A1 (fr) * 1998-03-18 1999-09-23 Flashpoint Technology, Inc. Dispositif numerique de capture d'images
WO2000008588A2 (fr) * 1998-08-04 2000-02-17 Cyberoptics Corporation Capteur perfectionne

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH11355625A (ja) * 1998-06-08 1999-12-24 Nippon Denki Computer System Kk インテリジェント・ビデオカメラおよび画像データ転送方法

Patent Citations (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4613942A (en) * 1982-02-19 1986-09-23 Chen Richard M Orientation and control system for robots
US4675993A (en) * 1984-12-26 1987-06-30 Tdk Corporation Electronic component mounting system having a vacuum suction head with a magnetic sensor for locating an electronic component with a magnet mark
US4890241A (en) * 1987-10-26 1989-12-26 Megamation Incorporated Robotic system
WO1989004578A1 (fr) * 1987-11-10 1989-05-18 Echelon Systems Reseau et cellule intelligente assurant la detection, les communications bidirectionnelles et la gestion
GB2257538A (en) * 1991-06-06 1993-01-13 Hitachi Ltd Air conditioning control.
US5821994A (en) * 1994-05-06 1998-10-13 Asahi Kogaku Kogyo Kabushiki Kaisha Image pickup and data processing apparatus having a plurality of special-purpose detachable substrates
US5903662A (en) * 1995-06-30 1999-05-11 Dci Systems, Inc. Automated system for placement of components
EP0777263A2 (fr) * 1995-11-30 1997-06-04 Lucent Technologies Inc. Méthode et dispositif d'alignement et de soudage
DE19642737A1 (de) * 1996-04-23 1997-10-30 Mitsubishi Electric Corp Bordsteuersystem zum Steuern von Einrichtungen, die in einem Motorfahrzeug installiert sind, und Verfahren zum Neuschreiben eines Steuerprogramms und von Variablen dafür
DE29615180U1 (de) * 1996-08-21 1996-10-31 Siemens AG, 80333 München Funktionsüberwachung für eine Näherungsschaltereinrichtung zur Beeinflussung einer Programmsteuereinrichtung
EP0932103A1 (fr) * 1998-01-27 1999-07-28 Deutsche Thomson-Brandt Gmbh Méthode et dispositif pour transférer des données bidirectionelles entre un bus IEEE 1394 et un appareil
WO1999048276A1 (fr) * 1998-03-18 1999-09-23 Flashpoint Technology, Inc. Dispositif numerique de capture d'images
WO2000008588A2 (fr) * 1998-08-04 2000-02-17 Cyberoptics Corporation Capteur perfectionne

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
PATENT ABSTRACTS OF JAPAN vol. 2000, no. 03, 30 March 2000 (2000-03-30) & JP 11 355625 A (NIPPON DENKI COMPUTER SYSTEM KK), 24 December 1999 (1999-12-24) *

Cited By (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2002065107A3 (fr) * 2001-02-09 2003-06-26 Wintriss Engineering Corp Systeme d'inspection de bandes
WO2002065107A2 (fr) * 2001-02-09 2002-08-22 Wintriss Engineering Corporation Systeme d'inspection de bandes
US8717456B2 (en) 2002-02-27 2014-05-06 Omnivision Technologies, Inc. Optical imaging systems and methods utilizing nonlinear and/or spatially varying image processing
US6867717B1 (en) 2002-04-04 2005-03-15 Dalsa, Inc. Digital encoder and method of encoding high dynamic range video images
EP2293236A1 (fr) * 2002-12-18 2011-03-09 Snap-On Technologies, Inc. Carte de camera à calcul par gradient
EP1573671B1 (fr) * 2002-12-18 2016-09-07 Snap-On Technologies, Inc. Carte de camera a calcul par gradient
EP1734425A2 (fr) * 2005-06-10 2006-12-20 Fanuc Ltd Commande pour robot
EP1734425A3 (fr) * 2005-06-10 2007-03-07 Fanuc Ltd Commande pour robot
US8514303B2 (en) 2006-04-03 2013-08-20 Omnivision Technologies, Inc. Advanced imaging systems and methods utilizing nonlinear and/or spatially varying image processing
US7911501B2 (en) 2006-04-03 2011-03-22 Omnivision Technologies, Inc. Optical imaging systems and methods utilizing nonlinear and/or spatially varying image processing
US8068163B2 (en) 2006-04-03 2011-11-29 Omnivision Technologies, Inc. Optical imaging systems and methods utilizing nonlinear and/or spatially varying image processing
WO2007118097A1 (fr) 2006-04-03 2007-10-18 Omnivision Cdm Optics, Inc. systèmes et méthodes d'imagerie optique utilisant UN traitement d'image non linéaire et/ou spatialement variable
EP1868313A3 (fr) * 2006-06-14 2008-05-21 Siemens Aktiengesellschaft Procédé destiné à la transmission de données entre un dispositif de commande à tête d'implantation de composants, automate d'implantation, tête d'implantation de composants, dispositif de transmission du côté émetteur et système pourvu du système de transmission des côtés émetteur et récepteur
EP1868313A2 (fr) * 2006-06-14 2007-12-19 Siemens Aktiengesellschaft Procédé destiné à la transmission de données entre un dispositif de commande à tête dýimplantation de composants, automate dýimplantation, tête dýimplantation de composants, dispositif de transmission du côté émetteur et système pourvu du système de transmission des côtés émetteur et récepteur
WO2009074708A1 (fr) * 2007-10-11 2009-06-18 Euroelektro International Oy Utilisation d'une caméra intelligente pour contrôler une commande à courant alternatif industrielle
CN111328257A (zh) * 2020-03-11 2020-06-23 广东省电信规划设计院有限公司 一种上下位机的数据同步方法及装置
CN111328257B (zh) * 2020-03-11 2022-03-22 广东省电信规划设计院有限公司 一种上下位机的数据同步方法及装置

Also Published As

Publication number Publication date
AU2001243537A1 (en) 2001-09-24
WO2001068326A3 (fr) 2002-05-30

Similar Documents

Publication Publication Date Title
US6988008B2 (en) Smart camera
US6985780B2 (en) Smart camera
US20010034557A1 (en) Hardware architecture and method for distributed machine control
US6845276B2 (en) Multiple axis modular controller and method of operating same
WO2001068326A2 (fr) Camera intelligente
KR20180129651A (ko) 촬영 장치, 시스템, 및 방법
US20050264429A1 (en) Device and method to establish temporal correspondence in multiple sensor configurations
JP2018024044A (ja) 情報処理システム、情報処理装置、ワークの位置特定方法、およびワークの位置特定プログラム
US11689439B2 (en) Method and apparatus for communication with a motor or load mounted device in motion applications
US20020007229A1 (en) Distributed machine control software architecture
CN112291523B (zh) 一种贴片机设备的图像数据接收系统及方法
JP2018024045A (ja) 情報処理システム、情報処理装置、ワークの位置特定方法、およびワークの位置特定プログラム
JP6400831B2 (ja) 多重化通信システム及び作業機
KR101038619B1 (ko) 로봇용 다자유도 스마트 액추에이터를 위한 네트워크 기반 제어 모듈
CN114390165A (zh) 摄像装置、信息处理方法和存储介质
JP3382374B2 (ja) 撮像装置
US20190279355A1 (en) Camera core of a smart camera for industrial image processing and method for industrial image processing
CN112040131B (zh) 基于EtherCAT工业以太网总线的智能相机及其控制方法
CN111902785B (zh) 控制系统、控制方法以及计算机可读存储介质
JP2017183908A (ja) 切替装置
WO2002049410A2 (fr) Dispositif d'acquisition d'images relatives a l'alignement de cartes a circuit imprime avec interface amelioree
US11269800B2 (en) Integrated communication unit
Scholles et al. IEEE 1394" FireWire" system design for industrial and factory automation applications
JP2000180119A (ja) 部品認識装置、及び部品装着装置
JP3635405B2 (ja) 画像処理装置及び電子カメラ

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A2

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BY BZ CA CH CN CR CU CZ DE DK DM DZ EE ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NO NZ PL PT RO RU SD SE SG SI SK SL TJ TM TR TT TZ UA UG UZ VN YU ZA ZW

AL Designated countries for regional patents

Kind code of ref document: A2

Designated state(s): GH GM KE LS MW MZ SD SL SZ TZ UG ZW AM AZ BY KG KZ MD RU TJ TM AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE TR BF BJ CF CG CI CM GA GN GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
DFPE Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101)
AK Designated states

Kind code of ref document: A3

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BY BZ CA CH CN CR CU CZ DE DK DM DZ EE ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NO NZ PL PT RO RU SD SE SG SI SK SL TJ TM TR TT TZ UA UG UZ VN YU ZA ZW

AL Designated countries for regional patents

Kind code of ref document: A3

Designated state(s): GH GM KE LS MW MZ SD SL SZ TZ UG ZW AM AZ BY KG KZ MD RU TJ TM AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE TR BF BJ CF CG CI CM GA GN GW ML MR NE SN TD TG

122 Ep: pct application non-entry in european phase
NENP Non-entry into the national phase in:

Ref country code: JP