JP6054917B2 - Method and system for optoelectronic detection and localization of objects - Google Patents

Description

[Field of the Invention]
The present invention relates to an optoelectronic sensor that detects an object and generates information representing the object, particularly information representing the position of the object.

  Optoelectronic sensors for detecting and locating objects are useful in a wide variety of fields. An example is industrial automation. To understand the operation, capabilities, and limitations of such sensors, consider one specific application, for example, printing alphanumeric characters on a label in a high-speed bottled line. In such a general application, the sensor detects the presence of a bottle at a given point as the bottle moves down the production line and commands the printer to start printing when the bottle is at that point. A signal may be generated.

  If the signal is generated when the bottle is in the proper position, the desired alphanumeric characters are printed at the desired position on the bottle label. On the other hand, if no bottle is detected, the label will not be printed, which may be a costly mistake. If a signal is generated when no bottle is present, the printer may scatter ink into the air and adversely affect the manufacturing process. If a signal is generated when the bottle is present but not in the proper position, the character will be printed in the wrong position on the label.

  As can be seen from this example, an optoelectronic sensor for detecting and locating an object must detect only the desired object and must detect only at the desired position.

  Photoelectric sensors, a type of optoelectronic sensor, have long been used for object detection and localization. Such sensors typically operate by emitting a light beam and detecting the received light. These sensors have a variety of configurations that fit a variety of purposes including:

  In one configuration, the emitter and receiver are placed at both ends of the path, and something that is not transparent across the path may be configured to block the light beam. An object is detected when the receiver finds a very small amount of light. The path is determined by the arrangement of the emitter and the receiver, thereby determining the position where the object is detected. Applications are limited so that only the desired object crosses the path and only needs to determine the position of the edge of the object. In bottled applications, this limitation means, for example, that the label is in a fixed position relative to the bottle edge.

  As a second configuration, the emitter and the receiver may be placed at one place, and a retroreflector may be disposed at the opposite end of the path that reflects the beam from the emitter to the receiver. This configuration is similar to the previous one, but is relatively easy to install. This is because all necessary wiring is not formed at both ends of the path, but only at one end of the path. However, if the object to be detected is reflective, erroneous detection and / or position identification errors may occur.

  In a third configuration, the emitter and receiver are placed in one place, and the emitter emits a focused light beam and something with sufficient reflectivity across the front of the beam reflects the light beam towards the receiver. May be configured to be An object is detected when the receiver finds an amount of light that exceeds a certain predetermined threshold. The position of the beam depends on the placement of the emitter / receiver assembly, which in turn determines the position where the object is detected. By using a focused beam, this position can be made relatively accurate, and the possibility of misdetecting this can be reduced because the beam is out of focus with respect to the object behind it. The object and its environment are limited so that the reflected light exceeds the threshold only when the desired object is in the desired position.

  The fourth configuration is a modification of the third configuration, and uses a diffused light beam instead of the focused beam. In this case, it is easy to detect an object whose position of the object is not so limited, but the accuracy of the position of the detected object is reduced, and detection is performed when there is no desired object in front of the beam. Probability increases.

  Examples of this type of photoelectric sensor include those manufactured and sold by Banner Engineering Corporation of Minneapolis, Minnesota, such as the MINI-BEAM line.

  Photoelectric sensors generally generate a single signal indicating that an object has been detected and further inform the location of the object. Such a signal has two states, sometimes referred to as “present” and “absent”. In the first and second configurations, the signal may be placed in a “present” state, for example, when a small amount of light is detected by the receiver. Also, in the third or fourth configuration, the signal may be placed in a “present” state when light above a threshold is detected by the receiver.

  An object is detected when the signal is in a “present” state. The moving object is located by the time of the signal transition from “absent” to “present”. That is, at the time of the transition, the object is in a known predetermined position, referred to herein as a “reference point”. In any of the above four photoelectric sensor configurations, the reference point is determined by the position of the beam and is adjusted by physically moving the sensor. In the present specification, the reference point means a desired position of the object, and the actual position of the object at the signal time may be different for various reasons as described in the present specification.

  Photoelectric sensors are typically used to detect a specific object and to locate an object at a specific position, for example, to detect a bottle moving downstream on a belt conveyor, or the tip of such a bottle Used to indicate when a specific reference point, for example, the front of the printer has been reached. None of the sensors make an object presence and location determination perfectly, if the desired object is not detected, an unwanted object or some other condition such as a light source causes a false detection, or If a desired object is detected but its position is incorrect, the result is a failure.

  In order to increase detection and localization reliability, the photoelectric sensor may use one or more techniques. For example, the various configurations described above find various tradeoffs between object misses, false detections, and other factors that meet the needs of a particular purpose. For example, a retroreflective configuration reduces the chance of missing an object, but two elements, a sensor and a reflector, must be installed in the proper location. With a focused beam, the possibility of detecting unwanted objects is reduced. This is because the focused beam has a narrow focusing range.

  In each sensor configuration, sensitivity adjustment is generally used as another trade-off between object miss and false detection. For example, in a focused beam configuration, a predetermined detection threshold is determined by sensitivity adjustment. Increasing the threshold decreases the frequency of false detections but increases the frequency of missing objects. When the threshold is decreased, the opposite trade-off is established.

  Furthermore, in order to reduce the frequency of false detections due to stray light, the sensor may use a suitably modulated light beam. For example, the beam may be pulsed on and off at a high frequency, and the receiver may be designed to detect only light pulsed at that frequency.

  “Background suppression”, a commonly used term for photoelectric sensors, is in the background (ie, beyond some predetermined distance) to reduce the frequency of false detections. By various means of ignoring the reflection of the beam emitted by the object. For example, US Pat. No. 5,760,390 describes such a device and also provides information regarding prior art devices.

  Photoelectric sensors are very well suited for object detection and location and are widely used, but have various limitations.

  Such sensors have very limited ability to distinguish the desired object from situations such as unwanted objects, object reflectance variations, complex markings of objects, or features. A sensor generally can only measure the amount of light it receives. For example, a gray object at a distance of 5 centimeters is as much as a white object at a distance of 10 centimeters, even if the beam is slightly out of focus at a 10 centimeter position. Of light may be reflected toward the receiver. In order to ensure the presence of a desired object at a desired position and to prevent the amount of received light from falling above or below a predetermined threshold due to other conditions, very careful handling must be performed. Thus, in particular, the method of attaching the sensor, the nature of the object to be detected or located, and the manner in which the object is presented to the sensor are limited. Furthermore, it is generally only the edge of an object that can be detected and located, it is difficult to detect and locate marks and other features on the object, and in many cases cannot be performed .

  When using a photoelectric sensor, it is generally necessary to physically move the sensor to determine or adjust the reference point. Such adjustments are difficult to make and, if not done carefully, can cause undesirable results. It is also difficult to determine the reference point with high accuracy.

  Photoelectric sensors have a natural delay between when an object crosses a reference point and when a signal transition occurs. This delay is commonly referred to as “latency” or “response time” and is typically on the order of hundreds of microseconds. During this delay, the object will move away from the reference point by a distance that increases as the speed of the object increases. Also, since the reference point is generally determined at the time of mounting using a stationary object, the actual position of the object at the signal time is always different during the manufacturing stage in which the object moves. If the delay and speed are known, it may be possible to compensate for latency, but latency compensation increases complexity and cost, so it can always be implemented. Not exclusively.

  Photoelectric sensor latency can generally be reduced by making some compromise on detection reliability. The shorter the time for which the sensor makes a detection determination, the lower the reliability of the determination.

  The photoelectric sensor has an inherent uncertainty at the signal transition time. This uncertainty is commonly referred to in the art as “reproducibility” and is typically on the order of hundreds of microseconds. This can be paraphrased as uncertainty of the position of the object at the signal time, and the uncertainty is proportional to the speed of the object.

  The accuracy of a photoelectric sensor in locating an object is limited by a combination of physical positioning, latency, and repeatability.

  In recent years, a new type of photoelectric sensor has been developed and is described in pending US patent application Ser. No. 10 / 865,155 filed on Jun. 9, 2004. These sensors, called “vision detectors,” solve some of the limitations of photoelectric sensors. The vision detector uses a two-dimensional imaging device (similar to that used for digital cameras) connected to a computer or other device that analyzes digital images, detects and determines position.

  Vision detectors have the ability to distinguish a desired object from other conditions that far exceed those that can be achieved using photoelectric sensors. This capability is achieved by detecting and locating objects using a two-dimensional luminance pattern. The reference point is set electrically using a man-machine interface. However, these devices have a very large latency, on the order of a few milliseconds, and are not as reproducible as photoelectric sensors, even under favorable conditions.

  Furthermore, vision detectors are very expensive compared to photoelectric sensors and are much more complex to set up for a given purpose.

The present invention provides an optoelectronic method and system that detects moving objects and generates information representing the object, such as, but not limited to, the position of the object. The disclosed methods and systems provide various embodiments having some or all of the following advantages.
Advanced ability to distinguish desired objects from other conditions based on measurements generated in a one-dimensional image, including various embodiments using normalized correlation pattern detection methods and processes・ Electrical setting and adjustment of the reference point using the interface ・ Adjustable optionally using the man-machine interface based on the prediction of the time when there is no waiting time or the object crosses the reference point Negative latency-Very high reproducibility based on very fast imaging and image analysis speeds and an accurate method to determine object position-Low cost-Setup and testing is relatively easy Other benefits described herein or that will be apparent to those skilled in the art

  The object to be detected and / or located is moving relative to the field of view of the optical sensor, and the optical sensor makes a light measurement within that field of view. In order to generate information representing the object, a one-dimensional image oriented substantially parallel to the direction of motion is taken and analyzed. This information can be used for any purpose and may be used, for example, to communicate with other systems using signals. In a typical embodiment, the optical sensor is a linear optical sensor that includes a one-dimensional array of light receiving elements.

  In this specification, the term “downstream” means “the direction of movement”, and the term “upstream” means “the direction opposite to the direction of movement”.

  By using a one-dimensional image that is oriented almost parallel to the direction of motion, it is much more efficient than an emitter / receiver configuration, a two-dimensional imager, or other prior art systems that use one-dimensional images of orientation. There are significant unexpected benefits. Combining object motion with the orientation of the image provides a time-series one-dimensional image of a portion of the object as it enters, moves through, and / or leaves the field of view. The movement of the object ensures that at least some of the plurality of captured images correspond to a plurality of positions of the object relative to the field of view. The result is, among other things, that multiple viewpoints are obtained and that much more information about the object can be obtained compared to information obtained from a single viewpoint. Using images obtained from multiple viewpoints makes it possible to reliably distinguish desired objects from other conditions, track object motion, and obtain position information with high reproducibility without waiting time. To help. Various embodiments including linear optical sensors and sufficiently powerful digital processors to process multiple one-dimensional images are very inexpensive.

  In contrast, the prior art emitter / receiver configuration is essentially a zero-dimensional sensor and simply measures the amount of light received. There is no ability to distinguish a desired object, for example by a pattern detection method, and there is no function to visually track an object passing through the field of view in order to improve detection reliability or obtain more accurate position information.

  By using multiple 1D images, it is possible to achieve very fast imaging and analysis speeds at low cost compared to prior art systems that use 2D images for detection and localization. It becomes possible. For example, the vision detector described in pending US patent application Ser. No. 10 / 865,155 processes images at a rate of 500 sheets per second. One embodiment of the present invention described below processes images at a rate of over 8000 per second and operates at a much lower cost. By combining this fast operating speed with the various processes described herein, latency can be eliminated and reproducibility can be dramatically improved. The above-referenced patent application can use multiple one-dimensional images oriented substantially parallel to the direction of motion for the purposes described herein, and can eliminate waiting time by any means. Is not described or considered.

  It has long been known as prior art to image a moving object using a linear array oriented substantially perpendicular to the direction of motion. One purpose of such a configuration is to obtain a two-dimensional image of an object from a single viewpoint, not to obtain a time series for a portion of the object at multiple positions. Such a linear array oriented substantially perpendicular to the direction of motion is merely an alternative to a two-dimensional camera and has certain advantages and disadvantages over such a camera. Such a linear array is not suitable for the purposes described herein. Other uses of linear array sensors that are oriented substantially perpendicular to the direction of motion are known, but are likewise not suitable for the purposes described herein.

  The methods and systems disclosed herein take a plurality of one-dimensional images of a field of view through which an object passes and analyze at least a portion of the image to generate information representing the object.

  In some embodiments, the information representing the object may include information that is responsive to the presence of the object in the field of view, for example, information indicating that the object is in the field of view or intersects the reference point.

  In some embodiments, information representing an object may include an estimate of the time at which the object crosses a reference point, referred to herein as “reference time”. According to these embodiments, it is possible to locate an object, that is, to search for an object at a reference point at a reference time. The estimated value of the reference time is obtained from information representing an object.

  Some embodiments may include an electrical man-machine interface that can adjust the reference point. Adjustment of the reference point is made possible by, for example, providing a push button for moving the reference point upstream or downstream. As a result, the reference point can be set very accurately without physically moving the sensor.

  In some embodiments, a signal serving to search for an object may be generated by indicating an estimated value of the reference time. This instruction can be made in various ways well known in the art and will be described in detail below. In some embodiments, this signal may be simply generated at that time, or may indicate the estimated time by responding to that time with some amount of delay or advance. The time at which a signal occurs in such an embodiment is referred to herein as “signal time”. A man-machine interface may be further provided to allow adjustment of the signal time relative to the estimated time.

  Depending on the embodiment, there may be no waiting time between the actual reference time and the signal time. In some embodiments, the estimated reference time may be determined by predicting the reference time.

  Some embodiments may include an exemplary object and a setup signal to set the reference point. When the example object is placed at the desired reference point, the setup signal indicates that the example object has been so placed. In response to the setup signal, one or more images of the field of view including the exemplary object are taken, analyzed, and a reference point is set. This makes it possible to set the reference point accurately without physically moving the sensor after the sensor has been attached to the approximate position. The setup signal may originate from a man-machine interface, may originate from another device, may originate from a software subroutine call, or It may originate from any means known in the art. In those embodiments, the signal is when the object crosses the reference point (no latency), the object is upstream from the reference point (negative latency), or the object is downstream from the reference point. Sometimes (positive waiting time) can be generated.

  In some embodiments, a test object for testing sensor mounting and a man-machine interface may be used. An appropriate indicator indicates whether or not a test object has been detected in the field of view, and if detected, indicates whether the test object is upstream or downstream of the reference point.

  In some embodiments, the method and system perform measurements on at least a plurality of captured images, select those measurements that are determined to be responsive to an object in the field of view, and select the selected measurements. Is used to make a decision to generate information representing the object, and in some cases, a signal that conveys the information may be generated.

  Depending on the embodiment, an “image measurement value” may be created for a captured image. The image measurement value includes any appropriate calculation result for the captured image. Examples of image measurements include brightness, contrast, edge quality, peak correlation values, peak correlation positions, and many other measurements well known in the art.

  A set of “object measurement values” is selected from the image measurement values. Object measurements consist of image measurements that are determined to be responsive to objects in the field of view. Some or all of the image measurements are used to select object measurements so that decisions are made based on various situations in the field of view, rather than blindly. One image measurement that is particularly suitable for such a determination is a model pattern and a peak correlation value using a normalized correlation process. However, many other types of measurements can also be used within the scope of the present invention.

  The object measurements are used to make various decisions that generate information representing the object. Information can be obtained in accordance with the method and system of the present invention that would exist to those skilled in the art, such as presence or state, events such as objects crossing a reference point, position, speed, brightness, contrast, etc. May represent a number of other information.

  In one exemplary embodiment, accurate location information may be obtained without waiting time by predicting when the object will cross the reference point and scheduling the signal to occur at the predicted time. This predicted value is obtained by fitting a curve such as a single line to a set of points (time, position). However, the time corresponds to the time when the image was taken, and the position is obtained from the image measurement value.

  In one exemplary embodiment, the latency can be adjusted using a man-machine interface, and a new capability may allow the latency to be negative.

  In one exemplary embodiment, the reference point may be accurately set and adjusted using a man-machine interface.

  In one exemplary embodiment, the model pattern used in the normalized correlation process to generate a specific image measurement is detected and indicates the desired appearance of the object to be located using the exemplary object. May be obtained using a training process.

  The present invention will be understood from the following detailed description and the accompanying drawings.

2 shows an exemplary application of a system according to the invention for detecting and locating individual objects moving down a production line. 1 shows an exemplary application of a system according to the invention for detecting and locating marks on a moving continuous web. FIG. 2 is a block diagram illustrating an example apparatus. It is a figure which shows the imaging | photography process for obtaining the image of a visual field, and an image is the direction substantially parallel with respect to the moving direction. FIG. 4 illustrates a portion of a training process used to obtain a model pattern used in the measurement process of an exemplary embodiment. FIG. 6 illustrates an object at a specific location as part of the measurement process of an exemplary embodiment. FIG. 6 is a timing diagram illustrating decision delay used in connection with latency and prediction descriptions. It is a figure which shows prediction of the time when an object crosses a reference point. However, since the crossing time is too far away, the signal cannot be scheduled. It is a figure which shows prediction of the time which an object crosses a reference point, and scheduling of the signal which generate | occur | produces at the time. It is a figure which shows prediction of the time when an object crosses a reference point. However, the predicted time is in the past, and the signal has already been generated. FIG. 6 illustrates a flow diagram illustrating portions of a selection process in an exemplary embodiment, various portions of other processes associated with the selection process, and memory used to hold values required by those processes. FIG. FIG. 3 illustrates a portion of a decision process in one embodiment, as well as various portions of other processes associated with the decision process. FIG. 2 illustrates a man machine interface used as part of a training process to set reference points, as well as exemplary objects used in the training process, in one embodiment. FIG. 3 shows a man machine interface used to test the operation of an exemplary embodiment of a system according to the present invention, as well as exemplary objects used for testing. FIG. 2 illustrates a man machine interface used to determine the direction of motion and put the system into operation in an exemplary embodiment, as well as an exemplary object used in its operation.

DETAILED DESCRIPTION OF THE INVENTION In the following detailed description of exemplary embodiments, reference is made to the accompanying drawings that form a part hereof. In the drawings, there are shown by way of illustration several specific embodiments in which the invention may be practiced. It should be considered that other embodiments may be used or structural changes may be made without departing from the scope of the invention.

Definitions As used herein, “linear detector” refers to an optoelectronic system according to an embodiment of the present invention that is used to generate information representing an object, sometimes referred to herein as “object information”. means. Similarly, “linear detection method” means a method according to an embodiment of the invention having the same purpose.

  Applications of the linear detector include, but are not limited to, for example, object analysis, detection, and location. In one embodiment for object detection, the object information may include information indicating that the object is in a particular state, for example, that the object is crossing a “reference point”. In one embodiment aimed at locating an object, the object information may include the time at which the object crosses a particular reference point.

  As used herein, “process” means a set of systematic operations having some purpose performed by any suitable apparatus, including but not limited to mechanisms, devices, Component, software, or firmware, or any combination thereof for co-operation at one location or at different locations to perform the intended operation. The system according to the invention may include suitable processes in addition to other elements.

  The description herein generally describes various embodiments of the system according to the invention, such system including various processes and other elements. As will be apparent to those skilled in the art, these descriptions are easy to interpret as describing the method according to the present invention, and the processes and other elements that make up the system according to the present invention may vary according to the method. Corresponds to the steps.

Exemplary Application FIG. 1 illustrates one application of an exemplary embodiment of the present invention for detecting and locating an object. The conveyor 100 moves the boxes 110, 112, 114, 116, and 118 in the direction of motion 102. Each box in this example has a label such as label 120 and a decorative mark such as mark 122. The printer 130 prints various characters, such as the character 132, on the label as each label passes through the printer. In the example of FIG. 1, the object to be detected and located is a label.

  The illustrated linear detector 150 sends a signal 134 to the printer 134 when the label to be printed crosses the printer or is in some desired position relative to the reference point 106. In one embodiment, signal 134 comprises a pulse indicating that a label has been detected, and the rising edge of the pulse occurs when the label is at the reference point 106, thereby serving to locate the label. To do.

  The linear detector 150 uses the lens 164 to form a one-dimensional image of the visual field 170 on the linear optical sensor 160 including the linear array 162 of light receiving elements. The visual field 170 and the linear array 162 of light receiving elements are arranged in such an orientation that a one-dimensional image is generated that is substantially parallel to the direction of motion. Each light receiving element performs light measurement within the visual field.

  In the example of FIG. 1, depending on the characteristics of the box, the use of a prior art photoelectric sensor may be inappropriate for label detection and location. Photoelectric sensors are generally well suited for detecting the edge of a box, but the label is not in a fixed position relative to the edge of the box. For example, the label on box 118 is farther from the left edge of the box than the label on box 116. Therefore, it is desirable to detect the label itself, but the prior art photoelectric sensor is unsuitable because the edge of the label cannot be distinguished from the edge of the decorative mark.

  The width of the label is much wider than the width of the decorative mark. However, for example by using a pattern detection process, the light pattern corresponding to the label in the field of view 170 can be distinguished from the light pattern corresponding to the decorative mark by appropriate analysis of the one-dimensional image. The present invention enables reliable detection and accurate localization by analyzing the image, as will be described in detail below.

  FIG. 2 illustrates another exemplary application of the present invention for detecting and locating objects. The conveyor 200 moves the continuous web 210 in the movement direction 202. Web 210 has periodic reference marks, such as marks 220, 222, and 224. As these marks move through the field of view 270 of the linear detector 250 that includes the linear optical sensor 260, a one-dimensional image of the field of view 270 is imaged on the linear optical sensor 260. The linear optical sensor 260 includes a linear array of light receiving elements 262. The visual field 270 and the linear array 262 of light receiving elements are arranged in such a direction that a one-dimensional image in a direction substantially parallel to the direction of motion 202 is generated.

  When the fiducial mark crosses the fiducial point 206 or when the fiducial mark is in some desired position relative to the fiducial point 206, the linear detector 250 sends a signal (not shown) to the appropriate automation device (not shown). (Not shown).

In the example of FIG. 2, fiducial marks include a pattern of gray shadows that can be distinguished from other marks, features, anomalies, or shadows that may appear on the web 210, according to such fiducial marks, Much more reliable detection is possible compared to what can be achieved with technical photoelectric sensors.
Exemplary device

  FIG. 3 is a block diagram illustrating one embodiment of a portion of a linear detector according to the present invention. A microcontroller 300 such as the AT91SAM7S64 sold by Atmel Corporation in San Jose, California includes an ARMv4T processor 310, a read / write SRAM memory 320, a read-only flash memory 322, a USART (Universal Synchronous / Asynchronous Receiver Transmitter) 330, a parallel An I / O interface (PIO) 332 and a timer 334 are included. The ARMv4T processor 310 controls other elements of the microcontroller 300 and executes software instructions stored in the SRAM 320 or flash memory 322 for various purposes. The SRAM 320 holds data used by the ARMv4T processor 310. The microcontroller 300 operates at a clock frequency of 50 MHz.

  A linear optical sensor 340, such as TSL3301-LF sold by Texas Advanced Optoelectronic Solutions (TAOS) in Plano, Texas, includes a linear array 102 of light receiving elements. The linear optical sensor 340 can expose the linear array of light receiving elements to light for an adjustable time period called an “exposure interval” according to the control of the ARMv4T processor 310 by a command issued using the USART 330, resulting in a result. The measured light measurement value 102 can be binarized, transmitted to the USART 330 in digital form, and stored in the SRAM 320. As described in the TAOS document TAOS0078, January 2006, the linear optical sensor 340 also binarizes them for each light measurement in each of the three zones, subject to the control of the ARMv4T processor 310. Before, adjustable analog gain and offset can be applied.

  As an exemplary embodiment, the linear optical sensor 340 may be calibrated to compensate for any non-uniformities in the illumination, optical characteristics, and response of individual light receiving elements. An object of uniform whiteness is placed in the field of view, and the gain is set for each of the three zones so that the brightest pixels are slightly below saturation. Next, a calibration value is calculated for each pixel. The calibration value is calculated as a value that, when multiplied by the corresponding calibration value for each gray value, produces a uniform image for an object of uniform whiteness. The calibration value is stored in the flash memory 322 and applied to the image that is subsequently taken. The calibration value is limited to a value such that each gray value is multiplied by a number of 8 or less. Each image used by this calibration procedure is obtained by averaging 512 the various captured images.

  The means for illuminating the field of view of the linear optical sensor 340 may be any suitable means. In an exemplary embodiment, two 630 nm LEDs may be used to illuminate the field of view from one side of the linear optical sensor 340 and two 516 nm LEDs may be used to illuminate the field of view from the other side. . A beam shaping diffuser, such as that manufactured by Luminit in Transformer, California, is placed in front of the LED and the LED beam is spread in a direction parallel to the linear optical sensor 340.

  A human user who is a linear detector, such as a manufacturing engineer, can use the Man Machine Interface (HMI) 350 to control the system. In an exemplary embodiment, the HMI 350 may have a series of buttons and indicator lights as described in detail below. The ARMv4T processor 310 controls the HMI 350 using the PIO interface 332. In other embodiments, the HMI may consist of a personal computer or other device. In yet another embodiment, the HMI may not be used at all.

  The linear detector shown in FIG. 3 generates a signal 362 for informing the object detection and position. Signal 362 is connected to automation device 360 as needed for a given purpose. The signal 362 is generated by the timer 334 according to the control of the ARMv4T processor 310. As an alternative embodiment, the signal 362 may be generated by a PIO interface 332 that is also controlled by the ARMv4T processor 310. Note that the automation device 360 is shown for illustrative purposes only. Signal 632 may be used for any purpose and need not be connected to any form of automation device, and signal 362 may not be used at all.

  As an exemplary embodiment, the various processes reside in a group of interrelated digital hardware elements as shown in the block diagram of FIG. 3, appropriate software instructions residing in SRAM 320 or flash memory 322, and SRAM 320. May be executed by data.

Processes used by the various embodiments of the system according to the invention As used herein, a “motion process” is a process that generates a relative motion in some direction of motion between an object and the field of view of an optical sensor. . As long as there is relative motion, the object, sensor and / or field of view can be moved. Examples of motion processes include, but are not limited to, a fixed sensor or a conveyor that moves an object through a movable sensor; a fixed arm or a robot arm that moves a sensor through a movable object. A fixed object that moves the field of view using some means, such as a movable mirror, or a fixed sensor; and a stationary sensor or a falling object that passes through the movable sensor.

  In this specification, the “imaging process” is a process of acquiring a one-dimensional image of the field of view of an optical sensor that performs optical measurement. The image may be in any form as long as it has information about light emitted from the field of view and is suitable for analysis by other processes as necessary. The image may be responsive to any measurable color, any wavelength, or any wavelength range, any polarization state, or any other light characteristic, or a combination thereof. The image may be analog, digital, or any combination thereof, and may be located on the optical sensor, in memory external to the sensor, or Any suitable form of analog signal processing, such as, but not limited to, gain and offset, resampling, resolution change, temporal filtering, and / or spatial filtering, And / or by digital signal processing.

  In the exemplary apparatus of FIG. 3, the imaging process takes 2 light measurements to obtain an array of numbers called “pixels” (the value of which is called “gray value” and corresponds to the light measurement). It consists of digitizing and transferring the pixels from the linear optical sensor 340 to the microcontroller 300 and storing the pixels in the SRAM 320 for further analysis. In the following, according to customary in the art, a pixel means a distance unit corresponding to a distance between one element of a numeric array or two neighboring elements of the array. An array of pixels is a one-dimensional digital image.

  The imaging process cooperates with the motion process to capture a plurality of one-dimensional images oriented substantially parallel to the motion direction. As a result, at least some of these images correspond to multiple positions of the object relative to the field of view, and when the object passes through the field of view, multiple time-series images of the object are generated. Is done.

  As will be apparent to those skilled in the art, “substantially parallel” means a time series of pieces of an object at multiple locations by a linear detector as the object enters, moves through, and exits the field of view. This means an arbitrary direction in which an image can be obtained. As is also apparent, the range of orientations suitable for use in the systems and methods described herein have different limitations depending on the application of the invention. For example, if the orientation of the image is such that the angle between the orientation of the image and the direction of motion is 5 degrees, every time the object moves 1 pixel parallel to that direction, the object It will be shifted by 0.09 pixels perpendicular to the direction of motion. For example, if the range of locations includes 30 pixels in the direction of motion, the shift will be only 2.6 pixels. If the system and method described herein function as intended, even if there is a 2.6 pixel misalignment, the 5 degree orientation will be a nearly parallel orientation in the application. I will. Of course, whether the systems and methods described herein function as intended will depend on the nature of the object, the desired capabilities of the invention (eg, accuracy and reliability), and other factors. Determined.

  Similarly, it will be apparent to those skilled in the art that the direction of motion need not be strictly uniform or constant as long as the systems and methods described herein function as intended.

  The use of a one-dimensional image enables very high-speed imaging and analysis at a much lower cost compared to a two-dimensional system of the prior art. According to high-speed imaging and analysis, it is possible to analyze a large number of images for each object as the object passes through the field of view. By moving the object, images can be obtained from a plurality of viewpoints, and much more information about the object can be obtained compared to information obtained from a single viewpoint. This information is the basis for reliable detection and accurate location.

  Prior art linear array sensors oriented generally perpendicular to the direction of motion typically generate a single two-dimensional image for each object. Such a system, while convenient, does not provide the capabilities of the present invention.

  The exemplary embodiments of FIGS. 1, 2 and 3 use linear optical sensors to generate a one-dimensional image oriented substantially parallel to the direction of motion, as will be apparent to those skilled in the art. Thus, other types of optical sensors can be used for the same purpose. For example, a two-dimensional optical sensor may be used to capture a two-dimensional image, and then that portion may be converted into a one-dimensional image oriented substantially parallel to the direction of motion. This conversion can be achieved by any suitable form of signal processing, for example by averaging light measurements approximately perpendicular to the direction of motion. As another example, a two-dimensional optical sensor having a function of photographing a so-called region of interest may be used. Many commercially available COMS optical sensors have this function. A one-dimensional region of interest oriented substantially parallel to the direction of motion will have the same function as a linear optical sensor. The image portion or region of interest does not necessarily have to be the same for each captured image, but as long as the system and method described herein function as intended, such image portion or region of interest is in some form. You may move it.

  FIG. 4 illustrates a portion of an exemplary imaging process for obtaining a one-dimensional image 430 of the field of view 410. In the figure, the image 430 is oriented substantially parallel to the movement direction 400.

  The light emitted from the field of view 410 is focused on the surface of an optical sensor (not shown), which performs optical measurements. In the exemplary imaging process of FIG. 4, 24 individual light measurements are made in 24 zones of the field of view 410, such as exemplary zone 440. For ease of illustration, the zones are depicted as rectangular, continuous, and non-overlapping, but in the case of typical sensors, for example, the nature of the optical elements that focus the light, The configuration becomes more complicated due to the requirements of

  In the exemplary imaging process of FIG. 4, each light measurement is binarized to generate 24 pixels, such as exemplary pixel 450 corresponding to exemplary zone 440, respectively. For example, pixel 450 has a gray value of 56, which is, for example, a light measurement of zone 440.

  The zones of FIG. 4 correspond, for example, to individual light receiving elements in a linear array sensor, or of a two-dimensional sensor converted to a single measurement value by any suitable form of signal processing as described above. It may correspond to a plurality of light receiving elements.

  The image orientation 420 of the image 430 is determined by the configuration of the light measurement zone of the field of view 410. If the orientation 420 of the image and the direction of motion 400 are the time series for a piece of object at multiple locations, depending on the imaging process, as the object enters, moves in and / or leaves the field of view. If the orientation and direction are such that a plurality of images can be obtained, the image 430 is substantially parallel to the movement direction 400. The image orientation 420 is a relative direction with respect to the visual field 410 as determined by the configuration of the light measurement values included in the image 430. The image itself is just an array of numbers that exists on the digital memory. Therefore, the orientation of the array with respect to the horizontal direction in FIG. 4 is meaningless.

  In this specification, the “measurement process” is a process of measuring a field of view by analyzing a captured image and generating a value called “image measurement value”. Examples of image measurement values include brightness, contrast, edge quality, edge position, number of edges, peak correlation value, peak correlation position, and the like. Many other image measurements that can be used within the scope of the present invention are known in the art. Image measurements can be obtained by any suitable form of analog signal processing and / or digital signal processing.

  In the exemplary embodiment of FIG. 3, the measurement process consists of digital calculations on the captured image stored in the SRAM 320 that are executed by the ARMv4T processor 310 according to control of software instructions stored in the SRAM 320 or flash memory 322. For example, the luminance image measurement may be generated by calculating an average of pixels for the image, or a portion thereof. Contrast image measurements may be generated by calculating the standard deviation of pixels for an image, or part thereof.

  It is not always necessary to generate the same image measurement values for all captured images, and it is not necessary to generate the same number of image measurement values. In one embodiment of the present invention, no analysis is performed on a particular captured image, and therefore image measurements may not be generated. However, for clarity of explanation, such images are ignored and it is assumed that all captured images are analyzed by the measurement process.

  In some applications of the present invention, a captured image is generated even when there is no object in the field of view, and therefore there may be no knowledge of the object from the image measurements. In other embodiments, the object is always in view, but not all image measurements may be relevant for determining object information. Also, although all image measurements are relevant, some object information may have to be determined at various points in order to be able to generate a signal. Accordingly, a linear detector according to some embodiments of the present invention may include a “selection process” for selecting an image measurement determined to be responsive to an object from a variety of image measurements. . The selected image measurement value is called an “object measurement value”. As will be described in detail later, the selection process uses at least a portion of the image measurements to determine which image measurements to select.

  In the exemplary embodiment of FIG. 3, the selection process is performed by the ARMv4T processor 310 in accordance with the control of software instructions stored in the SRAM 320 or flash memory 322, and the image measurements stored in the SRAM 320 or ARMv4T processor 310 registers. Includes digital calculations on values.

  A linear detector according to the present invention may include a “determination process” that analyzes object measurements and generates object information. In the exemplary embodiment of FIG. 3, the decision process is stored in SRAM 320 or stored in a register of the ARMv4T processor 310, executed by the ARMv4T processor 310 according to control of software instructions stored in the SRAM 320 or flash memory 322. Consisting of digital calculations on measured object values.

  A linear detector according to some embodiments of the present invention may optionally include a “signal process” for generating a signal carrying object information. The form of the signal may be any form known in the art, including but not limited to, for example, electrical pulse, analog voltage or current, serial line, USB line, or Ethernet (R) line, wireless transmission Or the form of the data transmitted in optical transmission, the message transmitted to a software routine, or the function call to a software routine, etc. is mentioned. The signal is generated spontaneously by a linear detector or in response to a request from another device.

  In the exemplary embodiment of FIG. 3, the signal process is stored in SRAM 320 or stored in a register of the ARMv4T processor 310, executed by the ARMv4T processor 310 according to the control of software instructions stored in the SRAM 320 or flash memory 322. Digital calculation of the detected object information and generation of the signal 362 by the timer means 334. Signal 362 is an electrical pulse and timer 334 is used by ARMv4T processor 310 to ensure that the pulse is generated at the correct time.

  Of course, any combination of processes that are somehow related to each other is also a process. Accordingly, the descriptions of the various embodiments described as including various specific processes are made for the sake of convenience only and have been made merely for ease of understanding. A reconfiguration of an embodiment that includes more or fewer processes that implements an equivalent set of operations for an equivalent purpose by a combination of processes also constitutes an equivalent embodiment. Should be considered.

Overview of the measurement process To enable the selection process to use an image measurement to select an object measurement that is determined to be responsive to the object, an image measurement that responds to the presence of an object in the field of view. It is often desirable to produce. Specifically, the measurement process may calculate a value (or set of values) referred to herein as a “score measurement” that indicates the likelihood that an object exists, ie, certainty. For example, the greater the value, the greater the likelihood, ie the certainty. Image measurements obtained from a given image can be selected when it is determined that the certainty that the object is present is sufficiently large according to some criteria. However, some image measurements are selected unconditionally or according to some other criteria.

  There are many types of image measurements that are suitable for determining the certainty that an object is present, and the choice is made depending on the intended use of the linear detector. Image measurements such as brightness, contrast, edge quality, number of edges, and peak correlation values can be used, as well as other image measurements that will occur to those skilled in the art.

  It may often be desirable to generate image measurements that are responsive to the position of an object in the field of view. Such measurements are referred to herein as “position measurements”. As can be seen from the subsequent description with respect to FIG. 11, the selection process can use such measurements. The decision process uses such measurements to locate the object. Various types of image measurements suitable for locating an object are known in the art. Exemplary embodiments are described below.

  Of particular importance is a measurement process that incorporates a “pattern detection” process. The pattern detection process is a process that responds to the spatial arrangement of gray values in the image. For example, a luminance measurement made by calculating the average of various pixels or a contrast measurement made by calculating the standard deviation of various pixels is not a pattern detection process. This is because these results are independent of the spatial arrangement of gray values. Examples of pattern detection processes include edge detection and correlation, for example, and many others are known in the art.

Measurement Process for Exemplary Embodiments FIGS. 5 and 6 illustrate image measurements in response to the presence of an object and the position of the object, as generated by the measurement process used by one embodiment. Yes. The measurement process includes a normalized correlation process based on the well-known normalized correlation process. The normalized correlation process is an example of a pattern detection process that uses a model pattern that corresponds to an expected pattern of gray values in an image of an object in the field of view. The normalized correlation process finds a position in the captured image that is most similar to the model pattern and generates an image measurement such as the similarity at that position. The similarity is an example of a score measurement value, and the position itself is an example of a position measurement value. Note that if there is no sufficiently similar position in the captured image, the position measurement value may be meaningless. Normalized correlation provides a number of advantages well known in the art.

The score measurement is responsive to the presence of the object, and the position measurement is responsive to the position of the object. In an exemplary embodiment, where r is a normalized correlation coefficient, the score measurement is min (r, 0) ² and, according to the position measurement, sub-parameters can be sub- Pixel accuracy is obtained.

  The model pattern may include any pattern of gray values, for example, a simple edge pattern consisting of two gray values, or a complex pattern consisting of multiple gray values. The model pattern can be obtained by various methods, for example, from prior knowledge of the gray value prediction pattern or from an exemplary object using a training process.

  FIG. 5 illustrates obtaining a model pattern utilizing a training process in one exemplary embodiment. An exemplary object 500 is placed in the field of view 502. The exemplary object has a light pattern that includes bright areas 510, 517, dark areas 513, 515, medium light areas 512, 514, and medium dark areas 511, 516.

  The training process takes a one-dimensional training image 540 from the example object 500. Training image 540 is shown by a graph 520 that plots gray values as a function of position in the image. The position of the pattern in the training image is freely defined as being at the zero position as indicated by the original line 532 passing through the boundary between regions 513 and 514. In this example, this 0 position is also a reference point.

  In the exemplary embodiment of FIG. 5, the model pattern 542 is obtained from a portion of the training image 540 that exists between a first position 530 at a position of −25 and a second position 534 at a position of +35. It is done. As shown here, since the training process 540 is subject to some noise, the model pattern 542 is averaged 512 and smoothed using a 3 pixel wide boxcar filter. Obtained by Any method known in the art may be used to obtain the model pattern from one or more training images, including no modification. One example of a known method is to use a plurality of exemplary objects and average the training images taken from each object.

  In the exemplary embodiment of FIG. 5, image 540 includes 100 pixels and model pattern 542 includes 60 pixels. In this embodiment, the normalized correlation process searches for peak positions only where the model pattern is present in the image in its entirety. As a result, there are 41 such positions. Various performance characteristics, such as speed, range, and discriminating ability, may be adjusted as needed by adjusting their size. Such adjustments can be performed by the HMI 350 by programming appropriate values in software, or by other well-known means. In other exemplary embodiments designed to operate at higher speeds but with a narrower range and somewhat smaller discrimination capabilities, the image size may be 84 pixels and the model pattern size may be 54 pixels. is there.

  In the embodiment according to the above description, the model needs to have at least 3 pixels, and the captured image needs to be at least as large as the model. However, there are no other restrictions on the image size and model size. As will be apparent to those skilled in the art, the difference in size results in various technical tradeoffs between speed, range, discriminating ability, and other characteristics.

  In the exemplary embodiment of FIG. 5, the direction of motion is from left to right, ie, the direction in which the position increases. As will be appreciated, the portion of the training image 540 used to obtain the model pattern 542 is offset in the downstream direction by 5 pixels in this example from the center of the field of view. As a result, it is possible to fit the model pattern 542 completely into the 100 pixel image 540 at more positions between the upstream end and the reference point at the location of the original line 532, and as a result, More data can be generated in preparation for the prediction described below. In general, the offset can be adjusted by HMI 350 by programming appropriate values into software, or may be adjusted by other well-known means. The offset can be calculated in response to the location of the reference point and / or the desired latency, and there may be no need to use the offset.

  FIG. 6 illustrates a measurement process for performing image measurements in response to object positions using a normalized correlation process in response to model pattern 542. As shown in the graph 620, an object 600 having a light pattern similar to that of the exemplary object 500 is somewhere in the field of view 602 when the image 640 is taken.

  The graph 622 shows the model pattern 642 at the peak position (in this example, the position of −15). Therefore, in this example, the measurement process performs image measurement of the object position in the captured image 640. The image measurement is -15 pixels.

  Since the model pattern 642 is very similar to the image 640 at the peak position, the score is high, so that the measurement process has a high certainty that the object is in the field of view at the time the image 640 was taken. As shown, an image measurement of the presence of the object will be generated. Because of the high certainty, the two image measurements are considered to be responsive to the object, and the two image measurements can be selected to be included in the object measurement.

  The above description of the measurement process is directed to certain specific embodiments and is similar to the teachings described herein, using major modifications that may occur to those skilled in the art, or entirely different methods. It is contemplated that effects or other desired effects can be achieved. For example, all numerical constants used should be construed as examples and not as limitations. The model can also be obtained from prior knowledge about the object rather than from training images. For example, instead of using a correlation and a model, a well-known edge detection method using a gradation and a model can be used to obtain a score measurement value and a position measurement value.

Latency and prediction Consider an object that moves across a reference point at a reference time. One common method of designing a sensor to detect and locate an object is to generate a signal, typically a pulse, at the exact time that the object crosses a reference point. The presence of the pulse serves to inform that the object has been detected, and the timing of the pulse serves to inform the position of the object by notifying the time when the object was at the position of the known reference point. By using the signal, the desired automation device can work on the object when the object is in a known position. What is important is that it generates a signal when an object is present and does not generate a signal in response to anything else that occurs in front of the sensor at other times.

  Of course, in practice, any system or method can only estimate the reference time, which causes some errors. The average error, ie the average interval from the reference time to the signal time, is usually referred to in the art as “latency” or “response time”. Prior art devices have some latency, typically hundreds of microseconds latency. The effect of the waiting time is that when the signal is generated, the object has already moved downstream from the reference point by an amount that depends on the speed of movement. If the waiting time and speed are known accurately, the automation device can be set to work on the object at a known location. Of course, in practice, latency and speed are known only inaccurately. Thus, the position error will increase as the speed or latency increases. If an encoder or similar device is used, insufficient information regarding speed can be compensated, but information regarding latency cannot be supplemented.

  The standard deviation of the error is a measure of the reproducibility of the system or method. Since these errors are random, they cannot be corrected. Overall accuracy is affected by both latency and repeatability.

  The reason for the waiting time is that the sensor requires some time to reliably determine that an object is present. In general, the more time is spent making a decision, the more reliable the decision. For example, a simple phototransistor may cause a small latency, on the order of a few nanoseconds, but will falsely respond to a large number of useless objects. Prior art sensors make a trade-off between reasonable latency and reasonable reliability.

  FIG. 7 illustrates this decision delay for the exemplary device of FIG. Image capture is initiated when the linear optical sensor 340 exposes its light receiving element to light over an exposure interval 700. Pixels are transferred to the SRAM 320 under the control of the ARMv4T processor 310. This transfer requires a transfer interval 710. Next, over the calculation interval 720, a measurement process, a selection process, and a decision process are performed on the ARMv4T processor 310. The earliest time at which any operation (for example, signal generation) can be performed based on the measurement value in the image corresponding to the exposure interval 700 is the time of the decision point 740, which is the midpoint of the exposure interval 700. 730 is delayed by a judgment delay 750. The decision process has only information about the past and no information about the present.

  Note that in the exemplary apparatus of FIG. 3, the exposure interval, transfer interval, and calculation interval can all be overlapped, so that the interval between images is not the sum of the three intervals, but of those intervals. Only the largest of the can be. As a result, very high time resolution is achieved, but the decision delay 750 is not reduced at all.

  Judgment delay is inevitable, but waiting time is not. The present invention eliminates latency by predicting when an object will cross the reference point and scheduling the signal to occur at that time. If you have enough knowledge about the motion of the object, both from the measurement of how the object has moved and the expectation that the object will continue to move that way for a sufficient amount of time in the future, the prediction is Become accurate.

  Further, according to the present invention, even when the waiting time is zero or negative, detection with much higher reliability is possible as compared with the prior art system. Detection is based on a large number of images in a time range that may be as long as many milliseconds, so that careful judgment can be made without sacrificing latency. In addition, detection using pattern detection such as normalized correlation can basically distinguish a desired object from other objects that rarely enter the field of view.

  By using a good prediction method, the latency can also be negative. In fact, there is a limit to the range of negative latency that can be achieved, but negative latency provides a variety of new capabilities compared to the prior art, which may be useful. . For example, if the automation device has a short latency between the reception of the signal and the operation, the negative latency in the linear detector according to the present invention is utilized so that the overall latency of the system is zero. In addition, the apparatus waiting time can be offset.

  Of course, the actual system or method will not have exactly zero latency. Thus, herein, “no latency” means a system or method that has significantly lower latency compared to reproducibility. For such systems or methods, the signal may sometimes arrive before or after the reference time.

Prediction in Exemplary Embodiment In the exemplary embodiment, the imaging process captures images in the same manner as described above, and for each image, generates an imaging time (t) corresponding to the middle of the exposure time. To do. For example, the ARMv4T processor 310 can determine the shooting time by reading the timer 334 in the middle of the exposure interval or at some other time and correcting the time difference between that time and the middle of the exposure interval.

  Using a measurement process that utilizes normalized correlation as described above, a set of score (s), position (x) pairs is obtained for an object moving through the field of view. Therefore, three values (s, t, x) are obtained for each captured image. That is, two image measurement values are obtained from the measurement process, and the photographing time is obtained from the photographing process. In this section, we will consider only (t, x) pairs, ignore the selection process, and consider only that part of the detection and the signal process related to prediction. Selection and detection are described in detail in other sections below.

  The prediction accuracy depends on various factors. For example, in an exemplary embodiment, a linear detector according to the present invention operates at a rate of 8000 or more images per second, and the operation is constant. Decision delay 750 is typically less than 300 μs. With a high temporal resolution of about 125 μs, it is possible to obtain accurate t measurements and a large number of (t, x) pairs for each object as it approaches the reference point. A short judgment delay means that any deviation from a constant speed occurs in a very short interval. By using normalized correlation with sub-pixel interpolation, an accurate x measurement is obtained.

  8-10 illustrate prediction and signal generation for an exemplary embodiment. The prediction in this embodiment is a combined effect of the imaging process, the measurement process, the selection process, and the judgment process that operate simultaneously.

  FIG. 8 is a plot 800 showing the relationship between the imaging time t and the object position x for an object moving through the field of view. The time value is shown in units of microseconds from any arbitrary starting point, and the position value is shown in units of pixels measured from the reference point 822. As described elsewhere, the time value is obtained from the imaging process and the position value is obtained from the measurement process. The measurement process further generates a score measurement s (not shown) for each point. For simplicity of illustration, the image capture / transfer / calculation speed is about 10 KHz, with a period of about 100 μs.

  Various (t, x) points, such as exemplary point 810, are plotted. A current time 832 is illustrated, and the current time corresponds to a decision point 740 for an image whose exposure interval center is at exposure time 830, ie, the most recent time at which a position measurement can be obtained. The judgment delay 840 is an interval between the exposure time 830 and the current time 832. So far, three points have been obtained.

  In an exemplary embodiment, three points with a sufficiently high score shall be sufficient to make a reference time prediction. The prediction is made using a best-fit line 820 that passes through the three points obtained so far, and the line 820 is calculated by the well-known least squares method. The predicted time 834 is a time when the best fit line 820 intersects the reference point 822.

  Of course, any suitable curve can be fitted to the (t, x) point, and the curve corresponds to some function f, where x = f (t). Of course, the best fit line 820 is an example of such a curve corresponding to a constant speed assumption. In other embodiments assuming constant acceleration, a best-fit parabola may be used.

  In the example of FIG. 8, the signal delay interval 842 from the current time 832 to the predicted time 834 is about 2170 μs. In an exemplary embodiment, a signal delay interval greater than 1000 μs may be considered too long to be reliable. It is not only the predicted time that is inaccurate, but there is not enough evidence to show that the object will actually cross the reference point. Therefore, no signal scheduling is performed.

  FIG. 9 shows a plot 900 similar to the plot of FIG. However, in this case, 15 (t, x) points including the exemplary point 910 are obtained. A current time 932, the most recent exposure time 930, and a decision delay 940 are shown. The best fit curve 920 intersects the reference point 922 at the predicted time 934. The signal delay interval 942 is about 190 μs. Since there are at least three points with a sufficiently high score and the signal delay interval 942 is less than 1000 μs, the signal is scheduled to occur at the predicted time 934.

  If the criteria for scheduling a signal are set as in the exemplary embodiment, it is clear that the signal has been scheduled multiple times before the current time 922. For example, at time 950, if the last point from which data was acquired was point 952, the criteria would be met and a pulse would be scheduled. The closer the object moves to the reference point 922, the more accurate the prediction. This is because the number of points that fit the line increases and the prediction is not extrapolated. Thus, when a signal is scheduled, it replaces the previously scheduled signal, so that the predicted signal time is updated for virtually all images as the object approaches the reference point. It will be.

  FIG. 10 shows a plot 1000 similar to the plot of FIG. However, in this case, 17 (t, x) points including the exemplary point 1010 are obtained. A current time 1032, the most recent exposure time 1030, and a judgment delay 1040 are shown. Best fit line 1020 intersects reference point 1022 at predicted time 1034. The signal delay interval 1042 is negative by about 130 μs, which means that the signal should have already occurred due to a previously scheduled signal.

  There are two cases. First, if the signal is actually generated by a pulse edge 1050 that occurred before the current time 1032 as shown in FIG. 10, there is nothing left to do with the current object. . No signal is scheduled. If for some reason the signal is not yet generated (if the signal is scheduled but not generated, or the signal is not scheduled), the signal is generated immediately.

  8-10 and the accompanying description describe an embodiment without latency, for example, positive or negative values at predicted reference times such as times 834, 934, and 1034. It is also possible to introduce positive or negative latency by adding. The latency may be adjustable using the HMI 350. Other than the additional means for adding this value, nothing has changed in the above embodiment. Even with latency, the signal is generated using the best available data.

  The added latency can also be thought of as reducing the decision delay, eg, 750, 840, 940, and 1040, by the latency value. A positive latency effectively reduces the decision delay and a negative latency effectively extends the decision delay. If the latency is equal to the actual decision delay, the actual decision delay is zero, signal generation is no longer predictive, and the extrapolation is interpolation. Further, as the waiting time increases, more data points for line fitting can be obtained. Therefore, the accuracy of signal timing increases as the latency increases.

  If there is no waiting time, the best reference point is the downstream end of the field of view. In this case, the entire field of view can be used, and the maximum number of (t, x) points used for prediction can be generated. As the reference point moves upstream, the field of view becomes less used, the number of available points decreases, and accuracy decreases. If the reference points are too far upstream in the field of view, the system will not work properly.

  As the reference point moves downstream beyond the field of view, the extrapolation increases and accuracy decreases. At any point in time, the maximum signal delay interval criterion of 1000 μs cannot be met. In the exemplary embodiment, when the object moves completely out of the field of view, the signal is scheduled regardless of the signal delay interval.

  The above description regarding the prediction relates to certain specific embodiments, and uses similar changes in accordance with the teachings described herein, using significant changes conceived by those skilled in the art, or entirely different methods, or It is believed that other desirable effects can be achieved. For example, all numerical constants used should be construed as examples and not as limitations. Not all signal delays that are too long need to be considered unreliable. The maximum reliable signal delay and minimum number of points required to schedule the signal may vary from image to image or from object to object based on some desired criteria. For example, a limit on the maximum number of points may be used so that once the limit is reached, no signal rescheduling takes place.

Exemplary Selection Process There are a wide variety of selection processes for the linear detector according to the present invention, and methods for configuring the decision process. In some embodiments, the two processes operate independently. That is, the selection process selects object measurements from the various image measurements and enters them into the decision process in response to image measurements rather than in response to something done by the decision process. hand over. In the exemplary embodiment, the selection process shown in FIG. 11 and described in this section operates concurrently with the decision process to select object measurements and use them to detect and locate objects. . Thus, the two processes will be depicted in separate figures and will be described in different sections, but the description needs to be somewhat overlapped. In addition, as described above, the selection process and the determination process may be equivalently combined as a single composite process. However, in order to make the description easy to understand, each process will be described individually.

  As described above, three values (s, t, x) are obtained for each captured image. Two image measurements are obtained from the measurement process and the imaging time is obtained from the imaging process. The selection process must select an object measurement to pass to the decision process from among the various image measurements (s, x). In the exemplary embodiment, the selection is made on a per image basis, and from a given image, a pair of values (s, x) is selected or not selected. Next, including the shooting time of the selected image, the decision process allows the triplet (s, t, x) to be obtained from the selected image. Within the scope of the present invention, any other suitable strategy for selecting object measurements from image measurements can be used, in which case imaging time and other data may or may not be used. It does not have to be.

  The selection process ignores image measurements when no object appears through the field of view. At those times, the selection process is called “inactive”.

  When an object that passes through the field of view appears, the selection process is said to be “active”. The selection process selects a set of triple values (s, t, x) used to determine whether an object has been detected in the determination process. When an object is detected, the decision process locates the object by calculating a reference time. This calculation may or may not be prediction according to the desired waiting time as described above.

  The selection process may select some or all of the triple values corresponding to each object. However, in the general case, only a part is selected. First, the rules are briefly described, and then they are described in detail with reference to FIG. In the following description, numerical constants corresponding to exemplary embodiments are used, but any other suitable value or method for performing the selection may be used within the scope of the present invention. it can.

  Selection of a given object is initiated when an image is found where s ≧ 0.6 and x is at least 2 pixels upstream from the reference point (ie, the selection process is active) . Such upstream requirements cause hysteresis in object detection so that a stationary or very slowly moving object is not detected multiple times when crossing a reference point.

  If the selection is active and s <0.6 for 3 consecutive images, the object is considered to have already passed the field of view and the selection becomes inactive.

  For every object, only 128 triple values are used. This is because the older the value, the lower the ability of the value to indicate what is currently happening. Three 128-element FIFO (First-In First-Out) buffers are used to hold the most recent triple values. When a new triple value is added to the FIFO, if the FIFO is full, the oldest triple value is deleted and the object measurements are as if none of those object measurements were selected from the beginning. Is deselected. As will be appreciated, the selection process becomes inactive when the FIFO is empty, and becomes active otherwise.

  If the FIFO is full and the x measurement in it is near that centered on the reference point, the selection ends and becomes inactive. This rule is useful when the desired latency is long enough that the reference time can be calculated by interpolation (not by prediction). Once the FIFO is full and the x measurement is near the center of the reference point, any additional measurements will reduce the accuracy of the interpolation.

  In the following description regarding FIG. 11, it is assumed that the direction of motion is positive x, and the downstream position has a larger x value than the upstream position. However, it will be apparent to those skilled in the art how to modify the description to obtain a negative direction of motion, i.e., to detect and locate objects moving in either direction.

  In FIG. 11, a reset step 1100 clears the FIFO (see FIG. 12) and deactivates the selection. This is used for system initialization and also when the selection is finished for a given object.

  In shooting step 1102 (part of the shooting process), the next image is shot, and a shooting time (t) 1164 is generated. Search step 1104 (part of the measurement process) uses normalized correlation to generate a score (s) 1160 and a position (x) 1162. Presence test step 1106 determines the certainty that an object is present in the field of view. If the score 1160 ≧ 0.6, the certainty is considered sufficient to indicate that the object is present. This does not mean that the object is actually determined to exist. The decision is deferred in preparation for a decision process based on the analysis of all selected triples.

  If it is determined that there is no object, and the selection is inactive (branch to “yes”), the first active test step 1108 instructs the selection process to continue looking for the object. If it is determined that no object is present, but the first active test step 1108 indicates that the selection is active, then the increment step 1110, the miss test step 1112 and the miss counter 1150 are used. Thus, three consecutive images in which no object appears are detected. If no more than two such images have been detected so far, control returns to the shooting step 1102.

  If the selection is active and three consecutive images with no object appearing are detected, a final decision step 1132 (part of the decision process) is entered. If the three conditions are met, the first signal step 1134 (part of the signal process) is scheduled so that the signal is generated at signal time (z) 1170 (calculated as described below). To do. The first condition is that there is no signal already scheduled or already generated. The second condition is that the “intersection” flag 1152 indicates that the object has actually crossed the reference point. The third condition is that the good point counter (g) 1174 (managed by the decision process) indicates that at least three triple values in the FIFO are s ≧ 0.75. This third condition is a criterion that the decision process uses to determine that an object has been detected. Regardless of whether the signal is scheduled, control returns to the reset step 1100.

  If the presence test step 1106 determines that the certainty is sufficient (score 1160 ≧ 0.6), the “missing” initialization step 1114 sets the missed counter 1150 to zero. A second active test step 1116 determines whether this is the first image of a new object. If it is the first image (branch to “yes”), the hysteresis test step 1121 determines whether the object appears sufficiently upstream from the reference point (r) 1154. If it does not appear well upstream (branch to “No”), control passes to the capture step 1102 and the selection remains inactive. If the object is sufficiently upstream, the “intersection” initialization step 1122 sets the “intersection” flag 1152 to false. This time, the selection becomes active.

  If the second active test step 1116 indicates that this is not the first image of the new object (branch to “No”), and if the object crosses the reference point 1154, the cross test Step 1118 and intersection set step 1120 set the “intersection” flag 1152 to true.

  When control moves to the line fit step 1124 (part of the decision process), the object measurement score 1160 and the position 1162 are already selected and the imaging time 1162 is available. These values are used to calculate the best fit line as described above in connection with FIGS. 8, 9, and 10 or as described below in connection with FIG. Examples of the calculation results include a signal time (z) 1170, an object speed (v) 1172, and a good point counter (g) 1174.

The execution decision step 1126 implements the signal scheduling rules described in connection with FIGS. 8, 9, and 10. When the following four conditions are met, the second signal step 1128 schedules the signal for signal time 1170:
1. 1. The signal has not yet been generated. The signal time 1170 is within 1 ms of the current time (including the case where the current time has passed the signal time 1170).
3. That the object velocity 1172 indicates downstream motion (here, the “intersection” flag 112 is not used because the object is simply predicted to cross the reference point 1154 at some future time. (The need for downstream movement is a suitable alternative.)
4). Good point counter 1174 indicates that at least three triples in the FIFO have s ≧ 0.75.

  If the FIFO is full and the x measurement is near the reference point 1154, the centralized test step 1130 implements the rules described above. Note that the test for concentrated measurements is represented by Σx> 128r, which means that the average x in the FIFO must be greater than the reference point 1154. Here, the assumption applied to the exemplary embodiment is that the direction of motion is positive and the FIFO contains 128 elements. Tests are represented to avoid division. Division is generally very expensive compared to multiplication. Of course, dividing by 128 is just an inexpensive shift. However, in some embodiments, the FIFO size may not be a power of two.

  If centralized test step 1130 is passed (branch to “yes”), control passes to final decision step 1132 and the selection is deactivated at reset step 1100. If the test fails (branch to “No”), control is passed to the capture step 1102 and the selection remains active.

  Note that in the exemplary embodiment, if a scheduled signal occurs, the selection will not be inactive by that signal. Since execution decision step 1126 and final decision step 1132 are always failed (branch to “no”), no further signals are scheduled. However, the selection becomes inactive after waiting for the object to pass through the field of view or for the x measurements to be centralized. This is yet another protective measure that prevents detecting an object multiple times (in addition to the hysteresis provided by hysteresis test step 1121).

  Yet another advantage is obtained by keeping the selection active after the signal is generated. One is that an interpolated reference time is obtained after some time has passed since the reference point was crossed. This interpolated reference time is more accurate than any predicted time, and by comparing with the predicted time, the accuracy of the system can be evaluated and automatic adjustment can eliminate latency errors in the system. . Another is that other properties of the object (eg, the velocity of the object) can be obtained and, if necessary, can be conveyed by other signals.

  The “missing” counter 1150, “intersection” flag 1152, reference point 1154, score 1160, position 1162, shooting time 1164, signal time 1170, speed 1172, and good point counter 1174 are stored in the memory 1180. The memory 1180 may be a part of the SRAM 320.

  The above description of the selection process is directed to certain specific embodiments and uses similar modifications in accordance with the teachings described herein, using significant variations that will occur to those skilled in the art, or entirely different methods. Or other desired effects could be achieved. For example, all numeric constants are to be interpreted as examples and not as limitations. The rules for scheduling signals may be changed as needed. There is no need to place a limit on the number of triplets used for an object. The good point counter 1174 may be omitted, or may be replaced with another statistical value such as an average value of scores in the FIFO.

Exemplary Decision Process FIG. 12 illustrates a decision process portion 1280 used in the example embodiment described above in connection with FIGS. 8, 9, 10, and 11. When the imaging process generates an image 1200, the normalized correlation process 1210 uses the model 1202 to generate an image measurement position (x) 1216 and a score (s) 1218. The shooting process uses the clock 1212 to generate a shooting time (t) 1214.

The selected triple value (s, t, x) is passed to the decision process portion 1280 and placed in the shooting time FIFO 1220, position FIFO 1222, and score FIFO 1224. Together, these FIFOs constitute a FIFO 1226 as described in detail above. The selected triple value (s, t, x) is further passed to the statistical element 1230. Statistical element 1230 calculates the known least squares line fit and Σt, Σt ² , Σx, Σtx, and Σ (s ≧ 0.75) as required for good point counter 1174. The summing element 1232 calculates the sum over all triple values in the FIFO 1226.

  When selecting a new triple value (s, t, x), if the FIFO 1226 is full, the oldest triple value is passed to the subtraction element 1234, which subtracts it as needed. Subtract from the sum of and effectively deselect the measurement. As will be appreciated, for each selected triple value (s, t, x), at most only one set of additions and one set of subtractions are required. For each triple value, there is no need to recalculate the total across the contents of FIFO 1226.

  In order to pursue numerical accuracy and computational efficiency, it is desirable to keep all the sums using the t value relative to the oldest time in the shooting time FIFO. This requires a slightly more complicated procedure for processing new triple values when the FIFO 1226 is full. However, the additional cost of doing so is well offset by other savings in many embodiments.

In the following equation for updating the total when the FIFO 1226 is full, the following definition is made:
t New shooting time 1214
x New position 1216
t ₀ oldest time obtained from the shooting time FIFO 1220 t ₁ second oldest time obtained from the shooting time FIFO 1220 x ₀ oldest position obtained from the position FIFO 1222 T current Σt
Q Current Σt ²
X Current Σx
C Current Σtx
Σt after T 'update
Q 'Σt ² after update
X 'Σx after update
C 'Σtx after update
The size of N FIFO 1226 (128 in the exemplary embodiment)
t _u t ₁ -t ₀
t _v t−t ₀

The update formula for the sum is as follows:
X ′ = X + x−x ₀
T ′ = T + t _v −Nt _u
Q ′ = Q + t _u [(N−1) t _u −2T] + (t _v −t _u ) ²
C ′ = C + t _u (x ₀ −X) + x (t _v −t _u )

  For each selected triplet (s, t, x), detection element 1240 determines whether there are at least three scores in score FIFO 1224 that are at least 0.75, ie, [Σ ( s ≧ 0.75)] ≧ 3 is determined. If not, the decision process determines that no objects have been detected yet, and therefore no signal is generated by the signal processing 1260. The detection element 1240 corresponds to a part of the execution determination step 1126 and the final determination step 1132, and a test is performed to determine whether or not the good point counter (g) 1174 ≧ 3.

  For each of the selected triples (s, t, x), the line fit element 1250 uses the sum calculated by the statistical element 1230 as well as the latency value 1252 and the reference point value 1254 to best fit. A line is calculated, and from the parameters of the line, the selection process of FIG. 11 and the signal time 1170 used by the signal processing 1260 are calculated, and further the object velocity 1172 used by the selection process of FIG. 11 is calculated. Well-known equations are used to calculate the best fit line, signal time 1170, and object velocity 1172 (which is just the slope of the best fit line).

  Latency value 1252 and reference point value 1254 are stored in SRAM 320 (FIG. 3) and can be adjusted using HMI 350 as needed.

  The above description of the decision process is directed to certain specific embodiments and is similar to the teachings described herein using major changes that will occur to those skilled in the art, or entirely different methods. It is contemplated that effects or other desired effects can be achieved. For example, all numerical constants used should be construed as examples and not as limitations. Instead of lines, other curves may be fitted to various points. A weighted line fit can also be used, for example, where the weight is a function of the elapsed time of the points.

Exemplary Signal Processing In the above, timer 334 and signal 362 in FIG. 3; signal scheduling associated with FIGS. 8, 9, and 10; pulse edge 1050 in FIG. 10; first signal step 1134 in FIG. The signal processing of the exemplary embodiment has been described in connection with the second signal step 1128; as well as the signal processing 1260 of FIG.

  Timer 334 is set to operate at 1/32 of the clock speed of microcontroller 300, producing a resolution of 0.64 μs. It is easy to set timer 334 to generate a pulse after a programmable delay (see ATMEL document 6175E-ATARM-04-Apr-06), and the programmable delay is determined by the desired signal time and It is desirable to set it equal to the difference between the current time.

  In order to achieve slightly higher timing accuracy, timer 334 is used for both clock 1212 and pulse edge 1050 (which appears in signal 362). When scheduling a signal for a specific time, the compare register of timer 334 is set to generate a pulse edge. The compare register is set on the fly without stopping the timer only if the pulse has not yet started due to previous scheduling. Any desired assembly language routine that operates without interruption can be used to achieve the desired operation.

  The above description relating to signal processing is directed to certain specific embodiments and is similar to the teachings herein, using significant changes that will occur to those skilled in the art, as well as entirely different methods and processes. It is also considered possible to achieve the above effect or other desired effects. The signal need not be a pulse edge and may be generated by other means well known in the art. A variety of different software and hardware elements can be used to achieve the desired effect.

Further object information generated by the decision process As will be apparent to those skilled in the art, according to the above description of the exemplary measurement process, selection process, and decision process, the decision process for analyzing object measurements It is possible to generate a great variety of object information. Without limitation, such object information includes, for example, information that is responsive to the presence of an object in the field of view, and more specifically, information that is responsive to existing states, or events that include transitions between such states, Both of them are mentioned.

Although it does not limit, as an example of the state which exists, for example,
-Object is in the field of view-Object is in the field of view upstream of the reference point-Object is in the field of view downstream of the reference point-Object is field of view at a speed above the specified speed A state where the object is moving • A state where the object is between two reference points in the field of view.

Although not limited, as an example of the event, for example,
An object enters the field of view.An object passes through the field of view.An object crosses the reference point in any direction.An object crosses the reference point in a specific direction.An object is within the field of view. Reaching the top of movement in the case ・ The object stops in the field of view.

  In order to more fully understand an exemplary embodiment for generating information in response to the presence of an object in the field of view, consider an example. Let M be a set of image measurements generated by the measurement process. Let P be a subset (possibly all) of M containing those image measurements that respond to image presence. The selection process responds to some or all of P and produces an object measurement S that is a subset of M. S may include a part of P, may include all, or may not include any element of P. S is used to obtain information in response to the presence of an object in the field of view. There may be no measurements in S that are responsive to the presence of the object, but all of S is still responsive to the object as is evident from the process in which they were selected.

  For clarity, assume that M includes a score measurement and a position measurement. The score measurement is responsive to the presence of the object. That is, the larger the value, the higher the possibility that an object exists. They are the subset P used by the selection step. Position measurements do not respond to object presence. That is, it cannot be determined from the position measurement value whether or not an object exists. If there is no object, the position measurement is meaningless (just noise), but it cannot be determined from the position measurement value. Thus, the selection process uses the score value (subset P) to select the measurement values in S. It may be just a position measurement. Their position measurements do not respond to the presence of the object, but the fact that they are selected means that they respond to the object. Because they have been chosen to respond to the score, they provide the correct object position. The decision process can use these selected object positions to determine, for example, whether the object has crossed a reference point.

Exemplary HMI for process training, reference point setting, testing, and / or execution
FIG. 13 shows an HMI and an exemplary box for the training process and for setting a reference point, such as may be used for setting the linear detector used in the application shown in FIG. Show.

  An exemplary box 1300 including an exemplary object 1302 (a label on the exemplary box 1300) and a decorative mark 1304 is placed such that the exemplary object 1302 is visible from the field of view of a linear detector (not shown). ing. A reference point 1310 is also shown.

  The HMI 1320 includes a training button 1330, an execute button 1332, a negative position indicator 1340, a positive position indicator 1342, a negative direction indicator 1350, and a positive direction indicator 1352. When the training button 1330 is pressed while the exemplary object 1302 is in the field of view 1312, one or more images are taken and used to generate a model as described in detail above in connection with FIG. The

  The training button 1330 may also be used to generate a setup signal for setting the reference point 1310. When the training button 1330 is pressed while the exemplary object 1302 is in the field of view 1312, one or more images are taken and used with the model, and the exemplary object 1302 is discovered by a normalized correlation process. The reference point 1310 is set to the found point.

  It should be noted that the training button 1330 may be used to achieve both by starting a training process, generating a setup signal for setting a reference point, or pressing a button.

  FIG. 14 shows the test object 1400 when in the next three test positions relative to the field of view 1412 and the reference point 1410. First test position 1450: At this time, the first test object 1400 is in a negative position relative to the reference point 1410 in the field of view 1412. Second test position 1452: At this time, the test object 1400 is in a positive position relative to the reference point 1410 in the field of view 1412. Third test position 1454: The test object 1400 is now far from the reference point 1410 and is no longer found in the field of view 1412.

  An HMI equivalent to the HMI 1320 of FIG. 3 is depicted in three states corresponding to the three test positions. The first HMI 1424 corresponds to the first test position 1450, the second HMI 1434 corresponds to the second test position 1452, and the third HMI 1444 corresponds to the third test position 1454. Note that the first HMI 1424, the second HMI 1434, and the third HMI 1444 represent the same HMI in three different states.

  Similarly, the first negative position indicator 1420, the second negative position indicator 1430, and the third negative position indicator 1440 represent the same indicator at three different test positions, the first positive position indicator 1422, the second Two positive position indicators 1432 and a third positive position indicator 1442 represent the same indicator at three different test positions.

  When the test object 1400 is in the first test position 1450, the first negative position indicator 1420 is on and the first positive position indicator 1422 is off and relative to the reference point 1410 in the field of view 1412. Indicates that the test object 1400 has been found in a negative position.

  When the test object 1400 is in the second test position 1452, the second negative position indicator 1430 is off and the second positive position indicator 1432 is on and is relative to the reference point 1410 in the field of view 1412. Indicates that the test object 1400 has been found at the positive position.

  When the test object 1400 is at the third test position 1454, the third negative position indicator 1440 and the third positive position indicator 1442 are both off and the test object 1400 can be found in the field of view 1412. Indicates no.

  By using a test object similar to the test object 1400 and an indicator as illustrated, the linear detector installer and other users can confirm that the appropriate model is being used and that the reference point is as desired. It can be confirmed. When an object with an appearance that closely resembles the model in use is placed in the field of view, the indicator indicates that a test object has been found, and when no such object exists, nothing is found. Must show.

  FIG. 15 shows an HMI 1520 that is used to determine the positive direction of motion 1512 and to activate the linear detector. When the execution button 1550 is pressed, the following three states are circulated as shown in the figure. That is, a training mode state where both the negative direction indicator 1542 and the positive direction indicator 1540 are off, a negative direction indicator 1542 is off, a positive direction indicator 1540 is on, an execution state where there is positive direction motion, and The negative direction indicator 1542 is on, the positive direction indicator 1540 is off, and there is an execution state in which there is a negative direction of motion. In the exemplary embodiment, the training process may be prohibited when the linear detector is in one of two execution states.

  In FIG. 15, the object 1500 is in the field of view 1514 and upstream of the reference point 1510. Since the movement direction 1512 is positive, this position corresponds to a negative position. Negative position indicator 1530 is on and positive position indicator 1532 is off, indicating that object 1500 has been found in a negative position within field of view 1514.

  The above description regarding HMI is directed to certain specific embodiments and uses the major changes conceived by those skilled in the art or entirely different methods and processes in accordance with the teachings described herein. It is believed that similar or other desirable effects can be achieved. The HMI may utilize a personal computer instead of or in addition to buttons and indicators. The HMI can use an alphanumeric indicator instead of a simple light, and the HMI allows the setting of any parameters used by one embodiment of the present invention.

  The above is a detailed description of various embodiments of the invention. Naturally, it is possible to change, omit, and add various forms and details without departing from the spirit and scope of the present invention. For example, the processors and computing devices described herein are exemplary and various processors and computers may be used in both stand-alone and distributed manners to perform the calculations described herein. can do. Similarly, the linear array sensors described herein are exemplary and can be modified or use different components within the scope of the teachings of the present invention. Also, some or all of the imaging process, measurement process, judgment process, and optional signal process can be combined with each other or separated into other forms. Although numerical constants are used herein in connection with various exemplary embodiments, any other suitable value or method within the scope of the present invention to perform the desired processing. Can be used. Therefore, this description should be taken as an example only and does not limit the scope of the present invention.

Exemplary embodiments of the present invention are listed below.
1. An optoelectronic system for generating information representing an object,
An optical sensor for measuring light in the field of view;
A motion process that generates relative motion between the object and the field of view in a direction of motion such that the object passes through the field of view;
A photographing process for photographing a plurality of one-dimensional images of the field of view,
Each of the plurality of images is responsive to an individual light measurement;
The image is oriented substantially parallel to the direction of motion;
An imaging process in which at least a portion of the image corresponds to a plurality of positions of the object relative to the field of view;
An optoelectronic system comprising: analyzing at least a part of the image and generating information representing the object.
2. The optoelectronic system according to 1, wherein the information representing the object includes information responsive to the presence of the object in the field of view.
3. The movement process causes the object to cross a reference point at a reference time;
2. The optoelectronic system according to 1, wherein the information representing the object includes an estimated value of the reference time.
4). 4. The optoelectronic system of 3, further comprising an electrical man-machine interface, wherein the reference point is adjusted in response to the man-machine interface.
5. 4. The optoelectronic system according to 3, further comprising a signal process for generating a signal indicating the estimated value of the reference time by being generated at a signal time responsive to the estimated value of the reference time.
6). 6. The optoelectronic system of 5, further comprising an electrical man-machine interface, wherein the signal time is adjusted in response to the man-machine interface.
7). 6. The optoelectronic system according to 5, wherein there is no waiting time between the reference time and the signal time.
8). 4. The optoelectronic system according to 3, wherein the estimated value of the reference time is determined by predicting the reference time.
9. An exemplary object;
A setup signal indicating that the exemplary object is at a setup position within the field of view, and the reference point is set in response to the exemplary object, the setup signal, and the setup position; 3. The optoelectronic system according to 3.
10. The optoelectronic system of claim 9, further comprising a signal process that generates a signal responsive to information representative of the object, wherein the signal occurs when the object crosses the setup position.
11. The optoelectronic system of claim 9, further comprising a signal process that generates a signal responsive to information representative of the object, wherein the signal occurs when the object is upstream of the setup position.
12 A test object,
The test object is detected upstream of the reference point;
And a man-machine interface including an indicator indicating a detection state selected from the fact that the test object was detected downstream from the reference point and the test object was not detected. 3. The optoelectronic system according to 3.
13. The optoelectronic system according to any one of 1 to 3, wherein the optical sensor is a linear optical sensor including a one-dimensional array of light receiving elements.
14 An optoelectronic system for generating information representing an object,
An optical sensor for measuring light in the field of view;
A motion process that generates relative motion between the object and the field of view in a direction of motion such that the object passes through the field of view;
A photographing process for photographing a plurality of one-dimensional images of the field of view,
Each of the plurality of images is responsive to an individual light measurement;
The image is oriented substantially parallel to the direction of motion;
An imaging process in which at least a portion of the image corresponds to a plurality of positions of the object relative to the field of view;
A measurement process that analyzes at least a portion of the image and generates image measurements;
A selection process for selecting an object measurement value consisting of the image measurement value determined to be responsive to the object from the image measurement value;
An optoelectronic system comprising: a determination process that analyzes the object measurements and generates information representative of the object.
15. At least a portion of the image measurement is responsive to the presence of an object in the field of view;
The selection process is responsive to at least a portion of the image measurement responsive to the presence of an object in the field of view;
15. The optoelectronic system according to 14, wherein the information representing the object includes information responsive to the presence of the object in the field of view.
16. The movement process causes the object to cross a reference point at a reference time;
At least a portion of the image measurement is responsive to a position of an object in the field of view;
At least a portion of the object measurement is selected from among image measurements responsive to the position of the object in the field of view;
The determination process is responsive to at least a portion of the object measurement selected from among image measurements responsive to a position of the object in the field of view;
15. The optoelectronic system according to 14, wherein the information representing the object includes an estimated value of the reference time.
17. 17. The shooting process according to 16, wherein each shooting time generates a plurality of shooting times that are responsive to the time at which the corresponding image was shot, and the determination process is responsive to at least one of the shooting times. Optoelectronic system.
18. The determination process includes curves for various points corresponding to object measurements selected from among the image measurements that include at least a portion of the imaging time and are responsive to the position of the object in the field of view. 18. The optoelectronic system according to 17, comprising fitting.
19. 17. The optoelectronic system according to 16, further comprising an electrical man machine interface, wherein the reference point is adjusted in response to the man machine interface.
20. 17. The signal processing further comprising: generating a signal responsive to information representative of the object, wherein the signal indicates information representative of the object by occurring at a signal time responsive to an estimate of the reference time. Of optoelectronic systems.
21. 21. The optoelectronic system of 20, further comprising an electrical man machine interface, wherein the signal time is adjusted in response to the man machine interface.
22. 21. The optoelectronic system according to 20, wherein there is no waiting time between the reference time and the signal time.
23. The system of claim 16, wherein the determining process determines an estimate of the reference time by predicting the reference time.
24. An exemplary object;
A setup signal indicating that the exemplary object is at a setup position within the field of view, and the reference point is set in response to the exemplary object, the setup signal, and the setup position; 17. The optoelectronic system according to 16.
25. 25. The optoelectronic system of 24, further comprising a signal process that generates a signal responsive to information representative of the object, the signal occurring when the object crosses the setup position.
26. 25. The optoelectronic system of 24, further comprising a signal process that generates a signal responsive to information representative of the object, wherein the signal occurs when the object is upstream of the setup position.
27. A test object;
The test object is detected upstream of the reference point;
And a man-machine interface including an indicator indicating a detection state selected from the fact that the test object was detected downstream from the reference point and the test object was not detected. 16. The optoelectronic system according to 16.
28. The optoelectronic system according to any one of claims 14 to 16, wherein the measurement process includes a pattern detection process.
29. An exemplary object;
Further comprising: a training process for obtaining a model pattern responsive to the exemplary object;
28. The optoelectronic system of 28, wherein the pattern detection process is responsive to the model pattern.
30. 28. The optoelectronic system of 28, wherein the pattern detection process includes a normalized correlation process.
31. The optoelectronic system according to any one of 14 to 16, wherein the optical sensor is a linear optical sensor including a one-dimensional array of light receiving elements.
32. An optoelectronic system for detecting and locating objects,
A linear optical sensor including a linear array of light receiving elements for measuring light within the field of view;
A motion process that generates relative motion between the object and the field of view in a direction of motion such that the object passes through the field of view and crosses the reference point at a reference time;
A shooting process of shooting a plurality of one-dimensional images of the field of view and generating a plurality of shooting times, each shooting time responding to the time when the corresponding image was shot;
Each of the plurality of images is responsive to an individual light measurement;
The image is oriented substantially parallel to the direction of motion;
An imaging process in which at least a portion of the image corresponds to a plurality of positions of the object relative to the field of view;
A measurement process that analyzes at least a portion of the plurality of images and generates position measurements and score measurements;
A determination process for generating an estimate of the reference time using the shooting time, the position measurement, and the score measurement;
An optoelectronic system comprising: a signal process that generates a signal indicating the reference time by being generated at a signal time in response to the estimated value of the reference time.
33. 33. The optoelectronic system according to 32, wherein the determination process estimates the reference time by predicting the reference time.
34. The optoelectronic system according to 32, wherein there is no waiting time between the reference time and the signal time.
35. An optoelectronic method for generating information representing an object, comprising:
Making optical measurements in the field of view using an optical sensor;
Generating relative motion between the object and the field of view in a direction of motion such that the object passes through the field of view;
Capturing a plurality of one-dimensional images of the field of view,
Each of the plurality of images is responsive to an individual light measurement;
The image is oriented substantially parallel to the direction of motion;
Capturing a plurality of one-dimensional images of the field of view, wherein at least a portion of the image corresponds to a plurality of positions of the object relative to the field of view;
Analyzing at least a portion of the image and generating information representative of the object.
36. 36. The optoelectronic method of 35, wherein the information representative of the object includes information responsive to the presence of an object in the field of view.
37. The relative motion is a motion such that the object crosses a reference point at a reference time,
36. The optoelectronic method according to 35, wherein the information representing the object includes an estimated value of the reference time.
38. 38. The optoelectronic method according to 37, further comprising adjusting the reference point using an electrical man-machine interface.
39. 38. The optoelectronic method according to 37, further comprising generating a signal indicative of the estimated value of the reference time by occurring at a signal time responsive to the estimated value of the reference time.
40. 40. The optoelectronic method according to 39, further comprising adjusting the signal time using an electrical man-machine interface.
41. 40. The optoelectronic method according to 39, wherein there is no waiting time between the reference time and the signal time.
42. 38. The optoelectronic method according to 37, wherein the analyzing step includes predicting the reference time.
43. Providing an exemplary object;
Generating a setup signal indicating that the exemplary object is at a setup position within the field of view;
38. The optoelectronic method of 37, further comprising: setting the reference point in response to the exemplary object, the setup signal, and the setup position.
44. 44. The optoelectronic method of 43, further comprising generating a signal responsive to information representative of the object, the signal occurring when the object crosses the setup position.
45. 44. The optoelectronic method of 43, further comprising generating a signal responsive to information representative of the object, wherein the signal occurs when the object is upstream of the setup position.
46. Preparing a test object;
The test object is detected upstream of the reference point;
Using a man-machine interface including an indicator indicating a detection state selected from the fact that the test object has been detected downstream from the reference point and the test object has not been detected. The optoelectronic method according to 37, further comprising:
47. The optoelectronic method according to any one of 35 to 37, wherein the optical sensor is a linear optical sensor including a one-dimensional array of light receiving elements.
48. An optoelectronic method for generating information representing an object, comprising:
Using an optical sensor for measuring light in the field of view;
Generating relative motion between the object and the field of view in a direction of motion such that the object passes through the field of view;
Capturing a plurality of one-dimensional images of the field of view,
Each of the plurality of images is responsive to an individual light measurement;
The image is oriented substantially parallel to the direction of motion;
Taking a plurality of one-dimensional images of the field of view, wherein at least a portion of the image corresponds to a plurality of positions of the object relative to the field of view;
Analyzing at least a portion of the plurality of images and generating image measurements;
Selecting an object measurement value comprising an image measurement value determined to be responsive to the object from the image measurement value;
Analyzing the object measurements and generating information representing the object.
49. At least a portion of the image measurement is responsive to the presence of an object in the field of view;
The selecting is responsive to at least a portion of the image measurement responsive to the presence of an object in the field of view;
49. The optoelectronic method of 48, wherein the information representative of the object includes information responsive to the presence of an object in the field of view.
50. The relative motion is a motion such that the object crosses a reference point at a reference time,
At least a portion of the image measurement is responsive to a position of an object in the field of view;
At least a portion of the object measurement is selected from among image measurements responsive to the position of the object in the field of view;
Analyzing the object measurement is responsive to at least a portion of the object measurement selected from image measurements responsive to a position of the object in the field of view;
49. The optoelectronic method according to 48, wherein the information representing the object includes an estimated value of the reference time.
51. The method further includes generating a plurality of shooting times, each shooting time being responsive to the time at which the corresponding image was shot, and the step of analyzing the object measurement value is further responsive to at least one of the shooting times 50. The optoelectronic method according to 50.
52. The step of analyzing the object measurement includes various points corresponding to the object measurement selected from among the image measurements including at least a part of the photographing time and responsive to the position of the object in the field of view. 52. The optoelectronic method according to 51, comprising fitting a curve.
53. 51. The optoelectronic method according to 50, further comprising adjusting the reference point using an electrical man-machine interface.
54. 51. The optoelectronic method of 50, further comprising generating a signal indicative of the estimated value of the reference time by occurring at a signal time responsive to the estimated value of the reference time.
55. 55. The optoelectronic method of 54, comprising using an electrical man-machine interface to adjust the signal time relative to the estimate of the reference time.
56. 55. The optoelectronic method according to 54, wherein there is no waiting time between the reference time and the signal time.
57. 51. The optoelectronic method according to 50, wherein the estimated value of the reference time is determined by predicting the reference time.
58. Providing an exemplary object;
Generating a setup signal indicating that the exemplary object is at a setup position within the field of view; and setting the reference point in response to the exemplary object, the setup signal, and the setup position. 51. The photoelectro-optical method of 50, further comprising:
59. 59. The optoelectronic method of 58, further comprising generating a signal responsive to the reference time estimate, wherein the signal occurs when the object crosses the setup position.
60. 59. The optoelectronic method of 58, further comprising generating a signal responsive to the reference time estimate, wherein the signal occurs when the object is upstream of the setup position.
61. Preparing a test object;
The test object is detected upstream of the reference point;
Using a man-machine interface including an indicator indicating a detection state selected from the fact that the test object has been detected downstream from the reference point and the test object has not been detected. The optoelectronic method according to 50, further comprising:
62. 51. The optoelectronic method according to any one of 48 to 50, wherein the step of analyzing at least a portion of the plurality of images includes performing pattern detection.
63. Providing an exemplary object;
Obtaining a model pattern responsive to the exemplary object; and
63. The optoelectronic method according to 62, wherein the pattern detection is responsive to the model pattern.
64. 63. The optoelectronic method according to 62, wherein performing the pattern detection comprises performing normalized correlation.
65. 51. The optoelectronic method according to any one of 48 to 50, wherein the optical sensor comprises a linear optical sensor including a one-dimensional array of light receiving elements.
66. An optoelectronic method for detecting and locating an object, comprising:
Using a linear optical sensor to perform light measurement in the field of view;
Generating a relative motion between the object and the field of view in a direction of motion such that the object passes through the field of view and crosses the reference point at a reference time;
Capturing a plurality of one-dimensional images of the field of view,
Each of the plurality of images is responsive to an individual light measurement;
The image is oriented substantially parallel to the direction of motion;
Capturing a plurality of one-dimensional images of the field of view, wherein at least a portion of the image corresponds to a plurality of positions of the object relative to the field of view;
Generating a plurality of shooting times, each shooting time responding to the time when the corresponding image was shot;
Analyzing at least a portion of the plurality of images to generate position measurements and score measurements;
Generating an estimate of the reference time using the imaging time, the position measurement, and the score measurement;
Generating a signal indicative of the reference time by generating it at a signal time in response to the estimated value of the reference time.
67. 69. The optoelectronic method of 66, wherein using the shadow time, the position measurement value, and the score measurement value to generate an estimate of the reference time comprises predicting the reference time.
68. 67. The optoelectronic system according to 66, wherein there is no waiting time between the reference time and the signal time.

Claims (38)

A computer comprising instructions for a data processing device for controlling a device for optically providing information representative of the object moving relative to the one-dimensional field of view so that the object passes through the one-dimensional field of view A non-transitory computer readable medium, wherein the instructions are transmitted to the data processing device,
Receiving data representing a plurality of images of the one-dimensional field of view substantially parallel to the direction of motion;
The image has a one-dimensional region of interest oriented substantially parallel to the direction of motion;
A computer program, wherein the plurality of images are obtained from a plurality of viewpoints with respect to the visual field, and calculate information representing the object based on at least a part of the images.   The computer program product of claim 1, wherein the information representing the object includes information responsive to the presence of an object in the field of view. The object crosses a reference point at a reference time;
The computer program according to claim 1, wherein the information representing the object includes an estimated value of the reference time.   The computer program according to claim 3, wherein the estimated value of the reference time is determined by predicting the reference time.   The computer program according to claim 3, further comprising instructions for causing the data processing device to provide a signal indicating an estimated value of the reference time.   4. A system including a processor for executing the computer program according to claim 3, wherein the computer program indicates an estimated value of the reference time by being generated at a signal time responsive to the estimated value of the reference time. The system further comprising a signal process for generating a signal.   The system of claim 6, further comprising an electrical man machine interface, wherein the signal time is adjusted in response to the man machine interface.   The system according to claim 6, wherein there is no waiting time between the reference time and the signal time. A system including a processor for executing the computer program according to claim 3,
An exemplary object;
A setup signal indicating that the exemplary object is at a setup position within the field of view, and the reference point is set in response to the exemplary object, the setup signal, and the setup position; system.   The system of claim 9, wherein the computer program further includes a signal process that generates a signal responsive to the information representing the object, the signal occurring when the object crosses the setup position. .   The computer program further includes a signal process that generates a signal responsive to the information representative of the object, the signal occurring when the object is upstream of the setup position. System. A system including a processor for executing the computer program according to claim 3,
A test object,
At least one of the test object detected upstream from the reference point, the test object detected downstream from the reference point, and the test object not detected. And a man-machine interface that includes an indicator that indicates a detection condition that includes one of the two.   A system comprising a processor for executing the computer program according to claim 3, further comprising a linear optical sensor having a one-dimensional array of light receiving elements for measuring light in the field of view. The computer program is stored in the data processing device.
Calculating an image measurement based on at least a portion of the image;
Selecting some of the image measurements calculated in response to the object as object measurements;
The computer program product according to claim 3, further comprising instructions for causing the information representing the object to be calculated, further comprising causing the information representing the object to be calculated based on the object measurement value.   15. The computer of claim 14, wherein at least one of the image measurements is responsive to the presence of an object in the field of view, and the information representing the object includes information responsive to the presence of an object in the field of view. program. The object crosses a reference point at a reference time;
At least a portion of the image measurement is responsive to a position of an object in the field of view;
The computer program is stored in the data processing device.
Selecting some of the image measurements in response to the position of the object as the object measurements;
The computer program according to claim 14, further comprising an instruction for causing the estimated value of the reference time to be calculated based on a position of the object as the information representing the object. The computer program is
A command for causing the data processing device to generate a plurality of shooting times, each shooting time responding to a time when a corresponding image was shot, a command for generating a plurality of shooting times;
The computer program according to claim 16, further comprising: an instruction for causing the data processing device to respond to at least one of the photographing times.   A command for causing the data processing device to respond to at least one of the photographing times is a command for causing the data processing device to include a plurality of the photographing times and the position of the object in the field of view. The computer program product of claim 17, comprising instructions for fitting a curve to various points including corresponding object measurements selected from:   The computer program further includes instructions for causing the data processing device to provide an electrical man-machine interface, wherein the reference point is adjusted in response to the man-machine interface. 16. The computer program according to 16.   The computer program further includes instructions for causing the data processing device to generate a signal responsive to the information representing the object, wherein the signal is generated at a signal time responsive to an estimate of the reference time. The computer program according to claim 16, wherein the information representing the object is represented by: The computer program further includes instructions for causing the data processing device to provide an electrical man-machine interface, wherein the signal time is adjusted in response to the man-machine interface. 20. The computer program according to 20.   The computer program according to claim 20, wherein there is no waiting time between the reference time and the signal time.   The command for causing the data processing device to respond to at least one of the photographing times causes the data processing device to determine an estimated value of the reference time by predicting the reference time. Computer program. An optoelectronic system for generating information representing an object,
An optical sensor that performs multiple light measurements in the field of view;
A motion process that generates relative motion between the object and the field of view in the direction of motion such that the object passes through the field of view;
An imaging process for capturing a plurality of one-dimensional images of the illuminated field of view on an imaging region, wherein the imaging of the plurality of images is performed using the optical sensor;
Each of the plurality of images is responsive to an individual light measurement;
The plurality of images are oriented substantially parallel to the direction of motion;
A photographing process in which the plurality of images are obtained from a plurality of viewpoints with respect to the field of view;
A measurement process that analyzes at least one of the plurality of images and generates a plurality of image measurements;
A selection process for selecting a plurality of object measurement values comprising at least a part of the plurality of image measurement values determined to be responsive to the object from the plurality of image measurement values; and the plurality of objects An optoelectronic system comprising: a processor configured to perform a decision process that analyzes measurements and generates information representative of the object. At least a portion of the image measurement is responsive to the presence of an object in the field of view;
The selection process is responsive to at least a portion of the image measurement responsive to the presence of an object in the field of view;
25. The optoelectronic system of claim 24, wherein the information representative of the object includes information responsive to the presence of an object in the field of view. The movement process causes the object to cross a reference point at a reference time;
At least a portion of the image measurement is responsive to a position of an object in the field of view;
At least a portion of the object measurement is selected from among image measurements responsive to the position of the object in the field of view;
The determining process is responsive to at least a portion of the plurality of object measurements selected from the plurality of image measurements responsive to a position of an object in the field of view;
The optoelectronic system according to claim 24, wherein the information representing the object includes an estimated value of the reference time.   The shooting process generates a plurality of shooting times, each shooting time responding to a time when a corresponding image was shot, and the determination process responds to at least one of the plurality of shooting times. 27. The optoelectronic system according to 26.   The determination process may be performed on various points including corresponding object measurements selected from among the plurality of image measurements in response to at least a portion of the plurality of imaging times and the position of the object in the field of view. 28. The optoelectronic system of claim 27, comprising fitting a curve.   The processor is further configured to perform a signal process that generates a signal responsive to information representative of the object, wherein the signal is generated at a signal time responsive to an estimate of the reference time, thereby causing the object 27. The optoelectronic system of claim 26, wherein the optoelectronic system indicates information representative of   30. The optoelectronic system of claim 29, wherein there is no waiting time between the reference time and the signal time.   27. The optoelectronic system of claim 26, wherein the determination process determines an estimate of the reference time by predicting the reference time. An exemplary object;
A setup signal indicating that the exemplary object is at a setup position within the field of view, and the reference point is set in response to the exemplary object, the setup signal, and the setup position; 27. The optoelectronic system according to claim 26.   33. The processor of claim 32, wherein the processor is further configured to perform a signal process that generates a signal responsive to information representative of the object, wherein the signal occurs when the object crosses the setup position. The described optoelectronic system. A test object,
The test object is detected upstream of the reference point;
And a man-machine interface including an indicator indicating a detection state selected from the fact that the test object was detected downstream from the reference point and the test object was not detected. 27. The optoelectronic system according to claim 26.   27. The optoelectronic system according to any one of claims 24 to 26, wherein the measurement process includes a pattern detection process. Further including an exemplary object;
The processor is further configured to perform a training process to obtain a model pattern responsive to the example object;
36. The optoelectronic system of claim 35, wherein the pattern detection process is responsive to the model pattern.   36. The optoelectronic system of claim 35, wherein the pattern detection process includes a normalized correlation process.   27. The optoelectronic system according to any one of claims 24 to 26, wherein the optical sensor comprises a linear optical sensor including a one-dimensional array of light receiving elements.

Images