JP2010135926A - Visual sensor synchronizing device and visual sensor synchronization method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a visual sensor synchronizing device and a visual sensor synchronization method for synchronizing the image pickup timing of a visual sensor, with the modulation timing of illumination light as a reference, without having to use private wirings or communications networks, being equipped with a special receiver for receiving synchronizing signals, or without having to require a special pixel circuit. <P>SOLUTION: The visual sensor synchronizer includes lighting device 11 to which intensity modulation is applied; an imaging element 13 for controlling the frame time; and an operation control circuit 14. The operation control circuit 14 obtains the information regarding the shift in time of the movement between the lighting device 11 and the imaging element 13 through post-processing of an image obtained by the imaging element 13 and minimizes the shift in time by the feedback control of the frame time based on the obtained information. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to timing synchronization of a visual sensor. For example, in the case of using a plurality of cameras using an image pickup device such as a CCD image sensor or a CMOS image sensor, and a plurality of visual processing systems in which an image processing device is added to them. The present invention relates to a visual sensor synchronization apparatus and a visual sensor synchronization method suitable for adjusting the timing of imaging between the two.

  When observing a moving object such as an object or a person in a non-contact manner, a video using a camera using an image sensor such as a CCD image sensor or a CMOS image sensor, and a visual processing system to which an image processing device is added. Obtaining images is widely done. These systems are collectively called visual sensors. A moving image is generally a sequence of still images taken at appropriate time intervals, and is usually taken at regular time intervals.

  As shown in FIG. 11, a plurality of visual sensors 51 are widely used when it is desired to observe an object from a plurality of directions, or when it is desired to observe a wide area of information. At this time, in order to integrate the information obtained from each visual sensor 51 and correctly obtain the position and shape of the original photographing object 50, the imaging timing of each visual sensor 51 needs to be synchronized. . This is particularly important when the movement of the subject 50 is high.

  Conventionally, in order to synchronize the imaging timing between visual sensors, it is common to perform transmission and reception of timing signals using dedicated electrical wiring. In recent years, with the spread of computer networks, a technique is also known in which the internal clock of each visual sensor is synchronized with each other by performing special communication on the network, thereby synchronizing the imaging timing (for example, Non-Patent Document 1).

  On the other hand, except for the case where a self-luminous object is used as the imaging target 50, the imaging by the visual sensor 51 always requires an illumination light source 52 as shown in FIG. A technique for detecting a relative deviation between the reference time system for intensity modulation of the illumination light source 52 and the reference time system for imaging of the visual sensor 51 by introducing a special technique on the visual sensor 51 side is known. (For example, see Non-Patent Document 2, 3 or 4).

G. Litoset al., “Synchronous Image Acquisition based on Network Synchronization”, Proceedings of Computer Vision and Pattern Recognition Workshop, 2006, p.167-172 S. Andoand A. Kimachi, “Conrrelation Image Sensor: Two-Dimensional Matched Detection of Amplitude-Modulated Light”, IEEE Transactions on Electron Devices, 2003, 50, (10), p.2059-2066 J. Ohtaet al., “An Image Sensor with an In-Pixel Demodulation Function for Detecting the Intensity of a Modulated Light Signal”, IEEE Transactions on ElectronDevices, 2003, 50, (1), p.166-172 B. Buxbaum et al., “PMD-PLL: receiver structure for incoherent communication andranging systems”, Proceedings of the SPIE, 1999, 3850, p.116-127

  In order to realize the imaging timing synchronization of the visual sensor, it is necessary to transmit and receive time information via a dedicated network or a communication network as described in Non-Patent Document 1, and it takes time and effort to install the visual sensor. There has been a problem that there is a restriction on the arrangement of the image. Although it is considered possible to perform communication based on a time synchronization protocol on a wireless communication network, there is a problem that synchronization accuracy is limited in a situation where delay variation cannot be ignored.

  As another means, it is possible to synchronize by receiving time information by Global Positioning System (GPS), receiving optical beacons, receiving beacons in a dedicated wireless communication band, etc. Therefore, it is necessary to provide dedicated devices such as a GPS receiver, a photodetector, and a wireless receiver separately from the image sensor, which causes a problem of increasing costs.

  On the other hand, if the techniques as described in Non-Patent Documents 2, 3 and 4 are used, the visual sensor itself can obtain a relative deviation from the optical image based on the intensity modulation timing of the illumination light. Yes, it can be applied to imaging timing synchronization, but each technology requires embedding a special circuit in the pixel of the vision sensor, and a system using a normal CCD image sensor or CMOS image sensor There was a problem that was not applicable.

  The present invention has been devised in view of such problems, and does not use a dedicated wiring or communication network, is not equipped with a special receiver for receiving a synchronization signal, and has a special pixel circuit. An object of the present invention is to provide a visual sensor synchronization device and a visual sensor synchronization method that can synchronize the imaging timing of a visual sensor with reference to the modulation timing of illumination light.

  In order to achieve the above object, a visual sensor synchronization device according to the present invention includes an illumination device that has been subjected to intensity modulation, an imaging device that can control a frame time, and post-processing of an image acquired by the imaging device. Computation control means that obtains information related to a time lag of operation between the apparatus and the image sensor and minimizes the time lag by feedback control of a frame time based on the obtained information is provided. .

  In the visual sensor synchronization device according to the present invention, the calculation control means includes a first time function that takes a value determined for each frame, and a second time that includes a feature amount calculated from an image of each frame. It is preferable to obtain information related to the time lag of the operation of the illumination device and the image sensor by calculating the time correlation between the functions. Further, in the visual sensor synchronization device according to the present invention, the calculation control means may use the time correlation value to minimize the time lag of the operation between the illumination device and the image sensor, based on the time correlation value. It is preferable to finely adjust the frame time.

  In the visual sensor synchronization device according to the present invention, the illumination device is intensity-modulated with a rectangular wave having a duty ratio of 1: 1, and the imaging device has a basic frame time set to a half of a modulation period of the illumination device. The arithmetic control means uses the function that takes 1 or −1 depending on the frame as the first time function that takes a value determined for each frame, thereby calculating the feature amount calculated from the image of each frame. It is preferable to obtain information related to the time lag by cumulatively adding and subtracting each frame and passing the result through a low-pass filter. Further, in the visual sensor synchronization device according to the present invention, the illumination device is intensity-modulated with a rectangular wave having a duty ratio of 1: 1, and the imaging device has a basic frame time set to a quarter of the modulation period of the illumination device. The arithmetic control means is calculated from the image of each frame by using a function that takes 1, 0, or -1 depending on the frame as the first time function that takes a value determined for each frame. The feature amount is cumulatively added / subtracted for each frame, and the result is passed through a low-pass filter to obtain information relating to the time lag, and the magnitude of the modulation amplitude is obtained. Information related to the time lag may be normalized.

  The visual sensor synchronization method according to the present invention includes an illumination device subjected to intensity modulation and an imaging device capable of controlling a frame time, and the illumination device and the imaging by post-processing of an image acquired by the imaging device. The present invention is characterized in that information related to the time lag of the operation of the element is obtained, and the time lag is minimized by feedback control of the frame time based on the obtained information.

  The visual sensor synchronization method according to the present invention is a time between a first time function that takes a value determined for each frame and a second time function that is composed of feature amounts calculated from the image of each frame. It is preferable to obtain information related to the time lag of the operation of the illumination device and the image sensor by calculating the correlation. In addition, the visual sensor synchronization method according to the present invention provides a fine adjustment of the frame time of the image sensor based on the value of the time correlation in order to minimize the time lag of the operation of the illumination device and the image sensor by feedback control. It is preferable to carry out.

  In the visual sensor synchronization method according to the present invention, the illumination device is intensity-modulated with a rectangular wave having a duty ratio of 1: 1, the basic frame time of the image sensor is set to a half of the modulation period of illumination, and is determined for each frame. By using a function that takes 1 or -1 depending on the frame as the first time function that takes the obtained value, the feature amount calculated from the image of each frame is cumulatively added and subtracted for each frame, and the result is reduced. It is preferable to obtain information related to the time difference by passing through a band-pass filter. Further, in the visual sensor synchronization method according to the present invention, the illumination device is intensity-modulated with a rectangular wave having a duty ratio of 1: 1, and the basic frame time of the image sensor is set to a quarter of the modulation period of the illumination device, The feature value calculated from the image of each frame is accumulated for each frame by using a function that takes 1, 0, or −1 depending on the frame as the first time function that takes a value determined for each frame. Addition / subtraction and passing the result through a low-pass filter obtains information related to the time shift, obtains the magnitude of the modulation amplitude, and normalizes the information related to the time shift by the magnitude of the modulation amplitude. Also good.

  The visual sensor synchronization device and the visual sensor synchronization method according to the present invention include an illumination device subjected to intensity modulation, and an image sensor that can control the frame time. Each time the image sensor acquires an image of each frame, it is preferable to calculate a feature amount from the image and obtain information related to a time lag between the operation of the lighting device and the image sensor based on the feature amount. Based on this information, the visual sensor is synchronized by performing feedback control of the frame time so as to minimize the time lag.

  In order to obtain information related to the time lag of the operation of the illumination device and the image sensor, a first time function that takes a value determined for each frame and a feature amount calculated from the image of each frame. Calculate the time correlation between two time functions. If information related to the time lag between the operation of the illumination device and the image sensor is obtained, feedback control that minimizes the time lag can be performed based on the phase-locked loop (PLL) theory.

  In particular, the intensity of the illuminator is modulated with a rectangular wave with a duty ratio of 1: 1, the basic frame time of the image sensor is set to one half of the modulation period of the illumination, and the time function taking a value determined for each frame When a value having a value of 1 or -1 is used, the time correlation can be obtained by cumulative addition / subtraction of feature amounts calculated from an image of each frame and application of a low-pass filter to the result. Synchronization is realized by increasing the frame time by a constant multiple of the value of the time correlation (decreasing if the correlation value is negative) with reference to the basic frame time.

  Here, it is necessary to appropriately set an amount to increase or decrease the frame time, that is, a gain, but an appropriate gain range depends on the amplitude of the modulated illumination light that is effectively received by the image sensor. In order to avoid instability of synchronization by operating without taking this dependency into account, the illumination device is intensity-modulated with a rectangular wave with a duty ratio of 1: 1, and the basic frame time of the image sensor is modulated with illumination. The time correlation between the second time function composed of the feature values calculated from the image of each frame and the first time function taking a value determined for each frame is set to a quarter of the period. Along with the calculation, an amount proportional to the amplitude of the second time function constituted by the feature amount calculated from the image of each frame is calculated. If information related to the time lag of the operation of the illumination device and the image sensor is obtained from the time correlation normalized by the amount proportional to the amplitude, feedback control that minimizes the time lag can be executed.

  According to the present invention, the imaging timing of the visual sensor can be illuminated without using a dedicated wiring or communication network, without a special receiver for receiving a synchronization signal, and without requiring a special pixel circuit. A visual sensor synchronization device and a visual sensor synchronization method that can be synchronized based on the modulation timing of light are obtained.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
A basic configuration of a visual sensor synchronization apparatus according to an embodiment of the present invention is shown in FIG.
As shown in FIG. 1, the visual sensor synchronization device includes a modulation illumination device 11, an optical system 12, an image sensor 13, and an arithmetic control circuit 14.

  The modulated illumination device 11 includes a light emitter 111 such as an LED (light emitting device) and a light emission control circuit 112. The intensity-modulated light emitted from the modulated illumination device 11 is irradiated onto the image sensor 13 through the optical system 12 directly or indirectly by being reflected by an object or the like, and converted into an electrical signal. The arithmetic control circuit 14 reads this electric signal, adds an appropriate calculation, and then controls the frame time of the image sensor 13 based on the calculation result. The image sensor 13 has a plurality of pixels 131.

  The arithmetic control circuit 14 constitutes an arithmetic control means, and may be a dedicated circuit that can execute the arithmetic and control described below, or may be realized as software on a microprocessor. It is necessary to select and use an image sensor 13 that can control the frame time, but it may be performed by giving an instruction to a control circuit built in the image sensor 13 and controls an external shutter trigger signal. It may be done by doing.

  As preparation for explaining the principle of the present embodiment, basic items of the PLL theory will be described. The present embodiment is based on a digital PLL (DPLL) in which input / output signals take only discrete values. The basic configuration of DPLL is shown in FIG. As shown in FIG. 2, a reference signal serving as a reference for synchronization is represented by f (t), and an output signal synchronized with f (t) is represented by g (t). As shown in FIG. 3, each of f (t) and g (t) is a rectangular function having a duty ratio of 1: 1 in which values 1 and −1 are alternately repeated. As shown in FIG. 2, the phase detector 21 calculates a product f (t) g (t) of f (t) and g (t). The low-pass filter 22 outputs a value q (φ) obtained by averaging the products. This output q (φ) is a time correlation between f (t) and g (t), and has a value depending on the phase difference φ between f (t) and g (t). FIG. 4 shows how the time correlation value q (φ) changes depending on the phase difference between f (t) and g (t).

  As shown in FIG. 2, the voltage controlled oscillator 23 is an oscillator that outputs an output signal g (t), and its oscillation cycle changes according to the time correlation value q output from the low-pass filter 22. When the time correlation value q is 0, the oscillation period operates in a predetermined basic oscillation period, and becomes longer as the time correlation value q becomes larger than 0, and becomes shorter as it becomes smaller than 0.

  As shown in FIG. 4, when the time correlation value q is positive, the phase of g (t) changes in a direction relatively delayed as the frame time becomes longer. Conversely, when it is negative, the phase of g (t) changes in a relatively advancing direction. Due to the effect of such feedback, the phase difference between f (t) and g (t) is π / 2 by appropriately selecting the gain when changing the oscillation period according to the change in the time correlation value q. At this equilibrium point, the operation of the system converges. The point where the phase difference is 3π / 2 is also an equilibrium point, but this equilibrium point is unstable and does not remain at this operating point in the actual situation where disturbances exist.

  In the present embodiment, the light emitted from the illumination device 11 and irradiated onto the image sensor 13 corresponds to the reference signal f (t). The light intensity irradiated to each pixel 131 in the image sensor 13 is usually not uniform, but in the following, for the sake of simplicity of discussion, the light intensity irradiated to each pixel 131 is one unless otherwise specified. Suppose that Therefore, in the following, the light intensity applied to the image sensor 13, the light intensity applied to each pixel 131, and the sum of the light intensity applied to the pixels 131 over all pixels are identified with f (t). There is.

ここで、Tは、時間平均を取る区間を表し、撮像素子のフレーム時間より十分に長いとする。 In the first aspect of the present embodiment, the output signal g (t) corresponds to a first time function that takes a value determined for each frame of the image sensor 13, and the reference signal f (t) This corresponds to a second time function composed of feature amounts calculated from the images. The time correlation q (φ) between f (t) and g (t), which is the output of the low-pass filter 22, is expressed as follows.

Here, T represents a section in which a time average is taken, and is assumed to be sufficiently longer than the frame time of the image sensor.

  In the above description of DPLL, f (t) has a value of 1 or -1. However, the intensity of the intensity-modulated light cannot take a negative value. Instead, as shown in FIG. 5, f (t) is a rectangular wave that takes a value of 0 or 1. That is, f (t) is expressed as f (t) = f ′ (t) / 2 + 1/2 where f ′ (t) is a rectangular wave having a value of 1 or −1.

f (t) = 1 corresponds to the case where the lighting device 11 is turned on, and f (t) = 0 corresponds to the case where it is turned off. Here, paying attention to the fact that the duty ratio of g (t) is 1: 1, the aforementioned DPLL operates in exactly the same manner. This is because the following holds.

  As shown in FIG. 5, the output function g (t) is assumed to take g (t) = 1 during odd frames and g (t) = − 1 during even frames.

The arithmetic control circuit 14 controls the operation of the image sensor 13 using the time correlation value of the formula [1]. However, the integrand f (t) g (t) here cannot be obtained directly. This is because the luminance value output from each pixel 131 of the image sensor 13 is an integral of the irradiated light intensity, that is, f (t) over one frame time, and is not the value of f (t) at an arbitrary moment. It is. On the other hand, if g (t) takes a fixed value within each frame period, the integral value of Equation 1 can be obtained as follows.

  Here, i is a sequential number given to each frame of the image sensor 13 in time order, and F [i] is a sum of the luminance values obtained by the pixels 131 in the frame i for all the pixels. The sum of the formula [3] is calculated over the number of frames corresponding to the interval T for which the time average is taken. Instead of directly executing this time averaging operation, it may be implemented as a recursive low-pass filter, or another more advanced low-pass filter may be used.

  The arithmetic control circuit 14 controls the frame time of the image sensor 13 based on the time correlation value q. Here, the center frequency of g (t) is set to be substantially the same as the frequency of f (t). That is, when the time correlation value q is 0, the frame time is set to ½ of the illumination modulation period, the frame time is lengthened as the time correlation value q is increased, and the frame time is shortened as it is decreased. By setting the gain at the time of changing the frame time according to the change of the time correlation value q to an appropriate value, the imaging timing of the imaging device 13 has the same period of f (t) and g (t). Yes, it converges to a point where the phase difference is π / 2. In other words, the frame time is ½ of the illumination modulation period, and the center time of each frame converges to a point where it matches the lighting on / off switching time.

  Until now, it has been described that the light intensity applied to each pixel 131 of the image sensor is uniform and can be identified with the modulated illumination light itself from the illumination device 11. The validity of this will be discussed in the light of a more practical situation.

  In general, light emitted from various light sources including the illumination device 11 is reflected by the surface of an object in the visual field of the visual sensor, and the reflected light is applied to the image sensor 13 through the optical system 12. The sum of luminance values obtained by the respective pixels 131 of the image sensor 13 over all the pixels is expressed as a sum of a component originating from the illumination device 11 and a component originating from another light source. Here, it can be considered that the former is proportional to the luminance of the light emitter 111 of the lighting device 11 and the latter is not dependent on the luminance of the light emitter 111. Assuming that the reflectance distribution of the object surface in the field of view does not change significantly between adjacent frames, the components originating from other light sources cancel out the odd frames and even frames, and the time correlation Does not contribute to the value q.

  To summarize the above discussion, only the component originating from the lighting device 11 among the luminance values obtained by each pixel 131 contributes to the time correlation value q, and the sum of all the components of all the pixels is the light emitter 111. Proportional to the brightness of Therefore, as described above, the sum of the luminance values obtained by each pixel 131 may be identified with the reference signal f (t).

  By the way, when the reflectance distribution of the object surface in the field of view changes greatly between adjacent frames, the above discussion is not valid. Such a situation is unlikely to occur under high-speed shooting conditions in which the frequency of f (t) is sufficiently high, that is, shooting at a high frame rate. Even when the frame rate is not sufficiently high, such a situation can be avoided by taking the sum of the luminance values over the other pixels excluding the region where the reflectance changes rapidly from the visual field. In this way, an appropriate pixel is selected or weighted, and the sum of luminance values of each pixel is calculated, which is called a feature amount calculated from an image.

  Heretofore, each pixel 131 of the image sensor 13 has been discussed as integrating the intensity of light applied over the entire frame time. In practice, this is not always the case, and the light intensity irradiated over a part of the frame time may be integrated by a mechanical or electronic shutter. The relationship between f (t) and g (t) in this case is shown in FIG. FIG. 7 shows how the time correlation value q (φ) changes due to the phase difference φ between f (t) and g (t) shown in FIG. Here, the phase difference φ is defined as a phase difference of 0 when the center time of the frame period in which g (t) = 1 matches the center time of the period in which f (t) = 1. ) Is represented by a phase difference of 2π. As shown in FIG. 7, in this case as well, only the point where the phase difference φ between f (t) and g (t) is π / 2 is the stable equilibrium point, and it can be seen that the operation is the same. Therefore, even when the integration operation is performed only during a part of the frame time, there is no problem in the operation.

  In the discussion so far, it was stated that the gain when changing the frame time according to the change of the time correlation value q needs to be set to an appropriate value. Here, it should be noted that the appropriate gain range depends on the amplitude of the modulated illumination component originating from the illumination device 11 in the light irradiated to the image sensor 13. Even if the phase difference between f (t) and g (t) is the same, if the amplitude is large, a relatively large change occurs in the frame time, which may make the operation unstable. In order to avoid this, it is necessary to reduce the gain so as to cancel the effect of the amplitude.

  However, the amplitude of the modulated illumination component irradiated to the image sensor 13 depends on the reflectance distribution in the field of view, the arrangement of the illumination device 11, and the like, so it is difficult to estimate in advance and can change with time. In such a case, it is difficult to set the gain in advance, and thus it is difficult to perform stable operation. In order to cope with the case where this becomes a problem, a means for estimating the modulation amplitude online and normalizing the gain by the value is required.

In the second aspect of the present embodiment that realizes this normalization, the intensity of the illumination device 11 is modulated with a rectangular wave having a duty ratio of 1: 1, and the basic frame time of the image sensor 13 is set to 4 minutes of the modulation period of the illumination device 11. 1 of As a first time function that takes a value determined for each frame of the image sensor 13, as shown in FIG. 8, in a frame in which the remainder when dividing the sequential number given to each frame in time order by 4 is 0 g ₁ and (t) = 1, 2 in the frame g ₁ (t) = -1,1 or g ₁ (t) = 0 and becomes function g ₁ is 3 frames (t), similarly the sequential number 4 divided by g ₂ in the remainder is 1 frame time (t) = a 1, 3 of the frame g ₂ (t) = -1, 0, or the second frame g ₂ (t) = 0 and becomes a function g ₂ ( Consider two of t).

For each of g ₁ (t) and g ₂ (t), the time correlation with f (t) is calculated in the same manner as in the first aspect of the present embodiment. The time correlation between f (t) and g ₁ (t) is represented as q ₁ (φ), and the time correlation between f (t) and g ₂ (t) is represented as q ₂ (φ). Assuming that the light intensity is integrated over the entire frame time of the image sensor 13, q ₁ (φ) and q ₂ (φ according to the phase difference φ between f (t) and g ₁ (t). FIG. 9 shows how each of these changes. Here, the phase difference φ is a phase difference of 0 when the center time of the frame period in which g ₁ (t) = 1 coincides with the center time of the period in which f (t) = 1, and these are expressed as f (t The case where it is shifted by one period of t) is expressed as a phase difference 2π.

As shown in FIG. 9, it is understood that the amplitude of the modulated illumination component can be estimated by max (| q ₁ (φ) |, | q ₂ (φ) |) regardless of the current phase difference φ value. . Therefore, similarly to the first aspect of the present embodiment, it is possible to change the frame time according to the value of q ₁ , normalize the gain at this time with the amplitude estimated as described above, and perform stable operation. Become. This normalization term may be subjected to post-processing such as time smoothing as necessary.

The second aspect of the present embodiment described above operates in the same manner even when the light intensity is integrated only during a part of the frame time of the image sensor 13. An example of changes in q ₁ (φ) and q ₂ (φ) in this case is shown in FIG. As shown in FIG. 10, the point where the phase difference φ is still π / 2 is the stable equilibrium point, and the amplitude of the modulated illumination component can be estimated in the same manner.

It is a block diagram which shows the visual sensor synchronizing device of embodiment of this invention. It is a block diagram which shows the basic composition of a digital phase locked loop (DPLL). 3 is a graph showing a reference signal f (t) and an output signal g (t) of the digital phase locked loop shown in FIG. 3 is a graph showing a time correlation value q (φ) between a reference signal f (t) and an output signal g (t) of the digital phase locked loop shown in FIG. It is a graph which shows the reference signal f (t) and output signal g (t) of the modulation | alteration illumination apparatus of the visual sensor synchronizer shown in FIG. 3 is a graph showing a reference signal f (t) and an output signal g (t) when an integration operation is performed only during a part of the frame time of the visual sensor synchronization device shown in FIG. 7 is a graph showing a time correlation value q (φ) between a reference signal f (t) and an output signal g (t) of the visual sensor synchronization device shown in FIG. 6. 4 is a graph showing a reference signal f (t), an output signal g ₁ (t), and an output signal g ₂ (t) when considering normalization of the gain of the visual sensor synchronization device shown in FIG. The time correlation value q ₁ (φ) between the reference signal f (t) and the output signal g ₁ (t) and the reference signal f (t) and the output signal g ₂ (t) of the visual sensor synchronization device shown in FIG. Is a graph showing the time correlation value q ₂ (φ). When considering normalization of the gain of the visual sensor synchronizer shown in FIG. 1, between the reference signal f (t) and the output signal g ₁ (t) when the integration operation is performed only in a part of the frame time. time correlation value q ₁ (phi) of a graph showing the time correlation value q ₂ (phi) between the reference signal f (t) · output signals g ₂ (t). It is a block diagram which shows the synchronization system of the conventional visual sensor.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 Illuminating device 111 Light-emitting body 112 Issue control circuit 12 Optical system 13 Image pick-up element 131 Pixel 14 Computation control circuit 21 Phase detector 22 Low-pass filter 23 Voltage-controlled oscillator

Claims (10)

An illumination device with intensity modulation;
An image sensor capable of controlling the frame time;
Information related to the time lag of the operation of the illumination device and the image sensor is obtained by post-processing of the image acquired by the image sensor, and the time lag is minimized by feedback control of the frame time based on the obtained information. Arithmetic control means for
A visual sensor synchronizer comprising:   The arithmetic control unit calculates a time correlation between a first time function taking a value determined for each frame and a second time function composed of feature amounts calculated from an image of each frame. The visual sensor synchronizing device according to claim 1, wherein information related to a time lag in operation between the illumination device and the imaging device is obtained.   The arithmetic control means performs a fine adjustment of the frame time of the image sensor according to the value of the time correlation in order to minimize the time lag of the operation between the illumination device and the image sensor by feedback control. The visual sensor synchronization apparatus according to claim 2. The illumination device is intensity-modulated with a rectangular wave with a duty ratio of 1: 1,
The imaging device has a basic frame time of one half of the modulation period of the lighting device,
The arithmetic and control means uses the function that takes 1 or −1 depending on the frame as the first time function that takes a value determined for each frame, and thereby calculates the feature amount calculated from the image of each frame. Accumulating and subtracting every time, and obtaining the information related to the time lag by passing the result through a low-pass filter,
The visual sensor synchronizing device according to claim 2 or 3, The illumination device is intensity-modulated with a rectangular wave with a duty ratio of 1: 1,
The imaging device has a basic frame time of a quarter of the modulation period of the lighting device,
The calculation control means uses the function that takes 1, 0, or −1 depending on the frame as the first time function that takes a value determined for each frame, thereby calculating the feature calculated from the image of each frame. The amount is cumulatively added / subtracted for each frame, and the result is passed through a low-pass filter to obtain information on the time lag, and the magnitude of the modulation amplitude is obtained. Normalizing such information,
The visual sensor synchronizing device according to claim 2 or 3,   An illumination device that has been subjected to intensity modulation and an image sensor that can control the frame time, and information relating to the time lag between the operation of the illumination device and the image sensor by post-processing of an image acquired by the image sensor. A visual sensor synchronization method characterized in that the time lag is minimized by frame time feedback control based on the obtained information.   By calculating a time correlation between a first time function taking a value determined for each frame and a second time function composed of feature amounts calculated from an image of each frame, the lighting device The method according to claim 6, further comprising: obtaining information related to a time lag of operation between the image sensor and the image sensor.   8. The frame time of the image sensor is finely adjusted according to the value of the time correlation in order to minimize a time lag of operation between the illumination device and the image sensor by feedback control. Visual sensor synchronization method.   The illumination apparatus is intensity-modulated with a rectangular wave having a duty ratio of 1: 1, the basic frame time of the image sensor is set to a half of the modulation period of illumination, and the first time taking a value determined for each frame By using a function that takes 1 or -1 depending on the frame as a function, the feature amount calculated from the image of each frame is cumulatively added and subtracted for each frame, and the result is passed through a low-pass filter to thereby shift the time difference. 9. The visual sensor synchronization method according to claim 7 or 8, characterized in that information relating to (1) is obtained. The intensity of the illuminating device is modulated with a rectangular wave having a duty ratio of 1: 1, the basic frame time of the imaging device is set to a quarter of the modulation period of the illuminating device, and the first value taking a value determined for each frame is taken. By using a function that takes 1, 0, or -1 depending on the frame as the time function of the above, the feature amount calculated from the image of each frame is cumulatively added and subtracted for each frame, and the result is passed through a low-pass filter. The information related to the time lag is obtained, the magnitude of the modulation amplitude is obtained, and the information related to the time lag is normalized by the magnitude of the modulation amplitude. Visual sensor synchronization method.

Images