JPH0624030B2 - Image analysis device - Google Patents

Description

The present invention relates to an image analysis device, and more particularly to a moving direction, moving distance and moving speed of a moving image from an image signal converted into a time series signal by two-dimensional scanning. The present invention relates to an image analysis device that analyzes motion vectors and the like.

[Outline of Invention]

The present invention extracts only a differential image signal between two temporally adjacent images from an image signal converted into a time-series signal by two-dimensional scanning, and detects a moving direction, a moving distance and a moving speed of a moving image, The present invention relates to a device for detecting a motion vector, etc., and when a difference image is two-dimensionally recorded in a spatial light modulator and the spatial light modulator is irradiated with readout light, the difference image is recorded in the spatial light modulator. The differential image signal of the series is converted into a two-dimensional differential optical image signal and taken out, and the differential optical image signal is two-dimensionally photoelectrically converted by a two-dimensional optical position detector to detect the barycentric coordinates of the differential image. This is the basic operation, and the moving direction, moving vector and moving speed, moving vector, etc. of the moving image can be obtained by continuously performing these series of operations.

[Conventional technology]

Conventionally, as this type of image analysis device, (A) a method for obtaining a deviation that minimizes a signal level difference between two images that are temporally consecutive (B) A method that finds the deviation that maximizes the image cross-correlation function is common, and all of these are obtained by numerically calculating a large number of sample values using a dedicated computing device that uses a computer or electronic circuit. (For example, Takahiko Fukibuki "Digital Signal Processing of Images" Nikkan Kogyo Shimbun P.221
~ 227).

[Problems to be Solved by the Invention] Image analysis methods and problems used in these apparatuses will be briefly described below.

In method (A), the relative position of two images that are temporally consecutive is slightly shifted, and the sum of the squares of the level differences between the sample values of both images is calculated, and the value is minimized. A method is used to determine the moving direction and moving distance of the moving image from the position. In other words, to represent the sample values of the coordinates (x, y) at time t in gt (x, y), each other at time t _1, t ₂ (zeta,
The sum of squares of the level difference between the sample values of the two images displaced by η) is ΣΣ {g _t2 (x-ζ, y-η) -g _t1 (x, y)} ² and ( In the method of A), the deviation (ζ,
The motion of the image is analyzed by obtaining η).

In the image analysis device using this method, (ζ, η) is given and the above calculation is repeated. Therefore, if the number of pixels increases, the amount of calculation becomes enormous, making it difficult to perform high-speed image analysis, and The group only moves in parallel in the horizontal direction or the vertical direction, which makes it difficult to analyze when the image is rotated or cycloidally moved.

On the other hand, in the (B) method, the cross-correlation function is given by ∬ g _t1 (x, y) g _t2 (x-ζ, y-η) dxdy, and the above function is maximized by changing the deviation (ζ, η). Since the position indicating the value is obtained, it has the same problem as the method (A).

The problems of the conventional techniques described above are caused by the fact that an image, which is originally two-dimensional information, is decomposed into pixels and the time-series processing is performed on the decomposed pixels.

Therefore, an object of the present invention is to reconstruct a two-dimensional differential image from image signals of two consecutive frames of a moving image signal represented by a time-series signal, without performing time-series processing, in order to solve the above problems. The recorded two-dimensional differential optical image signal is organized and recorded, and the moving direction, moving distance, moving speed, motion vector, etc. of the moving image are obtained at high speed by directly processing the recorded two-dimensional differential optical image signal using a light beam. Accordingly, it is an object of the present invention to provide an image analysis device capable of analyzing the movement of an image.

[Means for solving problems]

In order to achieve such an object, the present invention provides a means for forming a differential image signal of two temporally sequential images from an image signal converted into a time series signal by scanning, and a synchronization signal for the differential image signal. Forming means, means for forming a trigger signal that determines the timing of detecting the center of gravity of the difference image corresponding to the difference image signal, and a spatial light modulator that is a component of the spatial light modulating means at the timing of the synchronizing signal Signal writing means for converting into a recordable write signal, a light source for generating a light beam, a differential image signal is sequentially recorded and held on a two-dimensional plane by the write signal, and the light beam is used as a read light signal in the space. The time-series difference image signal is displayed in a two-dimensional space by guiding the light to the optical modulator and simultaneously modulating the readout light with the sequentially recorded difference image signal. A spatial light modulator for converting into an image, and a two-dimensional light position for photoelectrically converting the differential light image to generate a plurality of photocurrents, and obtaining barycentric coordinates of the differential light image from a plurality of photocurrent intensities and their generation positions. A detection means and a calculation means for obtaining at least one of the center of gravity of the differential optical image, the moving direction of the center of gravity, the moving distance and moving speed, and the motion vector based on a plurality of photocurrents. To do.

[Action]

In the present invention, the movement of the image is analyzed as described above, but here, as the input signal, not only a television image but also a communication image of a facsimile, an artificial satellite image, a sensing image of a traffic monitor or a robot, a gastric camera. It is possible to receive various input image signals such as medical images from the above, and it can be applied to moving image processing for these input image signals.

〔Example〕

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

The present invention will be described below with reference to an embodiment in which a television image is used as an input signal.

FIG. 1 shows a block diagram of a first embodiment of a moving image analysis apparatus according to the present invention.

In FIG. 1, reference numeral 1 forms a differential image signal 7 from a television signal 6 of two television images temporally preceding and following each other, a synchronizing signal 8 thereof, and a trigger signal 9 for determining the timing of detecting the center of gravity of the differential image. It is a signal processing unit. Two
Is a signal writing unit that converts the time-series difference image signal 7 into two-dimensional writing light 10 suitable for the spatial light modulator 4 at the timing of the synchronization signal 8. 3 is a two-dimensional difference image recorded in the spatial light modulator 4 by the two-dimensional writing light 10.
A light source unit for generating a reading light 11 for conversion into a two-dimensional differential optical image signal 12. Reference numeral 5 denotes a signal detection unit that converts the differential optical image signal 12 into an electric signal and obtains the moving direction, moving distance, moving speed, motion vector, etc. of the center of gravity of the differential image.

The signal processing unit 1, for example, as shown in FIG. 2, a delay line 13 for delaying a television signal by one frame, and an absolute value of a level difference between the television signal delayed by one frame and the television signal without delay. At the time when the differential image forming circuit 14 for forming the differential image is formed, the synchronizing signal is separated from the input television signal, and the differential image is recorded in the spatial light modulator 4 for one frame at the timing of the synchronizing signal. It has a synchronous control circuit 15 which sends out a trigger signal. In response to this trigger signal, the light source unit 3 and the signal detection unit 5 start detecting the barycentric coordinates of the difference image.

The signal writing unit 2 can be configured by a display device such as a television receiver, receives the differential image signal 7 and the synchronization signal 8 from the signal processing unit 1, displays the differential image on a display surface such as a cathode ray tube, and Display image, even fluorescent image writing light 10
To the spatial light modulator 4.

The light source section 3 receives the trigger signal 9 from the signal processing section 1 and generates a two-dimensional optical pulse having a spatially uniform intensity distribution as the reading light 11. However, the pulse width of the optical pulse is shorter than the vertical blanking period of the television signal. The spectrum of the optical pulse needs to be determined in consideration of the spectral characteristics of the spatial light modulator, but it is desirable that the spectrum width is narrow in the visible region.

The spatial light modulator 4 is composed of, for example, as shown in FIG. 3, optical components such as a spatial light modulator 16, lenses 17A and 17B, polarizing plates 18A and 18B, and a beam splitter 19. The spatial light modulator 16 and the polarizing plate 18B and the lens 17B on the output side are arranged to face each other, and the beam splitter 19 is arranged in the facing space. This beam splitter 19 has a reading light 11
Can be made incident on the spatial light modulator 16 via the lens 17A and the polarizing plate 18A, and the two-dimensional difference image recorded in the spatial light modulator 16 is read by the reading light 11. The read output light is taken out from the beam splitter 19 via the polarizing plate 18B and the lens 17B as a differential optical image signal 12 which is two-dimensionally distributed.

The spatial light modulator 16 is, for example, as shown in FIG. 3, a photoconductive layer 20 made of a photoconductive material that two-dimensionally records and holds writing light, and an electro-optical material that controls the phase and polarization direction of light. Comprising an electro-optical layer 21 and an intermediate thin film layer 22 for blocking the writing light 10 and reflecting the reading light 11 interposed between these two layers 20 and 21, and further comprising a photoconductive layer 20 and A transparent electrode 23 is provided on each outer surface of the electro-optical layer 21.

The spatial light modulator 16 (I) modulates the read light intensity according to the write light intensity (II) The time required to write at least one frame of the differential image signal into the spatial light modulator 16 (for example, NTSC (3) In the case of a TV signal, the write signal is recorded and held for 1/30 seconds or more. (III) The write signal is erased in a time shorter than the vertical blanking period. Instead of the spatial light modulator 16 in the form, various spatial light modulators such as a liquid crystal display can be used. These displays are described in detail in "Liquid Crystal Application" edited by Mitsuharu Okano and Shunsuke Kobayashi (Baifukan), and their explanations are omitted here.

For example, as shown in FIG. 4, the signal detector 5 receives the two-dimensional differential optical image signal 12 emitted from the spatial light modulator 4 and, according to the intensity of the optical signal 12, the barycentric coordinates of the differential image. -Dimensional optical position detector that generates multiple photocurrents corresponding to
24, and an arithmetic circuit for obtaining the moving direction, moving distance and moving speed of the center of gravity of the difference image, the moving vector, etc. from the plurality of photocurrent intensities obtained from the detector 24 and their generation positions.
It consists of 25 and. As the two-dimensional optical position detector 24,
For example, “Improvement of a semiconductor two-dimensional optical position detector” by Yoshitaka Terada and Akinaga Yamamoto (Optics, Vol. 12, No. 5, (1983) pp. 367-3.
Those described in detail in 73) can be used.

Next, as an example, as shown in FIGS. 5A to 5D, only high-luminance (white) squares with uniform brightness move horizontally, and low-luminance (black) background The operation of the present invention will be described by taking a series of still television images as an example.

When the television signals of FIGS. 5 (a) to 5 (d) are input to the signal processing unit 1, the signal processing unit 1 obtains the absolute value of the level difference between successive images. As a result, the differential image signal 7 as shown in FIGS. 5 (e) to 5 (g) is formed. That is,
5 (e), (f) and (g) are respectively shown in FIG. 5 (a) and
FIG. 5B shows the absolute value of the difference signal between the images in FIGS. 5B and 5C and FIGS. 5C and 5D. When the series of differential image signals 7 are displayed on the display surface of the display device in the signal writing section 2, the differential image signals 7 shown in FIGS. 5 (e) to 5 (g) are recorded in the spatial light modulator 16. can do.

At the time when the two-dimensional recording of one differential image signal 7 in the spatial light modulator 16 is completed, a two-dimensional optical pulse is sent from the light source unit 3 to the spatial light modulator 16 as the reading light 11, where:
In the signal writing section 2, the differential image signal 7 displayed on the display by scanning as a time series signal is converted into a two-dimensional differential optical image signal 12. This differential optical image signal 12
Is sent to the two-dimensional optical position detector 24 of the signal detection unit 5, where it is converted into a plurality of photocurrents corresponding to the barycentric coordinates of the difference image. Based on these photocurrents, the arithmetic circuit 25 can determine the center of gravity of the difference image.

If the center of gravity is calculated for each of the differential images shown in FIGS. 5 (e) to 5 (g), the moving distance, moving direction, moving speed, motion vector, etc. of the moving image can be calculated. For example, the barycentric coordinates of two consecutive difference images are (x _i , y _i ),
Assuming that (x _{i + 1} , y _{i + 1} ), the moving distance l of the moving image is given by the following equation.

l = {(x _i -x _{i + 1} ) ² + (y _i -y _{i + 1} ) ² } ^1/2 (1) Further, when the frame period is τ, the moving speed v is v = 1 / τ (2 ) Is given. Further, the moving direction is represented by an angle θ = tan ⁻¹ ｜ (y _i -y _{i + 1} ) / (x _i -x _{i + 1} ) | (3) formed by the straight line passing through the two barycentric coordinates and the x-axis. Be done. The motion vector of the moving image is determined from l and θ.

As shown in FIG. 5, when the image moves linearly,
The motion vector of the difference image is equal to the motion vector of the actual moving image. By the way, the motion of an image is generally curved, and when the motion is fast, the motion vector of the differential image and the motion vector of the moving image do not match. But,
Even in this case, the curvilinear motion can be approximated by a straight line by reducing the period of the light pulse (corresponding to τ in the equation (2)), and the motion vector of the difference image and the motion vector of the moving image can be obtained. Can be almost matched.

Although the embodiment of the present invention has been described above, as the delay circuit 13 of the signal processing section 1, a frame memory or a field memory can be used in addition to the one-frame delay line illustrated in FIG. Further, as the signal writing unit 2, various display devices such as a liquid crystal display, an electroluminescence display, a plasma display can be used in addition to the television receiver including the cathode ray tube illustrated here.

A block diagram of another embodiment of the present invention is shown in FIG. Here, the configurations and operations of the signal processing unit 1, the light source unit 3 and the signal detection unit 5 are the same as those of the embodiment shown in FIG. 1, but in this example, a transmissive type instead of the spatial light modulator 16 is used. The difference from the above embodiment is that the liquid crystal display 26 is used and the differential image signal 7 is recorded and held in the liquid crystal display 26.

The signal writing unit 2 uses the difference image signal 7 from the signal processing unit 1.
And a sync signal 8 and a liquid crystal drive circuit for forming a write signal 10 'for driving the liquid crystal display 26.

As shown in FIG. 6, the spatial light modulator 4 includes lenses 17A and 17A.
B and 17c, polarizing plates 18A and 18B, and a transmissive liquid crystal display 26. Polarizing plate 18 with the liquid crystal display 26 in between
A and 18B are arranged so as to face each other, and the read light 11 from the light source unit 3, for example, a He-Ne laser is converted into light having a two-dimensional spread through the lenses 17C and 17A and is made incident on the liquid crystal display 26. The transmitted light from the liquid crystal display 26 is sent to the signal detection unit 5 as the differential optical image signal 12 via the lens 17B.

As the liquid crystal display 26, there are a simple matrix drive type and an active matrix drive type, but the latter has advantages such as high image contrast and long write signal holding time, and thus is particularly useful as a spatial light modulator in the present invention. Are suitable.

In the embodiment shown in FIG. 1 or FIG. 6, the trigger signal 9 is sent to the light source unit 3, but as shown by the dotted line in the figure, the trigger signal 9 is sent to the signal detection unit 5 and the difference image At the time when is completed, the photocurrent from the two-dimensional position detector 24 is arithmetically operated at the timing of the trigger signal 9.
It is also possible to detect the center of gravity of the difference image by importing it into 25.

With this configuration, the light source unit 3 need only have the function of generating continuous light, and the configuration of the light source unit 3 can be simplified.

Next, in order to further clarify the function of the present invention, an experimental example in which a moving image is analyzed by using the image analysis device having the basic configuration shown in FIG. 6 will be shown. The configuration of the image analysis device used in that case is shown in FIG.

In FIG. 7, the signal processing unit 1 is composed of a personal computer 27, and this computer 27 has a function of forming a pseudo differential image signal 7'having a uniform brightness as shown in FIGS. 5 (e) to 5 (g). have. This personal computer 27
Also has a function of sending the trigger signal 9 to the signal detector 5. In the signal writing unit 2, the signal conversion circuit 28 has a function of converting the signal 7'from the personal computer 27 into an NTSC television signal. Further, the liquid crystal display drive circuit 29 forms a drive signal for displaying the differential image signal converted into the NTSC television signal on the liquid crystal display 26, that is, a write signal 10 '. The He-Ne laser 30 constitutes the light source unit 3 and has a function of sending laser light (wavelength 633 mm) having a spatially substantially uniform intensity distribution to the liquid crystal display 26. Here, the liquid crystal display 26 adopts an active matrix drive system and corresponds to the spatial light modulator 16.

The locus of the center of gravity (broken line) of the differential image 7 calculated in advance by the personal computer 27 when the rectangular pseudo differential image signal 7 created by the personal computer 27 is continuously moved on the circumference, and the device of FIG. When the locus of the center of gravity (solid line) actually measured is compared and shown in FIG.
In this case, the difference image occupies about 2.
It was 3%. The position of the differential image signal 7 created by the personal computer 27 changes every frame,
The moving distance and moving direction of the difference image every 1/30 second are represented as a motion vector on a broken line and a solid line. The two are almost the same, which shows that the apparatus according to the present invention can analyze a moving image at high speed and with high accuracy, although the apparatus has a simple structure.

Further, the light source unit 3, the spatial light modulator 4 and the two-dimensional light position detector 24 shown in FIG. 7 can be integrated to make the apparatus of the present invention smaller and more robust. An example thereof is shown in FIG. In the figure, 31 is a surface emitting light source (for example, a surface emitting cell utilizing plasma or electroluminescence), and 32 is a lens group formed into a two-dimensional array. A lens group 32 and a two-dimensional optical position detector 24 are arranged on both main surfaces of a transmissive liquid crystal display 26 via polarizing plates 18A and 18B. A surface emitting light source 31 is brought into close contact with the lens group 32, and two-dimensional light from the light source 31 is guided from the lens group 32 to the display 26 via the polarizing plate 18A. The light transmitted through the display 26 is guided to the detector 24 via the polarizing plate 18B.

The tenth example of the microlenses forming the lens array 32
Shown in the figure. This micro lens is a focusing rod lens 34
And its side surface is covered with a black resin or the like. The reason is that light having an incident angle larger than the maximum light receiving angle determined by the numerical aperture of the lens is emitted from the surface emitting cell 31 to the lens.
This is because when the light enters the lens 34, it is absorbed by the side surface of the lens and the deterioration of the contrast of the liquid crystal display 26, that is, the position detection accuracy is prevented.

The measurement accuracy of the embodiment shown in FIGS. 1, 6, and 7 mainly depends on the extinction ratio (the ratio between the maximum value and the minimum value of the output light) of the spatial light modulator 16 and the signal holding characteristic. . When the extinction ratio is small, the light emitted from the spatial light modulator 16 includes a large amount of background light corresponding to noise in addition to the differential optical image signal 12, and it is difficult to accurately detect the barycentric coordinates. become.

Therefore, in this example, the intensity of the background light in the absence of the differential light image signal 12 and its barycentric coordinate are obtained in advance, and then the intensity of the light beam having the differential light image signal 12 and the background light and its barycenter are obtained. This problem is solved by performing arithmetic processing for obtaining coordinates. That is, when the intensity of the differential optical image signal 12 is I _M , its barycentric coordinate is (x _M , y _M ), and the background light intensity is I _o , and its barycentric coordinate is (x _o , y _o ), the difference is The barycentric coordinates (x,
y) is Given in. The above equation can be rewritten as the following equation.

Therefore, I _o and (x _o , y _o ), and I _o + I _M (total intensity of the differential light image signal and background light) and the barycentric coordinates of the total light beam described above.
If (x, y) is measured, the barycentric coordinates (x _M , y
_M ) can be obtained. The experimental results shown in FIG. 8 are
The correction represented by the equations (6) and (7) is performed, and it is shown that this method is effective.

Next, the signal holding characteristic of the spatial light modulator 16 will be described.
When the signal holding time is short, brightness changes (shading) occur mainly in the vertical direction of the image. Such shading causes an error in the y-coordinate of the center of gravity of the moving image, but a light attenuating plate having a non-uniform transmittance distribution in which the transmittance of the upper portion is larger than that of the lower portion is provided between the light source unit 3 and the signal detecting unit 5. By inserting it, this error can be reduced.

〔The invention's effect〕

The present invention has the following effects as compared with the conventional moving image analysis apparatus.

(1) In the conventional device, when the number of pixels increases, the amount of calculation becomes enormous and high-speed processing becomes difficult. However, in the present invention, the calculation time for obtaining the barycentric coordinates of the difference image is the response time of the two-dimensional optical position detector. High-speed processing is possible because it depends only on. For example, in the image analysis device shown in FIG. 7, 2.5 × 1
A movement of an image consisting of 0 ⁷ pixels can be obtained at about 20 .mu.sec.

(2) In the conventional device, when the image is rotating at a high speed, it is difficult to analyze the movement, but in the present invention, when the center of gravity and the center of rotation of the differential optical image do not match, the direction of rotation or The rotation speed can be calculated.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration example of the image analysis apparatus of the present invention, FIG. 2 is a block diagram showing a configuration example of a signal processing unit in the image analysis apparatus of the present invention, and FIG. 3 is a diagram of the image analysis apparatus of the present invention. FIG. 4 is a block diagram showing a configuration example of a spatial light modulation unit, FIG. 4 is a block diagram showing a configuration example of a signal detection unit in the image analysis apparatus of the present invention, and FIG. 5 is a television for explaining the operation of FIG. 6 is a block diagram showing another configuration example of the image analysis device of the present invention, and FIG. 7 is a configuration example of a prototype image analysis device based on the configuration of FIG. FIG. 8 is a block diagram showing the calculation result (broken line) of the motion vector of the difference image and an example of an experimental result (practice), FIG. 9 is a sectional view showing a modification of the present invention, and FIG. FIG. 10 is a perspective view showing an example of the microlens shown in FIG. 9. 1 ... Signal processing unit, 2 ... Signal writing unit, 3 ... Light source unit, 4 ... Spatial light modulation unit, 5 ... Signal detection unit, 6 ... TV signal, 7 ... Differential image signal, 8 ... … Synchronization signal, 9 …… Trigger signal, 10 …… Write light, 10´ …… Write signal, 11 …… Read light, 12 …… Differential optical image signal, 13 …… 1 frame delay line, 14 …… Differential image Forming circuit, 15 …… Synchronous control circuit, 16 …… Spatial light modulator, 17A, 17B, 17C …… Lens, 18A, 18B …… Polarizing plate, 19 …… Beam splitter, 20 …… Photoconductive layer, 21… … Electro-optical layer, 22 …… Intermediate thin film layer, 23 …… Transparent electrode, 24 …… Two-dimensional optical position detector, 25 …… Computational circuit, 26 …… Liquid crystal display, 27 …… Personal computer, 28 …… Signal Conversion circuit, 29 …… LCD TV drive circuit, 30 …… He-Ne laser, 31 …… Surface emitting light source, 32 …… Lens group, 34 …… Focusing Rod lens.

Claims (6)

[Claims] 1. A means for forming a differential image signal of two temporally sequential images from an image signal converted into a time-series signal by scanning; a means for forming a synchronization signal for the differential image signal; Means for forming a trigger signal that determines the timing of detecting the center of gravity of the differential image corresponding to the differential image signal, and recording the differential image signal at the timing of the synchronization signal in a spatial light modulator that is a constituent element of the spatial light modulating means described later. Signal writing means for converting into a possible writing signal, a light source for generating a light beam, a differential image signal is sequentially recorded and held on the two-dimensional plane of the spatial light modulator by the writing signal, and the light beam The time-series difference image signal is obtained by guiding the read light to the spatial light modulator as read light and modulating the read light simultaneously with the sequentially recorded difference image signals. And a spatial light modulator for converting the differential light image simultaneously displayed in a two-dimensional space, and photoelectrically converting the differential light image to generate a plurality of photocurrents. A two-dimensional light position detecting means for obtaining barycentric coordinates of the differential light image; a barycenter of the differential light image based on the plurality of photocurrents;
An image analysis apparatus comprising: a calculation unit that calculates at least one of a moving direction of a center of gravity, a moving distance and a moving speed, and a motion vector. 2. The spatial light modulating means comprises a display device for displaying the differential image signal from the differential image signal forming means, and the spatial light modulating means is constituted by using the display light as the writing signal. The image analysis apparatus according to claim 1, wherein the image is written on a two-dimensional plane of the modulator. 3. The time required for the spatial light modulating means to write at least one frame of the differential image signal using the differential image signal displayed on the display device constituting the signal writing means as the writing signal. Over the above, it has a spatial light modulator for recording and holding the write signal, the light beam from the light source is simultaneously modulated by the write signal,
The time series of the write signals are simultaneously displayed as a differential optical image in a two-dimensional space.
The image analysis device according to the item. 4. The image analysis apparatus according to claim 2, wherein said signal writing means forms a signal for driving a liquid crystal display. 5. The image analysis apparatus according to claim 3, wherein the spatial light modulator of the spatial light modulator is a liquid crystal display. 6. A center of gravity of a light beam composed of a differential light image from the spatial light modulator and background light corresponding to noise other than that and its intensity, and a center of gravity of the background light and its intensity are separately measured. The image analysis apparatus according to any one of claims 1 to 5, wherein the center of gravity of the differential light image is obtained by