JPH03113989A - Movement extraction device for picture - Google Patents

Abstract

PURPOSE:To attain movement extraction of a picture in a real time at a high speed even to a high definition picture with lots of information quantity by varying the polarization state of a readout light at a 1st space optical modulator in response to movement picture information, reading an output of the 1st space optical modulator at a 2nd space optical modulator and varying the polarization state corresponding to the moving picture information. CONSTITUTION:An input moving picture introduced from an image forming optical system by a camera lens 2 is formed on space optical modulators 11a, 11b via a beam splitter 15c and a glass block 18b or the like. In order to prevent the inversion of the image at the branch of an input moving picture and production of fog of the image resulting from different optical path length from the camera lens 2 to the space optical modulators 11a, 11b, the beam splitter 15c and the mirror 17 are arranged to the optical system. Thus, the optical path length of two optical paths and the inversion of image are corrected. One beam is made incident in the write face of the space optical modulator 11a and the other beam is transmitted through the beam splitter 15c and the glass block 18b and incident in the write face of the space optical modulator 11b and the image is formed.

Description

DETAILED DESCRIPTION OF THE INVENTION [Field of Industrial Application] The present invention relates to an image motion extraction device that extracts motion components from image information in real time.

[Prior Art] Image information has a huge amount of information because it is two-dimensional information, but it has two-dimensional connections. Image processing is an attempt to make use of this two-dimensional connection to compact the enormous amount of information, and one such process is computational processing that extracts motion components from image information in real time. .

FIG. 6 is a block diagram for explaining a conventional image motion extraction device. lot is TV camera, 102,1
Reference numeral 03 represents a frame memory that stores two temporally sequential images from a television camera lot, and 104 represents a digital signal processing processor (DSP). The motion extraction calculation according to this conventional example is performed by the digital signal processing processor (DSP) 104 as follows. First, an image signal to be processed is converted into a time-series electric signal (for example, an NTSC image signal) by a TV camera 101 including a COD sensor or an image pickup tube, and this signal is converted into a digital signal and stored in a frame memory. 102,
103 in sequence. Thereafter, the digital signal processing processor 104 arranged at the next stage performs necessary operations such as detecting coincidence/mismatch of temporally preceding images stored in the frame memories 102 and 103 for each pixel or several pixels. The process was carried out one after another.

[Problems to be Solved by the Invention] However, in the image information motion extraction means using the digital signal processing processor in the above-mentioned conventional technology,
Although various image processing is possible through programming,
Due to limitations in signal transfer rate and calculation time, there are limits to real-time processing for high-speed, high-definition images. The calculation time limit is the digital signal processing processor 1
This is due to the serial nature of the processing in 04, and the fact that information on each two-dimensional point of image information must be processed one by one one by one.

The present invention was devised to solve the above-mentioned problems, and an object of the present invention is to provide an image motion extraction device that can perform real-time processing on high-speed, high-definition images.

[Means for Solving the Problems] The structure of the image motion extraction device of the present invention for achieving the above object is as follows: a first spatial light modulator that changes the polarization state of linearly polarized readout light and outputs the same; and a first spatial light modulator that changes the polarization state of linearly polarized readout light and outputs it, and a first spatial light modulator that corresponds to second moving image information on the writing surface in accordance with a second applied signal. a second spatial light modulator that changes the polarization state of the output light emitted from the device and outputs the output light; and a readout light source section that outputs the linearly polarized readout light to the first spatial light modulator according to a control signal. , the moving image light serving as the first and second moving image information is transmitted to the first and second moving image information.
an optical system that forms an image on the writing surface of the spatial light modulator; and a control unit that outputs each of the first applied signal, the second applied signal, and the control signal in conjunction with each other, and It is characterized in that the output light output from the spatial light modulator is used as a motion component of an image.

[Function] The present invention captures the moving image light imaged by the optical system on the writing surfaces of two spatial light modulators as two pieces of moving image information according to the applied signal from the control unit, and The first spatial light modulator changes the polarization state of the readout light in response to the information, and the second spatial light modulator uses the output of the first spatial light modulator as the readout light and changes its polarization state to the above video. By changing according to the image information, 2
A mismatch component, that is, a motion component, of the two moving image information is extracted. According to the present invention, this motion component extraction can be performed at high speed in real time by simultaneously performing the extraction of all pixels of moving image information on an image-by-image basis.

[Example] Hereinafter, an example of the present invention will be described in detail based on the drawings.

FIG. 1 is a block diagram showing a first embodiment of the present invention. l
2 is an optical processing unit that extracts image motion using the first spatial light modulator 11a, the second spatial light modulator itb, and the readout light source 12; 2 is a writing unit for each of the spatial light modulators 11a and llb; A camera lens that forms an imaging optical system that forms a moving image light onto a surface; 3 a video sensor that photoelectrically converts the output image of the optical processing unit 1 into an electrical video signal; 4 extracts the movement of the image. In order to do this, a signal V is applied to the first spatial light modulator lla, a signal V1 is applied to the second spatial light modulator Ilb, and a signal v is applied to control the lighting of the light source 12 for reading.
This is a control unit that outputs data in conjunction with 3. The image to be processed is captured from the front of the camera lens 2 (on the right side in the figure), the aperture and focus are adjusted by the camera lens 2, and the image is introduced into the optical processing unit l, where the motion component is extracted. , a photoelectric conversion is performed by a video sensor 3 composed of a COD array sensor or the like.

The optical processing section 1 is the spatial light modulator 11a described above.

11b and the reading light source 12, a lens 13a
, 13b, 13c, 13d, polarizers I4a, 14b, and beam splitters 15a, f5b.

15c, glass blocks 16a and 16b, a vapor deposition mirror 17, and a right-angle prism 18. The spatial light modulators 11a and 12b have a writing surface and a reading surface, and in the figure, the surface shown in black represents the writing surface, and the surface shown in white represents the reading surface. The second spatial light modulator 11b is located behind the camera lens 2 (to the left in the figure) via a beam splitter 15c and a glass block 16b.
Place it with the writing side facing up. An evaporation mirror 17 is placed on the vertical force side of the beam splitter 15c, and a first spatial light modulator 11a is placed on the vertical side of the beam splitter 15c with its writing surface facing. A beam distributor 15a, a lens 13a, and a polarizer 14a are arranged in this order on the readout surface side of the first spatial light modulator 11a, and a light source I2 such as a laser diode is arranged behind them.

The polarizer 14a is arranged, for example, in the p polarization direction, and the lens t3
Let a be such that the laser light from the light source 12 is made into a parallel beam. These light source 12, lens 13a, and polarizer 14a constitute a readout light source section of this embodiment.

On the other hand, behind the readout surface side of the second spatial light modulation 5ttb, a beam splitter 15b, a lens 13d, and a polarizer 14b are arranged in this order.The polarizer 14b is arranged in the S polarization direction, and the lens 13d is arranged in the second direction. Spatial modulator 1. I
It is assumed that the output image b is formed on the video sensor 3. First spatial light modulator of beam splitter 15a! ! a
A lens I3b, a glass block 16a, and a right-angle prism 18 are arranged in this order on the plane in which the output light is reflected. The reflection direction of the right-angle prism 18 is directed toward the beam splitter 15b, and a lens 13c is disposed between it and the beam splitter 15b.
is arranged so that when the image of the output light of the first spatial light modulator 11a is incident on the reading surface side of the second spatial light modulator ttb, the pixel position is the pixel of the image on the writing surface side. Make it correspond to the position. In the readout optical system, the optical length of the spatial light modulator 11a and lens 13b is set to 2.
f (f is the focal length M of the lens), lens 13b and lens 1
The optical length of 3c is 4f, the lens 13c and the spatial light modulator 11
An imaging system is configured in which the optical length of b is 2f. In addition, when observing a moving part directly without using photoelectric conversion, a light emitting diode is used as the readout light source 12, and a polarizer 14 is used.
The light emitted from b may be directly projected onto a screen or the like.

Next, the configuration of the spatial light modulators 11a and llb will be explained. FIGS. 2(a) and 2(b) are a side sectional view (a) and a top view (b) showing the configuration of the spatial light modulator. zt, zt
' is a glass substrate, t12 is a photoconductive film sensitive to writing light, 113 is a dielectric mirror, 114 is a ferroelectric liquid crystal (FLC), 115 115' is an alignment film for aligning the FLC 114, 116, 116', 116″
117 is a transparent electrode, 117 is an FLC, 114 is a spacer for keeping the thickness of the layer constant, 118 is an adhesive for sealing and fixing, and 119 is for connecting the upper transparent electrode 116 to one lead electrode 116a. It is a silver paste layer. A transparent electrode 116.116'' is formed on one glass substrate 111 on the writing surface side, a non-polar photoconductive film 112 is formed on the transparent electrode 116 by a film deposition method, and a photoconductive film is further formed on the transparent electrode 116. The dielectric mirror 1 is placed on 112 in order.
13 and an alignment film 115 are formed. Further, a transparent electrode 116' is formed on the other glass substrate 111' on the reading surface side, and an alignment film l! is formed on the transparent electrode 116'. form 5'. A spacer 11 is provided between the alignment films 115 and 115″.
7, a gap is formed, and the gap is filled with FLC1t4. 116b is the other electrode connected to the transparent electrode 116.

Next, the control unit 4 in FIG. 1 is constituted by a timing circuit that generates a plurality of pulses of a predetermined width in a predetermined order at a predetermined period. FIG. 3 is a timing diagram of the pulse output.

The signal V applied to the spatial light modulator 11a includes a reset pulse RP that resets the orientation state based on the previous input video image, and a set pulse SP that sets the orientation state according to the new input video image. The applied signal ■ to 11b is also the reset pulse RP.
and set pulse SP. In the above, the signal V,
What is signal V? The set pulses SP and SPt are applied alternately every frame, so that a time difference is provided between the set pulse SP and the set pulse SPt in order to extract a motion component, and each set pulse SP and the set pulse SPt are applied after the reset pulse RP. In each frame, in order to generate readout light for extracting motion components,
light source! A read pulse WP is sent out as an applied signal V to 2. In this embodiment, a 20 V, 300 μsec negative pulse is applied as the reset pulse RP, and a 20 V, 300 μsec positive pulse is applied as the set pulses SPI, SP2. Further, the readout μ pulse WP is, for example, an NTS
When reading using the C method, a pulse width of about 30 msec is required. In this case, motion extraction is performed for 300 microseconds.

The operation and effects of the first embodiment configured as above will be described.

First, in FIG. 1, the input moving image introduced from the imaging optical system by the camera lens 2 is transmitted to the beam splitter 15.
c, the spatial light modulator 11a via the glass block 16b, etc.
, llb. When splitting human-powered dynamic images, the beam splitter 15c and A mirror 17 is arranged in the optical system. This corrects the optical path length of the two optical paths and the inversion of the image. 1
One beam is incident on the writing surface of the spatial light modulator 2a, and the other beam is transmitted through the beam splitter 15c and the glass block 16b, and is incident on the writing surface of the spatial light modulator 11b to form an image.

Here, the spatial light modulator 11a shown in FIGS. 1 and 2,
The operation of 11b will be explained. Spatial light modulator 11a (llb
), if the transparent electrode 116 is regarded as a hot terminal and 116'' is regarded as a ground terminal, the readout light is modulated as follows.The ferroelectric liquid crystal 114 operates in response to the application of voltage Vf and pulse width t. has a threshold value C, i.e.
If a positive pulse of Vf·tic is applied, it switches to the up state, and if Vf-t<c, the past state is maintained. The same applies when the applied pulse is negative, -Vf-t
If l>c, it switches to the down state. When p-linearly polarized light is input as readout light, S-polarized light is read out in the up state, and p-polarized light is read out in the down state. When S-polarized light is incident, p-polarized light is in the up state, d.

In the wn state, S polarized light is read out. Spatial light modulator 11
Electrically, a(llb) can be considered to be a series circuit consisting of a resistor formed by the photoconductive film [12, a capacitance formed by the dielectric mirror 113, and the ferroelectric liquid crystal 114], so when a pulse voltage is applied, the voltage is applied to the photoconductive film f12. Ferroelectric liquid crystal 1 with or without light irradiation
The voltage applied to 14 varies. Therefore, a pulse of several times the threshold value C is applied to the spatial light modulator 11a (llb) in anticipation of the voltage drop caused by the dielectric mirror 113.
If this is done, the polarization of the readout light can be modulated by irradiating the XEJz2 with light.

For such spatial light modulators tla and llb, one pulse is sent from the control unit 4 at the timing explained in FIG.
Apply alternately every cycle. As mentioned above, this period t corresponds to one frame of the motion component image, and the set of signals Vl consisting of a reset pulse RP, a set pulse SP,
Alternatively, it consists of a set of signals (2) consisting of a reset pulse RP, a set pulse S P t, and a read pulse WP.

First, before applying the set pulse SP or SPt, a reset pulse RP (negative voltage) is applied to the spatial light modulator 1.
1a or ttb is initialized to the down state, and in this state, a voltage of Vf-t)c is applied to the threshold value C to ensure a reliable reset regardless of the brightness or darkness of the input moving image. . Next, during the first frame (ΔT1), a set pulse SP is applied only to the spatial light modulator 11a,
When (a positive voltage) is applied, only the bright portion of the input image is oriented in the υp state in the ferroelectric liquid crystal in the spatial light modulator 11a, but the bright portion remains in the down state oriented at the time of reset. This orientation state is maintained even after the set pulse SPI voltage is removed. On the other hand, if the set pulse S P t is applied only to the spatial light modulator 11b during the second frame (ΔT2), it operates in the same way as the spatial light modulator 11a.

Here, the principle of motion extraction when p-polarized light is used for readout light, 5 analyzers are used for readout detection, and spatial light modulators 11a and llb are connected in cascade with the readout surfaces is as follows. It is.

(1) When the input image is "bright" in the first frame and "bright" in the second frame, the first spatial light modulator 11a is in the up state, the reflected light is modulated into S-polarized light, and the second spatial light modulator 11a is in the up state. Modulator 11
b also shows an up state, and the S-polarized light incident from the first spatial light modulator 11a is returned to p-polarized light. Therefore, S
No output is obtained from the polarizer f4b arranged in the polarization direction.

(2) The input image is “bright” in the first frame, “dark” in the second frame
At this time, the first spatial light modulator 11a is in the up state and the reflected light is modulated into S-polarized light, but the second spatial light modulator zb
indicates a down state, and the S-polarized light incident from the first spatial light modulator 11a is reflected as S-polarized light. Therefore, a "bright" state output is obtained from the S polarizer 14b.

(3) The input image is “dark” in the first frame and “dark” in the second frame.
When the light is "bright," the first spatial light modulator 11a is in the down state, the reflected light maintains p-polarization, and the second spatial light modulator 11a is in the down state.
1b indicates an up state, and the p-polarized light incident from the first spatial light modulator 11a is output as S-polarized light. Therefore, a "bright" state output is obtained from the S polarizer 14b.

(4) When the input image is "dark" in the first frame and "dark" in the second frame, the spatial light modulators 11a and IIb are both do
It shows a wn state, and the reflected light is read out as p-polarized light.

Therefore, no output can be obtained from the S polarizer 14b.

Based on the explanations (1) to (4) above, only the portion of the input image where there is a change in brightness or darkness between the first frame and the second frame is detected by the polarizer 14b, which is the S analyzer. “Ming”
You will get the status output. In order to improve the contrast, the readout light source 1 is used as shown in
2, the readout light is lit in pulses by supplying a voltage to a laser diode or the like only when the set pulse 2 is applied. By continuously repeating the same operation using this as one frame, motion can be extracted from the input video image in real time.

Returning to FIG. 1 again, the operation will be explained.

The spatial light modulators 11a and llb are connected in cascade between their readout surfaces, and receive set pulses SP and llb, respectively.
, SP, with input video images with a time difference of l frames. Here is the light source for reading! When the write light pulse WP is applied to the polarizer 1, the read light is applied to the polarizer 1.
4a, only the p linearly polarized light component is taken out, and the lens 13
The beam is made into a parallel beam at point a, and enters the readout surface of the spatial light modulator 11a via the beam splitter 15a.

This readout light is modulated by the moving image writing light that is incident on the writing surface of the spatial light modulator 11a, and is again transmitted to the beam splitter 15a, the lens 13b, the glass block 16a, the prism 18. The light enters the readout surface of the spatial light modulator 11b via the lens 13c and the beam splitter 15b. This readout light is transmitted to the spatial light modulator 11b again.
modulated by beam splitter 15b and lens 13d.
.

The light is emitted from the polarizer 14b, photoelectrically converted by the video sensor 3, and read out using the NTSC method or the like.

FIG. 4 is a block diagram showing a second embodiment of the present invention. This example is an example of configuring an optical processing section having a transmission type structure that does not include a mirror film in the configuration of a spatial light modulator. The basic difference between this embodiment and the first embodiment in the optical processing section 1' is that in the first embodiment, the readout light is directed to the ferroelectric liquid crystal 114 side of the spatial light modulators 11a, . Since it is incident on the dielectric mirror 113 and reflected by the dielectric mirror 113,
Although the wavelength of the readout light source 12 could be freely selected,
In the embodiment, transmissive spatial light modulators 11c and li
Due to the circumstances of using spatial light modulators 11c, 1. It must not be absorbed by the photoconductive film in Id and must not cause photoconduction. Therefore, in the second embodiment, an 850 nm laser diode is used as the reading light source 12. 13e
, 13f and 13g are lenses, 14c and 14d are polarizers,
15d is a beam splitter, 16c and 16d are glass blowers, 17b is a mirror, 18c and 18d are right-angle prisms, 19a and 19b are graphical mirrors, and 3 is a video sensor. The moving image that is manually input from the camera lens of the circuit passes through the lens 13g and is sent to the beam splitter 15d.
It is branched at. One beam is beam splitter 15d
The light is reflected to the left by the prism 18b1 and enters the first spatial light modulator 11c via the dichroic mirror 19a. The other beam is reflected by the mirror 17b, and is again reflected to the right by the beam splitter 15d, and is then reflected to the prism 1.
8c, it is reflected by the gicroic mirror 19b. The guichroic mirrors 19a and 19b are designed to reflect light with a wavelength shorter than 700 nm and transmit light with a wavelength longer than 730 nm.

This embodiment has the advantage that the distance between each element can be shortened by using a transmissive spatial light modulator, so that the imaging optical system for the readout light can be simplified. The readout light is emitted from the laser diode light source I2, and the polarizer! It becomes a parallel beam of linearly polarized light by the 4c lens 13e. Spatial light modulator 11c
, lld passes through a lens 13f and a polarizer 14d, and enters a video sensor 3 using a COD array sensor or the like for only the moving portion. Spatial light modulator I! c,l
id and the signal pulse V l applied to the light source 12,
V t and V 3 are the same as in the first embodiment.

Also in the second embodiment with such a configuration, the first spatial light modulator flc and the second spatial light modulator 11d are oriented by the input moving images with a time difference, and the readout that is subsequently generated is Using light, motion components of these input moving images can be read out from the video sensor based on the same principle as in the first embodiment.

FIG. 5 is a block diagram showing a third embodiment of the present invention. In the first embodiment and the second embodiment, input moving images are directly input.
Although the configuration for writing with light has been shown, in the third embodiment, the optical processing unit 1'' configuration is shown when input moving images are converted into electrical image signals such as the NTSC system and manually performed. In the figure, 20a.

Reference numeral 20b denotes an electric writing type spatial modulator, which is a nematic liquid crystal panel with an electrode structure of an active matrix structure using thin simulated transistors. 1.1e.

11f is +1 similar to the spatial modulator in the first embodiment.
first and second spatial light modulators having a 1T structure, 12b;
12c is a writing light source, 12a is a reading light source, 1
3h, 13i, 13j, 13, 13Q is lens, +4
e, 14f is a polarizer, 15e.

15f is a beam splitter, and 3 is a video sensor.

The electric writing type spatial light modulators 20a and 20b are arranged on the writing surface side of the spatial light modulators 11a and llf arranged in the same plane, respectively, and the writing moonlight [12b and 12c] is passed through the lenses 13h and 13i, respectively. incident. A beam splitter 15e, a lens 13j2 and a polarizer 14e are provided on the readout surface side of the first spatial light modulator lie.
are arranged in this order, and the reading light source 12a is arranged in front of them. On the readout surface side of the second spatial light modulator Iff, a beam splitter 15f, a lens 1312° polarizer 14f
are arranged in this order, and the video sensor 3 is arranged in front of them.

A lens 13k is installed between the beam splitters 15e and 15f.
are arranged so that the output light of the first spatial light modulator lie is incident on the readout surface of the second spatial light modulator 11f with the pixel positions of the image corresponding to each other.

In the above configuration, the laser diodes 12b, 12
The electrical image signal is converted into a spatial optical signal by the writing light source c, lenses 13h and 131, and electric writing type spatial modulators 20a and 20b, which are input to the spatial light modulators lie and Iff, respectively. Spatial light modulator 11e and I
In order to give signals with a time difference to ff, by applying a positive voltage synchronized with the set pulse to the laser diodes 12b and 12c, images of odd frames are written to the spatial light modulator 1'1e, and images of even frames are written to the spatial light modulator 1'1e. Spatial light modulator! ! Write to f. The readout light is emitted from the laser diode 12a, passes through the spatial light modulators lie and llf, and a motion component is detected by the video sensor. The applied signals Vlv, such as the set pulses, are each applied to a spatial light modulator lie.

It is shared by Iff and the writing light sources 12b and 12c,
Together with the signal V3 applied to the reading light source 12a, this signal is applied from a control section configured similarly to the first embodiment of the flashlight. In this embodiment as well, the first spatial light modulator 11e and the second spatial light modulator 11f are oriented by the input moving images with a time difference, and the readout light generated subsequently causes the first spatial light modulator 11e and the second spatial light modulator 11f to
Based on the same principle as in the embodiment, the motion components of these input moving images can be read out from the video sensor 3.

[Effects of the Invention] As is clear from the above description, according to the image motion extraction device of the present invention, image motion extraction is performed simultaneously for all pixels by optical processing, so it is not as easy as a conventional digital signal processing processor. This eliminates the limitation on calculation time caused by sequential processing, and enables high-speed real-time image motion extraction even for high-definition images with a large amount of information. Therefore, the present invention can be incorporated into broadcast equipment, image processing devices, and the like.

[Brief explanation of drawings]

FIG. 1 is a configuration diagram showing a first embodiment of the present invention, and FIG. 2 (
a) and (b) are cross-sectional views and top views showing the configuration of the spatial light modulator of the first embodiment, FIG. 3 is an output timing diagram of the control section of the first embodiment, and FIG. 4 is the present invention. Fig. 5 is a block diagram showing the third embodiment of the present invention;
FIG. 6 is a block diagram of a conventional example. 1.1', 1''...Optical processing unit, 2...Camera lens, 3...Video sensor, 4...Control unit, tla
, Ilc. 11 e - first spatial light modulator, Ilb, lld. If... second spatial light modulator, 12, 12a...
Reading light source, 12b, 12c...Writing light source, 14a, 14bj4c, I4d, 14e, 14f.
=Polarizer, 15a, 15b, 15c, 15d, 15e1
5f...beam splitter! 8, 18b, 18c・
...Right angle prism, 19a, 19b...Gicroic mirror, 20a, 20b...Electric writing type spatial light modulator.

Claims (1)

[Claims] (1) A first spatial light modulator that changes the polarization state of linearly polarized readout light and outputs the linearly polarized readout light in response to the first moving image information on the writing surface according to the first applied signal; a second spatial light modulator that changes the polarization state of output light output from the first spatial light modulator in response to second moving image information on a writing surface in response to an applied signal; a readout light source unit that emits the linearly polarized readout light to the first spatial light modulator according to a signal; and a readout light source unit that emits the linearly polarized readout light to the first spatial light modulator;
an optical system that forms an image on the writing surface of the spatial light modulator; and a control unit that outputs each of the first applied signal, the second applied signal, and the control signal in conjunction with each other, and An image motion extraction device characterized in that output light output from a spatial light modulator is used as a motion component of an image.