KR20110137934A - Sham image projection device based on dmd for linear sensor and sham image projection method - Google Patents

Abstract

PURPOSE: A DMD(Digital Micro-mirror Device) based sham image projection device for a linear sensor and a projecting method thereof are provided to extract an image suitable for the reception of the linear sensor and accurately align an image projected at each pixel of a TDI(Time Delay Integration) senor. CONSTITUTION: An image modulation circuit(201) modules an image signal to be suitable for projection on a linear sensor. The image signal is received by applying a PWM(Pulse Width Modulation) using a digital torsion beam DMD. An optical system(202) transmits energy sources to be projected on the sensor system to the DMD. An optical system for projection(203) is composed of a DMD and a lens component and transmits energy to a sensor system.

Description

DMD-based simulated image projection apparatus for linear sensor and simulation image projection method using same {sham image projection device based on DMD for linear sensor and sham image projection method}

The present invention relates to a simulation image projection apparatus and a simulation image projection method using the same, and more particularly, to a simulation image projection apparatus suitable for projecting a simulation image on a sensor system of a one-dimensional scanning method and a simulation image projection method using the same. will be.

The image sensor detects electromagnetic radiation generated from an object, converts it into an electrical signal, and outputs it as a digital signal. Examples of systems in which image sensors are used are night and inclement weather vision systems and forward looking infrared (FLIR) navigation systems. Image sensors are widely used as the production cost of the system is reduced and performance is increased. As systems using image sensors increase in quantity and complexity, so does the need for composite image generation and projection of inputs to the image sensor. There are some difficult technical problems associated with image sensors, including initial design optimization, design verification, generation testing, and user training. These problems are generally more difficult when dealing with invisible systems than visible systems. Current invisible image simulation systems typically emit radiation from the projection device, resulting in lower resolution and lower contrast and lower refresh rates. Current systems are typically designed only for a single image sensor and generally do not generate phase information in the image.

State-of-the-art infrared system generation assays inject electrical test signals into sensor systems that are typically processed. In the case of signal injection, it is not an optimal test method because electronic devices, leveling devices, optics, etc. related to the image sensor are not tested. The optics and detectors of the system are typically tested separately, each with a very simple still image, a test pattern, or moving very slowly, but with a more complex series of processes for composite images. These two field test methodologies do not test the entire system end-to-end. To provide a realistic image for the input signal to the image sensor, complex systems require field testing, which can be costly for the image sensor. There is a need for a technology capable of implementing an image projection system that can generate and project high quality images in real time and non-real-time for input to an image sensor.

Projection apparatus according to the prior art is a device of the concept of projecting the infrared light by using an electroluminescent diode and the concept of reflecting and projecting the light source to a digital micro-mirror device (DMD).

1 is a general configuration of a DMD based projection apparatus. The configuration is composed of a projection unit system 200 and a system control unit 100 for controlling the projection unit system 200.

The system controller 100 is generally composed of a computer with convenient user input, and a program for controlling the projection unit system 200 is implemented. The system control unit 100 transmits a test image to the projection unit system 200, controls the projection unit system 200, and observes a state.

The projection unit system 200 includes an image modulation circuit 201, an illumination optical system 202, and a projection optical system 203. The image modulation circuit 201 modulates a received image signal in a form capable of DMD control. Various modulations, such as amplitude modulation using pulse width modulation (PWM) and phase modulation and amplitude modulation using analog pre-complexing, are performed. It is possible. Illumination optics 202 are lens components that will deliver energy to the DMD and the energy source to project to the sensor system. Projection optics 203 are comprised of DMD and lens components and are configured such that energy delivered from illumination optics 202 can be reflected from the DMD and delivered to the sensor system.

Conventional image extractors extract and project all images corresponding to a single field of view (FOV), and extract image data of areas (for example, between pixels and pixels) that are not actually seen by the pixels of the linear sensor. . In such a case, it is difficult to complete image processing and data transmission to the DMD within a short integration time. For example, in a linear sensor system with an integration time of 25 ms, the image transmission should be completed in about 5 ms except for the 20 ms DMD stabilization time.

It is an object of the present invention to provide a projection apparatus and a projection method capable of extracting an image suitable for reception by a linear sensor and accurately matching an image projected to each pixel position of a time delay integration sensor.

It is another object of the present invention to provide a projection apparatus and a projection method that can improve the projection intensity resolution of an optical signal by utilizing the time delay integral principle.

Still another object of the present invention is to provide a projection apparatus and a projection method capable of projecting as the image is obtained by rotating in a stationary state without the sensor system rotating.

Still another object of the present invention is to provide a projection apparatus and a projection method capable of projecting an image synchronized with a linear sensor system.

DMD-based simulation image projection apparatus for a linear sensor according to the present invention for achieving the above object is a DMD-based projection apparatus comprising a projection system and a system control unit for controlling the projection system, the projection apparatus system is An image modulation circuit for modulating a received image signal into a form suitable for projecting onto a linear sensor by applying pulse width modulation (PWM) using a digital torsion beam digital micro-mirror device (DMD); An illumination optical system for transmitting the energy source and energy to be projected to the sensor system to the DMD; And a projection optical system composed of a DMD and a lens component such that energy transmitted from the illumination optical system is reflected from the DMD and transmitted to the sensor system.

Detailed features of the DMD-based simulated image projection apparatus for a linear sensor according to the present invention include the image of the position where each pixel of the linear sensor is viewed in consideration of the condition of the linear sensor in order to extract only the minimum image necessary for the image modulation circuit to project. An image extraction unit for a time delay integration (TDI) linear sensor for extracting only data; A time delay integral (TDI) image processor for increasing the resolution of the projection light intensity by changing the image data value of a corresponding point when the image data of one point is moved to each pixel position of the TDI linear sensor and exposed; A dithering image processor for expressing one gray image pixel using a binary pixel; And a DMD image processor configured to calculate the projection pixel position of the DMD array and determine the projection pixel position in the initial DMD array.

Another detailed feature of the DMD-based simulated image projection apparatus for linear sensors according to the present invention is a field of view (FOV), pixel size, pixel spacing, TDI tap count and over sampling method which are the conditions of the linear sensor to extract only the minimum image. It includes at least one of the.

Another detailed feature of the DMD-based simulated image projection apparatus for a linear sensor according to the present invention is that the image extractor for the time delay integrated linear sensor increases the resolution of the projection light intensity to have a resolution from 0 to the number of time delay integrated taps. to be.

Another detailed feature of the DMD-based simulated image projection apparatus for a linear sensor according to the present invention is that the projection optical system is designed such that at least 4 (2 × 2) mirrors of the DMD are formed on one pixel of the linear sensor.

DMD-based simulated image projection method for a linear sensor according to the present invention is a DMD-based simulated image projection method using a DMD-based projection device comprising a projection device system and a system control unit for controlling the projection device system, TDI linear sensor An image extraction process for a time delay integral linear sensor for extracting an image of a position where each pixel is viewed by receiving a synchronization signal of the signal and synchronizing with the line speed of the linear sensor; A time delay integrated image processing process for processing an image using a time delay integral characteristic to solve the limitation on the projected optical resolution; Dithering image processing for representing one gray image pixel using 4 (2 × 2) or more miniature mirrors of DMD; And a DMD image processing process of calculating the projection pixel position in the DMD array to determine the projection pixel position in the initial DMD array.

Detailed features of the DMD-based simulated image projection method for linear sensors according to the present invention include the angle of the actual linear sensor in consideration of the conditions of the linear sensor in order to extract only the minimum image necessary for the projection of the time delay integral linear sensor. It only extracts the image data of the position that the pixel sees.

Another detailed feature of the DMD-based simulated image projection method for a linear sensor according to the present invention is a field of view (FOV), pixel size, pixel spacing, TDI tap count and over sampling method which are conditions of a linear sensor to extract only the minimum image. It includes at least one of the.

Another detailed feature of the DMD-based simulated image projection method for a linear sensor according to the present invention is that the time-delay-integrated linear sensor image extraction process extracts only an image corresponding to an over sampling position while moving from one pixel to the next. to be.

Another detailed feature of the DMD-based simulated image projection method for a linear sensor according to the present invention is that the time-delay-integrated image processing increases the resolution of the projection light intensity to have a resolution from zero to the time-delay-integrated taps.

Another detailed feature of the DMD-based simulation image projection method for a linear sensor according to the present invention is to project an image of a known pattern and move the position of each pixel projected from the DMD in the horizontal direction by the distance of each detection pixel. The method further includes adjusting each pixel detector and the image pixels projected by the DMD to coincide with each other.

DMD-based simulation image projection apparatus and projection method according to the present invention has the following effects.

First, it is possible to increase the resolution of the projection light intensity.

Second, an image suitable for reception by the linear sensor can be extracted, and the image projected at each pixel position of the time delay integral sensor can be accurately aligned.

Third, the sensor system can be rotated in a stationary state without rotation and project as if an image is obtained.

Fourth, image projection synchronized with the linear sensor system is possible.

1 is a block diagram showing a general configuration of a general DMD based projection apparatus.
2 is an exemplary view showing the configuration of a digital torsion beam DMD according to the present invention.
3 is an exemplary diagram illustrating a process of transmitting optical energy through a DMD.
4 is a block diagram showing the configuration of an image modulation circuit for a TDI linear sensor.
5 is an exemplary view illustrating an image extraction method for a TDI sensor.
6 is an exemplary diagram illustrating a principle of expressing a gray image using a dithering technique.
7 is an exemplary view of the image being projected in the DMD according to the embodiment of the present invention.
8 is an exemplary view of an image obtained by projecting on an infrared sensor system according to an embodiment of the present invention.
9 is a flowchart illustrating an image processing process in a simulation image projection method according to an embodiment of the present invention.

Hereinafter, with reference to the accompanying drawings will be described in detail the configuration of the simulation image projection apparatus according to the present invention and the progress of the projection method using the same.

In the present invention, a pulse width modulation (PWM) using a digital torsion beam DMD (Digital Micro-mirror Device) is applied to an image modulation circuit for modulating a received image signal into a form suitable for projecting onto a linear sensor. . Pulse width modulation (PWM) is generally used to represent gray images using DMD. The pulse width modulation method (PWM) is output for a predetermined time by differently adjusting the time that '1' of the DMD mirror (i.e., tilting in the + 12 ° direction) is maintained for each bit position in order to express gray image of 1 bit or more. Adjust the total amount of light. In this case, the time required to maintain the minimum '1' due to mechanical stabilization time is generally required to be 20㎲ or more, and this time may be larger or smaller depending on the type of DMD. If data is transmitted to the DMD during this stabilization time, an error may occur in the DMD mirror operation. Therefore, the extracted image must be transmitted to the DMD outside the stabilization time interval, and considering the linear sensor system having a short scanning period of several tens of microseconds, there is not much time for transmitting the image to the DMD. In addition, the simulated image projection apparatus for the moving or rotating sensor should move every time the sensor moves to project an image in the area viewed by the sensor. However, due to difficulties in stabilization, instrument development, and alignment between the sensor and the projection device, this method is not effective. Therefore, the simulated image projection device for this type of sensor should provide the effect of acquiring an image by moving or rotating in a stationary state. In addition, the number of bits of the video signal that can be represented is limited by the limitation of the number of DMD elements and the control function.

Unlike the 2D sensor system, the scanning sensor system that integrates the optical signal for a short time of several tens of microseconds (that is, the TDI integration time is very short) has a high bit resolution gray image by the conventional pulse width modulation method. it's hard to express.

For this reason, the projection apparatus and the projection method according to the present invention improve the projection intensity resolution to represent a sufficient gray image for a short time. In addition, the simulation image projection apparatus according to the present invention minimizes the simultaneous extraction image required for the test in consideration of the characteristics of the TDI linear sensor and at the same time to have the same movement or rotation effect.

The projection apparatus system according to the present invention comprises a projection apparatus system as shown in Figure 1 above, and a system control unit for controlling the projection apparatus system, the projection apparatus system, the projection apparatus system, digital torsion An image modulation circuit for modulating a received image signal into a form suitable for projecting onto a linear sensor by applying pulse width modulation (PWM) using a beam DMD; An illumination optical system for transmitting the energy source and energy to be projected to the sensor system to the DMD; And projection optics, comprised of a DMD and lens components, such that energy delivered from the illumination optics is reflected from the DMD and delivered to the sensor system.

Figure 2 shows an example of the digital torsion beam DMD applied in the present invention, Figure 3 shows the energy transfer method through the DMD. The DMD 3 uses an electrical signal to tilt a fine mirror having a pitch size (10.8 μm x 10.8 μm in this example) in the +/- 12 ° direction according to the digital state of '0' or '1'. The light input from 1) through the lens 2 is reflected in the -12 ° or + 12 ° direction. Only light reflected only in the + 12 ° direction is projected onto the sensor system 5 via the lens 4 of the projection optical system.

4 is a block diagram showing the configuration of an image modulation circuit in a projection apparatus according to the present invention for a TDI linear sensor. As shown, Time Delay Integration (TDI) for linear sensors extracts only the image data of the position of each pixel of the linear sensor considering the conditions of the linear sensor to extract only the minimum image required for projection. An image extractor 11; Time Delay Integral (TDI) image processor which increases the resolution of the projection light intensity by changing the image data value of the point when the image data of one point is moved to each pixel position of the TDI linear sensor. 12); A dithering image processor 13 expressing one gray image pixel using a binary pixel; And a DMD image processor 14 which calculates the projection pixel position in the DMD array and determines the projection pixel position in the initial DMD array.

The image extractor 11 receives a synchronization signal of the TDI linear sensor and extracts an image of a position viewed by each pixel in synchronization with the line speed of the linear sensor. Existing image extractors extract and project all images corresponding to one field of view (FOV). In this case, the image data of the area of the linear sensor that is not visible (for example, between pixels and pixels) can be extracted. do. In this case, it is difficult to complete image processing and data transmission to the DMD within a short integration time. For example, in a linear sensor system with an integration time of 25 ms, the image transmission should be completed within about 5 ms except for the 20 ms DMD mirror stabilization time. For this reason, in order to extract only the minimum image required for the projection, the condition of the linear sensor is to consider the image data of the position of each pixel of the actual linear sensor in consideration of at least one of FOV, pixel size, pixel spacing, TDI tap number and over sampling method. You can only extract it. In addition, in order to have the effect of moving or rotating even in a stopped state for a moving or rotating scan sensor, a pixel corresponding to the moved position is extracted and projected on the same DMD pixel position.

5 is an example of an image extraction method for a 4-tap TDI sensor with 2 over sampling. The TDI linear sensor, which performs 2 over sampling, generates a 2-pixel image by sampling the second image while scanning the space of one FOV and the next FOV. Due to this operation method, the image extraction in the image projection apparatus also extracts and transmits only the image corresponding to the 2 over sampling position while moving from one pixel to the next pixel. As shown in FIG. 4, when the extracted image pixels are synchronized with the line speed and projected at the same pixel position in the DMD array, the TDI linear sensor system may move and rotate the image as if it is over-sampling and acquires the image even though it is fixed. Has the same effect.

The TDI image processing unit 12 has a TDI of the sensor to compensate for the problem of the limitation in the optical resolution projected due to the limitation of the number of DMD mirrors corresponding to one pixel and the minimum '1' holding time for pulse width modulation. Perform image processing using features. TDI is a method in which a linear sensor moves a point image, acquires a signal, and then combines the acquired signals into a single pixel signal, thereby ultimately improving signal to noise ratio (SNR). Using this principle, when image data of a point is moved to each pixel position of the TDI linear sensor and exposed, changing the image data value of the point results in the TDI linear sensor changing the result of time delay integration of each pixel data. Will be generated.

Table 1 shows an example of TDI image processing for the case of a 4-tap TDI linear sensor for binary data.

Pixel data Sum
(decimal) # 1 pixel # 2 pixels # 3 pixels # 4 pixels 0 0 0 0 0 0 0 0 One One 0 0 One One 2 0 One One One 3 One One One One 4

Image data can only be expressed as '0' and '1', but if the image data is changed to '0' or '1' with time delay, the final result of adding each pixel in the TDI linear sensor is summed to the right. You can see that the change from 0 to 4. Therefore, the simulation image projection apparatus and the projection method according to the present invention increase the resolution of the projection light intensity so as to have a resolution from 0 to the time delay integral tap number.

The dithering image processor 13 expresses one image pixel using 4 (2x2) or more miniature mirrors of the DMD. The dithering technique is the same as the principle of expressing gray images using small black dots in a printer. In the present invention, the dithering technique is adopted and applied, and the projection optical system is designed such that at least 4 (2 × 2) mirrors of the DMD are formed on one pixel of the linear sensor.

6 is an example for explaining a principle of representing a gray image using a binary pixel by using a dithering technique. That is, since 3x3 binary pixels can be expressed from 0 to 9 in one image image, the bit resolution is improved to 3 bits or more as a result.

In order to stably project the image, the extracted image must be accurately projected on the DMD pixel corresponding to the IFOV viewed by each detection element of the TDI linear sensor. As an alignment method for this purpose, first, the length direction of the DMD is set to the length direction of the TDI linear sensor, and the projection pixel position is calculated in the DMD array in consideration of the physical arrangement of the detection element and the IFOV.

The DMD image processor 14 of FIG. 4 performs this calculation to determine the projection pixel position in the initial DMD array. In this case, the theoretically calculated position has a real position and an error, so it is necessary to make a fine adjustment function when connecting with the sensor system. 7 shows an image being projected on the DMD and an adjustment function of the projection position. To do this, project an image of a known pattern, and shift the position of each pixel projected from the DMD horizontally by the distance of each detection pixel, so that each pixel detector of the linear sensor and the image pixel projected from the DMD (for example, 2x2 pixels) View and adjust the output image of the linear sensor to match each other.

FIG. 8 is a 6-tap TDI linear array detection sensor, which rotates in a 360 degree azimuth direction and projects infrared energy using a simulated image projection device to a sensor system that acquires an infrared image.

9 is a flowchart illustrating an image processing process in a simulation image projection method according to an embodiment of the present invention. In a DMD based simulation image projection method using a DMD based projection apparatus comprising a projection system and a system control unit for controlling the projection system, receiving a simulation image (S1) and inputting a synchronization signal of a TDI linear sensor. An image extraction process (S2) for time delay integral linear sensor which extracts an image of a position where each pixel is viewed in synchronization with the line speed of the linear sensor; A time delay integrated image processing process (S3) of processing an image using a time delay integrated characteristic to solve the limitation on the projected optical resolution; A dither image processing step S4 of expressing one gray image pixel by using four (2 × 2) or more miniature mirrors of a DMD; DMD image processing (S5) for calculating the projection pixel position of the DMD array to determine the projection pixel position in the initial DMD array; Including the transmission and image projection process (S6) to the DMD. In this case, the image extraction process for the time delay integrated linear sensor may be configured to extract only the image data of the position viewed by each pixel of the linear sensor in consideration of the condition of the linear sensor in order to extract only the minimum image necessary for projection. At this time, in order to extract only the minimum image required for projection, the condition of the linear sensor is to consider the image data of the position of each pixel of the actual linear sensor in consideration of at least one of FOV, pixel size, pixel spacing, TDI tap number and over sampling method. Can be configured to extract only.

As described above, the present invention has been described in detail through preferred embodiments, but the present invention is not limited thereto, and various changes and applications are apparent to those skilled in the art without departing from the technical spirit of the present invention. Accordingly, the true scope of protection of the present invention should be construed according to the following claims, and all technical ideas within the scope of the same should be construed as being included in the scope of the present invention.

1: light source 2, 4: lens
3: DMD 5: Sensor System
11: image extraction unit for TDI sensor 12: TDI image processing unit
13: dither image processor 14: DMD image processor
100: system control unit 200: projection unit system
201: image modulation circuit 202: illumination optical system
203: projection optical system

Claims (11)

In the DMD-based projection apparatus comprising a projection device system and a system control unit for controlling the projection device system, the projection device system,
An image modulation circuit for modulating a received image signal into a form suitable for projecting onto a linear sensor by applying pulse width modulation (PWM) using a digital torsion beam digital micro-mirror device (DMD);
An illumination optical system for transmitting the energy source and energy to be projected to the sensor system to the DMD; And
DMD-based simulated image projection apparatus for a linear sensor, comprising a projection optical system consisting of a DMD and a lens component, the energy transmitted from the illumination optical system is reflected from the DMD and transmitted to the sensor system. The image modulation circuit of claim 1,
An image extraction unit for a time delay integration (TDI) linear sensor for extracting only image data of a position viewed by each pixel of the actual linear sensor in consideration of the condition of the linear sensor in order to extract only the minimum image necessary for projection;
A time delay integral (TDI) image processor for increasing the resolution of the projection light intensity by changing the image data value of the point when the image data of one point is moved to each pixel position of the TDI linear sensor and exposed;
A dithering image processor for expressing one gray image pixel using a binary pixel;
DMD-based simulation image projection apparatus for a linear sensor comprising a DMD image processing unit for calculating the projection pixel position of the DMD array to determine the projection pixel position in the initial DMD array. The linear sensor as claimed in claim 2, wherein the condition of the linear sensor to extract only the minimum image includes at least one of a field of view (FOV), a pixel size, a pixel interval, a number of TDI taps, and an over sampling method. DMD-based simulated image projection device for sensors. The DMD-based simulated image projection apparatus of claim 2, wherein the image extracting unit for the time delay integral linear sensor increases the resolution of the projection light intensity to have a resolution equal to the number of time delay integral taps from 0. The DMD-based simulation image projection apparatus of claim 2, wherein the projection optical system is designed such that at least 4 (2X2) mirrors of the DMD are formed on one pixel of the linear sensor. In the DMD-based simulated image projection method using a DMD-based projection device comprising a projection device system and a system control unit for controlling the projection device system,
An image extraction process for a time delay integrated linear sensor for receiving a synchronization signal of a TDI linear sensor and synchronizing with a line speed of the linear sensor to extract an image of a position seen by each pixel;
A time delay integrated image processing process for processing an image using a time delay integral characteristic to solve the limitation on the projected optical resolution;
Dithering image processing for representing one gray image pixel using 4 (2 × 2) or more miniature mirrors of DMD;
DMD-based simulated image projection method for a linear sensor comprising a DMD image processing to calculate the projection pixel position of the DMD array to determine the projection pixel position in the initial DMD array. The method of claim 6, wherein the image extraction process for the time delay integral linear sensor extracts only image data of a position viewed by each pixel of the linear sensor in consideration of the condition of the linear sensor in order to extract only the minimum image necessary for projection. DMD-based simulated image projection method for a linear sensor. The linear sensor as claimed in claim 7, wherein the condition of the linear sensor to extract only the minimum image includes at least one of a field of view (FOV), a pixel size, a pixel interval, a number of TDI taps, and an over sampling method. DMD-based simulated image projection method for sensors. 7. The DMD based simulation image projection method according to claim 6, wherein the image extraction process for the time delay integrated linear sensor extracts only an image corresponding to an over sampling position while moving from one pixel to the next pixel. 7. The DMD based simulation image projection method according to claim 6, wherein the time delay integrated image processing increases the resolution of the projection light intensity to have a resolution from 0 to the number of time delay integral taps. 7. The pixel detector of the linear sensor and the image pixels projected from the DMD coincide with each other by projecting an image of a known pattern and shifting the position of each pixel projected from the DMD by the distance of each detection pixel. DMD-based simulated image projection method for a linear sensor, characterized in that it further comprises the step of adjusting to.

Images