JP2886510B2 - Video signal data processing system - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates generally to thermal imaging systems and, more particularly, to a time division histogram chip for performing video line summing, video line capturing, and histogramming functions for video signal processing. And a video signal data processing system having the same.

[0002]

2. Description of the Related Art Scanning thermal imaging systems include surveillance systems,
It is used in various applications, including target detection / recognition systems. Such a system is typically included in a telescope lens assembly coupled to a scanner. The scanner scans energy from the scene and sends it through a lens of the imager to a detector array having a plurality of photo-responsive detector elements perpendicular to the scan direction. Each of these detector elements outputs an electrical signal proportional to the infrared flux at the particular detector element. The electrical signals generated from the detector elements are processed by system sensor electronics to generate an image that is subsequently displayed on a system output device. To improve sensitivity, some of these systems have detectors parallel to the scan direction. Ideally, the outputs of these detectors are time delayed with respect to each other and the delayed outputs are added (integrated) so that, ideally, the scanned image is output simultaneously on all of the parallel detectors. This process is called time delay and integration (TDI). In the thermal imaging system described above, the system sensor electronics processes the signals from the detector elements, thereby providing a clean output video signal to the system output device. Important elements in system electronics are:
A histogram chip that collects and processes the data obtained from the video signal so that the processor can examine the data, thereby controlling the video data compression function. Associated system hardware is the element that performs the summing function of each video line to correct for imbalance between channels and to ensure that the outputs of the detector elements are equal in voltage level and voltage gain.
The system hardware also corrects pixel misalignment associated with each channel output from the detector array, thereby ensuring that one row of output display pixels is aligned with an adjacent row of pixels. For the device.

[0003]

While conventional thermal imaging systems exhibit adequate performance characteristics, there is room for improvement in the art. In particular, this design specification requires that a thermal imaging based application be installed in a smaller footprint. Therefore, there is a need to integrate the separate functions and system hardware elements of a conventional histogram chip into a small package. Further, such thermal imaging systems have relatively high costs associated with installation due to the large number of hardware and software-based components required for installation. Further, there is a continuing need to increase the accuracy of the system as much as possible.

Accordingly, histogramming, line sum,
There is a need for a time division histogram chip that allows the performance of the line capture function and thereby minimizes the cost of the system while at the same time increasing the accuracy of the overall system.

[0005]

According to the contents of the present invention,
Histogram chips are provided for use in thermal imaging systems that can perform a number of functions conventionally performed by separately provided hardware elements. Thus, the histogram chip of the present invention eliminates the need for these separate hardware elements, thereby reducing the cost of the system. Further, the histogram chip of the present invention is provided in a smaller footprint by eliminating previously required hardware elements, thereby leaving additional space for other thermal imaging system applications. The histogram chip of the present invention further increases the overall system accuracy due to the unique implementation method.

In particular, the present invention provides a system for processing video signal data. The system includes a processor for controlling the operation of the system and processing the video signal input to the system. The histogram chip collects the video signal data and accumulates the data in a form readable by the processor. The histogram chip operates to perform histogramming, video line summing and video line capture functions. Further, a histogram chip mode controller is associated with the processor for controlling selection of a histogram chip mode of operation by the processor.

[0007] Other objects and advantages of the present invention will become more apparent upon review of the following detailed description and upon reference to the accompanying drawings.

[0008]

The following description of the preferred embodiment is merely exemplary in nature and is in no way intended to limit the invention or its application or use.

Referring to the drawings, FIG. 1 shows a LAV-25 light armored vehicle 10 which constitutes a preferred embodiment of the present invention. As shown in FIGS. 1 and 2, the present invention provides for reflected energy returning from a detected target scene 14 through a system telescope assembly 16 coupled to an externally mounted head mirror 18. Is configured as a part of the thermal imaging device 12 that processes the image.

Preferably, the thermal imaging device 12 is a Hughes Infra
red Equipment (HIRE) Thermal image sensor device. HIR
The E-Equipment is a high-performance, lightweight, modular launch control visual and thermal imaging system that can provide excellent visibility capabilities through total darkness, smoke, dust, and other adverse conditions. The HIRE system is configurable in a variety of environments, including LAV-25, Piranha, Desert Warrior, and
Included in various armored vehicles such as the LAV-105. The thermal imaging system 10 has a stand-alone thermal imaging capability,
Further, it can be configured for use in a TOW missile launch control system. The equipment includes several commercially available major components, thereby reducing the logical demands by the commonality of such things as repair equipment, support equipment, training programs, and spare parts. The EFL compensator of the present invention, by means of a thermal image sensor device, can significantly improve the image quality and aim setting capabilities of an image device over conventional thermal image systems as described below.

Referring to FIGS. 1-5, a telescope device 16 in which a target search and aim setting function is performed is provided in a protected environment within the vehicle 10. The head mirror 18 is configured to relay the detected scene to the telescope device 16. As described below, after the thermal energy signal of the target scene has been processed by the thermal imaging device, the scene is coupled to the display control panel 20 for operation of the shooter display 19,
And through the command display 21 operatively coupled to the command display control panel 22.

As shown in FIG. 3, energy from the detected scene is transmitted through the thermal imaging device 12 to a polygon mirror scanner 23 which is rotated by a scanner motor 24. The scanner comprises eight facets 23a through 23h, each of which is cut at an angle to displace the scanned scene energy by a discrete amount on the detector array. The cuts and displacements made by each facet are shown below: Table I Facet Cut Detector Array Energy Displacement (in Pixels) 23a Normal 0 23b Interlaced -1/2 23c Up +1 23d Interlaced -1/2 23e Down -1 23f interlaced -1/2 23g normal 0 23h interlaced -1/2

As the scanner rotates, the scanner mirror reflects scene energy at continuously changing angles through an imager device, generally designated 25. The imager device includes an imager lens such as lens 25a,
Detector array with this lens housed in detector device 27
Project the scene onto 26. The imager device 25 also has an imager optical system temperature sensor 25b for monitoring the temperature of the imager.
Contains. The detector device 27 is housed in a dewar 28 and cooled to a low temperature by a cooling device 28a. A cold shield 29, housed in a dewar 28, allows the detector element to detect only scene energy input through the optics of the telescope assembly and to provide energy input into the system, such as energy from the hot side of the housing. Limits the thermal energy that can be observed by the detector so as not to detect other peripheral features of the Thereby cold shield
29 reduces input noise and improves the overall system video quality.

As shown partially in FIG. 3 and in more detail in FIG. 4, the detector array 26 of the present invention comprises two offset 120 × 4 subarrays 26a of detector elements. , 26b, each element being sensitive to light in the infrared spectrum and having a respective detector element output. When the scanner scans the image of the scene across the detectors in the direction indicated by arrow A in FIG. 4, the output of each detector is a read-integration circuit (ROIC) 27a (FIG. 4) associated with the detector assembly. 5) and this circuit
27 samples the output, performs a time delay and integration (TDI) of four parallel detector elements in each detector element row, and divides the 240 resulting TDI detector channels into four.
Multiplexed into a plurality of video output channels 31, 32, with output channel 31 transmitting an output signal from a first 120 × 4 detector subarray 26a and output channel 32 transmitting an output signal from a second detector subarray 26b. . ROIC27a,
A TDI clock 27b that determines when the detector output is sampled at the TDI, a multiplexer 27c,
RO preferably having a 60: 1 sample period
And a fast detector clock 27d for the IC multiplexer.

In a preferred embodiment, the four multiplexed output channels of the detection assembly further include an input high speed clock (H
CLK) rate, multiplexed into one channel by the signal processing electronics, which preferably has a minimum of 240: 1 sample period and is associated with the system electronics described below with reference to FIG. EFL compensator uses TDI clock 27
The sample rate of DCLK 27d is changed to control the sample rate of b.

[0016] Currently provided detector arrays typically comprise 60 to 120 detector elements, each having an associated output wire. Thus, the detector array of the present invention exhibits higher resolution due to the additional detector elements. In addition, the detector array of the present invention utilizes multiplexed detector array output lines, thereby minimizing detector element output wires, minimizing the area required to provide the array, and facilitating assembly and repair. I do.

Referring to FIG. 5, the operation of the video system components is typically controlled by system electronics 34. The system electronics 34 is comprised of three cards coupled to a system motherboard 35. The cards include an analog video processing card (AVPC) 36, a scene based histogram processor card (SHPC) 38, and a memory output symbol card (MOSC) 40. The relevant functions of those three cards are described in more detail below. Also coupled to the motherboard 35 is a power supply card 42, which receives power input from the vehicle in which the system is located and outputs power to various system components at the voltage levels required by the individual system components.

Referring specifically to FIG. 6, the entire block diagram shows the components provided on the three cards 36,38,40. Referring first to the AVPC card 36, the channel outputs 31, 32 are input to an S / HMUX 52 having an associated high speed system multiplier clock (HCLK) 53.
Preferably a total of 960 detector elements (240 pixels)
Are clocked during the clock sampling period. S / H
MUX 52 is a custom integrated circuit from Hughes, part number 6364060PGA-DEV, preferably designed to sample and further multiplex the multiplexed detector element outputs.
It is. These multiplexed signals are sampled at an adjustable sampling rate. However, the signal is converted to a voltage signal via an IV converter 54 for further signal processing. Once these signals have been converted, they are digitized by an analog-to-digital converter 56.

After conversion to a digital signal, the detector element output signal is input to a signal equalizer 60. The signal equalizer 60 adds the associated gain and level values stored in the memory 62 and enhances the image quality with a uniform multiplexed digital signal output for each of the 240 detector pixels at 63 To correct the gain and level difference from each detector pixel signal.

Further referring to the AVPC card 36,
The digital input signal (to the signal equalizer 60) is 12 bits. However, when the signal equalizer corrects for signal gain and level differences, it increases the digital signal output to 19 digit bits. When the signal contains only 15 bits of usable data, saturation detector 64 sets all data beyond the 15-bit range to saturation level 1 and all data below the 15-bit range to saturation level 0. Therefore, only useful data within the 15 bit range is SHPC
Output to the card 38. The AVPC card also includes a timing / control processor 68 having a clock 53 and line timing for clocking the multiplexed signal from the S / HMUX during the sampling period. Preferably, the line timing HCLK is 240 TDIs per sampling period plus 16 clock quiescent times.
It has the clock sampling rate of the channel. However, this speed may be varied in accordance with the present invention when required as described below. The AVPC card also includes an interface 70 that connects the AVPC card components to the system microprocessor bus 72.

Next, considering the SHPC card 38, the signal output from the saturation detector 64 is input to a look-up table 74.
Generally, the output dynamic range of the digitization and signal equalization process is larger than the maximum dynamic range of a conventional image display. In addition, there are regions of the output dynamic range that have very little or no information. Therefore, the output signal of the digitization and signal equalization processing is
Entered at 74, compresses the information into the dynamic range of the display. Lookup tables are large input dynamic
A programmable method for mapping a range to a small output dynamic range is provided. The mapping is
It can be varied continuously based on either manual input from a system operator or an automatic histogram-based method. Prior to the look-up table, the video is input to the histogram / accumulator 80. Histogram / accumulator 80 performs certain programmable functions such as video line sum (hereinafter simply line sum), digitized information (hereinafter simply line capture), and histogramming of digitized information. . Search table
74 converts the 15-bit signal output from the saturation detector to 8
Convert to a bit output signal. The search table shows the Integrated Device Technology Model No. ID
32k x 8 well known in the art, such as T71256
It is preferably a random access memory (RAM), which can change continuously based on either manual input from a system operator or an automatic gain algorithm. 15 output from the saturation detector
The bit signal is also converted to a 10-byte signal through the video shifter 76.

On the SHPC card 38, microprocessors 82 and 84 are arranged. As described above, a number of functions are performed under the control of these microprocessors, including a histogramming function, a line sum function, a line capture function, and the like. The microprocessor 84 performs a number of control-related operations associated with the control panel, controls the TDI clock speed and the histogram / accumulator function for EFL compensation, level equalization values for each pixel, global level control values, And calculate the values in the lookup table. Microprocessor 82 performs more system-based processing associated with the function,
Operates in conjunction with AM86 and EEPROM90. RA
Both M86 and EEPROM 90 store software-based instructions for controlling an electronically effective focal length compensator according to a preferred embodiment of the present invention, the functions of which are described in detail below.

Next, referring to the MOSC card 40 shown in FIG. 7, the 8-bit output signal from the look-up table 74 is input through the pixel buffers 92 and 94, scan-converted via the frame memory, and After being converted back to an analog signal through the digital-analog converter 96, the analog signal is output to both the shooter display 19 and the commander display 21. Before being output through the digital-to-analog converter 96, the sign is also switched by the sign processor 98 for any pixel in the image signal. Such code data can be displayed on the commander display 21 or the archer display.
Includes status indications, aiming at aiming crosshairs, and instruction text in any of the nineteen.

Before being output to the display, the digitized signal is scan converted. Generally, a scanner scans a scene horizontally, and thus the data is multiplexed along vertical columns. However, standard video displays require that data be output along horizontal lines. Therefore, the digitized data must be converted from a vertical column input format to a horizontal line output format. Further, the digitized data from the sub-arrays are delayed in time from each other due to the separation between the sub-arrays of detectors. This delay must be removed. The delay depends on the effective focal length of the imager, and since the data is digitized, proper removal of the delay depends on accurate compensation for changes in the focal length of the image. The EFL compensator installed in the system electronics performs both of these functions.

Referring to FIGS. 8 and 9, the field programmable gate array (FPG) shown in FIG.
A schematic block diagram of A) is shown generally at 100. FPGA 100 includes two main sub-arrays, a counter sub-array 102 and a microprocessor / FPGA command sub-array 104.

Referring to counter sub-array 102, video input line 108 receives a 15-bit video input signal from saturation detector 64. Lookup table counter loop 110
Is provided for loading data into the LUT, as described below, and is provided every 7.5 Hertz = 133 ms when data is loaded into the LUT. LU
The output of T counter loop 110 is multiplexed with the FLIR video input signal at multiplexer 111 and the output on line 113.

In addition, the line synchronization and field activation lines, indicated generally at 112, are input from a system timing generator (see FIG. 6). In particular, input line 112 controls the operation of column counter 115 and row counter 116. Column counter 115 and row counter 116 are selectively enabled for histogram 80, both of which provide a control address. The row counter is incremented by one from 0 to 239 as each line of video signal data is loaded into the LUT and resets for each data load line. Column counter, row counter
Increment by 1 each time 116 is reset. The column counter signal is output on line 118 while the row counter signal is output on line 120. Then output lines 118,120
Is input to a multiplexer 122 along with a multiplexed video data input signal line 113. When activity stops at the detector array, i.e., when the detector array does not detect energy from the target scene, the input line
112 relays this information to counters 115 and 116, which reset the counters.

The look-up table addresses the counter loop 110 and the outputs of the column counter 115 and the row counter 116 are multiplexed together in a multiplexer 122. The histogram MUX selection line shown at 124 is
The histogram mode control signal from the microprocessor 82 is input to the multiplexer, thereby
6 Controls the histogram mode control signal output above. Table 1 below shows the various histogram MUX selection command inputs and the corresponding output command signal outputs at 126.

Histogram MUX selection mode 00 Histogram function 01 Line sum function 10 Line capture function 11 Asynchronous 24 function

In operation, the hardware implemented counter sub-array 102 includes a histogram chip to perform histogramming, line summing, and line capture functions in accordance with the requirements of the particular system as dictated by the microprocessor 82. Controlled by software programmed in RAM 86 to enable 80. Upon receiving the 00 signal from the processor 82, the multiplexer 122
Is output on line 126 to switch to the histogramming mode. When in the histogramming mode, the histogram chip generates a histogram of the video signal data. Video signal data is LU
Processed through T. The processor reads this histogram data and uses it in distinguishing valid video data from discardable data in video signal data compression applications.

Upon receiving the 01 signal from processor 82, multiplexer 122 outputs a line sum command signal on line 126 to the histogram chip that switches the histogram chip to line sum mode. In the line sum mode, the histogram sums the data over each of the 240 lines of video signal data output from the detector array. Therefore, at the address level, the address of one of the rows is
And 26b show the pixels of the first row. Histogram 80 sums the video data input to the histogram with the video data previously stored at position 1 of the row. The histogram chip is
When operating in line sum mode, the detector array 26
For applications associated with the processor, such as correcting for non-uniformities in each of the 240 TDI channels output from the LUT, the data from the row counter 116 is used when the video signal data is loaded into the LUT.

When receiving ten signals from the processor,
Multiplexer 122 outputs on line 126 a line capture command signal that switches the histogram to line capture mode. In the line capture mode, the histogram chip will read 2 when the column counter is incremented on each input line.
Capture one of the 40 lines. Thus, as the address is incremented by one column, the data from the previous row is entered at that row address, and processor 84 reads the data from the histogram chip. When operating in line capture mode, the histogram chip performs LUT processing on the video signal data to perform functions such as pixel alignment applications to enhance the quality of the output video signal.
Uses the data from the video input signal associated with the column counter 115 and the processor when loaded. For example, if the histogram captures two adjacent video lines and detects that the pixels in line X are not aligned with the pixels in line X + 1, the processor may use a pixel alignment function to correct the detected problem. (Effective focal length compensation) can be performed.

Upon receiving the eleven signals from the processor 82, the multiplexer 122 outputs a processor read command to the histogram chip, followed by the data accumulated by the histogram being read by the processor and into one of the applications described above. Switch the histogram chip to the mode used for it.

FPGA / microprocessor subarray
104 is coupled on line 130 to the microprocessor data bus. Through line 130, the microprocessor inputs start, stop, and most significant bit data load functions to control the loading of data into look-up table 74 through registers 132a-132c. Register outputs 134a-c are coupled to look-up table address counter loop 111 as shown in counter sub-array 102 at 136a and 136b. In addition, sub-array 104 includes a command register 132d coupled to input line 130 and output line 134d and having an input to multiplexer 140 along with outputs 134a-134c for processor readability. The command register 132d functions as an input / output port and also functions to initialize the histogram 80 in any of the modes input on the histogram MUX select line 124. Registers 134a-134d allow the processor to command the FPGA to load a piece of data into rows and columns in LUT 74.

The most significant bit register 132c is constructed on the assumption that the seven most significant bits of the start and stop address registers are equal. The most significant bit register 132c allows the system to identify which bank of memory in the LUT is to be loaded with data and this bank of memory is 1024
, Only the block load can be performed.

Referring to FIG. 10, a flow diagram of the execution of the preferred method of the histogram chip according to the present invention is generally shown.
Shown as 150. Initially, at step 152, the histogram chip 80 receives a mode command from the processor 82. If, at step 154, the histogram chip is placed in the histogramming mode by the mode command, the histogram chip accumulates the input video signal data at step 156 for the purpose of compressing the video signal. If the command is not a histogram mode command in step 154, the method proceeds to step 15
Proceed to 8. If the command places the chip in line sum mode at step 158, then at step 160 the address of the histogram chip is used to correct the imbalance between channels, so that the video signal is line by line for each of 240 lines of video data. Sum the data.
If the command is not in line sum mode at step 158, the method proceeds to step 162. If the chip was placed in line capture mode by command in step 162, the address of the histogram chip will be
For each of the 240 output video lines above 9,21, increment the input video signal data at the column-by-column level in step 164 for pixel alignment purposes. If, in step 162, the histogram chip is not in line capture mode, the method proceeds to step 162.
Proceed to 166. In step 166, the histogram chip determines whether the command places the chip in asynchronous processor read mode. If so, in step 168, the histogram chip
Is switched to a mode in which data is read from the chip. If it is determined by this method that none of the operating modes of the above-described histogram are selected, the application will proceed until the histogram chip receives a mode command from the processor at step 152.
The method ends at 170, at which point the method is repeated.

In considering the above detailed description, the provision of the multi-function time-division histogram chip of the present invention allows the hardware components required to provide conventional separate line sum and line capture functions to be implemented. It should be appreciated that the need and associated expenditure is eliminated because it is integrated with the histogram chip of the present invention. Thus, the histogram chip of the present invention reduces the area required for system installation due to its large number of software-driven elements, and separate hardware previously required to provide line summing and line capture functions. It provides flexibility and growth capability to thermal imaging systems by eliminating the need for wear elements. The histogram chip of the present invention also reduces the cost and complexity of the system, while maintaining high operation of the entire system.

[0038] Various other advantages of the present invention will become apparent to one of ordinary skill in the art after reviewing the above description and drawings in conjunction with the appended claims.

[Brief description of the drawings]

FIG. 1 is a perspective view of a LAV-25 light armored vehicle embodying the present invention.

FIG. 2 is a perspective view of a thermal imaging system provided with a preferred embodiment of the present invention.

FIG. 3 is a partially exploded view of the thermal imaging optics and detector system shown in FIG.

FIG. 4 is a schematic view of the arrangement of the detector elements shown partially in FIG. 3;

FIG. 5 is a block diagram of the thermal imaging system unit shown in FIG. 2;

FIG. 6 is a schematic block diagram of a system electronic device of the thermal imaging system of the present invention.

FIG. 7 is a schematic block diagram of a system electronic device of the thermal imaging system of the present invention.

FIG. 8 is a schematic diagram illustrating the system hardware of the present invention used to control the histogram mode of operation.

FIG. 9 is a schematic diagram illustrating the system hardware of the present invention used to control the histogram mode of operation.

FIG. 10 is a flow diagram illustrating a preferred method of performing a histogram chip according to a preferred embodiment of the present invention.

Continuation of the front page (72) Inventor Christopher S. Johns Kensington Road Number 74450, Los Angeles, 90066, California, United States (56) References JP-A-1-205279 (JP, A) JP-A-5-205 20447 (JP, A) (58) Field surveyed (Int. Cl. ⁶ , DB name) G06T 1/00 H04N 7/18

Claims (8)

(57) [Claims] 1. A collects a processor for processing the data of the input video signal, the data of the inputted video signal, that collection
The data of the video signal accumulates in a format for processing by the processor, histogramming function, video line summary function, and the video line capturing <br/> that is configured to perform捉function histogram chips A histogram chip mode control device for controlling the operation of the histogram chip in each operation mode for performing the functions of the histogramming function , the video line total function , and the video line capture function. Is
A data processing system for an input video signal. 2. The system of claim 1, wherein said histogram chip mode controller is implemented through a field programmable gate array. 3. The system of claim 2, wherein said field programmable gate array includes a command register for receiving a mode control command from said processor. 4. The video line summing function is input
To the control of the gain and level of uniformity of the video signal
The system according to claim 1, which is used . Wherein said video line capture function is used to control the pixel alignment of the signal exiting processing said video <br/> O signals
The system of claim 1, wherein the. 6. The apparatus according to claim 1, wherein said histogramming function is input.
The system of claim 1, wherein used for the compression of the video signal. 7. The system according to claim 6 , wherein the input video signal is compressed from 15 bits to 8 bits. 8. The system of claim 1 further comprising a look-up table coupled to said histogram chip and said processor for loading said input video signal for processing by said processor. .