JP3225417B2 - Image processing device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus which can be used for inputting an image in a range corresponding to a field of view of a driver of a car, for example, and detecting necessary information.

[0002]

2. Description of the Related Art For example, if image information in a range corresponding to the field of view of a driver of a car is processed, the position of a white line on the road surface or the position of another vehicle traveling ahead can be recognized. By using the image processing result, the angle of the steering wheel is controlled so as to maintain the own vehicle within a predetermined traveling lane, and the distance between the own vehicle and another vehicle is maintained constant. Autopilot such as adjusting the vehicle speed to be able to do so becomes possible.

In this type of image processing, it is generally necessary to first detect a feature point indicating the contour position of an image.
In order to detect a feature point, generally, a change amount of an image density, that is, a differential value is detected, and then the differential value is binarized by a predetermined threshold value. Further, the position of the central portion of the characteristic region having a certain width, which is indicated by the binarized differential value, is detected as the position of the characteristic point (contour) by the thinning process. When calculating the amount of change in the image density, the value (density) of the pixel of interest is referred to by referring to a plurality of pixels in an n × n pixel matrix including the pixel of interest as disclosed in, for example, JP-A-61-77965. Is used.

[0004]

When this type of image processing is performed, generally, detection processing is repeatedly performed on all pixels in an input image. Therefore, for example, 512 ×
In order to process an image composed of 512 pixels, processing for each pixel must be repeated about 260,000 times, which requires a long time. Therefore, the present applicant has already proposed that only a rectangular area is to be processed by excluding an area that does not need to be detected in advance (Japanese Patent Application No. 63-70372).
issue). However, even if the processing area is limited to such a rectangular area, the number of repetitions is reduced to only about ２ at most, and the processing still requires a long time.

Accordingly, it is an object of the present invention to provide an image processing apparatus capable of detecting feature point information in an image at high speed.

[0006]

In order to solve the above problems, an image processing apparatus according to the present invention comprises: an image memory means (27) for holding predetermined image data which can be read; a clock signal having a constant period; Clock signal generating means (13
3); at least a plurality of sets of scan table coordinates and scan end coordinates, or scan table coordinates and scan length information, which are freely readable and writable by scan table memory means (103); each scan start on the scan table memory means; Scanning address generating means (125, 128, 129) for generating scanning address information which changes in synchronization with the clock signal and for applying the scanning address information to the image memory means; Scan line end detection means (134, 135) for identifying whether or not each scan line is at the end position based on the information of each scan end coordinate or scan length. A table address generating means (121) for generating a table address which changes each time it is detected and applying it to an address terminal of said scanning table memory means; A feature point detecting means (150) for inputting time-series image data read from the image memory means in synchronization with the clock signal and identifying whether or not the image data at the scanning position is a feature point; And a feature point storage means (112) for storing scan address information output by the scan address generation means when the feature point detection means detects a feature point.

The image processing apparatus according to a second aspect of the present invention further comprises a table selecting means (106) for determining an initial value of a table address output from the table address generating means.

In a third aspect of the present invention, the scanning table memory means further comprises an area for storing scanning direction information (V / H) indicating whether a scanning direction is a vertical direction or a horizontal direction, and The means comprises first counter means (128) for counting the scanning position in the vertical direction and second counter means (129) for counting the scanning position in the horizontal direction. A clock signal is applied to one of the first counter means and the second counter means according to the scanning direction information on the scanning table memory means.

The symbols shown in the parentheses indicate the reference numerals of the corresponding elements in the embodiments described later for reference, but each component of the present invention is a specific element in the embodiments. It is not limited to only.

[0010]

In the image processing apparatus according to the present invention, when the operation is started, the image data stored in the image memory means is sequentially read out in synchronization with the clock signal and output as a time series signal. Since the scan address information applied to the image memory means changes in synchronization with the clock signal, the pixel position of the output time-series signal is, for example, 1, 2, 3, 4 in the X-axis direction (or Y-axis direction). ,... Sequentially. This time-series image data is processed by the feature point detecting means, and the feature point detecting means detects a feature point in the image.

The scanning position in the image is generated by the scanning address generation means. The scan address information output by the scan address generation means has an initial value determined by the scan start coordinates on the scan table memory means, and sequentially increases / decreases in synchronization with the clock signal. On the other hand, the scanning line end detecting means detects the end of one scanning line based on information on each scanning end coordinate or scanning length on the scanning table memory means. When the scanning line end detecting means detects the end of one scanning line,
The table address information output from the table address generating means is updated. Since this table address information is applied to the address terminal of the scanning table memory means,
When the table address information is updated, the data (scan start coordinates, scan end coordinates) read from the scan table memory means is automatically switched to the next group. Then, scanning is performed for the switched new scanning position (scanning line) in the same manner as described above, and the feature point is detected. These operations are repeated based on the data group on the scanning table memory means. For example, if data relating to 10 sets of scanning lines is written in the scanning table memory means, scanning from the scanning start coordinates to the scanning end coordinates is performed for each scanning line indicated by the 10 sets of data. Then, detection of a feature point is performed during scanning.

For example, in the case of detecting an outline of a white line (lane) in the traveling direction in an image as shown in FIG. 14, an area where the white line exists is usually limited to a certain narrow range. It is sufficient to scan only image data in a narrow range as a detection target and perform feature detection.
However, since the detection target area in this case has a complicated shape such as a trapezoid that is inclined obliquely, such a scan cannot be performed by the conventional apparatus.

However, in the case of the present invention, the scanning area is determined by a combination of scanning line information (scanning start coordinates and scanning end coordinates) written on the scanning table memory means, and there is no limitation on the shape of the scanning area. By scanning only the necessary minimum area, detection can be completed in a short time.

According to the second invention, the initial value of the table address output from the table address generating means can be changed by changing the contents of the table selecting means. Therefore, for example, if a plurality of types of scanning line information groups having different scanning shapes are written in advance in a plurality of address areas on the scanning table memory means, the table can be obtained.
The scanning shape of the scanning line information group read at the time of scanning can be instantaneously switched only by changing the contents of the cable selection means.

According to a third aspect of the present invention, the scanning tape
Whether the scanning direction is the vertical direction or the horizontal direction can be designated by the data to be written on the table memory means. For example,
When the designated scanning area has a horizontally long shape, the repetition of the horizontal scanning has a smaller number of repetitions than the repetition of the vertical scanning, so that the required processing time is shorter, and the scanning table memory means is not required. Writing data is also simplified. That is, a scanning direction suitable for the scanning shape can be selected as needed.

[0016]

DESCRIPTION OF THE PREFERRED EMBODIMENTS FIG. 1 shows the configuration of the entire apparatus according to one embodiment.
This device is mounted on a car, for example, as shown in FIG. 14, inputs an image similar to the image reflected in the driver's field of view, recognizes the input image, and automatically generates various information necessary for driving. To detect. This will be described with reference to FIG. The TV camera 22 is fixedly arranged at a position close to the driver's eyes in the vehicle interior, captures a subject in front as shown in FIG. 14, and outputs the captured image as a monochrome video signal. The video signal output from the TV camera 22 is an analog signal, and the A / D converter 24 converts the density information into an 8-bit digital signal. The digital video signal output from the A / D converter 24 is input to an image memory 27 and at the same time is applied to a TV monitor 26 via a CRT driver 25. Therefore, the TV monitor 26 can display the video input by the TV camera 22 as a two-dimensional image, similarly to a general television device. Further, a signal output from the system controller 10 is also applied to the CRT driver 25. Therefore, the information output by the system controller 10 can be
The TV monitor 2 superimposes the image input by the TV camera 22.
6 can be displayed on the screen.

The image memory 27 has an area for storing data of 8 bits per pixel, and constitutes a set of frame memories of 512 × 512 pixels. The digital video signal input by the TV camera 22 is imaged.
When writing to the memory 27, the address signal of the image memory 27 changes in synchronization with the scanning signal of the TV camera 22. That is, the horizontal direction is divided into 512 and the vertical direction is 51.
Image memory 27 associated with each position (coordinate) of the two-dimensional scanning position of the TV camera 22 divided into two.
, 8-bit pixel information at each position is written. As shown in FIG. 14, in this example, the coordinates of the upper left corner of the two-dimensional image are (0, 0: horizontal and vertical), and the coordinates of the lower right corner are (511, 511).

The system controller 10 and the high-speed image data processing unit 100 are connected to the image memory 27. Both the system controller 10 and the high-speed image data processing unit 100 Image data written in the memory 27 can be read out.
In this embodiment, mainly, the high-speed image data processing unit 100 reads out the image data written in the image memory 27 and detects the information of the characteristic points in the image. Information detected by the high-speed image data processing unit 100 is held in a memory in the high-speed image data processing unit 100, read out by the system controller 10, and used as basic data for image recognition. .

The system controller 10 has a CPU (microprocessor) 11, a ROM 12, a RAM 13, and an interface 14. Operation board 23
Is provided with a number of key switches and a display device, and issues various instructions to the system controller 10 by a key operation of a driver and displays various information output from the system controller 10. be able to.

FIG. 2 schematically shows the structure of the high-speed image data processing unit 100 shown in FIG. Referring to FIG. 2, the high-speed image data processing unit 100 includes a start register 101, a stop register 102, a scan table memory 103, an arithmetic expression switching coordinate register 104, a feature point number limit value register 105, a scan table. Position register 106, status register 107, feature point storage number memory 111, feature point memory 112, interface 11
3, an internal control circuit 120 and a feature point detection circuit 150 are provided. The system controller 10 writes data to a start register 101, a stop register 102, a scan table memory 103, an arithmetic expression switching coordinate register 104, a feature point number limit value register 105, and a scan table position register 106. The contents of the status register 107, the feature point storage number memory 111, and the feature point memory 112 can be read. Further, the system controller 10 can write data into various registers (described later) provided in the feature point detection circuit 150. The image data of the image memory 27 is applied to the feature point detection circuit 150 via the interface 113. An address signal and a control signal for reading out image data from the image memory 27 are generated by the internal control circuit 120 and applied to the image memory 27 via the interface 113.

The function of each component shown in FIG. 2 will be briefly described. The start register 101 is a register in which a signal for instructing the start of the feature point detection operation of the high-speed image data processing unit 100 is written. The stop register 102 instructs the cancellation of the feature point detection operation during operation. Is a register in which a signal to be written is written. The scanning table memory 103 is a memory for storing information (scanning pattern) such as an image area and an operation condition to be subjected to a feature point detection operation, and has a capacity capable of storing a large number of various scanning patterns. It has. Operation expression switching coordinate register 104
Is a register for holding a scanning position at which the content of an arithmetic expression for detecting a feature point in the feature point detection circuit 150 is switched. The feature point number limit value register 105 is a register that holds a value that limits the number of feature points detected on one scanning line. The scan table position register 106 is a register for holding an address value (see FIG. 16) of the scan table memory 103, which is first referred to in the feature point detection operation. The status register 107 is used for the internal control circuit 1
A register holding the status signal at 20. By reading this status signal, it is possible to know whether or not the feature point detection process is in operation and whether the process is completed normally or abnormally. When the operation of the feature point detection processing is completed, an interrupt signal is automatically sent to the system controller 10.
The feature point storage number memory 111 is a memory that holds the number of feature points detected for each scanning line. Feature point memory 112
Is a memory for storing the coordinates of each detected feature point and status information.

High-speed image data processing unit 1 shown in FIG.
3, 4, 5 and 5 show a specific configuration of 00. All the elements indicated by reference numerals other than those shown in FIG.
This corresponds to the internal control circuit 120 in FIG. In each figure, various memories,
Descriptions of strobe signals and clock signals for reading and writing to registers, counters and the like are omitted except for particularly important parts.

First, a description will be given with reference to FIG. An address bus and an output terminal of the counter 121 are connected to an address terminal of the scanning table memory 103 via a selector 122. The selector 122 is connected to the system control.
When the scanner 10 accesses the scanning table memory 103, the contents of the address bus are applied to the address terminals of the scanning table memory 103.
The output value of 1 is applied to the address terminal of the scan table memory 103. When the operation is started, a value held in the scan table position register 106 is preset in the counter 121, and this value is counted up every time a signal CHL described later is applied. On the other hand, a data bus is connected to a data terminal of the scanning table memory 103 via a bidirectional buffer 126. Therefore, the system controller 10 can write and read data to and from the scanning table memory 103.

Each of the scan table memories 103 has 3
It has a storage capacity capable of registering a large number of 2-bit data. Each of the 32 bits has a vertical scanning start coordinate value Vs to which 9 bits are allocated, a horizontal scanning start coordinate value Hs to which 9 bits are allocated, and a scanning end coordinate value to which 9 bits are allocated. L, unused 1 bit, and 4
It is composed of bit control information (MT, D / U, V / H, TE).

The function of each bit of the control information is defined as follows.

MT: information for determining an arithmetic expression to be selected first. If MT = 0, arithmetic expression A (1 × 5 matrix). If MT = 1, arithmetic expression B (1 × 7 matrix). D / U: scanning direction (down / Up) If D / U = 0 and V / H = 0, scan in the direction → If D / U = 0 and V / H = 1, scan in the direction ↓ If D / U = 1 and V / H = 0 Scan in the direction of ← If D / U = 1 and V / H = 1, scan in the direction of ↑ V / H: Information specifying the vertical and horizontal scanning directions V / H = 0, horizontal scanning V / H = 1, vertical scanning TE: information indicating whether or not the last scanning line to be referred to. If TE = 0, refer to the following scanning line. If TE = 1, end scanning. Also, the scanning end coordinate value L includes horizontal scanning when V / H = 0. The coordinate values are registered. When V / H = 1, the coordinate values of the vertical scanning are registered.

When the feature detection operation is started by writing data to the start register 101, the 32-bit parallel data registered in the scan table memory 103 is read from the start address held in the scan table position register 106. Date
Each time the data is read out and the signal CHL appears, the address is updated and the next 32-bit parallel data is read out.
This operation is repeated. This repetition operation ends when data in which the control information TE is set to 1 is read.

Of the 32-bit parallel data read from the scan table memory 103, the 9-bit vertical scanning start coordinate Vs is applied to the selector 127 and the counter 128, and the 9-bit horizontal scanning start coordinate Hs is selected by the selector. The 9-bit scanning end coordinate value L is applied to the counter 130, and the 4-bit control information is applied to the control bit register 132. Also, the 4-bit control information is latched in the control bit register 132, and the control signal generation circuit 133
Is applied to The control signal generation circuit 133 generates various control signals according to the input control information. By the control signal, the values of Vs, Hs and L are preset in the counters 128, 129 and 130, respectively. Further, when V / H = 0, Hs is applied, and when V / H = 1, Vs is applied to the counter 125 via the selector 127, and is preset in the counter 125.

A counter 128 generates a vertical coordinate value PV of the image memory 27, a counter 129 generates a horizontal coordinate value PH of the image memory 27, and
Numeral 30 generates a horizontal or vertical scanning end coordinate value L0.
Further, the counter 125 generates a vertical or horizontal scanning position CHP during the feature point detection processing.

By the way, in this embodiment, a delay for outputting a plurality of pixel data at the same timing,
Since there are delays required for various operations, delays for adjusting the timing of a plurality of operation results, and the like, the difference between the image data read from the image memory 27 and the image data targeted for feature detection is obtained. There is a time difference between them, that is, a scanning position shift. Therefore, the coordinates (CHP) of the detected feature point do not match the read address (PV, PH) of the image memory 27 at that time. Therefore, in this embodiment, a counter (128, 129) for generating a read address of the image memory 27 and a counter (125) for generating feature point detection coordinates are provided independently. First, the value Vs (or Hs) preset in the counter 125 and the value Vs (or counter 1) preset in the counter 128 are set.
Although the value Hs) preset in 29 is the same, the value of each counter is corrected by the correction signal output from the control signal generation circuit 133. The respective correction amounts are as follows.

When D / U = 0 and V / H = 0 (→ direction scanning): Counter 128: 0 (PV = Vs) Counter 129: −6 (PH = Hs−6) Counter 130: +14 (L0 = L + 14) Counter 125: -20 (CHP = Hs-20) When D / U = 0 and V / H = 1 (downward scanning): Counter 128: +6 (PV = Vs + 6) Counter 129: 0 (PH = Hs) ) Counter 130: +14 (L0 = L + 14) Counter 125: -20 (CHP = Vs-20) When D / U = 1, V / H = 0 (← direction scanning): Counter 128: 0 (PV = Vs) Counter 129: +6 (PH = Hs + 6) Counter 130: -14 (L0 = L-14) Counter 125: +20 (CHP = Hs + 20) When D / U = 1, V / H = 1 (↑ direction running) Inspection): Counter 128: -6 (PV = Vs-6) Counter 129: 0 (PH = Hs) Counter 130: -14 (L0 = L-14) Counter 125: +20 (CHP = Vs + 20) According to the above correction amount Thus, the control signal generation circuit 133 outputs a control signal. Actually, the counter 125 has an up / down switching signal UDC corresponding to the sign (+/-) of the correction amount.
Is applied, a clock pulse of the number of pulses corresponding to the correction amount is applied as a signal CLK, and an up / down switching signal UDV corresponding to the sign (+/-) of the correction amount is applied to the counter 128, and then the correction is performed. A clock pulse of the pulse number corresponding to the amount is applied as the signal CKV, and the counter 129
After applying an up / down switching signal UDH corresponding to the sign (+/-) of the correction amount, a clock pulse of the number of pulses corresponding to the correction amount is applied as a signal CKH. After the application of the up / down switching signal UDL according to (+/-), the number of clock pulses corresponding to the correction amount is applied as the signal CKL.
The counters 125, 128, 129 and 130 are all up / down counters, each of which increases or decreases a preset value according to an applied signal.

When the above preparation operation is completed, the control signal generation circuit applies predetermined signals to the counters 125, 128 and 129 according to the direction signals (D / U and V / H) to start scanning. . The contents of the count of each counter are as follows.

When D / U = 0 and V / H = 0 (→ direction scanning): Counter 128: Stop Counter 129: Increment (PH ← PH +
1) Counter 125: Increment (CHP ← CHP)
+1) When D / U = 0, V / H = 1 (downward scanning): Counter 128: Increment (PV ← PV +
1) Counter 129: Stop Counter 125: Increment (CHP ← CHP)
+1) When D / U = 1, V / H = 0 (← direction scanning): Counter 128: Stop Counter 129: Decrement (PH ← PH-1) Counter 125: Decrement (CHP ← CHP−)
1) When D / U = 1 and V / H = 1 (↑ scanning): Counter 128: Decrement (PV ← PV-1) Counter 129: Stop Counter 125: Decrement (CHP ← CHP−)
1) In practice, controlling the signals UDV, UDH and UDC,
After setting the increment / decrement of each counter, for the counter that counts, the internally generated continuous clock pulse is applied to the signal lines CKV, CKH and C
Apply to LK. Therefore, one of the counters 128 and 129 and the counter 125 count up or down in synchronization with the clock pulse, and the coordinate values PV or PH and CHP change continuously. Therefore, from the image memory 27, data of a pixel group continuous in the designated direction is sequentially read from the pixel at the position of the scanning start coordinates (Vs, Hs) written in the scanning table memory 103. Is read out.

On the other hand, the comparator 134 compares the coordinate value PH output from the counter 129 with the scan end coordinate value L0 output from the counter 130, and when they match, a match signal EQ is output.
H is output. The comparator 135 compares the coordinate value PV output from the counter 128 with the scan end coordinate value L0 output from the counter 130, and when they match, a match signal EQ.
Output V. The end of scanning of one scanning line can be detected by these coincidence signals EQH or EQV. When the scanning of one scanning line is completed, a signal CHL appears, the value of the counter 121 is incremented, and the read address of the scanning table memory 103 changes, so that the information of the next scanning line is read from the scanning table memory 103. And the above operation is repeated. Next, a description will be given with reference to FIG. The image data read from the image memory 27 is input to the feature point detection circuit 150 via the buffer 146 for each pixel in synchronization with the clock pulse ACLK, and for each pixel, whether or not it is a feature point is determined. Are identified instantaneously, ie in real time. The feature point detection circuit 150
It outputs a signal CH indicating whether or not it is a feature point and a status signal. The feature point signal CH is applied to the respective count input terminals of the counters 144 and 149. The count output of the counter 144 is applied to the address terminal of the feature point memory 112 via the buffer 141. The data of the feature point memory 112
The coordinate information DV and DH and the status signal output from the feature point detection circuit 150 are applied to the data terminal via the latch 136. These information are synchronized with the generation of the feature point signal CH. The data is written to the feature point memory 112. As shown in FIG. 3, the coordinate information DV is PV or CHP,
The selector 123 selects one according to the direction signal V / H. The coordinate information DH is PH or CHP,
The selector 123 selects one according to the direction signal V / H.

On the other hand, the counter 149 outputs the characteristic point signal CH
And the number of feature points of each scanning line is counted. Comparator 1
Reference numeral 48 compares the number of feature points detected by the counter 149 with the value of the feature point number limit value register 105, and outputs a match signal EQM when they match. Or gate 147
E selected according to the coincidence signal EQM and the direction signal V / H
When either QH or EQV appears, it outputs a scan line end signal CHL. The counter 145 counts the scanning line end signal CHL. The output value (scanning line number) of the counter 145 is applied to the address terminal of the feature point storage number memory 111 via the buffer 143. The data terminal of the feature point storage number memory 111 has a counter 14
The output value of 4 (the number of feature points detected in each scanning line) is applied via the buffer 142. That is, the feature point storage number memory 111 stores the number of feature points detected in each scanning line at independent addresses.

The address terminal of the feature point memory 112 is connected to an address bus via a buffer 138, and the data terminal is connected to a data bus via a bidirectional buffer 137. An address terminal of the feature point storage number memory 111 is connected to an address bus via a buffer 140, and a data terminal is connected to a data bus via a bidirectional buffer 139. Therefore, the system controller 10 can read information of all the feature points held in the feature point memory 112 after the feature point detection operation is completed, and can be held in the feature point storage number memory 111. The number of feature points of all the scanning lines can be read.

FIG. 5 shows the configuration of the feature point detection circuit 150 shown in FIG.
Shown in Referring to FIG. 5, input image data is
Through the shift register 153, two types of operation units 154
And 155 respectively. The determination unit 156 generates a feature point signal CH and a status signal based on one of the information output from the calculation units 154 and 155. Which of the two sets of information is adopted is determined by a signal output from the size switching circuit 158. One arithmetic unit 154 performs an arithmetic operation by referring to five consecutive pixel regions in the scanning direction, and the other arithmetic unit 155 performs an arithmetic operation by referring to seven continuous pixel regions in the scanning direction. The calculation unit 154 is suitable for detecting a feature point in a region where the density gradient is relatively large, and the calculation unit 155 is suitable for detecting a feature point in a region where the density gradient is relatively small. Feature point detection circuit 15
0 indicates a threshold register group 1 connected to the data bus.
51, 157, a mode register 152, and an arithmetic expression switching coordinate register 104. These registers store values required by the system controller 10, respectively.

FIG. 6 shows a specific configuration of the shift register 153 and the arithmetic unit 154 shown in FIG. 5, and FIG. 7 shows a specific configuration of the arithmetic unit 155. First, a description will be given with reference to FIG. The input image data in which each pixel is composed of 8 bits is supplied to the shift register 15 in synchronization with the clock pulse (ACLK).
3 are sequentially applied. In this example, shift register 1
Reference numeral 53 denotes a 10-stage flip-flop (indicated by F / F), from which signals can be extracted. That is, serial image data sequentially input for each pixel is applied to the input of the shift register 153, but image data of consecutive 10 pixels is output from the shift register 153 in parallel at the same timing. Date
Can be output as data.

The operation unit 154 shown in FIG. 6 is configured to output G-2 and G of the image data output from the shift register 153.
-1, G + 1, G + 2, G + 3, G + 4, G + 5 and G
Eight sets of +6 are input and used. In FIG. 6, each component of the arithmetic unit 154 (all except 153)
Are described in a size and a position corresponding to the clock timing so as to coincide with the time delay and the processing timing. For example, the adder 211 requires two clocks from the input to the output of the signal, and the subtractor 232 requires three clocks from the input to the output of the signal. The output signal of the adder 211 and the output signal of the adder 212 appear at the same timing. Accordingly, as can be seen from FIG. 6, a large number of signals AJR output from the arithmetic unit 154 are output.
U, AJRL, AJTL, AJTE, AJHL, AJH
E, ADI9, AJDN, AJDP, AJSU, AJS
L, AJWU and AJWL all appear simultaneously for one pixel of interest during scanning.

In FIG. 6, threshold values ARGU, ARGL, ASGU, ASGL, A are applied to each comparator.
WGU, AWGL, ADIN, and ADIP are values held by each register included in the threshold register group 151, respectively.

In this embodiment, it is necessary to recognize the shadow or white line of the automobile reflected on the road from the image as shown in FIG. Therefore, the arithmetic unit 154 basically includes a circuit for detecting a change in density (density differential value) with reference to density data of five consecutive pixels, a circuit for detecting the density of the area, and a circuit for detecting five pixels. A circuit is provided for detecting the density at the scanning position before the area.

Actually, of the data of five consecutive pixels G + 2, G + 1, G, G-1, and G-2, G + 2 and G
+1 is added by the adder 213, G−1 and G−2 are added by the adder 214, and the output of the adder 213 is subtracted from the output of the adder 214 by the subtractor 232, and the density differential value (DIFF
= ((G-1) + (G-2))-((G + 2) + (G + 1))). Also, referring to the output value of the adder 214, 5
The density of the pixel area is identified. Further, the values of four pixels G + 6, G + 5, G + 4 and G + 3 before the five-pixel area are added by the adders 211, 212 and 231 to detect the density before the point of interest G.

The comparators 233 and 234 are provided to determine whether or not the density of the five-pixel area is within a density range that can be regarded as a shadow portion of the vehicle. The comparators 235 and 236 have five pixels. The comparators 251 and 252 are provided to identify whether the density of the area is within a density range that can be regarded as a white line portion on the road surface. It is provided to identify whether or not the concentration is within the recognizable concentration range. ASGU
And ASGL are the upper and lower limits of the density range of the shadow portion, respectively, and AWGU and AWGL are the upper and lower limits of the density range of the white line portion, respectively.
RGU and ARGL are the upper limit value and the lower limit value of the density range of the road surface portion, respectively.

The reason for referring to not only the density differential value but also the density of the attention area is to prevent occurrence of an error in feature point detection. For example, when scanning the vicinity of the center of the image as shown in FIG.
First, a large density differential value is obtained in the shadow area below the vehicle. However, for example, if a mark indicating the inter-vehicle distance is displayed on the road surface between the own vehicle and the vehicle in front, a large density differential value is obtained at the inter-vehicle distance mark first. If only the differential value is referred to, it is not possible to distinguish between a mark on the road surface and a shadow. By checking whether or not the density is within a predetermined range, the road surface, the white line, the shadow, and the like can be distinguished without error.

The comparators 261 and 262 are provided with a subtractor 232
Output the density differential value DIFF output by the threshold AD
As compared with IN and ADIP, the direction of the density change and whether or not the change amount is sufficiently large is identified. That is, the comparator 26
No. 2 discriminates whether or not the density change (differential value) when the density changes from dark to light is larger than the positive threshold ADIP. The comparator 261 determines whether the density is light to dark. It is determined whether or not the amount of density change in the case of the change in the direction is larger than the threshold value ADIN on the negative side (in absolute value).

When a density differential value obtained by scanning an image is binarized by a comparator, for example, as shown in FIG. 13, a portion where a feature is detected has a certain width (length). become. For example, FIG. 21 shows an image obtained by inputting an image similar to that shown in FIG. 14, scanning the density upward from the bottom, detecting the density differential value, and binarizing the density differential value on a two-dimensional screen. However, on this two-dimensional screen,
The feature information is output as a set of lines. Therefore, in order to detect the actual feature point, that is, the position of the contour, it is necessary to perform a thinning process on the binarized feature information. However, since thinning by software processing usually takes a very long time, in this embodiment, processing to make thinning unnecessary is executed inside the feature point detection circuit 150 as described below. ing.

The arithmetic unit 154 includes a flip-flop 253 that outputs the density differential value DIFF output from the subtractor 232 with a delay of one clock, and a flip-flop 26 that outputs the signal (Z) output from the subtracter 232 with a further delay.
3 are provided, and the density differential value at each position (Z + 1, Z, Z-1) of three pixels appearing continuously in the scanning direction is obtained at the same timing. Then, the comparator 271 compares the density differential value at the pixel position Z + 1 with the density differential value at the pixel position Z to generate two signals AJTL and AJTE, and the comparator 272 generates the density differential value at the pixel position Z. And the density differential value at the pixel position of Z-1 are compared to generate two signals AJHL and AJHE. That is, in the embodiment shown in FIG.
Based on JTL, AJTEA, JHL and AJHE,
The change shape of the density differential value DIFF is examined, and the top of the peak and the bottom of the valley of the change appearing in DIFF are detected as contour positions. Note that the remaining many flip-flops 201 are delay elements provided to make the timings at which many signals are output coincide with each other.

The operation section 155 will be described with reference to FIG.
In the arithmetic unit 155, of the image data output from the shift register 153, G-3, G-2, G + 2, G +
Seven sets of 3, G + 4, G + 5 and G + 6 are input and used. Also, as in FIG. 6, each component of the arithmetic unit 155 illustrated in FIG. 7 is described in a size and a position corresponding to a clock timing so as to coincide with a time delay and processing timing. Accordingly, as can be seen from FIG. 7, a large number of signals BJRU, BJ
RL, BJTL, BJTE, BJHL, BJHE, BD
I9, BJDN, BJDP, BJSU, BJSL, BJ
WU and BJWL all appear simultaneously for one pixel of interest during scanning. In FIG. 7, threshold values BRGU, BRGL, BSGU, BS applied to each comparator are shown.
GL, BWGU, BWGL, BDIN and BDIP
Each is a value held by each register included in the threshold register group 157.

The arithmetic unit 155 basically has seven consecutive
Density change (density differential value) with reference to pixel density data
And a circuit for detecting the density of the area,
A circuit is provided for detecting the density at the scanning position before the seven pixel area.

In practice, G + 3, G + 2, G + 1, G,
G + 2 and G + 3 of the data of seven consecutive pixels G-1, G-2 and G-3 are added by the adder 217, and G-2
And G-3 are added by the adder 218, and the output of the adder 217 is subtracted from the output of the adder 218 by the subtractor 238, and the density differential value (DIFF = ((G-3) + (G-2)) − ((G + 2)
+ (G + 3))). Further, the density of the seven-pixel area is identified with reference to the output value of the adder 218. Further, G + 6, G + 5, G + 4, and G before the 7-pixel area
The values of the four pixels of +3 are added by the adders 215, 216 and 237 to detect the density before the point of interest G.

The comparators 221 and 222 are provided for identifying whether or not the density of the seven-pixel area is within a density range that can be regarded as a shadow portion of the vehicle. The comparators 223 and 224 are provided with seven pixels. The comparators 254 and 255 are provided to identify whether or not the density of the area is within a density range that can be regarded as a white line portion on the road surface. It is provided to identify whether or not the concentration is within the recognizable concentration range. BSGU
And BSGL are the upper and lower limits of the density range of the shadow portion, respectively, and BWGU and BWGL are the upper and lower limits of the density range of the white line portion, respectively.
RGU and BRGL are the upper limit value and the lower limit value of the density range of the road surface portion, respectively. The configuration and operation of the remaining circuit are the same as those of the arithmetic unit 154 described above.

The specific configuration of the determination unit 156 in FIG. 5 is shown in FIGS. 8, 9 and 10. First, referring to FIG. 8, in the upper part of the circuit, the signals AJDP, AJDN, AJRU, AJRL, AJS output from the arithmetic unit 154 are provided.
A signal AJG is generated based on L, AJSU, ADI9, AJWL, and AJWU and the mode signals G, P, and N. In the lower half, signals BJDP, BJDN, BJRU, BJRL, BJSL,
A signal BJG is generated based on BJSU, BDI 9, BJWL and BJWU and mode signals G, P and N.

The mode signals G, P and N are signals output from the mode register 152 shown in FIG. 5, and determine the feature point detection conditions. The relationship between the value (mode value) set in the mode register 152 and the detection condition is as follows.

Mode value 0001 (binary display): Detects density change (differential value) from dark to light direction Density value is not considered Mode value 0010: light to dark direction Mode value 0011: Detects both density changes in the direction from dark to bright and in the direction from light to dark. Density value is not considered. Mode value 0101 In case of: Detect density change (differential value) from dark to light direction Judgment taking into account density value In case of mode value 0110: Detect density change from light to dark direction (differential value) In the case of mode value 0111: detection of both density changes in the direction from dark to light and in the direction from light to dark Judgment in consideration of density values In this embodiment, the above six types of modes are used. Although only the mode switching is possible, for example, a large number of No. AJRU, AJRL, AJTL, AJTE, A
JHL, AJHE, ADI9, AJDN, AJDP, A
JSU, AJSL, AJWU and AJWL, and arithmetic unit 1
Numerous signals BJRU, BJRL, BJT output by 55
L, BJTE, BJHL, BJHE, BDI9, BJD
N, BJDP, BJSU, BJSL, BJWU and BJ
If the configuration is changed so that whether to ignore each WL can be individually determined by the output signal of the mode register 152, characteristics can be detected under various kinds of conditions.

A description will be given with reference to FIG. This circuit includes signals AJTL, AJTE, AJH output from the arithmetic unit 154.
L, AJHE, and ADI9, and signals BJTL, BJTE, BJHL, BJHE, and BD output by the arithmetic unit 155.
I9 and signals AJG and BJ output from the circuit of FIG.
G and the control signal FMT, the characteristic point determination signal CH
Generate

The control signal FMT is a signal output from the size switching circuit 158 in FIG. In the circuit shown in FIG. 9, the arithmetic expression used for detecting the feature point is switched according to whether the control signal FMT is 0 or 1. Specifically, the control signal FM
When T is 0, determination of a feature point is performed using a signal group output from the arithmetic unit 154, and when TMT is 1, determination of a feature point is performed using a signal group output from the arithmetic unit 155. Is carried out. That is, when the control signal FMT is 0, the size of the pixel region referred to for detecting the feature point is 1 × 5 pixels, and when the control signal FMT is 1, the size is 1 × 7 pixels.
Pixel.

For example, as shown in FIG. 14, there are various objects from a short-distance subject to a far-distance subject in an image for feature point detection, and in the image of FIG. In the area on the side, a subject at a short distance tends to be displayed, and in the area on the upper side, a subject at a long distance tends to be displayed. In addition, a subject at a short distance occupies a larger area in an image than a subject at a long distance. Thus, for example, when scanning an image from bottom to top to detect a shadow reflected on the ground of a vehicle traveling ahead, a shadow area is detected in a large image area when the distance of the vehicle is short. If the distance is long, it is detected in a small image area.
The density gradient (density change with respect to the unit position change) tends to be relatively small when the distance of the vehicle is short and large when the distance is long.

Therefore, if the size of the pixel area to be referred to for detecting a feature point is relatively small, a subject at a short distance with a small density gradient may not be detected as a feature point. If the size is relatively large,
A subject at a long distance with a small area may be missing and not detected as a feature point.

FIG. 15 shows an image as shown in FIG. 14, in which two types of image information of a case where the distance to a vehicle traveling ahead is short and a case where the distance is long are inputted. Each density distribution at the time of scanning is shown. FIG.
It can be seen from FIG. 7 that the shadow area reflected on the lower end of the front vehicle is large when the distance to the front vehicle is short, and is small when the distance to the front vehicle is long. Therefore, in order to reliably recognize the shadow of the lower vehicle, the characteristics of the small pixel area and the characteristics of the large pixel area must be detectable. In order to cope with such a change in the size of the detection target region in the image, the size of the reference pixel region is switched by the control signal FMT.

FIG. 11 shows the configuration of size switching circuit 158 for generating control signal FMT. Referring to FIG. 11, the control signal FMT is output from the flip-flop FF1. The state of the flip-flop FF1 is initially the signal MT
, And then switched by the signal CNG output from the comparator CP1. That is, the control information M included in the information of each scanning line on the scanning table memory 103
The value of T is read from the scan table memory 103,
A signal which is applied to the size switching circuit 158 via the control bit register 132 to control the preset and clear terminals of the flip-flop FF1 is generated, and
The initial state of F1 is set. The signal OEs is a scanning
This is an enable signal for a bull memory output, and is applied to the size switching circuit 158 together with the MT. For example, when scanning the image shown in FIG. 14 from bottom to top, 1 is set in the control information MT and FF1 is preset, so that the control signal FMT becomes 1 and the pixel area size of 1 × 7 is Selected. The comparator CP1 compares the scanning position information CHP output by the counter 125 with the value held by the arithmetic expression switching coordinate register 104, and when they match, a signal C
NG is output. When the signal CNG appears, the state of the flip-flop FF1 is inverted, and the control signal FMT is inverted. That is, the scanning position is designated (register 15
9), when MT is 1, FMT changes from 1 to 0.
When MT is 0, FMT is inverted from 0 to 1.

In the circuit portion shown in FIG. 10 of the judgment section 156, a large number of signals AJRU, AJ
RL, ADI9, AJDN, AJDP, AJSU, AJ
SL, AJWU and AJWL, and a large number of signals BJRU, BJRL, BDI9, BJD output from the arithmetic unit 155
N, BJDP, BJSU, BJSL, BJWU and BJ
Various status signals are generated based on the WL and the control signal FMT. These status signals are written into the feature point memory 112 as feature point information together with coordinate information. The contents of the information indicated by each status signal are as follows.

N: Result of comparison between density differential value DIFF and differential threshold value (ADIN, BDIN) P: Density differential value DIFF and differential threshold value (ADIP,
Rb: Density of the scanning position before the point of interest and the road surface upper limit threshold value (ARGU, BRGU) Rb: Density of the scanning position before the point of interest and the road surface lower limit threshold Results of comparison with values (ARGL, BRGL) Sw: Density of target area and shadow area upper threshold (ASG)
U, BSGU) Sb: Density of attention area and shadow area lower threshold (ASG)
Lw, BSGL) Ww: Density of attention area and white line area upper threshold (AWG)
(U, BWGU) Wb: Density of attention area and white line area lower threshold (AWG)
L, BWGL) Ds: sign of density differential value DIFF As described above, high-speed image data processing unit 10
In the case of 0, scanning is performed in accordance with the data written in the scanning table memory 103, and detection of a feature point is performed. The scanning area (window) is determined by a set of scanning line information including scanning start coordinates, scanning end coordinates, and scanning direction information. Therefore, the feature points can be detected only within the range of the special shape as shown in FIG. When such scanning is performed, the amount of data to be processed is greatly reduced, and the time required for feature point detection is reduced.

The area to be scanned (window) is determined by the scanning line information from the address specified by the scanning table position register 106 to the position where the table end mark (TE = 1) exists. In the scan table memory 103, a plurality of sets of data of various scan patterns having different positions and shapes can be registered in advance, and only the contents of the scan table position register 106 are rewritten. Thus, the scanning pattern can be instantaneously switched.

For example, when detecting a characteristic point of a white line portion from an image as shown in FIG. 14, the direction and the position of the white line on the screen change according to the road condition and the driving condition of the automobile. If only the range is defined as the scanning window, the white line cannot be detected unless the range exactly matches the actual position of the white line.

Therefore, in this embodiment, when detecting a characteristic point of a white line, data of nine types of windows whose positions and / or inclinations are different from each other as shown in FIG.
An appropriate window is previously registered in the scanning table memory 103, and an appropriate window is selected by the processing shown in FIG. 17 based on the white line information detected in the previous recognition processing, and the head of the data of the selected window is selected. The address is set in the scan table position register 106.

Referring to FIG. 18, windows 1 to 9
Have a parallelogram shape as shown in FIG. 19, and the windows 1, 2, and 3 have their right sides at the center position x0 of the screen.
It is determined to match. Windows 4, 5 and 6
Respectively correspond to those moving windows 1, 2, and 3 slightly to the left, and windows 7, 8, and 9 correspond to moving windows 1, 2, and 3, respectively, slightly to the right. The windows 1, 2 and 3 have different inclinations. Windows 4, 5 and 6 and window 7,
8 and 9 are similar. Therefore, by switching the windows 1 to 9, it is possible to cope with a change in the direction or position of the white line on the screen.

In FIG. 18, yU1, yU2, y
U3, yU4, yU5, yU6, yU7, yU8 and y
U9 has windows 1, 2, 3, 4, 5, 6, respectively
Y coordinate of upper left side of 7, 8, and 9, yL1, yL2, y
L3, yL4, yL5, yL6, yL7, yL8 and y
L9 is window 1, 2, 3, 4, 5, 6, respectively
The y-coordinates xL1, xL4, and xL7 on the lower left side of 7, 8, and 9 are x values on the left side of windows 1, 4, and 7, respectively.
The coordinates xR1, xR4 and xR7 are the x coordinates of the right side of windows 1, 4 and 7, respectively. Further, xV is the x coordinate of the intersection of the left white line and the right white line detected in the previous recognition process, and yLW is the y coordinate of the intersection between the left white line detected in the previous recognition process and the right side of the window. Coordinates.

In the processing of FIG. 17, first, xV <(xR1
+ XR4) / 2, and if yes, 1 is stored in the register NU; if no, xV> (x
R1 + xR7) / 2 is identified, and if yes, 2 is stored in the register NU, and if no, 0 is stored in NU. Subsequently, it is determined whether or not yLW <(yL1 + yU2) / 2. If yes, 1 is stored in the register NL. If no, yLW> (yL2 + yU3).
/ 2, and if yes, stores 3 in NL; if no, stores 2 in NL. And NU
The result of the calculation of × 3 + NL is stored in the register No. The value of No is the window number to be selected.

[0069]

As described above, according to the present invention, the scanning area is determined by a combination of scanning line information (scanning start coordinates and scanning end coordinates) written on the scanning table memory means. There is no limitation on the shape, and the detection can be completed in a short time by scanning only the necessary minimum area.

According to the second aspect, the initial value of the table address output from the table address generating means can be changed by changing the contents of the table selecting means. Therefore, for example, if a plurality of types of scanning line information groups having different scanning shapes are written in advance in a plurality of address areas on the scanning table memory means, the table can be obtained.
The scanning shape of the scanning line information group read at the time of scanning can be instantaneously switched only by changing the contents of the cable selection means.

According to the third aspect of the present invention, the scanning tape
Whether the scanning direction is the vertical direction or the horizontal direction can be designated by the data to be written on the table memory means. For example,
When the designated scanning area has a horizontally long shape, the repetition of the horizontal scanning has a smaller number of repetitions than the repetition of the vertical scanning, so that the required processing time is shorter, and the scanning table memory means is not required. Writing data is also simplified. That is, a scanning direction suitable for the scanning shape can be selected as needed.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of an entire apparatus according to an embodiment.

FIG. 2 is a block diagram schematically showing a configuration of a high-speed image data processing unit 100 of FIG.

FIG. 3 is a block diagram showing the configuration of a part of the high-speed image data processing unit 100 in detail.

FIG. 4 is a block diagram showing the configuration of the remaining portion of the high-speed image data processing unit 100 in detail.

FIG. 5 is a block diagram showing a configuration of a feature point detection circuit 150 of FIG.

FIG. 6 is a block diagram illustrating a configuration of a calculation unit 154 of FIG.

FIG. 7 is a block diagram illustrating a configuration of a calculation unit 155 of FIG.

8 is a block diagram showing a configuration of a part of a determination unit 156 of FIG.

FIG. 9 is a block diagram illustrating a configuration of a part of a determination unit 156 of FIG. 5;

FIG. 10 is a block diagram showing a configuration of a remaining part of the determination unit 156 of FIG.

FIG. 11 is a block diagram showing a configuration of a size switching circuit 158 of FIG.

FIG. 12 is a graph showing detection conditions for APEAK and ADIP.

FIG. 13 is a waveform diagram showing the contents of a conventional feature detection process.

FIG. 14 is a plan view illustrating an example of a two-dimensional image to be a feature detection target.

FIG. 15 is a graph showing the density distribution in the vertical direction in two types of images.

FIG. 16 is a map showing an example of the contents of a scanning table memory.

FIG. 17 is a flowchart showing a scan window selection process.

FIG. 18 is a plan view showing shapes and positions of nine types of windows used for white line detection.

FIG. 19 is a plan view showing an example of a scanning pattern.

FIG. 20 is a plan view showing a pixel region of interest of a filter for detecting a density differential value and the contents of calculation.

FIG. 21 is a plan view showing, on a two-dimensional screen, the result of binarizing the density differential value of image data with a predetermined threshold value.

[Explanation of symbols]

10: System controller 11: CPU 12: ROM 13: RAM 14: Interface 22: TV camera 23: Operation board 24: A / D converter
Table 25: CRT driver 26: TV monitor 27: Image memory 100: High-speed image data processing unit 101: Start register 102: Stop register 103: Scanning table memory 104: Arithmetic expression switching coordinate register 105: Limit of the number of feature points Value register 106: Scan table position register 107: Status register 111: Feature point storage number memory 112: Feature point memory 113: Interface 120: Internal control circuit 121, 125, 128, 129, 130: Counters 122, 123, 124, 127: selector 126: bidirectional buffer 132: control bit register 133: control signal generation circuit 134, 135: comparator 136: latch 141, 146: buffer 144, 145, 149: counter 148 : Comparator 47: CHL generation circuit 150: feature point detection circuit 151, 157: threshold register group 152: mode register 153: shift register 154, 155: arithmetic unit 156: determination unit 158: size switching circuit 201, 253, 263: Flip-flops 211, 212, 213, 214, 215, 216, 2
17, 218, 231, 237: adder 232: subtractor 221, 222, 223, 224, 233, 234, 2
35,236,251,252,254,255,26
1,262,271,272: Comparator

──────────────────────────────────────────────────続 き Continuation of the front page (56) References JP-A-62-139057 (JP, A) JP-A-2-112075 (JP, A) JP-A-58-189763 (JP, A) JP-A-58-18976 189780 (JP, A) JP-A-3-280172 (JP, A) JP-A-1-33673 (JP, A) JP-A-3-99594 (JP, A) JP-A-61-118888 (JP, A) JP-A-3-95689 (JP, A) JP-A-61-150080 (JP, A) (58) Fields investigated (Int. Cl. ⁷ , DB name) G06T 1/20 G06T 1/60 450 G06T 3 / 00 400 G06T 7/00 250 G06T 7/60 200

Claims (3)

(57) [Claims] 1. An image memory means for holding predetermined readable image data; a clock signal generating means for outputting a clock signal having a constant period; at least a plurality of sets of scanning start coordinates and scanning end coordinates, or Scanning table memory means capable of freely reading and writing information on scanning start coordinates and scanning length; an initial value is determined according to each scanning start coordinate on the scanning table memory means, and a scanning address which changes in synchronization with the clock signal Scanning address generating means for generating information and applying the information to the image memory means; based on information on each scanning end coordinate or scanning length on the scanning table memory means;
Scanning line end detecting means for identifying whether or not each scanning line is at the end position; generating a table address which changes each time the scanning line end detecting means detects the end position of the scanning line; Table address generating means for applying to an address terminal of the table memory means; input of time-series image data read from the image memory means in synchronization with the clock signal; Image processing comprising: feature point detecting means for identifying whether a point is a point; and feature point storing means for storing scanning address information output by the scanning address generating means when the feature point detecting means detects a feature point. apparatus. 2. The image processing apparatus according to claim 1, further comprising table selection means for deciding an initial value of a table address output from said table address generation means. 3. The scanning table memory means further includes an area for storing scanning direction information indicating whether a scanning direction is a vertical direction or a horizontal direction, and the scanning address generating means counts a scanning position in the vertical direction. A first counter for counting the scanning position in the horizontal direction; a second counter for counting the horizontal scanning position, wherein the clock signal generating means performs the first scanning in accordance with the scanning direction information on the scanning table memory during the scanning. The image processing apparatus according to claim 1, wherein a clock signal is applied to one of the counter unit and the second counter unit.