JPH0140379B2 - - Google Patents

Description

[Detailed description of the invention]

(Field of Application of the Invention) The present invention extracts features obtained from an image, such as end points, intersections, branch points, etc., or points that most match a template in pattern matching, and detects their number, position, etc. The present invention also relates to an image processing apparatus that performs pattern recognition of characters, objects, etc. based on this, and particularly relates to an image processing apparatus suitable for detecting positional information such as the above-mentioned features and matching points at high speed. (Background of the Invention) An example of processing for extracting features from an image by a conventional image processing device is shown in FIGS. 1 and 2. FIG. 1 shows that end points 11 to 14, which are geometric feature points, and an intersection point 15 are extracted from an X-shaped image 10 as shown in the figure. The feature extraction processor 20 extracts end points and intersection points obtained from the image 10, converts them into numerical values, and outputs them at positions corresponding to the input image 10.
are doing. In the output 30 of this example, the end points 11 to 14
"A" is assigned to the hexadecimal number, and the numeric value (code) "B" is assigned to the intersection 15. FIG. 2 shows a pattern matching processor 60.
2 shows pattern matching processing for detecting pixel-by-pixel consistency with the template (standard pattern) 50. In the example of this figure, among the various character strings of the input image 40, the template 50 (“A” in the illustrated example)
Find the most matching character and output the corresponding output 7
It shows that matching points 71 and 72 (indicated by black circles in the figure) are output at the 0 position. In visual recognition, such as recognizing characters and objects,
It is common to perform a preprocessing process for performing feature extraction processing as described above, and a process for final recognition based on the features obtained by the preprocessing, that is, a postprocessing process. In conventional image processing devices, pre-processing is typically performed using dedicated hardware, and post-processing is performed using a general-purpose computer, and suitable hardware for post-processing has not yet been realized. . Here, we will discuss the conventional method for "detecting the position on an image of the feature obtained from pre-processing," which is one of the most important processes in post-processing and is the direct subject of the present invention. Explain how. Figure 3,
1 is a configuration block diagram of an image processing apparatus for explaining a conventional method. First, FIG. 3 will be explained. The first image memory 200 is for storing images to be processed. Similar second image memory 250
is for storing the results of image processing, that is, the digitized features. Here, for convenience of explanation, two image memories 2
Although the image memories 200 and 250 are shown as being separated, the image memories 200 and 250 may be integrated in some systems. Also, image memory 2
00 and 250 can be freely accessed with the host computer. The image processing unit 300 is for extracting features obtained from an image, converting them into numerical values, and outputting them.
This includes a conventionally known feature extraction processor 20 or pattern matching processor 60 as shown in FIGS. The address processor 100 is for outputting a read address 101 for controlling reading from the first image memory 200 and a write address 102 for controlling writing to the second image memory 250. Note that, normally, the write address 102 is an address obtained by correcting the read address 101 in consideration of processing delays by the image processing unit 300. Thereby, the features extracted from the first image memory 200 are stored in the second image memory 250 without positional deviation. During operation, an image is read from the first image memory 200 according to the read address 101 sent from the address processor 100, and the information is read out from the first image memory 200.
The data is input to the image processing unit 300 via the image memory read bus 201. The image processing unit 300 stores the obtained features in the second image memory 2 via the image memory write bus 301.
Send to 50. In the second image memory 250, the features on the image memory write bus 301 are written based on the write address 102 sent from the address processor 100. After the above pre-processing is completed, post-processing, that is, feature position detection is performed by a host computer (hereinafter abbreviated as CPU). As described above, the second image memory 250 in which features are stored is
The CPU reads out the entire area and stores all read addresses when a predetermined feature is detected. As is clear, this CPU read address corresponds to the location of the feature. Such a conventional example has the advantage of being highly versatile because the image processing apparatus has a simple configuration and can be handled by CPU software. However, on the other hand, it has the disadvantage that it takes a huge amount of time to process because it is necessary to sequentially access a large capacity image memory by the CPU. Next, FIG. 3, which is another conventional example, will be explained. The difference between this conventional example and the one in FIG. 3 is that a feature address detector 400, which is dedicated hardware, is used to detect the position of a feature on an image.
This is because we have established the following. The feature address detector 400 includes the image processing unit 30
0 is input, and when a predetermined feature is extracted, the write address 102 sent from the address processor 100 is input to the internal buffer.
This is for real-time memorization. A specific circuit example of the feature address detector 400 is shown in FIG. The main components of feature address detector 400 are:
As is clear from the figure, a pair of first-in-first-out buffers (hereinafter abbreviated as FIFO) 410, for storing the write address 102, that is, the X address and the Y address, respectively, are the position information of the feature. 420, said FIFO
410 and 420, and a binarization circuit 440 that detects only predetermined features from among various features that are digitized and output. Now, the end points 11 to 1 of the image 10 shown in FIG.
It is assumed that a position address of 4 is to be detected. As mentioned above, each end point is assigned the value "A" in hexadecimal. The feature code “A” at the end point is transferred to the image memory write bus 301 and is output to the binarization circuit 440.
is input. The binarization circuit 440 binarizes the input data Fi as follows based on the upper and lower thresholds THmax and THmin. (1) When THmin<Fi≦THmax, outputs “1”. (2) Outputs “0” when FiTHmin or Fi > THmax. The upper and lower thresholds are determined in advance by the binarization circuit 4 via the system bus 801 under the control of the CPU.
It is given to 40. Here, for example, THmin="9" and THmax="A". When "A" is input as the input data Fi, the binarization circuit 440 outputs "1" to the binary data bus 441 according to the binarization conditions described above, so that the input data Fi becomes If the value is outside the range, "0" is output. In write controller 430, binary data bus 44
The signal on 1 is “1” and FIFO410, 42
When the input ready signals 411 and 412 sent from the IR output of 0 are each “1”, the FIFO
Load clock 431, which is a write permission signal to 410 and 420, is sent to each LDCK input. Here, input ready signals 411, 421
is a general FIFO output signal, which is
When it is “1”, it means that there is still free space in the memory area in the buffer and data can be written.
On the other hand, when it is "0", it has the opposite meaning. However, some FIFOs have the opposite meaning to the above. The write address 102 when the load clock 431 is sent is the DIN of the FIFO 410, 420.
Provided to input and written. In this case, since the images are arranged two-dimensionally, the image memory write bus 102 is separated into addresses in the X direction and addresses in the Y direction. In the configuration example shown in Figure 4, for example, the X address is FIFO
At 410, the Y address is stored in FIFO 420. The above write operation can be performed in parallel with the feature extraction process by the image processing 300 described above. The CPU reads out the position addresses of the features stored in the FIFOs 410 and 420 using a read request signal 801 included in the system bus 801.
is performed by being input to the UNCK inputs of the FIFOs 410 and 420. X position address 412 and Y position address 42 which are read output from FIFO 410, 420
2 are sent to the system bus 801 via gates 460 and 470, respectively. At this time, FIFO
Output ready signals 413 and 414, which are the OR outputs of signals 410 and 420, are sent to system bus 801 via OR circuit 450 and gate 480. Note that the output ready signal 413,
414 is also a general FIFO output signal, and when it is "1", it means that the storage section in the buffer has stored data and it is possible to read the data. When it is "0", it has the opposite meaning (although some FIFOs have the opposite meaning). Therefore, this output ready signal 413,
It is only necessary to continue reading from the CPU until the transmission of 414 is stopped. The configuration shown in FIG. 4 has the advantage that the characteristic position address can be detected at high speed simply by reading out the FIFOs 410 and 420. However, on the other hand, it has the disadvantage that it can only realize a single function of detecting a positional address due to its configuration. In addition, commonly available high-speed FIFOs are
Since the storage capacity is mainly small, there is a problem that it is not suitable for the storage section of the position address detector in the configuration shown in FIG. 4, which requires a large capacity storage element for storing position addresses. That is, for example, FIFO 410, 42 in FIG.
In order to achieve 0, it is necessary to prepare a large number of commercially available FIFOs, which also has the disadvantage of increasing costs. (Objective of the Invention) An object of the present invention is to provide an image processing device that can detect positional information of features obtained from an image at high speed while suppressing complication of structure and increase in cost. (Summary of the Invention) The present invention provides means for extracting and coding features of an original image, and a method for extracting desired specific feature points from a plurality of coded features by binarization processing. a digitization means, a labeling means for assigning individual numbers (labels) to desired and specific feature points obtained by the binarization means, and a label assigned to each feature point by the labeling means. The present invention is characterized in that a histogram processing means is provided which can extract the appearance position of each label in the image, that is, the maximum and minimum coordinates in the X and Y directions, in parallel with the processing by the labeling means. In addition, another feature of the present invention is that the histogram processing means is configured to store a predetermined maximum and minimum value in a third storage device whose address is the label data of the specified feature point, and in all addresses of the third storage device. A means for writing one of the values in advance, and a position address (coordinates) on the original image of the specified feature point corresponding to certain label data as a first input, and a means for writing one of the values in the label data in the third storage means. Therefore, the data read from the specified address is used as the second input, and the magnitude relationship between the first and second inputs is compared, and a high value or a low value is determined according to a preset function. an arithmetic unit that outputs the input; and means for supplying the output of the arithmetic unit to the data input of the third storage means and storing it at a specified address according to the label data of the specified feature point. It is in the point of composition. (Embodiment of the Invention) Hereinafter, an embodiment of the present invention will be described with reference to FIG. The main configuration of this embodiment is as listed below. (1) Image memory 280 that stores input images and image processing results. This simultaneously includes image memories 200 and 250 of the conventional configuration shown in FIG. (2) Binarization circuit 440. This is also the same as the conventional configuration shown in FIG. (3) A conventionally known feature extraction processor 500 that extracts and outputs features obtained from an image. (4) A labeling processor 600, also known in the art, which assigns individual numbers (labels) to independent objects or graphical elements. (5) A histogram processor 700 that performs various histogram processes in parallel on the processing results of the feature extraction processor 500 and the labeling processor 600, which will be described in detail later. (6) Address processor 100 similar to that described in the conventional example of FIG. (7) A CPU 800 that controls various processors and controls reading and writing of image memory. Now, in the embodiment shown in FIG. 5, assume that an image is taken in from a television camera 900, which is an image input means, and the input image is stored in the image memory 280 via the A/D converter 920. Address processor 100 sends memory address 112 to image memory 280. The memory address 112 mentioned above is the read address 10 of the image memory explained in the conventional example of FIG.
1 and write address 102 are output on the same bus in a time-division manner. In other words, the memory address 112 is set by the address processor 100 so that the memory plane in which the input image is stored is read from the image memory 280, and at the same time, the memory plane in which the image processing result is stored is written into the image memory 280. controlled from. The image data read from the image memory 280 is sent via the memory read bus 201 in advance.
The image is binarized by the binarization circuit 440 based on a threshold value for object or image identification (not feature extraction) given by a command from the CPU 800. The binary data output from the binarization circuit 440 is
First, it is sent to the feature extraction processor 500 via the binary data bus 441A. The feature extraction processor 500 extracts features obtained from the image, such as end points and intersection points, and assigns codes to the extracted features. Here, the end point is “A” in hexadecimal,
The intersection point is also assigned the code "B" in hexadecimal. Feature code data 501, which is the output of feature extraction processor 500, is temporarily written to image memory 280 via memory write bus 202. As a result, contents such as those shown at 30 in FIG. 1 are stored in an appropriate area within image memory 280. Next, the operation of the embodiment shown in FIG. 5 will be described when position information of only end points is extracted. It is now assumed that feature codes "A" and "B" obtained by the feature extraction processor 500 are stored in the image memory 280 as indicated by the reference numeral 30 in FIG. The feature code is read from the image memory 280 in the same manner as in the case of feature extraction described above. The read characteristic code is input to the binarization circuit 440 via the memory read bus 201. At this time, in the binarization circuit 440, only the end points are extracted in the binarization manner described above with respect to the conventional example shown in FIG. The extracted endpoint information is then input to labeling processor 600 via binary data bus 441B. The labeling processor 600 allocates labels equal to the number of end points and outputs label data 601. At the same time, the histogram processor 700
Label data 6 via memory write bus 202
01 and further address processor 100.
The write address 104 sent from the end point is taken in, and the appearance position information of the label assigned to the end point is stored in an internal storage unit. Table 1 shows an example of the storage format of the label number and appearance position information in the storage section in the histogram processor 700. Here, details of the histogram processor 700 will be explained with reference to FIG. The main components of histogram processor 700 are as follows. (1) Histogram memory 710 in which the results of histogram processing are stored; (2) A and B data input terminals;

[Table] 5 5 5 5
An arithmetic unit 720 that has a function of comparing the data input to A and B and outputting the larger value or smaller value (which one is selected is set in advance) to the output terminal Y as a result. , (3) Selector 730 that selects the address to be input to the histogram memory 710, (4) Selector 740 that selects the data input to be supplied to the data input terminal B of the arithmetic unit 720, (5) Based on instructions from the CPU 800. , selector 7
30,740 switching control and computing unit 720
A control register 75 that selects the calculation function of
There are roughly two operating modes of the histogram processor 700 shown in FIG. One is CPU8
00 is the system control mode, and the other is the image processing mode using the address processor 100. In the former system control mode, the following settings and processing are performed. (1) After the image processing mode ends, reading the histogram memory 710 in which the histogram processing results are extracted and stored, and before entering the image processing mode, setting data from the CPU 800 to the control register 750, and (2) ) Initializing the histogram memory 710 - For example, if the arithmetic function of the arithmetic unit 720 is specified to output the smaller value of both inputs A and B, the histogram memory 710
0, the maximum value (the data width (bit length) of the histogram must be at least the size of one side of the image, for example, if it is N bits, (2 ^N - 1)) is set in advance for all entries. Each of these operations will be explained below. First, the histogram memory 71 by the CPU 800
The process of reading 0 is performed as follows. Address selection signal 752, which is the output of control register 750, is sent to the S input of selector 730 to select data on system bus 801, which is the B input to selector 730. Meanwhile, the output control signal 755 of the control register 750 opens the gate of the output buffer 760. As a result, the histogram read data 71 which is the read data of the histogram memory 710 is
1 is placed on the system bus 801. At that time, the input control signal 754 of the control register 50
closes the gate of input buffer 770, and arithmetic unit output control signal 756 closes the gate of arithmetic unit output control buffer 780. Next, the writing process to the histogram memory 710 by the CPU 800 will be explained. Address selection signal 752 of control register 750
and arithmetic unit output control signal 756 are the same as when reading, while input control signal 754 and output control signal 755 are set to the opposite state from when reading. In the latter image processing mode, control from the CPU 800 is performed as follows. Address selection signal 752 of control register 750
selects the data on the memory write bus 202 which is the A input of the selector 730. Further, the input control signal 754 and the output control signal 755 are input to the input buffer 770 and the output buffer 76, respectively.
0 is closed, and the arithmetic unit output control signal 756 opens the gate of the arithmetic unit output control buffer 780. Next, in order to detect the feature appearance position information by the histogram processor 700, the arithmetic function of the arithmetic unit 720 selects either the smaller value or the larger value of the A input and the B input. It is set to output as follows. Here, the control register 750 is configured to select the former, that is, the smaller of both the A and B inputs.
The arithmetic unit function selection signal 753 of the arithmetic unit 720 is
Sent to input. The write address 104 sent from the address processor 100 is sent to the B input of the arithmetic unit 720. In this case, as shown in the conventional example in FIG. 4, the write address 104 includes addresses in the X-axis direction and Y-axis direction, so they are separated into the A and B inputs of the selector 740, respectively. Supplied. Therefore, in the first image processing mode, selector 7
The data selection signal 751 of the control register 750 is sent out so that the A input of 40 is selected, and the B input is selected in the second image processing mode. If there is enough hardware in the system, you can prepare two sets of histogram memory 710 and two arithmetic units 720, and assign them to each for the X-axis direction and the Y-axis direction. Since the processing is completed in the image processing mode, further speeding up can be achieved. After the above settings are completed, the histogram processor 700 shifts to image processing mode. First, label data assigned to each end point of the image is sent to the memory write bus 202 by the labeling processor 600. At the same time, a write address 104 is sent from the address processor 100 in accordance with the appearance position of the label data, and is sent to the B of the arithmetic unit 720 via the selector 740.
entered into the input. If the label data is “M”, that is, if the Mth end point is detected, “M” is sent to the ADR input of the histogram memory 710 via the histogram address bus 731. The histogram memory 710 reads the data at address M and outputs it as histogram read data 711 from the DOUT output. By the initialization process of the histogram memory 710 in the system control mode described above,
Since the maximum value (2 ^N -1) is set, this value becomes the histogram read data 711 in this case and is input to the A input of the arithmetic unit 820. Arithmetic unit 720 compares the values of A input and B input, and outputs the smaller value from Y output. Here, since the B input is always smaller than the A input, the write address 104, which is the B input, is selected and output from the arithmetic unit 720. Arithmetic unit output data 7 which is the output of the arithmetic unit 720
21 passes through the arithmetic unit output buffer 780, becomes histogram write data 712, and is input to the DIN input of the histogram memory 710. At that time, the ADR input of histogram memory 710 is indicating M. Therefore, the histogram write data 712, that is, the write address 104, which is the appearance position information (X and Y direction address) of the label M, is stored at the location (address) of M in the histogram memory 710. Histogram processor 700 shown in FIG.
Since the read modification write process of the histogram memory 710 can be completed within the data determination time of one pixel (in this case, the determination time of one label data), the feature extraction processor 500 and labeling described above can be completed. Parallel processing with the processor 600 can be realized. Another embodiment of the present invention will be described with reference to FIG. The difference from the embodiment shown in FIG. 5 is that a pattern matching processor 550 is incorporated as a component instead of the feature extraction processor 500. The pattern matching processor 550 is a conventionally known one, and the pattern matching value 551, which is the result obtained by the pattern matching processor, is the result of a pixel matching value 551 in a pixel-by-pixel comparison between the template and the input image. It is a number. As is clear, it can be said that the larger this value is, the greater the matching degree is. Next, as in the case of the embodiment shown in FIG. 5, the pattern matching value 551 is supplied to the binarization circuit 440, and only points satisfying a predetermined degree of matching are extracted. The subsequent processing is the same as the embodiment shown in FIG. That is, labeling and histogram processing are performed, and the positions of matched points are detected. (Effects of the Invention) As is clear from the above description, according to the embodiments shown in FIGS. 5 and 6, the labeling process and the histogram process after feature extraction can be executed in parallel. This has the advantage that positional information on an image of arbitrary feature points such as end points and intersections, or matched points can be detected at high speed. Another advantage is that the image processing function can be expanded simply by changing or adding functions to the arithmetic unit 720, which is a component of the histogram processor according to the embodiment of the present invention. For example, as a function of the arithmetic unit 720 described above, A
Function to increment the input value (A+1→
By adding Y), an effective image processing function can be realized, in which feature code extraction and labeling processing are performed, and at the same time, feature code and label data appearance frequency distribution extraction is performed through histogram processing. That is, the present invention includes a labeling means in addition to a means for extracting image features, and a histogram processor capable of detecting feature appearance position information in parallel with the operation of the labeling means. By providing hardware, it eliminates the need for a full-screen search of two-dimensional large-capacity image information using software processing, making it possible to quickly detect positional information of features obtained from images. There is an advantage that

[Brief explanation of drawings]

Figure 1 is a schematic diagram for explaining a feature extraction process that digitizes and outputs features obtained from an image, and Figure 2 describes a process that uses pattern matching to detect matching points with a template. 3 and 4 are block diagrams showing an example of a conventional image processing device.
The figure is a block diagram showing an example of a conventional feature position detection device, FIG. 5 is a block diagram of an image processing device that is an embodiment of the present invention, and FIG. 6 is a block diagram of a histogram processor that is a component of the present invention. FIG. 7 is a block diagram showing an example of the configuration of an image processing apparatus according to another embodiment of the present invention. DESCRIPTION OF SYMBOLS 100... Address processor, 280... Image memory, 440... Binarization circuit, 500... Feature extraction processor, 550... Pattern matching processor, 600... Labeling processor, 700... Histogram processor, 710... Histogram memory, 720 ...Arithmetic unit, 730,740...Selector, 750...Control register, 800...CPU
(Central Processing Unit).

Claims (1)

[Scope of Claims] 1. Means for extracting only pre-specified feature points from information including feature codes at corresponding position addresses on the original image; a labeling means for individually numbering feature points and outputting them as label data; 1. An image processing apparatus comprising: a histogram processing means that executes histogram processing in parallel. 2. The means for extracting only pre-specified feature points is performed by comparing the feature code with preset upper and lower thresholds, and detects when the feature code is between the upper and lower thresholds. 2. The image processing apparatus according to claim 1, wherein the image processing apparatus is a binarization means that generates only an output. 3 means for extracting only pre-specified feature points from information including feature codes at corresponding position addresses on the original image; labeling means for numbering and outputting as label data; and a histogram processing in parallel with the operation of either the feature extraction means or the labeling means, which is supplied with the label data and the write address. In the image processing apparatus, the histogram processing means includes a storage means whose address is the label data of the designated feature point, and a storage means that stores scheduled maximum and minimum values in all addresses of the storage means. A means for writing either one of them in advance, and a position address (coordinates) on the original image of the specified feature point corresponding to certain label data as a first input, and a point specified according to the label data in the storage means. The data read from the address is used as a second input,
and an arithmetic unit that compares the magnitude relationship between the first and second inputs and outputs either a high value or a low value input according to a preset function; An image processing apparatus characterized by comprising means for supplying data to a data input and storing it at a designated address according to label data of the designated feature point.