JPH0896132A - Image processor and processing method - Google Patents

Abstract

PURPOSE: To perform a high-speed matching with image data of a body to be processed which is picked up at various tilt angles by using a small number of model images. CONSTITUTION: Picked-up image data from a TV camera are registered in an image memory 8 and a tilt detection part 9a finds the tilt angle of the object. In a RAM 12, a model image having a 0 deg. tilt angle is registered. A recognition part 9b makes a serial search for the image data to make a rough search for detecting the presence position of the object in the image data and then makes a random search for the image data on the basis of the detected presence position data of the object, and makes the data match with the model image. A memory control part 7 when reading the image data out of the image memory perform affine transformation on the basis of the detected tilt to rotate the picked-up image data by a specific angle so that the data match with the angle of the model image, and outputs and sends the result to a recognition part 9b.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus and a processing method, and more specifically, to a matching process in the case where a directional object has its inclination randomly placed in an imaging area. , Improvement of access system to data in image memory for high-speed and accurate operation.

[0002]

2. Description of the Related Art In an image processing apparatus such as a visual inspection apparatus for inspecting the appearance of a product for defects, an image of an object to be inspected is picked up by a camera and the obtained image data and a model image registered in advance. And are compared (pattern recognition) to determine whether or not they match. By the way, such a visual inspection device is arranged on the carry-out side of the manufacturing line, and is designed to perform an inspection process on an inspection object that moves randomly on the transfer device, and enters the imaging area of the camera. The orientation of the incoming inspection object is not constant.

Therefore, a model image for each rotation angle obtained by rotating a model of the same shape as a comparison target (reference) at predetermined angle intervals is registered, and an image is obtained when performing the pattern recognition. The image data of the inspection object is sequentially compared with each rotated model image, and if the degree of coincidence with the model image at a certain rotation angle is high, there is an inspection object that coincides with the model image related to the angle direction. That is, it is determined that the product is non-defective, and if it does not match the model image of any rotation angle, it is determined that the product is defective.

Therefore, as a preparation for operating the visual inspection apparatus, it is necessary to register the rotation model composed of the plurality of model images. Therefore, in the past, while changing the angle of the inspection object that is a normal sample little by little, each time the image was taken by the camera, and after performing the predetermined image processing (region cutout, etc.) on each of the taken images, the model Register each model image in the memory as a pair with the angle data, or conversely, fix a normal inspection object and take the image while gradually changing the angle of the camera and perform the above processing. There is.

[0005]

However, the above-mentioned conventional apparatus has the following various problems. That is, the model image required for recognizing one type of object requires a plurality of model images for each rotation angle rotated by a predetermined angle, and requires a large memory capacity for storing them. Become. Then, the actual recognition inspection (pattern matching) is compared with a plurality of model images for each rotation angle, and if the degree of coincidence (correlation value) with any one of the model images is high, it is a normal object. Judgment is made, if the angle of the actual object to be inspected matches the angle prepared as the model image, but if they do not match, the degree of matching inevitably becomes low and there is a risk of erroneous recognition. is there. Therefore, in order to solve such a problem, it is sufficient to reduce the rotation angle interval and increase the number of models to be registered. However, if this is done, a large memory capacity for models is required, and not only the above problem becomes noticeable. However, the number of matching processes increases and high-speed determination cannot be performed.

Further, when performing the registration processing of the rotation model for each rotation angle, at least one of the camera and the inspection object is actually rotated and an image is taken each time, and a predetermined process is performed on the obtained image data. Therefore, the processing becomes complicated.

Further, since it is unknown at which position in the image pickup area the object to be processed exists, a serial search (coarse search) is performed on all the pixels of the imaged image data, and the object is processed. It is generally performed to specify an approximate location and then perform a matching process on a portion of the location by random access.
Then, a model image for serial search and a model image for random search are required, and the above problems (increase in memory capacity, speed reduction, etc.) become more prominent.

The present invention has been made in view of the above background. An object of the present invention is to solve the above problems and to process objects imaged at various tilt angles using a few model images. It is possible to perform matching processing with the image data of the model, reduce the memory capacity for storing the model image, reduce the number of times of matching, and perform high-speed processing. To provide an image processing apparatus and a processing method capable of performing accurate recognition processing by performing matching processing in a state where the angles are aligned with each other, and further reducing the number of parts and simplifying configuration and control. is there.

[0009]

In order to achieve the above object, in the image processing method according to the present invention, serial search is performed on given image data, and an object existing in the image data is searched. After performing a rough search for detecting the existence position, a random search is performed on the image data based on the existence position data of the object detected by the rough search to match the model image. Based on the image processing method, first, the inclination of the object in the given image data is detected. Then, affine transforming the image data based on the detected inclination,
After rotating at least a predetermined angle to match the angle of the object in the image data with the angle of the model image registered in advance, the rough search and matching are performed on the image data after the affine transformation.

Then, in the image processing apparatus according to the present invention, which is suitable for carrying out the above method, serial search is performed on the given image data to detect the existing position of the object existing in the image data. Rough search means, matching means for performing a random search on the image data based on the position data of the object detected by the rough search means to match with a model image, and imaged image data In an image processing apparatus provided with an image storage means for storing a model image and a model storage means for registering the model image, an inclination detection means for detecting an inclination of an object in given image data is provided, and the image storage When the image data stored in the means is read out, affine transformation is performed based on the tilt detected by the tilt detecting means, and image pickup is performed. Which was at least a predetermined angular rotation of the image data, and constituted by a memory control means to provide the converted image data to match the angle of the previously registered model image on the coarse search means and matching means.

Further, preferably, the memory control means includes an affine transformation unit for outputting an image for coarse search,
That is, the affine transformation unit for outputting an image for matching is configured to be common.

Further, preferably, the apparatus further comprises monitor means for displaying the object obtained by the image pickup, and the display mode output to the monitor means is the image data after the affine transformation. Further, the memory control means receives the coordinate value of the predetermined area on the monitor means, performs the affine transformation based on the coordinate value, obtains an address on the image storage means, and stores the image on the basis of the obtained address. It is more preferable to read the image data stored in the means.

[0013]

When the image data is picked up and the image data stored in the image storage means is read out, the memory control means performs affine transformation to read out the image data in a predetermined inclination angle direction. Then, regardless of the angle of the imaged object, the image read out from the image storage means is rotated by a predetermined angle to have the same rotation angle as the model image. Therefore, the converted image data and the model image are compared and a predetermined recognition process is performed. Therefore, at least one model image to be compared (one each for rough search and matching) may be registered. Since the number of model images used in the comparison process is small, the recognition process is performed in a short time.

Further, by appropriately setting the rotation angle at the time of performing the affine transformation, since the angle of the imaged object is matched with the angle of the model image, the rotation angle is made up of many rotation angles. The same effect as having the model image is obtained, and the recognition rate is not reduced due to the angle mismatch.

Further, if the coarse search (serial search) and the matching (random search) are both executed by an affine transformation unit composed of common hardware,
The operation speed of the affine transformation is extremely short, the image data is read out at high speed, and the entire processing time for recognition is also short.

[0016]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of an image processing apparatus and a processing method according to the present invention will be described in detail below with reference to the accompanying drawings. FIG. 1 shows a schematic configuration of an image processing system to which the present invention is applied. As shown in FIG. 1, a TV camera 1 captures an image of an object to be processed, and the obtained image data is used as an image processing device 2.
Then, a predetermined matching process is performed on the object imaged therein based on the registered model image. In addition, the imaged image data is displayed on the video monitor 3 after being subjected to predetermined image processing so that the operator can visually confirm it.

Further, as shown in FIG. 2, the inside of the image processing apparatus 2 receives an A / D signal from a video signal sent from a TV camera.
After being received by the conversion unit 5 and converted into a digital signal there,
It outputs to two video buses (video bus 0, video bus 1).

An image memory 8 is connected to each video bus via a memory control unit 7, and image data picked up by a TV camera is temporarily stored in an image memory 8 serving as an image storage unit via the memory control unit 7. It is supposed to be stored.
The image data stored in the image memory 8 is read out via the memory control unit 7 and sent to the measuring unit 9 or the screen control unit 10 via the above-mentioned predetermined video bus, or via the CPU bus. And is output to the CPU 11 or the RAM 12 as a model storage means.

The screen control unit 10 outputs a predetermined control signal for displaying on the video monitor 3 based on the image signal (digital signal) sent via the video bus. After being converted into a video signal (analog) by the A converter 13, the video signal is output to the video monitor 3 (not shown). Since the screen control unit 10 is the same as the conventional one, its detailed description is omitted. In addition, the CPU 11 is adapted to execute a predetermined process described later according to a program stored in the ROM 1.

Next, each unit will be described in detail.
Includes an inclination detection unit 9a and a recognition unit 9b. The inclination detection unit 9a obtains an inclination angle with respect to the X axis of a processing target obtained by capturing an image with a TV camera.
This is calculated by binarizing and extracting the outer shape of the object and the shape (outer shape of the pattern) of a predetermined characteristic portion inside the object, and determining the principal axis angle of the extracted shape.
Further, two characteristic portions of the object are detected, the inclination angle of the line segment connecting the detected characteristic portions with respect to the X axis is obtained, and the inclination angle of the corresponding line segment with respect to the X axis when the rotation angle is 0 degree. Various means can be used in addition to obtaining the rotation angle of the currently imaged object from the difference between

The recognition section 9b executes an image recognition process consisting of a predetermined algorithm, and compares the model image registered in the RAM 12 with the image data of the processing object stored in the image memory 18. Then, the existence position of the object in the image data is detected, and the quality of the object is determined by matching. That is, it also serves as a rough search means and a matching means.

In the present invention, the model image serving as a comparison reference when performing the matching process in the recognition unit 9b is R.
Although it is stored in the AM 12, the image data actually stored in the memory (RAM) is such that only one image having a predetermined angle (for example, a rotation angle of 0 degree) is registered for one type of model. Then, during the matching process, the coordinate system is converted (affine transformation) so that the image data of the object temporarily stored in the image memory 8 is rotated by the rotation angle detected by the tilt detection unit 9a. The matching between the converted image obtained as described above and the model image stored in the RAM 12 is obtained by the recognition unit 9b.

The memory controller 7 which is the main part of the present invention
The CPU interface (I / F) unit 20 is connected to the CPU bus as shown in FIG.
Enables bidirectional communication with the CPU 11 via the CPU bus,
Data control signals such as pixel coordinate values (coordinate values in the coordinate system after conversion) given from the CPU 11 via the CPU interface unit 20 are sent to the serial address generator (address generating unit) 21 or the random address generator (address generating unit). ) 22 to be sent.

Then, the serial address generator 2
1 is synchronized with a video bus for performing a rough search to search for an object existing position in the image memory and for displaying on a monitor, and a coordinate value on the monitor for the affine transformation unit 23 at the next stage ( X, Y) are sequentially output pixel by pixel in the raster direction from the upper left (0, 0) to the lower right.

The random address generator 22
Synchronizes with the video bus to perform final recognition processing (matching), and monitors a predetermined area (model area for performing matching processing on an object) on the monitor for the affine transformation unit 23 at the next stage. The above coordinate values are output one pixel at a time.

Further, the affine transformation unit 23 includes the CPU 11
The coordinate values given from the generators 21 and 22 are affine-transformed and stored in the corresponding image memory 8 based on the conversion parameter (rotation angle θ of the object detected by the tilt detection unit 9a) set from Find the address coordinates,
It is sent to the image memory 8 via the memory interface section 24, and the corresponding pixel data is output to the video bus.

That is, for example, when the object K is tilted as shown in FIG. 4A in the image memory 8 in which the object is imaged and stored, the object K is rotated by a predetermined angle θ. As shown in FIG. 7B, the coordinates can be converted into a position with a rotation angle of 0 degree, and the state is displayed on the monitor. Then, the coordinate values on the monitor (XA, YA
) In the image memory corresponding to the above coordinate values (X'A, Y'A)
With respect to the relation with, when the coordinates of the center of rotation are (Xc, Yc)

[0028]

[Equation 1] Becomes Therefore, when the coordinate value of one coordinate system is input based on the relational expression, the rotation angle θ is set as a parameter, and thus the coordinate value of the other coordinate system is obtained.
The affine transformation unit 23 is composed of hardware that performs a predetermined calculation based on the above relational expression.

As shown in FIG. 2, a keyboard, a mouse, and other input devices 25 are connected to the CPU bus. The input devices 25 capture image data (displayed on the monitor 3). An area (model area) to be registered as a model in the inside can be designated, and the image data of the designated area is registered in the RAM 11.

Next, while explaining the operation of the above-mentioned device,
An embodiment of the image processing method according to the present invention will be described. First, before performing the matching process, registration for returning is performed as preprocessing. That is, as shown in FIG. 5, image data obtained by capturing an image of an object serving as a comparison reference is registered in the image memory 8 (ST11). At this time, if the object is placed at the position where the rotation angle is 0 degree, the data stored in the image memory 8 is called as it is and displayed on the monitor 3.

Next, the operator looks at the monitor 3 and
Using the input device 25, a model area to be registered as a model image is designated (ST12). Then, the CPU 11
The coordinate value of the coordinate system on the monitor 3 in the designated model area is output, and the corresponding image data is copied from the image memory 8 to the RAM 11 (ST13). Note that the images registered at this time are the model image for rough search that detects the approximate location of the target object for performing the measurement process, and the model for performing the matching process such as the final matching degree of the target object. Both of the images are performed, but in each case, one image is registered in the RAM 12 for one model image (rotation angle is 0 degree). This completes the model registration process. Since the model for coarse search is for detecting and specifying the existing position of the object in the captured image data, it is a small area such as a characteristic part such as a corner of the object. Becomes This completes the model registration process.

When the object serving as the comparison reference is picked up at an arbitrary rotation angle, affine transformation is performed during serial search, which will be described later, and the object is rotated and monitored at a rotation angle of 0 degrees. 3 Display on top. Then, the operator designates the area on the monitor. Then, the CPU 11
The coordinate value on the monitor is sent to the memory control unit 7, where the image data stored at the predetermined address in the image memory is read out and registered in the RAM 12 while performing affine transformation at the time of random search described later. In any case, the operator and the CPU 11 that receives an instruction from the operator need only issue a command based on the coordinate values on the monitor.

On the other hand, in the measurement processing, the object invading the image pickup area of the TV camera 1 is sequentially picked up and stored in the image memory 8. At the same time as this image is taken in, the inclination detection unit 9a of the measurement unit 9 obtains the inclination of the object (position correction measurement) (ST14).

Next, based on the obtained inclination, the CPU
At step 11, affine transformation parameters are obtained and set in the affine transformation section 23 (ST15). And CPU
The control signal from 11 operates the affine transformation unit 23 via the serial addressless generator 21 to serialize the data in the image memory 8 with respect to the image data registered in the image memory 8 so that the rotation angle becomes 0 degree. And sends it to the video bus, displays it on the monitor 3 via the screen control unit 10, and sends data to the recognition unit 9b of the measuring unit 9. As a result, the image displayed on the monitor 3 and sent to the recognition unit 9b, regardless of the inclination of the imaged object, is displayed at a rotation angle of 0, as shown in FIG. 6A, for example.
Degrees (the same as the angle of the model image stored in the RAM 12).

Then, the recognition unit 9b performs a serial search process on the sequentially transmitted data based on the model image registered in the RAM 12 (in FIG. 6B,
(Search in the direction of the arrow), the same area as the model image existing in the image data is detected, and, for example, FIG.
The coordinate value of the candidate point P (coordinate value in the coordinate system on the monitor) is output to the CPU 11 as shown in (ST16). Note that, as shown in the figure, the image to be subjected to the serial search process always has a rotation angle of 0 degree, and therefore only one model needs to be compared, so that high-speed processing is possible and the candidate points of the object K are instantly displayed. P can be detected.

Next, when the candidate point P is detected in this way, the CPU 11 obtains the area R for random access determined based on the candidate point P, and then the area R
The coordinate values on the monitor 3 existing therein are sequentially sent to the random address generator 22. Then, based on the address data output from the address generator 22 via the affine transformation unit 23, the image data (R portion in FIG. 6D) in the image memory 8 corresponding to the region R is read and recognized. The image data is sent to the section 9b.
Then, in the recognition unit 9b, as shown in FIG.
The model is compared with the model registered in AM12, a predetermined matching process is performed, and the degree of coincidence is obtained (ST17).
Even when this matching process is performed, the target image always has a rotation angle of 0 degrees as shown in the figure, and therefore the matching process only needs to find the degree of coincidence with one model, so that high-speed recognition processing can be performed. .

Further, in this example, when performing the affine transformation for both the serial search (coarse search) and the random search, the affine transformation unit 2 composed of hardware.
Since 3 is used, it is only necessary to give the coordinate value on the image monitor from the CPU 11, so that the address in the image memory by the coordinate conversion can be obtained and read at high speed.

Moreover, since the conversion units (elements) for performing the affine conversion for both searches are commonly used, the simplification of the configuration and the cost reduction can be achieved along with the reduction of the number of parts. Even with such commonization, since the affine transformation parameters (tilt angle, etc.) in the serial search and the random search are the same, the parameter setting process is only required once, and the control algorithm is It will be simple.

[0039]

As described above, in the image processing apparatus and the processing method according to the present invention, at least the model images of the representative rotation angles need to be registered as the model images registered for one object. The memory capacity of the model storage means can be reduced.

When a predetermined image recognition process is performed on the image data picked up and stored in the image storage means, the image data obtained by performing the affine transformation on the image data is serialized. Since the search (coarse search) and the matching are respectively performed, the number of times of matching can be reduced and high-speed processing can be performed. Moreover, by appropriately setting the rotation angle when performing the affine transformation, the same effect as indirectly having a model image of an arbitrary desired angle can be obtained, and processing performed at various tilt angles is performed. The matching process with the image data of the object can be performed accurately.

Further, if the coarse search (serial search) and the matching (random search) are both executed by an affine transformation unit composed of common hardware,
The number of parts can be reduced and the configuration and control can be simplified. Moreover, the CPU side outputs the coordinates of the image on the monitor, and the memory control means performs predetermined arithmetic processing.
Moreover, since such arithmetic processing is composed of hardware, it can be processed at extremely high speed.

[Brief description of drawings]

FIG. 1 is a diagram showing a schematic configuration of a system to which an image processing apparatus according to the present invention is applied.

FIG. 2 is a block diagram showing a preferred embodiment of an image processing apparatus according to the present invention.

FIG. 3 is a diagram showing an internal configuration of a memory control unit which is a main part thereof.

FIG. 4 is a diagram illustrating an operation.

FIG. 5 is a flowchart showing an embodiment of a processing method according to the present invention.

FIG. 6 is a diagram for explaining the operation.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 TV camera 2 Image processing device 3 Monitor 7 Memory control unit 8 Image memory 11 CPU 12 RAM (model memory means) 21 Serial address generator 22 Random address generator 23 Affine conversion unit

Claims (5)

[Claims] 1. A serial search is performed on given image data, and after a coarse search is performed to detect the existence position of an object existing in the image data, the object detected by the rough search is detected. An image processing method in which a random search is performed on the image data based on existence position data to match a model image, the inclination of an object in the given image data is detected, and then, The image data is affine-transformed based on the detected inclination, and at least a predetermined angle is rotated to match the angle of the object in the image data with the angle of the model image registered in advance, and then the affine-transformed image data. An image processing method in which the rough search and matching are performed with respect to. 2. A coarse search means for performing a serial search in given image data to detect the existence position of an object existing in the image data, and the object detected by the rough search means. Matching means for performing a random search on the image data based on the existence position data to match with the model image, image storage means for storing the captured image data, and model storage means for registering the model image In an image processing apparatus including: a tilt detecting means for detecting a tilt of an object in given image data, and detecting the tilt detecting means when reading the image data stored in the image storage means. Affine transformation is performed based on the obtained inclination, and the imaged image data is rotated at least a predetermined angle and registered in advance. The image processing apparatus provided with a memory control means to provide the converted image data to match the angle of the Dell image on the coarse search means and matching means. 3. The memory control means according to claim 2, wherein the affine transformation unit for outputting an image for coarse search and the affine transformation unit for outputting an image for matching are commonly used. Image processing device. 4. The monitor according to claim 2, further comprising monitor means for displaying the object obtained by the image pickup, wherein the display mode output to the monitor means is the image data after the affine transformation. Image processing device. 5. The memory control means receives a coordinate value of a predetermined area on the monitor means, performs the affine transformation based on the coordinate value, obtains an address on the image storage means, and based on the obtained address. The image processing apparatus according to claim 4, wherein the image data stored in the image storage means is read out.