JPH0729008A - Picture processor - Google Patents

Abstract

PURPOSE:To provide a picture processor which can automatically set a parameter for recognizing a work and which can always be processed with optimum even by a time band or a work type. CONSTITUTION:A recognition condition setting circuit 2 for setting a window becoming the recognition range of a picture which is image-picked up by a TV camera 1 and the number of retrying times, a picture pre-processing circuit 3 executing a picture pre-processing by receiving the setting of a binary level value and arrangement data for removing noise and a picture memory 4 storing the picture are provided. Furthermore, a recognition execution circuit 5 recognizing the input picture, a parameter setting memory 9 for setting the parameter in the recognition condition setting circuit and the picture pre-processing circuit, a result judgement means 10 which rewrites and resets the parameter of the parameter setting memory at all times based on the judgement of the recognition result by the recognition execution circuit and a system switching circuit 7 which changes over a mode are provided. The parameter is searched at a test run mode, and the parameter is optimized in accordance with the time band or the work type at the system run mode.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus for setting optimum values of parameters when performing image processing of a recognition object in a visual sensor used in a transfer robot, for example.

[0002]

2. Description of the Related Art As a parameter of a visual sensor for image-processing a recognition object, there are, for example, a binary level, a grayscale preprocessing set value, and a recognition window setting. These are set when the system is started up, but in the setting of the binary level, they are fixed values or flexible values that are automatically set.

FIG. 11 is a hardware block diagram showing a processing process of a conventional typical visual sensor as an image processing apparatus. In the figure, 1 is a TV camera for image input,
Reference numeral 2 is an image recognition condition setting circuit for limiting the area of an image obtained by the TV camera 1, 3 is an image preprocessing circuit for removing image noise, 4 is an image memory, 5 is a recognition execution circuit, 6 is a result output circuit. A parameter setting memory 9 stores the setting values of the recognition condition setting circuit 2 and the image preprocessing means 3.

Further, FIG. 12 is a diagram of Japanese Patent Laid-Open No. 63-108469.
2 is a block diagram of an image processing device disclosed in Japanese Patent Publication No. Here, optimization is performed by focusing on the threshold value for binarization. In the figure, 1 is a TV camera, 4 is an image memory, 18 is a comparison circuit, and 11 is an image binarization process. The image is usually shaded in the binarization circuit, but in order to simplify and speed up the process, the image is processed into a binary state of black and white with a threshold value of a certain level. 12 is 2
A digitized image memory, 13 is a threshold suitability determination circuit, 14 is an object position indexing circuit, 15 is a shape specifying memory, 16 is a threshold value setting memory, and 6 is a result output circuit.

According to the configuration of FIG. 11 described above, in the conventional visual sensor, the image obtained by the TV camera 1 is
An image area is restricted by the recognition condition setting circuit 2, noise and the like are removed through the image preprocessing circuit 3, and then the image is captured in the image memory 4. At this time, the setting value of the recognition condition setting circuit 2 and the setting value of the image preprocessing circuit 3 are
A fixed value preset in the parameter setting memory 9 is used.

The image processing apparatus disclosed in Japanese Patent Laid-Open No. 63-108469, as shown in FIG. 12, is limited to the process of binarizing an image and compared with the data of the shape specifying memory 15. The value suitability determination circuit 13 optimizes the threshold value.

That is, the recognition object and its background imaged by the TV camera 1 are sent to the binarization circuit 11 via the image memory 4 and the comparison circuit 18. Further, the threshold value setting memory 16 stores the initial threshold value in advance, receives the signal from the threshold value suitability determination circuit 13, corrects the initial threshold value, and the binary circuit through the comparison circuit 18. Output to 11.

Next, the binarization circuit 11 binarizes the input image data and sends it to the threshold suitability determination circuit 13 via the binarized image memory 12, and the threshold suitability determination circuit 13 specifies the shape. The information peculiar to the object in the memory 15 is compared with the image data from the binarized image memory 12 to determine the appropriateness of binarization. As a result, if it is appropriate, the binary image is output to the result output circuit 6 via the object position indexing circuit 14, and if it is incorrect, the correction signal is the threshold setting memory 1
6 is output. Thereby, the position of the work object or the like can be accurately grasped.

[0009]

However, in the image processing apparatus shown in FIG. 11 described above, since the setting parameter is a fixed value, the recognition environment such as the unevenness of the work to be recognized or the change of the outside light at the time of image input is considered. , Recognition execution circuit 5
Has a large recognition margin during image processing (even if the image condition is not so good, a reasonable value is given for image recognition), which may result in poor recognition accuracy or a high unrecognizable rate. There was a problem sometimes. The setting of parameters largely depends on the experience of the operator, and even if the parameters are calculated and set, it takes time to set them.

Further, since the image processing apparatus shown in FIG. 12 for the purpose of optimizing the binary level is limited to the level of the binary level, it is doubtful whether it can be applied to a harsh field system. , If it deviates from the preset shape specific value, there is no choice but to deal with it by changing the threshold value, so it is necessary to change the threshold value step by step, large exploratory factors are required, and fast recognition speed is required. The system is disadvantageous.

Further, when there are a large number and a large number of target works, how to put data into the shape specifying memory 15 in advance is a problem. For example, if the target work is a cardboard box, there are many types handled in warehouses and the like, and the quantity is large. Even if a person performs registration work for 10 minutes for each type, if there are about 100 types, the adjustment time becomes enormous and the burden on the operator is heavy.

The present invention has been made in order to solve the problems of the above-mentioned conventional example, and all the parameter settings for a large number of works can be automatically realized.
An object of the present invention is to obtain an image processing apparatus which can always optimally deal with the operating time zone based on the recognition result even when the system is in operation, and can also optimally deal with unevenness due to the work.

[0013]

An image processing apparatus according to claim 1 of the present invention sets a camera for picking up a recognition target, a window as a recognition range of an image picked up by the camera, and a retry count. Recognition condition setting means,
An image preprocessing unit that executes image preprocessing upon receiving settings of binary level values and array data for noise removal, an image memory that stores the image preprocessed image, and an input image through the image memory. Recognition execution means for executing recognition by obtaining the position of the center of gravity, parameter setting memory for setting parameters in the recognition condition setting means and the image preprocessing means, and based on the judgment of the recognition result by the recognition execution means The result determination means for searching the optimum value of the parameter by rewriting and resetting the parameter of the parameter setting memory at any time is provided.

An image processing apparatus according to a second aspect of the present invention includes a camera for picking up an image of a recognition target, a recognition condition setting means for setting a window which is a recognition range of an image picked up by the camera, and a retry count. Image pre-processing means for executing image pre-processing upon receiving value level values and setting of array data for noise removal, an image memory for storing the image pre-processed image, and the center of gravity of the input image via the image memory A recognition executing means for executing recognition by obtaining a position, a parameter setting memory for setting parameters in the recognition condition setting means and the image preprocessing means, and the above based on the judgment of the recognition result by the recognition executing means Result determination means for searching the optimum value of the parameter by rewriting and resetting the parameter of the parameter setting memory at any time, The system includes a system switching means for switching between the system test mode and the system run mode. It is designed to be converted.

Further, the image processing apparatus according to claim 3 is
It receives a camera for imaging the recognition target, a recognition condition setting means for setting the window which is the recognition range of the image picked up by the camera and the number of retries, and setting of the binary level value and array data for noise removal. Image preprocessing means for executing image preprocessing by means of image processing, image memory for storing the image preprocessed image, recognition executing means for executing recognition by determining the barycentric position of the input image through the image memory, A parameter setting memory for setting parameters in the recognition condition setting means and the image preprocessing means, and parameters by rewriting and resetting the parameters in the parameter setting memory at any time based on the judgment of the recognition result by the recognition executing means. Result determination means for searching for the optimum value of the system, and switching between system test mode and system run mode And a system switching unit for performing, the result judgment circuit, the system run mode, in which so as to optimize the recognition runtime parameter based on the history information of past recognition data of different object work.

[0016]

According to the first aspect of the present invention, temporary data is obtained in the system test run mode by searching the optimum value of the parameter by rewriting and resetting the parameter as needed based on the judgment of the recognition result. It can be set, and it reduces the burden on the operator to find various optimum parameters for various works.

Further, according to the second aspect, in the system run mode, by optimizing the parameters at the time of recognition execution based on the history information of the past recognition data according to the time zone, the change in the sun altitude depending on the time zone, It is optimized by changing parameters that change depending on weather, day and night, forced lighting, etc., and recognition accuracy and reliability are increased.

Further, according to the third aspect, in the system run mode, the parameters at the time of recognition execution are optimized based on the history information of the past recognition data for each target work, so that each of them can be automatically operated. Changing and optimizing the parameters peculiar to the target work not only reduces the burden on the worker, but also greatly increases the recognition accuracy and reliability.

[0019]

【Example】

Example 1. The first embodiment of the present invention will be described below. FIG. 1 shows the configuration of the image processing apparatus according to the first embodiment, which is an example of a hardware block for automatically setting parameters. In the figure, the same reference numerals as those in FIGS. 11 and 12 denote the same parts, 1 is a TV camera for inputting an image, 2 is a window setting and recognition execution when recognizing an image input from the TV camera 1. In the recognition condition setting circuit for setting the recognition condition such as the setting of the number of retries, the window setting sets the coordinate value on the diagonal line of the window when setting the recognition range on the image. Three
Is an image preprocessing circuit that performs binary processing based on the setting of binary level values and image noise removal, and 4 is an image preprocessing circuit 3
An image memory for recording an image subjected to the image pre-processing after 5 is a recognition execution circuit, and 6 is a result output circuit.

As a new code, 7 is a switching circuit for switching between the system run and the test run, 8 is a recognition result memory in which the recognition result is stored, and 9 is a parameter setting in which the parameter setting value is stored. When the recognition is executed in the memory, the default value is initially set, but then the recognition is executed, the value is changed to an appropriate value, and this is repeated an appropriate number of times to obtain the optimum value. Reference numeral 10 is a result determination circuit that has a processor and a program storage memory built-in, determines setting values and average values from the recognition data, and determines whether or not the target image is within the window area.

Next, the operation in the test run mode according to the configuration of FIG. 1 will be described with reference to the triangular image processing example shown in FIG. An image input from the TV camera 1, that is, a triangular figure (see FIG. 2A) passes through the recognition condition setting circuit 2 in which the recognition conditions of the window and the number of retries are set, and the image preprocessing circuit 3 performs the image preprocessing. It is processed and recorded in the image memory 4.

In this case, the window setting given from the parameter setting memory 9 to the recognition condition setting circuit 2 is the coordinate point A (x ₁ , y ₁ ) of the screen area shown in FIG. 2B.
And B (x ₂ , y ₂ ) are set. At this time, as the set value given from the parameter setting memory 9 to the image preprocessing circuit 3, a binary level value (shading level of 0 to 255) and array data for removing image noise are set. (See FIG. 2 (c)).

Here, the array data for removing the image noise is given a parameter value of a weighting matrix in an image filter in which a pixel near the center is heavily weighted, for example, and the array data for removing the image noise is set between the image data and the array data. Noise is removed by the product-sum operation with the pixel values of the image, and the binary image input image shown in FIG. 2D is stored in the image memory 4.

The image data recorded in the image memory 4 is processed by the recognition execution circuit 5 to obtain, for example, the barycentric position of the triangle to be recognized, and the result judgment circuit 10 checks the number of retries and the recognition result. That is, the window position is checked from the position of the center of gravity, and the binary level is checked from the recognition result, and the recognition result is stored in the recognition result memory 8.

Here, the position of the center of gravity of the triangle shown in FIG. 2 (e) is determined by points An (xan, yan) and Bn (xbn, yb) in the figure.
n), Cn (xcn, ycn), and Dn (xdn, ydn), Δxn = Bn−An = xbn−xan, Δyn = Dn−Cn = ydn−ycn, and the area is A _X = ΣΔxn, _{a Y = Σ △ yn, m} x = Σx · △ xn, when the m _y = Σy · △ yn, the gravity center position _{G = (m x / a X} , m y / a Y) becomes.

The center of gravity position and the recognition result (OK, NG) are stored in the recognition result memory 8, and the center of gravity position and the error code due to the check are output to the outside through the result output circuit 10. Here, each of the above parameter values, that is, the window position, the binary level value, the number of retries, and the matrix data for noise removal are rewritten and reset by the result determination circuit 10 at any time.

FIG. 3 shows an example in which parameter setting values are determined by taking a binary level as an example. In this example, recognition is executed at each level in the system test run mode, and an optimum value is searched for from the result. That is, the set value at each recognition number is sequentially set to 10, 11, 12, 13, ...
-, The result from the 4th time to the i-th time is OK in FIG. For this reason, the center values of them are given as the set value PB ₁ of the binary level for recognition. That is, as the set value PB ₁ , PB ₁ = {f ₁ (4) + f ₁ (i)} / 2, and as the average value PB, PB = ΣPB _k / N (where k is 1 to N) ).

FIG. 4 shows an embodiment in which a window for designating an image area to be recognized is set. Obtain the recognition position (center of gravity position) during system operation and record this.
After measuring a certain number of times, the average value is updated as a new window position each time. As a result, the optimum window position is determined. That is, here, the position of the center of gravity the same as G described above _{= (m x / A X,} m y / A Y) to determine, if calculating the center of gravity, the recognition execution circuit 5, takes the target image in the window It is possible to determine whether (a frame of the window crosses the image data).

Now, in FIG. 4, W is set as an initial value.
_{When c1} (x, y) = (R _X1 , R _Y1 ) is set and the measurement is performed a predetermined number of times n, the average value Wc (x, y) of them is updated as the window center position, that is, a new window position each time. The optimum window position is determined by going forward. Here, the average value Wc (x, y) is obtained by the following equation. Wc (x, y) = ΣR _Xi / n (where n is 1 to n)

Next, FIG. 5 shows an example in which gradation conversion is performed as image preprocessing of an image to be recognized in the processing performed in the image preprocessing circuit 3. In this example, if the trial 1 (first recognition) cannot be recognized, the trial 2 (second recognition) and the trial 3 (third recognition) are retried and executed. That is, if the recognition cannot be performed, the recognition is executed up to three times. Here, the set value set in each try mode is changed every certain number of times based on the recognition result. As a result, the optimum gradation conversion parameter is determined for each target work.

That is, each set value is weighted from the results from the i-th to the i + 3-th results, and the weighted average is calculated to newly obtain i +.
Determine the setting value from the 4th. For example, the set values P _k1 , P _k2 , and P _k3 of the gradation conversion parameter for the work A are: P _k1 = (P _k1 × 1 + P _k2 × 1 + P _k3 × 2) / 4 P _k2 = (P _k2 × 1 + P _k3 × 2 ) / 3 P _k3 remains unchanged.

For example, in order to sharpen the gray level of the original image before gradation conversion, the respective set values P _k1 , P _k2 ,
P _k3 is a lightness level of 100 to 200, and a lightness level of 0 to 2
55, change the lightness level 120-200 to the lightness level 0-2
55, change the lightness level 140-210 to the lightness level 0-2
55 when sequentially corresponding to each other, in the case of FIG. 5, OK
Since the weight is determined by the number of the set values and the set value is determined, assuming that the number of _OKs is 4, the set value P _k1 is the set values P _k1 , P _k2 , P.
_{If k3} is used, P _k1 = {(100, 200) × 1 + (120, 200)
× 1 + (140, 210) × 2)} / 4, which can be newly set.

FIG. 6 shows, for example, 2 according to claim 1 of the present invention.
This is a mode for searching an optimum binary level in the system test run mode. Here, in the configuration of FIG. 1, the system switching circuit 7 switches to the system test run mode, the result determination circuit 10 searches for the lower limit and upper limit of the recognizable binary level, and the image setting preprocessing circuit from the memory setting memory 9 is searched. Determine the optimal binary level to set to 3.

According to this search flow, temporary data can be set in the system test run mode by searching for the optimum value of the parameter by rewriting and resetting the parameter as needed based on the judgment of the recognition result. It is possible to reduce the burden on the operator in obtaining various optimum parameters for various works.

FIG. 7 shows a parameter retrieval flow in the system run mode according to the third aspect. In the system run mode by switching the system switching circuit 7, normal recognition processing (for example, in the case of using a robot, recognition processing such as recognition of an object, position correction and gripping a work) is performed. In parallel with this, parameter optimization processing is performed. In this, the recognition position is recorded for n pieces of work, and the recognition window position of the work n + 1th piece is newly determined from the results of the past n pieces. This is repeated one after another.

Therefore, according to this search flow, in the system run mode, the parameters at the time of recognition execution are optimized based on the history information of the past recognition data for each target work, so that the respective parameters are automatically controlled during the automatic operation. Changing and optimizing the parameters unique to the target work can not only reduce the burden on the worker, but also greatly increase the recognition accuracy and reliability.

Further, FIG. 8 shows a search flow according to the embodiments of claims 2 and 3. Perform normal recognition processing in system run mode. In parallel with this, parameter optimization processing is performed. Here, the recognition result is recorded for n works, and the gradation conversion parameter for the work n + 1 is newly determined from the results for the past n works. At this time,
The results are compiled for each work type, and the parameters are optimized for each work. While repeating this one after another, at this time, by performing the optimization of the gradation conversion parameter as a parameter at the time of recognition based on the history information of the past recognition data by the time zone, the change in the sun altitude depending on the time zone, Optimization is performed by changing the gradation conversion parameter that changes depending on the weather, day and night, forced illumination, and the like.

Therefore, according to the above-described search flow, in the system run mode, the parameters at the time of recognition execution are optimized based on the history information of the past recognition data for each target work, so that each parameter is automatically controlled during the automatic operation. Changing and optimizing the parameters unique to the target work not only reduces the burden on the operator, but also greatly increases the recognition accuracy and reliability, and at this time, the history information of past recognition data depending on the time zone is used. By optimizing the parameters at the time of recognition based on this, it is possible to increase the recognition accuracy and reliability by changing the parameters that change due to the change of the sun altitude depending on the time of day, weather, day and night, forced lighting, etc. it can.

In the first embodiment, an example of optimizing the parameters is shown by a simple logic, but the same effect can be obtained by using a logic such as fuzzy or neuro.

Example 2. Next, FIG. 9 shows an example of application to the image processing apparatus of Japanese Patent Application Laid-Open No. 63-108469 shown in FIG. 12 described above. When the present invention is applied, the result judging circuit 10, the system switching circuit 7 and the recognition are shown. As a result, the memory 8 is added, and the operation of the system shown in FIG.
Unlike the above, if the parameters are optimized according to the flow shown in FIG. 6 to FIG. 8, in the system test run (test run) mode, so-called automatic tuning is performed to optimize the optimum parameters and reduce the load on the operator. In addition, the system run (automatic operation) mode has the effect of always being able to flexibly set optimal parameter conditions for things that change during automatic operation, such as the work time zone and work type. .

Example 3. Next, FIG. 10 shows an example of application of the present invention to a robot control system. In the figure, 20 is a robot, 21 is a fixed camera, 22 is a hand, 23 is a hand camera attached to the hand, and 24 is a hand camera.
Is a work stacked on the pallet 25, 26 is an image processing apparatus having the configuration shown in FIG. 1 or 9 for performing image processing by recognizing the work 24 by the fixed camera 21 and the hand camera 23, and 27 controls the robot 20. A robot controller 28 for controlling the above is a central controller for controlling the ideal collection place 226 and the furnace bit controller 27.

According to the robot control system having such a configuration, the parameters are optimized by the flow shown in FIGS. 6 to 8, and in the system test run (test run) mode, so-called automatic tuning is performed to optimize the parameters. Various parameters can be realized by reducing the load on the operator so far, and in the system run (automatic operation) mode, it is always optimally flexible for things that change during automatic operation such as the work time zone and work type. Since the conditions of various parameters can be set, the system stop factor such as the error occurrence rate or the robot hand interference can be reduced, and the efficiency of system operation can be improved.

[0043]

As described above, according to claim 1 of the present invention, the system test is performed by searching the optimum value of the parameter by rewriting and resetting the parameter as needed based on the judgment of the recognition result. There is an effect that provisional data can be set in the run mode, and the burden on the operator can be reduced in obtaining various optimum parameters for various works.

Further, according to the second aspect, in the system run mode, by optimizing the parameters at the time of recognition execution based on the history information of the past recognition data according to the time zone, the change in the sun altitude depending on the time zone, The parameters that change depending on the weather, day and night, forced illumination, etc. are changed to be optimized, and the recognition accuracy and reliability are increased.

Further, according to claim 3, in the system run mode, the parameters at the time of recognition execution are optimized on the basis of the history information of the past recognition data for each target work, so that each of them can be automatically operated. Changing and optimizing the parameters unique to the target work not only reduces the burden on the worker, but also has the effect of significantly increasing the recognition accuracy and reliability.

[Brief description of drawings]

FIG. 1 is a block diagram showing a hardware configuration example of an image processing apparatus according to a first embodiment of the present invention.

FIG. 2 is an explanatory diagram of a processing process based on the configuration of FIG.

FIG. 3 is an explanatory diagram in a case where the parameters according to the first embodiment are limited to binary level set values.

FIG. 4 is an explanatory diagram of setting parameters of a window (recognition position) according to the first embodiment.

FIG. 5 is an explanatory diagram of an example of setting gradation conversion parameters according to the first embodiment.

FIG. 6 relates to the first embodiment and is an explanatory diagram showing an operation flow of claim 1 of the present invention.

FIG. 7 is an explanatory diagram according to the first embodiment and showing an operation flow corresponding to claim 3 of the present invention.

FIG. 8 is an explanatory diagram according to the first embodiment and showing an operation flow corresponding to claims 2 and 3 of the present invention.

FIG. 9 is a hardware configuration diagram when the present invention is applied to a conventional example disclosed in Japanese Patent Laid-Open No. 63-108469.

FIG. 10 is a block diagram when the present invention is applied to a robot control system.

FIG. 11 is a system configuration diagram of a conventional example.

FIG. 12 is a hardware configuration diagram of a conventional example disclosed in Japanese Patent Laid-Open No. 63-108469.

[Explanation of symbols]

 1 TV camera 2 Recognition condition setting circuit 3 Image preprocessing circuit 4 Image memory 5 Recognition execution circuit 6 Result output circuit 7 System switching circuit 8 Recognition result memory 9 Parameter setting memory 10 Result judgment circuit

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁶ Identification code Office reference number FI Technical display area H04N 7/18 KG06F 15/64 340 B 400 J 9191-5L 15/68 320 Z

Claims (3)

[Claims] 1. A camera for picking up a recognition target, a recognition condition setting means for setting a window as a recognition range of an image picked up by the camera and a retry count, and a binary level value and an array for removing noise. An image preprocessing means that receives image data and executes image preprocessing, an image memory that stores the image preprocessed image, and a recognition that executes recognition by determining the barycentric position of the input image through the image memory. Execution means, parameter setting memory for setting parameters in the recognition condition setting means and the image preprocessing means, and rewriting and rewriting the parameters in the parameter setting memory as needed based on the judgment of the recognition result by the recognition executing means. An image processing apparatus comprising: result determination means for searching for an optimum value of a parameter by setting. 2. A camera for picking up a recognition target, a recognition condition setting means for setting a window as a recognition range of an image picked up by the camera and a number of retries, and a binary level value and an array for removing noise. An image preprocessing means that receives image data and executes image preprocessing, an image memory that stores the image preprocessed image, and a recognition that executes recognition by determining the barycentric position of the input image through the image memory. Execution means, parameter setting memory for setting parameters in the recognition condition setting means and the image preprocessing means, and rewriting and rewriting the parameters in the parameter setting memory as needed based on the judgment of the recognition result by the recognition executing means. Result determination means for searching the optimum value of the parameter by setting, system test mode and system run And a system switching means for switching to a mode, the result determination circuit, in the system run mode, to optimize the parameters at the time of recognition based on the history information of the past recognition data by time zone. Image processing device. 3. A camera for picking up a recognition target, a recognition condition setting means for setting a window as a recognition range of an image picked up by the camera and a retry count, and a binary level value and an array for removing noise. An image preprocessing means that receives image data and executes image preprocessing, an image memory that stores the image preprocessed image, and a recognition that executes recognition by determining the barycentric position of the input image through the image memory. Execution means, parameter setting memory for setting parameters in the recognition condition setting means and the image preprocessing means, and rewriting and rewriting the parameters in the parameter setting memory as needed based on the judgment of the recognition result by the recognition executing means. Result determination means for searching the optimum value of the parameter by setting, system test mode and system run And a system switching means for switching to a mode, and the result determination circuit optimizes a parameter at the time of recognition execution based on history information of past recognition data for each target work in the system run mode. Image processing device.