WO1996030870A1

WO1996030870A1 - Image processing system and image processing method

Info

Publication number: WO1996030870A1
Application number: PCT/JP1995/000591
Authority: WO
Inventors: Yukihiro Kawamata; Kimiya Yamaashi; Masayuki Tani; Masami Ito; Syohei Fukuoka; Masaaki Nakajima; Gen Tobita
Original assignee: Hitachi, Ltd.; Hitachi Information & Control Systems, Inc.
Priority date: 1995-03-29
Filing date: 1995-03-29
Publication date: 1996-10-03

Abstract

An image processing system which takes an image of a machine whose motion is controlled by control information, and processes the image. The system has a control information collecting means for obtaining the control information, and a setting means for determnining setting conditions concerning the image processing on the basis of the control information taken by the collecting means. Since the image processing conditions corresponding to the condition of an object to be recognized and background are set by determining the brightness, saturation, hue and visible portions of the object and background and the variation of the background by using control information obtained from a machine, such as a plant, the recognition accuracy is improved, and the recognition processing is simplified.

Description

Specification

Image processing system and image processing method

The present invention relates to an image processing apparatus and method for a device controlled by control information such as a plant system. Background art

(1) In the conventional image processing device, the target object is recognized using only the information of the captured image (for example, brightness, difference between images, optical flow, etc.).

For example, “Extracting Moving Objects by Sequential Matching,” IEICE Transactions Vol.J76-D—II No.10 pp.2196—2203 (1993) states that even if the brightness or background changes, Methods that can be recognized are being studied, and the method recognizes the object in each frame by sequentially matching the contour of the object recognized in the previous frame with the edge image of the current frame.

First, when cutting out an object (moving object part) from a frame, three consecutive frames Fl, F2, and F3 are taken in, and the difference regions between F1 and F2 and between F2 and F3 are obtained. Then, a common part of the two difference regions is cut out as an object of F2. Then, when the extracted object is used as a template for pattern matching, matching is first performed between the edge image and the entire contour of the object, and then the contour and edge are partially matched. Take.

(2) —On the other hand, in the conventional cold rolling plant, the rolled material is coiled. If the tip of the rolled material is not accurately fitted into the center axis of the coil, the quality of the coil will be degraded. (Called detection). Conventionally, this meandering detection has been performed by the operator monitoring the deviation between the edge of the rolled material in the video captured by the ITV camera at the site and the guideline previously drawn on the monitor screen.

In the prior art (1), the recognition accuracy decreased as the background and the change of the recognition target increased. In addition, since the state (size, distortion) of the recognition target is estimated only from the image, the recognition process is complicated.

An object of the present invention is to solve this problem, improve recognition accuracy in recognition processing, and simplify recognition processing.

In the conventional rolling plant of the above (2), it is difficult to recognize the visible part of the rolled material to be recognized due to the reflection of the rolled material on the roll due to the movement of the roll.

An object of the present invention is to solve this problem and to make it possible to detect meandering of a rolled material. Disclosure of the invention

In order to achieve the above object, the present invention provides a control information acquisition unit that captures control information for controlling the operation of a device related to a recognition target, and an image processing condition setting unit that sets a setting condition related to image processing based on the control information. And provided. Also, in order to solve the problems of the conventional rolling plant, position information of the roll was taken in, and based on the information, the position and size of the region were set so that the roll did not enter the image processing region. .

Each of the above components operates as follows. The control information acquisition means captures control information for controlling the operation of the device related to the recognition target, and the image processing condition setting means measures, for example, the background change, luminance, saturation, hue, and hue of the recognition target that are captured by the captured control information. By setting the setting conditions related to image processing, such as the threshold for binarization and the parameters of the image processing filter, based on such information, the recognition rate of the recognition target can be improved and the recognition processing can be simplified. become.

Further, in the meandering detection system for the rolling plant according to the present invention, the position information of the roll is taken in, and based on the information, the position and size of the region are determined so that the roll does not enter the image processing region. By setting, the meandering of the rolled material can be detected regardless of the roll movement. BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 is a block diagram of an image processing system according to an embodiment of the present invention.

FIG. 2 is a flowchart of an image processing method according to an embodiment of the present invention. FIG. 3 is a diagram showing an image when a rolled material is reflected on a brand image taken by an ITV camera of the image processing system according to one embodiment of the present invention. FIG. 4 is a view showing a manner of estimating a portion where the rolled material is no longer visible in one embodiment of the present invention.

FIG. 5 is a diagram showing a state of background removal according to one embodiment of the present invention.

FIG. 6 is a diagram showing a state of edge position detection according to one embodiment of the present invention.

FIG. 7 is a diagram showing a method for detecting a displacement of a rolled material according to one embodiment of the present invention.

FIG. 8 is a block diagram of an image processing system according to an embodiment of the present invention. You.

FIG. 9 is a diagram showing the behavior of the recognition target according to one embodiment of the present invention.

FIG. 10 is a flowchart of an image processing method according to an embodiment of the present invention. FIG. 11 shows a method of detecting the position of an end point of a rolled material using pattern matching according to an embodiment of the present invention. FIG. BEST MODE FOR CARRYING OUT THE INVENTION

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

Example 1

FIG. 1 shows a system configuration of a first embodiment of the present invention.

The present embodiment is an example in which image processing is performed to detect a displacement of a leading end of a rolled material caused by meandering of a rolled material when winding a coil in a cold rolling plant.

When the rolled material is wound into a coil, the quality of the coil will deteriorate if the tip of the rolled material is not accurately fitted into the center axis of the coil. Therefore, the end points of the rolled material are detected using image processing. However, in cold rolling, since the rolled material to be recognized is glossy, its brightness, saturation, and hue change depending on the location of the plant, and the visible portion changes due to the movement of the plant, so recognition is not possible. Have difficulty. Therefore, the state of the image, such as the luminance, saturation, hue, and the visible portion of the recognition target and the recognition target is estimated using the control information of the plant device, and the recognition target is recognized using the estimated state. Thus, the position of the leading end of the rolled material is detected.

In this system, the end point of the rolled material is detected as follows. First, plant images are taken in, and plant control information such as roll speed and position S in cold rolling is taken in. Then, take that Using the control information and the plant image, the visible part of the rolled material is estimated and the range for image processing is set. Next, using the captured control information and the plant image, the background, the contrast of the rolled material and the background, and the brightness of the rolled material are estimated, and the image processing method is set. Finally, the edge of the rolled material is detected from the image processed, and the end point is detected from the edge of the rolled material. FIG. 1 is a system configuration diagram of an embodiment of the present invention.

This system consists of an ITV camera 101 that captures images of the end points of the rolled material, an image capturing unit 102 that captures images captured by the ITV camera 101, and plant control information (roll rotation speed, Using the control information acquisition unit 103 that captures the roll position, etc.) and the control information of the plant captured by the control information acquisition unit 103, the position, brightness, visible part, and edge of the rolled material are used. Estimating unit 104 for estimating the state of the image processing target such as the contrast of the image and the background, and using the luminance information estimated by the state estimating unit 104 to adjust the binarization threshold of the image. In addition, the type of filter to be subjected to image processing is selected using the contrast information estimated by the state estimating unit 104, and an image is obtained from the position and the visible portion of the rolled material estimated by the state estimating unit 104. Image processing conditions that set the area to be processed In the setting unit 105 and the image capturing unit 102, the image processing area set in the image processing condition setting unit 105 is cut out, and the image in that region is selected in the image processing condition setting unit 105. The image processing unit 106 processes the image with the image processing filter and binarizes the processed image, and the image estimated by the state estimating unit 104 from the image processed by the image processing unit 106 An image recognition unit 107 that recognizes the recognition target using the state of the processing target, a control unit 108 that controls the entire system such as image capture processing, image state estimation processing, and end processing, and an ITV camera 10 The image taken in step 1 and the image display section 109 displaying the recognition result It consists of a vows issuing unit 110 that issues an alarm when there is.

Next, an operation example of the present embodiment will be described.

FIG. 2 is a flowchart showing the processing of this embodiment.

First, in step 202, the image capturing unit 102 captures a plan image captured by the ITV camera 101.

It is composed of an alarm issuing unit 110 that issues an alarm when it occurs.

Next, an operation example of the present embodiment will be described.

FIG. 2 is a flowchart showing the processing of this embodiment.

First, in step 202, the image capturing unit 102 captures a blank image captured by the ITV camera 101.

In step 203, the image capturing unit 102 determines whether the rolled material has entered the plant image captured in step 202. Fig. 3 is an image of a brand image taken by the ITV camera 101 when a rolled material is being conveyed. On the screen 300, the rolled material 301 advances from the upper right part of the screen toward the lower left part of the rolls 302 and 303. An area 350 for determining whether or not the rolled material has entered the vicinity of the upper end 310 of the screen 300 is provided. This area is called an approach determination area. To determine whether the rolled material has entered the screen 300, the difference between the approach determination area of the plant image captured in the past and the approach determination area of the currently captured plant image is calculated. If there is no difference, it is determined that the rolled material has not entered the screen 300, and if there is a difference, it is determined that the rolled material 301 has entered the screen 300. If it is determined that the rolled material has not entered, the process returns to step 202 to capture the plant image. If it is determined that the rolled material has entered, the process proceeds to step 204.

In step 204, the plant image taken with ITV camera 101 is The image is captured by the image capturing unit 102, and the captured image is displayed on the image display unit 109.

In step 205, the control information acquisition unit 103 captures the roll position and roll rotation speed, which are the control information of the plant, from the plant. <In step 206, the state estimation unit 104 determines the roll position. The position of the rolled material is estimated using the rotation speed information, and the image processing condition setting unit 105 determines the position of the region to be image-processed from the position of the rolled material.

Here, a method for estimating the area to be image-processed from the rotation speed of the roll will be described.

On screen 300, the coordinate system is such that the upper end of the screen is 310, the X axis is right, the right direction is positive, the left end of the screen, 311 is y axis, the lower direction is positive, and the upper left end is 312, the origin. Take. The minimum unit of the coordinate system is one pixel.

The rolled material 300 flows from the upper right to the lower left of the screen 300. The rolled material 301 is traveling at a speed of V [mZ second] in a direction inclined by 0 [rad] from the X axis. Assuming that the radius of the rolls 302 and 303 is r [m] and the rotation speed is ω [radZ seconds], the speed of the rolled material 301 is r · ω [mZ seconds].

Here, when the rolled material 301 moves α [pixel] on the screen 300, it actually moves g [m]. The rolled material 301 moves on the screen 300 at a speed of a Z g · r • ω [pixels per second]. Therefore, the speed of the rolled material 301 on the screen 300 in the X-axis direction is one a Z g · r · ω · cos (θ) [pixel seconds], and the speed in the y-axis direction is a Z g · r · ω -sin (Θ) [pixel seconds].

The moving speed on the screen of the region 320 where the image processing is to be performed is the same as the moving speed of the rolled material 301 on the screen. Here, the area for which image processing is to be performed is referred to as an image processing area. The image processing area 320 is directed from the upper right to the lower left of the screen so as to track the end point 305 of the rolled material 301. It starts moving on a straight line. The time t [sec] has elapsed since the rolled material 310 appeared on the screen 300, and the position of the lower right point 3 2 1 of the area when the image processing area 320 starts moving (m , n), the coordinates of the lower right point 3 2 1 of the image processing area 3 2 0 are (− a / g 'r' w 'cos (e)-t + m, a / g-r · ω · sin (θ) · t + n). The coordinates of the lower right point are defined as the position of the image processing area.

In step 2007, the state estimating unit 104 estimates the portion where the rolled material is hidden by the roll using the position information of the movable roll captured in step 205, and sets the image processing condition setting unit 100. In step 5, the size of the image processing area is set from the part where the estimated roll is hidden.

Here, a method of setting the size of the image processing area from the position information of the movable roll will be described.

FIG. 4 is an example of estimating a portion where the rolled material becomes invisible according to the position of the movable roll in detecting the end point of the rolled material and setting the size of the image processing area. The movable roll 4 1 2 moves up and down. Then, the rolled material 4 10 is sandwiched between the movable roll 4 1 2 and the port 4 1 1. Image 400 has the movable roll 4 1 2 at the top, Image 4 50 has the movable roll 4 1 2 at the bottom <Image 4 5 0 shows the rolled material when the movable roll 4 12 comes down Since 410 is hidden, the edge detection processing of the hidden part is not performed. Now, the position of the movable roll 4 12 can be taken in as brand control information. When the center of the lower end of the movable roll 4 15 enters the image processing area, change the size of the image processing area 4 20 so that the movable roll 4 12 does not enter. The X axis is taken at 401, the y-axis is taken at the left end of the screen 402, and the origin is taken at the top left of the screen 4003. Similarly, the screen 450 has the X axis at the top edge of the screen and the y axis at the left edge of the screen. Is the origin. Also, the size of the screen 400 and the screen 450 is set to 320 [pixels] horizontally and 240 [pixels] vertically.

Here, when the movable roll 4 1 2 moves up and down 1 [m], it moves 100 [pixels] on the screen. Also, the lower end center 4 15 of the movable roll 4 12 is set as the position of the movable roll 4 12.

When the movable roll 4 12 is at the top (screen 4 00), the image processing area 4 2 0 is horizontal 2 0 [pixels], vertical 1 2 [pixels], and position (2 2 0, 18 0) And And the position of the movable roll 4 1 2 when it is at the top is (1 4 0,

3 0).

Consider the case where the movable roll 4 1 2 moves downward by 0.9 [m]. _{C The} movable roll 4 1 2 moves 90 pixels in the y-axis direction on the screen, and the position of the movable roll after the movement is (140, 120). In the image processing condition setting unit 105, the upper end of the image processing area 460 is positioned below the lower center part 415 of the movable roll so that the movable roll 412 does not enter the image processing area. Change the size of the image processing area. Therefore, the image processing area after the change

460 has a vertical length of 60 [pixels]. In this way, the image processing area can be set so that no extra background is included in the area.

In step 208, the image capture area 102 cuts out the image processing area set in step 206 and step 207 from the captured image.

In step 209, the image processing unit 106 removes the background from the image cut out by the image capturing unit 102.

Here, a method of removing a background from an image will be described. To remove the background from an image, the difference between the previously captured background image and the captured image is taken. However, in the case of cold rolling, the movement of the movable roll and the The background changes greatly due to the change in the degree of reflection of the roll due to the movement of the roll part.

Therefore, a background image captured in advance from the position information of the movable roll is selected, and the selected background image is subtracted from the image cut out in step 208. FIG. 5 shows the position information of the movable roll 4 In this example, the same background image as the captured image is selected from the previously captured background images, and the background image is subtracted from the captured image to remove the background. The image processing unit 106 has a previously captured background image as a database. Hereinafter, this is referred to as a background image database. The background image database 500 is a background image 501, 503, 505, 507, 509, 511 of any movable roll position when the rolled material is not passing through. 1 and the amount of movement of the movable roll at that time are 50 2, 504, 506, 508, and δ 10 _: 5 12.

Here, the background image 501 is an image when the moving amount of the movable roll is 0 [m] or more and less than 0.15 [m] (502), and the background image 503 is a moving image of the movable opening. The image and background image 505 when the moving amount is 0.15 [m] or more and less than 0.30 [m] (504) are those where the moving amount of the movable roll is 0.30 [m] or more and 0.45 [ m] and below (506) Image and background image 507 is the image and background when the moving amount of the movable roll is 0.45 [m] or more and 0.60 [m] or less (508) Image 509 is the image when the moving amount of the movable roll is 0.60 [m] or more and 0.75 [m] or less (5 10), and background image 5 11 is the moving amount of the movable roll. It is an image when it is 0.75 [m] or more and 0.90 [m] or less (5 1 2).

Now, consider the case where the moving amount of the movable roll of the image 520 captured by the image capturing unit 102 is 0.40 [m]. First, in the background image database 500, the movable roll is 0.30 [m] or more and 0.45 [m]. The background image 505 when less than is selected. Next, the image processing area 532 obtained in steps 206 and 207 is cut out from the selected background image 505, and the difference between the image processing area 532 cut out in step 205 and the image 531 is obtained. Take. Here, the image obtained by taking the difference is referred to as a difference image 5 33. This makes it possible to remove the background of the image processing area adapted to the state of the plant.

In step 210, the state estimator 104 estimates the contrast of the edge portion and the background portion of the rolled material in the image processing area extracted in step 206 from the plant control information.

Here, a method for estimating the clarity of the edge portion of the rolled material in the image processing area will be described. The rolled material 301 flows from the back of the plant equipment (above screen 300) to the front (below screen 300). When the rolled material 301 is in the back, light is blocked by the plant equipment, the contrast of the rolled material 301 and the background is poor, and the rolled material 301 is unclear, and the rolled material 301 is not clear. When it is in front, the rolled material 301 and rolls 302 and 303 are exposed to light, so the contrast between the background and the rolled material 301 is good, and the rolled material 301 should be clear. In such a case, the clarity of the edge of the rolled material can be estimated from the estimated position of the rolled material.

Now, let the size of the screen 300 be 320 [pixels] horizontally and 240 [pixels] vertically. The position of the leading end of the rolled material is as estimated in step 206. When the y-axis coordinate of the leading end of the rolled material exceeds 100 [pixels], it is determined that the rolled material is clearly visible, and the y-axis coordinate of the leading end of the rolled material is 100 [pixel]. If the value does not exceed the value, the rolled material is determined to be invisible. In this way, the contrast between the rolled material and the background can be estimated.

Here, when the contrast between the rolled material and the background is determined to be clear, the control is performed. In the control unit 108, the process proceeds so as to proceed to step 2 13. If it is determined that the process is unclear, the control unit 108 proceeds to step 2 11.

In step 211, the image processing unit 106 applies an expansion filter to the difference image.

In step 2 12, the image processing unit 106 applies a contraction filter to the difference image subjected to the dilation processing.

If the contrast between the rolled material and the background is poor, the edge of the rolled material in the difference image is intermittent. Therefore, by performing the dilation processing and then performing the erosion processing as in steps 211 and 212, the broken portion of the edge can be eliminated.

In step 2 13, the image processing unit 106 performs edge enhancement processing on the difference image by applying a differential filter such as a Laplacian filter.

In step 2 14, a threshold value for performing the binarization processing in the state estimating unit 104 is set.

Here, a method of setting the threshold value when performing the binarization process will be described. The rolled material 301 flows from the back to the front. When the rolled material 301 is in the back, light is blocked by the plant equipment, so that the brightness of the rolled material 301 is low.When the rolled material 301 is in front, the rolled material 301 is exposed to light. Therefore, it is assumed that the luminance is large. As described above, since the luminance of the rolled material 301 is related to the position of the rolled material, the luminance of the edge of the rolled material can be estimated from the estimated position information of the rolled material. In such a case, the threshold value of the binarization process can be obtained from the estimated brightness information of the edge of the rolled material. For example, when the y-coordinate of the leading end position of the rolled material 301 is greater than or equal to 0 [pixels] and less than 60 [pixels], the threshold value for binarization is set to 5, and the threshold is set to 60 or more [pixels] 15 When the value is less than 0 [pixels], the threshold value for binarization is set to 8, and when it is greater than or equal to 150 [pixels] and less than 240 [pixels], the threshold value for binarization is set to 12. Set. Here, the threshold value indicates luminance, and in this embodiment, the luminance is 256 gradations.

By doing so, it is possible to perform image binarization processing suitable for changing plant conditions.

In step 2 15, the image processing unit 106 performs binarization processing using the threshold value obtained in step 2 14, and extracts the edge portion of the rolled material from the image.

From step 216 to step 220, the image recognition unit 107 detects the edge position of the rolled material from the binarized image. Then, in step 222, the end point of the rolled material is detected. The end point of the rolled material is detected by detecting the edge of the rolled material and detecting the intersection of the edge and the edge.

FIG. 6 is a diagram showing a method of detecting an edge position. The image 600 is an image that has been binarized by the image recognition unit 107. Image 600 has two edges, a leading edge 610 and a right edge 611. The leading edge 6110 and the rightmost edge 611 are intermittent due to noise. And there is noise other than the edge part.

Therefore, it is difficult to extract edges using only the histogram of the image 600.

For the image 600, a coordinate system is set in which the upper left point of the image is the origin, the upper end of the image is the X axis, and the left end of the image is the y axis. And the throne In the reference system, the image downward direction and the image left direction are defined as positive. The coordinate system set on the image 600 is set as a reference coordinate system, and the number of pixels when the binary image is projected on the y-axis of the reference coordinate system is called a y-axis projection value.

The binary image 630 is an image obtained by subjecting the image 600 to affinity conversion. The y-axis projection value 640 of the image 630 is given a value even in places other than the edge part due to noise, making it difficult to determine the edge part. The peak 6 41 of the y-axis projection value of the edge portion takes the maximum value when the edge portion and the y-axis are perpendicular. Therefore, the peak 641 of the y-axis projection value is obtained while rotating the binary image 600 little by little, and a search is made for a place where the peak 641 of the y-axis projection value becomes maximum. The angle of rotation of the image when the peak 641 of the y-axis projection value is maximum is the edge inclination.

Image 660 is a binary image when the peak of the y-axis projection value is at a maximum. For the image 660, a coordinate system having an origin at the upper left point 661 of the image, an X axis at the upper end 626 of the image, and a y axis at the left end 666 of the image is set. Image 660 is obtained by rotating image 600 by Θ [rad]. There is an edge at the peak 671 of the y-axis projection value of the image 660. The edge is perpendicular to the y-axis, and the y-axis coordinate value 672 of the peak 671 is b, and the edge equation on the image 660 in the reference coordinate system is y = b.

By converting the equation of the edge on the image 660 in the reference coordinate system into the coordinate system of the image 660, an edge equation in the coordinate system on the image is obtained.

Here, the processing from step 216 to step 219 is called loop processing, and the number of times of the loop processing is called the number of loops. The number of loops in the first loop processing is set to one. Edge detection first detects the leading edge and then the right edge. Tip tip It is assumed that the image is always downward-sloping on the image 600, and the angle between the image and the X axis is small. Also, the right edge and the tip edge are assumed to be almost perpendicular.

In step 211, the affine transformation is performed so that the binary image 600 is rotated clockwise (0.02 X loop times + initial value) [rad] around the origin of the reference coordinate system. When detecting the leading edge, which is the first edge to be detected, the initial value is 0 [rad].

In step 217, the y-axis projection value 640 of the binary image 630 after the affinity conversion is obtained.

In step 218, a peak 61 is obtained from the y-axis projection values 640 obtained in step 217.

In step 211, the peak of the y-axis projection value obtained in step 218 is compared with the peak of the y-axis projection value obtained in the previous loop processing, and the peak of the y-axis projection value is calculated. Find the place with the largest value. If the peak of the y-axis projection value obtained in step 218 is large, the process returns to step 2 16. If it is small, the process proceeds to step 220. If the number of loops is one, the process returns to step 2 16.

In step 220, an edge equation is obtained from the image 660 after the affinity conversion. The angle of the affinity transformation when the peak of the y-axis projection value takes the maximum value is the edge inclination, and the y-coordinate value b (6 7 2) having the affinity transformation angle 0 and the maximum value of the y-axis projection value ), The equation of the edge part can be found by performing coordinate transformation to the coordinate system of the image 660.

Here, a method for obtaining the equation of the edge portion from the angle of the affinity conversion and the y-coordinate value of the peak of the y-axis projection value will be described. The image 660 after the affinity conversion is obtained by rotating the image 600 by 0 [rad]. Image 660 tip edge 670 is on the straight line y = b (673) in the reference coordinate system. For the image 660, the origin is at the upper left part 661 of the image, the X axis is at the upper part 626 of the image, and the left end is

6 Set the coordinate system that takes the y-axis in 3. The image 6 described in the reference coordinate system to obtain the equation of the leading edge 6 10 in the coordinate system (reference coordinate system) of the image 600 on the image 600 before the affinity conversion If the straight line y = b (673) representing the leading edge 610 of 60 is coordinate-transformed to the coordinate system of the image 660, the equation of the edge is as follows.

y =-tan (θ) x + b / cos (Θ)

In step 220, it is determined whether the detected edge is a leading edge or a right edge (determining whether the processing in step 220 is the first or second time), and if the detected edge is a leading edge. After setting the initial value of the rotation angle of the affinity transformation π 2 [rad] so that the right edge can be detected, return to step 2 16 and if it is the right edge, go to step 2 2 Proceed to 1.

In step 221, the intersection of the straight lines of the two edges obtained in step 220 from step 216 is determined in order to detect the end point of the edge of the rolled material.

In step 222, the control unit 108 detects the displacement of the rolled material.

FIG. 7 is a diagram showing a method for detecting a displacement of a rolled material. On the screen 700, a trajectory 720 of an end point 70 of the rolled material 701 is drawn. The points 711, 712, 713, 713, 714, and 715 are the locations of the end points detected by this system, which are blocked on the screen 700. User can use rolled material

A guide line 730 indicating the correct trajectory of the end point of 70 1 can be set in advance <Also, the user can set a guide line according to the type of sheet width of the rolled material, For example, if three types of rolled material with a plate width of 470 mm, 56 Omni and 640 mm flow through the plant, the user specifies three types of guide lines according to the type of plate width. I do.

If the distance between the guide line 730 and the end point 710 of the rolled material exceeds a preset threshold value, it is determined that the position of the leading end of the rolled material is deviated and step 2 2 4 If the threshold value is not exceeded, it is determined that the position of the leading end of the rolled material is correct, and the process proceeds to step 223.

In step 222, the image recognition unit 107 determines whether the leading end of the rolled material has passed the lower end of the screen.If the leading end has passed the lower end of the image, the process proceeds to step 225. If not, the flow returns to step 204 to detect the position of the end point of the rolled material again.

In step 224, the newsletter 110 notifies the user of the displacement of the rolled material, and proceeds to step 225.

In step 225, the system is terminated.

Example 2

A second embodiment of the present invention will be described with reference to the system configuration shown in FIG. 8. <This embodiment is an example in which the positional deviation of the leading end of a rolled material is detected using pattern matching.

This system consists of an ITV camera 101, an image capture unit 102, a control information acquisition unit 8001, which captures plant control information (roll rotation speed, roll position, and plate type). A state estimator 8002 for estimating the state of the image processing target such as the position of the rolled material, the brightness of the rolled material, and the visible portion using the control information of the plant captured by the information acquisition unit 8001, and a state estimator. The area of image processing is set from the position of the rolled material estimated by the part 800 and the visible part, and the type information of the plate taken in by the control information acquisition part 801 The threshold value for binarization is set using the control information acquisition unit 8001, and the template for pattern matching for detecting the end point of the rolled material is set using the position information of the roll taken in by the control information acquisition unit 800. An image processing condition setting unit 803, and an image processing unit 804 that performs background removal and binarization processing of the captured image and _: rolling is performed using the template set in the image processing condition setting unit 803. An image recognition unit 805 that performs pattern matching on the end points of the material, a control unit 806 that controls the entire system such as image capture processing, image state estimation processing, and termination processing, and an image display unit 109 , And the alarm issuing section 110.

FIG. 9 is a diagram showing the behavior of a rolled material to be recognized in the present embodiment. On the screen 900, the rolled material 900 advances from the back of the plant (center of the screen) toward the front of the plant (bottom of the screen) between the movable rolls 902 and 903. The movable roll 902 moves up and down, and the rolled material 901 is sandwiched between the movable roll 902 and the roll 903 when passing directly below the movable roll 902. When the rolled material 901 is not sandwiched by the movable roll 902, the rolled material 901 advances in the front direction of the screen with the leading edge 904 facing the screen. Then, when the rolled material 90 1 is sandwiched between the movable rolls 90 2 and the rolls 90 3, the rolled material 90 1 advances toward the screen with the surface of the rolled material facing the screen, and moves down the screen so as to be rolled up by the rolls 90 3. Go out. When the rolled material 901 is not sandwiched between the rolls, the brightness of the edge 904 of the rolled material is large, and when the rolled material 901 is sandwiched between the rolls, the surface 9 5. Assume that the brightness of 1 is large.

Next, an operation example of the present embodiment will be described. FIGS. 10 (a) and (b) are flowcharts of the present embodiment.

First, in step 1002, the control information acquisition section 8001 —Acquires the rotation speed information of the

In step 1003, the image is taken in the image taking section 102.

In step 104, the image capturing unit 102 determines whether or not the rolled material has entered the plant image captured in step 1002 by taking a difference from the image of the immediately preceding frame. I do.

In step 1005, the control information acquisition unit 801 fetches the roll position, roll rotation speed, and plate type, which are control information, from the plant.

In step 106, the plant image captured by the ITV camera 101 is captured by the image capturing unit 102, and the captured image is displayed on the image display unit 109.

In step 107, the control unit 806 determines whether or not the rolled material is sandwiched between the rolls based on the position information of the movable roll captured by the control information acquisition unit 801.

Here, when the movable roll moves 0.9 m or more downward from the uppermost position, it is determined that the movable roll has been caught by the rolled material. Then, in the control unit 806, if the rolled material is sandwiched between the rolls, the process proceeds to step 1101. If not, the process proceeds to step 1008. .

Before the rolled material 901 is sandwiched between the rolls, since the rolled material 901 is located above the roll 903, the image processing area 910 is located between the movable roll 902 and the roll sole 903. Provide. Then, after the rolled material 901 is sandwiched between the rolls, since the rolled material 901 is present at the portal 903, an image processing area 960 is provided at the roll 903.

Before the rolled material is sandwiched between rolls, the brightness of the edge of the rolled material After the rolled material is sandwiched between the rolls, the brightness of the surface of the rolled material increases, so a template for detecting the edge at the tip is set before the rolled material is sandwiched, and the surface is Set a template for part detection. In steps 1008 to 1101, the processing is performed when the rolled material is not sandwiched between the rolls.In steps 101 to 1, the rolled material is sandwiched between the rolls. Perform the processing for the case.

In step 1008, an image processing area 910 (hereinafter, referred to as an upper image processing area) is provided in the image processing condition setting unit 803 between the movable door 902 and the roll 903.

At step 109, the image processing condition setting unit 803 sets a template for detecting the edge of the rolled material for pattern matching of the edge of the rolled material.

In step 110, the image processing condition setting section 803 estimates a portion where the rolled material is hidden by the movable mallet, and determines the size of the area so that the hidden part does not enter the upper image processing area. To change.

In step 101, the image processing condition setting unit 803 sets an image processing area 960 (hereinafter referred to as a lower image processing area) in the roll 903.

In step 1102, the image processing condition setting unit 803 sets a template for detecting the surface portion of the rolled material for performing pattern matching on the surface portion of the rolled material.

In step 101, in the image capturing unit 102, the image processing area set by the image processing condition setting unit 803 is cut out.

In step 104, the background is removed from the image in the image processing area. In step 1 0 15, the image processing condition setting section 8 03 The threshold value of the binarization processing is set from the type information of the plate taken in by the information acquisition unit 8001.

Here, a method of setting a threshold value for binarization will be described.

The brightness of the board depends on the type. Therefore, the threshold value of the binarization process is changed according to the type of plate. For example, in the rolling plant of this embodiment, iron plates, copper plates, and aluminum plates are transported. Aluminum plate has the highest brightness, copper plate has the next highest brightness, and iron plate has the lowest brightness. Here, in order to extract different types of boards from the captured image, the upper and lower limits of the board brightness are set, the value between the upper and lower limits is set to 1, and all other values are set to 0 and binarized. Perform processing. In this way, different types of boards can be extracted from the image. Here, the threshold values of the binarization process are set to lower limit 30 and upper limit 40 for aluminum plate, lower limit 2 2 and upper limit 32 for copper plate, and lower limit for iron plate. The value is 15 and the upper limit is 25.

In the rolling plant of the present embodiment, since the iron plate is transported, the threshold values of the binarization processing are set to the lower limit value 15 and the upper limit value 25.

The threshold value indicates luminance, and the luminance is 256 gradations.

In step 106, the image processing section 804 binarizes the image in the image processing area using the threshold value set in the image processing condition setting section 803, and extracts the rolled material. .

In step 107, the image recognition unit 805 performs pattern matching using the template set in the image processing condition setting unit 803, and detects the end point of the rolled material from the matched position. .

Here, a method of detecting the end point of the rolled material will be described.

FIG. 11 is a diagram showing a method for detecting a beard at an end point of a rolled material using pattern matching. The image 110 of the upper image processing area 910 after the binarization processing has the edge 904 of the rolled material extracted, and the lower image processing area 960 after the binarization processing has been extracted. In the image 1150, the surface 951 of the rolled material is extracted. In the upper image processing area 9110, pattern matching is performed between the image 1100 after binarization processing and the tip edge detection template 1110 selected in step 1109. In the lower image processing area 960, the image 1150 after binarization processing is subjected to pattern matching with the surface detection template 1160 selected in step 1102.

On screen 900, the upper end of the screen is set to the positive direction with the upper end of the screen as the X axis, the left end of the screen as the y axis, the lower direction is set as the positive direction, and the upper left end of the screen is set as the origin. Take a coordinate system. Then, on the screen 950, the upper end 1171 of the screen is set as the X axis, the right direction is set to the positive direction, the left end 1172 of the screen is set as the y axis, the lower direction is set as the positive direction, and the upper left end 1170 is set as the origin. Take a coordinate system.

Further, the position of the leading edge detection template 111 is defined as the end point position (xl, yl) (111) of the rolled material in the template 111, and the surface detection template 111 is used. The position 60 is the position (x2, y2) (1161) of the end point of the rolled material in the template 1160.

By doing so, the position of the end point of the rolled material can be determined from the position of the template subjected to pattern matching.

In step 11018, the image recognition unit 805 detects a displacement of the rolled material. Here, the displacement of the rolled material is detected by calculating the distance between the end point of the rolled material and a preset guide line, and determining whether the distance exceeds a preset threshold value.

When the position of the leading end of the rolled material is shifted in the control unit 806, the process proceeds to step 120, where the position of the leading end of the rolled material is not shifted. If not, the process proceeds to step 109.

In step 109, the control unit 806 determines whether or not the leading end of the rolled material has passed the lower end of the screen, and if the leading end has exceeded the lower end of the image, proceeds to step 1021. Proceed and if not, step

Returning to the step 105, the position of the end point of the rolled material is detected again.

In step 1020, the alarm issuing unit 110 notifies the user of the displacement of the rolled material. Then, an operation procedure is displayed on the screen display section 109 so as to return the rolled material to a normal position.

In step 1021, the end processing of the present system is performed.

The image processing system configured as described above has the following effects.

(1) Using control information obtained from plant equipment to know the brightness, saturation, hue, visible part, and background change of an object, image recognition processing according to the recognition target and the state of the background can be performed. Since the recognition accuracy is high, the recognition process can be simplified.

(2) Roll position information is acquired, and based on the information, the position and size of the area are set so that the roll does not enter the image processing area. The meandering of the material can be detected. Industrial applicability

As described above, the image processing system according to the present invention determines image processing conditions using control information acquired from a plant device or the like, and is useful for improving recognition accuracy and simplifying recognition processing.

Claims

1. An image processing system that performs image processing on the image by capturing an image related to a device whose movement is controlled by control information. A control information acquisition unit that captures the control information, and the control information that is captured by the unit. An image processing condition setting means for determining setting conditions relating to the image processing based on the image processing system.

2. The image processing system according to claim 1, wherein the image processing condition setting means sets the contents of an image processing filter based on the control information.

3. The image processing system according to claim 2, wherein the image processing condition setting means sets a parameter of a matrix of the image processing filter based on the control information. .

4. The image processing system according to claim 1, wherein the image processing condition setting means sets a binarization threshold in an image binarization process based on the control information. .

5. The image processing system according to claim 1, wherein the image processing condition setting means sets the content of a background image to be referred to at the time of background removal based on the control information.

6. The image processing system according to claim 1, wherein the image processing condition setting means sets a recognition target based on the control information.

7. The image processing system according to claim 6, wherein the image processing condition setting means sets a template for pattern matching of a recognition target based on the control information.

8. The image processing system according to claim 1, wherein the image processing condition setting means is provided. The image processing system according to claim 1, wherein at least one of a position, a size, and a shape of an area to be image-processed is set based on the control information. 9. The image processing system according to claim 1, further comprising a state estimating unit, wherein the state estimating unit estimates a state of a recognition target based on the control information.

10. The image processing system according to claim 9, wherein the state estimating unit estimates at least one of brightness, saturation, and hue of the recognition target based on the control information. system.

11. The image processing system according to claim 9, wherein the state estimating unit estimates at least one of a position and a speed of the recognition target based on the control information.

12. The image processing system according to claim 9, wherein the state estimating unit estimates at least one of a shape and a size of the recognition target based on the control information.

13. An image processing method for capturing an image related to a device whose movement is controlled by control information and performing image processing on the image, the control information acquiring step of capturing the control information, and the step of capturing the control information. An image processing condition setting step of determining a setting condition for the image processing based on the control information.

14. The image processing method according to claim 13, wherein the image processing condition setting step sets contents of an image processing filter based on the control information.

15. The image processing method according to claim 14, wherein the image processing condition setting step sets a matrix parameter of the image processing filter based on the control information. .

16. The image processing method according to claim 13, wherein the image processing condition setting step sets a binarization threshold value in an image binarization process based on the control information. Image processing method.

17. The image processing method according to claim 13, wherein the image processing condition setting step sets the content of a background image to be referred to at the time of background removal based on the control information. Method.

18. The image processing method according to claim 13, wherein the image processing condition setting step sets an object to be recognized based on the control information.

19. The image processing method according to claim 13, wherein the image processing condition setting step sets a template for pattern matching of a recognition target based on the control information. .

20. The image processing method according to claim 13, wherein the image processing condition setting step sets at least one of a position, a size, and a shape of an area to be image-processed based on the control information. An image processing method comprising: 2 1. An image processing method characterized by affinity-converting a recognized image and recognizing a linear component in the image based on a density distribution projected on an arbitrary axis after the affinity conversion.

22. In a rolling material meandering detection system that captures an image of a rolled material rolled by a rolling roll whose movement is controlled by control information and performs image processing on the image to detect the meandering of the rolled material, A control information acquiring unit for acquiring setting information, and an image processing condition setting unit for determining a setting condition relating to the image processing based on the control information acquired by the unit. Detection system.

23. The meandering detection system according to claim 22, wherein the control information obtaining means is provided. The position information of the roll is fetched in the image processing condition setting means. Based on the position information of the roll, at least one of the position, size, and shape of the area is set so that the roll does not enter the area where the image processing is performed. A meandering detection system characterized by setting one.

24. The meandering detection system according to claim 22, further comprising state estimating means, wherein the control information acquiring means fetches the rotation speed information of the roll, and the state estimating means stores the rotation speed information of the roll in the rotation speed information. A meandering detection system that estimates the position of a rolled material in a captured image based on the information.

25. The image processing system according to claim 24, wherein, based on the position information of the rolled material estimated by the state estimating means, the region to be recognized of the rolled material falls within the region to be image-processed. A meandering detection system characterized by setting at least one of a position, a size, and a shape.

26. The image processing system according to claim 22, wherein the control information acquisition means captures the width information of the rolled material, and the image processing condition setting means determines that the rolled material is normal based on the width information of the rolled material. A meandering detection system characterized by setting a guide line for measuring a positional deviation between a track when a sheet is conveyed and a track actually conveyed.