CN100393486C - A Fast Tracking Method Based on Object Surface Color - Google Patents

Abstract

A fast tracking method and device based on the surface color of an object. An image acquisition card is installed in a computer, and images of moving objects are collected into the computer through a camera and the image acquisition card. Then use a specific image processing algorithm to select the required object according to the characteristics of the color block on the surface of the object, and give the centroid position of the object image. The difference between the position of the center of mass of the object image and the position of the given image point is used as the feedback control amount to control the movement of the robot, thereby driving the camera movement, and realizing fast tracking of the object. The image processing method of the invention is simple and fast, and independently becomes a unit, and has strong adaptability and transplantability. The learning method based on color information is adopted in image processing, which has good adaptability to changes in objects and ambient light. Ability to keep objects within the camera's field of view at all times. The invention is suitable for the fields of intelligent monitoring, automatic detection of industrial products, visual control of assembly lines and the like.

Description

A Fast Tracking Method Based on Object Surface Color

technical field

The invention belongs to the technical field of visual tracking in the field of robots, in particular to a method and device for obtaining a surface image of a moving object, selecting a specific object and realizing fast tracking according to the color characteristics of the image.

Background technique

At present, in the vision-based fast-moving object tracking research, the object to be tracked has obvious color characteristics, so the method of pasting a single color color scale on the surface of the object is used (see "Hu Ying, Zhao Shuying, Xu Xin for its typical structure. and, Research on Color Scale Design and Recognition Algorithm, Chinese Journal of Image and Graphics, Vol. 7 (Edition A), No. 12, December 2002, pp. 1291~1295"). The method of pasting the color code has certain limitations and cannot be applied to occasions such as intelligent monitoring and assembly line parts tracking.

Contents of the invention

The object of the present invention is to provide a fast tracking method based on the surface color of an object, which is a simple, fast and effective image processing method suitable for visual tracking.

Another object of the present invention is to provide a device for implementing a fast tracking method based on the surface color of an object.

In order to achieve the above object, the technical solution of the present invention is to provide a fast tracking method based on the surface color of the object, which performs image recognition according to the following steps during the movement of the object:

Step 1: First, separate the object that needs to be identified and tracked from the background, and then collect the image in real time. For each image, after comparing with the calculated HSV threshold, the pixel area that meets the color range is reserved. , and the rest is removed as the background to segment the object image:

F(x,y)=1(t1<=F(x,y)<=t2)

F(x, y) = 0 (other);

Step 2: binarize the segmented object to generate a binarized black-and-white image, and filter the binary image to obtain a smooth black-and-white image;

Step 3: use the Canny operator to sharpen the edge, and use the expansion algorithm to achieve the effect of removing small holes;

Step 4: Use the edge extraction algorithm to get the outline of the object;

Step 5: Image recognition based on shape features, according to the geometric model of the tracked object, eliminate the pixel area that does not conform to the geometric model, and find the centroid of the image of the tracked object;

Step 6: After determining the centroid of the object image, take the difference between the position of the centroid of the object image and the position of the given image point as the feedback control amount, control the robot to drive the camera to move, keep the image of the object within the field of view of the camera, and track the moving objects.

In the fast tracking method, the fifth step: image recognition based on shape features uses shape parameters, and the shape parameter F describes the compactness of the region to a certain extent, which is based on the perimeter B of the region and the region The area A is calculated from:

F＝B*B/(4*PI*A)

Among them, the shape parameter takes the minimum value of 1 for the circular area, and F is always greater than 1 when the area is of other shapes.

Described fast tracking method, it also comprises before motion tracking, learns, and it adopts the method for online learning: a) before carrying out tracking task, obtain a digitized RGB color image through image acquisition card; b) The user uses the mouse to select the rectangular area of the object to be tracked; c) the operating system stores the selected partial image on the computer in the form of a BMP file, which is used as the threshold for subsequent recognition and the basis for segmenting the real-time collected image; d) The local color image is transformed into an HSV model, and the histograms of its H and S components are made respectively to obtain the H and S thresholds of the selected area.

In the fast tracking method, the H and S thresholds, in the subsequent real-time image recognition, the thresholds will not change as the criteria for object segmentation until the user re-learns.

In the fast tracking method, without calibrating the camera, the error between a given point in the image and the center of mass of the object is used as the control feedback to realize fast and accurate visual tracking.

The fast tracking method, based on the color information of the object surface, recognizes the object and performs visual tracking.

A device for realizing a fast tracking method based on the surface color of an object, comprising a robot, a robot control system, and a vision processing system. The robot control system is composed of a main control computer and a robot controller. The vision processing system is composed of a camera, an image acquisition card and Composition of image processing computer, wherein the camera is installed at the end of the robot, the output terminal of the camera is connected to the image acquisition card, the image acquisition card is placed in the image processing computer, the robot is electrically connected to the robot controller, the robot controller and the image processing computer are respectively connected to the main computer electrical connection.

The device, wherein the robot is a five-degree-of-freedom robot, is composed of a three-degree-of-freedom rectangular coordinate robot and a two-degree-of-freedom rotating wrist, the rotating wrist is installed at the end of the vertical axis of the rectangular coordinate robot, and the rotating wrist A camera is fixedly connected to the robot; the robot is controlled by a master computer and a robot controller.

Described device, its described image acquisition card and image processing computer are to select the PCI bus image acquisition card for use, and the image card is installed in the general-purpose PC that main frequency is ≥ 2.8G, constitutes the image processing system.

The outstanding feature of the present invention is that the camera does not need to be calibrated, and does not need to paste color marks, and can quickly track objects whose surfaces are covered with multiple colors.

The image processing method of the invention is simple, fast, and effective, and independently forms a unit, with strong adaptability and strong portability. In the image processing, the learning method based on color information is adopted, which has good adaptability to changes in objects and ambient light. In the case of not calibrating the camera, use the camera and the image acquisition card to obtain the image of the moving object, and use a special image processing algorithm to learn the color features and the area of the color block on the surface of the object, and the learning result is used as the moving object tracking During the process, objects are identified and criteria for defining objects. After obtaining the image of the tracked object, calculate the image centroid position c(u _c , v _c ), the distance between the object image centroid position c(u _c , v _c ) and the given point s(u _s , v _s ) in the image The pixel difference e is used as the visual feedback to control the robot to drive the camera to move, and use the robot's rotating joints to track the moving object. The response is fast, the tracking speed is fast, and the image of the object can always be kept in the camera's field of view. The visual processing method described in the invention is not sensitive to changes in ambient light, and is suitable for tracking moving objects covered by multiple colors on the surface.

The invention is suitable for the fields of intelligent monitoring, automatic detection of industrial products, visual control of pipelines and the like.

Description of drawings

Fig. 1 is the schematic diagram of the device for realizing the fast tracking method based on the surface color of the object in the present invention;

FIG. 2 is a schematic diagram of the process of processing a moving image by the fast tracking method based on the surface color of an object in the present invention.

Detailed ways

A device for implementing a fast tracking method based on the surface color of an object, including a robot, a robot control system, and a vision processing system. The principle of the overall device is shown in Figure 1. The robot control system device is composed of a main control computer and a robot controller, and the vision processing system is composed of a camera, an image acquisition card, and an image processing computer. Wherein, in the present invention, the camera is installed at the end of the robot, the output terminal of the camera is connected to the image acquisition card, the image acquisition card is placed in the image processing computer, the robot is electrically connected to the robot controller, and the robot controller and the image processing computer are respectively electrically connected to the main computer .

In the image processing algorithm, the online learning method is used to obtain a digitized RGB color image through the image acquisition card before performing the tracking task. The user uses the mouse to select a rectangular area of the object to be tracked. The system stores the selected partial images on the computer in the form of BMP files, which will be used as the threshold for later recognition and the basis for segmenting real-time collected images. The local color image is transformed into an HSV model, and the histograms of its H and S components are made respectively to obtain the H and S thresholds of the selected area. In the subsequent real-time image recognition, the threshold will not change as a criterion for object segmentation until the user re-learns.

The advantage of this learning process is that when the tracking object changes, there is no need to make any changes to the program. Every time when the conditions change, such as the light changes significantly, and the tracked object changes, you only need to take a picture before tracking. The current picture, select the object to be tracked with the mouse to complete the learning process.

When the tracking starts, the program first reads the BMP file of the image of the area of the object, and generates the HSV histogram and threshold of the tracking object for the BMP file. The image card collects images in parallel in real time, and compares each image with the threshold , remove the background, segment the object, and find the image edge and center of the object. In the case that the tracked object does not change and the light does not change strongly, there is no need to re-learn until the tracking process is completed. The complete process is shown in Figure 2.

When the moving object is tracked, the robot drives the camera to move, so that the moving object is always within the field of view of the camera. During this process, the steps of object image recognition are as follows:

Step 1: First, separate the object to be recognized and tracked from the background. The background is the collection of stationary pixels on the image, it does not belong to any moving object in front of the camera. Then collect images in real time. For each image, after comparing with the HSV threshold value just calculated, keep the pixel area that meets the color range, and remove the rest as the background to segment the image of the object.

F(x,y)=1(t1<=F(x,y)<=t2)

F(x, y) = 0 (other)

Step 2: binarize the segmented object to generate a binarized black-and-white image, and filter the binary image to obtain a smooth black-and-white image;

Step 3: use the Canny operator to sharpen the edge, and use the expansion algorithm to achieve the effect of removing small holes;

Step 4: Use the edge extraction algorithm to get the outline of the object;

Step 5: Image recognition based on shape features, find the center point of the object according to the geometric model of the object. Using the shape parameter, the shape parameter F describes the compactness of the region to a certain extent, which is calculated from the perimeter B of the region and the area A of the region:

F＝B*B/(4*PI*A)

Among them, the shape parameter takes the minimum value of 1 for the circular area, and when the area is of other shapes, F is always greater than 1. For example, when recognizing a spherical object, the noise with too small area is firstly filtered out by the area threshold. Then considering the area where F is closest to 1, the regular circle in the picture can be distinguished from other shapes. If other regular shapes need to be recognized, such as a square, the F value of the square can be obtained through the characteristics of the square to be close to 4/PI (=1.3).

Step 6: After determining the center point of the object, control the robot to drive the camera to move, keep the image of the object in the field of view of the camera, and track the moving object.

An example of the present invention is given below. In the example, the camera is installed at the end of a five-degree-of-freedom robot. The robot is composed of a three-degree-of-freedom rectangular coordinate robot and a two-degree-of-freedom rotating wrist. The rotating wrist is installed at the end of the vertical axis of the rectangular coordinate robot. The robot Controlled by a main control computer and controller. Fix an industrial standard color camera on the rotating wrist, select the OK series PCI bus image acquisition card, and install the image card in a general-purpose PC with a main frequency of 2.8G to form an image processing system. The working principle of the whole device is shown in Figure 1.

The application example system is used to track a remote control car under natural lighting. The surface of the remote control car is yellow and green, and the front and rear windows are black. Using the learning method described in the present invention, before the motion tracking, learning is carried out to obtain the H and S thresholds of the surface color of the remote control car. Use the remote control to control the movement of the car. According to the process shown in Figure 2, the image recognition method from the first step to the sixth step is used to realize the motion tracking of the remote control car.

It can be seen that the method and device of the present invention can realize fast visual tracking of objects with complicated surface colors without the need for pasting color marks when the camera is not calibrated.

Claims (6)

1. A fast tracking method based on object surface color, characterized in that: in the process of object motion, image recognition is carried out according to the following steps: Step 1: First, separate the object that needs to be identified and tracked from the background, and then collect the image in real time. For each image, after comparing with the calculated HSV threshold, the pixel area that meets the color range is reserved. , and the rest will be removed as the background, and the object image will be segmented. This step will segment multiple objects with similar colors: F(x,y)=1(t1<=F(x,y)<=t2) F(x, y) = 0 (other); Step 2: binarize the segmented object image to generate a binarized black-and-white image, and filter the binary image to obtain a smooth black-and-white image; Step 3: use the Canny operator to sharpen the edge, and use the expansion algorithm to achieve the effect of removing small holes; Step 4: Use the edge extraction algorithm to get the outline of the object; Step 5: Image recognition based on shape features, according to the geometric model of the tracked object, eliminate the pixel area that does not conform to the geometric model, and find the centroid of the image of the tracked object; Step 6: After determining the centroid of the object image, take the difference between the position of the centroid of the object image and the position of the given image point as the feedback control amount, control the robot to drive the camera to move, keep the image of the object within the field of view of the camera, and track the moving objects. 2. The fast tracking method as claimed in claim 1, characterized in that: the fifth step: the image recognition based on shape features is to use shape parameters, and the shape parameter F describes the compactness of the region to a certain extent, it It is calculated based on the perimeter B of the area and the area A of the area: F＝B*B/(4*PI*A) Among them, the shape parameter takes the minimum value of 1 for the circular area, and F is always greater than 1 when the area is of other shapes. 3. fast tracking method as claimed in claim 1, it is characterized in that: also comprise before motion tracking, learn, and it adopts the method for on-line learning: a) before carrying out tracking task, through image acquisition card, obtain a A digitized RGB color image; b) the user uses the mouse to select the rectangular area of the object to be tracked; c) the operating system stores the selected partial image on the computer in the form of a BMP file, which is used as the threshold for later identification and for real-time acquisition The basis for segmenting the image; d) convert the local color image into an HSV model, and make histograms of its H and S components respectively to obtain the H and S thresholds of the selected area. 4. The fast tracking method according to claim 3, characterized in that: the H and S thresholds, in the subsequent real-time image recognition, the thresholds will not change as the criteria for object segmentation until the user relearns. 5. The fast tracking method according to claim 1, characterized in that: without calibrating the camera, the error between a given point in the image and the center of mass of the object is used as the control feedback amount to realize fast and accurate visual tracking . 6. The fast tracking method according to claim 1, characterized in that: the object is recognized and visually tracked based on the color information of the object surface.

Images