KR101486308B1 - Tracking moving objects for mobile robots control devices, methods, and its robot - Google Patents

Abstract

The present invention relates to a method and an apparatus for following a moving object for a mobile robot. The apparatus for controlling a mobile robot comprises: a camera unit which is attached to a fixed position on a mobile robot in order to photograph images; a laser scanner unit which photographs laser signals through a laser scanner; and a control unit which filters the photographed images, calculates the photographed laser signals, and generates a control signal based on the filtered images and the calculated laser signals in order to track a moving object. Therefore, the apparatus can easily determine changes in lighting or the shape of the moving object and can overcome the existing problems due to a narrow angle of view.

Description

 Technical Field [0001] The present invention relates to a mobile robot control device, a method, and a mobile robot for tracking a moving object,

The present invention relates to an object recognition technology of a mobile robot, and more particularly, to a mobile robot control method and apparatus for tracking a moving object for a mobile assistance using a complementary tracker of a camera and a laser scanner.

Recently, robot technology is developing in various fields. Especially, it is a tendency that many robots are being developed for the purpose of service and mobility assistance as human-centered technology.

These robots are effective in providing various services by moving close to human on the extension line of Human-Robot Interaction (HRI). Computer vision technology in robots is responsible for vision, which is implemented using cameras.

However, there are many problems in object tracking in mobile robots because they are tracked under various lighting conditions such as day, night, indoor, etc and various object color and shape changes. In general, the factors that cause tracking failure are internal and external.

First, the internal factor is the movement of the moving object itself, such as the change of the posture and shape of the object to be tracked. External factors are the camera shake, the change of the viewpoint, the change of illumination, the complex background, It is a phenomenon that appears in addition to the same moving object being changed. Many of these internal and external factors reduce tracking performance.

In order to solve the above problem, robust object tracking is still being studied. However, since most of the existing studies have cameras installed in the same direction as the direction of the robot, the tracking object is less likely to deviate from the camera angle of view.

This is because if the heading angle of the robot is adjusted simply before the person departs from the field of view of the camera, the subject can be positioned at the center of the image.

In addition, since the calculation of the target point required for the tracking of the mobile robot uses the distances and angles to the target to be tracked, the single camera only has its limitations.

Korean Patent Application No. 10-2010-0129410 discloses a tracking system for tracking the position of an object such as a person or an object by fusing a laser scanner and a video camera and is provided with a horizontal The present invention relates to an apparatus and a method for automatically correcting an arbitrary bias and measuring a horizontal bias correction value. In the method of recognizing and filtering a background in a crowded image, .

 Korean Patent Application No. 10-2011-0119438 discloses a robot in which a human operator's tracking mobile robot has a space capable of holding predetermined tools or objects while moving forward and backward. In the robot, The present invention relates to an apparatus and method for monitoring whether an object wears a reflective band for recognizing a worker through an optical signal reflected by an object, It is different from the present invention.

Korean Patent Application No. 10-2009-0039481 uses distance measuring sensors in recognizing a target object but recognizing objects in a similar way. Since only the sensor for measuring the distance is used, the method is different from the method for recognizing the object through the laser of the present invention.

Korean Patent Application No. 10-2009-0134721 is similar to the present invention in recognizing and following an object, but it is different from the present invention in that a method of identifying and recognizing a shape of a leg is different from that of the present invention, , And how to recognize the target is also different from the present invention. Also, in the case of laser sensors, there is also a difference in the method of recognizing the object, since the region that recognizes the object is limited to the legs. Therefore, there is a difference from the present invention.

SUMMARY OF THE INVENTION The present invention provides a mobile robot control device for extracting shape characteristics from a video signal of a moving object and solving the problem of recognition capability of the extracted video signal to follow a moving object not disturbed by the external environment, And a mobile robot.

According to another aspect of the present invention, there is provided a mobile robot control device for tracking a moving object that recognizes and tracks an object on a front side as well as a front side based on a camera installed on a side surface of the mobile robot and a laser scanner installed on the front side, To provide a robot.

Another object of the present invention is to provide a mobile robot control device, a method, and a mobile robot for following a moving object that keeps track of a moving object even when the moving object deviates from an angle of view of a camera.

A camera unit for photographing an image through a camera attached to a predetermined position of the mobile robot, a laser scanner unit for photographing a laser signal through a laser scanner, filtering the photographed image, calculating the photographed laser signal, And a control unit for generating a control signal for tracking the moving object based on at least one of the calculated image and the computed laser signal.

And the camera unit photographs an image of a moving path side of the mobile robot.

The laser scanner unit includes an angle of view that overlaps with the camera unit, has a wider angle of view than the angle of view of the camera unit, and can capture the front of the robot path to generate a laser signal.

The control unit can confirm the geometrical relationship between the laser scanner and the camera using the installed chess board.

The laser scanner unit may generate a laser signal that is corrected according to the identified geometric relationship.

The control unit may include a filtering unit that detects a moving object to be detected based on the photographed image signal based on the detected shape characteristic and a moving object to be followed based on the detected shape characteristic, A position of at least one of the moving object recognized by the filtering unit and the moving object recognized by the laser signal processing unit is calculated and a control signal for moving the mobile robot in accordance with the calculated position is generated And a position calculation unit.

The filtering unit may include a shape feature detecting unit for extracting shape features of the image captured by the camera unit, and a moving object recognizing unit for recognizing the moving object based on the extracted shape feature.

The shape feature detector may detect the shape feature by applying a Histogram of Oriented Gradients (HOG) algorithm.

The moving object recognizing unit may recognize the moving object by applying a particle filter algorithm after applying a bidirectional PCA (BDPCA) algorithm.

The laser signal processing unit may apply a mean-shift algorithm for extracting a relative distance and an angle between the mobile robot and the moving object based on the photographed laser signal.

Wherein the position calculating unit generates a control signal based on the image signal and, when the image is out of the angle of view of the image, tracks the moving object based on the laser signal of the laser scanner unit, Signal can be generated.

The method includes capturing an image with a camera attached to a predetermined position of a mobile robot, photographing a laser signal through a laser scanner, filtering the photographed image, computing the photographed laser signal, And generating a control signal for tracking the moving object based on at least one of the computed laser signals.

The step of photographing the image may include photographing an image of a side of the movement path of the mobile robot.

The step of photographing the laser signal includes a step of generating a laser signal by photographing the forward part of the path of the mobile robot including an angle of view overlapping with the photographed image and having a wider angle of view than the angle of view of the photographed image .

The generating of the control signal may include finding a geometric relationship between the laser scanner and the camera using a chess board installed.

The step of photographing the laser signal may generate a laser signal corrected according to the identified geometric relationship term.

The generating of the control signal may include detecting a shape feature based on the photographed image signal and recognizing a moving object to follow based on the detected shape characteristic, A position of at least one of a moving object recognized based on the video signal and a moving object recognized based on the laser signal is calculated, and the moving robot is moved according to the calculated position And generating a control signal.

The step of recognizing the moving object based on the video signal may include extracting a shape characteristic of the photographed image signal and recognizing the moving object based on the extracted shape characteristic.

The step of extracting feature features of the image may include detecting the feature features by applying a Histogram of Oriented Gradients (HOG) algorithm.

The step of recognizing the moving object may include recognizing the moving object by applying a particle filter algorithm after applying a bidirectional PCA (BDPCA) algorithm.

The step of recognizing the moving object based on the laser signal may include applying a mean-shift algorithm for extracting a relative distance and an angle between the mobile robot and the moving object based on the photographed laser signal . ≪ / RTI >

The generating of the control signal may include generating a control signal based on the image signal, tracking the moving object based on the laser signal when the image signal is out of the angle of view of the image, And generating a control signal based on the control signal.

A camera unit for photographing an image attached to a predetermined position of the mobile robot, a laser scanner unit for photographing a laser signal through a laser scanner, filtering the photographed image, calculating the photographed laser signal, And a control unit for generating a control signal for tracking the moving object on the basis of the calculated laser signal, and a mobile device operating based on the control signal.

According to the apparatus and method for controlling a mobile robot for tracking a moving object according to the present invention, a shape characteristic is extracted by applying a HOG algorithm to an image signal of a moving object, and a particle filter is traced based on the extracted image signal. It is possible to have a strong cognitive ability to change the shape of the object or illumination change.

 The angle of view of the camera for photographing the side and the angle of view of the laser scanner for photographing the front are overlapped with each other so that the moving object can be recognized by the signal of the laser scanner even when the moving object deviates from the angle of view of the camera.

When the angle of the camera is out of the angle of view, the laser scanner that captures the front side can recognize the object and overcome the angle of view problem of the camera and combine the signals of the camera and the laser scanner to continuously track the moving object.

1 is a block diagram illustrating a mobile robot control apparatus for following a moving object according to an embodiment of the present invention.
FIG. 2 is a block diagram illustrating a configuration of a controller of a mobile robot control apparatus for following a moving object according to an embodiment of the present invention. Referring to FIG.
3 is a block diagram illustrating a configuration of a filtering unit of a mobile robot control apparatus for following a moving object according to an embodiment of the present invention.
4 is a diagram illustrating a process of detecting a shape feature of a moving object in a shape feature detector of a mobile robot controller for tracking a moving object according to an exemplary embodiment of the present invention.
5 is a conceptual diagram illustrating calibration of a camera and a laser scanner using a chess board in a mobile robot controller for tracking a moving object according to an embodiment of the present invention.
6 is a view showing an image of a laser signal photographed by a laser scanner unit of a mobile robot controller for tracking a moving object according to an embodiment of the present invention.
7 is a simplified flowchart of a mean-shift algorithm applied to a laser signal processing unit of a mobile robot control apparatus for tracking a moving object according to an embodiment of the present invention.
8 is a diagram illustrating an example in which a moving object is recognized in a laser signal processing unit of a mobile robot control apparatus for tracking a moving object according to an embodiment of the present invention.
9 is a diagram illustrating a process of tracking a moving object by combining a captured image signal and a photographed laser signal in a mobile robot controller for tracking a moving object according to an embodiment of the present invention.
FIG. 10 is a flowchart illustrating a process of recognizing and tracking an object by a mobile robot controller for tracking a moving object according to an exemplary embodiment of the present invention. Referring to FIG.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings, in order to facilitate a person skilled in the art to easily carry out the technical idea of the present invention. .

The various features used in object tracking can be divided into color features and shape features. Color information of an object is the most intuitive feature, but object tracking in a robot requires more robust features because background and lighting conditions change easily. Therefore, in the present invention, a shape feature that is less influenced by illumination than a color-based feature is used.

1 is a block diagram illustrating a mobile robot control apparatus for following a moving object according to an embodiment of the present invention.

Referring to FIG. 1, the camera unit 110 is attached to one side of a robot and photographs an external image using a camera. The camera may be attached to a side surface of the robot to take a side view of the path of the mobile robot.

The control unit 150 can detect the shape characteristic of the moving object based on the image signal acquired by the camera unit 110. [ The method for detecting the shape feature may be a histogram of gradients (HOG) algorithm.

The control unit 150 can recognize the moving object based on the detected shape characteristics. The moving object may be recognized by using a bidirectional PCA (BDPCA) algorithm and a particle filter algorithm, which are robust object detection algorithms.

The laser scanner unit 130 is provided with a laser scanner at an angle of view overlapping the camera unit 110 and can receive a laser signal through the laser scanner. The laser scanner unit 130 can sense a range of 240 degrees forward. The laser scanner unit 130 may be a laser scanner having a wider angle of view than the angle of view of the camera installed in the camera unit 110.

The control unit 150 can recognize the moving object from the laser signal acquired by the laser scanner unit 130. [ The control unit 150 may combine the laser signal photographed by the laser scanner unit 130 and the image signal photographed by the camera unit 110 because the laser scanner unit 130 and the camera unit 110 include the overlapping angle of view The geometric relationship can be known. And the laser signal corrected according to the geometric relationship can be generated. A mean-shift algorithm may be applied to the laser signal. The control unit 150 can recognize the moving object from the laser signal and calculate the relative distance and angle between the robot and the moving object.

The control unit 150 can determine the position of the moving object by combining the video signal photographed by the camera unit 110 and the laser signal photographed by the laser scanner unit 130.

Since the control unit 150 knows the geometric relationships of the sensors of the camera unit 110 and the laser scanner unit 130, the tracking result in the camera image can be matched with the tracking result of the laser signal. The control unit 150 can also track an object outside the angle of view of the camera through the laser signal. The control unit 150 may generate a control signal for following the moving object based on the tracked moving object.

The mobile device 20 may be operated based on the generated control signal. The mobile device 20 can drive to follow the moving object.

FIG. 2 is a block diagram illustrating the configuration of a controller 150 of a mobile robot controller for following a moving object according to an embodiment of the present invention. Referring to FIG.

2, the control unit 150 may include a filtering unit 120, a laser signal processing unit 155, and a position calculating unit 157.

The filtering unit 120 can detect the shape characteristic of the moving object based on the video signal photographed by the camera unit 110. In addition, the filtering unit 120 may recognize the moving object based on the detected shape characteristics.

The laser signal processing unit 155 can recognize the moving object from the laser signal acquired by the laser scanner unit 130. The laser signal processing unit 155 can recognize the moving object from the obtained laser signal and calculate the relative distance and angle between the robot and the moving object.

The position calculation unit 157 may combine the image signal and the laser signal received from the filtering unit 120 and the laser signal processing unit 155 to determine the current position of the robot and the current position of the moving object. The position calculating unit 157 may generate a control signal for controlling the mobile device 20 based on the position of the detected moving object.

3 is a block diagram illustrating the configuration of a filtering unit 120 of a mobile robot control apparatus for following a moving object according to an embodiment of the present invention.

Referring to FIG. 3, the filtering unit 120 may include a shape feature detection unit 122 and a moving object recognition unit 124.

The morphological feature detecting unit 122 can detect the morphological feature of the moving object based on the video signal photographed by the camera unit 110. A method of detecting the shape feature can apply a histogram of gradients (HOG) algorithm that is resistant to changes in the external environment. The detected feature information may be transmitted to the moving object recognizing unit 124.

The moving object recognizing unit 124 can recognize the moving object based on the detected shape characteristics. A method of recognizing a moving object based on the detected shape characteristics may be a particle filter algorithm. The particle filter algorithm is an algorithm that represents a probability distribution as a set of samples and belongs to a recursive Bayesian framework along with a Kalman filter. Before tracking the moving object, a moving object can be detected using a BDPCA (bidirectional PCA) algorithm, which is a robust object detection algorithm.

A moving object signal detected by the BDPCA (bidirectional PCA) algorithm can be used as a square filter by applying each signal to a particle filter. Thus, the signal of the detected moving object can be defined as a state variable as shown in Equation (2).

와

는 속도를 의미한다. 또한, h는 사각형의 세로를 의미하고, 고정된 사각형 비율(aspect ratio)를 사용하기 때문에 사각형의 가로는 w = 1.1h 일 수 있다.In Equation (1), x and y represent the position of the corresponding sample in the image

Wow

Means speed. Also, h means the height of the rectangle, and since the fixed aspect ratio is used, the width of the rectangle can be w = 1.1h.

The particle filter is largely composed of a prediction step and an update step, and each of the steps may be represented by [Equation 2] and [Equation 3].

에

을 곱해 예측

을 수행하고 식 [수학식 3]는 예측된 현재 상태를 기반으로 새로운 관측

을 통해 현재 상태

를 갱신한다. In Equation (2) and Equation (3), z is a measurement variable and Equation (2)

on

Multiply by

And equation (3) is based on the predicted current state and a new observation

Current state through

.

[Mathematical expression 2] can be expressed as a matrix, and can be expressed as [Mathematical expression 4].

는 각 변수의 잡음을 발생시키기 위한 난수의 분산 값이다.In Equation (4)

Is the variance of the random number to generate the noise of each variable.

를 누적하여 작성된 히스토그램

과 현재 샘플에 해당하는 히스토그램

를 비교할 수 있다. 상기 유사성을 판단하는 과정은 [수학식 5]과 같이 Bhattacharyya distance를 이용할 수 있다. 이때, 두 히스토그램이 완벽하게 일치하면 0, 불일치하면 1이다.The update step is done through a likelihood calculation, which can be done using the previously created HOG descriptor. To determine the similarity between the state vector of each sample and the learned model

Histogram < RTI ID = 0.0 >

And a histogram corresponding to the current sample

Can be compared. The process of determining the similarity can use the Bhattacharyya distance as shown in Equation (5). At this time, if the two histograms are perfectly matched, 0 is obtained.

The reliability of each sample obtained in Equation (5) can be converted into a probability value in the range of 0 to 1 through normalization as in Equation (6) and Equation (7).

The shape characteristic of the moving object recognized by the moving object recognizing unit 124 may be transmitted to the controller 150.

4 is a diagram illustrating a process of detecting a shape feature of a moving object in a shape feature detector 122 of a mobile robot controller for tracking a moving object according to an exemplary embodiment of the present invention. As a method of detecting the shape of the moving object, a Histogram of Gradient (HOG) algorithm, which is one type of shape characteristic, can be used.

Referring to FIG. 4, HOG (Histogram of Gradientients) uses 2 × 3 descriptors in one square, and each descriptor is represented by eight directions. The vectors 410, 420, 430, 440, 450, and 460 represent descriptors in each grid.

In Equation (8), m _i (x, y) means the magnitude of the gradient and θ _i (x, y) means the angle of the gradient. x and y represent the position of the corresponding lattice.

5 is a conceptual diagram illustrating calibration of a camera 110 and a laser scanner 130 using a chess board 510 in a mobile robot controller for tracking a moving object according to an embodiment of the present invention.

Referring to FIG. 5, the laser scanner 130 can photograph the front of the mobile robot. The camera 110 can photograph the side of the mobile robot. The laser scanner 130 and the camera 110 may include an overlapping angle of view. The geometric relationship between the camera 110 and the laser scanner 130 must be known in order to share the information of the camera 110 and the camera 110 having different sensing ranges and installation directions. Therefore, the chess board 510 can be disposed at the overlapping angle of view between the camera 110 and the laser scanner 130, and the Q.Zhang algorithm can be applied based on the image and the laser distance value obtained at the same time.

Calibration is a process of obtaining a 3 × 3 rotation matrix Φ and a 3D motion vector Δ from the laser scanner 130 to the camera 110 as shown in Equation (9).

The coordinate of arbitrary laser data on the chess board 510 is referred to as P _l , and this coordinate is represented by P _c in the camera coordinate system. In Equation (9), K is a parameter inside the camera and p is a coordinate in the image.

로 표현될 수 있다. 상기 캘리브레이션(calibration)을 위해 레이저 스캐너(130)는 지면과 수평으로 놓고, 카메라(110)는 레이저 스캐너(130) 위에서 오른쪽으로 90도 정도 회전시켜 장착할 수 있다. 상기 레이저 스캐너(130)의 레이저 신호가 카메라(110)의 화각에 들어오게 하기 위해 한 변이 100mm인 격자무늬를 이용하여 10×7의 체스보드(510)를 이용할 수 있다.The calibration between the camera 110 and the laser scanner 130 is performed using a moving matrix and a rotation vector

. ≪ / RTI > The laser scanner 130 may be horizontally placed on the ground for the calibration and the camera 110 may be mounted on the laser scanner 130 by turning it 90 degrees to the right. A 10 × 7 chess board 510 can be used by using a grid pattern having a side length of 100 mm so that the laser signal of the laser scanner 130 enters the angle of view of the camera 110.

는 [수학식 10]과 같다.In one embodiment of the present invention,

Is expressed by Equation (10).

The laser signal processing unit 155 can project the photographed laser signal into the image using the rotation matrix and the motion vector as shown in Equation (10) and using Equation (9).

6 is a view showing an image of a laser signal photographed by a laser scanner unit of a mobile robot controller for tracking a moving object according to an embodiment of the present invention.

Referring to FIG. 6, the calibration of the camera 110 and the laser scanner 130 may be combined to project the laser signal data into an image. The image 511 is displayed on the laser scanner unit 130 and can be recognized as the laser signal 515. Further, the image 521 may be photographed by the laser scanner unit 130 and recognized as a laser signal 525. [

7 is a simplified flowchart of a mean-shift algorithm applied to the laser signal processor 155 of the mobile robot controller for tracking a moving object according to an embodiment of the present invention.

Referring to FIG. 7, the mean-shift algorithm indicates a process of recognizing a person in the laser signal processor 155.

The mean-shift algorithm is known to be a robust method of finding local extrema in the density distribution of a data set. The mean-shift algorithm works well for continuous distributions, which can be viewed as a 'hill climbing' method that is well applied to the density histogram of the data. In a discrete data set, however, the kernel operates using only the data points existing in the window. The kernel used in this algorithm uses a uniform kernel with all the weights in the window and can perform mean-shift tracking in the 2D laser domain. Since the relative distance and angle between the robot and the person can be extracted using the mean-shift algorithm, the robot can be controlled.

The laser signal processing unit 155 estimates the kernel density based on the laser signal photographed by the laser scanner unit 130 (S100).

The laser signal processing unit 155 estimates the density gradient based on the estimated kernel density (S110).

The laser signal processing unit 155 calculates a vector based on the estimated density gradient (S120). The vector may be a mean-shift vector.

The laser signal processing unit 155 may determine whether the calculated vector is converged (S130). The convergent vector can be recognized as a moving object. If there is no vector converged in step S130, the process of step S120 may be performed again.

8 is a diagram illustrating an example in which a moving object is recognized in a laser signal processing unit of a mobile robot control apparatus for tracking a moving object according to an embodiment of the present invention.

Referring to FIG. 8, the laser signal processor 155 may track a moving object using a mean-shift algorithm. In the mean-shift algorithm, the relative distance and angle between the robot and the moving object can be extracted. The recognition point 711, the recognition point 721, the recognition point 731, the recognition point 741, the recognition point 751 and the recognition point 761 may represent the recognized moving object. The recognition point 715, the recognition point 725, the recognition point 735, the recognition point 745, the recognition point 755 and the recognition point 765 can represent the current position of the mobile robot.

The laser signal processing unit 155 may extract the relative distance and angle between the current position of the mobile robot and the recognized moving object by determining the recognition points.

9 is a diagram illustrating a process of tracking a moving object by combining a captured image signal and a photographed laser signal in a mobile robot controller for tracking a moving object according to an embodiment of the present invention.

9, a mobile robot controller for tracking a moving object according to an exemplary embodiment of the present invention synthesizes signals from a camera unit 110 and a laser scanner unit 130, Even if you do, you can see that tracking is done.

The recognition point 811, the recognition point 821, the recognition point 831, and the recognition point 841 can represent the current position of the mobile robot. The recognition point 813, the recognition point 823, the recognition point 833, and the recognition point 843 may represent positions where the moving object is recognized.

The image 815 can determine the moving object 817 based on the video signal and can determine the current position 813 based on the laser signal. The image 815 may recognize the moving object 817 and may combine with the laser signal to determine the current location 813 of the moving object. The image 825 can determine the moving object 827 based on the video signal and can determine the current position 823 based on the laser signal. The image 825 can recognize the moving object 827 and combine with the laser signal to determine the current position 823 of the moving object.

The image 835 can determine the moving object 837 based on the video signal and can determine the current position 833 based on the laser signal. The image 835 may recognize the moving object 837 even with the obstacle 839 and may combine with the laser signal to determine the current position 823 of the moving object.

The image 845 can not determine the moving object based on the video signal, such as the image 847, because there is no moving object. However, the current position 843 of the moving object may be determined based on the laser signal, and the moving device 20 of the mobile robot may be driven to continuously track the moving object.

FIG. 10 is a flowchart illustrating a process of a mobile robot control device for tracking a moving object according to an embodiment of the present invention.

Referring to FIG. 10, when tracking is performed using the mobile robot control method for tracking a moving object according to an embodiment of the present invention, an object outside the angle of view of the camera can be tracked.

The laser scanner unit 130 captures a laser signal from the laser scanner (S200). The laser scanner photographs the path ahead of the mobile robot and can have an angle of view of 240 degrees. The laser scanner may have a geometric relationship with the camera. The laser signal may be transmitted to the laser signal processing unit 155.

The laser signal processing unit 155 determines a moving object based on the laser signal (S210). The method of determining the moving object may be a mean-shift algorithm. The determined laser signal may be transmitted to the control unit 150.

The camera unit 110 captures an image from an attached camera (S220). The camera may be attached to one side of the mobile robot to photograph the side of the mobile robot path. The photographed image may be transmitted to the filtering unit 120.

The filtering unit 120 processes the photographed image to recognize a moving object (S230). In the process of processing the image, a HOG algorithm and a particle filter algorithm for detecting a shape feature may be applied. The photographed image may not be recognized as a moving object. The recognized video signal may be transmitted to the control unit 150.

The control unit 150 recognizes the moving object by combining the laser signal transmitted from the laser signal processing unit 155 and the image transmitted from the filtering unit 120 at step S240. The control unit 150 may determine whether the moving object is recognized in the combined image.

If the moving object is not recognized in step S240, the position calculating unit 157 may generate a control signal based on the laser signal (S250). The generated control signal may be transmitted to the mobile device 20.

If the moving object is recognized in step S240, the position calculating unit 157 may generate a control signal based on the video signal (S260). The generated control signal may be transmitted to the mobile device 20.

The mobile device 20 operates based on the generated control signal (S250). The mobile device 20 can move following the person.

20: mobile device 110: camera part
120: filtering unit 122: shape feature detecting unit
124: moving object recognition unit 130: laser scanner unit
150: control unit 155: laser signal processing unit
157:

Claims (23)

A camera unit for photographing an image through a camera attached to a side surface of the mobile robot;
A laser scanner unit including an angle of view overlapping with an angle of view of the camera and inputting a geometrical relationship with the camera and generating a laser signal by photographing a front of the robot path; And
And a controller for controlling the movement of the moving object on the basis of the calculated laser signal when the moving object is not present in the photographed image, And a controller for generating a control signal for tracking the moving object based on the filtered image if the moving object is present.
delete delete delete delete The method according to claim 1,
Wherein,
A filtering unit for detecting a moving object to be detected based on the detected shape characteristic on the basis of the detected shape characteristic and a moving object for tracking based on the detected shape characteristic;
A laser signal processor for recognizing a moving object to be followed based on the photographed laser signal; And
And a position calculation unit for calculating a position of at least one of the moving object recognized by the filtering unit and the moving object recognized by the laser signal processing unit and generating a control signal for moving the mobile robot according to the calculated position Wherein the mobile robot is a mobile robot.
The method according to claim 6,
Wherein the filtering unit comprises:
A shape feature detector for extracting a shape feature of an image photographed by the camera unit; And
And a moving object recognizing unit for recognizing the moving object based on the extracted shape characteristic.
8. The method of claim 7,
The shape feature detecting unit,
Wherein the shape feature is detected by applying a Histogram of Oriented Gradients (HOG) algorithm to the mobile robot.
8. The method of claim 7,
The moving object recognizing unit,
Wherein the mobile robot recognizes the moving object by applying a particle filter algorithm after applying a bidirectional PCA (BDPCA) algorithm.
The method according to claim 6,
Wherein the laser signal processing unit comprises:
And a mean-shift algorithm for extracting a relative distance and an angle between the mobile robot and the moving object based on the photographed laser signal is applied to the mobile robot.
delete Capturing an image with a camera attached to a side surface of the mobile robot;
Generating a laser signal by capturing an image of a front side of the path of the mobile robot, wherein the geometric relationship with the camera is input, including an angle of view overlapping with an angle of view of the camera unit; And
And a controller for controlling the movement of the moving object on the basis of the calculated laser signal when the moving object is not present in the photographed image, And generating a control signal for tracking the moving object on the basis of the filtered image if the moving object is present.
delete delete delete delete 13. The method of claim 12,
Wherein the step of generating the control signal comprises:
Detecting a shape feature based on the photographed image signal and recognizing a moving object to follow based on the detected shape characteristic;
Recognizing a moving object to be followed based on the photographed laser signal; And
Calculating a position of at least one of a moving object recognized based on the video signal and a moving object recognized based on the laser signal and generating a control signal for moving the mobile robot according to the calculated position, And controlling the moving robot to follow the moving object.
18. The method of claim 17,
Wherein the step of recognizing the moving object based on the video signal comprises:
Extracting shape characteristics of the photographed image signal; And
And recognizing the moving object on the basis of the extracted shape characteristic.
19. The method of claim 18,
Wherein the step of extracting feature features of the image comprises:
And detecting the shape feature by applying a Histogram of Oriented Gradients (HOG) algorithm to the mobile robot.
19. The method of claim 18,
Wherein the recognizing of the moving object comprises:
And a step of recognizing the moving object by applying a particle filter algorithm after applying a bidirectional PCA (BDPCA) algorithm to the moving object.
19. The method of claim 18,
Wherein the step of recognizing the moving object based on the laser signal comprises:
And applying a mean-shift algorithm for extracting a relative distance and an angle between the mobile robot and the moving object based on the photographed laser signal. Robot control method.
delete A camera unit attached to a side surface of the mobile robot for capturing an image;
A laser scanner part including an angle of view overlapping with an angle of view of the camera part and inputting a geometrical relationship with the camera part and generating a laser signal by photographing the forward part of the robot path;
And a controller for controlling the movement of the moving object on the basis of the calculated laser signal when the moving object is not present in the photographed image, A control unit for generating a control signal for tracking the moving object based on the filtered image when the moving object exists; And
And a mobile device operating on the basis of the control signal.

Images