CN106886216B - Robot automatic tracking method and system based on RGBD face detection - Google Patents
Robot automatic tracking method and system based on RGBD face detection Download PDFInfo
- Publication number
- CN106886216B CN106886216B CN201710028570.5A CN201710028570A CN106886216B CN 106886216 B CN106886216 B CN 106886216B CN 201710028570 A CN201710028570 A CN 201710028570A CN 106886216 B CN106886216 B CN 106886216B
- Authority
- CN
- China
- Prior art keywords
- image
- classifier
- face
- human body
- detection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0231—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
- G05D1/0246—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means
- G05D1/0253—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means extracting relative motion information from a plurality of images taken successively, e.g. visual odometry, optical flow
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0212—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory
- G05D1/0225—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory involving docking at a fixed facility, e.g. base station or loading bay
Abstract
The invention relates to a robot automatic tracking method and system based on RGBD face detection, wherein the system comprises the following steps: the system comprises an RGBD camera, an image algorithm processing platform and a robot motion control unit; the RGBD camera is used for collecting RGB images and depth images of a human body and is used for identifying the upper half body and the face images of the human body; the image algorithm processing platform is used for receiving a human body image acquired by the RGBD camera, realizing human face recognition of a human body through algorithm processing, and calculating the distance between the robot and the human face; the robot motion control unit obtains the longitudinal forward and backward distance of the human face through the image algorithm processing platform and controls the robot to move along with the movement of the human face according to the distance change information. The invention solves the problems of face tracking accuracy and face longitudinal accurate distance measurement, and ensures that the robot moves along with the face with higher accuracy and stronger human-like function.
Description
Technical Field
The invention relates to the technical field of robot control, in particular to an automatic robot tracking method and system based on RGBD face detection.
Background
With the development of computer science and automatic control technology, more and more intelligent robots appear in production and life, and a vision system is more and more valued by people as an important subsystem in an intelligent robot system. The intelligent robot vision system takes the intelligent behavior ability of a human as a blueprint, realizes the aspects of the robot such as environment perception organization, complex scene adaptation, interaction and cooperation, concept formation and integration, knowledge acquisition and reasoning, autonomous cognition and advanced decision making, humanoid intelligent behavior and the like, and develops the intelligent development of the robot. The face detection follows a part of the intellectualization of the robot at any time, and is mainly used for rapid and accurate face detection and tracking, so that better experience is obtained in the communication between the human and the robot.
The domestic patent CN103093212A discloses a method and a device for intercepting face images based on face detection and tracking, the system comprises a detection module, a tracking module, a judgment module and an interception module, and a cascade classifier is adopted to carry out face detection on images to be detected; when the face target is detected, carrying out face tracking on the face target by using a mean value tracking algorithm; when the face target leaves the detection area, judging whether the face detection and the face tracking in the same frame correspond to the same face target or not on each frame according to the position of the target, and selecting each frame of which the face detection and the face tracking correspond to the same face target; calculating the contact ratio of a window of face detection and a window of face tracking in the same frame in each selected frame, and taking a face image obtained by face detection on the frame with the maximum contact ratio as an intercepted face image; the invention has the disadvantages that the cascade classifier is singly used, the classifier has high false detection rate and is not beneficial to being used in a complex environment, and the used 2D camera has no image depth information, so that the size can be only roughly estimated when the face moves forwards and backwards, and the actual distance of the movement forwards and backwards cannot be measured.
The domestic patent CN101477616 discloses a face detection and tracking method, which is executed by a computer or a microprocessor with computing power to identify the face and its position in the image, and the face detection is adopted to perform face tracking on each frame of image, record the face position, and perform face detection once again on the image after several frames of images, under the condition of the found face position, so as to quickly find out other faces that may be added newly.
Accordingly, there is a need in the art for improvements.
Disclosure of Invention
The invention discloses an automatic robot tracking method and system based on RGBD face detection, which are used for enabling a robot to move along with a face with higher accuracy and stronger human-like function.
On one hand, the robot automatic tracking method based on the RGBD face detection provided by the embodiment of the present invention includes:
acquiring human body sample image data, and detecting technical indexes of a classifier through the human body sample image;
using a cascade classifier to detect the upper half of the human body, using depth image data to perform numerical analysis, and confirming the detected human body image;
using a cascade classifier to detect a human face in an image for detecting the upper half of the human body by using a method of integrating image depth information until the human face is detected, and confirming the detected human face image, wherein the cascade classifier is formed by connecting a plurality of strong classifiers which are weighted by a plurality of weak classifiers;
after the human face area is detected, the control module is used for extracting a corresponding area from the depth image to perform distance operation, and the accurate distance between the human and the robot is obtained.
In another embodiment of the robot automatic tracking method based on RGBD face detection according to the present invention, the acquiring human body sample image data includes the following steps:
preparing training data, and making a human body image training sample, wherein the human body image training sample comprises: a negative sample and a positive sample, the negative sample referring to an image not including an object; the positive sample is an image of an object to be detected and cannot contain any negative sample;
using an image test classifier of a finished training sample, and analyzing technical indexes of the classifier according to a test result, wherein the technical indexes comprise: the detection success rate and the false detection rate of the upper half body image and the success rate and the false detection rate of the human face detection;
adjusting technical parameters of a classifier, wherein the parameters adjusted by the classifier comprise: the number of positive samples used during training, the number of negative samples used during training of each stage of classifier, the stage number of the classifier, the haar feature type and the minimum detection rate of each stage of the classifier.
In another embodiment of the robot automatic tracking method based on RGBD face detection according to the present invention, the performing upper body detection on a human body by using a classifier and performing numerical analysis by using depth image data, and the determining an image of the detected human body includes:
firstly, carrying out gray level conversion on an RGB image to change a color image into a gray level image of 0 to 255 orders;
enhancing the brightness of the gray scale image by using a histogram equalization method;
selecting an image detection area, and carrying out integral processing on the selected area, wherein when the rectangular characteristic is calculated by adopting an integral method, the sum of pixel gray values in a rectangle does not need to be counted again every time, only the integral of a plurality of corresponding points of the rectangle needs to be calculated, namely the rectangular characteristic value is calculated, and the calculation time does not change along with the change of the size of the rectangle;
scanning the image detection area once in a whole to obtain a detection factor coefficient;
multiplying the first-stage detection result by the detection factor coefficient to obtain a weak classifier result, and cascading weak classifiers to obtain a strong classifier detection result;
zooming an original image by half, and scanning an image to be detected by using a haar window with a fixed size as a basic window;
calculating the mean value and the variance in windows of different sizes, traversing the image to be detected, and considering the window as the image of the upper half of the human body when the set conditions are met;
calculating Haar-like characteristics in the candidate region, and transmitting the characteristics to a cascade adaboost classifier for further judgment;
if three windows exist and can simultaneously detect the upper half body image of the human body, the output result of the cascade classifier is determined to be a true value, the upper half body coordinate is output, and if the output result of the cascade classifier does not exist, the output result of the cascade classifier is determined to be a non-true value;
calculating distance (x, y) Pix (x, y)/1000 for all pixel values in the upper half body region depth map;
and performing consistency analysis on the measured distance data, and if the measured distance data is consistent, confirming that the target detected in the RGB image exists in the depth image, otherwise, confirming that the target does not exist in the depth image.
In another embodiment of the robot automatic tracking method based on RGBD face detection according to the present invention, the using a classifier to perform face detection on the image in which the upper half of the human body is detected by using a method of merging image depth information until the human face is detected, and confirming the detected face image includes:
pre-loading face detection classifier data, setting the size of a scanning window, wherein the size of the scanning window is not limited and is specifically determined according to the requirements in actual projects;
and scanning the image according to the window with the set scale until the human face is detected.
In another embodiment of the robot automatic tracking method based on RGBD face detection according to the present invention, the performing distance calculation on the region corresponding to the depth image extraction includes:
obtaining the distance faceDis (x, y) of each pixel point in the face area in the depth map as image (x, y)/1000;
and summing and averaging the distances of all the pixel points to finally obtain the actual distance between the robot and the person.
An embodiment of the present invention further provides an RGBD face detection-based robot automatic tracking system, including: the system comprises an RGBD camera, an image algorithm processing platform and a robot motion control unit;
the RGBD camera is used for collecting RGB images and depth images of a human body and is used for identifying the upper half body and the face images of the human body; the image algorithm processing platform is used for receiving a human body image acquired by the RGBD camera, realizing human face recognition of a human body through algorithm processing, and calculating the distance between the robot and the human face; the robot motion control unit obtains the longitudinal forward and backward distance of the human face through the image algorithm processing platform and controls the robot to move along with the movement of the human face according to the distance change information;
the RGBD camera comprises an RGB camera and a depth camera, the RGB camera is used for shooting RGB images of a human body, the depth camera is used for shooting depth images of the human body, and after the RGB camera and the depth camera are calibrated by a camera, images shot by the depth camera are overlapped with images shot by the RGB camera;
the RGBD camera is an intel depth processing camera.
In another embodiment of the robot automatic tracking system based on RGBD face detection according to the present invention, the RGBD camera further includes a cascade classifier, the cascade classifier is configured to detect an upper half image of a human body in an image captured by the RGB camera, and when the upper half image is detected, the cascade classifier is used again for face detection on the upper half region of the image.
In another embodiment of the above robot automatic tracking system based on RGBD face detection according to the present invention, the cascade classifier comprises a plurality of strong classifiers connected together for operation, and the strong classifier comprises a plurality of weak classifiers composed of a plurality of weak classifiers by weighting.
In another embodiment of the robot automatic tracking system based on RGBD face detection according to the present invention, the image algorithm processing platform performs image algorithm processing by adding a haar algorithm and an adboost algorithm to a target depth information analysis method, and includes:
firstly, carrying out gray level conversion on an RGB image to change a color image into a gray level image of 0 to 255 orders;
enhancing the brightness of the gray scale image by using a histogram equalization method;
selecting an image detection area, and carrying out integral processing on the selected area, wherein when the rectangular characteristic is calculated by adopting an integral method, the sum of pixel gray values in a rectangle does not need to be counted again every time, only the integral of a plurality of corresponding points of the rectangle needs to be calculated, namely the rectangular characteristic value is calculated, and the calculation time does not change along with the change of the size of the rectangle;
scanning the image detection area once in a whole to obtain a detection factor coefficient;
multiplying the first-stage detection result by the detection factor coefficient to obtain a weak classifier result, and cascading weak classifiers to obtain a strong classifier detection result;
zooming an original image by half, and scanning an image to be detected by using a haar window with a fixed size as a basic window;
calculating the mean value and the variance in windows of different sizes, traversing the image to be detected, and considering the window as the image of the upper half of the human body when the set conditions are met;
calculating Haar-like characteristics in the candidate region, and transmitting the characteristics to a cascade adaboost classifier for further judgment;
if three windows exist and can simultaneously detect the upper half body image of the human body, the output result of the cascade classifier is determined to be a true value, the upper half body coordinate is output, and if the output result of the cascade classifier does not exist, the output result of the cascade classifier is determined to be a non-true value;
calculating distance (x, y) Pix (x, y)/1000 for all pixel values in the upper half body region depth map;
and performing consistency analysis on the measured distance data, and if the measured distance data is consistent, confirming that the target detected in the RGB image exists in the depth image, otherwise, confirming that the target does not exist in the depth image.
In another embodiment of the robot automatic tracking system based on RGBD face detection according to the present invention, the robot motion control unit is configured to perform a distance operation on a region corresponding to the depth image extraction, where the distance operation includes:
obtaining the distance faceDis (x, y) of each pixel point in the face area in the depth map as image (x, y)/1000;
summing and averaging the distances of all the pixel points to finally obtain the actual distance between the robot and the person;
and controlling the robot behavior according to the calculated actual concrete behaviors of the robot and the human.
Compared with the prior art, the invention has the following advantages:
the invention relates to a robot automatic tracking method and a system based on RGBD face detection, which comprises the steps of firstly collecting RGB images through an RGB camera of the RGBD camera, carrying out image detection on the upper half of a human body by using a cascade classifier, detecting the upper half of the human body by using an image algorithm processing platform, then carrying out face detection on the upper half of the image by using the cascade classifier again, determining the region of the depth information of the face in the image formed by the depth camera according to the output coordinate information after detecting the face, extracting the face depth information, carrying out distance operation to obtain the longitudinal forward and backward distance of the face, and controlling a robot to move along with the movement of the face according to the distance change information The human-like function is stronger.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, a brief description will be given below to the drawings used in the description of the embodiments or the prior art.
Fig. 1 is a schematic structural diagram of an embodiment of the robot automatic tracking system based on RGBD face detection of the present invention.
Fig. 2 is a flowchart of an embodiment of the robot automatic tracking method based on RGBD face detection of the present invention.
Fig. 3 is a flowchart of another embodiment of the robot automatic tracking method based on RGBD face detection of the present invention.
Fig. 4 is a flowchart of another embodiment of the robot automatic tracking method based on RGBD face detection of the present invention.
Fig. 5 is a flowchart of another embodiment of the robot automatic tracking method based on RGBD face detection of the present invention.
Fig. 6 is a flowchart of another embodiment of the robot automatic tracking method based on RGBD face detection of the present invention.
In the figure: the system comprises a 1RGBD camera, a 11RGB camera, a 12 depth camera, a 13 cascade classifier, a 2 image algorithm processing platform and a 3 robot motion control unit.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Fig. 1 is a schematic structural diagram of an embodiment of the robot automatic tracking system based on RGBD face detection of the present invention, as shown in fig. 1, the robot automatic tracking system based on RGBD face detection includes:
the robot comprises an RGBD camera 1, an image algorithm processing platform 2 and a robot motion control unit 3;
the RGBD camera 1 is used for collecting RGB images and depth images of a human body and identifying the upper half body and face images of the human body; the image algorithm processing platform 2 is used for receiving the human body image acquired by the RGBD camera 1, realizing human face recognition of the human body through algorithm processing, and calculating the distance between the robot and the human face; the robot motion control unit 3 obtains the longitudinal forward and backward distance of the human face through the image algorithm processing platform 2, and controls the robot to move along with the movement of the human face according to the distance change information.
The RGBD camera 1 comprises an RGB camera 11 and a depth camera 12;
the RGB camera 11 is used for shooting RGB images of a human body;
the depth camera 12 is used for shooting a depth image of a human body;
after the RGB camera 11 and the depth camera 12 are calibrated by the cameras, the image shot by the depth camera 12 coincides with the image shot by the RGB camera.
The RGBD camera 1 further includes a cascade classifier 13, the cascade classifier 13 is configured to detect an upper half body image of a human body in the image captured by the RGB camera 11, and when the upper half body is detected by the image, the cascade classifier 13 is used again to detect a human face in the upper half body region of the image.
The cascade classifier 13 includes a plurality of strong classifiers, which are connected together to operate, and the strong classifier includes a plurality of weak classifiers, which are weighted by the plurality of weak classifiers.
The image algorithm processing platform 2 adds a target depth information analysis method to perform image algorithm processing by adopting a haar algorithm and an adboost algorithm, and comprises the following steps:
firstly, carrying out gray level conversion on an RGB image to change a color image into a gray level image of 0 to 255 orders;
enhancing the brightness of the gray scale image by using a histogram equalization method;
selecting an image detection area, and carrying out integral processing on the selected area, wherein when the rectangular characteristic is calculated by adopting an integral method, the sum of pixel gray values in a rectangle does not need to be counted again every time, only the integral of a plurality of corresponding points of the rectangle needs to be calculated, namely the rectangular characteristic value is calculated, and the calculation time does not change along with the change of the size of the rectangle;
scanning the image detection area once in a whole to obtain a detection factor coefficient;
multiplying the first-stage detection result by the detection factor coefficient to obtain a weak classifier result, and cascading weak classifiers to obtain a strong classifier detection result;
zooming an original image by half, and scanning an image to be detected by using a haar window with a fixed size as a basic window;
calculating the mean value and the variance in windows of different sizes, traversing the image to be detected, and considering the window as the image of the upper half of the human body when the set conditions are met;
calculating Haar-like characteristics in the candidate region, and transmitting the characteristics to a cascade adaboost classifier for further judgment;
if three windows exist and can simultaneously detect the upper half body image of the human body, the output result of the cascade classifier 13 is determined to be a true value, the upper half body coordinate is output, and if the three windows do not exist, the output result of the cascade classifier 13 is determined to be a non-true value;
calculating distance (x, y) Pix (x, y)/1000 for all pixel values in the upper half body region depth map;
and performing consistency analysis on the measured distance data, and if the measured distance data is consistent, confirming that the target detected in the RGB image exists in the depth image, otherwise, confirming that the target does not exist in the depth image.
The robot motion control unit 3 is configured to perform distance calculation in a region corresponding to the depth image extraction, where the distance calculation includes:
obtaining the distance faceDis (x, y) of each pixel point in the face area in the depth map as image (x, y)/1000;
summing and averaging the distances of all the pixel points to finally obtain the actual distance between the robot and the person;
and controlling the robot behavior according to the calculated actual concrete behaviors of the robot and the human.
Fig. 2 is a flowchart of an embodiment of the robot automatic tracking method based on RGBD face detection of the present invention, and as shown in fig. 2, the robot automatic tracking method based on RGBD face detection includes:
acquiring human body sample image data, and detecting technical indexes of a classifier through the human body sample image; the human body sample image data is shot by an RGBD camera 1, and the technical indexes of the classifier comprise the detection success rate and the false detection rate of the upper half body and the success rate and the false detection rate of human face detection;
20, using a cascade classifier 13 to detect the upper body of a human body, using depth image data to perform numerical analysis, and confirming a detected human body image, wherein the cascade classifier 13 is formed by connecting a plurality of strong classifiers which are weighted by a plurality of weak classifiers together for operation;
30, using a cascade classifier 13 to detect the human face in the image for detecting the upper half of the human body by using a method of integrating image depth information until the human face is detected, and confirming the detected human face image;
and 40, after the human face area is detected, using a control module to extract a corresponding area from the depth image for distance calculation to obtain the accurate distance between the human and the robot.
The method for detecting the human body upper part of the body by using the classifier and carrying out numerical analysis by using the depth image data is a cascade classification method, wherein the cascade classification method is to connect a plurality of strong classifiers together for operation, and the strong classifier is formed by weighting a plurality of weak classifiers.
For example, the image detection of the upper half of the human body uses 10 weak classifiers which are cascaded together to form a cascaded strong classifier, and because the distinguishing accuracy of each strong classifier on the negative samples is very high, once the detected target bit negative sample is found, the following strong classifiers are not called continuously, so that the detection time is reduced; since many areas to be detected in one image are negative samples, the cascade classifier 1 abandons the complex detection of many negative samples in the early stage of the classifier, so the speed of the cascade classifier is very high; only the positive samples are sent to the next strong classifier for secondary inspection, so that the probability of false positive of the finally output positive samples is very low; the cascade structure classifier consists of a plurality of weak classifiers, and each stage is more complex than the previous stage; each classifier allows all positive examples to pass through, and filters most negative examples, so that the number of positive examples to be detected in each stage is less than that in the previous stage, a large number of non-detection targets are eliminated, and the detection speed is greatly improved.
Fig. 3 is a flowchart of another embodiment of the robot automatic tracking method based on RGBD face detection of the present invention, and as shown in fig. 3, the acquiring human body sample image data includes the following technical indicators of a human body sample image detection classifier:
preparing training data, and making a human body image training sample, wherein the human body image training sample comprises: a negative sample and a positive sample, the negative sample referring to an image not including an object; the positive sample is an image of an object to be detected and cannot contain any negative sample; the negative samples are more diversified, cannot be related to the positive samples, and are better to be taken from daily life;
12, using an image test classifier of the finished training sample, and analyzing technical indexes of the classifier according to a test result, wherein the technical indexes comprise: the detection success rate and the false detection rate of the upper half body image and the success rate and the false detection rate of the human face detection;
and 13, adjusting technical parameters of the classifier, wherein the parameters adjusted by the classifier comprise: the number of positive samples used during training, the number of negative samples used during training of each stage of classifier, the stage number of the classifier, the haar feature type and the minimum detection rate of each stage of the classifier.
The preparing of the training data and the making of the human body image training sample comprise:
using an RGBD camera 1 to obtain an image of a training sample, wherein the training sample is divided into a positive sample and a negative sample, and the negative sample refers to an image without an object; the positive sample is an image of an object to be detected and cannot contain any negative sample; the negative samples are more diversified, cannot be related to the positive samples, and are better to be taken from daily life;
calculating an integral image of the sample image, and constructing a characteristic model;
calculating a characteristic value of the characteristic model to obtain a characteristic set;
determining a domain value, and generating a corresponding weak classifier according to the feature set to obtain a weak classifier set;
training a strong classifier by using an adaboost algorithm;
adding part of the negative sample set again, and calculating an integral image of an integral sample image;
and finishing the acquisition of the human body sample by the cascade classifier.
Fig. 4 is a flowchart of another embodiment of the robot automatic tracking method based on RGBD face detection according to the present invention, and as shown in fig. 4, the detecting the upper body of the human body by using the classifier and performing numerical analysis by using the depth image data includes:
firstly, carrying out gray scale conversion on an RGB image to change a color image into a gray scale image of 0 to 255 orders;
202, enhancing the brightness of the gray-scale image by using a histogram equalization method;
203, selecting an image detection area, and performing integration processing on the selected area, wherein the integration method is adopted to calculate the rectangular characteristic without counting the sum of pixel gray values in the rectangle again every time, only the integration of a plurality of corresponding points of the rectangle is required to be calculated, namely the rectangular characteristic value is calculated, and the calculation time is not changed along with the change of the size of the rectangle;
204, scanning the image detection area once to obtain a detection factor coefficient;
205, multiplying the first-stage detection result by the detection factor coefficient to obtain a weak classifier result, and cascading the weak classifiers to obtain a strong classifier detection result; setting a 40 × 80 window, scanning the image once to obtain a detection factor coefficient, multiplying the first-stage detection result by the detection factor coefficient to obtain a weak classifier result, and cascading the weak classifiers to obtain a final detection result.
206, zooming the original image by half, and scanning the image to be detected by using a fixed-size haar window, wherein the fixed-size haar window is a basic window;
207, calculating the mean value and the variance in the windows with different sizes, traversing the image to be detected, and considering the window as the image of the upper half of the human body when the set conditions are met;
208, calculating Haar-like characteristics in the candidate region, and transmitting the characteristics to a cascade adaboost classifier for further judgment;
209, if three windows exist to simultaneously detect the upper half body image of the human body, determining that the output result of the cascade classifier is a true value, and outputting the coordinates of the upper half body, and if the three windows do not exist, determining that the output result of the cascade classifier is a non-true value;
210, calculating distance (x, y) Pix (x, y)/1000 for all pixel values in the upper body region depth map;
and 211, performing consistency analysis on the measured distance data, if the measured distance data is consistent, confirming that the target detected in the RGB image exists in the depth image, and otherwise, confirming that the target does not exist in the depth image.
The method is characterized in that target depth information analysis is added on the basis of a haar algorithm and an adboost algorithm. The Haar algorithm is essentially characterized by a statistical principle, inherent defects exist in classification by using a statistical method, namely fuzzy characteristics exist in the statistical principle, the defect of the statistical principle can be effectively made up by adding target depth information, the accuracy of target detection is greatly improved, and the false detection rate is effectively reduced.
Fig. 5 is a flowchart of another embodiment of the robot automatic tracking method based on RGBD face detection according to the present invention, and as shown in fig. 5, the performing face detection using the method of merging image depth information in an image in which the upper half of the human body is detected by using a classifier until a face is detected, and confirming the detected face image includes:
31, pre-loading face detection classifier data, and setting the size of a scanning window, wherein the size of the scanning window is not limited and is specifically determined according to the requirements in an actual project;
and 32, scanning the image according to the window with the set scale until the human face is detected.
Fig. 6 is a flowchart of another embodiment of the robot automatic tracking method based on RGBD face detection in the present invention, and as shown in fig. 6, the performing distance calculation on the region corresponding to the depth image extraction includes:
41, obtaining the distance faceDis (x, y) of each pixel point in the face area in the depth map as image (x, y)/1000;
and 42, summing and averaging the distances of all the pixel points to finally obtain the actual distance between the robot and the person.
The method and the system for automatically tracking the robot based on the RGBD face detection provided by the present invention are described in detail above, and a specific example is applied in the text to explain the principle and the implementation of the present invention, and the description of the above embodiment is only used to help understanding the method and the core idea of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.
Finally, it should be noted that: although the present invention has been described in detail with reference to the foregoing embodiments, it will be apparent to those skilled in the art that modifications may be made to the embodiments or portions thereof without departing from the spirit and scope of the invention.
Claims (8)
1. An automatic robot tracking method based on RGBD face detection is characterized by comprising the following steps:
acquiring human body sample image data, and detecting technical indexes of a classifier through the human body sample image;
using a cascade classifier to detect the upper half of the body of the human body, using depth image data to perform numerical analysis, and confirming a detected human body image;
using a cascade classifier to detect a human face in an image for detecting the upper half of the human body by using a method of integrating image depth information until the human face is detected, and confirming the detected human face image, wherein the cascade classifier is formed by connecting a plurality of strong classifiers which are weighted by a plurality of weak classifiers;
after the face area is detected, a control module is used for extracting a corresponding area from the depth image to perform distance operation to obtain an accurate distance between the robot and the person;
using the classifier to carry out human upper body detection to using the depth image data to carry out numerical analysis, confirming detecting human images includes:
firstly, carrying out gray level conversion on an RGB image to change a color image into a gray level image of 0 to 255 orders;
enhancing the brightness of the gray scale image by using a histogram equalization method;
selecting an image detection area, and carrying out integral processing on the selected area, wherein when the rectangular characteristic is calculated by adopting an integral method, the sum of pixel gray values in a rectangle does not need to be counted again every time, only the integral of a plurality of corresponding points of the rectangle needs to be calculated, namely the rectangular characteristic value is calculated, and the calculation time does not change along with the change of the size of the rectangle;
scanning the image detection area once in a whole to obtain a detection factor coefficient;
multiplying the first-stage detection result by the detection factor coefficient to obtain a weak classifier result, and cascading weak classifiers to obtain a strong classifier detection result;
zooming an original image by half, and scanning an image to be detected by using a haar window with a fixed size as a basic window;
calculating the mean value and the variance in windows of different sizes, traversing the image to be detected, and considering the window as the image of the upper half of the human body when the set conditions are met;
calculating Haar-like characteristics in the candidate region, and transmitting the characteristics to a cascade adaboost classifier for further judgment;
if three windows exist and can simultaneously detect the upper half body image of the human body, the output result of the cascade classifier is determined to be a true value, the upper half body coordinate is output, and if the output result of the cascade classifier does not exist, the output result of the cascade classifier is determined to be a non-true value;
calculating distance (x, y) Pix (x, y)/1000 for all pixel values in the upper half body region depth map;
and performing consistency analysis on the measured distance data, and if the measured distance data is consistent, confirming that the target detected in the RGB image exists in the depth image, otherwise, confirming that the target does not exist in the depth image.
2. The method of claim 1, wherein the obtaining of the human body sample image data and the detecting of the technical index of the classifier through the human body sample image comprises:
using an image test classifier of a finished training sample, and analyzing technical indexes of the classifier according to a test result, wherein the technical indexes comprise: the detection success rate of the upper half body image and the error preparation training data are used for making a human body image training sample, and the human body image training sample comprises: a negative sample and a positive sample, the negative sample referring to an image not including an object; the positive sample is an image of an object to be detected and cannot contain any negative sample;
the detection rate, the success rate of face detection and the false detection rate;
adjusting technical parameters of a classifier, wherein the parameters adjusted by the classifier comprise: the number of positive samples used during training, the number of negative samples used during training of each stage of classifier, the stage number of the classifier, the haar feature type and the minimum detection rate of each stage of the classifier.
3. The method according to claim 1, wherein the using the classifier to perform face detection using a method of merging image depth information into an image in which the upper body of the human body is detected until the face is detected, and the confirming the detected face image comprises:
pre-loading face detection classifier data, setting the size of a scanning window, wherein the size of the scanning window is not limited and is specifically determined according to the requirements in actual projects;
and scanning the image according to the window with the set scale until the human face is detected.
4. The method of claim 1, wherein the distance operation on the corresponding region extracted from the depth image comprises:
obtaining the distance faceDis (x, y) of each pixel point in the face area in the depth map as image (x, y)/1000;
and summing and averaging the distances of all the pixel points to finally obtain the actual distance between the robot and the person.
5. An automatic robot tracking system based on RGBD face detection is characterized by comprising: the system comprises an RGBD camera, an image algorithm processing platform and a robot motion control unit;
the RGBD camera is used for collecting RGB images and depth images of a human body and is used for identifying the upper half body and the face images of the human body; the image algorithm processing platform is used for receiving a human body image acquired by the RGBD camera, realizing human face recognition of a human body through algorithm processing, and calculating the distance between the robot and the human face; the robot motion control unit obtains the longitudinal forward and backward distance of the human face through the image algorithm processing platform and controls the robot to move along with the movement of the human face according to the distance change information;
the RGBD camera comprises an RGB camera and a depth camera, the RGB camera is used for shooting RGB images of a human body, the depth camera is used for shooting depth images of the human body, and after the RGB camera and the depth camera are calibrated by a camera, images shot by the depth camera are overlapped with images shot by the RGB camera;
the RGBD camera is an intel depth processing camera;
the image algorithm processing platform adopts a haar algorithm and an adboost algorithm to add a target depth information analysis method for image algorithm processing, and comprises the following steps:
firstly, carrying out gray level conversion on an RGB image to change a color image into a gray level image of 0 to 255 orders;
enhancing the brightness of the gray scale image by using a histogram equalization method;
selecting an image detection area, and carrying out integral processing on the selected area, wherein when the rectangular characteristic is calculated by adopting an integral method, the sum of pixel gray values in a rectangle does not need to be counted again every time, only the integral of a plurality of corresponding points of the rectangle needs to be calculated, namely the rectangular characteristic value is calculated, and the calculation time does not change along with the change of the size of the rectangle;
scanning the image detection area once in a whole to obtain a detection factor coefficient;
multiplying the first-stage detection result by the detection factor coefficient to obtain a weak classifier result, and cascading weak classifiers to obtain a strong classifier detection result;
zooming an original image by half, and scanning an image to be detected by using a haar window with a fixed size as a basic window;
calculating the mean value and the variance in windows of different sizes, traversing the image to be detected, and considering the window as the image of the upper half of the human body when the set conditions are met;
calculating Haar-like characteristics in the candidate region, and transmitting the characteristics to a cascade adaboost classifier for further judgment;
if three windows exist and can simultaneously detect the upper half body image of the human body, the output result of the cascade classifier is determined to be a true value, the upper half body coordinate is output, and if the output result of the cascade classifier does not exist, the output result of the cascade classifier is determined to be a non-true value;
calculating distance (x, y) Pix (x, y)/1000 for all pixel values in the upper half body region depth map;
and performing consistency analysis on the measured distance data, and if the measured distance data is consistent, confirming that the target detected in the RGB image exists in the depth image, otherwise, confirming that the target does not exist in the depth image.
6. The system of claim 5, wherein the RGBD camera further comprises a cascade classifier, and the cascade classifier is configured to detect an upper body image of a human body in the image captured by the RGB camera, and when the upper body image is detected, perform face detection on the upper body region of the image again using the cascade classifier.
7. The system of claim 6, wherein the cascaded classifier comprises a plurality of strong classifiers operatively connected together, the strong classifiers comprising a plurality of weak classifiers, weighted by the plurality of weak classifiers.
8. The system of claim 5, wherein the robot motion control unit is configured to perform a distance operation on the corresponding region extracted from the depth image, and the distance operation comprises:
obtaining the distance faceDis (x, y) of each pixel point in the face area in the depth map as image (x, y)/1000;
summing and averaging the distances of all the pixel points to finally obtain the actual distance between the robot and the person;
and controlling the robot behavior according to the calculated actual concrete behaviors of the robot and the human.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710028570.5A CN106886216B (en) | 2017-01-16 | 2017-01-16 | Robot automatic tracking method and system based on RGBD face detection |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710028570.5A CN106886216B (en) | 2017-01-16 | 2017-01-16 | Robot automatic tracking method and system based on RGBD face detection |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106886216A CN106886216A (en) | 2017-06-23 |
| CN106886216B true CN106886216B (en) | 2020-08-14 |
Family
ID=59176339
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710028570.5A Active CN106886216B (en) | 2017-01-16 | 2017-01-16 | Robot automatic tracking method and system based on RGBD face detection |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106886216B (en) |
Families Citing this family (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108647555A (en) * | 2017-11-21 | 2018-10-12 | 江苏鸿山鸿锦物联技术有限公司 | Whether there is or not the detection methods of personnel in a kind of car based on video image |
| CN109895140A (en) * | 2017-12-10 | 2019-06-18 | 湘潭宏远电子科技有限公司 | A kind of robotically-driven trigger device |
| CN107977636B (en) * | 2017-12-11 | 2021-11-30 | 北京小米移动软件有限公司 | Face detection method and device, terminal and storage medium |
| CN108197507A (en) * | 2017-12-30 | 2018-06-22 | 刘智 | A kind of privacy real-time protection method and system |
| CN108537843A (en) * | 2018-03-12 | 2018-09-14 | 北京华凯汇信息科技有限公司 | The method and device of depth of field distance is obtained according to depth image |
| CN108734083B (en) * | 2018-03-21 | 2023-04-25 | 北京猎户星空科技有限公司 | Control method, device, equipment and storage medium of intelligent equipment |
| CN108527366B (en) * | 2018-03-22 | 2020-12-04 | 北京理工华汇智能科技有限公司 | Robot following method and device based on depth of field distance |
| CN108647662A (en) * | 2018-05-17 | 2018-10-12 | 四川斐讯信息技术有限公司 | A kind of method and system of automatic detection face |
| CN109299595A (en) * | 2018-09-08 | 2019-02-01 | 太若科技(北京)有限公司 | Method, apparatus and AR equipment based on hand skin texture information unlock AR equipment |
| CN111259684B (en) * | 2018-11-30 | 2023-07-28 | Tcl科技集团股份有限公司 | Method and device for determining distance from camera to face |
| CN110276271A (en) * | 2019-05-30 | 2019-09-24 | 福建工程学院 | Merge the non-contact heart rate estimation technique of IPPG and depth information anti-noise jamming |
| CN112115913B (en) * | 2020-09-28 | 2023-08-25 | 杭州海康威视数字技术股份有限公司 | Image processing method, device and equipment and storage medium |
| CN117519488B (en) * | 2024-01-05 | 2024-03-29 | 四川中电启明星信息技术有限公司 | Dialogue method and dialogue system of dialogue robot |
Family Cites Families (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101477625B (en) * | 2009-01-07 | 2013-04-24 | 北京中星微电子有限公司 | Upper half of human body detection method and system |
| CN102214291B (en) * | 2010-04-12 | 2013-01-16 | 云南清眸科技有限公司 | Method for quickly and accurately detecting and tracking human face based on video sequence |
| US20160191995A1 (en) * | 2011-09-30 | 2016-06-30 | Affectiva, Inc. | Image analysis for attendance query evaluation |
| US8861800B2 (en) * | 2010-07-19 | 2014-10-14 | Carnegie Mellon University | Rapid 3D face reconstruction from a 2D image and methods using such rapid 3D face reconstruction |
| CN103093212B (en) * | 2013-01-28 | 2015-11-18 | 北京信息科技大学 | The method and apparatus of facial image is intercepted based on Face detection and tracking |
| CN103995747B (en) * | 2014-05-12 | 2017-05-31 | 上海大学 | Distributed pedestrian detecting system and method based on mobile robot platform |
| CN105187721B (en) * | 2015-08-31 | 2018-09-21 | 广州市幸福网络技术有限公司 | A kind of the license camera and method of rapid extraction portrait feature |
| CN105182983A (en) * | 2015-10-22 | 2015-12-23 | 深圳创想未来机器人有限公司 | Face real-time tracking method and face real-time tracking system based on mobile robot |
| CN106250850B (en) * | 2016-07-29 | 2020-02-21 | 深圳市优必选科技有限公司 | Face detection tracking method and device, and robot head rotation control method and system |
-
2017
- 2017-01-16 CN CN201710028570.5A patent/CN106886216B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN106886216A (en) | 2017-06-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106886216B (en) | Robot automatic tracking method and system based on RGBD face detection | |
| KR100612858B1 (en) | Method and apparatus for tracking human using robot | |
| CN104616002A (en) | Facial recognition equipment used for judging age groups | |
| JP2016015045A (en) | Image recognition device, image recognition method, and program | |
| CN110084830B (en) | Video moving object detection and tracking method | |
| CN110287907B (en) | Object detection method and device | |
| CN114693661A (en) | Rapid sorting method based on deep learning | |
| CN110942450A (en) | Multi-production-line real-time defect detection method based on deep learning | |
| CN111091057A (en) | Information processing method and device and computer readable storage medium | |
| CN114721403B (en) | Automatic driving control method and device based on OpenCV and storage medium | |
| CN113822352A (en) | Infrared dim target detection method based on multi-feature fusion | |
| CN107045630B (en) | RGBD-based pedestrian detection and identity recognition method and system | |
| CN101320477B (en) | Human body tracing method and equipment thereof | |
| CN106529441A (en) | Fuzzy boundary fragmentation-based depth motion map human body action recognition method | |
| CN108563997B (en) | Method and device for establishing face detection model and face recognition | |
| CN111178405A (en) | Similar object identification method fusing multiple neural networks | |
| CN113021355B (en) | Agricultural robot operation method for predicting sheltered crop picking point | |
| CN112949451B (en) | Cross-modal target tracking method and system through modal perception feature learning | |
| US20220405527A1 (en) | Target Detection Methods, Apparatuses, Electronic Devices and Computer-Readable Storage Media | |
| CN110334703B (en) | Ship detection and identification method in day and night image | |
| WO2022263904A1 (en) | Target detection methods, apparatuses, electronic devices and computer-readable storage media | |
| CN115908886A (en) | Image classification method, image processing apparatus, and storage device | |
| CN112926676B (en) | False target identification method and device and computer equipment | |
| CN117523428B (en) | Ground target detection method and device based on aircraft platform | |
| CN114155475B (en) | Method, device and medium for identifying end-to-end personnel actions under view angle of unmanned aerial vehicle |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |