CN112132865A - Personnel identification method and system - Google Patents

Abstract

The present invention discloses a person following method, which is used for a robot to follow a single person, which includes the following steps: S1. The robot confirms the number of people in an image captured by the robot and identifies the personnel; S2. the robot obtains real-time feature information and path information of the personnel; S3. the robot predicts the next position of the personnel; and S4. the robot predicts according to the S3 step The position information of the person determines the detection range of the robot in the next frame and follows the person. By predicting the next position of the person, the robot determines the range that needs to be detected in the next frame in advance, which improves the speed of the robot to detect and recognize pedestrians so that the robot can follow the person.

Description

Personnel identification method and system

technical field

The invention relates to the field of robot identification methods and systems, in particular to a person identification method and system.

Background technique

With the advancement of technology, existing robots can replace human beings to achieve more and more functions, and it is an essential function for robots to follow a specific person. Existing methods for robots to follow people are divided into generative methods and discriminative methods. Among them, the generative method uses a feature model to describe the appearance features of the target, and then minimizes the reconstruction error between the tracking target and the candidate target to confirm the target; the generative method focuses on the feature extraction of the target itself, ignoring the background information of the target. When the appearance changes drastically or is occluded, it is prone to target drift or target loss. Since the discriminative following method can significantly distinguish the difference between the background and the person, and has excellent performance and strong anti-interference ability, the discriminative following method has occupied the mainstream position in recent years, and the discrimination represented by correlation filtering and deep learning The method has achieved a satisfactory follow-up effect.

However, the existing discriminative following methods require robots to perform deep learning to accurately identify the difference between people and backgrounds. However, the application of deep learning in the field of people following robots is not smooth. Because of the particularity of target following tasks, robots Only the image data of the initial frame can be used, and there is a lack of a large amount of data for the neural network to learn. Only by migrating the data to the target tracking through the convolutional neural network can the robot follow the pedestrians. Therefore, in the existing pedestrian tracking methods, The reasoning speed of the robot is slow, and it is difficult to re-follow the person after losing the information of the person.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a pedestrian identification method, which can solve the defects of the existing pedestrian identification methods, improve the reasoning speed of the robot, and can also follow the person again after the robot loses the personnel information. Specifically, a person following method, which is used for a robot to follow a single person, includes the following steps: S1. The robot confirms the number of people in the image captured by the robot and identifies the person the robot needs to follow S2. described robot obtains the real-time feature information and path information of described personnel; S3. described robot predicts the position of the next step of described personnel; The position information of the person determines the detection range of the robot in the next frame and follows the person.

According to an embodiment of the present invention, the step S1 includes: S101. The robot confirms the number of persons in the image; and S102. If the number of persons in the robot image is one, the robot extracts the number of persons in the image. The characteristics of the person; if the number of the people in the robot image is at least two, the robot performs feature matching on the people in the image to determine the person the robot needs to follow.

According to an embodiment of the present invention, in the step S102, when the robot performs feature matching on the person in the image, the robot performs matching based on the depth field information and the feature information of the person.

According to an embodiment of the present invention, the step S2 includes: S201. The robot reduces the image to a rectangular image including all the information of the person; S202. The robot analyzes the content in the rectangular image frame to Acquire characteristic information of the person; and S203. The robot acquires the path information of the person according to the moving direction and distance of the person.

According to an embodiment of the present invention, the step S4 includes: S401. The robot predicts the position of the person according to the Kalman filter algorithm; and S402. The robot predicts the position of the person according to the prediction result of the position of the person. Images captured by the robot are cropped.

The present invention also discloses a personnel following system, which can realize the above-mentioned personnel following method. for capturing images; a first identification module, the first identification module identifies the number of persons in the images captured by the capturing module and confirms the identity of the persons followed by the robot; a second identification module, the first identification module The second recognition module recognizes the feature information and path information of the person according to the image captured by the shooting module; a prediction module, which predicts the position of the person; and an adjustment module, which determines the robot The detection range of the next frame.

According to an embodiment of the present invention, the first identification module includes: a quantity identification unit, and the quantity identification unit is configured to identify the number of the persons in the image captured by the photographing module and classify the image according to the The number of the persons is classified into a first image including one person and a second image including at least two persons; a primary identification unit, the primary identification unit is used to identify the person in the first image and a secondary identification unit, the secondary identification unit is configured to perform feature matching on the person in the second image to determine the person the robot needs to follow.

According to an embodiment of the present invention, the secondary identification unit performs matching in combination with depth field information and feature information of the person.

According to an embodiment of the present invention, the second recognition module includes: a first cropping unit, the first cropping unit is to crop the image captured by the photographing module to a rectangular shape including all the information of the person ; a second recognition unit, which analyzes the content in the rectangular figure to obtain real-time feature information of the person; and a path acquisition unit, which is based on an image captured by the shooting unit Obtain real-time path information of the person.

According to an embodiment of the present invention, the prediction module includes: a prediction unit, the prediction unit predicts the position of the person according to the recognition result of the first recognition unit or the second recognition unit; and a second cropping unit, The second cropping unit crops the next frame of image photographed by the photographing module according to the prediction result of the predicting unit.

By adopting the above-mentioned technical scheme, the present invention mainly has the following technical effects:

1. By predicting the next position of the person, the robot determines the range to be detected in the next frame in advance, which improves the speed of the robot to detect pedestrians;

2. The robot matches the feature information of the person in combination with the depth information, the training speed of the robot is fast and the detection accuracy of the person is high;

3. The robot extracts the identity information of the person, extracts and records the real-time feature information of the person, and can also use the real-time feature information to retrieve the person after the robot loses the image of the person;

4. The position of the person is predicted by Kalman filter, which improves the detection speed of the robot. At the same time, reducing the detection position can ensure that the person enters the deep learning model with the original size, which improves the accuracy of detection.

Description of drawings

1 is a schematic diagram of a person following method according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of a person following system according to an embodiment of the present invention.

In the figure: 1, shooting module; 2, first identification module; 21, quantity identification unit; 22, primary identification unit; 23, secondary identification unit; 3, second identification module; 31, first cutting unit; 32, 33, a path acquisition unit; 4, a prediction module; 41, a prediction unit; 42, a second cropping unit; 5, an adjustment module.

Detailed ways

The specific embodiments of the present invention will be described below with reference to the accompanying drawings.

The invention discloses a person tracking method, which is used for a robot to follow a single person, and specifically includes the following steps:

S1. The robot confirms the number of people in the image captured by the robot and identifies the person the robot needs to follow;

S2. the robot obtains real-time feature information and path information of the person;

S3. the robot predicts the next position of the person; and

S4. The robot determines the detection range of the robot in the next frame according to the position information of the person predicted in the step S3 and follows the person.

Wherein, step S1 specifically includes S101. The robot confirms the number of people in the image; and S102. If the number of people in the robot image is one, the robot extracts the characteristics of the person in the image; if When the number of the persons in the image captured by the robot is at least two, the robot performs feature matching on the persons in the image to determine the persons that the robot needs to follow. The robot first determines the number of people in the initial recognition image it shoots, and adopts different recognition methods according to the number of people. In this embodiment, in order to improve the recognition speed of the robot, when there is only one person in the image, the robot adopts the personnel re-identification method. The identification algorithm is used to extract the characteristic information of the person. When there are at least two members in the image, the robot uses the matching algorithm to match the predetermined target, so as to determine which person the robot needs to follow from among multiple people. Moreover, in order to be able to follow the person again after the robot loses the image of the person, the person feature information extracted by the robot in this embodiment includes the identity information of the person, and the robot can quickly re-follow the person through the identity information. In addition, in order to quickly distinguish the person that the robot needs to follow among multiple people, when the robot matches the features of the people in the field of view, the robot matches the people by combining the depth field information and the feature information of the person.

Specifically, the above-mentioned S2 step specifically includes S201. The robot reduces the image to a rectangular image including all the information of the person; S202. The robot analyzes the content in the rectangular field of view to obtain the person's information. feature information; and S203. The robot acquires the path information of the person according to the moving direction and distance of the person. By reducing the image size, the size of the image that the robot needs to recognize during the recognition process is smaller, which reduces the content that the robot needs to recognize, thereby improving the speed of the robot's recognition and detection. By reducing the image to a rectangular figure that includes all the characteristic information of the person, the robot only needs to recognize the reduced figure when it recognizes the figure, and does not need to detect all the information in the image. In addition, the robot obtains the path information of the person, which can help the robot to predict the follow-up path and position of the person.

In addition, step S4 in this embodiment specifically includes S401. The robot predicts the position of the person according to the Kalman filter algorithm; and S402. The robot photographs the robot according to the predicted result of the position of the person. image is cropped. By adopting the above technical solution, the robot only needs to identify the entire content of the captured image at the beginning stage in the process of following the person to identify the specific feature information and path information of the person the robot needs to follow. The robot predicts the location of the person by In addition to identifying all images of the initial frame, the robot only needs to recognize the cropped image in the image of the subsequent frame, thereby improving the speed of the robot to recognize the characteristic information of persons.

In addition, in order to realize the above-mentioned person identification method, this embodiment also discloses a pedestrian following system, which includes a shooting module 1, a first identification module 2, a second identification module 3, a prediction module 4 and an adjustment module arranged on the robot 5. The photographing module 1 is arranged on the robot to photograph the image of the person, the first recognition module 2 identifies the number of persons in the image photographed by the photographing module 1 and confirms the identity of the person followed by the robot, and the second recognition module 3 is based on the retreat image captured by the photographing module 1. Ang identifies the characteristic information and path information of the person, the prediction module 4 predicts the position of the person, and the adjustment module 5 determines the detection range of the next frame of the robot.

Meanwhile, in order to detect images of different numbers of persons respectively, the first recognition module 2 in this embodiment includes a number recognition unit 21 , a primary recognition unit 22 and a number recognition unit 21 for recognizing the number of persons in the images captured by the photographing module 1 Secondary identification unit 23 . The number recognition unit 21, after recognizing the number of persons in the image, classifies the images into a first image including only one person and a second image including at least two persons, and the primary recognition unit 22 is used to recognize the first image in the image The secondary identification unit 23 is configured to perform feature matching on the person in the second image to determine the person the robot needs to follow. In this embodiment, the primary identification unit 22 uses a person re-identification algorithm to extract personnel feature information, and the secondary identification unit 23 uses a matching algorithm to match with a predetermined target to determine the person the robot needs to follow. In addition, in order to improve the matching accuracy of the secondary identification unit 23, the secondary identification unit 23 in this embodiment performs person matching in combination with the depth field information and the characteristic information of the person.

The second identification module 3 in this embodiment includes a first cropping unit 31 , a second identification unit 32 and a path obtaining unit 33 , wherein the first cropping unit 31 crops the image captured by the photographing module 1 , and crops the image including the person. For a rectangular figure of all information, the second recognition unit 32 analyzes the content in the rectangular image to obtain real-time feature information of the person, and the path obtaining unit 33 obtains the real-time path information of the person according to the image captured by the shooting unit. The path obtaining unit 33 in this embodiment may calculate and obtain the real-time path of the person according to the historically captured images of the shooting module 1, or may obtain the path information of the person in other ways.

The prediction module 4 in this embodiment includes a prediction unit 41 and a second cropping unit 42. The prediction unit 41 predicts the position of the person according to the recognition result of the first recognition unit or the second recognition unit 32, and the second cropping unit 42 predicts the position of the person according to The prediction result of the prediction unit 41 crops the next frame of image captured by the capturing module 1 . The prediction unit 41 in this embodiment uses the Kalman filter algorithm to predict the subsequent position and path of the person.

The above embodiments are only used to illustrate the present invention, but not to limit the present invention. Those of ordinary skill in the relevant technical field can also make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, all Equivalent technical solutions also belong to the scope of the present invention, and the patent protection scope of the present invention should be defined by the claims.

Claims (10)

1. a personnel following method, it is used for the robot to follow a single person, it is characterized in that, comprise the following steps: S1. The robot confirms the number of people in the image captured by the robot and identifies the people the robot needs to follow; S2. the robot obtains real-time feature information and path information of the person; S3. the robot predicts the next position of the person; and S4. The robot determines the detection range of the robot in the next frame according to the position information of the person predicted in the step S3 and follows the person. 2. following method according to claim 1 is characterized in that: The S1 step includes: S101. The robot confirms the number of people in the image; and S102. If the number of people in the robot image is one, the robot extracts the characteristics of the person in the image; if the number of people in the robot image is at least two, the robot The person in the feature matching is performed to determine the person the robot needs to follow. 3. following method according to claim 2 is characterized in that: In the step S102, when the robot performs feature matching on the person in the image, the robot performs matching based on the depth field information and the feature information of the person. 4. following method according to claim 1 is characterized in that: The S2 step includes: S201. The robot reduces the image to a rectangular image including all the information of the person; S202. The robot analyzes the content in the rectangular image frame to obtain characteristic information of the person; and S203. The robot acquires the path information of the person according to the moving direction and distance of the person. 5. following method according to claim 1 is characterized in that: The S4 step includes: S401. the robot predicts the position of the person according to the Kalman filter algorithm; and S402. The robot crops the image captured by the robot according to the predicted result of the position of the person. 6. A personnel following system is characterized in that, comprising: a photographing module, the photographing module is arranged on the robot, and the photographing module is used for photographing images; a first identification module, the first identification module identifies the number of the persons in the images captured by the photographing module and confirms the identity of the persons followed by the robot; a second identification module, the second identification module identifies the feature information and path information of the person according to the image captured by the shooting module; a prediction module that predicts the location of the person; and An adjustment module, the adjustment module determines the detection range of the robot in the next frame. 7. The following system according to claim 6, characterized in that: The first identification module includes: Quantity recognition unit, the quantity recognition unit is used for recognizing the number of the persons in the images captured by the photographing module and classifying the images according to the number of persons in the images, into the number of persons including one person. an image and a second image including at least two persons; a primary identification unit for identifying the characteristics of the person in the first image; and A secondary identification unit, the secondary identification unit is configured to perform feature matching on the person in the second image to determine the person the robot needs to follow. 8. The following system according to claim 7, characterized in that: The secondary identification unit performs matching in combination with the depth field information and the characteristic information of the person. 9. The following system according to claim 6, characterized in that: The second identification module includes: a first cropping unit, the first cropping unit crops the image captured by the photographing module to a rectangular figure including all the information of the person; a second identification unit that analyzes the content in the rectangular graphic to obtain real-time feature information of the person; and A path acquisition unit, which acquires real-time path information of the person according to the image captured by the shooting unit. 10. The following system according to claim 6, characterized in that: The prediction module includes: a prediction unit, the prediction unit predicts the position of the person according to the recognition result of the first recognition unit or the second recognition unit; and a second cropping unit, the second cropping unit performs cropping on the next frame of image photographed by the photographing module according to the prediction result of the predicting unit.

Images