CN111291632A - A pedestrian state detection method, device and device - Google Patents

Abstract

The invention discloses a pedestrian state detection method, a pedestrian state detection device and pedestrian state detection equipment, which comprise the following steps: acquiring an existing pedestrian image; establishing a pedestrian state detection model, and processing the existing pedestrian image by using the pedestrian state detection model; inputting the processed pedestrian image into a pedestrian state detection model for training to obtain a trained pedestrian state detection model; and inputting the real-time pedestrian image into the trained pedestrian state detection model, and outputting a judgment result of the pedestrian state by the pedestrian state detection model. According to the pedestrian state detection method, the pedestrian state detection model is established, and the acquired human body detection image, the acquired face attribute detection image and the acquired pedestrian attribute detection image are input into the pedestrian state detection model, so that the pedestrian state detection model can comprehensively judge the state information of the pedestrian, the defect that the comprehensive condition of the pedestrian cannot be judged in the conventional pedestrian state detection method is overcome, and the pedestrian state detection method has important significance in practical application.

Description

A pedestrian state detection method, device and device

technical field

The present invention relates to the technical field of pedestrian detection, in particular to a pedestrian state detection method, device and device.

Background technique

With the development of science and technology, people can identify various information in the image through the image, and at present, image recognition is also widely used in the field of security. Pass, etc. However, in the prior art, only the number of pedestrians can be determined by obtaining the status information of passing pedestrians through images, and it is impossible to comprehensively detect whether the pedestrians have luggage, children and the age status of the pedestrians. Pedestrians guide, cannot and remind pedestrians to put their luggage in the luggage belt or provide priority access for the elderly.

To sum up, the methods for detecting the state of pedestrians in the prior art have the disadvantage that the comprehensive situation of pedestrians cannot be judged.

SUMMARY OF THE INVENTION

The present invention provides a pedestrian state detection method, device and equipment, which solves the problem that the pedestrian state detection method in the prior art cannot judge the comprehensive situation of the pedestrian.

A pedestrian state detection method provided by the present invention includes the following steps:

Step S1: obtain an existing pedestrian image;

Step S2: establishing a pedestrian state detection model, and using the pedestrian state detection model to process the existing pedestrian images;

Step S3: input the processed pedestrian image into the pedestrian state detection model for training, and obtain a trained pedestrian state detection model;

Step S4: Input the real-time pedestrian image into the trained pedestrian state detection model, and the pedestrian state detection model outputs the judgment result of the pedestrian state.

Preferably, step S2 specifically includes the following steps:

Step S201: Input the pedestrian image into the pedestrian state detection model, and the pedestrian state detection model detects the human body in the pedestrian image to obtain the human body detection image;

Step S202: the pedestrian state detection model normalizes the human body detection image to obtain the pedestrian attribute detection image;

Step S203: The pedestrian state detection model performs face detection on the human body detection image and aligns it to obtain a face attribute detection image.

Preferably, attribute annotation is performed on the human body detection image, the face attribute detection image and the pedestrian attribute detection image respectively.

Preferably, the attribute of the human body detection image is the number of pedestrians; the attribute category of the face attribute detection image includes: gender, age, ethnicity, expression, face posture and face wear; the attribute category of the pedestrian attribute detection image includes: backpack, handbag , satchels, luggage, carrying children, wheelchairs and strollers.

Preferably, in step S3, the following steps are specifically included:

Step S301: Divide the human body detection image, the face attribute detection image and the pedestrian attribute detection image into a training set and a test set in a certain proportion;

Step S302: Input the training set of the human body detection image, the training set of the face attribute detection image and the training set of the pedestrian attribute detection image into the pedestrian state detection model in order to optimize the weight parameters of the pedestrian state detection model;

Step S303: Input the test set into the trained pedestrian state detection model to obtain optimal weight parameters of the pedestrian state detection model.

Preferably, in the face attribute detection training set of the face attribute detection image, the attribute of each face attribute detection image is classified as a class, and the pedestrian attribute training set of the pedestrian attribute detection image takes the attribute of each pedestrian attribute detection image as one type.

Preferably, in step S301, the human body detection image is used as a positive sample of the human body detection training set, and the image without pedestrians in the existing image is input into the pedestrian state detection model as a negative sample of the human body detection training set. The detection training set contains all categories of the attributes of the face attribute detection images and the number of each category is equal, and the pedestrian attribute training set contains all the categories of the pedestrian attribute detection images and the number of each category is equal.

Preferably, the pedestrian state detection model includes a human body detection model, a face attribute detection model and a pedestrian attribute detection model, and the face attribute detection model and the pedestrian attribute detection model are composed of a convolutional neural network, and the convolutional neural network At least one layer of deconvolution layer and at least one layer of inception layer are arranged between the first layer of convolution layer and the last layer of convolution layer, and the inception adopts multiple convolution kernels of different sizes when performing convolution; the The human detection model adopts the faceboxes algorithm.

A pedestrian state detection device, comprising a pedestrian image acquisition module, an image preprocessing module and a pedestrian state detection model module;

The pedestrian image acquisition module is used to acquire images of pedestrians;

The image preprocessing module is used for preprocessing images of pedestrians;

The pedestrian state detection model module is used to establish a pedestrian state detection model and output the result of pedestrian state detection.

A pedestrian state detection device, the device includes a processor and a memory;

the memory is used to store program code and transmit the program code to the processor;

The processor is configured to execute the pedestrian state detection method according to any one of claims 1-8 according to the instructions in the program code.

As can be seen from the above technical solutions, the present invention has the following advantages:

In the embodiment of the present invention, by establishing a pedestrian state detection model, the obtained human body detection image, face attribute detection image and pedestrian attribute detection image are input into the pedestrian state detection model, so that the pedestrian state detection model can synthesize the state information of pedestrians It solves the problem that the pedestrian state detection method in the prior art cannot judge the comprehensive situation of the pedestrian, and is beneficial to the security personnel to guide the pedestrians passing through the security inspection channel, which is of great significance in practical applications.

Description of drawings

In order to explain the embodiments of the present invention or the technical solutions in the prior art more clearly, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without any creative effort.

FIG. 1 is a flowchart of a method, apparatus, and device for detecting a pedestrian state according to an embodiment of the present invention.

FIG. 2 is a device frame diagram of a pedestrian state detection method, device, and device provided by an embodiment of the present invention.

FIG. 3 is a device frame diagram of a pedestrian state detection method, device, and device according to an embodiment of the present invention.

Detailed ways

The embodiments of the present invention provide a pedestrian state detection method, device and device, which are used to solve the technical problem that the pedestrian state detection method in the prior art cannot judge the comprehensive situation of the pedestrian.

In order to make the purpose, features and advantages of the present invention more obvious and understandable, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the following The described embodiments are only some, but not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

Example 1

Please refer to FIG. 1. FIG. 1 is a flowchart of a method, apparatus, and device for detecting a pedestrian state according to an embodiment of the present invention.

As shown in Figure 1, a pedestrian state detection method provided by the present invention includes the following steps:

Step S1: obtain an existing pedestrian image;

The existing pedestrian images can be selected from the historical images captured by the cameras on the security check ramp. The pedestrian images in the security check ramp cover most types of pedestrian state images, which are beneficial to the subsequent pedestrian detection model for the learning and detection of pedestrian images.

Step S2: establishing a pedestrian state detection model, and using the pedestrian state detection model to process the existing pedestrian images;

Step S3: input the processed pedestrian image into the pedestrian state detection model for training, optimize the pedestrian state detection model, and obtain a trained pedestrian state detection model;

Input the processed existing pedestrian images into the pedestrian state detection model for training. The pedestrian state detection model detects and recognizes images of different types and images of the same type with different attribute categories and optimizes itself, so as to obtain trained pedestrians. State detection model.

Step S4: Input the real-time pedestrian image into the trained pedestrian state detection model, and the pedestrian state detection model outputs the judgment result of the state of the pedestrian.

The real-time pedestrian image is input into the trained pedestrian state detection model, and the pedestrian state detection model can directly detect and judge the real-time pedestrian image, so as to directly judge the comprehensive state of the pedestrian.

As a preferred embodiment, step S2 specifically includes the following steps:

Step S201: Input the pedestrian image into the pedestrian state detection model, and the pedestrian state detection model detects the human body in the pedestrian image to obtain the human body detection image;

Step S202: the pedestrian state detection model normalizes the human body detection image to obtain the pedestrian attribute detection image;

Step S203: The pedestrian state detection model performs face detection on the human body detection image and aligns it to obtain a face attribute detection image.

Dividing the existing pedestrian images into human body detection images can make the pedestrian state detection model only focus on the human body in the image; dividing the existing pedestrian images into face attribute detection images can make the pedestrian state detection model only focus on the people in the image. face; dividing the existing pedestrian images into pedestrian attribute detection images can make the pedestrian state detection model only focus on the attributes of pedestrians; by dividing the existing pedestrian images into different categories, the pedestrian state detection model can only focus on the image. The specific point is beneficial to improve the learning efficiency of the pedestrian state detection model.

The pedestrian state detection model detects the pedestrian image to obtain the sub-image formed by the human body matrix, and normalizes the sub-image to obtain a 128*256 human body matrix sub-image. Use the face detection algorithm (ie mtcnn or faceboxes) to get the face image, and align the face image to get the face attribute detection image, the size is 96*112.

As a preferred embodiment, attribute labeling is performed on the human body detection image, the face attribute detection image, and the pedestrian attribute detection image, respectively.

By annotating the attributes in the human body detection image, the face attribute detection image and the pedestrian attribute detection image, the characteristics that need to be recognized by the pedestrian state detection model in different types of images can be highlighted, which is beneficial to reduce the interference items in the image and improve the pedestrian state detection. The learning rate of the model.

As a preferred embodiment, the attribute of the human body detection image is the number of pedestrians; the attribute categories of the face attribute detection image include: gender, age, expression, ethnicity, face posture and face wear; the attribute category of the pedestrian attribute detection image includes : Backpacks, bags, satchels, suitcases, lead children, wheelchairs and strollers.

Gender, age, expression, ethnicity, face pose and face wear are the most intuitive attributes of a face. By learning this attribute, the pedestrian state detection model can infer the comprehensive information of the pedestrian's appearance. The attributes of the pedestrian attribute detection image can enable the pedestrian state detection model to infer the comprehensive shape information of the pedestrian, so as to determine whether the pedestrian is carrying a backpack, carrying luggage, or whether the pedestrian is a baby or a disabled person in a wheelchair, which is convenient for the staff to guide the pedestrian. .

As a preferred embodiment, in step S3, the following steps are specifically included:

Step S301: Divide the image in the human body detection image, the image in the face attribute detection image, and the image in the pedestrian attribute detection image into a training set and a test set in a certain proportion; divide the image into a training set for the pedestrian state detection model. In the training, the test set is used to test the detection effect of the pedestrian state detection model.

Step S302: Input the training set of the human body detection image, the training set of the face attribute detection image and the training set of the pedestrian attribute detection image into the pedestrian state detection model in order to optimize the weight parameters of the pedestrian state detection model;

The training set in the human body detection image, the training set in the face attribute detection image, and the training set in the pedestrian attribute detection image are input into the pedestrian state detection model in sequence, and the pedestrian state detection model adjusts its own state detection model according to the three training sets. weight parameter.

Step S303: Input the test set into the pedestrian state detection model to obtain optimal weight parameters of the pedestrian state detection model.

Input the test set in the human body detection image, the test set in the face attribute detection image and the test set in the pedestrian attribute detection image into the trained pedestrian state detection model, and the pedestrian state detection model further adjusts its own parameters , to find its own optimal weight parameters.

As a preferred embodiment, in step S301, the cropped human body detection image is used as a positive sample of the human body detection training set, and the image without pedestrians in the existing image is used as a negative sample of the human body detection training set and input to the pedestrian state detection model , the number of negative samples is three times that of positive samples, the face attribute detection training set contains all categories of the attributes of the face attribute detection image and the number of each category is equal, and the pedestrian attribute training set contains the attributes of the pedestrian attribute detection image. of all categories with an equal number of each category.

As a preferred embodiment, the face attribute detection training set of the face attribute detection image takes the attribute of each face attribute detection image as a class, and the pedestrian attribute training set of the pedestrian attribute detection image uses the attribute of each pedestrian as a class.

As a preferred embodiment, the pedestrian state detection model is composed of a convolutional neural network, and the pedestrian state detection model includes a human body detection model, a face attribute detection model, and a pedestrian attribute detection model. The face attribute detection model and The pedestrian attribute detection model is composed of a convolutional neural network, and at least one deconvolution layer and at least one inception layer are set between the first convolutional layer and the last convolutional layer of the convolutional neural network. When inception performs convolution, multiple convolution kernels of different sizes are used; the human detection model uses the faceboxes algorithm.

At least one deconvolution layer and at least one inception layer are set between the first convolutional layer and the last convolutional layer of the convolutional neural network in the face attribute detection model and the pedestrian attribute detection model. The layer can make the convolutional neural network better adapt to the change of the input image size, while increasing the diversity of feature images, multi-scale fusion of feature images, and reduce the amount of computation, the deconvolution layer can restore the size of the feature image, right The feature image is supplemented and expanded, which can not only learn high-level features, but also reduce the model parameters of the network, so that the pedestrian state detection model can adapt to changes in the size of the input image, and the convolution layer can learn more abundant features. .

After inputting the real-time pedestrian image into the pedestrian state detection model, the human body detection model first detects the real-time pedestrian image to detect the number of pedestrians in the image. After that, the face attribute detection model and the convolutional neural network in the pedestrian attribute detection model The shared feature sub-network (such as convolutional layer, deconvolutional layer and inception layer) in the network will extract the attribute features of real-time pedestrian images. For example, the fully connected layer will extract the attributes of real-time pedestrian images, and the output layer will be treated by the features of shared attributes. The real-time pedestrian image's face attribute detection image attributes and the pedestrian attribute detection image attributes are calculated to obtain the recognition result of the comprehensive state of each pedestrian in the real-time pedestrian image.

Example 2

Embodiment 2 of the present application provides a face attribute recognition device. For the convenience of description, only parts related to the present application are shown. As shown in FIG. 2 , a pedestrian state detection device includes a pedestrian image acquisition module 401, an image a preprocessing module 402 and a pedestrian state detection model module 403;

The pedestrian image acquisition module 401 is used to acquire images of pedestrians;

The image preprocessing module 402 is used to preprocess the image of the pedestrian;

The pedestrian state detection model module 403 is used to establish a pedestrian state detection model and output the result of pedestrian state detection.

It should be noted that the information exchange, execution process and other contents between the above-mentioned devices/units are based on the same concept as the method embodiments of the present application. For specific functions and technical effects, please refer to the method embodiments section. It is not repeated here.

Example 3

As shown in FIG. 3 , a pedestrian state detection device 50 includes a processor 500 and a memory 501;

The memory 501 is used to store program codes 502 and transmit the program codes 502 to the processor;

The processor 500 is configured to execute the steps in the above-mentioned embodiment of the pedestrian state detection method according to the instructions in the program code 502 , for example, steps S1 to S5 shown in FIG. 1 . Alternatively, when the processor 500 executes the computer program 502, the functions of the modules/units in the foregoing device embodiments are implemented, for example, the functions of the modules 401 to 403 shown in FIG. 2 .

Exemplarily, the computer program 502 may be divided into one or more modules/units, and the one or more modules/units are stored in the memory 501 and executed by the processor 500 to complete the this application. The one or more modules/units may be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program 502 in the terminal device 50 . For example, the computer program 502 can be segmented to include a pedestrian image acquisition module, an image preprocessing module, and a pedestrian state detection model module;

The pedestrian image acquisition module is used to acquire images of pedestrians;

The image preprocessing module is used for preprocessing images of pedestrians;

The pedestrian state detection model module is used to establish a pedestrian state detection model and output the result of pedestrian state detection.

The terminal device 50 may be a computing device such as a desktop computer, a notebook, a palmtop computer, and a cloud server. The terminal device may include, but is not limited to, a processor 500 and a memory 501 . Those skilled in the art can understand that FIG. 3 is only an example of the terminal device 50, and does not constitute a limitation on the terminal device 50. It may include more or less components than the one shown, or combine some components, or different components For example, the terminal device may further include an input and output device, a network access device, a bus, and the like.

The so-called processor 500 may be a central processing unit (Central Processing Unit, CPU), and may also be other general-purpose processors, digital signal processors (Digital Signal Processors, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), Off-the-shelf programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, and the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The memory 501 may be an internal storage unit of the terminal device 50 , such as a hard disk or a memory of the terminal device 50 . The memory 501 may also be an external storage device of the terminal device 50, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) equipped on the terminal device 50. card, flash card (Flash Card) and so on. Further, the memory 501 may also include both an internal storage unit of the terminal device 50 and an external storage device. The memory 501 is used to store the computer program and other programs and data required by the terminal device. The memory 501 can also be used to temporarily store data that has been output or will be output.

Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system, device and unit described above may refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

In the several embodiments provided in this application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the apparatus embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, and may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware, or may be implemented in the form of software functional units.

The integrated unit, if implemented in the form of a software functional unit and sold or used as an independent product, may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes: U disk, removable hard disk, Read-Only Memory (ROM, Read-Only Memory), Random Access Memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program codes.

As mentioned above, the above embodiments are only used to illustrate the technical solutions of the present invention, but not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand: The technical solutions described in the embodiments are modified, or some technical features thereof are equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions depart from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims (10)

1. A pedestrian state detection method is characterized by comprising the following steps:

step S1: acquiring an existing pedestrian image;

step S2: establishing a pedestrian state detection model, and processing the existing pedestrian image by using the pedestrian state detection model;

step S3: inputting the processed pedestrian image into a pedestrian state detection model for training to obtain a trained pedestrian state detection model;

step S4: and inputting the real-time pedestrian image into the trained pedestrian state detection model, and outputting a judgment result of the pedestrian state by the pedestrian state detection model.

2. The pedestrian state detection method according to claim 1, wherein the step S2 specifically includes the steps of:

step S201: inputting the pedestrian image into a pedestrian state detection model, and detecting a human body in the pedestrian image by the pedestrian state detection model to obtain a human body detection image;

step S202: the pedestrian state detection model normalizes the human detection image to obtain a pedestrian attribute detection image;

step S203: the pedestrian state detection model carries out face detection on the human body detection image and aligns the human body detection image to obtain a human face attribute detection image.

3. The pedestrian state detection method according to claim 2, wherein attribute labeling is performed on the human detection image, the human face attribute detection image, and the pedestrian attribute detection image, respectively.

4. The pedestrian state detection method according to claim 3, wherein the attribute of the human body detection image is a pedestrian number; the attribute classes of the face attribute detection image include: gender, age, ethnicity, expression, facial pose, and facial wear; the attribute categories of the pedestrian attribute detection image include: backpacks, handbags, satchels, luggage, carrying children, wheelchairs, and strollers.

5. The pedestrian state detection method according to claim 4, wherein in step S3, the method specifically comprises the following steps:

step S301: dividing a human body detection image, a human face attribute detection image and a pedestrian attribute detection image into a training set and a test set according to a certain proportion;

step S302: inputting a training set of human body detection images, a training set of face attribute detection images and a training set of pedestrian attribute detection images into a pedestrian state detection model in sequence to optimize weight parameters of the pedestrian state detection model;

step S303: and inputting the test set into the trained pedestrian state detection model to obtain the optimal weight parameters of the pedestrian state detection model.

6. The method according to claim 5, wherein the attribute of each of the face attribute detection images is classified into one class in the face attribute detection training set of the face attribute detection images, and the attribute of each of the pedestrian attribute detection images is classified into one class in the pedestrian attribute training set of the pedestrian attribute detection images.

7. The method according to claim 6, wherein in step S301, the human body detection images are used as positive samples of a human body detection training set, images without pedestrians in the existing images are used as negative samples of the human body detection training set and input into the pedestrian state detection model, all classes containing the attributes of the human face attribute detection images in the human face attribute detection training set are equal in number, and all classes containing the attributes of the pedestrian attribute detection images in the pedestrian attribute training set are equal in number.

8. The pedestrian state detection method according to claim 1, wherein the pedestrian state detection model comprises a human body detection model, a face attribute detection model and a pedestrian attribute detection model, the face attribute detection model and the pedestrian attribute detection model are composed of a convolutional neural network, at least one deconvolution layer and at least one interception layer are arranged between a first layer of convolutional layer and a last layer of convolutional layer of the convolutional neural network, and a plurality of convolution kernels of different sizes are adopted when the interception is performed; the human body detection model adopts a faceboxes algorithm.

9. A pedestrian state detection device is characterized by comprising a pedestrian image acquisition module, an image preprocessing module and a pedestrian state detection model module;

the pedestrian image acquisition module is used for acquiring an image of a pedestrian;

the image preprocessing module is used for preprocessing the image of the pedestrian;

the pedestrian state detection model module is used for establishing a pedestrian state detection model and outputting a pedestrian state detection result.

10. A pedestrian state detection apparatus, characterized in that the apparatus comprises a processor and a memory;

the memory is used for storing program codes and transmitting the program codes to the processor;

the processor is configured to execute a pedestrian condition detection method according to any one of claims 1 to 8 in accordance with instructions in the program code.

Images