CN111626165A - Pedestrian recognition method and device, electronic equipment and storage medium - Google Patents

Pedestrian recognition method and device, electronic equipment and storage medium Download PDF

Info

Publication number
CN111626165A
CN111626165A CN202010424166.1A CN202010424166A CN111626165A CN 111626165 A CN111626165 A CN 111626165A CN 202010424166 A CN202010424166 A CN 202010424166A CN 111626165 A CN111626165 A CN 111626165A
Authority
CN
China
Prior art keywords
pedestrian
area
suspected
image
target
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202010424166.1A
Other languages
Chinese (zh)
Other versions
CN111626165B (en
Inventor
杨帆
祖春胜
张澄宇
张飞
曾伟
丁钊
李涛
孙宝
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Anhui Jianghuai Automobile Group Corp
Original Assignee
Anhui Jianghuai Automobile Group Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Anhui Jianghuai Automobile Group Corp filed Critical Anhui Jianghuai Automobile Group Corp
Priority to CN202010424166.1A priority Critical patent/CN111626165B/en
Publication of CN111626165A publication Critical patent/CN111626165A/en
Application granted granted Critical
Publication of CN111626165B publication Critical patent/CN111626165B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/103Static body considered as a whole, e.g. static pedestrian or occupant recognition
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/22Matching criteria, e.g. proximity measures
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/20Image preprocessing
    • G06V10/26Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G06V10/267Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/75Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
    • G06V10/751Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Data Mining & Analysis (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Databases & Information Systems (AREA)
  • Evolutionary Biology (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Human Computer Interaction (AREA)
  • Traffic Control Systems (AREA)
  • Image Analysis (AREA)

Abstract

The present invention relates to the field of automatic driving technologies, and in particular, to a pedestrian recognition method, apparatus, electronic device, and storage medium. The method comprises the following steps: determining suspected pedestrian areas in an obstacle image, and acquiring confidence values of the suspected pedestrian areas; taking a suspected pedestrian area with the confidence value in a first confidence value interval as a target area, and determining a pedestrian target in the target area; taking a suspected pedestrian area with the confidence value in a second confidence value interval as an undetermined area, and respectively obtaining a person comprehensive value of the undetermined area according to a preset sheltering object model; when the figure comprehensive value is larger than or equal to a preset comprehensive value, determining a figure target in an undetermined area corresponding to the figure comprehensive value as a comprehensive pedestrian target; and generating a pedestrian recognition image according to the pedestrian target and the comprehensive pedestrian target. The accuracy of pedestrian discernment among the autopilot process has been promoted, the pedestrian target of various forms of accurate discernment.

Description

Pedestrian recognition method and device, electronic equipment and storage medium
Technical Field
The present invention relates to the field of automatic driving technologies, and in particular, to a pedestrian recognition method, apparatus, electronic device, and storage medium.
Background
The safety problem of automatic driving is more and more concerned by people, and the main problem in the problem is the identification and protection of pedestrians by an automatic driving system, and a radar sensor can detect obstacles, but the effect of identifying the pedestrians and the states of the pedestrians is poor, so that the forward-looking pedestrian identification completed by using a visual algorithm has important significance in the development of automatic driving, and the quality of the safety performance of automatic driving is determined to a certain extent.
The human body overall contour method detects a human body as an integral body, and when the target is incomplete or shielded, the error is large; meanwhile, in automatic driving, forward looking pedestrians are dynamic, and it is difficult for the template to include all pedestrian states. In the existing human body part identification method, the legs and the heads are mostly used as important parts, and if a pedestrian carries an umbrella or wears a skirt, the legs or the heads are shielded, so that the problem of missing detection of the pedestrian is easily caused, and potential safety hazards are generated.
The above is only for the purpose of assisting understanding of the technical aspects of the present invention, and does not represent an admission that the above is prior art.
Disclosure of Invention
The invention mainly aims to provide a pedestrian recognition method, a pedestrian recognition device, electronic equipment and a storage medium, and aims to solve the technical problem of pedestrian recognition.
In order to achieve the above object, the present invention provides a pedestrian recognition method, including:
determining suspected pedestrian areas in an obstacle image, and acquiring confidence values of the suspected pedestrian areas;
taking a suspected pedestrian area with the confidence value in a first confidence value interval as a target area, and determining a pedestrian target in the target area;
taking a suspected pedestrian area with the confidence value in a second confidence value interval as an undetermined area, and respectively obtaining a person comprehensive value of the undetermined area according to a preset sheltering object model;
when the figure comprehensive value is larger than or equal to a preset comprehensive value, determining a figure target in an undetermined area corresponding to the figure comprehensive value as a comprehensive pedestrian target;
and generating a pedestrian recognition image according to the pedestrian target and the comprehensive pedestrian target.
Preferably, the step of determining the suspected pedestrian areas in the obstacle image and acquiring the confidence values of the suspected pedestrian areas specifically includes:
determining an undetermined pedestrian area in an obstacle image, and performing area amplification on the undetermined pedestrian area to obtain a suspected pedestrian area;
and carrying out image matching on the suspected pedestrian areas according to a preset human body model so as to obtain confidence values of all the suspected pedestrian areas.
Preferably, the step of determining an undetermined pedestrian area in an obstacle image and performing area amplification on the undetermined pedestrian area to obtain a suspected pedestrian area specifically includes:
determining an undetermined pedestrian area in the obstacle image according to a pedestrian area formula;
performing regional amplification on the area of the pedestrian to be detected to obtain a suspected pedestrian area;
wherein the pedestrian area formula is:
Figure BDA0002494771320000021
wherein, ROI is the region of undetermined pedestrian, LwIs the aspect ratio of the region, SpIs the ratio of the area of the regions, S is the area of the regions, SminPresetting a minimum area, S, for a regionmaxPresetting a maximum area, L, for a regionminPresetting a minimum aspect ratio, L, for a regionmaxPresetting a maximum aspect ratio, S, for a regionpmaxFor presetting the area ratio of the maximum segmentation region to the image display page, SpminThe area ratio of the minimum segmentation region to the image presentation page is preset.
Preferably, the step of performing image matching on the suspected pedestrian areas according to a preset human body model to obtain a confidence value of each of the suspected pedestrian areas specifically includes:
determining a human body position area of a suspected pedestrian target in the suspected pedestrian area;
and carrying out image matching on the human body position area through a preset human body model so as to obtain a confidence value of each suspected pedestrian area.
Preferably, the step of taking the suspected pedestrian area with the confidence value in the second confidence value interval as an undetermined area and respectively obtaining the person comprehensive value of the undetermined area according to a preset blocking object model specifically includes:
taking a suspected pedestrian area with the confidence value in a second confidence value interval as an undetermined area;
respectively acquiring the similarity of each shelter target in the area to be determined according to a preset shelter model;
adding the confidence value and the similarity according to a comprehensive formula to obtain a person comprehensive value of the region to be determined;
wherein, the comprehensive formula is as follows:
Figure BDA0002494771320000031
wherein, B is a human comprehensive value, a is a confidence value, l is a weight corresponding to the confidence value, BiSimilarity corresponding to the object of the obstruction,/iAnd the weight corresponding to the object of the shelter.
Preferably, the step of respectively obtaining the similarity of each of the shelter targets in the area to be fixed according to a preset shelter model specifically includes:
respectively acquiring a shelter position area corresponding to each shelter target in the area to be determined according to a preset shelter model;
and carrying out image matching on the position area of the shielding object through the preset shielding object model so as to obtain the corresponding similarity of each shielding object target.
Further, to achieve the above object, the present invention also provides a pedestrian recognition apparatus, comprising:
the confidence value acquisition module is used for determining suspected pedestrian areas in the obstacle image and acquiring the confidence value of each suspected pedestrian area;
the determining module is used for taking a suspected pedestrian area with the confidence value in the first confidence value interval as a target area and determining a pedestrian target in the target area;
the comprehensive value acquisition module is used for taking a suspected pedestrian area with the confidence value in the second confidence value interval as an undetermined area and respectively acquiring a person comprehensive value of the undetermined area according to a preset shelter model; the pedestrian comprehensive value judging module is also used for judging the human target in the undetermined area corresponding to the human comprehensive value as a comprehensive pedestrian target when the human comprehensive value is larger than or equal to a preset comprehensive value;
and the generation module is used for generating a pedestrian recognition image according to the pedestrian target and the comprehensive pedestrian target.
In addition, to achieve the above object, the present invention also provides an electronic device, including: a memory, a processor and a pedestrian identification program stored on the memory and executable on the processor, the pedestrian identification program being configured to implement the steps of the pedestrian identification method as described above.
Furthermore, to achieve the above object, the present invention also proposes a storage medium having stored thereon a pedestrian recognition program which, when executed by a processor, implements the steps of the pedestrian recognition method as described above.
According to the method, suspected pedestrian areas in an obstacle image are determined, and confidence values of the suspected pedestrian areas are obtained; taking a suspected pedestrian area with the confidence value in a first confidence value interval as a target area, and determining a pedestrian target in the target area; taking a suspected pedestrian area with the confidence value in a second confidence value interval as an undetermined area, and respectively obtaining a person comprehensive value of the undetermined area according to a preset sheltering object model; when the figure comprehensive value is larger than or equal to a preset comprehensive value, determining a figure target in an undetermined area corresponding to the figure comprehensive value as a comprehensive pedestrian target; and generating a pedestrian recognition image according to the pedestrian target and the comprehensive pedestrian target. The obstacle area is preliminarily recognized by matching the head and shoulder shape and the human body shape structure, a preliminarily recognized pedestrian target and a suspected pedestrian target are obtained, then interference objects which are easy to shield the human body, such as umbrellas and riding tool-shaped objects, are subjected to feature matching with the suspected pedestrian target, pedestrians are screened out from the suspected pedestrian target, non-pedestrians are eliminated, and the feature combination method for the obstacle area is capable of overcoming the shielding problem and enriching the pedestrian feature information. The problem of shielding is overcome, the target identification precision of forward-looking pedestrians is improved, and the safety of automatic driving is improved. And a more accurate dynamic obstacle image is obtained, a more stable foundation is provided for the identification of a pedestrian target, and the problems of incomplete detection target of a frame difference method and omission of the background difference method are solved by combining the advantages of the frame difference method and the background difference method.
Drawings
Fig. 1 is a schematic structural diagram of an electronic device in a hardware operating environment according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating a pedestrian recognition method according to a first embodiment of the present invention;
FIG. 3 is a flowchart illustrating a pedestrian recognition method according to a second embodiment of the present invention;
fig. 4 is a block diagram showing the structure of the pedestrian recognition apparatus according to the first embodiment of the present invention.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
Referring to fig. 1, fig. 1 is a schematic structural diagram of an electronic device in a hardware operating environment according to an embodiment of the present invention.
As shown in fig. 1, the electronic device may include: a processor 1001, such as a Central Processing Unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Wherein a communication bus 1002 is used to enable connective communication between these components. The user interface 1003 may include a Display screen (Display), an input unit such as a Keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a WIreless interface (e.g., a WIreless-FIdelity (WI-FI) interface). The Memory 1005 may be a Random Access Memory (RAM) Memory, or may be a Non-Volatile Memory (NVM), such as a disk Memory. The memory 1005 may alternatively be a storage device separate from the processor 1001.
Those skilled in the art will appreciate that the configuration shown in fig. 1 does not constitute a limitation of the electronic device and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components.
As shown in fig. 1, a memory 1005, which is a storage medium, may include therein an operating system, a network communication module, a user interface module, and a pedestrian recognition program.
In the electronic apparatus shown in fig. 1, the network interface 1004 is mainly used for data communication with a network server; the user interface 1003 is mainly used for data interaction with a user; the processor 1001 and the memory 1005 of the electronic device of the present invention may be provided in the electronic device, and the electronic device calls the pedestrian recognition program stored in the memory 1005 through the processor 1001 and executes the pedestrian recognition method provided by the embodiment of the present invention.
An embodiment of the present invention provides a pedestrian identification method, and referring to fig. 2, fig. 2 is a flowchart illustrating a first embodiment of a pedestrian identification method according to the present invention. In this embodiment, the pedestrian identification method includes the following steps:
step S10: and determining suspected pedestrian areas in the obstacle image, and acquiring confidence values of all the suspected pedestrian areas.
The invention acquires the image of the obstacle through the front-view camera of the automatic driving automobile, performs preliminary pedestrian recognition on the area of the obstacle through matching the head and shoulder shape and the human body shape structure, then performs matching of the part shape of the obstacle with the umbrella-shaped object, the riding tool-shaped object, the leg-shaped object and the similar human face object, and calculates the confidence value according to the output weight of the matching result. The umbrella and the riding tool which are used for shielding objects and the head, the shoulder and the legs of the shielded parts become secondary characteristics of pedestrians, so that the shielding problem is overcome, and the characteristic information of the pedestrians is enriched.
Step S10 specifically includes: determining an undetermined pedestrian area in an obstacle image, and performing area amplification on the undetermined pedestrian area to obtain a suspected pedestrian area; and carrying out image matching on the suspected pedestrian areas according to a preset human body model so as to obtain confidence values of all the suspected pedestrian areas.
It should be understood that the obstacle image mainly includes vehicle obstacles besides pedestrians, and it is necessary to select the suspected pedestrian area in the obstacle image, the human body shape structure is different from other object shapes, and the human body height-to-body width ratio L is selectedwPedestrian region area S and area ratio S of barrier division region area to rectangular window areapAnd as an evaluation standard, obtaining a suspected pedestrian area through a pedestrian area formula.
The pedestrian zone formula is:
Figure BDA0002494771320000061
wherein, ROI is the region of undetermined pedestrian, LwIs the aspect ratio of the region,SpIs the ratio of the area of the regions, S is the area of the regions, SminPresetting a minimum area, S, for a regionmaxPresetting a maximum area, L, for a regionminPresetting a minimum aspect ratio, L, for a regionmaxPresetting a maximum aspect ratio, S, for a regionpmaxFor presetting the area ratio of the maximum segmentation region to the image display page, SpminThe area ratio of the minimum segmentation region to the image presentation page is preset.
And obtaining the suspected pedestrian obstacle area according to the judgment standard. And finally, the suspected area is expanded outwards by preset times appropriately, so that the pedestrian target in the suspected area is contained as much as possible.
Step S20: and taking a suspected pedestrian area with the confidence value in the first confidence value interval as a target area, and determining a pedestrian target in the target area.
The step of performing image matching on the suspected pedestrian areas according to a preset human body model to obtain confidence values of the suspected pedestrian areas specifically includes: determining a human body position area of a suspected pedestrian target in the suspected pedestrian area; and carrying out image matching on the human body position area through a preset human body model so as to obtain a confidence value of each suspected pedestrian area.
It should be understood that the horizontally projected histogram of the binary image of the suspected pedestrian object reflects the width information of the suspected pedestrian object, and the vertically projected direct force reflects the height information of the suspected pedestrian object. And calculating a horizontal projection histogram of the suspected pedestrian target binary image, wherein the distance from a starting point to a first minimum value point is the width W of the head, and the overall height of the head and the shoulder is obtained by a formula H of 0.3W/1.3.
Furthermore, a matching area is limited according to the position of the human body part, the suspected pedestrian target is located in the area, the pedestrian is necessarily located in the center of the image, the head is necessarily located in the middle-upper position of the area, the omega-shaped head and shoulder shape is matched with the head and shoulder part which is difficult to completely shield, the confidence value of the target is output, and the initial identification is completed.
Step S30: and taking the suspected pedestrian area with the confidence value in the second confidence value interval as an undetermined area, and respectively obtaining the person comprehensive value of the undetermined area according to a preset sheltering object model.
Step S30 specifically includes: taking a suspected pedestrian area with the confidence value in a second confidence value interval as an undetermined area; respectively acquiring the similarity of each shelter target in the area to be determined according to a preset shelter model; and adding the confidence value and the similarity according to a comprehensive formula to obtain a person comprehensive value of the region to be determined.
The step of respectively obtaining the similarity of each shelter target in the area to be determined according to a preset shelter model specifically comprises the following steps: respectively acquiring a shelter position area corresponding to each shelter target in the area to be determined according to a preset shelter model; and carrying out image matching on the position area of the shielding object through the preset shielding object model so as to obtain the corresponding similarity of each shielding object target.
Further, according to the preliminarily recognized confidence value, the first confidence value interval corresponds to a pedestrian target, the second confidence value interval corresponds to a suspected pedestrian target, and targets which do not belong to the preset confidence value interval are non-pedestrian targets and are excluded. For a suspected pedestrian target, more part information judgment is required.
In a specific implementation, a blocking object (a part template such as an umbrella, a riding tool, a leg, a skirt and a human face, which are common pedestrian objects) is matched with a suspected pedestrian object, so that the information content of the suspected pedestrian object is increased, a matching area is limited by the matched part position, the part corresponding to the blocking object generally has a fixed position (such as a cap corresponding to the head of a pedestrian, an upper half of the pedestrian corresponding to the umbrella, a lower half of the skirt corresponding to the lower half of the pedestrian and the like), and the obtained similarity is multiplied by the corresponding weight and then added with the confidence value of the initial recognition according to a comprehensive formula.
In the specific implementation, the embodiment is further explained, for example: the front obstacle is recognized as a face avatar having a head and shoulders, and recognized as a pseudo-pedestrian target that is an advertiser avatar on a bus in front of the autonomous vehicle, but there is no object in the figure to which the lower body (legs) can be matched, in combination with the actual situation. For example: the suspected pedestrian target is successfully matched with the head and the shoulders, but the lower half of the body lacks legs, the suspected pedestrian target is found to be a person riding a motorcycle through matching with the vehicle, and the suspected pedestrian target is judged to be a comprehensive pedestrian target.
Wherein, the comprehensive formula is as follows:
Figure BDA0002494771320000081
b is the comprehensive value of human beings, a is the confidence value, l is the weight corresponding to the confidence value, BiSimilarity corresponding to the object of the obstruction,/iAnd the weight corresponding to the object of the shelter. The shelter can be a human body shape structure, a head and shoulder model, an umbrella, a riding tool, a leg, a human face and the like, and i is the number or the type of the shelter.
Step S40: and when the figure comprehensive value is larger than or equal to a preset comprehensive value, determining the figure target in the undetermined area corresponding to the figure comprehensive value as a comprehensive pedestrian target.
Step S50: and generating a pedestrian recognition image according to the pedestrian target and the comprehensive pedestrian target.
It should be understood that, since the legs of the pedestrian are generally in contact with the ground and perpendicular to the ground, the leg parts appear in the lower half area of the image, and the detection is to scan the lower half area of the image, detect a plurality of leg targets, and obtain leg information through non-maximum suppression. And detecting the human face of the cut head-shoulder region according to the Haar characteristics to obtain confidence coefficient because the head-shoulder model is obtained. The invention mainly aims at detecting a side-looking riding tool-shaped object, and the riding tool-shaped object is characterized in that the riding tool-shaped object is positioned below a pedestrian area and is provided with two wheels which are in contact with the ground.
The method comprises the steps of preliminarily identifying an obstacle area by matching head and shoulder shapes and human body shape structures, obtaining a preliminarily identified pedestrian target and a suspected pedestrian target, carrying out feature matching on interference objects which are easy to shield a human body, such as umbrellas and riding tool shapes, and the suspected pedestrian target, screening pedestrians from the suspected pedestrian target to eliminate non-pedestrians, and enriching pedestrian feature information. The problem of shielding is overcome, the target identification precision of forward-looking pedestrians is improved, and the safety of automatic driving is improved.
Referring to fig. 3, fig. 3 is a flowchart illustrating a pedestrian recognition method according to a second embodiment of the present invention. Based on the first embodiment described above, the pedestrian recognition method of the present embodiment further includes, before the step S10:
step S11: and acquiring an image to be recognized, and performing background difference processing on the image to be recognized to acquire a preliminary recognition image.
Step S11 specifically includes: and performing interframe difference processing on the image to be identified to obtain a background image. And carrying out differential operation on the background image to obtain a moving target image. And carrying out difference operation on the moving target image and the current image to be recognized so as to obtain a preliminary recognition image.
It should be noted that background difference processing is performed by using a Surendra algorithm, difference processing is performed on the image to be identified of the current frame and the adjacent image to be identified of the previous frame, and the pixel value of the image to be identified after difference is compared with a preset threshold value, so as to obtain a binary image. Representing a motion area with a numerical value of 1 in the binary image without updating; and the non-motion area with the numerical value of 0 is updated according to the updating coefficient, and a stable background image can be obtained through iterative operation of a plurality of frames of images.
It is easy to understand that the background image and the image to be recognized of each frame are subjected to differential operation to obtain a moving target image.
It is easy to understand that the moving target image and the image to be identified of the current frame are differentiated to obtain a preliminary identification image.
Step S12: and carrying out three-frame difference processing on the image to be identified to obtain an edge difference image.
The method specifically comprises the following steps: performing edge detection on a previous frame image of the image to be identified, a next frame image of the image to be identified and the image to be identified so as to obtain corresponding edge images; and carrying out difference processing on the edge image to obtain a first difference image and a second difference image. And taking the first difference image and the second difference image as the edge difference image.
It should be understood that the previous frame image of the image to be recognized, the next frame image of the image to be recognized, and the image to be recognized are continuous three frames of images, and the three frames of images are subjected to edge detection to obtain three frames of corresponding edge images.
It should be understood that, the three frame edge images are subjected to adjacent frame edge image difference operation, and two difference images, namely a first difference image and a second difference image, are obtained, namely the edge difference images.
Step S13: and obtaining an obstacle image according to the preliminary identification image and the edge difference image.
The method specifically comprises the following steps: and carrying out logical OR operation according to the preliminary identification image and the first difference image to obtain a medium-term identification image. And performing logical AND operation according to the medium-term identification image and the second difference image to obtain a dynamic obstacle image. And carrying out dynamic obstacle detection according to the dynamic obstacle image.
It is easy to understand that the first differential map and the preliminary recognition map are logically or-ed, the operation result and the second differential map are logically and-ed, and the dynamic obstacle result is detected.
According to the embodiment of the invention, a more accurate dynamic obstacle image is obtained through the method, a more stable foundation is provided for the identification of a pedestrian target, and the incomplete detection target problem of the frame difference method and the omission problem of the background difference method are solved by combining the advantages of the frame difference method and the background difference method.
Referring to fig. 4, fig. 4 is a block diagram of a first embodiment of the pedestrian recognition apparatus according to the present invention.
As shown in fig. 4, the routine person identifying apparatus of the present invention includes:
and the confidence value acquisition module is used for determining suspected pedestrian areas in the obstacle image and acquiring the confidence value of each suspected pedestrian area.
The invention acquires the image of the obstacle through the front-view camera of the automatic driving automobile, performs preliminary pedestrian recognition on the area of the obstacle through matching the head and shoulder shape and the human body shape structure, then performs matching of the part shape of the obstacle with the umbrella-shaped object, the riding tool-shaped object, the leg-shaped object and the similar human face object, and calculates the confidence value according to the output weight of the matching result. The umbrella and the riding tool which are used for shielding objects and the head, the shoulder and the legs of the shielded parts become secondary characteristics of pedestrians, so that the shielding problem is overcome, and the characteristic information of the pedestrians is enriched.
It should be noted that the following are specifically used: determining an undetermined pedestrian area in an obstacle image, and performing area amplification on the undetermined pedestrian area to obtain a suspected pedestrian area; and carrying out image matching on the suspected pedestrian areas according to a preset human body model so as to obtain confidence values of all the suspected pedestrian areas.
It should be understood that the obstacle image mainly includes vehicle obstacles besides pedestrians, and it is necessary to select the suspected pedestrian area in the obstacle image, the human body shape structure is different from other object shapes, and the human body height-to-body width ratio L is selectedwPedestrian region area S and area ratio S of barrier division region area to rectangular window areapAnd as an evaluation standard, obtaining a suspected pedestrian area through a pedestrian area formula.
The pedestrian zone formula is:
Figure BDA0002494771320000101
wherein, ROI is the region of undetermined pedestrian, LwIs the aspect ratio of the region, SpIs the ratio of the area of the regions, S is the area of the regions, SminPresetting a minimum area, S, for a regionmaxPresetting a maximum area, L, for a regionminPresetting a minimum aspect ratio, L, for a regionmaxPresetting a maximum aspect ratio, S, for a regionpmaxFor presetting the area ratio of the maximum segmentation region to the image display page, SpminFor presetting minimum segmentation area and image displayArea ratio of the page.
And obtaining the suspected pedestrian obstacle area according to the judgment standard. And finally, the suspected area is expanded outwards by preset times appropriately, so that the pedestrian target in the suspected area is contained as much as possible.
And the determining module 20 is configured to use the suspected pedestrian area with the confidence value in the first confidence value interval as a target area, and determine a pedestrian target in the target area.
The step of performing image matching on the suspected pedestrian areas according to a preset human body model to obtain confidence values of the suspected pedestrian areas specifically includes: determining a human body position area of a suspected pedestrian target in the suspected pedestrian area; and carrying out image matching on the human body position area through a preset human body model so as to obtain a confidence value of each suspected pedestrian area.
It should be understood that the horizontally projected histogram of the binary image of the suspected pedestrian object reflects the width information of the suspected pedestrian object, and the vertically projected direct force reflects the height information of the suspected pedestrian object. And calculating a horizontal projection histogram of the suspected pedestrian target binary image, wherein the distance from a starting point to a first minimum value point is the width W of the head, and the overall height of the head and the shoulder is obtained by a formula H of 0.3W/1.3.
Furthermore, a matching area is limited according to the position of the human body part, the suspected pedestrian target is located in the area, the pedestrian is necessarily located in the center of the image, the head is necessarily located in the middle-upper position of the area, the omega-shaped head and shoulder shape is matched with the head and shoulder part which is difficult to completely shield, the confidence value of the target is output, and the initial identification is completed.
And the comprehensive value acquisition module 30 is configured to use the suspected pedestrian area with the confidence value in the second confidence value interval as an undetermined area, and respectively acquire a person comprehensive value of the undetermined area according to a preset blocking object model.
The method is specifically used for: taking a suspected pedestrian area with the confidence value in a second confidence value interval as an undetermined area; respectively acquiring the similarity of each shelter target in the area to be determined according to a preset shelter model; and adding the confidence value and the similarity according to a comprehensive formula to obtain a person comprehensive value of the region to be determined. The step of respectively obtaining the similarity of each shelter target in the area to be determined according to a preset shelter model specifically comprises the following steps: respectively acquiring a shelter position area corresponding to each shelter target in the area to be determined according to a preset shelter model; and carrying out image matching on the position area of the shielding object through the preset shielding object model so as to obtain the corresponding similarity of each shielding object target.
Further, according to the preliminarily recognized confidence value, the first confidence value interval corresponds to a pedestrian target, the second confidence value interval corresponds to a suspected pedestrian target, and targets which do not belong to the preset confidence value interval are non-pedestrian targets and are excluded. For a suspected pedestrian target, more part information judgment is required.
In a specific implementation, a blocking object (a part template such as an umbrella, a riding tool, a leg, a skirt and a human face, which are common pedestrian objects) is matched with a suspected pedestrian object, so that the information content of the suspected pedestrian object is increased, a matching area is limited by the matched part position, the part corresponding to the blocking object generally has a fixed position (such as a cap corresponding to the head of a pedestrian, an upper half of the pedestrian corresponding to the umbrella, a lower half of the skirt corresponding to the lower half of the pedestrian and the like), and the obtained similarity is multiplied by the corresponding weight and then added with the confidence value of the initial recognition according to a comprehensive formula.
In the specific implementation, the embodiment is further explained, for example: the front obstacle is recognized as a face avatar having a head and shoulders, and recognized as a pseudo-pedestrian target that is an advertiser avatar on a bus in front of the autonomous vehicle, but there is no object in the figure to which the lower body (legs) can be matched, in combination with the actual situation. For example: the suspected pedestrian target is successfully matched with the head and the shoulders, but the lower half of the body lacks legs, the suspected pedestrian target is found to be a person riding a motorcycle through matching with the vehicle, and the suspected pedestrian target is judged to be a comprehensive pedestrian target.
Wherein, the comprehensive formula is as follows:
Figure BDA0002494771320000121
b is the comprehensive value of human beings, a is the confidence value, l is the weight corresponding to the confidence value, BiSimilarity corresponding to the object of the obstruction,/iAnd the weight corresponding to the object of the shelter. The shelter can be a human body shape structure, a head and shoulder model, an umbrella, a riding tool, a leg, a human face and the like, and i is the number or the type of the shelter.
The comprehensive value obtaining module 30 is further configured to determine, when the human comprehensive value is greater than or equal to a preset comprehensive value, a human target in an undetermined area corresponding to the human comprehensive value as a comprehensive pedestrian target.
And the generating module 40 is used for generating a pedestrian recognition image according to the pedestrian target and the comprehensive pedestrian target.
It should be understood that, since the legs of the pedestrian are generally in contact with the ground and perpendicular to the ground, the leg parts appear in the lower half area of the image, and the detection is to scan the lower half area of the image, detect a plurality of leg targets, and obtain leg information through non-maximum suppression. And detecting the human face of the cut head-shoulder region according to the Haar characteristics to obtain confidence coefficient because the head-shoulder model is obtained. The invention mainly aims at detecting a side-looking riding tool-shaped object, and the riding tool-shaped object is characterized in that the riding tool-shaped object is positioned below a pedestrian area and is provided with two wheels which are in contact with the ground.
The device further comprises an obstacle image processing module 50, which is used for acquiring an image to be recognized and performing background difference processing on the image to be recognized to acquire a preliminary recognition image.
The method is specifically used for: and performing interframe difference processing on the image to be identified to obtain a background image. And carrying out differential operation on the background image to obtain a moving target image. And carrying out difference operation on the moving target image and the current image to be recognized so as to obtain a preliminary recognition image.
It should be noted that background difference processing is performed by using a Surendra algorithm, difference processing is performed on the image to be identified of the current frame and the adjacent image to be identified of the previous frame, and the pixel value of the image to be identified after difference is compared with a preset threshold value, so as to obtain a binary image. Representing a motion area with a numerical value of 1 in the binary image without updating; and the non-motion area with the numerical value of 0 is updated according to the updating coefficient, and a stable background image can be obtained through iterative operation of a plurality of frames of images.
It is easy to understand that the background image and the image to be recognized of each frame are subjected to differential operation to obtain a moving target image.
It is easy to understand that the moving target image and the image to be identified of the current frame are differentiated to obtain a preliminary identification image.
The obstacle image processing module 50 is further configured to perform three-frame difference processing on the image to be identified to obtain an edge difference map. The method is specifically used for: performing edge detection on a previous frame image of the image to be identified, a next frame image of the image to be identified and the image to be identified so as to obtain corresponding edge images; and carrying out difference processing on the edge image to obtain a first difference image and a second difference image. And taking the first difference image and the second difference image as the edge difference image.
It should be understood that the previous frame image of the image to be recognized, the next frame image of the image to be recognized, and the image to be recognized are continuous three frames of images, and the three frames of images are subjected to edge detection to obtain three frames of corresponding edge images.
It should be understood that, the three frame edge images are subjected to adjacent frame edge image difference operation, and two difference images, namely a first difference image and a second difference image, are obtained, namely the edge difference images.
And the obstacle image processing module 50 is further configured to obtain an obstacle image according to the preliminary identification map and the edge difference map. The method is specifically used for: and carrying out logical OR operation according to the preliminary identification image and the first difference image to obtain a medium-term identification image. And performing logical AND operation according to the medium-term identification image and the second difference image to obtain a dynamic obstacle image. And carrying out dynamic obstacle detection according to the dynamic obstacle image.
It is easy to understand that the first differential map and the preliminary recognition map are logically or-ed, the operation result and the second differential map are logically and-ed, and the dynamic obstacle result is detected.
The obstacle area is preliminarily recognized by matching the head and shoulder shape and the human body shape structure, the preliminarily recognized pedestrian target and the suspected pedestrian target are obtained, then the interference objects which are easy to shield the human body, such as the umbrella-shaped objects, the riding tool-shaped objects and the like, are subjected to feature matching with the suspected pedestrian target, pedestrians are screened out from the suspected pedestrian target, non-pedestrians are eliminated, and the feature combination method for not only overcoming the shielding problem but also enriching the pedestrian feature information is achieved. The problem of shielding is overcome, the target identification precision of forward-looking pedestrians is improved, and the safety of automatic driving is improved. And a more accurate dynamic obstacle image is obtained, a more stable foundation is provided for the identification of a pedestrian target, and the problems of incomplete detection target of a frame difference method and omission of the background difference method are solved by combining the advantages of the frame difference method and the background difference method.
Furthermore, an embodiment of the present invention further provides a storage medium, on which a pedestrian recognition program is stored, where the pedestrian recognition program is executed by a processor to perform the steps of the pedestrian recognition method as described above.
Since the storage medium adopts all technical solutions of all the embodiments, at least all the beneficial effects brought by the technical solutions of the embodiments are achieved, and no further description is given here.
It should be understood that the above is only an example, and the technical solution of the present invention is not limited in any way, and in a specific application, a person skilled in the art may set the technical solution as needed, and the present invention is not limited thereto.
It should be noted that the above-described work flows are only exemplary, and do not limit the scope of the present invention, and in practical applications, a person skilled in the art may select some or all of them to achieve the purpose of the solution of the embodiment according to actual needs, and the present invention is not limited herein.
In addition, the technical details that are not described in detail in this embodiment may refer to the pedestrian identification method provided in any embodiment of the present invention, and are not described herein again.
Further, it is to be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present invention or portions thereof that contribute to the prior art may be embodied in the form of a software product, where the computer software product is stored in a storage medium (e.g., Read Only Memory (ROM)/RAM, magnetic disk, optical disk), and includes several instructions for enabling a terminal electronic device (e.g., a mobile phone, a computer, a server, or a network electronic device) to execute the method according to the embodiments of the present invention.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.

Claims (10)

1. A pedestrian identification method, characterized in that the method comprises:
determining suspected pedestrian areas in an obstacle image, and acquiring confidence values of the suspected pedestrian areas;
taking a suspected pedestrian area with the confidence value in a first confidence value interval as a target area, and determining a pedestrian target in the target area;
taking a suspected pedestrian area with the confidence value in a second confidence value interval as an undetermined area, and respectively obtaining a person comprehensive value of the undetermined area according to a preset sheltering object model;
when the figure comprehensive value is larger than or equal to a preset comprehensive value, determining a figure target in an undetermined area corresponding to the figure comprehensive value as a comprehensive pedestrian target;
and generating a pedestrian recognition image according to the pedestrian target and the comprehensive pedestrian target.
2. The method of pedestrian identification according to claim 1, wherein the step of determining areas of suspected pedestrian in an image of an obstacle and obtaining confidence values for each of the areas of suspected pedestrian is preceded by the step of:
acquiring an image to be identified, and carrying out background difference processing on the image to be identified to acquire a primary identification image;
performing three-frame difference processing on the image to be identified to obtain an edge difference image;
and obtaining an obstacle image according to the preliminary identification image and the edge difference image.
3. The method according to claim 2, wherein the step of determining the suspected pedestrian areas in the obstacle image and obtaining the confidence values of each of the suspected pedestrian areas comprises:
determining an undetermined pedestrian area in an obstacle image, and performing area amplification on the undetermined pedestrian area to obtain a suspected pedestrian area;
and carrying out image matching on the suspected pedestrian areas according to a preset human body model so as to obtain confidence values of all the suspected pedestrian areas.
4. The pedestrian recognition method according to claim 3, wherein the step of determining a region of a pedestrian to be detected in the obstacle image and performing region expansion on the region of the pedestrian to obtain the region of the pedestrian to be detected specifically comprises:
determining an undetermined pedestrian area in the obstacle image according to a pedestrian area formula;
performing regional amplification on the area of the pedestrian to be detected to obtain a suspected pedestrian area;
wherein the pedestrian area formula is:
Figure FDA0002494771310000021
wherein, ROI is the region of undetermined pedestrian, LwIs the aspect ratio of the region, SpIs the ratio of the area of the regions, S is the area of the regions, SminPresetting a minimum area, S, for a regionmaxPresetting a maximum area, L, for a regionminPresetting a minimum aspect ratio, L, for a regionmaxPresetting a maximum aspect ratio, S, for a regionpmaxFor presetting the area ratio of the maximum segmentation region to the image display page, SpminThe area ratio of the minimum segmentation region to the image presentation page is preset.
5. The pedestrian identification method according to claim 4, wherein the step of performing image matching on the suspected pedestrian areas according to a preset human body model to obtain confidence values of the suspected pedestrian areas specifically comprises:
determining a human body position area of a suspected pedestrian target in the suspected pedestrian area;
and carrying out image matching on the human body position area through a preset human body model so as to obtain a confidence value of each suspected pedestrian area.
6. The pedestrian recognition method according to claim 5, wherein the step of taking the suspected pedestrian area with the confidence value in the second confidence value interval as the undetermined area and respectively obtaining the comprehensive values of the people in the undetermined area according to a preset blocking object model specifically comprises:
taking a suspected pedestrian area with the confidence value in a second confidence value interval as an undetermined area;
respectively acquiring the similarity of each shelter target in the area to be determined according to a preset shelter model;
adding the confidence value and the similarity according to a comprehensive formula to obtain a person comprehensive value of the region to be determined;
wherein, the comprehensive formula is as follows:
Figure FDA0002494771310000022
wherein, B is a human comprehensive value, a is a confidence value, l is a weight corresponding to the confidence value, BiSimilarity corresponding to the object of the obstruction,/iAnd the weight corresponding to the object of the shelter.
7. The pedestrian recognition method according to claim 6, wherein the step of obtaining the similarity of each blocking object target in the region to be determined according to a preset blocking object model includes:
respectively acquiring a shelter position area corresponding to each shelter target in the area to be determined according to a preset shelter model;
and carrying out image matching on the position area of the shielding object through the preset shielding object model so as to obtain the corresponding similarity of each shielding object target.
8. A pedestrian recognition apparatus, characterized in that the apparatus comprises:
the confidence value acquisition module is used for determining suspected pedestrian areas in the obstacle image and acquiring the confidence value of each suspected pedestrian area;
the determining module is used for taking a suspected pedestrian area with the confidence value in the first confidence value interval as a target area and determining a pedestrian target in the target area;
the comprehensive value acquisition module is used for taking a suspected pedestrian area with the confidence value in the second confidence value interval as an undetermined area and respectively acquiring a person comprehensive value of the undetermined area according to a preset shelter model; the pedestrian comprehensive value judging module is also used for judging the human target in the undetermined area corresponding to the human comprehensive value as a comprehensive pedestrian target when the human comprehensive value is larger than or equal to a preset comprehensive value;
and the generation module is used for generating a pedestrian recognition image according to the pedestrian target and the comprehensive pedestrian target.
9. An electronic device, characterized in that the electronic device comprises: memory, a processor and a pedestrian identification program stored on the memory and executable on the processor, the pedestrian identification program being configured to implement the steps of the pedestrian identification method according to any one of claims 1 to 7.
10. A storage medium, characterized in that the storage medium has stored thereon a pedestrian recognition program which, when executed by a processor, implements the steps of the pedestrian recognition method according to any one of claims 1 to 7.
CN202010424166.1A 2020-05-15 2020-05-15 Pedestrian recognition method and device, electronic equipment and storage medium Active CN111626165B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010424166.1A CN111626165B (en) 2020-05-15 2020-05-15 Pedestrian recognition method and device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010424166.1A CN111626165B (en) 2020-05-15 2020-05-15 Pedestrian recognition method and device, electronic equipment and storage medium

Publications (2)

Publication Number Publication Date
CN111626165A true CN111626165A (en) 2020-09-04
CN111626165B CN111626165B (en) 2024-02-02

Family

ID=72259865

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010424166.1A Active CN111626165B (en) 2020-05-15 2020-05-15 Pedestrian recognition method and device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN111626165B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112329724A (en) * 2020-11-26 2021-02-05 四川大学 Real-time detection and snapshot method for lane change of motor vehicle

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105844232A (en) * 2016-03-21 2016-08-10 联想(北京)有限公司 Image identification method and device
JP2017117244A (en) * 2015-12-24 2017-06-29 Kddi株式会社 Image analysis device for detecting a person displayed in photographic image of camera
WO2018223955A1 (en) * 2017-06-09 2018-12-13 北京深瞐科技有限公司 Target monitoring method, target monitoring device, camera and computer readable medium
CN110781821A (en) * 2019-10-25 2020-02-11 上海商汤智能科技有限公司 Target detection method and device based on unmanned aerial vehicle, electronic equipment and storage medium

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2017117244A (en) * 2015-12-24 2017-06-29 Kddi株式会社 Image analysis device for detecting a person displayed in photographic image of camera
CN105844232A (en) * 2016-03-21 2016-08-10 联想(北京)有限公司 Image identification method and device
WO2018223955A1 (en) * 2017-06-09 2018-12-13 北京深瞐科技有限公司 Target monitoring method, target monitoring device, camera and computer readable medium
CN110781821A (en) * 2019-10-25 2020-02-11 上海商汤智能科技有限公司 Target detection method and device based on unmanned aerial vehicle, electronic equipment and storage medium

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
曹丽武;: "视频图像序列中人体目标的提取与跟踪" *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112329724A (en) * 2020-11-26 2021-02-05 四川大学 Real-time detection and snapshot method for lane change of motor vehicle
CN112329724B (en) * 2020-11-26 2022-08-05 四川大学 Real-time detection and snapshot method for lane change of motor vehicle

Also Published As

Publication number Publication date
CN111626165B (en) 2024-02-02

Similar Documents

Publication Publication Date Title
JP4203512B2 (en) Vehicle periphery monitoring device
KR101971866B1 (en) Method and apparatus for detecting object in moving image and storage medium storing program thereof
JP3934119B2 (en) Vehicle periphery monitoring device
JP4263737B2 (en) Pedestrian detection device
JP6276519B2 (en) Person counting device and human flow line analyzing device
US9092868B2 (en) Apparatus for detecting object from image and method therefor
US9298990B2 (en) Object tracking method and device
WO2022183682A1 (en) Target determination method and apparatus, electronic device, and computer-readable storage medium
JPWO2008020544A1 (en) Vehicle detection device, vehicle detection method, and vehicle detection program
KR101483742B1 (en) Lane Detection method for Advanced Vehicle
EP2821935B1 (en) Vehicle detection method and device
US10255512B2 (en) Method, system and apparatus for processing an image
CN111626165A (en) Pedestrian recognition method and device, electronic equipment and storage medium
WO2011108258A1 (en) Vehicle-surroundings monitoring apparatus and vehicle-surroundings monitoring method
CN110660000A (en) Data prediction method, device, equipment and computer readable storage medium
KR101553012B1 (en) Apparatus and method for extracting object
US20240071104A1 (en) Image processing device, image processing method, and recording medium
CN112329508A (en) Gesture recognition method, device, equipment and readable storage medium
CN112347853A (en) License plate data desensitization method based on video, storage medium and server
CN113298811B (en) Automatic counting method, device and equipment for number of people in intelligent classroom and storage medium
CN115457644B (en) Picture identification method and device for obtaining target based on expansion space mapping
CN116612279B (en) Method, device, network equipment and storage medium for target detection
KR101895678B1 (en) Efficient Search Window Set-Up Method for the Automotive Image Recognition System
CN117485345A (en) Method, system, equipment and storage medium for determining forward-looking perception of parking scene
Rosebrock et al. Using the shadow as a single feature for real-time monocular vehicle pose determination

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant