CN111626165B - Pedestrian recognition method and device, electronic equipment and storage medium - Google Patents
Pedestrian recognition method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN111626165B CN111626165B CN202010424166.1A CN202010424166A CN111626165B CN 111626165 B CN111626165 B CN 111626165B CN 202010424166 A CN202010424166 A CN 202010424166A CN 111626165 B CN111626165 B CN 111626165B
- Authority
- CN
- China
- Prior art keywords
- pedestrian
- area
- suspected
- target
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 70
- 238000012545 processing Methods 0.000 claims description 19
- 230000011218 segmentation Effects 0.000 claims description 10
- 230000003321 amplification Effects 0.000 claims description 7
- 238000003199 nucleic acid amplification method Methods 0.000 claims description 7
- 238000005516 engineering process Methods 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 10
- 238000001514 detection method Methods 0.000 description 9
- 238000004891 communication Methods 0.000 description 5
- 230000000877 morphologic effect Effects 0.000 description 5
- 238000003708 edge detection Methods 0.000 description 4
- 241001424438 Surendra Species 0.000 description 2
- 230000037237 body shape Effects 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 230000001629 suppression Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/103—Static body considered as a whole, e.g. static pedestrian or occupant recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/751—Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
Abstract
The present invention relates to the field of automatic driving technologies, and in particular, to a pedestrian recognition method, a device, an electronic apparatus, and a storage medium. The method comprises the following steps: determining suspected pedestrian areas in the obstacle image, and acquiring confidence values of the suspected pedestrian areas; taking a suspected pedestrian area with the confidence value in a first confidence value interval as a target area, and determining a pedestrian target in the target area; taking the suspected pedestrian area with the confidence value in the second confidence value interval as a to-be-determined area, and respectively acquiring the character comprehensive value of the to-be-determined area according to a preset shelter model; when the character comprehensive value is larger than or equal to a preset comprehensive value, judging the character target in the undetermined area corresponding to the character comprehensive value as a comprehensive pedestrian target; and generating a pedestrian identification image according to the pedestrian target and the comprehensive pedestrian target. The accuracy of pedestrian identification in the automatic driving process is improved, and pedestrian targets in various forms are accurately identified.
Description
Technical Field
The present invention relates to the field of automatic driving technologies, and in particular, to a pedestrian recognition method, a device, an electronic apparatus, and a storage medium.
Background
The safety problem of automatic driving is more and more focused, and the most important of the problems is that an automatic driving system recognizes and protects pedestrians, a radar sensor can detect obstacles, but the effect of recognizing pedestrians and states thereof is poor, so that the recognition of forward-looking pedestrians is significant in the development of automatic driving by utilizing a visual algorithm, and the safety performance of automatic driving is determined to a certain extent.
The human body overall contour method regards human body as a whole for detection, and when the incomplete or shielding of the target occurs, the error is larger; meanwhile, in automatic driving, forward looking pedestrians are dynamic, and it is difficult for templates to include all pedestrian states. In the existing human body part recognition method, the legs and the heads are mostly used as important parts, if a pedestrian carries an umbrella or wears a skirt, the legs or the heads are shielded, the problem of missed detection of the pedestrian is easy to occur, and potential safety hazards are generated.
The foregoing is provided merely for the purpose of facilitating understanding of the technical solutions of the present invention and is not intended to represent an admission that the foregoing is prior art.
Disclosure of Invention
The invention mainly aims to provide a pedestrian recognition method, a device, electronic equipment and a storage medium, and aims to solve the technical problem of pedestrian recognition.
In order to achieve the above object, the present invention provides a pedestrian recognition method, the method comprising:
determining suspected pedestrian areas in the obstacle image, and acquiring confidence values of the suspected pedestrian areas;
taking a suspected pedestrian area with the confidence value in a first confidence value interval as a target area, and determining a pedestrian target in the target area;
taking the suspected pedestrian area with the confidence value in the second confidence value interval as a to-be-determined area, and respectively acquiring the character comprehensive value of the to-be-determined area according to a preset shelter model;
when the character comprehensive value is larger than or equal to a preset comprehensive value, judging the character target in the undetermined area corresponding to the character comprehensive value as a comprehensive pedestrian target;
and generating a pedestrian identification image according to the pedestrian target and the comprehensive pedestrian target.
Preferably, the step of determining the suspected pedestrian area in the obstacle image and obtaining the confidence value of each suspected pedestrian area specifically includes:
determining a to-be-determined pedestrian area in the obstacle image, and carrying out area amplification on the to-be-determined pedestrian area to obtain a suspected pedestrian area;
and performing image matching on the suspected pedestrian areas according to a preset human body model to obtain confidence values of the suspected pedestrian areas.
Preferably, the step of determining a pending pedestrian area in the obstacle image and performing area amplification on the pending pedestrian area to obtain a suspected pedestrian area specifically includes:
determining a pending pedestrian area in the obstacle image according to a pedestrian area formula;
performing region amplification on the to-be-determined pedestrian region to obtain a suspected pedestrian region;
wherein, the pedestrian area formula is:
wherein the ROI is a pending pedestrian area, L w Is the aspect ratio of the region S p Is the area ratio of the areas, S is the area of the area, S min Preset minimum area for region S max Preset maximum area for region L min Presetting minimum aspect ratio for region, L max Presetting maximum aspect ratio for region S pmax S is to preset the area ratio of the maximum segmentation area to the image display page pmin The area ratio of the minimum segmentation area to the image display page is preset.
Preferably, the step of performing image matching on the suspected pedestrian areas according to a preset human body model to obtain confidence values of the suspected pedestrian areas specifically includes:
determining a human body position area of a suspected pedestrian target in the suspected pedestrian area;
and carrying out image matching on the human body position areas through a preset human body model so as to obtain confidence values of the suspected pedestrian areas.
Preferably, the step of taking the suspected pedestrian area with the confidence value in the second confidence value interval as a pending area and respectively acquiring the character comprehensive value of the pending area according to a preset shelter model specifically includes:
taking the suspected pedestrian area with the confidence value in the second confidence value interval as a pending area;
respectively acquiring the similarity of each shielding object target in the undetermined area according to a preset shielding object model;
adding the confidence value and the similarity according to a comprehensive formula to obtain a character comprehensive value of the undetermined area;
wherein, the comprehensive formula is:
wherein B is a human-object comprehensive value, a is a confidence value, l is a weight corresponding to the confidence value, B i For the corresponding similarity of the shielding object, l i Is the weight corresponding to the object of the shielding object.
Preferably, the step of obtaining the similarity of each occlusion object in the undetermined area according to a preset occlusion model includes:
respectively acquiring a shade position area corresponding to each shade target in the undetermined area according to a preset shade model;
and carrying out image matching on the position area of the shielding object through the preset shielding object model so as to obtain the similarity corresponding to each shielding object target.
In addition, in order to achieve the above object, the present invention also proposes a pedestrian recognition apparatus including:
the confidence value acquisition module is used for determining suspected pedestrian areas in the obstacle image and acquiring confidence values of the suspected pedestrian areas;
the determining module is used for taking a suspected pedestrian area with the confidence value in the first confidence value interval as a target area and determining a pedestrian target in the target area;
the comprehensive value acquisition module is used for taking a suspected pedestrian area with the confidence value in the second confidence value interval as a to-be-determined area and respectively acquiring the character comprehensive value of the to-be-determined area according to a preset shelter model; and the method is also used for judging the character target in the undetermined area corresponding to the character comprehensive value as a comprehensive pedestrian target when the character comprehensive value is larger than or equal to a preset comprehensive value;
and the generation module is used for generating a pedestrian identification image according to the pedestrian target and the comprehensive pedestrian target.
In addition, to achieve the above object, the present invention also proposes an electronic device including: the pedestrian recognition system comprises a memory, a processor and a pedestrian recognition program stored on the memory and capable of running on the processor, wherein the pedestrian recognition program is configured to realize the steps of the pedestrian recognition method.
In addition, in order to achieve the above object, the present invention also proposes a storage medium having stored thereon a pedestrian recognition program which, when executed by a processor, implements the steps of the pedestrian recognition method as described above.
The method comprises the steps of determining suspected pedestrian areas in an obstacle image and obtaining confidence values of the suspected pedestrian areas; taking a suspected pedestrian area with the confidence value in a first confidence value interval as a target area, and determining a pedestrian target in the target area; taking the suspected pedestrian area with the confidence value in the second confidence value interval as a to-be-determined area, and respectively acquiring the character comprehensive value of the to-be-determined area according to a preset shelter model; when the character comprehensive value is larger than or equal to a preset comprehensive value, judging the character target in the undetermined area corresponding to the character comprehensive value as a comprehensive pedestrian target; and generating a pedestrian identification image according to the pedestrian target and the comprehensive pedestrian target. The method comprises the steps of carrying out preliminary recognition on an obstacle region through matching the shape of a head and a shoulder and the morphological structure of a human body, obtaining a preliminary recognized pedestrian target and a suspected pedestrian target, carrying out feature matching on an interference object which is easy to shield a human body, such as an umbrella-shaped object, a riding tool-shaped object and the like, and the suspected pedestrian target, and screening pedestrians from the suspected pedestrian target to exclude non-pedestrians, thereby overcoming the shielding problem and enriching the pedestrian feature information. The shielding problem is overcome, the recognition precision of the forward-looking pedestrian target is improved, and the safety of automatic driving is improved. And a more accurate dynamic obstacle image is obtained, a more stable foundation is provided for the identification of pedestrian targets, the advantages of a frame difference method and a background difference method are combined, and the problems of incomplete detection targets and omission of the background difference method of the frame difference method are solved.
Drawings
FIG. 1 is a schematic diagram of an electronic device of a hardware operating environment according to an embodiment of the present invention;
FIG. 2 is a flowchart of a first embodiment of a pedestrian recognition method according to the present invention;
FIG. 3 is a flowchart of a pedestrian recognition method according to a second embodiment of the present invention;
fig. 4 is a block diagram showing the construction of a first embodiment of the pedestrian recognition device of the present invention.
The achievement of the objects, functional features and advantages of the present invention will be further described with reference to the accompanying drawings, in conjunction with the embodiments.
Detailed Description
It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
Referring to fig. 1, fig. 1 is a schematic structural diagram of an electronic device in a hardware running environment according to an embodiment of the present invention.
As shown in fig. 1, the electronic device may include: a processor 1001, such as a central processing unit (Central Processing Unit, CPU), a communication bus 1002, a user interface 1003, a network interface 1004, a memory 1005. Wherein the communication bus 1002 is used to enable connected communication between these components. The user interface 1003 may include a Display, an input unit such as a Keyboard (Keyboard), and the optional user interface 1003 may further include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a WIreless interface (e.g., a WIreless-FIdelity (WI-FI) interface). The Memory 1005 may be a high-speed random access Memory (Random Access Memory, RAM) Memory or a stable nonvolatile Memory (NVM), such as a disk Memory. The memory 1005 may also optionally be a storage device separate from the processor 1001 described above.
Those skilled in the art will appreciate that the structure shown in fig. 1 is not limiting of the electronic device and may include more or fewer components than shown, or may combine certain components, or may be arranged in different components.
As shown in fig. 1, an operating system, a network communication module, a user interface module, and a pedestrian recognition program may be included in the memory 1005 as one type of storage medium.
In the electronic device shown in fig. 1, the network interface 1004 is mainly used for data communication with a network server; the user interface 1003 is mainly used for data interaction with a user; the processor 1001 and the memory 1005 in the electronic device of the present invention may be provided in the electronic device, and the electronic device invokes the pedestrian recognition program stored in the memory 1005 through the processor 1001 and executes the pedestrian recognition method provided by the embodiment of the present invention.
Referring to fig. 2, fig. 2 is a schematic flow chart of a first embodiment of a pedestrian recognition method according to the present invention. In this embodiment, the pedestrian recognition method includes the following steps:
step S10: and determining suspected pedestrian areas in the obstacle image, and acquiring confidence values of the suspected pedestrian areas.
In the invention, the obstacle image is acquired by the front-view camera of the automatic driving automobile, the area where the obstacle is located is primarily identified by matching the shape of the head and the shoulder and the human body morphological structure, then the shape of the part of the obstacle is matched with umbrella-shaped objects, riding tool-shaped objects, leg-shaped objects and human-like objects, and the confidence value is calculated according to the output weight of the matching result. The umbrella, the riding tool, the head shoulder and the leg of the shielded part become possible secondary characteristics of pedestrians, so that the shielding problem is solved and the pedestrian characteristic information is enriched.
It should be noted that step S10 specifically includes: determining a to-be-determined pedestrian area in the obstacle image, and carrying out area amplification on the to-be-determined pedestrian area to obtain a suspected pedestrian area; and performing image matching on the suspected pedestrian areas according to a preset human body model to obtain confidence values of the suspected pedestrian areas.
It should be understood that the obstacle image mainly includes vehicle obstacles in addition to pedestrians, and the suspected pedestrian area in the obstacle image needs to be selected, the human morphology structure is different from other object morphologies, and the human height-to-body width ratio L is selected w Area S of pedestrian area, area of obstacle dividing area and area ratio S of rectangular window p As an evaluation criterion, a suspected pedestrian area is obtained by a pedestrian area formula.
The pedestrian area formula is:
wherein the ROI is a pending pedestrian area, L w Is the aspect ratio of the region S p Is the area ratio of the areas, S is the area of the area, S min Preset minimum area for region S max Preset maximum area for region L min Presetting minimum aspect ratio for region, L max Presetting maximum aspect ratio for region S pmax S is to preset the area ratio of the maximum segmentation area to the image display page pmin The area ratio of the minimum segmentation area to the image display page is preset.
And obtaining a suspected pedestrian obstacle area according to the judgment standard. And finally, expanding the suspected area outwards by a preset multiple to enable the pedestrian target in the suspected area to be contained as much as possible.
Step S20: and taking the suspected pedestrian area with the confidence value in the first confidence value interval as a target area, and determining a pedestrian target in the target area.
The step of performing image matching on the suspected pedestrian areas according to a preset human body model to obtain confidence values of the suspected pedestrian areas specifically includes: determining a human body position area of a suspected pedestrian target in the suspected pedestrian area; and carrying out image matching on the human body position areas through a preset human body model so as to obtain confidence values of the suspected pedestrian areas.
It should be understood that the horizontal projection histogram of the binary image of the suspected pedestrian object reflects the width information of the suspected pedestrian object and the vertical projection straight force reflects the height information of the suspected pedestrian object. Calculating a horizontal projection histogram of the suspected pedestrian target binary image, wherein the distance from the first minimum point to the point is the width W of the head, and the head-shoulder overall height is obtained through the formula H=0.3×W/1.3.
Further, the matching area is limited according to the position of the human body part, the suspected pedestrian is located in the area where the target is located, the pedestrian is located at the center of the image, the head is located at the middle upper part of the area, the head and shoulder parts which are least prone to being completely shielded are matched through the omega-shaped head and shoulder shape, the confidence value of the target is output, and the preliminary identification is completed.
Step S30: and taking the suspected pedestrian area with the confidence value in the second confidence value interval as a to-be-determined area, and respectively acquiring the character comprehensive value of the to-be-determined area according to a preset shelter model.
The step S30 specifically includes: taking the suspected pedestrian area with the confidence value in the second confidence value interval as a pending area; respectively acquiring the similarity of each shielding object target in the undetermined area according to a preset shielding object model; and adding the confidence value and the similarity according to a comprehensive formula to obtain the character comprehensive value of the undetermined area.
The step of respectively obtaining the similarity of each occlusion object in the undetermined area according to a preset occlusion model specifically comprises the following steps: respectively acquiring a shade position area corresponding to each shade target in the undetermined area according to a preset shade model; and carrying out image matching on the position area of the shielding object through the preset shielding object model so as to obtain the similarity corresponding to each shielding object target.
Further, according to the initially recognized confidence value, the first confidence value interval corresponds to a pedestrian target, the second confidence value interval corresponds to a suspected pedestrian target, and the target which does not belong to the preset confidence value interval is a non-pedestrian target and is eliminated. For a suspected pedestrian target, more location information decisions are required.
In a specific implementation, a shielding object (an umbrella-shaped object, a riding tool object, a leg-shaped object, a skirt, a human face and other pedestrian targets are matched with a suspected pedestrian target, the information quantity of the suspected pedestrian target is increased, meanwhile, the matched position limits a matching area, the position corresponding to the shielding object is generally provided with a fixed position (such as a head of a cap corresponding to a pedestrian, an upper half of the umbrella corresponding to the pedestrian, a lower half of the skirt corresponding to the pedestrian and the like), the obtained similarity is multiplied by a corresponding weight, and then the obtained similarity is added with a confidence value of preliminary identification according to a comprehensive formula.
In specific implementation, this embodiment is further explained, for example: the front obstacle is identified as a face head portrait, has a head and a shoulder, is identified as a suspected pedestrian target, but there is no object in the figure that the lower body (leg object) can be matched, and the suspected pedestrian target is an advertisement portrait on a bus in front of an automatic driving vehicle in combination with the actual situation. For example: the suspected pedestrian target is successfully matched with the head and the shoulder, however, the lower body lacks legs, the suspected pedestrian target is found to be a person riding a motorcycle after being matched with the vehicle, and the suspected pedestrian target is judged to be a comprehensive pedestrian target.
Wherein, the comprehensive formula is:
b is a comprehensive value of the human body, a is a confidence value, l is a weight corresponding to the confidence value, B i For the corresponding similarity of the shielding object, l i Is the weight corresponding to the object of the shielding object. The shielding objects can be human body shape structures, head and shoulder models, umbrella-shaped objects, riding tool-shaped objects, leg-shaped objects, faces and the like, and i is the number or the type of the shielding objects.
Step S40: and when the character comprehensive value is greater than or equal to a preset comprehensive value, judging the character target in the undetermined area corresponding to the character comprehensive value as a comprehensive pedestrian target.
Step S50: and generating a pedestrian identification image according to the pedestrian target and the comprehensive pedestrian target.
It should be appreciated that since the pedestrian's leg is generally ground-contacting and vertical, the leg portion will appear in the lower half of the image, and detection is by scanning the lower half of the area to detect multiple leg targets, and then obtaining leg information via non-maximum suppression. And as the head-shoulder model is already obtained, the detection of the human face is carried out according to the Haar characteristics in the cut head-shoulder area, and the confidence is obtained. The invention is mainly aimed at detecting the side-looking riding tool object, and the riding tool object is characterized in that the riding tool object is positioned below a pedestrian area and two wheels which contact the ground are arranged at the same time.
The method of the embodiment carries out preliminary recognition on the obstacle region by matching the shape of the head and the shoulder and the human morphological structure, obtains a preliminary recognized pedestrian target and a suspected pedestrian target, carries out characteristic matching on the interference objects which are easy to shield the human body, such as umbrella-shaped objects, riding tool-shaped objects and the like, and the suspected pedestrian target, screens out pedestrians from the suspected pedestrian target to exclude non-pedestrians, and not only overcomes the shielding problem, but also enriches the characteristic information of the pedestrians. The shielding problem is overcome, the recognition precision of the forward-looking pedestrian target is improved, and the safety of automatic driving is improved.
Referring to fig. 3, fig. 3 is a flowchart of a second embodiment of a pedestrian recognition method according to the present invention. Based on the first embodiment, the present implementation routine identification method further includes, before the step S10:
step S11: and obtaining an image to be identified, and carrying out background difference processing on the image to be identified to obtain a preliminary identification image.
The step S11 specifically includes: and carrying out inter-frame difference processing on the image to be identified to obtain a background image. And carrying out differential operation on the background image to obtain a moving object image. And carrying out differential operation on the moving target image and the current image to be identified so as to obtain a preliminary identification image.
It should be noted that, background difference processing is performed by using Surendra algorithm, difference processing is performed on the image to be identified of the current frame and the image to be identified of the adjacent previous frame, and pixel values of the image to be identified after difference are compared with a preset threshold value to obtain a binarized image. The motion area of the binary image with the value of 1 is represented and is not updated; and the non-motion area with the value of 0 is updated according to the updating coefficient, and a stable background image can be obtained through iterative operation of multi-frame images.
It is easy to understand that the background image and the image to be identified of each frame are subjected to differential operation to obtain the moving object image.
It is easy to understand that the moving object image and the image to be identified of the current frame are differentiated to obtain a preliminary identification image.
Step S12: and carrying out three-frame difference processing on the image to be identified to obtain an edge difference map.
The method specifically comprises the following steps: performing edge detection on a previous frame image of the image to be identified, a next frame image of the image to be identified and the image to be identified so as to obtain a corresponding edge image; and carrying out differential processing on the edge image to obtain a first differential graph and a second differential graph. And taking the first differential graph and the second differential graph as the edge differential graph.
It should be understood that the previous frame of the image to be identified, the next frame of the image to be identified and the image to be identified are continuous three frames of images, and the three frames of images are subjected to edge detection to obtain edge images corresponding to the three frames.
It should be understood that, performing differential operation on the edge images of the three frames by using the edge images of the adjacent frames to obtain two differential images, namely a first differential image and a second differential image, namely the edge differential images.
Step S13: and obtaining an obstacle image according to the preliminary identification image and the edge difference image.
The method specifically comprises the following steps: and carrying out logical OR operation according to the preliminary identification graph and the first differential graph to obtain a mid-term identification graph. And performing logical AND operation according to the medium-term identification graph and the second differential graph to acquire a dynamic obstacle image. And detecting the dynamic obstacle according to the dynamic obstacle image.
It is easy to understand that the first differential diagram and the preliminary identification diagram are logically or, the operation result and the second differential diagram are logically and, and a dynamic obstacle result is detected.
According to the embodiment of the invention, a more accurate dynamic obstacle image is obtained by the method, a more stable foundation is provided for the identification of the pedestrian target, the advantages of the frame difference method and the background difference method are combined, and the problems of incomplete detection target and omission of the background difference method of the frame difference method are solved.
Referring to fig. 4, fig. 4 is a block diagram showing the construction of a first embodiment of the pedestrian recognition device of the present invention.
As shown in fig. 4, the present invention implements a routine identification device including:
the confidence value acquisition module is used for determining suspected pedestrian areas in the obstacle image and acquiring confidence values of the suspected pedestrian areas.
In the invention, the obstacle image is acquired by the front-view camera of the automatic driving automobile, the area where the obstacle is located is primarily identified by matching the shape of the head and the shoulder and the human body morphological structure, then the shape of the part of the obstacle is matched with umbrella-shaped objects, riding tool-shaped objects, leg-shaped objects and human-like objects, and the confidence value is calculated according to the output weight of the matching result. The umbrella, the riding tool, the head shoulder and the leg of the shielded part become possible secondary characteristics of pedestrians, so that the shielding problem is solved and the pedestrian characteristic information is enriched.
It should be noted that the method is specifically used for: determining a to-be-determined pedestrian area in the obstacle image, and carrying out area amplification on the to-be-determined pedestrian area to obtain a suspected pedestrian area; and performing image matching on the suspected pedestrian areas according to a preset human body model to obtain confidence values of the suspected pedestrian areas.
It should be understood that the obstacle image mainly includes vehicle obstacles in addition to pedestrians, and the suspected pedestrian area in the obstacle image needs to be selected, the human morphology structure is different from other object morphologies, and the human height-to-body width ratio L is selected w Area S of pedestrian area, area of obstacle dividing area and area ratio S of rectangular window p As an evaluation criterion, a suspected pedestrian area is obtained by a pedestrian area formula.
The pedestrian area formula is:
wherein the ROI is a pending pedestrian area, L w Is the aspect ratio of the region S p Is the area ratio of the areas, S is the area of the area, S min Preset minimum area for region S max Preset maximum area for region L min Presetting minimum aspect ratio for region, L max Presetting maximum aspect ratio for region S pmax S is to preset the area ratio of the maximum segmentation area to the image display page pmin The area ratio of the minimum segmentation area to the image display page is preset.
And obtaining a suspected pedestrian obstacle area according to the judgment standard. And finally, expanding the suspected area outwards by a preset multiple to enable the pedestrian target in the suspected area to be contained as much as possible.
The determining module 20 is configured to take a suspected pedestrian area with the confidence value located in the first confidence value interval as a target area, and determine a pedestrian target in the target area.
The step of performing image matching on the suspected pedestrian areas according to a preset human body model to obtain confidence values of the suspected pedestrian areas specifically includes: determining a human body position area of a suspected pedestrian target in the suspected pedestrian area; and carrying out image matching on the human body position areas through a preset human body model so as to obtain confidence values of the suspected pedestrian areas.
It should be understood that the horizontal projection histogram of the binary image of the suspected pedestrian object reflects the width information of the suspected pedestrian object and the vertical projection straight force reflects the height information of the suspected pedestrian object. Calculating a horizontal projection histogram of the suspected pedestrian target binary image, wherein the distance from the first minimum point to the point is the width W of the head, and the head-shoulder overall height is obtained through the formula H=0.3×W/1.3.
Further, the matching area is limited according to the position of the human body part, the suspected pedestrian is located in the area where the target is located, the pedestrian is located at the center of the image, the head is located at the middle upper part of the area, the head and shoulder parts which are least prone to being completely shielded are matched through the omega-shaped head and shoulder shape, the confidence value of the target is output, and the preliminary identification is completed.
The comprehensive value obtaining module 30 is configured to take a suspected pedestrian area with the confidence value in the second confidence value interval as a pending area, and obtain the person comprehensive value of the pending area according to a preset occlusion model.
The method is particularly used for: taking the suspected pedestrian area with the confidence value in the second confidence value interval as a pending area; respectively acquiring the similarity of each shielding object target in the undetermined area according to a preset shielding object model; and adding the confidence value and the similarity according to a comprehensive formula to obtain the character comprehensive value of the undetermined area. The step of respectively obtaining the similarity of each occlusion object in the undetermined area according to a preset occlusion model specifically comprises the following steps: respectively acquiring a shade position area corresponding to each shade target in the undetermined area according to a preset shade model; and carrying out image matching on the position area of the shielding object through the preset shielding object model so as to obtain the similarity corresponding to each shielding object target.
Further, according to the initially recognized confidence value, the first confidence value interval corresponds to a pedestrian target, the second confidence value interval corresponds to a suspected pedestrian target, and the target which does not belong to the preset confidence value interval is a non-pedestrian target and is eliminated. For a suspected pedestrian target, more location information decisions are required.
In a specific implementation, a shielding object (an umbrella-shaped object, a riding tool object, a leg-shaped object, a skirt, a human face and other pedestrian targets are matched with a suspected pedestrian target, the information quantity of the suspected pedestrian target is increased, meanwhile, the matched position limits a matching area, the position corresponding to the shielding object is generally provided with a fixed position (such as a head of a cap corresponding to a pedestrian, an upper half of the umbrella corresponding to the pedestrian, a lower half of the skirt corresponding to the pedestrian and the like), the obtained similarity is multiplied by a corresponding weight, and then the obtained similarity is added with a confidence value of preliminary identification according to a comprehensive formula.
In specific implementation, this embodiment is further explained, for example: the front obstacle is identified as a face head portrait, has a head and a shoulder, is identified as a suspected pedestrian target, but there is no object in the figure that the lower body (leg object) can be matched, and the suspected pedestrian target is an advertisement portrait on a bus in front of an automatic driving vehicle in combination with the actual situation. For example: the suspected pedestrian target is successfully matched with the head and the shoulder, however, the lower body lacks legs, the suspected pedestrian target is found to be a person riding a motorcycle after being matched with the vehicle, and the suspected pedestrian target is judged to be a comprehensive pedestrian target.
Wherein, the comprehensive formula is:
b is a comprehensive value of the human body, a is a confidence value, l is a weight corresponding to the confidence value, B i For the corresponding similarity of the shielding object, l i Is the weight corresponding to the object of the shielding object. The shielding objects can be human body shape structures, head and shoulder models, umbrella-shaped objects, riding tool-shaped objects, leg-shaped objects, faces and the like, and i is the number or the type of the shielding objects.
The comprehensive value obtaining module 30 is further configured to determine, when the comprehensive value of the person is greater than or equal to a preset comprehensive value, a person target in a to-be-determined area corresponding to the comprehensive value of the person as a comprehensive pedestrian target.
A generating module 40, configured to generate a pedestrian recognition image according to the pedestrian target and the comprehensive pedestrian target.
It should be appreciated that since the pedestrian's leg is generally ground-contacting and vertical, the leg portion will appear in the lower half of the image, and detection is by scanning the lower half of the area to detect multiple leg targets, and then obtaining leg information via non-maximum suppression. And as the head-shoulder model is already obtained, the detection of the human face is carried out according to the Haar characteristics in the cut head-shoulder area, and the confidence is obtained. The invention is mainly aimed at detecting the side-looking riding tool object, and the riding tool object is characterized in that the riding tool object is positioned below a pedestrian area and two wheels which contact the ground are arranged at the same time.
The device further comprises an obstacle image processing module 50, which is used for acquiring an image to be identified and performing background differential processing on the image to be identified to acquire a preliminary identification image.
The method is particularly used for: and carrying out inter-frame difference processing on the image to be identified to obtain a background image. And carrying out differential operation on the background image to obtain a moving object image. And carrying out differential operation on the moving target image and the current image to be identified so as to obtain a preliminary identification image.
It should be noted that, background difference processing is performed by using Surendra algorithm, difference processing is performed on the image to be identified of the current frame and the image to be identified of the adjacent previous frame, and pixel values of the image to be identified after difference are compared with a preset threshold value to obtain a binarized image. The motion area of the binary image with the value of 1 is represented and is not updated; and the non-motion area with the value of 0 is updated according to the updating coefficient, and a stable background image can be obtained through iterative operation of multi-frame images.
It is easy to understand that the background image and the image to be identified of each frame are subjected to differential operation to obtain the moving object image.
It is easy to understand that the moving object image and the image to be identified of the current frame are differentiated to obtain a preliminary identification image.
The obstacle image processing module 50 is further configured to perform three-frame difference processing on the image to be identified, so as to obtain an edge difference map. The method is particularly used for: performing edge detection on a previous frame image of the image to be identified, a next frame image of the image to be identified and the image to be identified so as to obtain a corresponding edge image; and carrying out differential processing on the edge image to obtain a first differential graph and a second differential graph. And taking the first differential graph and the second differential graph as the edge differential graph.
It should be understood that the previous frame of the image to be identified, the next frame of the image to be identified and the image to be identified are continuous three frames of images, and the three frames of images are subjected to edge detection to obtain edge images corresponding to the three frames.
It should be understood that, performing differential operation on the edge images of the three frames by using the edge images of the adjacent frames to obtain two differential images, namely a first differential image and a second differential image, namely the edge differential images.
The obstacle image processing module 50 is further configured to obtain an obstacle image according to the preliminary identification map and the edge difference map. The method is particularly used for: and carrying out logical OR operation according to the preliminary identification graph and the first differential graph to obtain a mid-term identification graph. And performing logical AND operation according to the medium-term identification graph and the second differential graph to acquire a dynamic obstacle image. And detecting the dynamic obstacle according to the dynamic obstacle image.
It is easy to understand that the first differential diagram and the preliminary identification diagram are logically or, the operation result and the second differential diagram are logically and, and a dynamic obstacle result is detected.
The device of the embodiment carries out preliminary recognition on the obstacle region by matching the shape of the head and the shoulder and the human body morphological structure, obtains a preliminary recognized pedestrian target and a suspected pedestrian target, carries out characteristic matching on the interference objects which are easy to shield the human body, such as umbrella-shaped objects, riding tool-shaped objects and the like, and the suspected pedestrian target, screens out pedestrians from the suspected pedestrian target, eliminates non-pedestrians, and not only overcomes the shielding problem, but also enriches the characteristic combination method of the pedestrian characteristic information. The shielding problem is overcome, the recognition precision of the forward-looking pedestrian target is improved, and the safety of automatic driving is improved. And a more accurate dynamic obstacle image is obtained, a more stable foundation is provided for the identification of pedestrian targets, the advantages of a frame difference method and a background difference method are combined, and the problems of incomplete detection targets and omission of the background difference method of the frame difference method are solved.
In addition, the embodiment of the invention also provides a storage medium, wherein the storage medium stores a pedestrian recognition program, and the pedestrian recognition program is used for executing the steps of the pedestrian recognition method by a processor.
Because the storage medium adopts all the technical schemes of all the embodiments, the storage medium has at least all the beneficial effects brought by the technical schemes of the embodiments, and the description is omitted here.
It should be understood that the foregoing is illustrative only and is not limiting, and that in specific applications, those skilled in the art may set the invention as desired, and the invention is not limited thereto.
It should be noted that the above-described working procedure is merely illustrative, and does not limit the scope of the present invention, and in practical application, a person skilled in the art may select part or all of them according to actual needs to achieve the purpose of the embodiment, which is not limited herein.
In addition, technical details that are not described in detail in this embodiment may refer to the pedestrian recognition method provided in any embodiment of the present invention, and are not described herein again.
Furthermore, it should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.
The foregoing embodiment numbers of the present invention are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments.
From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. Read Only Memory)/RAM, magnetic disk, optical disk) and including several instructions for causing a terminal electronic device (which may be a mobile phone, a computer, a server, or a network electronic device, etc.) to perform the method according to the embodiments of the present invention.
The foregoing description is only of the preferred embodiments of the present invention, and is not intended to limit the scope of the invention, but rather is intended to cover any equivalents of the structures or equivalent processes disclosed herein or in the alternative, which may be employed directly or indirectly in other related arts.
Claims (7)
1. A method of pedestrian identification, the method comprising:
determining suspected pedestrian areas in the obstacle image, and acquiring confidence values of the suspected pedestrian areas;
taking a suspected pedestrian area with the confidence value in a first confidence value interval as a target area, and determining a pedestrian target in the target area;
taking the suspected pedestrian area with the confidence value in the second confidence value interval as a to-be-determined area, and respectively acquiring the character comprehensive value of the to-be-determined area according to a preset shelter model;
when the character comprehensive value is larger than or equal to a preset comprehensive value, judging the character target in the undetermined area corresponding to the character comprehensive value as a comprehensive pedestrian target;
generating a pedestrian recognition image according to the pedestrian target and the comprehensive pedestrian target;
the method for determining the suspected pedestrian areas in the obstacle image and obtaining the confidence value of each suspected pedestrian area comprises the following steps:
determining a pending pedestrian area in the obstacle image according to a pedestrian area formula;
performing region amplification on the to-be-determined pedestrian region to obtain a suspected pedestrian region;
wherein, the pedestrian area formula is:
the method comprises the steps of determining a region of a pedestrian to be determined, wherein the region is an area ratio of a region, lw is an area ratio of the region, S is an area of the region, smin is a preset minimum area of the region, smax is a preset maximum area of the region, lmin is a preset minimum aspect ratio of the region, lmax is a preset maximum aspect ratio of the region, spmax is an area ratio of a preset maximum segmentation region to an image display page, and Spmin is an area ratio of a preset minimum segmentation region to the image display page;
performing image matching on the suspected pedestrian areas according to a preset human body model to obtain confidence values of the suspected pedestrian areas;
the method for obtaining the person comprehensive value of the undetermined area according to the preset shelter model comprises the following steps of:
taking the suspected pedestrian area with the confidence value in the second confidence value interval as a pending area;
respectively acquiring the similarity of each shielding object target in the undetermined area according to a preset shielding object model;
adding the confidence value and the similarity according to a comprehensive formula to obtain a character comprehensive value of the undetermined area;
wherein, the comprehensive formula is:
wherein B is a comprehensive value of the person, a is a confidence value, l is a weight corresponding to the confidence value, bi is a similarity corresponding to the object of the shielding object, and li is a weight corresponding to the object of the shielding object.
2. The pedestrian recognition method according to claim 1, wherein before the step of determining suspected pedestrian areas in the obstacle image and acquiring confidence values for the respective suspected pedestrian areas, the method includes:
acquiring an image to be identified, and carrying out background differential processing on the image to be identified to acquire a preliminary identification image;
performing three-frame difference processing on the image to be identified to obtain an edge difference image;
and obtaining an obstacle image according to the preliminary identification image and the edge difference image.
3. The pedestrian recognition method according to claim 1, wherein the step of performing image matching on the suspected pedestrian areas according to a preset human body model to obtain confidence values of the suspected pedestrian areas comprises the following steps:
determining a human body position area of a suspected pedestrian target in the suspected pedestrian area;
and carrying out image matching on the human body position areas through a preset human body model so as to obtain confidence values of the suspected pedestrian areas.
4. The pedestrian recognition method according to claim 1, wherein the step of obtaining the similarity of each occlusion object in the undetermined area according to a preset occlusion model, comprises the following steps:
respectively acquiring a shade position area corresponding to each shade target in the undetermined area according to a preset shade model;
and carrying out image matching on the position area of the shielding object through the preset shielding object model so as to obtain the similarity corresponding to each shielding object target.
5. A pedestrian recognition device, the device comprising:
the confidence value acquisition module is used for determining suspected pedestrian areas in the obstacle image and acquiring confidence values of the suspected pedestrian areas;
the determining module is used for taking a suspected pedestrian area with the confidence value in the first confidence value interval as a target area and determining a pedestrian target in the target area;
the comprehensive value acquisition module is used for taking a suspected pedestrian area with the confidence value in the second confidence value interval as a to-be-determined area and respectively acquiring the character comprehensive value of the to-be-determined area according to a preset shelter model; and the method is also used for judging the character target in the undetermined area corresponding to the character comprehensive value as a comprehensive pedestrian target when the character comprehensive value is larger than or equal to a preset comprehensive value;
the generation module is used for generating a pedestrian identification image according to the pedestrian target and the comprehensive pedestrian target;
the confidence value acquisition module is further used for determining a pending pedestrian area in the obstacle image according to a pedestrian area formula;
performing region amplification on the to-be-determined pedestrian region to obtain a suspected pedestrian region;
wherein, the pedestrian area formula is:
the method comprises the steps of determining a region of a pedestrian to be determined, wherein the region is an area ratio of a region, lw is an area ratio of the region, S is an area of the region, smin is a preset minimum area of the region, smax is a preset maximum area of the region, lmin is a preset minimum aspect ratio of the region, lmax is a preset maximum aspect ratio of the region, spmax is an area ratio of a preset maximum segmentation region to an image display page, and Spmin is an area ratio of a preset minimum segmentation region to the image display page;
performing image matching on the suspected pedestrian areas according to a preset human body model to obtain confidence values of the suspected pedestrian areas;
the comprehensive value acquisition module is further used for taking a suspected pedestrian area with the confidence value in the second confidence value interval as a pending area;
respectively acquiring the similarity of each shielding object target in the undetermined area according to a preset shielding object model;
adding the confidence value and the similarity according to a comprehensive formula to obtain a character comprehensive value of the undetermined area;
wherein, the comprehensive formula is:
wherein B is a comprehensive value of the person, a is a confidence value, l is a weight corresponding to the confidence value, bi is a similarity corresponding to the object of the shielding object, and li is a weight corresponding to the object of the shielding object.
6. An electronic device, the electronic device comprising: a memory, a processor and a pedestrian recognition program stored on the memory and executable on the processor, the pedestrian recognition program configured to implement the steps of the pedestrian recognition method of any one of claims 1 to 4.
7. A storage medium having stored thereon a pedestrian recognition program which, when executed by a processor, implements the steps of the pedestrian recognition method of any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010424166.1A CN111626165B (en) | 2020-05-15 | 2020-05-15 | Pedestrian recognition method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010424166.1A CN111626165B (en) | 2020-05-15 | 2020-05-15 | Pedestrian recognition method and device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111626165A CN111626165A (en) | 2020-09-04 |
| CN111626165B true CN111626165B (en) | 2024-02-02 |
Family
ID=72259865
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010424166.1A Active CN111626165B (en) | 2020-05-15 | 2020-05-15 | Pedestrian recognition method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111626165B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112329724B (en) * | 2020-11-26 | 2022-08-05 | 四川大学 | Real-time detection and snapshot method for lane change of motor vehicle |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105844232A (en) * | 2016-03-21 | 2016-08-10 | 联想(北京)有限公司 | Image identification method and device |
| JP2017117244A (en) * | 2015-12-24 | 2017-06-29 | Kddi株式会社 | Image analysis device for detecting a person displayed in photographic image of camera |
| WO2018223955A1 (en) * | 2017-06-09 | 2018-12-13 | 北京深瞐科技有限公司 | Target monitoring method, target monitoring device, camera and computer readable medium |
| CN110781821A (en) * | 2019-10-25 | 2020-02-11 | 上海商汤智能科技有限公司 | Target detection method and device based on unmanned aerial vehicle, electronic equipment and storage medium |
-
2020
- 2020-05-15 CN CN202010424166.1A patent/CN111626165B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2017117244A (en) * | 2015-12-24 | 2017-06-29 | Kddi株式会社 | Image analysis device for detecting a person displayed in photographic image of camera |
| CN105844232A (en) * | 2016-03-21 | 2016-08-10 | 联想(北京)有限公司 | Image identification method and device |
| WO2018223955A1 (en) * | 2017-06-09 | 2018-12-13 | 北京深瞐科技有限公司 | Target monitoring method, target monitoring device, camera and computer readable medium |
| CN110781821A (en) * | 2019-10-25 | 2020-02-11 | 上海商汤智能科技有限公司 | Target detection method and device based on unmanned aerial vehicle, electronic equipment and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 曹丽武 ; .视频图像序列中人体目标的提取与跟踪.计算机与数字工程.2011,(03),全文. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111626165A (en) | 2020-09-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11450146B2 (en) | Gesture recognition method, apparatus, and device | |
| JP4203512B2 (en) | Vehicle periphery monitoring device | |
| Zhang et al. | Image segmentation based on 2D Otsu method with histogram analysis | |
| JP5867596B2 (en) | Three-dimensional object detection apparatus and three-dimensional object detection method | |
| US9092868B2 (en) | Apparatus for detecting object from image and method therefor | |
| JP5223675B2 (en) | Vehicle detection device, vehicle detection method, and vehicle detection program | |
| CN101896920A (en) | Image processing method and device based on motion scan | |
| Ray et al. | Dynamic blindspots measurement for construction equipment operators | |
| KR20150119736A (en) | System for detecting obstacle using a road surface model setting and method thereof | |
| EP2821935B1 (en) | Vehicle detection method and device | |
| KR20140148171A (en) | Lane Detection method for Advanced Vehicle | |
| CN111626165B (en) | Pedestrian recognition method and device, electronic equipment and storage medium | |
| JP2010262576A (en) | Subject detecting apparatus and program | |
| CN108629225B (en) | Vehicle detection method based on multiple sub-images and image significance analysis | |
| JP2011165170A (en) | Object detection device and program | |
| Rotaru et al. | Extracting road features from color images using a cognitive approach | |
| CN114639159A (en) | Moving pedestrian detection method, electronic device and robot | |
| US20240071104A1 (en) | Image processing device, image processing method, and recording medium | |
| KR102283053B1 (en) | Real-Time Multi-Class Multi-Object Tracking Method Using Image Based Object Detection Information | |
| CN114529715A (en) | Image identification method and system based on edge extraction | |
| KR20180071552A (en) | Lane Detection Method and System for Camera-based Road Curvature Estimation | |
| KR101895678B1 (en) | Efficient Search Window Set-Up Method for the Automotive Image Recognition System | |
| CN116612279B (en) | Method, device, network equipment and storage medium for target detection | |
| Rosebrock et al. | Using the shadow as a single feature for real-time monocular vehicle pose determination | |
| JP3216356B2 (en) | License plate position detector |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |