CN112434671B - Pedestrian snapshot optimization method and device, computer equipment and storage medium - Google Patents
Pedestrian snapshot optimization method and device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN112434671B CN112434671B CN202011503821.9A CN202011503821A CN112434671B CN 112434671 B CN112434671 B CN 112434671B CN 202011503821 A CN202011503821 A CN 202011503821A CN 112434671 B CN112434671 B CN 112434671B
- Authority
- CN
- China
- Prior art keywords
- snapshot
- snapshots
- pedestrian
- target object
- sets
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/55—Clustering; Classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
Abstract
The invention discloses a pedestrian snapshot optimization method, a pedestrian snapshot optimization device, computer equipment and a storage medium. Detecting a target object in a video, and capturing snapshots of the target object in different video frames; detecting key point information in each snapshot, screening the snapshots with complete postures according to the key point information, confirming the shooting angles of target objects in the snapshots, and classifying each snapshot according to the shooting angles to obtain multiple groups of snapshot sets classified in different angles; and then comparing the confidence degrees and the interception times of all the snapshots in each group of snapshot sets, selecting the snapshots with the confidence degrees from high to low, sending the snapshots into a pedestrian database until the number of the snapshots sent into the pedestrian database in each group of snapshot sets reaches the preset number requirement, wherein the difference value of the interception times of adjacent snapshots in the selected snapshots is also larger than the preset time interval. The invention provides the pedestrian snapshot optimization strategy, and the selected pedestrian snapshot has the advantage of high identification accuracy.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to a pedestrian snapshot optimization method, a pedestrian snapshot optimization device, computer equipment and a storage medium.
Background
The pedestrian snapshot in the video refers to a pedestrian image which is recorded by a camera in a real scene and collected in the video, and the image can be used for computer vision tasks such as pedestrian recognition. In a real scene, the cameras are arranged at different angles, different heights and the like, the shooting environment is not controllable, the pedestrian images obtained by the cameras are large in change, and the image quality is uneven, so that most of pedestrian recognition tasks need to find local unchangeable features including clothes colors, clothes textures, handbags, backpacks and the like without being judged by the whole pedestrian.
The existing method for selecting the pedestrian snapshot from the video is mainly as follows: through the target detection network, the pedestrian is discerned in the video to use and detect the frame and carry out the frame with this pedestrian and get, then judge the image input pedestrian recognition network that the frame was taken out, however based on the variability of reality scene and the accuracy of target detection network, the pedestrian image that the camera of different angles obtained often appears the quality problem, and the main probably condition of occurrence has:
(1) the target detection network mistakenly identifies other objects as pedestrians, including fire hydrants, nameplates and the like;
(2) the detected pedestrian is seriously shielded;
(3) the detected pedestrian snapshot comprises more than two pedestrians;
(4) the condition of leg lack, head lack and the like can occur due to incomplete pedestrian images caused by instability of the detected pedestrian surrounding frame;
the pedestrian snapshots under the above conditions lose a large amount of global and local information of pedestrians, which brings great difficulty to pedestrian identification, and in the existing documents disclosing pedestrian identification, only a judgment method for pedestrian identification is explained, and how to select pedestrian snapshots under different cameras for pedestrian identification is not considered.
Therefore, the existing method for selecting the pedestrian snapshot in the video has the following limitations: the pedestrian snapshot optimization strategy cannot be given, and the pedestrian snapshots acquired by the cameras at different angles are directly adopted and can only be applied to scenes with single environment; for the scenes such as cities, public places and the like with quite variable and complicated scenes, the pedestrian identification accuracy and the identification effect can be greatly reduced.
Disclosure of Invention
The invention aims to provide a pedestrian snapshot optimizing method, a pedestrian snapshot optimizing device, computer equipment and a storage medium, and aims to solve the problem that pedestrian snapshots under different cameras cannot be selected aiming at improving pedestrian recognition rate.
In a first aspect, an embodiment of the present invention provides a pedestrian snapshot optimizing method, which includes:
detecting a target object in a video, and capturing snapshots of the target object in different video frames;
detecting key point information in each snapshot, and screening out the snapshots with complete postures according to the key point information;
confirming the shooting angle of the target object in each screened snapshot, and classifying each screened snapshot according to the shooting angle to obtain a plurality of groups of snapshot sets with different angle classifications, wherein one group of snapshot sets corresponds to one angle classification;
and comparing the confidence degrees and the interception time of all the snapshots in each group of snapshot sets, wherein the confidence degree indicates the probability value of whether the snapshots are the target object, the snapshots are sequentially selected from high to low according to the confidence degree and sent into the pedestrian database, and the interval of the interception time of the snapshots sequentially sent into the pedestrian database is greater than the preset time interval until the number of the snapshots sent into the pedestrian database in each group of snapshot sets reaches the preset number requirement.
In a second aspect, an embodiment of the present invention provides a pedestrian snapshot optimizing apparatus, including:
the capturing unit is used for detecting a target object in the video and capturing snapshots of the target object in different video frames;
the screening unit is used for detecting key point information in each snapshot and screening the snapshots with complete postures according to the key point information;
the classification unit is used for confirming the shooting angle of the target object in each screened snapshot and classifying each screened snapshot according to the shooting angle to obtain a plurality of groups of snapshot sets with different angle classifications, wherein one group of snapshot sets corresponds to one angle classification;
and the selection unit is used for comparing the confidence degrees and the interception time of all the snapshots in each group of snapshot sets, wherein the confidence degrees indicate whether the probability values are the probability values of the target object, the snapshots are sequentially selected from high confidence degrees to low confidence degrees and sent into the pedestrian database, and the interval of the interception time of the snapshots sequentially sent into the pedestrian database is greater than the preset time interval until the number of the snapshots sent into the pedestrian database in each group of snapshot sets reaches the preset number requirement.
In a third aspect, an embodiment of the present invention further provides a computer device, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor, when executing the computer program, implements the pedestrian snapshot preference method described in the first aspect.
In a fourth aspect, the embodiment of the present invention further provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and the computer program, when executed by a processor, causes the processor to execute the pedestrian snapshot preference method according to the first aspect.
The embodiment of the invention discloses a pedestrian snapshot optimization method, a pedestrian snapshot optimization device, computer equipment and a storage medium. Detecting a target object in a video, and capturing snapshots of the target object in different video frames; detecting key point information in each snapshot, screening the snapshots with complete postures according to the key point information, confirming the shooting angles of target objects in the snapshots, and classifying each snapshot according to the shooting angles to obtain multiple groups of snapshot sets classified in different angles; and then comparing the confidence degrees and the interception times of all the snapshots in each group of snapshot sets, selecting the snapshots with the confidence degrees from high to low, sending the snapshots into a pedestrian database until the number of the snapshots sent into the pedestrian database in each group of snapshot sets reaches the preset number requirement, wherein the difference value of the interception times of adjacent snapshots in the selected snapshots is also larger than the preset time interval. The embodiment of the invention provides an optimal strategy for the pedestrian snapshot, and the selected pedestrian snapshot has the advantage of high identification accuracy.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a schematic flow chart of a pedestrian snapshot optimization method according to an embodiment of the present invention;
fig. 2 is a sub-flow diagram of a pedestrian snapshot optimization method according to an embodiment of the present invention;
fig. 3 is a schematic sub-flow chart of a pedestrian snapshot optimization method according to an embodiment of the present invention;
FIG. 4 is a schematic view of another sub-flow chart of a pedestrian snapshot optimization method according to an embodiment of the present invention;
FIG. 5 is a schematic view of another sub-flow chart of a pedestrian snapshot optimization method according to an embodiment of the present invention;
FIG. 6 is a schematic diagram of a preferred apparatus for snapshot of pedestrians according to an embodiment of the present invention;
fig. 7 is a schematic diagram of a computer device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It will be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It is also to be understood that the terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in the specification of the present invention and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
It should be further understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.
Referring to fig. 1, fig. 1 is a schematic flow chart of a pedestrian snapshot optimization method according to an embodiment of the present invention;
as shown in fig. 1, the method includes steps S101 to S104.
S101, detecting a target object in a video, and capturing snapshots of the target object in different video frames.
In this embodiment, a plurality of videos are acquired based on cameras at different angles in a real scene, the videos are detected, a target object (namely, a pedestrian) in the videos is captured, snapshots of the target object in different video frames are captured, snapshots of the target object at various angles in the videos can be obtained, a large number of snapshots are collected for subsequent screening, and accuracy of identifying the target object is improved based on the screened snapshots.
In one embodiment, as shown in fig. 2, the step S101 includes:
s201, detecting a target object in a video through a target detection network, capturing snapshots of the target object in different video frames, and recording capturing time;
s202, calculating the confidence of each snapshot.
In this embodiment, the target detection network is an identification network, and can detect and track an object in a video, where the object may be a person or an article; finding the target object in a video through the target detection network, tracking, capturing snapshots of the target object at different moments in the tracking process, and recording capturing time of each snapshot; and detecting the confidence coefficient of each snapshot through the target detection network, wherein the confidence coefficient represents the probability value of whether the snapshot is a target object, and the higher the confidence coefficient is, the higher the probability of judging the snapshot is. However, since the target object is continuously walking in the video, the target object may be complete or occluded on the screen at different times, so that the confidence of each snapshot changes, and the confidence cannot represent the quality of the snapshot, the snapshot with high confidence may also have an occlusion problem, and the snapshot with low confidence may also have a complete target object.
S102, detecting key point information in each snapshot, and screening the snapshots with complete postures according to the key point information.
The embodiment is a process of screening snapshots with complete postures, that is, a process of screening snapshots which only exist in the target object and have a substantially visible body of the target object, and specifically, the snapshots with complete postures can be screened by detecting key point information in each snapshot and according to the detected key point information.
In one embodiment, as shown in fig. 3, the step S102 includes:
s301, carrying out human body posture estimation detection on each snapshot to obtain the number of objects and key point information in each snapshot, wherein the key point information comprises the number and the type of key points;
s302, if the number of the objects is 2 or more than 2, removing corresponding snapshots;
s303, if the number of the objects is 1 and the number of the key points is less than a preset number threshold, or if the number of the objects is 1 and key points of important types are missing, removing the corresponding snapshots;
s304, if the number of the objects is 1 and the number of the key points is larger than or equal to a preset number threshold, or the number of the objects is 1 and key points of important types are not missed, keeping the corresponding snapshots.
In this embodiment, the collected snapshots are screened; specifically, the collected snapshots are detected through a human body posture estimation network, and key point information of objects appearing in the snapshots is obtained; the human body posture estimation network is a neural network for estimating the posture position of the human body, and after the snapshot is input into the human body posture estimation network, the specific body part positions and the specific part numbers of the head position, the elbow position and the like of the target object in the snapshot can be output, and the output part positions and the part numbers are the key point information. Based on the actual scene, there may be a plurality of objects or incomplete postures of the objects in the snapshot, so the screening may be performed by detecting the number of the objects in the snapshot and key point information, where the key point information is each part of the body of the object, specifically, 25 key points of a nose, a neck, a left ear, a right ear, a left eye, a right shoulder, a left hip, a middle hip, a right elbow, a left elbow, a right wrist, a left wrist, a right hip, a left hip, a right knee, a left knee, a right ankle, a left heel, a right heel, a left thumb, and a right thumb, and the key points belonging to the head, the elbow, and the leg belong to important kinds of key points.
Specifically, the screening can be divided into three types: the first is that 2 or more than 2 objects are detected to appear in the snapshot, and each object has respective key point information; the second method is to detect that only 1 object appears in the snapshot but the body of the object is occluded and the number of the key points is less than a preset number threshold, or only 1 object appears but the body of the object is occluded and the object lacks key points of important types, where the preset number threshold may be 25 or other values lower than 25, and a larger preset number threshold indicates that the snapshot is more strictly screened; and the third method is that only 1 object is detected to appear in the snapshot, and the number of the key points of the object is greater than or equal to a preset number threshold, or only 1 object appears but the object does not lack key points of important categories. Based on the three conditions, the first and the second cases are removed, and the snapshot corresponding to the third case is reserved.
S103, confirming the shooting angle of the target object in each selected snapshot, and classifying each selected snapshot according to the shooting angle to obtain multiple groups of snapshot sets with different angle classifications, wherein one group of snapshot sets corresponds to one angle classification.
In this embodiment, the screened snapshot with the complete posture is input to the pedestrian attribute classification network, the pedestrian attribute classification network is a classification convolutional neural network, a snapshot with the complete posture is input, and the shooting angle of the target object in the snapshot can be output; thus, according to the shooting angle of the target object in each snapshot, a plurality of groups of snapshot sets with different angle classifications can be obtained, and each group of snapshot set corresponds to one angle classification.
In one embodiment, as shown in fig. 4, the step S103 includes:
s401, inputting each screened snapshot into a pedestrian attribute classification network, and outputting a shooting angle of each snapshot;
s402, classifying the shooting angles of all the snapshots to obtain a front posture snapshot set, a side posture snapshot set and a back posture snapshot set.
In the embodiment, each selected snapshot is input into a pedestrian attribute classification network, and the shooting angle of each snapshot is output, wherein the shooting angle comprises a front shooting angle, a side shooting angle and a back shooting angle; thus, according to the shooting angle of the target object in each snapshot, a front posture snapshot set, a side posture snapshot set and a back posture snapshot set can be obtained.
S104, comparing the confidence degrees and the interception time of all the snapshots in each group of snapshot sets, wherein the confidence degree indicates whether the probability value is the probability value of the target object, selecting the snapshots in sequence from high to low according to the confidence degree and sending the snapshots into the pedestrian database, and the interval of the interception time of the snapshots which are sent into the pedestrian database in sequence is greater than the preset time interval until the number of the snapshots which are sent into the pedestrian database in each group of snapshot sets reaches the preset number requirement.
In this embodiment, the snapshots in each group of snapshot sets obtained by screening already have a higher identification accuracy, but in order to further improve the identification accuracy, the snapshots in each group of snapshot sets need to be screened again and then sent into the pedestrian database, and specifically, screening can be performed according to the confidence degrees and the interception times of all the snapshots in each group of snapshot sets, which can be sequentially selected from high to low according to the confidence degrees, and the interval of the interception times between the selected snapshots needs to be greater than a preset time interval, so that the purpose of screening is to ensure that the selected snapshots can be dispersed over a time span, and thus, the selected snapshots are high-quality snapshots, and the identification efficiency and accuracy of the target object can be improved.
And screening the snapshots in each group of snapshot sets again until the number of the snapshots sent into the pedestrian database in each group of snapshot sets reaches a preset number requirement, wherein the preset number requirement can be 3, namely that at least 3 snapshots need to be sent into the pedestrian database in each group of snapshot sets.
In one embodiment, as shown in fig. 5, the step S104 includes:
s501, comparing the confidence degrees and the interception times of all snapshots in each group of snapshot sets, and sequentially selecting the snapshots from high to low according to the confidence degrees;
s502, judging whether the difference value between the interception time of the currently selected snapshot and the interception time of the last selected snapshot is larger than a preset time interval or not;
and S503, if so, sending the currently selected snapshot into the pedestrian database, and continuously selecting the next snapshot until the number of snapshots collectively sent into the pedestrian database in each group of snapshots reaches the preset number requirement.
In this embodiment, a specific selection process of snapshots in each group of snapshot sets is as follows: sorting the snapshots in each group of snapshot sets from high to low according to the confidence degrees, firstly selecting the snapshot with the highest confidence degree to be sent to the pedestrian database, then selecting the next highest confidence degree snapshot according to the sorting, calculating the difference value between the interception time of the next highest snapshot and the interception time of the last selected snapshot, if the difference value is greater than the preset time interval, sending the currently selected next highest snapshot to the pedestrian database, and then continuously selecting the next snapshot; and if the difference value between the interception time of the currently selected next-highest snapshot and the interception time of the last selected snapshot is less than or equal to the preset time interval, directly skipping the currently selected next-highest-confidence-degree snapshot, and selecting the next snapshot according to the sequence for judgment until the number of the snapshots collectively sent into the pedestrian database in each group of snapshots reaches 3.
In one embodiment, the step S503 includes:
and aiming at each group of snapshot sets, when all snapshots are selected and the number of the snapshots which are sent into the pedestrian database in the corresponding snapshot set still does not meet the requirement of the preset number, continuously capturing the snapshots of the target object in the video and screening out new snapshots with complete postures.
In this embodiment, after all snapshots in each group of snapshot sets are selected, if all the snapshots are selected and the number of snapshots which are sent into the pedestrian database in the corresponding snapshot set does not meet the requirement of the preset number, snapshot supplementation needs to be performed on the corresponding snapshot set; specifically, the video is continuously detected again through the target detection network, snapshots of the target object in different video frames are captured, and new snapshots with complete gestures are screened out; and then, classifying the shooting angles of the screened new snapshots through a pedestrian attribute classification network so as to supplement the new snapshots into corresponding snapshot sets, then selecting the new snapshots in the snapshot sets, and sending the selected new snapshots into the snapshots of the pedestrian database until the number of the snapshots sent into the pedestrian database by the corresponding snapshot sets reaches the preset number requirement.
In an embodiment, the step S104 further includes:
and judging whether the difference value between the interception time of the currently selected snapshot and the interception time of the last selected snapshot is smaller than or equal to a preset time interval, and if so, removing the currently selected snapshot from the corresponding snapshot set.
In this embodiment, if the difference between the capturing time of the currently selected snapshot and the capturing time of the last selected snapshot is less than or equal to the preset time interval, it indicates that the previously selected snapshot does not meet the requirement of being sent into the pedestrian database, and therefore, the previously selected snapshot does not need to be retained, and the previously selected snapshot is directly removed from the corresponding snapshot set; meanwhile, the method also aims to prevent confusion when a new snapshot is added to the snapshot set subsequently.
The embodiment of the invention also provides a pedestrian snapshot optimizing device, which is used for executing any embodiment of the pedestrian snapshot optimizing method. Specifically, referring to fig. 6, fig. 6 is a schematic block diagram of a pedestrian snapshot optimizing apparatus according to an embodiment of the present invention.
As shown in fig. 6, the pedestrian snapshot preference apparatus 600 includes: an intercepting unit 601, a screening unit 602, a classifying unit 603 and a selecting unit 604.
An intercepting unit 601, configured to detect a target object in a video, and intercept snapshots of the target object in different video frames;
the screening unit 602 is configured to detect key point information in each snapshot, and screen out snapshots with complete gestures according to the key point information;
a classifying unit 603, configured to determine a shooting angle of the target object in each screened snapshot, and classify each screened snapshot according to the shooting angle to obtain multiple groups of snapshot sets with different angle classifications, where one group of snapshot sets corresponds to one angle classification;
the selecting unit 604 is configured to compare confidence levels and interception times of all snapshots in each group of snapshot sets, where the confidence level indicates a probability value of whether the snapshots are target objects, select the snapshots in sequence from high to low according to the confidence levels and send the snapshots into the pedestrian database, and an interval of the interception times of the snapshots sequentially sent into the pedestrian database is greater than a preset time interval until the number of the snapshots sent into the pedestrian database in each group of snapshot sets reaches a preset number requirement.
The device adopts the deep learning technology to carry out snapshot filtering when selecting the snapshots in the video, can obtain high-quality pedestrian snapshots, and can effectively improve the pedestrian identification efficiency and accuracy in complex scenes such as cities and public places.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
The pedestrian snapshot preference means may be implemented in the form of a computer program that can be run on a computer device as shown in fig. 7.
Referring to fig. 7, fig. 7 is a schematic block diagram of a computer device according to an embodiment of the present invention. The computer device 700 is a server, which may be an independent server or a server cluster composed of a plurality of servers.
Referring to fig. 7, the computer device 700 includes a processor 702, memory, and a network interface 705 coupled via a system bus 701, where the memory may include a non-volatile storage medium 703 and an internal memory 704.
The non-volatile storage medium 703 may store an operating system 7031 and a computer program 7032. The computer program 7032, when executed, causes the processor 702 to perform a pedestrian snapshot preference method.
The processor 702 is configured to provide computing and control capabilities to support the operation of the overall computing device 700.
The internal memory 704 provides an environment for running a computer program 7032 on the non-volatile storage medium 703, and the computer program 7032, when executed by the processor 702, causes the processor 702 to perform the pedestrian snapshot preference method.
The network interface 705 is used for network communication, such as providing transmission of data information. Those skilled in the art will appreciate that the configuration shown in fig. 7 is a block diagram of only a portion of the configuration associated with aspects of the present invention and is not intended to limit the computing device 700 to which aspects of the present invention may be applied, and that a particular computing device 700 may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
Those skilled in the art will appreciate that the embodiment of a computer device illustrated in fig. 7 does not constitute a limitation on the specific construction of the computer device, and that in other embodiments a computer device may include more or fewer components than those illustrated, or some components may be combined, or a different arrangement of components. For example, in some embodiments, the computer device may only include a memory and a processor, and in such embodiments, the structures and functions of the memory and the processor are consistent with those of the embodiment shown in fig. 7, and are not described herein again.
It should be appreciated that, in embodiments of the present invention, the Processor 702 may be a Central Processing Unit (CPU), and the Processor 702 may also be other general purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Wherein a general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
In another embodiment of the invention, a computer-readable storage medium is provided. The computer readable storage medium may be a non-volatile computer readable storage medium. The computer readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the pedestrian snapshot preference method of an embodiment of the present invention.
The storage medium is an entity and non-transitory storage medium, and may be various entity storage media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-only memory (ROM), a magnetic disk, or an optical disk.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described apparatuses, devices and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
While the invention has been described with reference to specific embodiments, the invention is not limited thereto, and various equivalent modifications and substitutions can be easily made by those skilled in the art within the technical scope of the invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (10)
1. A pedestrian snapshot preference method, comprising:
detecting a target object in a video, and capturing snapshots of the target object in different video frames;
detecting key point information in each snapshot, and screening out the snapshots with complete postures according to the key point information;
confirming the shooting angle of the target object in each screened snapshot, and classifying each screened snapshot according to the shooting angle to obtain a plurality of groups of snapshot sets with different angle classifications, wherein one group of snapshot sets corresponds to one angle classification;
and comparing the confidence degrees and the interception time of all the snapshots in each group of snapshot sets, wherein the confidence degree indicates the probability value of whether the snapshots are the target object, the snapshots are sequentially selected from high to low according to the confidence degree and sent into the pedestrian database, and the interval of the interception time of the snapshots sequentially sent into the pedestrian database is greater than the preset time interval until the number of the snapshots sent into the pedestrian database in each group of snapshot sets reaches the preset number requirement.
2. The pedestrian snapshot optimizing method according to claim 1, wherein the detecting a target object in a video, and capturing snapshots of the target object in different video frames comprises:
detecting a target object in a video through a target detection network, capturing snapshots of the target object in different video frames and recording capturing time;
the confidence level of each snapshot is calculated.
3. The pedestrian snapshot optimizing method according to claim 1, wherein the detecting key point information in each snapshot and screening out the snapshot with complete posture according to the key point information comprises:
carrying out human body posture estimation detection on each snapshot to obtain the number of objects and key point information in each snapshot, wherein the key point information comprises the number and the type of key points;
if the number of the objects is 2 or more than 2, removing the corresponding snapshots;
if the number of the objects is 1 and the number of the key points is less than a preset number threshold, or if the number of the objects is 1 and key points of important types are missing, removing the corresponding snapshots;
if the number of the objects is 1 and the number of the key points is greater than or equal to a preset number threshold, or if the number of the objects is 1 and key points of important types are not missing, the corresponding snapshot is retained.
4. The pedestrian snapshot optimizing method according to claim 1, wherein the determining a shooting angle of the target object in each of the screened snapshots and classifying each of the screened snapshots according to the shooting angle to obtain a plurality of groups of snapshot sets with different angle classifications comprises:
inputting each screened snapshot into a pedestrian attribute classification network, and outputting the shooting angle of each snapshot;
and classifying the shooting angles of all the snapshots to obtain a front posture snapshot set, a side posture snapshot set and a back posture snapshot set.
5. The pedestrian snapshot optimizing method according to claim 1, wherein the comparing of the confidence degrees and the capturing times of all the snapshots in each group of snapshot sets, the selecting of the snapshots sequentially from high to low according to the confidence degrees and the sending of the snapshots into the pedestrian database, and the capturing time intervals of the snapshots sequentially sent into the pedestrian database being greater than a preset time interval until the number of the snapshots sent into the pedestrian database in each group of snapshot sets reaches a preset number requirement, comprises:
comparing the confidence degrees and the interception times of all the snapshots in each group of snapshot sets, and selecting the snapshots in sequence from high to low according to the confidence degrees;
judging whether the difference value between the interception time of the currently selected snapshot and the interception time of the last selected snapshot is greater than a preset time interval or not;
and if so, sending the currently selected snapshot into the pedestrian database, and continuously selecting the next snapshot until the number of snapshots collectively sent into the pedestrian database in each group of snapshots reaches the preset number requirement.
6. The pedestrian snapshot optimizing method according to claim 5, wherein if yes, sending the currently selected snapshot into the pedestrian database, and continuing to select the next snapshot until the number of snapshots sent into the pedestrian database in each group of snapshot sets reaches a preset number requirement, includes:
and aiming at each group of snapshot sets, when all snapshots are selected and the number of the snapshots which are sent into the pedestrian database in the corresponding snapshot set still does not meet the requirement of the preset number, continuously capturing the snapshots of the target object in the video and screening out new snapshots with complete postures.
7. The pedestrian snapshot optimizing method according to claim 5, wherein the comparing of the confidence degrees and the capturing times of all the snapshots in each group of snapshot sets is performed, the snapshots are sequentially selected from high to low according to the confidence degrees and sent to the pedestrian database, and the capturing time intervals of the snapshots sequentially sent to the pedestrian database are greater than a preset time interval until the number of the snapshots sent to the pedestrian database in each group of snapshot sets reaches a preset number requirement, further comprising:
and judging whether the difference value between the interception time of the currently selected snapshot and the interception time of the last selected snapshot is smaller than or equal to a preset time interval, and if so, removing the currently selected snapshot from the corresponding snapshot set.
8. A pedestrian snapshot preference apparatus, comprising:
the capturing unit is used for detecting a target object in the video and capturing snapshots of the target object in different video frames;
the screening unit is used for detecting key point information in each snapshot and screening the snapshots with complete postures according to the key point information;
the classification unit is used for confirming the shooting angle of the target object in each screened snapshot and classifying each screened snapshot according to the shooting angle to obtain a plurality of groups of snapshot sets with different angle classifications, wherein one group of snapshot sets corresponds to one angle classification;
and the selection unit is used for comparing the confidence degrees and the interception time of all the snapshots in each group of snapshot sets, wherein the confidence degrees indicate whether the probability values are the probability values of the target object, the snapshots are sequentially selected from high confidence degrees to low confidence degrees and sent into the pedestrian database, and the interval of the interception time of the snapshots sequentially sent into the pedestrian database is greater than the preset time interval until the number of the snapshots sent into the pedestrian database in each group of snapshot sets reaches the preset number requirement.
9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the pedestrian snapshot preference method according to any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program which, when executed by a processor, causes the processor to perform the pedestrian snapshot preference method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011503821.9A CN112434671B (en) | 2020-12-18 | 2020-12-18 | Pedestrian snapshot optimization method and device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011503821.9A CN112434671B (en) | 2020-12-18 | 2020-12-18 | Pedestrian snapshot optimization method and device, computer equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112434671A CN112434671A (en) | 2021-03-02 |
| CN112434671B true CN112434671B (en) | 2021-08-06 |
Family
ID=74696711
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011503821.9A Active CN112434671B (en) | 2020-12-18 | 2020-12-18 | Pedestrian snapshot optimization method and device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112434671B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP3547215A1 (en) * | 2018-03-26 | 2019-10-02 | Cohda Wireless Pty Ltd. | Systems and methods for automatically training neural networks |
| CN110598540A (en) * | 2019-08-05 | 2019-12-20 | 华中科技大学 | Method and system for extracting gait contour map in monitoring video |
| CN110609920A (en) * | 2019-08-05 | 2019-12-24 | 华中科技大学 | Pedestrian hybrid search method and system in video monitoring scene |
| EP3588136A1 (en) * | 2018-06-28 | 2020-01-01 | Veoneer Sweden AB | Object representation and classification based on vehicle sensor detections |
| CN111062239A (en) * | 2019-10-15 | 2020-04-24 | 平安科技（深圳）有限公司 | Human body target detection method and device, computer equipment and storage medium |
-
2020
- 2020-12-18 CN CN202011503821.9A patent/CN112434671B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP3547215A1 (en) * | 2018-03-26 | 2019-10-02 | Cohda Wireless Pty Ltd. | Systems and methods for automatically training neural networks |
| EP3588136A1 (en) * | 2018-06-28 | 2020-01-01 | Veoneer Sweden AB | Object representation and classification based on vehicle sensor detections |
| CN110598540A (en) * | 2019-08-05 | 2019-12-20 | 华中科技大学 | Method and system for extracting gait contour map in monitoring video |
| CN110609920A (en) * | 2019-08-05 | 2019-12-24 | 华中科技大学 | Pedestrian hybrid search method and system in video monitoring scene |
| CN111062239A (en) * | 2019-10-15 | 2020-04-24 | 平安科技（深圳）有限公司 | Human body target detection method and device, computer equipment and storage medium |
Non-Patent Citations (2)
| Title |
|---|
| "Modelling Unidirectional Crowd Motion in a Corridor with Statistical Characteristics of Pedestrian Movements";Chen T;《Mathematical Problems in Engineering》;20200630;第1-11页 * |
| "视频摘要技术在监控系统中的应用";陈想;《万方》;20170630;第1-78页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112434671A (en) | 2021-03-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| AU2022252799B2 (en) | System and method for appearance search | |
| US10147163B2 (en) | Systems and methods for automated image cropping | |
| CN109076198B (en) | Video-based object tracking occlusion detection system, method and equipment | |
| US8903123B2 (en) | Image processing device and image processing method for processing an image | |
| US8644563B2 (en) | Recognition of faces using prior behavior | |
| CN110264493A (en) | A kind of multiple target object tracking method and device under motion state | |
| JP4373840B2 (en) | Moving object tracking method, moving object tracking program and recording medium thereof, and moving object tracking apparatus | |
| US20110063441A1 (en) | Imager and surveillance system | |
| JP5937823B2 (en) | Image collation processing apparatus, image collation processing method, and image collation processing program | |
| CN110674837A (en) | Video similarity obtaining method and device, computer equipment and storage medium | |
| JP6448212B2 (en) | Recognition device and recognition method | |
| JP6349448B1 (en) | Information processing apparatus, information processing program, and information processing method | |
| JP2018055367A (en) | Image processing device, image processing method, and program | |
| JP6798609B2 (en) | Video analysis device, video analysis method and program | |
| US9286707B1 (en) | Removing transient objects to synthesize an unobstructed image | |
| CN112883940A (en) | Silent in-vivo detection method, silent in-vivo detection device, computer equipment and storage medium | |
| CN112434671B (en) | Pedestrian snapshot optimization method and device, computer equipment and storage medium | |
| JP2014157453A (en) | Image processing apparatus, image processing method, and image processing program | |
| Colombari et al. | Background initialization in cluttered sequences | |
| WO2018128138A1 (en) | Image processing device, video monitoring system, image processing method, and recording medium storing program | |
| CN114387296A (en) | Target track tracking method and device, computer equipment and storage medium | |
| JP6350331B2 (en) | TRACKING DEVICE, TRACKING METHOD, AND TRACKING PROGRAM | |
| JP2009003615A (en) | Attention region extraction method, attention region extraction device, computer program, and recording medium | |
| JP2021012631A (en) | Image processing system, information processing device, information processing method, and program | |
| JP6468642B2 (en) | Information terminal equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |