WO2023098081A1 - Method and apparatus for acquiring passenger flow data, and electronic device and storage medium - Google Patents

Abstract

Provided in the present disclosure are a method and apparatus for acquiring passenger flow data, and an electronic device and a storage medium. The method for acquiring passenger flow data provided in the embodiments of the present disclosure comprises: acquiring a video clip, which is obtained by means of performing video recording on a target area, and acquiring a discrimination mark, wherein the video clip comprises a plurality of frames of target images, and the target images comprise entrance and exit areas of the target area; respectively performing object recognition and tracking on each of the plurality of frames of the target images, so as to obtain a plurality of target objects, and key point information and moving speed information, which respectively correspond to the plurality of target objects; and determining passenger flow data of the target area on the basis of the key point information and the moving speed information, which respectively correspond to the plurality of target objects, and the discrimination mark.

Description

Method, device, electronic device and storage medium for obtaining passenger flow data

cross-reference statement

This application claims priority to a Chinese patent application with application number 202111446732.X filed with the China Patent Office on November 30, 2021, the entire contents of which are incorporated herein by reference.

technical field

The present disclosure relates to the technical field of image processing, and in particular, to a method, device, electronic device, and storage medium for acquiring passenger flow data.

Background technique

Many places need to understand the passenger flow information, and formulate relevant control strategies based on the passenger flow information to ensure the safety and normal operation of the corresponding places. For example, for subway places, it is necessary to know the passenger flow information during the peak commuting period in order to adopt corresponding passenger flow control strategies; for another example, for offline physical commercial places, it is necessary to understand the passenger flow information in order to formulate relevant operation strategies. However, the current passenger flow statistics for a certain place are not accurate enough.

Contents of the invention

Embodiments of the present disclosure at least provide a method, device, electronic device, and storage medium for acquiring passenger flow data.

In the first aspect, an embodiment of the present disclosure provides a method for acquiring passenger flow data, including: acquiring a video segment obtained by shooting a target area, and a distinguishing mark; the video segment includes multiple frames of target images; the target image includes The entrance and exit area of the target area; performing object recognition and tracking on each frame of the target image in the multiple frames of the target image, to obtain multiple target objects, key point information and moving speed information respectively corresponding to the multiple target objects ; Wherein, the key point information is used to characterize the position of the target object in the target image to which the target object belongs; the moving speed information is used to characterize the position of the target object in the target image composed of the target object The moving speed in the target image sequence: determining passenger flow data in the target area based on the key point information and the moving speed information respectively corresponding to the plurality of target objects, and the discriminant marks.

In this aspect, object recognition, object tracking, and key point detection can be realized more accurately through image recognition, and then the passenger flow data in the target area can be determined more accurately by combining the detected key point information and the moving speed of the target object. Compared with the prior art that uses head and shoulder detection of objects for passenger flow statistics, using key points for passenger flow statistics can effectively avoid the defect of low detection accuracy caused by occlusion, and combine the movement of target objects on the basis of key point information The speed can further improve the accuracy of passenger flow detection.

In a possible implementation manner, the key point information includes first position information indicating the position of at least one key point of the target object in the target image to which the target object belongs; the discriminant mark includes a flow rate discriminant line; wherein , the determining the passenger flow data in the target area based on the key point information and the moving speed information respectively corresponding to the multiple target objects, and the discriminant flag includes: for the multiple targets For each target object in the target object, based on the first position information of the at least one key point corresponding to the target object and the second position information of the flow judgment line, determine the line crossing result corresponding to the target object information; the cross-line result information is used to represent whether the target object crosses the flow judgment line; based on the cross-line result information and the moving speed information corresponding to the multiple target objects, determine the target area of the passenger flow data.

In this embodiment, according to the first position information of the key point corresponding to the target object and the second position information of the flow discrimination line, it can be more accurately determined whether the target object has crossed the flow discrimination line, that is, more accurate cross-line result information can be obtained; Afterwards, combined with the cross-line result information and the moving speed information of the target object, the passenger flow statistics of the target area can be more accurately counted.

In a possible implementation manner, the key points of each target object include at least one set of relative key points, and the key point information includes at least one set of relative key point information corresponding to the at least one set of relative key points, wherein, The relative key point information includes foot relative key point information composed of left foot key point information and right foot key point information of the target object, left knee key point information and right knee key point information of the target object. At least one of knee relative key point information composed of point information, leg relative key point information composed of left leg key point information and right leg key point information of the target object.

In this embodiment, the position information of key points such as footstep key points, knee key points or leg key points can more accurately represent the position of the target object, so that it can be more accurately determined whether the target object crosses the flow rate line.

In a possible implementation manner, the target object is determined based on the first position information of the at least one key point corresponding to the target object and the second position information of the flow discrimination line The corresponding line-crossing result information includes: based on the first position information and the second position information, it is determined that there is at least one set of relative points where a line connecting two corresponding key points intersects the flow discrimination line. In the case of key point information, determining that the line-crossing result information includes information indicating that the target object crosses the flow judgment line.

In this embodiment, if the connection line of a certain group of relative key points intersects the flow discrimination line, it means that the target object has crossed the flow discrimination line. At this time, the line crossing result information needs to be set to include the information indicating that the target object has crossed the flow discrimination line , so that the cross-line result information obtained is more accurate.

In a possible implementation manner, the target object is determined based on the first position information of the at least one key point corresponding to the target object and the second position information of the flow discrimination line The corresponding line-crossing result information includes: determining the line-crossing result information when the first position information of the at least one key point of the target object matches the second position information Include information indicating that the target object crosses the flow discrimination line.

In this embodiment, when a certain key point is on the flow judgment line, it means that the target object has crossed the flow judgment line. At this time, the line crossing result information needs to be set to include information indicating that the target object has crossed the flow judgment line. In this way, The obtained cross-line result information is more accurate.

In a possible implementation manner, the determining the passenger flow data in the target area based on the cross-line result information and the moving speed information corresponding to the multiple target objects includes: based on the video segment, determining the moving direction information of the target object; determining the passenger flow data in the target area based on the cross-line result information, the moving speed information, and the moving direction information corresponding to the multiple target objects .

In this embodiment, on the basis of the cross-line result information and moving speed information, combined with the moving direction information of the target object, it can be determined more accurately whether the target object enters the target area or leaves the target area, so as to obtain a more detailed passenger flow data.

In a possible implementation manner, the passenger flow data includes a first number of target objects entering the target area; wherein, the cross-line result information corresponding to the plurality of target objects, the movement The speed information and the moving direction information, determining the passenger flow data in the target area, includes: filtering the line-crossing result information from the multiple target objects to indicate crossing the traffic judging line, the moving direction The information indicates that the moving direction when crossing the flow discrimination line is the first object that enters the target area and the speed indicated by the moving speed information is greater than the first preset speed; the number of first objects is taken as the first number .

In this embodiment, according to the line-crossing result information, the moving speed information and the moving direction information, the target objects entering the target area can be more accurately screened out, that is, the above-mentioned first number can be determined more accurately.

In a possible implementation manner, the passenger flow data includes a second number of target objects leaving the target area; wherein, the cross-line result information corresponding to the plurality of target objects, the movement The speed information and the moving direction information, determining the passenger flow data in the target area, includes: filtering the line-crossing result information from the multiple target objects to indicate crossing the traffic judging line, the moving direction The information indicates that the moving direction when crossing the flow judgment line is the second object that leaves the target area and the speed indicated by the moving speed information is greater than a second preset speed; the number of the second objects is taken as the second object Two quantities.

In this embodiment, according to the line-crossing result information, the moving speed information and the moving direction information, the target objects leaving the target area can be more accurately screened out, that is, the above-mentioned second quantity can be determined more accurately.

In a possible implementation manner, the passenger flow data includes a third number of target objects currently staying in the target area; wherein, the cross-line result information corresponding to the plurality of target objects, the determining the passenger flow data in the target area based on the moving speed information and the moving direction information, further comprising: determining the third number based on the first number and the second number; wherein the first number is the The crossing result information in the plurality of target objects indicates crossing the flow judgment line, the moving direction information indicates that the moving direction when crossing the flow judgment line is to enter the target area, and the moving speed information indicates The number of first objects whose speed is greater than the first preset speed; the second number is that the line crossing result information indicates crossing the flow judgment line, and the moving direction information indicates crossing the The moving direction at the time of the flow determination line is the number of second objects leaving the target area whose speed indicated by the moving speed information is greater than a second preset speed.

In this embodiment, according to the number of target objects entering the target area, that is, the above-mentioned first number, and the number of target objects that leave the target area, that is, the above-mentioned second number, the number of target objects currently staying in the target area can be determined more accurately , which is the third quantity mentioned above.

In a possible implementation manner, the performing object recognition and tracking on each frame of target images in the multiple frames of target images to obtain the multiple target objects includes: respectively performing Object recognition, obtaining a plurality of object detection frames; wherein, each object detection frame corresponds to an object; determining the degree of overlap between the object detection frames respectively belonging to two frames of adjacent target images, and when the degree of overlap is greater than a preset When the value is set, it is determined that the object detection frames belonging to two adjacent frames of the target image correspond to the same object, and the object is used as the target object.

In this embodiment, according to the degree of overlap between the two object detection frames, it can be more accurately determined whether the objects corresponding to the two object detection frames are the same object, that is, the object tracking can be performed more accurately.

In a possible implementation manner, after obtaining the multiple target objects, the method further includes: for each target object in the multiple target objects, the target is not tracked within a preset time In the case of an object, it is determined that tracking of the target object has failed.

In this embodiment, for a target object that has not been detected within a certain period of time, there is a high probability that the tracking fails. At this time, such target objects can be set as tracking failure to improve the accuracy of object tracking.

In a possible implementation manner, the discrimination mark includes a flow discrimination line and a preset judgment frame, wherein the flow judgment line is at least partly located within the preset judgment frame; Performing object recognition and tracking on each frame of the target image in the target image to obtain the plurality of target objects, including: based on the third position information of the preset discrimination frame, separately identifying the multiple frames of the target image corresponding to the preset Object recognition is performed on the image area of the judgment frame, and multiple objects appearing in the image area corresponding to the preset judgment frame in the video clip are respectively tracked to obtain the multiple target objects.

This embodiment only recognizes and tracks objects within the preset judgment frame, which can effectively reduce the amount of data processing and improve efficiency and accuracy.

In a possible implementation manner, after the first object and the second object are determined, the method further includes: sending the time when the first object enters the target area, the The time when the second object leaves the target area, the entry identification information of the first object entering the target area, the exit identification information of the second object leaving the target area, and the second position of the traffic determination line at least one of the information, so as to realize passenger flow statistics in the target area through the remote server.

In this embodiment, information such as the entry time and entry sign of objects entering the target area, and the departure time and exit sign of objects leaving the target area are transmitted to the remote server, which can improve the accuracy of passenger flow statistics through the remote server. sex.

In the second aspect, the embodiment of the present disclosure also provides a device for acquiring passenger flow data, including: an information acquisition module, configured to acquire a video clip obtained by shooting a target area, and a distinguishing mark; the video clip includes multiple frames of target images The target image includes the entrance and exit area of the target area; the information processing module is used to perform object recognition and tracking on each frame of the target image in the multi-frame target image, and obtain multiple target objects, the multiple Key point information and moving speed information respectively corresponding to the target object; wherein, the key point information is used to characterize the position of the target object in the target image to which the target object belongs; the moving speed information is used to characterize the target The moving speed of the object in the target image sequence composed of the target images to which the target object belongs; the passenger flow statistics module, which is used to base on the key point information and the moving speed information respectively corresponding to the multiple target objects, and the The discrimination flag is used to determine the passenger flow data of the target area.

In a third aspect, an embodiment of the present disclosure further provides an electronic device, including: a processor, a memory, and a bus, the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the The processor communicates with the memory through the bus, and when the machine-readable instructions are executed by the processor, the steps in the above-mentioned first aspect or any possible implementation manner of the first aspect are performed.

In the fourth aspect, the embodiments of the present disclosure further provide a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, any one of the above-mentioned first aspect or the first aspect is executed. steps in a possible implementation.

For the effect description of the above device for obtaining passenger flow data, the electronic equipment, and the computer-readable storage medium, please refer to the description of the above method for obtaining passenger flow data, which will not be repeated here.

In order to make the above-mentioned objects, features and advantages of the present disclosure more comprehensible, preferred embodiments will be described in detail below together with the accompanying drawings.

Description of drawings

In order to illustrate the technical solutions of the embodiments of the present disclosure more clearly, the following will briefly introduce the accompanying drawings used in the embodiments. The accompanying drawings here are incorporated into the specification and constitute a part of the specification. The drawings show the embodiments consistent with the present disclosure, and are used together with the description to explain the technical solutions of the present disclosure. It should be understood that the following drawings only show some embodiments of the present disclosure, and therefore should not be regarded as limiting the scope. For those skilled in the art, they can also make From these drawings other related drawings are obtained.

FIG. 1 shows a flow chart of a method for acquiring passenger flow data provided by an embodiment of the present disclosure;

FIG. 2 shows a flow chart of determining passenger flow data in another method for obtaining passenger flow data provided by an embodiment of the present disclosure;

FIG. 3 shows a schematic diagram of an area for performing passenger flow statistics provided by an embodiment of the present disclosure;

Fig. 4 shows a schematic diagram of a device for obtaining passenger flow data provided by an embodiment of the present disclosure;

Fig. 5 shows a schematic diagram of an electronic device provided by an embodiment of the present disclosure.

Detailed ways

In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only It is a part of the embodiments of the present disclosure, but not all of them. The components of the disclosed embodiments generally described and illustrated in the figures herein may be arranged and designed in a variety of different configurations. Accordingly, the following detailed description of the embodiments of the present disclosure provided in the accompanying drawings is not intended to limit the scope of the claimed disclosure, but merely represents selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative effort shall fall within the protection scope of the present disclosure.

After research, it is found that at present, many places need to carry out passenger flow statistics, but the current passenger flow statistics for a certain place are not accurate enough. Aiming at this, the present disclosure provides a method, device, electronic device and storage medium for acquiring passenger flow data. The present disclosure can more accurately realize object recognition, object tracking, and key point detection through image recognition, and then combine the detected key point information and the moving speed of the target object to more accurately determine the passenger flow data in the target area. Compared with the prior art that uses head and shoulder detection of objects for passenger flow statistics, using key points for passenger flow statistics can effectively avoid the defect of low detection accuracy caused by occlusion, and combine the movement of target objects on the basis of key point information The speed can further improve the accuracy of passenger flow detection.

The method for acquiring passenger flow data provided by the embodiments of the present disclosure will be described below by taking the execution subject as a device with computing capability as an example.

As shown in FIG. 1 , the method for acquiring passenger flow data provided by the present disclosure may include steps S110-S130.

S110. Acquire a video segment obtained by shooting the target area, and a distinguishing mark; the video segment includes multiple frames of target images; the target image includes an entrance and exit area of the target area.

The aforementioned target area is the area where passenger flow data statistics are to be performed, for example, a shopping mall. The above video clips may be taken by cameras installed in the entrance and exit areas of the target area.

The above-mentioned discrimination mark may include a flow judgment line, and only when an object (for example, a customer, passenger, etc.) crosses the flow judgment line, the information of the object may become valid data in passenger flow statistics. In addition, the discrimination mark may also include a preset judgment frame, only when the object enters the preset judgment frame, the information of the object may become valid data in passenger flow statistics. The above-mentioned discrimination mark can be set, for example, in the entrance and exit area of the target area.

The aforementioned multi-frame target images may refer to all frame images included in the video clip, or may refer to partial frame images selected from the video clip.

S120. Perform object recognition and tracking on each frame of target images in the multiple frames of target images to obtain multiple target objects, key point information of the target objects, and moving speed information of the target objects.

Wherein, the key point information represents the position of the target object in the target image to which the target object belongs; the moving speed information represents the position of the target object in the target image sequence composed of the target images to which the target object belongs movement speed in .

Exemplarily, the trained neural network can be used to perform object recognition and tracking on the target image, and to determine key point information of the target object.

When performing object recognition on a target image, the object detection frames of each object in the target image can be determined. Object tracking can then be performed according to the Intersection-over-Union (IoU) of the object detection frame. Specifically, in two frames of target images that are adjacent or have a shorter shooting time interval, set the Intersection-over-Union ratio greater than the preset The two objects corresponding to the two object detection frames of the value are regarded as the same object. At this time, the object is considered to be tracked successfully, so the object can be used as the target object.

While performing object recognition, the trained neural network can be used to determine the key point information corresponding to each object. Alternatively, after the target object is determined, the trained neural network may be used to detect the target object to determine key point information corresponding to the target object. The neural network for object recognition and tracking on the target image and the neural network for determining key point information may be the same neural network or different neural networks.

The above key point information may be information capable of characterizing the position of the target object in the target image to which it belongs, and specifically may include first position information indicating the position of at least one key point of the target object in at least one frame of the target image. For example, the key point information may be at least one of information corresponding to key points of the target object's feet, information corresponding to key points of the target object's knees, and information corresponding to key points of the target object's legs.

According to the tracking of the target object, the multi-frame target images to which the target object belongs can be determined, and these images are sorted according to the shooting time to obtain the above target image sequence.

After the target image sequence of a certain target object is determined, the moving distance of the target object within the shooting time interval of any two frames of target images in the target image sequence may be determined. Afterwards, according to the shooting time interval and moving distance of the two frames of target images, the moving speed of the target object within the shooting time interval can be determined. In this way, any two frames of target images can be selected multiple times from the sequence of target images, and multiple corresponding moving speeds can be obtained. Based on a plurality of the above-mentioned moving speeds, the above-mentioned moving speed information may be determined. Exemplarily, multiple moving speeds may be directly used as the moving speed information.

S130. Determine passenger flow data in the target area based on the key point information and the moving speed information respectively corresponding to the multiple target objects, and the discrimination flag.

Exemplarily, it can be determined whether the target object has crossed the traffic discrimination line according to the key point information and the discrimination mark, and the crossing speed of the target object when crossing the traffic judgment line can be determined according to the moving speed information, and only the crossing speed meets the preset requirements The information of the target object can become effective data in the passenger flow data. Therefore, the passenger flow data in the target area can be determined more accurately according to the key point information, moving speed information, and discriminant signs.

Exemplarily, the passenger flow data of the target area can be determined through the following steps: first, for each target object, based on the first position information of at least one key point corresponding to the target object and the second position information of the flow discrimination line The position information is used to determine the line-crossing result information corresponding to the target object; the line-crossing result information is used to indicate whether the target object crosses the flow judgment line. Afterwards, passenger flow data in the target area is determined based on the line-crossing result information corresponding to the multiple target objects and the moving speed information.

In some embodiments, at least one set of relative key point information corresponding to at least one set of relative key points of the target object may be used to determine the line-crossing result information corresponding to the target object. For example, the relative key point information may include the relative key point information of the foot composed of the key point information of the left foot and the key point information of the right foot of the target object, and the key point information of the left knee and the key point information of the right knee of the target object. At least one item of the relative key point information of the leg formed by the relative key point information of the knee, the key point information of the target object's left leg, and the key point information of the right leg. The above relative key point information includes at least the position information of the corresponding key point, for example, the relative key point information of the target object’s foot includes at least the first position information of the target object’s left foot key point and the first position information of the right foot key point; For another example, the relative knee key point information of the target object includes at least the first position information of the left knee key point and the first position information of the right knee key point of the target object.

The position information of the key points of footsteps, knees, or legs can more accurately represent the position of the target object, so that it can be more accurately determined whether the target object has crossed the flow discrimination line according to these key points.

During specific implementation, the following method can be used to determine the cross-line result information corresponding to the target object: based on the first position information and the second position information, it is determined that there are two corresponding key points between the line and the flow rate In the case of at least one set of relative key point information intersected by the judgment line, it is determined that the line-crossing result information includes information that the target object crosses the flow judgment line.

The connection line of a certain group of relative key points intersects the flow discrimination line, which means that the target object has crossed the flow discrimination line. At this time, it is necessary to set the cross-line result information to include information indicating that the target object crosses the flow discrimination line. Line result information is more accurate.

In some embodiments, a single key point can also be used to determine the line-crossing result information corresponding to the target object. Specifically, in the case that the first position information of at least one key point of the target object matches the second position information, determining the line crossing result information includes indicating that the target object crosses the flow judgment line information. The above matching can specifically be that the first position information of the key point is the same as the second position information. At this time, it can indicate that a key point of the target object is located on the flow discrimination line, that is, the target object is located on the flow discrimination line. At this time, it is considered The target object crosses the flow discrimination line.

When a certain key point is on the flow discrimination line, it means that the target object has crossed the flow discrimination line. At this time, it is necessary to set the cross-line result information to include information indicating that the target object crosses the flow judgment line. In this way, the obtained cross-line result information more precise.

On the basis of the above-mentioned cross-line result information and moving speed information, the passenger flow data of the target area can be determined in combination with the moving direction information of the target object, so that it can be further accurately determined whether the target object enters the target area or leaves Target area, so as to get more detailed passenger flow data.

The above-mentioned moving direction information can be determined according to the above-mentioned video clips, such as determining the target image sequence corresponding to the target object from the video clip, and determining the position of the target object in any two frames of the target image in the target image sequence, and then according to the two frames The shooting time interval of the target image can determine the moving direction of the target object within the shooting time interval of the two frames of the target image. Here, any two frames of target images may be two frames of adjacent images in the target image sequence. In this way, any two frames of target images can be selected multiple times from the sequence of target images, and multiple corresponding moving directions can be obtained. Based on a plurality of moving directions, the above-mentioned moving direction information may be determined. Exemplarily, multiple moving directions may be directly used as the moving direction information. It can be seen that, according to the above moving direction information, the moving direction of the target object when it crosses the flow determination line can be determined.

In some embodiments, as shown in FIG. 2, the passenger flow data in the target area is determined based on the cross-line result information corresponding to the multiple target objects, the moving speed information, and the moving direction information. Specifically, Use the following steps to achieve:

S210. From a plurality of target objects, filter the line-crossing result information indicating crossing the traffic judging line, and the moving direction information indicating that the moving direction when crossing the traffic judging line is entering the target area, the moving direction the first object whose speed indicated by the speed information is greater than the first preset speed; taking the number of the first object as the first number of target objects entering the target area;

S220. From the plurality of target objects, filter the line-crossing result information indicating crossing the traffic judging line, and the moving direction information indicating that the moving direction when crossing the traffic judging line is to leave the target area, a second object whose speed indicated by the moving speed information is greater than a second preset speed; taking the number of the second object as the second number of target objects leaving the target area;

S230. Determine a third quantity based on the first quantity and the second quantity; for example, calculate the difference between the first quantity and the second quantity, and use the difference as the third quantity, here the third The number is the number of target objects currently staying in the target area;

S240, using the first quantity, the second quantity and the third quantity as the passenger flow data of the target area.

According to the line-crossing result information, the moving speed information and the moving direction information, the number of target objects leaving the target area, the number of target objects entering the target area, and the target objects currently staying in the target area can be more accurately screened out The quantity effectively improves the accuracy of the determined passenger flow data.

In some embodiments, the above object recognition and tracking are performed on each frame of the target image in the multiple frames of the target image to obtain multiple target objects, which can be specifically implemented by the following steps:

First, object recognition is performed on the multi-frame target images respectively to obtain a plurality of object detection frames; wherein, each object detection frame corresponds to an object; after that, it is determined between object detection frames belonging to two adjacent frames of target images respectively degree of overlap, and when the degree of overlap is greater than a preset value, it is determined that the object detection frames belonging to two frames of adjacent target images correspond to the same object, and the object is taken as the target object.

During specific implementation, it is also possible to determine the degree of overlap between two object detection frames that belong to target images with a shorter shooting time interval between any two frames, and if the degree of overlap is greater than a preset value, determine the degree of overlap between the two The two objects corresponding to the object detection frames are the same object, and the object is used as the target object.

Whether the objects corresponding to the two object detection frames are the same object can be determined more accurately according to the degree of overlap between the two object detection frames, that is, object tracking can be performed more accurately.

The degree of overlap here is specifically the intersection and union ratio of two object detection frames.

In some embodiments, if a certain target object is not tracked within a preset time, it is determined that the target object has failed to be tracked. For a target object that has not been detected within a certain period of time, there is a high probability that the tracking will fail. At this time, such target objects can be set as tracking failure to improve the accuracy of object tracking.

After a certain target object is determined for the first time, an object identifier will be set for the target object. For a target object that fails to be tracked, the object identifier of the target object will not be assigned to other target objects detected subsequently.

In some embodiments, the above-mentioned judging signs include not only flow judging lines, but also preset judging boxes. At this time, the above-mentioned object recognition and tracking are respectively performed on each frame of the target image in the multi-frame target images to obtain a plurality of target objects, which can be specifically implemented by using the following steps:

Based on the third position information of the preset judgment frame, object recognition is performed on image regions corresponding to the preset judgment frame in multiple frames of target images, and object recognition is performed on the video clips corresponding to the preset judgment frame Multiple objects appearing in the image area of the image area are respectively tracked to obtain multiple target objects; wherein the flow determination line is at least partially located within the preset determination frame.

In specific implementation, according to the third position information of the preset discrimination frame, the corresponding sub-image can be intercepted from the target image, and the sub-image sequence formed by sorting the sub-images of multiple frames according to the shooting time is subjected to object recognition and tracking, and the obtained multiple target audiences.

Recognizing and tracking only objects within the preset discrimination frame can effectively reduce the amount of data processing and improve efficiency and accuracy.

The above-mentioned preset judgment frame and flow judgment line are determined after calibrating the shooting device that shoots the above-mentioned video clip. The preset discrimination frame may be a frame of regular shape such as a rectangle, or a frame of irregular shape, which is not limited in this application. When setting the preset discrimination frame, it is necessary to ensure that the upper body of the object entering or leaving the target area is inside the frame, so as to improve the accuracy of object recognition, tracking and passenger flow data statistics.

Exemplarily, FIG. 3 shows a schematic diagram of an area for performing passenger flow statistics provided by an embodiment of the present disclosure. As shown in FIG. 3 , the entrance and exit area of the target area is marked with 310 , and a preset judgment box 320 is set in the entrance and exit area 310 , and the embodiment of the present disclosure only recognizes and tracks objects appearing in the preset judgment box. The flow judgment line 330 is at least partially located within the preset judgment box 320 . The statistics of passenger flow data need to be carried out according to whether the tracked target object crosses the flow judgment line 330 . Specifically, as shown in FIG. 3 , a certain group of relative key points of the target object 350 (for example, if the midpoint of the sole of the sole is used as a key point, then this group of relative key points can be the midpoint of both feet of the target object 350) 370 intersects the flow discrimination line 330, at this time, the target object 350 is considered to cross the flow discrimination line 330; a certain key point of another target object 360 is located on the flow discrimination line 330, and at this time, another target object 360 is also considered to cross the flow discrimination line 330.

Further, if the tracking results show that the same target object is the target object 350 and the target object 360 at different times, the moving direction of the target object can be determined according to the time when the image to which the target object 350 belongs and the time when the image to which the target object 360 is captured information. For example, if the time to capture the image of the target object 350 is earlier than the time to capture the image of the target object 360 , it is determined that the moving direction information of the target object indicates that the moving direction of the target object when crossing the flow determination line is to enter the target area. If the speed indicated by the moving speed information of the target object determined according to the time when the image to which the target object 350 belongs and the time when the image to which the target object 360 belongs is determined to be greater than the first preset speed, then it is determined that the target object is an object entering the target area (i.e. the first object above). The above-mentioned first quantity, that is, the quantity of objects entering the target area, can be determined according to the counted quantity of this type of target objects.

Further, if the time of shooting the image to which the target object 350 belongs is later than the time of shooting the image to which the target object 360 belongs, it is determined that the moving direction information of the target object indicates that the moving direction of the target object when it crosses the flow judgment line is to leave the target area . If the speed indicated by the moving speed information of the target object determined according to the time when the image to which the target object 350 belongs and the time when the image to which the target object 360 belongs is determined to be greater than the second preset speed, then it is determined that the target object is an object leaving the target area ( i.e. the second object above). The above-mentioned second quantity, that is, the quantity of objects leaving the target area, can be determined according to the counted quantity of this type of target objects.

Before carrying out passenger flow data statistics, it is also necessary to determine the first direction of entering the target area and the second direction of leaving the target area. The first direction and the second direction are used to determine the moving direction of the target object when crossing the flow judgment line. area or away from the target area.

After determining the first object and the second object, the time when the first object enters the target area, the time when the second object leaves the target area, and the time when the first object enters the target area can also be calculated. The entry identification information of the target area, the exit identification information of the second object leaving the target area, the second position information of the flow judgment line, the identification of the flow judgment line, video clips, and the corresponding At least one of the target image sequences is sent to the remote server, so as to realize passenger flow counting in the target area through the remote server.

The information such as the entry time and entry sign of objects entering the target area, and the departure time and exit sign of objects leaving the target area are transmitted to the remote server, which can improve the accuracy of passenger flow statistics through the remote server.

The methods in the above embodiments of the present disclosure may also be executed by a shooting device that shoots video clips, and the present disclosure does not limit the execution subject of the methods in the above embodiments.

Those skilled in the art can understand that in the above method of specific implementation, the writing order of each step does not mean a strict execution order and constitutes any limitation on the implementation process. The specific execution order of each step should be based on its function and possible The inner logic is OK.

Based on the same inventive concept, the embodiment of the present disclosure also provides a device for obtaining passenger flow data corresponding to the method for obtaining passenger flow data, because the problem-solving principle of the device in the embodiment of the present disclosure is similar to the above-mentioned method for obtaining passenger flow data in the embodiment of the present disclosure , so the implementation of the device can refer to the implementation of the method, and the repetition will not be repeated.

Referring to FIG. 4 , FIG. 4 is a schematic structural diagram of a device for obtaining passenger flow data provided by an embodiment of the present disclosure. The device includes an information obtaining module 410 , an information processing module 420 and a passenger flow statistics module 430 .

The information acquiring module 410 is used to acquire video clips obtained by shooting the target area and identification marks; the video clips include multiple frames of target images; the target images include the entrance and exit areas of the target area.

The information processing module 420 is configured to perform object recognition and tracking on each frame of the target images in the multiple frames of target images, and obtain multiple target objects, key point information and moving speed information respectively corresponding to the multiple target objects.

Wherein, the key point information is used to represent the position of the target object in the target image to which the target object belongs; the moving speed information is used to represent the position of the target object in the target image composed of the target object to which the target object belongs. The speed of movement in the image sequence.

The passenger flow counting module 430 is configured to determine passenger flow data in the target area based on the key point information and the moving speed information respectively corresponding to the plurality of target objects, and the discriminant flag.

In some embodiments, the key point information includes first position information indicating the position of at least one key point of the target object in the target image to which the target object belongs; the discrimination mark includes a flow discrimination line;

Wherein, when the passenger flow statistics module 430 determines the passenger flow data in the target area based on the key point information and the moving speed information respectively corresponding to the multiple target objects, and the discrimination flag, At:

For each of the plurality of target objects, based on the first position information of the at least one key point corresponding to the target object and the second position information of the flow discrimination line, determine the target Line-crossing result information corresponding to the object; the line-crossing result information is used to represent whether the target object crosses the flow discrimination line;

The passenger flow data in the target area is determined based on the line crossing result information and the moving speed information corresponding to the multiple target objects.

In some embodiments, the key points of each target object include at least one set of relative key points, and the key point information includes at least one set of relative key point information corresponding to the at least one set of relative key points, wherein the relative key points The key point information includes the relative key point information of the foot which is composed of the key point information of the left foot and the key point information of the right foot of the target object, the key point information of the left knee and the key point information of the right knee of the target object At least one item of the knee relative key point information, and the leg relative key point information composed of the target object's left leg key point information and right leg key point information.

In some embodiments, the passenger flow statistics module 430 determines the When the cross-line result information corresponding to the target object is used for:

When it is determined based on the first position information and the second position information that there is at least one set of relative key point information in which a line connecting two corresponding key points intersects the flow discrimination line, determine the cross The line result information includes information indicating that the target object crosses the traffic determination line.

In some embodiments, the passenger flow statistics module 430 determines the When the cross-line result information corresponding to the target object is used for:

If the first position information of the at least one key point of the target object matches the second position information, determining the line crossing result information includes indicating that the target object crosses the flow judgment line Information.

In some embodiments, when determining the passenger flow data in the target area based on the cross-line result information and the moving speed information corresponding to the multiple target objects, the passenger flow statistics module 430 is configured to:

determining moving direction information of the target object based on the video clip;

The passenger flow data in the target area is determined based on the line-crossing result information, the moving speed information, and the moving direction information corresponding to the multiple target objects.

In some embodiments, the traffic data includes a first number of target objects entering the target area;

Wherein, when the passenger flow statistics module 430 determines the passenger flow data in the target area based on the cross-line result information, the moving speed information, and the moving direction information corresponding to the multiple target objects, it uses At:

From the plurality of target objects, filtering the line-crossing result information indicates crossing the traffic judging line, and the moving direction information indicates that the moving direction when crossing the traffic judging line is entering the target area, the moving direction a first object whose speed indicated by the speed information is greater than a first preset speed;

The quantity of the first object is used as the first quantity.

In some embodiments, the traffic data includes a second number of target objects leaving the target area;

Wherein, when the passenger flow statistics module 430 determines the passenger flow data in the target area based on the cross-line result information, the moving speed information, and the moving direction information corresponding to the multiple target objects, it uses At:

From the plurality of target objects, filter the line-crossing result information indicating crossing the traffic judging line, and the moving direction information indicating that the moving direction when crossing the traffic judging line is to leave the target area, the moving direction a second object whose speed indicated by the speed information is greater than a second preset speed;

The quantity of the second object is used as the second quantity.

In some embodiments, the traffic data includes a third number of target objects currently staying in the target area;

Wherein, when the passenger flow statistics module 430 determines the passenger flow data in the target area based on the cross-line result information, the moving speed information, and the moving direction information corresponding to the multiple target objects, it uses At:

determining the third quantity based on the first quantity and the second quantity;

Wherein, the first quantity is that the line-crossing result information in the plurality of target objects indicates that the flow judgment line is crossed, and the moving direction information indicates that the moving direction when crossing the flow judgment line is to enter the target area, the number of first objects whose speed indicated by the moving speed information is greater than a first preset speed;

The second quantity is that the line-crossing result information in the plurality of target objects indicates that the flow judgment line is crossed, the moving direction information indicates that the moving direction when crossing the flow judgment line is to leave the target area, The moving speed information indicates the number of second objects whose speed is greater than a second preset speed.

In some embodiments, when the information processing module 420 performs object recognition and tracking on each frame of target images in the multiple frames of target images to obtain the multiple target objects, it is used to:

Respectively performing object recognition on the multi-frame target images to obtain a plurality of object detection frames; wherein, each object detection frame corresponds to an object;

Determining the degree of overlap between the object detection frames respectively belonging to two adjacent frames of target images, and in the case that the degree of overlap is greater than a preset value, determining the objects respectively belonging to two frames of adjacent target images The detection boxes correspond to the same object, and the object is used as the target object.

In some embodiments, after obtaining the multiple target objects, the information processing module 420 is further configured to:

For each target object among the plurality of target objects, if the target object is not tracked within a preset time, it is determined that the tracking of the target object fails.

In some embodiments, the judgment mark includes a flow judgment line and a preset judgment box, wherein the flow judgment line is at least partially located within the preset judgment box;

Wherein, when the information processing module 420 performs object recognition and tracking on each frame of the target image in the multiple frames of target images to obtain the multiple target objects, it is used to:

Based on the third position information of the preset judgment frame, perform object recognition on image regions corresponding to the preset judgment frame in the multi-frame target images, and perform object recognition on the video clips corresponding to the preset judgment frame Multiple objects appearing in the image area of the discrimination frame are respectively tracked to obtain the multiple target objects.

In some embodiments, after the passenger flow counting module 430 determines the first object and the second object, it is further configured to:

sending the time when the first object enters the target area, the time when the second object leaves the target area, the entry identification information of the first object entering the target area, and the second object to the remote server. At least one of the departure identification information of the object leaving the target area and the second position information of the flow judgment line, so as to implement passenger flow statistics for the target area through the remote server.

Based on the same technical idea, an embodiment of the present disclosure also provides an electronic device. Referring to FIG. 5 , it is a schematic structural diagram of an electronic device 500 provided by an embodiment of the present disclosure, including a processor 51 , a memory 52 , and a bus 53 . Wherein, the memory 52 is used to store execution instructions, including a memory 521 and an external memory 522; the memory 521 here is also called an internal memory, and is used to temporarily store the calculation data in the processor 51 and the data exchanged with the external memory 522 such as a hard disk, The processor 51 exchanges data with the external memory 522 through the memory 521 .

When the electronic device 500 is running, the processor 51 communicates with the memory 52 through the bus 53, so that the processor 51 performs the following operations when executing the instructions: acquire the video segment obtained by shooting the target area, and identify the mark; The segment includes a multi-frame target image; the target image includes an entrance and exit area of the target area; object recognition and tracking are performed on each frame of the target image in the multi-frame target image, and multiple target objects, the multiple Key point information and moving speed information respectively corresponding to the target object; wherein, the key point information is used to characterize the position of the target object in the target image to which the target object belongs; the moving speed information is used to characterize the target The moving speed of the object in the target image sequence composed of the target images to which the target object belongs; based on the key point information and the moving speed information respectively corresponding to the multiple target objects, and the discriminant flag, determine the Passenger flow data in the target area.

An embodiment of the present disclosure also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is run by a processor, the steps of the method for obtaining passenger flow data described in the above-mentioned method embodiments are executed . Wherein, the storage medium may be a volatile or non-volatile computer-readable storage medium.

The computer program product of the method for obtaining passenger flow data provided by the embodiments of the present disclosure includes a computer-readable storage medium storing program codes, and the instructions included in the program code can be used to execute the method for obtaining passenger flow data described in the above method embodiments For the steps of the method, refer to the above-mentioned method embodiment for details, and details are not repeated here. The computer program product can be specifically realized by means of hardware, software or a combination thereof. In an optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK) etc. wait.

Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working process of the above-described system and device can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed systems, devices and methods may be implemented in other ways. The device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or May be integrated into another system, or some features may be ignored, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some communication interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit.

If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium executable by a processor. Based on this understanding, the technical solution of the present disclosure is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including Several instructions are used to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in various embodiments of the present disclosure. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disc and other media that can store program codes. .

Finally, it should be noted that: the above-mentioned embodiments are only specific implementations of the present disclosure, and are used to illustrate the technical solutions of the present disclosure, rather than limit them, and the protection scope of the present disclosure is not limited thereto, although referring to the aforementioned The embodiments have described the present disclosure in detail, and those skilled in the art should understand that any person familiar with the technical field can still modify the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present disclosure Changes can be easily imagined, or equivalent replacements can be made to some of the technical features; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in this disclosure. within the scope of protection. Therefore, the protection scope of the present disclosure should be defined by the protection scope of the claims.

Claims (15)

1. A method for obtaining passenger flow data, comprising:

   Obtaining a video clip obtained by shooting the target area, and a distinguishing mark; the video clip includes a multi-frame target image; the target image includes an entrance and exit area of the target area;

   Respectively performing object recognition and tracking on each frame of the target image in the multiple frames of target images to obtain multiple target objects, key point information and moving speed information respectively corresponding to the multiple target objects;

   Wherein, the key point information is used to represent the position of the target object in the target image to which the target object belongs; the moving speed information is used to represent the position of the target object in the target image composed of the target object to which the target object belongs. speed of movement in the image sequence;

   The passenger flow data of the target area is determined based on the key point information and the moving speed information respectively corresponding to the plurality of target objects, and the discriminant marks.
2. The method according to claim 1, wherein the key point information includes first position information indicating the position of at least one key point of the target object in the target image to which the target object belongs; judgment line,

   Wherein, the determination of the passenger flow data in the target area based on the key point information and the moving speed information respectively corresponding to the plurality of target objects, and the discrimination mark includes:

   For each of the plurality of target objects, based on the first position information of the at least one key point corresponding to the target object and the second position information of the flow discrimination line, determine the target Line-crossing result information corresponding to the object; the line-crossing result information is used to represent whether the target object crosses the flow discrimination line;

   The passenger flow data in the target area is determined based on the line crossing result information and the moving speed information corresponding to the multiple target objects.
3. The method according to claim 2, wherein the keypoints of each target object comprise at least one set of relative keypoints,

   The key point information includes at least one set of relative key point information corresponding to the at least one set of relative key points,

   Wherein, the relative key point information includes foot relative key point information composed of left foot key point information and right foot key point information of the target object, left knee key point information and right knee key point information of the target object At least one item of knee relative key point information composed of knee key point information, leg relative key point information composed of left leg key point information and right leg key point information of the target object.
4. The method according to claim 3, wherein, based on the first position information of the at least one key point corresponding to the target object and the second position information of the flow discrimination line, determine The cross-line result information corresponding to the target object includes at least one of the following:

   When it is determined based on the first position information and the second position information that there is at least one set of relative key point information in which a line connecting two corresponding key points intersects the flow discrimination line, determine the cross The line result information includes information indicating that the target object crosses the flow judgment line;

   In the case where the first position information of the at least one key point of the target object matches the second position information, determining the line crossing result information includes indicating that the target object crosses the flow discrimination line information.
5. The method according to any one of claims 2 to 4, characterized in that, based on the cross-line result information and the moving speed information corresponding to the multiple target objects, the Traffic data, including:

   determining moving direction information of the target object based on the video clip;

   The passenger flow data in the target area is determined based on the line-crossing result information, the moving speed information, and the moving direction information corresponding to the multiple target objects.
6. The method of claim 5, wherein the traffic data includes a first number of target objects entering the target area,

   Wherein, the determination of the passenger flow data in the target area based on the cross-line result information, the moving speed information, and the moving direction information corresponding to the multiple target objects includes:

   From the plurality of target objects, filtering the line-crossing result information indicates crossing the traffic judging line, and the moving direction information indicates that the moving direction when crossing the traffic judging line is entering the target area, the moving direction a first object whose speed indicated by the speed information is greater than a first preset speed;

   The quantity of the first object is used as the first quantity.
7. The method according to claim 5 or 6, wherein the passenger flow data includes a second number of target objects leaving the target area,

   Wherein, the determination of the passenger flow data in the target area based on the cross-line result information, the moving speed information, and the moving direction information corresponding to the multiple target objects includes:

   From the plurality of target objects, filter the line-crossing result information indicating crossing the traffic judging line, and the moving direction information indicating that the moving direction when crossing the traffic judging line is to leave the target area, the moving direction a second object whose speed indicated by the speed information is greater than a second preset speed;

   The quantity of the second object is used as the second quantity.
8. The method according to any one of claims 5 to 7, wherein the passenger flow data includes a third number of target objects currently staying in the target area,

   Wherein, the determining the passenger flow data in the target area based on the cross-line result information, the moving speed information, and the moving direction information corresponding to the multiple target objects further includes:

   determining the third quantity based on the first quantity and the second quantity;

   Wherein, the first quantity is that the line-crossing result information in the plurality of target objects indicates that the flow judgment line is crossed, and the moving direction information indicates that the moving direction when crossing the flow judgment line is to enter the target area, the number of first objects whose speed indicated by the moving speed information is greater than a first preset speed;

   The second quantity is that the line-crossing result information in the plurality of target objects indicates that the flow judgment line is crossed, the moving direction information indicates that the moving direction when crossing the flow judgment line is to leave the target area, The moving speed information indicates the number of second objects whose speed is greater than a second preset speed.
9. The method according to claim 8, wherein after determining the first object and the second object, the method further comprises:

   sending the time when the first object enters the target area, the time when the second object leaves the target area, the entry identification information of the first object entering the target area, and the second object to the remote server. At least one of the departure identification information of the object leaving the target area and the second position information of the flow judgment line, so as to implement passenger flow statistics for the target area through the remote server.
10. The method according to any one of claims 1 to 9, wherein the discrimination mark includes a flow discrimination line and a preset judgment frame, wherein the flow judgment line is at least partly located within the preset judgment frame,

    Wherein, the object recognition and tracking are carried out to each frame of the target image in the multi-frame target image respectively, and the multiple target objects are obtained, including:

    Based on the third position information of the preset judgment frame, perform object recognition on image regions corresponding to the preset judgment frame in the multi-frame target images, and perform object recognition on the video clips corresponding to the preset judgment frame Multiple objects appearing in the image area of the discrimination frame are respectively tracked to obtain the multiple target objects.
11. The method according to any one of claims 1 to 10, wherein the step of performing object recognition and tracking on each frame of target images in the multiple frames of target images to obtain the plurality of target objects includes:

    Respectively performing object recognition on the multi-frame target images to obtain a plurality of object detection frames; wherein, each object detection frame corresponds to an object;

    Determining the degree of overlap between the object detection frames respectively belonging to two adjacent frames of target images, and in the case that the degree of overlap is greater than a preset value, determining the objects respectively belonging to two frames of adjacent target images The detection boxes correspond to the same object, and the object is used as the target object.
12. The method according to claim 11, wherein after obtaining the plurality of target objects, the method further comprises:

    For each target object among the plurality of target objects, if the target object is not tracked within a preset time, it is determined that the tracking of the target object fails.
13. A device for obtaining passenger flow data, characterized in that it comprises:

    An information acquisition module, configured to acquire a video clip obtained by shooting the target area, and a distinguishing mark; the video clip includes multiple frames of target images; the target image includes an entrance and exit area of the target area;

    An information processing module, configured to perform object recognition and tracking on each frame of target images in the multiple frames of target images, to obtain multiple target objects, key point information and moving speed information respectively corresponding to the multiple target objects;

    Wherein, the key point information is used to represent the position of the target object in the target image to which the target object belongs; the moving speed information is used to represent the position of the target object in the target image composed of the target object to which the target object belongs. speed of movement in the image sequence;

    The passenger flow counting module is configured to determine the passenger flow data of the target area based on the key point information and the moving speed information respectively corresponding to the plurality of target objects, and the discrimination marks.
14. An electronic device, characterized in that it includes: a processor, a memory and a bus, the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the The memories communicate through the bus, and the machine-readable instructions are executed by the processor to execute the steps of the method for obtaining passenger flow data according to any one of claims 1 to 12.
15. A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the method of obtaining passenger flow data according to any one of claims 1 to 12 is executed. steps of the method.

Images