WO2023077797A1

WO2023077797A1 - Method and apparatus for analyzing queue

Info

Publication number: WO2023077797A1
Application number: PCT/CN2022/097274
Authority: WO
Inventors: 刘诗男; 杨昆霖; 侯军; 伊帅
Original assignee: 上海商汤智能科技有限公司
Priority date: 2021-11-05
Filing date: 2022-06-07
Publication date: 2023-05-11
Also published as: CN114049378A

Abstract

A method and apparatus for analyzing a queue are provided, and applied to a terminal device, the method comprises: performing target detection on one or more image frames in a video stream, and determining one or more target objects in a queue in the one or more image frames; tracking the one or more target objects in the video stream, and assigning a tracking identifier to each of the one or more target objects; and determining a queuing analysis result of the queue according to the tracking identifiers of the one or more target objects in at least one image frame in the video stream.

Description

Queue analysis method and device

Related Application Cross Reference

This application claims the priority of the Chinese patent application with the application number 202111308427.4 and the filing date of November 05, 2021. The entire content of the Chinese patent application is hereby incorporated by reference into this application.

technical field

The embodiments of the present disclosure relate to the technical field of computer vision, and in particular to a queuing analysis method and device.

Background technique

Queuing events in life can be seen everywhere, such as queuing at the cashier counter in the supermarket, queuing for subway tickets, queuing for security checks, queuing for canteens, etc. Traditional queuing analysis methods, such as the queuing number extraction method, are not suitable for the above-mentioned various queuing scenarios, and the information about the queuing situation that traditional queuing analysis methods can obtain is limited.

Contents of the invention

According to a first aspect of the present disclosure, there is provided a queuing analysis method applied to a terminal device, the method comprising: performing target detection on one or more image frames in a video stream, and determining the one or more image frames One or more target objects in the queuing queue; track the one or more target objects in the video stream, and assign a tracking identifier to each target object in the one or more target objects; according to The tracking identification of the one or more target objects in at least one image frame in the video stream determines the queuing analysis result of the queuing queue.

According to a second aspect of the present disclosure, a queuing analysis device is provided, the device comprising: an object detection module, configured to perform target detection on one or more image frames in a video stream, and determine the one or more image frames One or more target objects in the queue in the frame; an object tracking module, configured to track the one or more target objects in the video stream, for each target in the one or more target objects Object allocation tracking identification; a result analysis module, configured to determine the queuing analysis result of the queuing queue according to the tracking identification of the one or more target objects in at least one image frame in the video stream.

According to a third aspect of the present disclosure, there is provided an electronic device, the device includes a memory and a processor, the memory is used to store computer instructions executable on the processor, and the processor is used to execute the computer The instruction implements the queuing analysis method described in any embodiment of the present disclosure.

According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the queuing analysis method described in any embodiment of the present disclosure is implemented.

According to a fifth aspect of the present disclosure, a computer program product is provided, the product includes a computer program, and when the computer program is executed by a processor, the queuing analysis method described in any embodiment of the present disclosure is implemented.

Description of drawings

In order to more clearly illustrate the technical solutions in one or more embodiments of the present disclosure, the drawings that need to be used in the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only one example of the present disclosure. Or some embodiments described in multiple embodiments, for those skilled in the art, other drawings can also be obtained according to these drawings without creative work.

Fig. 1A is a flowchart of a queuing analysis method according to at least one embodiment of the present disclosure;

Fig. 1B is a flowchart of another queuing analysis method according to at least one embodiment of the present disclosure;

Fig. 2A is a schematic diagram of a subway queuing scene according to at least one embodiment of the present disclosure;

Fig. 2B is a schematic diagram of a queuing scenario according to at least one embodiment of the present disclosure;

Fig. 2C is a statistical diagram of similarity shown according to at least one embodiment of the present disclosure;

Fig. 3 is a block diagram of a queuing analysis device according to at least one embodiment of the present disclosure;

Fig. 4 is a block diagram of another queuing analysis device according to at least one embodiment of the present disclosure;

Fig. 5 is a schematic diagram of a hardware structure of an electronic device according to at least one embodiment of the present disclosure.

Detailed ways

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numerals in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the following exemplary examples do not represent all implementations consistent with this specification. Rather, they are merely examples of apparatuses and methods consistent with aspects of the present specification as recited in the appended claims.

The terms used in this specification are for the purpose of describing particular embodiments only, and are not intended to limit this specification. As used in this specification and the appended claims, the singular forms "a", "the", and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. It should also be understood that the term "and/or" as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

It should be understood that although the terms "first", "second", "third", etc. may be used in this specification to describe various information, the information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this specification, first information may also be called second information, and similarly, second information may also be called first information. Depending on the context, the word "if" as used herein may be interpreted as "at" or "when" or "in response to a determination."

If the disclosed technical solution involves user information, the product applying the disclosed technical solution has clearly informed the user of the information processing rules and obtained the user's independent consent before processing the user information. If the disclosed technical solution involves sensitive user information, the product applying the disclosed technical solution has obtained the separate consent of the user before processing the sensitive user information, and at the same time meets the requirement of "express consent". For example, at the user information collection device such as a camera, set a clear and prominent sign to inform that the user information has entered the collection range, and the user information will be collected. If the user voluntarily enters the collection range, it is deemed to agree to the collection of his user information; or On the user information processing device, when the user information processing rules are notified with obvious signs/information, the user authorization is obtained through pop-up information or asking the user to upload their user information; among them, the user information processing rules may include user Information processor, user information processing purpose, processing method, type of user information processed and other information.

There are many queuing scenarios in life, and service providers need to collect statistics on user queuing to optimize service configuration. The usual queuing statistics management scheme mainly has two methods: 1. Use the queuing number to count the number of queuing people. This method is limited to the scene and is not flexible. It is only suitable for small-scale scene queuing. Suitable for large-scale scenes such as playgrounds and zoos. 2. The method of using mobile terminals to count the number of people in the queue requires users to actively access the network of mobile terminals, and is also limited to local spaces. For example, it is only applicable to scenarios such as toilets and closed venue passages, and because there People queuing can also access the network of mobile terminals, so the statistical accuracy is not high, and the queuing information that can be obtained is also limited.

In view of this, at least one embodiment of the present disclosure provides a queuing analysis method. When performing queuing analysis on the queuing queue, the method analyzes the video stream in the queuing scene, so that it is not limited by the queuing scene. , you can control the queued information.

Fig. 1A is a flowchart showing a queuing analysis method according to at least one embodiment of the present disclosure, and the method may include steps 102 to 106.

In step 102, target detection is performed on one or more image frames in the video stream, and one or more target objects in the queue in the one or more image frames are determined.

In this embodiment, the video stream includes a plurality of image frames collected from the queuing scene, and the video stream may be obtained by real-time monitoring of the queuing queue, or may be a recorded video of the queuing queue.

In this step, performing object detection on the image frame may include performing object detection on the entire image frame, or may include performing object detection on the marked queuing area in the image frame.

This embodiment does not limit the manner of detecting the image frame in the video stream, for example, it may be detected by a neural network, or may be detected by other methods.

In step 104, the one or more target objects are tracked in the video stream, and a tracking identifier is assigned to each of the one or more target objects.

Wherein, the tracking identifier is used to mark the same target object in different image frames. The same target object may exist in different video frames, and one or more target objects in multiple image frames may be tracked to determine the position of the same target object in different image frames and mark the target object with a tracking mark.

This embodiment does not limit the method used for tracking the target object. For example, the kalman (Kalman) filter tracking algorithm, the tracking algorithm based on siamsRPN (visual target tracking network), etc. can be used to track the target object.

In step 106, a queuing analysis result of the queuing queue is determined according to the tracking identifiers of the one or more target objects in at least one image frame in the video stream.

The queuing analysis result may include a preliminary analysis result obtained by analyzing each image frame, and may also include a result obtained by further summarizing the preliminary analysis results.

For example, the queuing analysis result can include the number of people in the queue (the number of target objects in the queue), and according to the number of tracking marks of the target object in the image frame, the number of people in the queue at the acquisition time corresponding to the image frame can be determined . Moreover, by further analyzing the number of people in the queue in multiple image frames of the video stream, the number of people in the queue in different time periods can be obtained, so as to know the peak value, peak time, valley value, valley time and other information of the queue number.

The queuing analysis method provided by the technical solution of the embodiment of the present disclosure uses the video stream in the queuing scene to track and analyze the target objects in the queuing queue, and can obtain richer queuing information, which is more practical and is not limited by the queuing scene. Various queuing scenarios have relatively good adaptability, and can control queuing information in order to optimize resource allocation such as manpower and material resources for queuing objects, thereby greatly improving service efficiency and reducing costs.

Fig. 1B is a flow chart of a queuing analysis method provided according to at least one embodiment of the present disclosure. The method combines the subway queuing scene shown in Fig. 2A and describes in more detail how to perform queuing analysis by detecting image frames in a video stream process. As shown in FIG. 1B , the method may include steps 202 to 208 , where it should be noted that, this embodiment does not limit the execution order of the steps.

In step 202, object detection is performed on any one of the one or more image frames in the video stream to obtain detection frames of each object in the image frame.

In this step, an object detection network may be used to detect the image frames in the video stream. The object detection network is a pre-trained neural network for detecting objects, and correspondingly obtains the detection frame of each object in the image frame. In other implementation manners, the head key point detection network may also be used to detect the image frames in the video stream, and correspondingly obtain the detection frames of the heads of the objects in the image frames. Alternatively, detection frames of other parts of the object may also be detected, for example, human feet, human legs, car wheels, animal feet, and the like.

The image frame is input to the object detection network, and the detection frame of each object in the image frame is output. The detection frame contains coordinate information, which is used to represent the position of the object.

In step 204, in response to detecting that the detection frame is in the preset queuing area in the image frame, the object corresponding to the detection frame is determined as the target object in the queuing queue.

The position and size of the queuing area can be delineated in advance in the video images collected by the camera near the subway ticket vending machine. In particular, for a scene where there may be multiple queuing queues, the queuing area in the queuing area that needs to be analyzed can be designated by delimiting the queuing area in the image frame of the video stream.

Generally speaking, since the viewing angle position of the camera that collects the video stream in the queuing scene is fixed, it is sufficient to pre-mark the queuing area only once for the video stream collected by the camera. You can also choose one or several sides of the queuing area and indicate the direction of the queue.

In this embodiment, the preset queuing area in the image frame may be an area in a quadrangular frame composed of black lines as shown in FIG. 2A , and the direction of leaving the queue is shown by the arrow. It can also be the area in the box formed by the black lines shown in Figure 2B, and the direction of leaving the team is shown by the arrow.

In this step, whether the object corresponding to the detection frame is the target object in the queuing queue can be judged by whether the feature points in the detection frame are in the queuing area. In other examples, it can also be determined according to whether the edge of the detection frame is in the queuing area. or the degree of overlap between the detection frame and the queuing area.

The feature point is a point in the inner area of the detection frame or a point on the edge of the detection frame. This embodiment does not limit the selection of the feature point. For example, the feature point can be the midpoint position of the lower edge of the detection frame, as shown at the bottom of the detection frame in Figure 2A. Indicated by the white circle. Feature points can also be located in the lower left corner or lower right corner of the detection frame.

For the detection frame whose feature point is located in the queuing area, it is determined that the target object corresponding to the detection frame is also located in the queuing area, that is, the target object is in the queuing queue.

In step 206, one or more target objects are tracked in the video stream, and a tracking identifier is assigned to each of the one or more target objects.

It should be noted that, for a target object that is in the queuing queue in a certain image frame, in other image frames, the target object may not be in the queuing queue at this time because the queuing has not yet started, but the target object is in the other image frame. The area of an image frame outside the queuing queue or outside the queuing area. Therefore, when tracking the target object, the detection frame of the target object in the queuing area and the detection frame of the surrounding objects in the queuing area can be tracked. The same object in the image frame at the moment is associated, and then a tracking identifier (trackID) is assigned to each object.

The tracking identifiers can be represented by numbers, for example, the numbers 0-6 in Figure 2A mark seven target objects in the queuing queue. Exemplarily, the tracking identifier can be determined according to the chronological order in which the target object appears in the frame of the video stream, for example, the target object 1 marked by the tracking identifier 1 can be earlier than the target object 5 marked by the preceding tracking identifier 5 To the queuing scene, but target object 1 starts queuing after target object 5 enters the queuing queue.

The position of the target object marked by the tracking mark may use the position of the feature point, the position of the center of the detection frame, or the position of other points on the detection frame.

In step 208, a queuing analysis result of the queuing queue is determined according to the tracking identifiers of the one or more target objects in at least one image frame in the video stream.

By sequentially analyzing the changes in the tracking marks in the queuing area at each moment, information such as the position, serial number, and real-time waiting time of each target object in the queuing queue at each moment can be obtained. Analyzing and counting these information can Get the queuing analysis result.

Exemplarily, the queuing analysis result may include the queuing waiting time and service usage time of each target object, the number of target objects in the queuing queue at each moment, and the average queuing time and service usage time.

For example, for any image frame in which the target object marked by the tracking mark is at the end of the queuing queue, the queuing start time of the target object marked by the tracking mark can be determined; For the image frame at the head of the queue, determine the queuing end time of the target object marked by the tracking identifier; determine the queuing waiting time of the target object according to the queuing start time and queuing end time of the target object.

In some embodiments, when calculating the queuing waiting time of the target object, for a plurality of image frames in which the target object marked by any tracking identifier is at the end of the queuing queue, the earliest of the plurality of image frames may be The acquisition time is determined as the queue start time of the target object marked by the tracking identifier.

For the plurality of image frames in which the target object marked by the tracking identifier is at the head of the queue, the earliest acquisition time among the plurality of image frames is determined as the queuing end time of the target object marked by the tracking identifier.

The queuing waiting time of the target object is determined according to the queuing start time of the target object and the queuing end time of the target object.

For example, the target object at the head of the queue is generally located at the end of the direction of leaving the queue, that is, the target object at the end of the direction pointed by the arrow is using the services provided by the subway ticket vending machine, such as ticket collection, ticket purchase, or query service. An object can be thought of as a target object at the head of a queuing queue. On the contrary, the target object at the end of the queuing queue is generally located at the other end of the queue, for example, the target object newly entered the queuing queue is at the end of the queuing queue.

According to the position of the tracking mark of a certain target object, it can be determined that the target object is located at the end of the queue, and the image frame with the earliest acquisition time is when the target object marked by the tracking mark just entered the queue The image frame of the image frame, the acquisition time of the image frame is recorded as the queuing start time of the target object.

Similarly, according to the position of the tracking mark of the target object, it can also be determined that the target object is located in a plurality of image frames at the head of the queue, and the image frame with the earliest collection time is the target object marked by the tracking mark and has just entered The image frame when the leader of the queuing queue is about to start using the subway ticket vending machine. The acquisition time of the image frame is recorded as the queuing end time of the target object, or the service start time of the target object.

The queuing start time and queuing end time of the target object are subtracted to obtain the queuing waiting time of the target object.

In addition, based on the calculated queuing waiting time of multiple target objects in the queuing queue, the queuing waiting time of the target object that has just entered the queuing queue can be estimated, for example, the average queuing waiting time of multiple target objects can be used as the estimate The queuing waiting time can also be estimated according to the equation fitted by the relationship between the queuing waiting time of multiple target objects and the number of people waiting in line.

For another example, for any one of the tracking identifiers, a plurality of image frames in which the target object marked by the tracking identifier is at the head of the queue is determined.

Determining the earliest acquisition time among the plurality of image frames as the service start time of the target object marked by the tracking identifier.

Based on detecting that the target object marked by the tracking mark leaves the queuing queue, the service end time of the target object marked by the tracking mark is determined. For example, when the target object appears at the head of the queue for the last time, it may be considered that the target object has been detected to leave the queue, and the latest acquisition time among the plurality of image frames is determined as the target object marked by the tracking identifier. Service end time.

The service usage time of the target object is determined according to the service start time and service end time of the target object.

The method for calculating the service usage time is similar to the method for calculating the queue waiting time in the above example, and the queue end time of the target object can be used as the service start time of the target object.

A plurality of image frames in which the target object is located at the head of the queue determined according to the position of the tracking mark of the target object, wherein the image frame with the latest collection time is that the target object marked by the tracking mark has used up the subway ticket vending machine, When the image frame is ready to leave the queuing queue, the acquisition time of the image frame is recorded as the service end time of the target object.

The service usage time of the target object can be obtained by subtracting the service start time and the service end time of the target object.

For each target object, the above-mentioned queuing waiting time and service usage time can be calculated, and the average queuing time and service usage time can be further obtained.

For another example, in response to the fact that the number of tracking markers in adjacent image frames in the video stream is different, and the target object marked by the tracking marker added in the image frame after the acquisition time is not located in the queue of the queuing queue At the end, it is determined that the target object in the queuing queue is queued.

For two adjacent image frames in the video stream, when the number of tracking identifiers in the queuing queue is different, usually there is a target object newly entering the queuing queue, or a target object newly leaving the queuing queue. Normally, the target object newly entering the queuing queue should be at the end of the queuing queue, if the target object marked by the new tracking mark in the image frame after the acquisition time is not at the end of the queuing queue, then determine the new The target object that enters the enqueuing queue jumps into the queue.

The queuing analysis method provided by the technical solution of the embodiment of the present disclosure can mark the position and size of the queuing area in the image frame in the video stream, track and analyze the target objects in the queuing queue in the queuing area, and can be applied to various types of queuing Scenarios, flexibly configure the queuing areas that need to be analyzed, and control queuing information in a targeted manner. It has better adaptability to various queuing scenarios, so that service operators can optimize the allocation of resources such as human and material resources for queuing objects, thereby greatly Improve service efficiency and reduce costs.

In the above embodiment, when judging whether the target object at the head of the queuing queue is replaced, that is, when judging whether the target object has just entered the head of the queuing queue or just left the head of the queuing queue, the judgment is made based on the position of the tracking mark of. However, sometimes the tracking identification at the head of the queuing queue may jump, for example, when two target objects are very close together, or when the result of object tracking is inaccurate, the method in the above-mentioned embodiment It is possible to make a wrong judgment on whether the target object of the queue head has changed, which in turn affects the accuracy of the queuing analysis results.

In one embodiment, on the basis of the above-mentioned embodiments, the present disclosure provides a queuing analysis method to make the queuing analysis result more accurate. When the method judges whether the target object at the head of the queue leaves, the A method for comparing the ReID (Person re-identification, pedestrian re-identification) feature of the target object at the head of the queue, after step 104 or step 206 of the above-mentioned embodiment, the method of the above-mentioned embodiment also includes: extracting the video The first feature information of the first target object in the first image frame in the stream, and extract the second feature information of the second target object in the second image frame adjacent to the first image frame, and compare the first The characteristic information and the second characteristic information, in response to the similarity between the first characteristic information and the second characteristic information being less than a similarity threshold, it is determined that the first target object leaves the queuing queue.

Wherein, the first target object is the target object at the head of the queuing queue in the first image frame, and the second target object is the target object at the head of the queuing queue in the second image frame, and the first image frame and the second image The frames are adjacent image frames in the video stream, and the acquisition time of the first image frame is before the acquisition time of the second image frame.

The first feature information and the second feature information may include the information of the ReID feature of the target object, and the ReID feature may include features of various types of attributes of the target object, such as attributes of clothing, posture, hairstyle, and limbs of the human body.

Generally speaking, the target object in the queuing queue includes three states: entering the queue, in the queue, and exiting the queue, and in special cases, it also includes the state of cutting the queue.

Among them, entering the queue: for each new tracking identifier that appears at the end of the queue, we consider it to be a target object that newly enters the queue.

In the queue: the target object that persists in the queuing queue, we think that this target object has been queuing in the queuing queue.

Dequeue: The target object that leaves the queuing queue. This is very important for the analysis and judgment of the queuing queue. Since the change of the tracking logo may jump, we need to use the ReID feature of the target object at the head of the line in the previous frame of the video stream and the target at the head of the line in the current frame. The ReID feature of the object is compared to determine whether the target object at the head of the queue in the previous frame is out of the queue.

In some embodiments, the ReID feature of the target object at the head of the line in each image frame of the video stream can be extracted using the pedestrian re-identification technology, and the ReID feature of the target object at the head of the line in two adjacent image frames can be compared , to obtain the similarity between the two, in other words, comparing the first feature information of the first target object in the first image frame with the second feature information of the second target object in the second image frame, the greater the similarity between the two , then the possibility that the first target object and the second target object are the same target object is higher, and vice versa, the possibility that the first target object and the second target object are the same target object is lower.

Exemplarily, for a certain video stream, the result of comparing the similarity between each image frame and the adjacent previous image frame is shown in Figure 2C. In Figure 2C, the abscissa is the serial number of the image frame, "0" indicates the starting point of the video stream, "500" indicates the 500th image frame in the video stream, "2000" indicates the 2000th image frame in the video stream, and the ordinate is the similarity, the closer the similarity is to "1.0", the more the team leader in two adjacent image frames is the same person.

The similarity threshold can be set to 0.5. If the similarity is less than 0.5, it is considered that the target object at the head of the team in two adjacent image frames is not the same person. The first target object at the head of the queue in the previous frame has left the queue at the time corresponding to the current frame, the acquisition time corresponding to the image frame of the previous frame can be determined as the service end time of the first target object, and the image frame of the current frame The corresponding collection time is determined as the service start time of the second target object, or the queue end time of the second target object.

Similar to the method for judging that the first target object leaves the queuing queue, it can also be judged that the second target object leaves the queuing queue at the acquisition time corresponding to a certain image frame.

In response to determining that the second target object leaves the queuing queue, the service usage time corresponding to the second target object is determined by subtracting the time when the first target object leaves the queuing queue and the time when the second target object leaves the queuing queue.

Similarly, the queuing service time of the target object can also be calculated according to the queuing end time of the target object determined in the method of this embodiment combined with the queuing start time of the target object.

In this embodiment, the method of ReID feature comparison is used to judge whether the target object at the head of the queue has left, which effectively avoids the wrong judgment of whether the target object is out of the queue caused by inaccurate tracking, and improves the accuracy of queuing analysis. When analyzing the queuing area, make more accurate statistics on the queuing waiting time and service usage time, so as to better improve the queuing experience of the target object based on the queuing analysis results.

Fig. 3 is a block diagram showing a queuing analysis device according to at least one embodiment of the present disclosure, the device includes an object detection module 31, an object tracking module 32, and a result analysis module 33.

Wherein, the object detection module 31 is configured to perform target detection on one or more image frames in the video stream, and determine one or more target objects in the queue in the one or more image frames.

The object tracking module 32 is configured to track the one or more target objects in the video stream, assign a tracking identifier to each target object in the one or more target objects, and the tracking identifier is used for marking The same target object in different said image frames.

The result analysis module 33 is configured to determine the queuing analysis result of the queuing queue according to the tracking identifiers of the one or more target objects in at least one image frame in the video stream.

In one example, the human body detection module 31 is configured to: perform target detection on one or more image frames in the video stream to obtain the detection frame of each object in the image frame; In the preset queuing area in the image frame, the object corresponding to the detection frame is determined as the target object in the queuing queue.

In one example, the queuing analysis result includes the number of the one or more target objects in the queuing queue; the result analysis module 33 is configured to: according to the tracking identification in one image frame in the at least one image frame in the video stream The number determines the number of target objects in the queuing queue at the acquisition time corresponding to the image frame.

In one example, the queuing analysis result includes queuing waiting time; the result analysis module 33 is configured to: for any image frame in which the target object marked by any tracking mark is at the end of the queuing queue, determine the target object marked by the tracking mark The starting time of queuing; for the image frame in which the target object marked by the tracking mark is at the head of the queuing queue, determine the queuing end time of the target object marked by the tracking mark; according to the queuing start time of the target object and the target The queuing end time of the object determines the queuing waiting time of the target object.

In one example, the queuing analysis result includes service usage time; the result analysis module 33 is configured to: for any one of the tracking markers, determine a plurality of image frames in which the target object marked by the tracking marker is at the head of the queuing queue; Determining the earliest acquisition time among the plurality of image frames as the service start time of the target object marked by the tracking mark; based on detecting that the target object marked by the tracking mark leaves the queuing queue, determining the target marked by the tracking mark The service end time of the object; according to the service start time of the target object and the service end time of the target object, the service usage time of the target object is determined.

In one example, the result analysis module 33 is configured to: respond to the number of tracking identifiers of the one or more target objects in adjacent image frames in the video stream being different, and the image frame whose acquisition time is later The target object marked by the newly added tracking identifier in is not located at the end of the queuing queue, and it is determined that the target object marked by the newly added tracking identifier in the queuing queue is inserted into the queue.

In one example, as shown in FIG. 4 , the device further includes: a feature comparison module 34 .

extracting first feature information of a first target object in a first image frame in the video stream, and extracting second feature information of a second target object in a second image frame adjacent to the first image frame; Wherein, the first target object is the target object at the head of the queuing queue in the first image frame, and the second target object is the head of the queuing queue in the second image frame the target object; in response to the similarity between the first characteristic information and the second characteristic information being less than a similarity threshold, determining that the first target object leaves the queuing queue.

In one example, the result analysis module 33 is configured to: in response to determining that the second target object leaves the queuing queue, according to the time when the first target object leaves the queuing queue and the time when the second target object leaves the queuing queue , to determine the service usage time corresponding to the second target object.

For the implementation process of the functions and effects of each module in the above-mentioned device, please refer to the implementation process of the corresponding steps in the above-mentioned method for details, and details will not be repeated here.

An embodiment of the present disclosure also provides an electronic device. As shown in FIG. The device 52 is configured to implement the queuing analysis method described in any embodiment of the present disclosure when executing the computer instructions.

An embodiment of the present disclosure further provides a computer program product, which includes a computer program/instruction, and when the computer program/instruction is executed by a processor, implements the queuing analysis method described in any embodiment of the present disclosure.

An embodiment of the present disclosure also provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the queuing analysis method described in any embodiment of the present disclosure is implemented.

As for the device embodiment, since it basically corresponds to the method embodiment, for related parts, please refer to the part description of the method embodiment. The device embodiments described above are only illustrative, and the modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in One place, or it can be distributed to multiple network modules. Part or all of the modules can be selected according to actual needs to achieve the purpose of the solution in this specification. It can be understood and implemented by those skilled in the art without creative effort.

The foregoing describes specific embodiments of this specification. Other implementations are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. Multitasking and parallel processing are also possible or may be advantageous in certain embodiments.

Other embodiments of the specification will be readily apparent to those skilled in the art from consideration of the specification and practice of the disclosure. This description is intended to cover any modification, use or adaptation of this description. These modifications, uses or adaptations follow the general principles of this description and include common knowledge or conventional technical means in this technical field for which this description does not apply . The specification and examples are to be considered exemplary only, with a true scope and spirit of the specification being indicated by the following claims.

It should be understood that this specification is not limited to the precise constructions which have been described above and shown in the accompanying drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the specification is limited only by the appended claims.

The above descriptions are only some examples of this specification, and are not intended to limit this specification. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of this specification shall be included in the protection of this specification. within the range.

Claims

A queuing analysis method applied to a terminal device, comprising:

Perform target detection on one or more image frames in the video stream, and determine one or more target objects in the queue in the one or more image frames;

Tracking the one or more target objects in the video stream, and assigning a tracking identifier to each of the one or more target objects;

A queuing analysis result of the queuing queue is determined according to the tracking identifiers of the one or more target objects in at least one image frame in the video stream.
The method according to claim 1, wherein performing target detection on one or more image frames in the video stream, and determining the one or more target objects in the queue in the image frame, comprising:

performing target detection on any one of the one or more image frames to obtain a detection frame of each object in the image frame;

In response to detecting that the detection frame is in the preset queuing area in the image frame, the object corresponding to the detection frame is determined as the target object in the queuing queue.
The method according to claim 1 or 2, wherein said queuing analysis result comprises the number of said one or more target objects of said queuing queue;

Determining a queuing analysis result of the queuing queue according to the tracking identifiers of the one or more target objects in at least one image frame in the video stream, including:

According to the number of tracking markers in one image frame of the at least one image frame, the number of target objects located in the queuing queue at the acquisition time corresponding to the image frame is determined.
The method according to any one of claims 1-3, wherein said queuing analysis result comprises queuing waiting time;

Determining a queuing analysis result of the queuing queue according to the tracking identifiers of the one or more target objects in at least one image frame in the video stream, including:

For any image frame in which the target object marked by the tracking mark is at the end of the queuing queue, determine the queuing start time of the target object marked by the tracking mark;

For the image frame in which the target object marked by the tracking mark is at the head of the queuing queue, determine the queuing end time of the target object marked by the tracking mark;

The queuing waiting time of the target object is determined according to the queuing start time of the target object and the queuing end time of the target object.
The method according to any one of claims 1-4, wherein the queuing analysis result includes service usage time;

Determining a queuing analysis result of the queuing queue according to the tracking identifiers of the one or more target objects in at least one image frame in the video stream, including:

For any of the tracking marks, determine a plurality of image frames in which the target object marked by the tracking mark is at the head of the queuing queue;

Determining the earliest acquisition time among the plurality of image frames as the service start time of the target object marked by the tracking identifier;

determining the service end time of the target object marked by the tracking mark based on detecting that the target object marked by the tracking mark leaves the queuing queue;

The service usage time of the target object is determined according to the service start time of the target object and the service end time of the target object.
The method according to any one of claims 1-5, further comprising:

extracting first feature information of a first target object in a first image frame in the video stream, and extracting second feature information of a second target object in a second image frame adjacent to the first image frame; Wherein, the first target object is the target object at the head of the queuing queue in the first image frame, and the second target object is the head of the queuing queue in the second image frame target audience;

In response to the similarity between the first feature information and the second feature information being less than a similarity threshold, it is determined that the first target object leaves the queuing queue.
The method of claim 6, further comprising:

In response to determining that the second target object leaves the queuing queue, determining the second target object based on the time when the first target object left the queuing queue and the time when the second target object left the queuing queue The service usage time corresponding to the object.
The method according to any one of claims 1-7, wherein the queuing analysis result of the queuing queue is determined according to the tracking identifiers of the one or more target objects in at least one image frame in the video stream, include:

In response to the fact that the number of tracking identifiers of the one or more target objects in adjacent image frames in the video stream is different, and the target object marked by the new tracking identifier in the image frame after the acquisition time is not located The queue tail of the queuing queue determines that the target object marked by the newly added tracking identifier in the queuing queue jumps into the queue.
A queuing analysis device, comprising:

An object detection module, configured to perform target detection on one or more image frames in the video stream, and determine one or more target objects in the queue in the one or more image frames;

An object tracking module, configured to track the one or more target objects in the video stream, and assign a tracking identifier to each of the one or more target objects;

The result analysis module is configured to determine the queuing analysis result of the queuing queue according to the tracking identifiers of the one or more target objects in at least one image frame in the video stream.
An electronic device, comprising a memory and a processor, the memory is used to store computer instructions that can be run on the processor, and the processor is used to implement any one of claims 1 to 8 when executing the computer instructions method.
A computer program product, the product includes a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is realized.
A computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the method according to any one of claims 1 to 8 is realized.