WO2023077783A1

WO2023077783A1 - Method and apparatus for determining queuing waiting time

Info

Publication number: WO2023077783A1
Application number: PCT/CN2022/095777
Authority: WO
Inventors: 刘诗男; 杨昆霖; 侯军; 伊帅
Original assignee: 上海商汤智能科技有限公司
Priority date: 2021-11-05
Filing date: 2022-05-27
Publication date: 2023-05-11
Also published as: CN114037146A

Abstract

A method and apparatus for determining queuing waiting time, the method comprises: obtaining a queue video stream of a target region, where the target region includes a queuing region and a dequeuing region, and the dequeuing region includes a dequeuing line which needs to be passed to end queuing and is used to determine whether queuing has ended; identifying the dequeuing region in the queue video stream, tracking objects in the dequeuing region, determining a number of dequeuing objects that has crossed the dequeuing line in a unit time, and obtaining a dequeuing speed; obtaining a number of queuing objects in the queuing region from a target image frame of the queue video stream; obtaining a queuing waiting time on the basis of the dequeuing speed and the number of queuing objects.

Description

Method and device for determining waiting time in queue

Related Application Cross Reference

This application claims the priority of the Chinese patent application with the application number 202111308447.1 and the filing date of November 05, 2021. The entire content of the Chinese patent application is hereby incorporated by reference into this application.

technical field

This specification relates to the field of computer vision technology, and in particular to a method and device for determining the waiting time in a queue.

Background technique

In real life, for better management, the waiting time in the queue will be estimated so that the queue objects can arrange their time reasonably.

Generally, the head tracking method is used to estimate the waiting time in the queue. However, in a scene with dense heads (such as a serpentine formation), due to the increase in the difficulty of head tracking, it is difficult to guarantee the accuracy of tracking, and the accuracy of the estimated waiting time is also reduced.

Contents of the invention

According to the first aspect of the embodiments of this specification, there is provided a method for determining the waiting time in queuing, the method comprising: acquiring the queuing video stream of the target area, wherein the target area includes a queuing area and an out-of-queue area, and the out-of-queue The area includes the exit line that needs to pass through to end the queuing and judge whether to end the queuing; identify the exit area in the queued video stream, track the objects in the exit area, and determine the line that crosses the line in the unit time. Describe the quantity of dequeuing objects of the dequeuing line to obtain the dequeue speed; obtain the quantity of queuing objects in the queuing area from the target image frame of the queuing video stream; according to the dequeue speed and the number of queuing objects, Get the waiting time in line.

According to the second aspect of the embodiment of this specification, there is provided a device for determining the waiting time in queuing, the device includes: an acquisition module, configured to acquire the queuing video stream of the target area, wherein the target area includes at least a queuing area and a dequeue Area, the dequeue area includes the dequeue line that needs to pass through to end the queuing, and judges whether to end the queuing; the dequeue speed determination module is used to identify the dequeue area in the queued video stream, and for the dequeue Objects in the area are tracked to determine the number of dequeue objects crossing the dequeue line per unit time to obtain the dequeue speed; the number of queuing objects is determined by a module for obtaining the queuing from the target image frame of the queuing video stream The number of queuing objects in the area; the queuing waiting time determination module, configured to obtain the queuing waiting time according to the dequeue speed and the number of queuing objects.

According to a third aspect of the present specification, there is provided a computer device, including a memory, a processor, and a computer program stored on the memory and operable on the processor, wherein, when the processor executes the program, the above-mentioned Any one of the methods for determining the waiting time in queue.

According to the fourth aspect of the present specification, there is provided a computer-readable storage medium, on which computer instructions are stored, and when the instructions are executed by a processor, the method for determining the queuing waiting duration as described above is implemented.

According to the fifth aspect of the present specification, a computer program product is provided, including a computer program stored in a memory, and when the computer program instructions are executed by a processor, the method for determining the queuing waiting time as described above is implemented.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the specification.

Description of drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the specification and together with the description serve to explain the principles of the specification.

Fig. 1 is a schematic diagram of a scene according to an exemplary embodiment of this specification.

Fig. 2 is a schematic flowchart of a method for determining the waiting time in a queue according to an exemplary embodiment of the present specification.

3A and 3B are schematic diagrams of a queuing area scene according to an exemplary embodiment of the present specification, and FIG. 3B is a schematic diagram of a queuing area scene including object distribution.

Fig. 4 is a schematic diagram showing another queuing area scene according to an exemplary embodiment of the present specification.

5A and 5B are schematic diagrams of another queuing area scene according to an exemplary embodiment of the present specification, and FIG. 5B is a schematic diagram of a queuing area scene including object distribution.

Fig. 6 is a schematic diagram showing object positioning points according to an exemplary embodiment of the present specification.

Fig. 7 is a schematic diagram of a scene of a multi-camera instrument according to an exemplary embodiment of the present specification.

Fig. 8 is a schematic flowchart of another method for determining the waiting time in a queue according to an exemplary embodiment of the present specification.

Fig. 9 is a schematic block diagram of a device for determining a queue waiting time according to an exemplary embodiment of the present specification.

Fig. 10 is a schematic diagram of a computing device according to an exemplary embodiment of this specification.

Detailed ways

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numerals in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the following exemplary examples do not represent all implementations consistent with this specification. Rather, they are merely examples of apparatuses and methods consistent with aspects of the present specification as recited in the appended claims.

The terms used in this specification are for the purpose of describing particular embodiments only, and are not intended to limit the specification. As used in this specification and the appended claims, the singular forms "a", "the", and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. It should also be understood that the term "and/or" as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

It should be understood that although the terms first, second, third, etc. may be used in this specification to describe various information, the information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this specification, first information may also be called second information, and similarly, second information may also be called first information. Depending on the context, the word "if" as used herein may be interpreted as "at" or "when" or "in response to a determination."

If the disclosed technical solution involves user information, the product applying the disclosed technical solution has clearly informed the user of the information processing rules and obtained the user's independent consent before processing the user information. If the disclosed technical solution involves sensitive user information, the product applying the disclosed technical solution has obtained the separate consent of the user before processing the sensitive user information, and at the same time meets the requirement of "express consent". For example, at the user information collection device such as a camera, set a clear and prominent sign to inform that the user information has entered the collection range, and the user information will be collected. If the user voluntarily enters the collection range, it is deemed to agree to the collection of his user information; or On the user information processing device, when the user information processing rules are notified with obvious signs/information, the user authorization is obtained through pop-up information or asking the user to upload their user information; among them, the user information processing rules may include user Information processor, user information processing purpose, processing method, type of user information processed and other information.

In real life, in many cases, it is necessary to estimate the waiting time in line so that the queuing objects can arrange their time reasonably. For example, there are many entertainment items in the amusement park, and the manager of the amusement park needs to use the terminal device to estimate the waiting time of each item. Make an estimate, and then push the waiting time in line to the tourists entering the amusement park, so that tourists can reasonably arrange the order of play according to the waiting time in line for each item. In other embodiments, it may be necessary to estimate the waiting time of vehicles in the vehicle congestion area, such as at the entrance and exit of the expressway. During holidays, vehicles are congested to a certain expressway entrance and exit, and the driver may want to know the queuing time at a certain expressway intersection. Therefore, a request is made to the server to know the waiting time of the vehicle in line.

When estimating the queuing time, it is generally necessary to track each queuing object, time the queuing time of each queuing object, and then estimate the queuing time of new queuing objects based on the queuing time of each queuing object duration.

This method for determining the waiting time in queue needs to track each queuing object. Therefore, in the case of dense queuing objects (such as serpentine queuing), the tracking difficulty will increase. On the one hand, the queuing objects are dense and need to be tracked. The number of queuing objects is large, and the difficulty will increase. On the other hand, for each queuing object, it takes a long time to track, and the probability of tracking failure and tracking error will increase. The accuracy of estimating the queuing waiting time of queuing objects based on this method for determining the queuing waiting time will be greatly reduced.

Based on this, this specification provides a method for determining the waiting time in a queue, which is to track the head of the queued objects in the area near the queue (ending the queue), determine the number of queued objects per unit time, and obtain the queued speed. At the same time, the number of queuing objects in the queuing area is counted to obtain the number of queuing objects. According to the dequeue speed and the number of queued objects, the queue waiting time is obtained.

Through one or more embodiments of this specification, it is only necessary to track the queuing objects in the dequeue area that is about to end queuing. On the one hand, since the tracking area is reduced and the number of tracking targets is reduced, the tracking difficulty will be reduced, making the calculation Resource consumption will also be reduced. On the other hand, since the duration of tracking the target is shortened, the probability of tracking failure and error is reduced, so the tracking accuracy is improved, and the estimated waiting time in line is more accurate.

Next, the method for determining the queuing waiting time proposed in this specification will be described in detail.

It should be noted that the method for determining the queuing waiting time provided in this manual is for the person who is about to enter the queuing area and become the last queuing object. The estimate of , that is, the estimate of the waiting time in line.

Firstly, the application scenario of this manual is described. As shown in FIG. 1 , it is a schematic diagram of a scene shown in this specification, 101 is a camera, and an area 102 is an area where queuing objects line up. Of course, the actual queuing video stream can also be obtained through other means, such as a mobile phone with a camera.

Then the camera device uses the queued video stream to estimate the queued waiting time, or the camera device sends the queued video stream to a certain terminal device, and the terminal device uses the queued video stream to estimate the queued waiting time.

Wherein, the method for determining the waiting time in line can be performed by electronic equipment such as a terminal device or a server, and the terminal device can be a fixed terminal or a mobile terminal, such as a mobile phone, a tablet computer, a game console, a desktop computer, an advertising machine, an all-in-one machine, a vehicle-mounted terminal, etc. terminal, unmanned aerial vehicle, aircraft, etc., the server includes a local server or a cloud server, etc., and the method can also be realized by calling a computer-readable instruction stored in a memory by a processor.

As shown in FIG. 2 , FIG. 2 is a schematic flowchart of a method for determining the waiting time in a queue according to an exemplary embodiment of the present specification, including steps 201 to 207 .

Step 201, acquire queued video streams in the target area.

Wherein, the target area includes the queuing area and the dequeue area, and the dequeue area includes the dequeue line that needs to pass through to end the queuing and judge whether to end the queuing.

Queued video streams can be acquired in real time or from historical video streams.

Step 203: Identify the dequeue area in the queuing video stream, track the objects in the dequeue area, determine the number of dequeue objects crossing the dequeue line per unit time, and obtain the dequeue speed.

Wherein, the out-of-queue area is an area range including the out-of-queue line.

The exit line is the line that the queuing object needs to pass through when it ends (completes) the queuing. It is used to judge whether any queuing object ends the queuing line. It is generally calibrated according to the shooting angle and the actual queuing situation, or generated by the computing system.

The dequeuing area is the area where object tracking needs to be performed, so it needs to include the queuing area where the queuing object is about to end queuing and the non-queuing area where the queuing has just ended.

The method shown in this description needs to track the head in the dequeue area, determine the number of dequeue objects crossing the dequeue line per unit time, and obtain the dequeue speed. Therefore, in order to determine whether the queued object is dequeue Object, the dequeue area for head tracking needs to include the dequeue line.

In some embodiments, the dequeuing area may also include a queuing end area and a non-queuing area, at least part of the queuing end area overlaps with the queuing area, and the dequeuing line is used as a bridge between the first queuing area and the non-queuing area. dividing line.

Wherein, the last queuing area refers to an area where queuing is about to end, as shown in FIG. 3A and FIG. 4 , the area 302 can be used as the last queuing area.

The method shown in this manual only needs to track the people located in the dequeue area. Specifically, when a person enters the dequeue area, he is assigned a tracking ID and tracked. When he leaves the dequeue area, Determines to end tracking of the person, deleting the tracking ID assigned to the person. For methods of object tracking, refer to specific implementations of related methods, such as head tracking, body tracking, etc., which will not be described in detail in this specification.

Among them, the queued video stream obtained is a queued video stream with a relatively fixed viewing angle. In other words, the queued video stream will not change the viewing angle in a short time. The queuing area, dequeue area, and dequeue line.

The exit area and exit line can be determined according to the actual scene. The location of the camera equipment that shoots the queuing video stream relative to the queue area is different, and the obtained exit area and line will be different. For example, the exit line, exit The team area may be in the upper left part of the image frame, may also be in the lower right part of the image frame, and may also be in the middle part of the image frame, etc.

As shown in Figure 3A and Figure 4, it is a schematic diagram of two scenes shown in this specification, the queue line and the queue area in Figure 3A are in the middle of the right side of the image frame, and the queue line and the queue area in Figure 4 are in the middle of the image frame Middle left. Wherein, FIG. 3B is the same as the area schematic diagram shown in FIG. 3A , and an example of objects is added in FIG. 3B , the direction of the dotted line is the direction of queuing, and each black circle represents a queuing object.

When determining the dequeued object, the following method may be used to determine, including step 2031 to step 2032 .

Step 2031, track the objects in the dequeuing area.

Step 2032: Determine the dequeue object based on the movement trajectory obtained by tracking the objects in the dequeue area.

Wherein, the dequeuing object is determined according to the movement trajectory of the object moving from the queuing area across the dequeuing line to the non-queuing area.

When performing object tracking, such as head tracking, in continuous multi-frame images, the position of the head of the same person in different frames of images will be different. According to the timing, the movement of the head of the object can be obtained The trajectory, the moving trajectory line will have a direction, pointing according to the time sequence, and the moving trajectory line with direction will be obtained. As long as it is determined that the direction of the moving trajectory is from the queuing area to the non-queuing area, and passes through the dequeuing line, it can be determined that the object corresponding to the moving trajectory is the dequeuing object.

In some embodiments, the objects in the dequeue area may be tracked based on the dequeue area of the queued video stream, and the moving trajectory of each object may be determined respectively. When the moving trajectory of any object intersects the dequeuing line, and the moving direction of the moving trajectory is from the queuing area to the non-queuing area, the object is determined to be the dequeuing object. In other words, when the moving trajectory of any queuing object intersects with the dequeuing line, and the moving direction of the moving trajectory is from the queuing area to the non-queuing area, it can be determined that the queuing object is dequeued, and the queuing object is determined to be dequeued. For dequeuing objects.

Step 205, acquiring the number of queuing objects in the queuing area from the target image frame in the queuing video stream.

Wherein, since the number of queuing objects in the queuing area does not change within a short period of time, the target image frame is an image frame in the queuing video stream.

In this description, the queuing area refers to the area where the queuing objects are queued, and other areas are non-queuing areas, as shown in Figure 3A and Figure 4, which are schematic diagrams of the queuing area shown in this specification, and area 301 and area 302 are queuing areas, Other areas are non-queuing areas.

As shown in Figure 5A, it is a schematic diagram of another queuing area shown in this specification, wherein, area 301 and area 302 are queuing areas, area 305 is the position where the working object stands, and the working object will not move along with the queuing object. Therefore, the position where the work object stands does not belong to the queuing area, therefore, the area 305 is not included in the queuing area and belongs to the non-queuing area, so, in the scene shown in Figure 5A, the non-queuing area includes area 305, area 303 , area 304 .

As shown in Figure 5B, it is the same as the schematic diagram of the area shown in Figure 5A, and a schematic diagram of object distribution is added. The direction of the dotted line is a schematic diagram of the queuing path. This object provides the estimated queuing time, and the objects marked on the right are the objects that have just finished queuing.

The above are schematic diagrams of some queuing areas exemplified in this manual. In practical applications, the queuing area may be more complicated. The queuing area can be divided according to the angle of the shooting instrument and the actual queuing situation. For example, due to emergencies, The expansion of the queuing area, or setting up staff to stand guard in some areas, does not need to divide the area into the queuing area, or changes in the shooting angle, etc., can cause changes in the actual queuing area, thereby redefining the queuing area.

Therefore, in one or more embodiments, when it is determined that the queuing area changes (for example, the distribution of the area changes from FIG. 4 to FIG. 5A), the changed queuing area is obtained, and the changed queuing area is used as a new queuing area.

Therefore, in one or more embodiments of this specification, before step 205, the current queuing area may be determined first, and then the target image frame is obtained from the queuing video stream, and the number of queuing objects in the current queuing area is obtained according to the target image frame.

Wherein, when acquiring the number of queuing objects located in the queuing area from the target image frame in the queuing video stream, the head of the object in the target image frame can be first positioned to obtain the head points of each object in the target image frame gather. Then, in the set of head points, based on the queuing area in the target image frame, the number of head points located in the queuing area is obtained to obtain the number of queuing objects. As shown in Figure 6, it is a schematic diagram of the scene shown in this specification. Some objects are in the queuing area, and some objects are not in the queuing area. , the coordinates of B, C, and D are [(5,5), (5,20), (25,5), (25,20)], after positioning each head point in the target image frame, we get Several head point coordinate sets, if the abscissa of the head point coordinates is greater than 5 and less than 25, and the ordinate is greater than 5 and less than 20, then the head point coordinates are located in the queuing area and belong to the queuing object; otherwise, the head point coordinates The point coordinates are located in the non-queuing area and belong to the non-queuing object. Then, in the set of head points, determine the number of head point coordinates that are queued objects, and obtain the number of queued objects. A subject's head point may refer to the center point of the subject's head region.

The above is the method for determining the number of queued objects in the way of head point positioning shown in this specification. In practical applications, the head area (that is, the head area) of the target image frame can also be positioned, and any object corresponding to Head area, if the area of the head area of the object in the queuing area is greater than the preset value, then it is determined that the object is in the queuing area and determined as the queuing object (for example, the preset value set is 90%, 100% 90 and above objects whose head area is in the queuing area are determined to be in the queuing area and determined to be queuing objects).

Of course, there are other methods for determining the number of queuing objects in the queuing area, such as locating each object according to body structure, locating each object according to facial features, etc., and this description will not repeat them one by one.

Step 207: Obtain the queue waiting time according to the dequeue speed and the number of queued objects.

Among them, the queue waiting time can be obtained by dividing the number of queued objects by the dequeue speed.

For example, the dequeue speed can be v, the unit is (for example, person/hour) or (for example, person/minute), the number of queued objects is s, and the unit is (for example, 1), then the waiting time in line is t=s /v, in hours or minutes.

Assuming that the speed of leaving the queue is 2 people per minute, and the number of people in the queue is 30, then the estimated waiting time in the queue is 15 minutes.

In practical applications, there are many other factors that affect the waiting time in the queue. Therefore, the waiting time in the queue can be t=s/v+α, where α is the time spent or error by other influencing factors.

For example, a merchant needs to replace the equipment every hour, and the equipment replacement takes about two minutes. When determining the waiting time in the queue, it is necessary to determine the estimated length of the waiting time in the queue based on the time of the last equipment replacement. This needs to be considered. The time α spent by the device and so on.

In practical applications, there will be many other factors that affect the waiting time in the queue, which will not be described in detail here.

In practical applications, the queuing speed is not constant, it will change with time, and the waiting time in queuing will also change with time. Therefore, in one or more embodiments of this specification, in the schematic flowchart shown in FIG. 8 , the method for determining the waiting time in queue may include steps 801 to 807.

Step 801. Obtain queued video streams in the target area.

Wherein, the target area includes a queuing area and a dequeue area, and the dequeue area includes a dequeue line that needs to pass through to end the queuing and judge whether to end the queuing.

Step 803: Based on the multiple image frames in the queuing video stream at the first moment and the predetermined time period before the first moment, track the objects in the dequeuing area in the multiple image frames, and determine the unit time Get the dequeue speed corresponding to the first moment by the number of dequeue objects that cross the dequeue line.

Wherein, the first moment may be a certain moment corresponding to the queued video stream, but not the moment corresponding to the first image frame, and there need to be multiple image frames within a predetermined period of time before the first moment. For example, before the time 9:40, there needs to be an image frame of the previous minute, that is, there needs to be an image frame corresponding to the time period from 9:39 to 9:40.

The predetermined time period can be set according to the actual situation. For example, if the actual queuing situation is good and the speed of leaving the queue changes rapidly, the predetermined time period can be shorter; if the actual queuing situation is poor and the speed of leaving the queue changes slowly, the predetermined time period can be longer.

For example, the predetermined period of time may be two minutes, and for a certain moment, based on the image frames within the two minutes before the moment and the moment, head tracking is performed on the queued objects in the out-of-queue area in these image frames, and the unit time is determined Get the number of heads of the dequeue objects that cross the dequeue line, and get the corresponding dequeue speed at that moment.

In practical applications, if the time difference between two adjacent frames of images is too small, the content of the two frames of images will be almost the same, for example, the time difference between the first frame of images and the second frame of images is 0.01s.

Therefore, in one or more embodiments of this specification, the plurality of image frames within the predetermined time period before the first moment may be part of the images within the predetermined period before the first moment, for example, from the queued video stream One frame of image is acquired every 1s, and 120 frames of images will be obtained for the image frames within two minutes before the moment, and head tracking is performed on the image corresponding to the moment and the queuing objects in the out-of-queue area in the 120 frames of images. Determine the number of heads of dequeue objects that cross the dequeue line per unit time, and obtain the dequeue speed corresponding to the first moment.

Step 805: Obtain the number of queuing objects in the queuing area from the image frame corresponding to the first moment in the queuing video stream.

Step 807, according to the dequeue speed and the number of queued objects corresponding to the first moment, the queue waiting time corresponding to the first moment is obtained.

In practical applications, if the queuing situation changes slowly, the number of queuing objects will not change within a certain period of time. For example, the queuing speed is slow, for example, the number of queuing objects does not change within 30s. Therefore, in one or more embodiments of this specification , Step 805 may further include: acquiring the number of queuing objects in the queuing area from any image frame in the corresponding video stream within a fixed period of time before the first moment in the queuing video stream.

In addition, in practical applications, the queuing area is very large, and it is difficult for one camera to capture all the queuing areas, and multiple cameras are required to take pictures. As shown in Figure 7, it is a schematic diagram of the multi-camera collaboration scene shown in this manual. The queuing area includes 702A and 702B, and the imaging equipment includes 701A and 701B. The imaging equipment 701A is used for shooting the area 702A, and the imaging equipment 701B is used for shooting the area 702B.

Therefore, in one or more embodiments of the present specification, the queued video stream includes: multiple sub-video streams respectively captured by multiple cameras, wherein the area captured by each of the sub-video streams includes a part of the queuing area.

At this time, step 205 may include: according to the image frames corresponding to the plurality of sub-video streams corresponding to the second moment, determine the objects located in the queuing area in each image frame; count the objects located in the queue area in each image frame The number of objects in the queuing area corresponding to the second moment is obtained from the number of objects in the queuing area.

Wherein, the second moment may be any moment of the video stream, and it is only necessary to ensure that the image frames of each sub-queuing video stream are simultaneously acquired at the second moment.

In this way, when the area photographed by one imaging device cannot cover all the queuing areas, multiple imaging devices can be used to cooperate together, regardless of the shape of the queuing area and how wide the scope is, it can be achieved through the multi-camera shown in this manual. The instruments cooperate with each other to make statistics on the objects in the queue, and then get the waiting time in the queue.

Corresponding to the foregoing method embodiments, this specification also provides embodiments of a device and a terminal to which it is applied.

This specification provides a device for determining the queuing waiting time length, as shown in FIG. 9 , the device includes: an acquisition module 901, configured to acquire the queuing video stream of the target area, wherein the target area includes a queuing area and an out-of-queue area , the dequeue area includes the dequeue line that needs to pass through to end the queuing and judge whether to end the queuing; the dequeue speed determination module 903 is used to identify the dequeue area in the queued video stream, and for the dequeue Objects in the area are tracked, and the number of dequeue objects crossing the dequeue line in unit time is determined to obtain the dequeue speed; the queued object quantity determination module 905 is used to obtain all the objects from the target image frames in the queued video stream. The number of queuing objects in the queuing area; the queuing waiting duration determination module 907, configured to obtain the queuing waiting duration according to the dequeue speed and the number of queuing objects.

Wherein, the out-of-queue area may include: a queuing end area and a non-queuing area; at this time, at least part of the area in the queuing end area overlaps with the queuing area, and the out-of-queue area in the out-of-queue area The line is used as a boundary between the first queuing area and the non-queuing area.

The queuing object quantity determining module 905 may also be used for: performing head positioning on objects in the target image frame in the queuing video stream to obtain a set of head points in the target image frame; point collection and the queuing area, obtaining head points located in the queuing area; counting the number of head points in the queuing area to obtain the number of queuing objects.

In addition, the dequeuing speed determination module 903 may also be configured to: based on the first moment and the plurality of image frames in the queued video stream within a predetermined period of time before the first moment, the plurality of image frames Objects in the dequeue area in the frame are tracked, the number of dequeue heads crossing the dequeue line per unit time is determined, and the dequeue speed corresponding to the first moment is obtained.

At this time, the queuing object quantity determination module 905 is used to: acquire the queuing object quantity in the queuing area from the image frame corresponding to the first moment in the queuing video stream; the queuing waiting duration determination module 907 is used to : Obtain the queue waiting time corresponding to the first moment according to the dequeue speed and the number of queued objects.

In addition, the dequeue speed determining module 903 is configured to: track the objects in the dequeue area; determine the dequeue object based on the moving track of the object obtained by tracking the objects in the dequeue area ;Determine the number of dequeue objects per unit time, and get the dequeue speed.

The dequeue speed determination module 903 can also be used for: tracking the objects in the dequeue area; based on tracking the objects in the dequeue area, respectively determine the movement trajectory of each object; respond to any The moving track line of an object intersects with the described dequeuing line, and the moving direction of the described moving track line is from the queuing area to the non-queuing area, and it is determined that the object is the dequeuing object; the number of dequeuing objects per unit time is determined, Get the queue speed.

In addition, the queuing video stream may include: a plurality of sub-video streams respectively captured by multiple cameras, wherein the area captured by each of the sub-video streams includes a part of the queuing area; at this time, the queuing object quantity determination module 905 It can be used to: determine the objects located in the queuing area in each of the image frames according to the image frames corresponding to the plurality of sub-video streams corresponding to the second moment; count the objects located in the queuing area in each of the image frames The number of objects in the queue is obtained to obtain the number of queued objects corresponding to the second moment.

For the implementation process of the functions and effects of each module in the above-mentioned device, please refer to the implementation process of the corresponding steps in the above-mentioned method for details, and details will not be repeated here.

As for the device embodiment, since it basically corresponds to the method embodiment, please refer to the part description of the method embodiment for relevant parts. The device embodiments described above are only illustrative, and the modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in One place, or it can be distributed to multiple network modules. Part or all of the modules can be selected according to actual needs to achieve the purpose of the solution in this specification. Objects of ordinary skill in the art can be understood and implemented without creative effort.

Embodiments of the apparatus in this specification can be applied to computer equipment, such as servers or terminal equipment. The device embodiments can be implemented by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by reading the corresponding computer program instructions in the non-volatile memory into the memory for operation through the processor of the file processing where it is located.

Correspondingly, this specification also provides a computer device, including a memory, a processor, and a computer program stored in the memory and operable on the processor, wherein, when the processor executes the program, any one of the above-mentioned The method for determining the waiting time in the queue described above.

As shown in FIG. 10 , it is a schematic diagram of a more specific hardware structure of a computing device provided by the embodiment of this specification. The device may include: a processor 1010 , a memory 1020 , an input/output interface 1030 , a communication interface 1040 and a bus 1050 . The processor 1010 , the memory 1020 , the input/output interface 1030 and the communication interface 1040 are connected to each other within the device through the bus 1050 .

The processor 1010 may be implemented by a general-purpose CPU (Central Processing Unit, central processing unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute related programs to realize the technical solutions provided by the embodiments of this specification.

The memory 1020 can be implemented in the form of ROM (Read Only Memory, read-only memory), RAM (Random Access Memory, random access memory), static storage device, dynamic storage device, etc. The memory 1020 can store operating systems and other application programs. When implementing the technical solutions provided by the embodiments of this specification through software or firmware, the relevant program codes are stored in the memory 1020 and invoked by the processor 1010 for execution.

The input/output interface 1030 is used to connect the input/output module to realize information input and output. The input/output/module can be configured in the device as a component (not shown in the figure), or can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, mouse, touch screen, microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, and the like.

The communication interface 1040 is used to connect a communication module (not shown in the figure), so as to realize the communication interaction between the device and other devices. The communication module can realize communication through wired means (such as USB, network cable, etc.), and can also realize communication through wireless means (such as mobile network, WIFI, Bluetooth, etc.).

Bus 1050 includes a path that carries information between the various components of the device (eg, processor 1010, memory 1020, input/output interface 1030, and communication interface 1040).

It should be noted that although the above device only shows the processor 1010, the memory 1020, the input/output interface 1030, the communication interface 1040 and the bus 1050, in the specific implementation process, the device may also include other components. In addition, those skilled in the art can understand that the above-mentioned device may only include components necessary to implement the solutions of the embodiments of this specification, and does not necessarily include all the components shown in the figure.

This specification also provides a computer-readable storage medium, on which computer instructions are stored, and when the instructions are executed by a processor, the steps of any method for determining the waiting time in a queue as described above are realized.

This specification also provides a computer program product, including a computer program stored in a memory, and when the computer program instructions are executed by a processor, the steps of the method for determining the queuing waiting time described in any one of the above are implemented.

Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can be implemented by any method or technology for storage of information. Information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Flash memory or other memory technology, Compact Disc Read-Only Memory (CD-ROM), Digital Versatile Disc (DVD) or other optical storage, Magnetic cassettes, disk storage, quantum memory, graphene-based storage media or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by computing devices. As defined herein, computer-readable media excludes transitory computer-readable media, such as modulated data signals and carrier waves.

The foregoing describes specific embodiments of this specification. Other implementations are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. Multitasking and parallel processing are also possible or may be advantageous in certain embodiments.

Other embodiments of the specification will readily occur to those skilled in the art from consideration of the specification and practice of the invention claimed herein. This description is intended to cover any modification, use or adaptation of this description. These modifications, uses or adaptations follow the general principles of this description and include common knowledge or conventional technical means in this technical field for which this description does not apply . The specification and examples are to be considered exemplary only, with a true scope and spirit of the specification being indicated by the following claims.

It should be understood that this specification is not limited to the precise constructions which have been described above and shown in the accompanying drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the specification is limited only by the appended claims.

The above descriptions are only some examples of this specification, and are not intended to limit this specification. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of this specification shall be included in the protection of this specification. within the range.

Claims

A method for determining the waiting time in a queue, applied to a terminal device, comprising:

Acquiring the queuing video stream of the target area, wherein the target area includes a queuing area and an out-of-queue area, and the out-of-queue area includes an out-of-queue line that needs to pass through to end the queuing and judge whether to end the queuing;

Identify the dequeue area in the queuing video stream, track the objects in the dequeue area, determine the number of dequeue objects crossing the dequeue line per unit time, and obtain the dequeue speed;

Obtain the number of queuing objects in the queuing area from the target image frame of the queuing video stream;

According to the dequeue speed and the number of queuing objects, the queuing waiting time is obtained.
The method according to claim 1, wherein the out-of-queue area further comprises: an end-of-queue area and a non-queuing area;

At least a part of the queuing end area overlaps with the queuing area, and the dequeuing line in the out queuing area is used as a boundary between the queuing end area and the non-queuing area.
The method according to claim 1, wherein obtaining the number of queuing objects in the queuing area from the target image frame of the queuing video stream comprises:

Perform head positioning on objects in the target image frame in the queued video stream to obtain a set of head points in the target image frame;

Based on the set of head points and the queuing area, determine the number of head points located in the queuing area to obtain the number of queuing objects.
The method according to claim 1, wherein tracking the objects in the dequeue area, determining the number of dequeue objects crossing the dequeue line per unit time, and obtaining the dequeue speed includes:

Based on the first moment and the plurality of image frames in the queuing video stream within a predetermined time period before the first moment, tracking the objects in the out-of-queue area in the plurality of image frames, and determining the unit time The number of dequeue objects that cross the dequeue line in the interior, and obtain the dequeue speed corresponding to the first moment;

Obtaining the number of queuing objects in the queuing area from the target image frame of the queuing video stream includes:

Obtaining the number of queuing objects in the queuing area from the image frame corresponding to the first moment in the queuing video stream;

According to the dequeue speed and the number of queuing objects, the queuing waiting time is obtained, including:

According to the dequeue speed corresponding to the first moment and the number of queuing objects, the queuing waiting time corresponding to the first moment is obtained.
The method according to claim 1, wherein tracking the objects in the dequeue area, determining the number of dequeue objects crossing the dequeue line per unit time, and obtaining the dequeue speed includes :

Tracking objects in the dequeuing area;

determining the dequeuing object based on the movement track of the object obtained by tracking the object in the dequeuing area;

Determine the number of dequeue objects per unit time to obtain the dequeue speed.
The method according to claim 5, wherein, based on the moving track of the object obtained by tracking the objects in the dequeue area, determining the dequeue object comprises:

Based on tracking the objects in the dequeuing area, respectively determine the movement trajectory of each object;

In response to the movement trajectory of any object intersecting the dequeue line, and the movement direction of the movement trajectory is from the queuing area to the non-queuing area, it is determined that the object is the dequeue object.
The method according to claim 1, wherein the queuing video stream comprises: a plurality of sub-video streams respectively captured by a plurality of cameras, wherein the area captured by each of the sub-video streams includes a part of the queuing area;

Acquiring the number of queuing objects in the queuing area in the target image frame of the queuing video stream, including:

According to the image frames corresponding to the plurality of sub-video streams corresponding to the second moment, determine the objects located in the queuing area in each of the image frames;

Count the number of objects in the queuing area in each of the image frames to obtain the number of queuing objects corresponding to the second moment.
A device for determining the waiting time in a queue, comprising:

An acquisition module, configured to acquire a queuing video stream in a target area, wherein the target area includes a queuing area and an out-of-queue area, and the out-of-queue area includes an out-of-queue line that needs to pass through to end the queuing and judge whether to end the queuing;

The dequeue speed determination module is used to identify the dequeue area in the queuing video stream, track the objects in the dequeue area, and determine the number of dequeue objects crossing the dequeue line per unit time , get the dequeue speed;

A queuing object quantity determination module, configured to acquire the queuing object quantity in the queuing area from the target image frame of the queuing video stream;

The queue waiting time determination module is configured to obtain the queue waiting time according to the dequeue speed and the number of queued objects.
A computer device, comprising a memory, a processor, and a computer program stored in the memory and operable on the processor, wherein the method according to any one of claims 1-7 is implemented when the processor executes the program .
A computer-readable storage medium, on which computer instructions are stored, and when the instructions are executed by a processor, the method according to any one of claims 1-7 is realized.
A computer program product comprising a computer program stored in a memory, the computer program instructions implementing the method according to any one of claims 1-7 when executed by a processor.