WO2019127132A1

WO2019127132A1 - Waiting timeout prompting method and cloud system

Info

Publication number: WO2019127132A1
Application number: PCT/CN2017/119097
Authority: WO
Inventors: 南一冰; 廉士国
Original assignee: 深圳前海达闼云端智能科技有限公司
Priority date: 2017-12-27
Filing date: 2017-12-27
Publication date: 2019-07-04
Also published as: CN108323202A

Abstract

Provided are a waiting timeout prompting method and a cloud system. The method comprises: acquiring video image information; determining a waiting duration of a pre-set person in the video image information; and when the waiting duration of the pre-set person exceeds a pre-set duration, sending waiting timeout information of the pre-set person. By means of the present application, a member of staff can provide a more humanized and diversified service for a corresponding pre-set person according to waiting timeout information, thereby effectively improving the service quality of staff.

Description

Prompt method for timeout waiting and cloud system

Technical field

The present application relates to the field of intelligent monitoring technologies, and in particular, to a prompting method for a timeout waiting and a cloud system.

Background technique

The business halls of the banking, communications and other industries provide customers with all the services they need. After the customer enters the business hall to get the queue number, they need to wait for a period of time before they can handle related business, and the customer waits for too long, and the customer behavior is abnormal. The service methods of the existing business hall staff are not perfect enough to provide a humanized and diversified service for the situation that the customer waits for too long.

Summary of the invention

The embodiment of the present application proposes a reminder method for timeout waiting and a cloud system to solve the technical problem that the customer waiting time is too long and the business hall staff does not have a corresponding perfect service mode in the existing business hall work.

In one aspect, an embodiment of the present application provides a prompting method for timeout waiting, including:

Obtaining video image information;

Determining a waiting time of a preset person in the video image information;

When the waiting time of the preset person exceeds the preset duration, the timeout waiting information of the preset person is sent.

In another aspect, the embodiment of the present application provides a prompt cloud system for waiting for a timeout, including:

a camera device for collecting video image information;

An algorithm server for acquiring video image information; and,

Determining a waiting time of a preset person in the video image information; and,

In another aspect, an embodiment of the present application provides an electronic device, where the electronic device includes:

Transceiver, memory, one or more processors;

One or more modules, the one or more modules being stored in the memory and configured to be executed by the one or more processors, the one or more modules comprising Instructions for each step.

In another aspect, embodiments of the present application provide a computer program product for use with an electronic device, the computer program product comprising a computer readable storage medium and a computer program mechanism embedded therein, the computer program mechanism Instructions are included for performing the various steps in the above methods.

The benefits are as follows:

In this embodiment, the algorithm server determines the waiting time of the preset person in the video image information according to the video image information collected by the camera device, and sends the pre-waiting time when the waiting time of the preset person exceeds the preset duration. Set the timeout information of the personnel, so that the staff can provide more personalized and diversified services to the corresponding preset personnel according to the timeout waiting information, thereby effectively improving the service quality of the staff.

DRAWINGS

Specific embodiments of the present application will be described below with reference to the accompanying drawings, in which:

1 is a schematic diagram of a method for waiting for a timeout in the first embodiment of the present application;

2 is a schematic flowchart of a method for waiting for a timeout in the first embodiment of the present application;

3 is a structural diagram of a cloud system in which a timeout waiting prompt is performed in Embodiment 2 of the present application;

FIG. 4 is a schematic structural diagram of an electronic device according to Embodiment 3 of the present application.

Detailed ways

The essence of the technical solution of the embodiment of the present invention is further clarified by specific examples below.

The exemplary embodiments of the present application are further described in detail below with reference to the accompanying drawings, in which the embodiments described are only a part of the embodiments of the present application, but not all embodiments. An exhaustive example. And in the case of no conflict, the features in the embodiments and the embodiments in the description can be combined with each other.

The inventor noticed during the invention:

When the customer enters the business hall of the banking, communication and other industries to get the queue number, it takes a while to process the relevant business, but the waiting time for the customer is too long, and there are problems such as impatience. The service mode of the existing business hall cannot be The client provides services related to emotional comfort.

In view of the above-mentioned shortcomings/based on the above, the embodiment of the present application proposes that during the process of the customer waiting in the waiting area, by monitoring the waiting time of the customer in the video, the customer is impatient, such as going back and forth, repeatedly entering and leaving, and the staff of the business hall appear. In the waiting area, the distinction between staff and customers, and the Human Re-ID (Person Re-Identification) technology based on the Convolutional Neural Network (CNN) The waiting time of the customer, identifying the abnormal behavior of the customer, so that the statistical time is more accurate, the identification of the customer's complex abnormal behavior is more adaptable, and the customer does not need to carry additional authentication equipment, thereby effectively improving the staff of the business hall. service quality.

In order to facilitate the implementation of the present application, the following examples are described.

Example 1

1 is a schematic diagram of a method for timeout waiting for prompting in the first embodiment of the present application, and FIG. 2 is a schematic flowchart showing a method for timeout waiting for prompting in the first embodiment of the present application, as shown in FIG. 2, as shown in FIG. As shown, the method includes:

Step 101: Acquire video image information.

Step 102: Determine a waiting time of a preset person in the video image information.

Step 103: Send a timeout waiting information of the preset person when the waiting time of the preset person exceeds a preset duration.

In step 101, the camera device collects the video image in the preset area of the business hall in real time, and sends the video image to the algorithm server. The algorithm server obtains the video image information according to the video image, and the video image information includes the personnel information in the preset area.

In step 102, the algorithm server determines whether the person appearing in the video image is a preset person. If it is a preset person, assigns a unique ID to each preset person according to the characteristics of the human body region, and counts the waiting for each ID. Duration; if it is not a preset person, it will not be processed.

In step 103, after the set information sending time interval is reached, information such as the human body area screenshot and the corresponding waiting time of the preset person whose waiting time is the longest, or exceeds the preset duration is obtained, and is sent to the scheduling server.

In the implementation, the execution entity of the foregoing step may be a cloud server, and the scheduling server in the cloud server may wait for the longest waiting time, or the information of the human body area of the preset person exceeding the preset duration and the corresponding waiting time and other information are distributed to the business office. A handheld mobile terminal of a person (for example, a lobby manager), after receiving the information, the worker searches for a corresponding preset person in the waiting area according to the preset human body area characteristic image, and provides a corresponding service.

In this embodiment, the preset person is a client, and determining a waiting time of the client in the video image information includes:

Identifying whether the person is a customer according to a human body region characteristic of a person in the video image information;

If the person is identified as a customer, the waiting time of the customer is determined.

In this embodiment, the personnel include a staff member and a customer, and identifying whether the person is a customer includes:

Finding a human body region feature of the person from a preset staff feature database;

If the human body area feature of the person is in a preset staff feature database, determining that the person is a staff member;

If the human body area feature of the person is not in the preset staff feature database, it is determined that the person is a customer.

In this embodiment, the determining the waiting time of the client includes:

If the customer's human body area feature is not in the preset customer feature database, the customer's human body area feature is stored in the preset customer feature database, and the waiting time of the customer is set to 0;

If the customer's human body area feature is in the preset customer feature database, the waiting time of the customer is updated.

In the implementation, the human body area feature of all the people in the video image is extracted to determine whether it is a customer, and if it is a customer, the customer feature database and the customer waiting time are updated. Specifically:

1) determining whether the location of each person in the video image is within the defined waiting area, and if in the waiting area, extracting the human body region characteristics of each person based on the CNN Person Re-ID algorithm;

2) According to the characteristics of the human body area of each person, find out whether there are similar human body area characteristics in the staff characteristic database and the customer feature database (for example, the similarity reaches the set threshold Th), if a certain person is not in the staff Found in the feature database and customer feature database, the person is considered to be the customer, and the human body region feature is stored in the customer feature database, and a unique ID number is assigned to it, denoted by n, and the current time T is entered as the customer. Waiting area start time Sn, Sn=T, and set the customer's current waiting time Wn to 0; if a person is found in the staff feature database, no processing; if found in the customer feature database, then Updating the human body area feature of the customer in the customer feature database as the current human body area feature, and updating the waiting time length Wn corresponding to the customer according to the start time Sn and the current time T, Wn=T-Sn;

3) Sort all customer IDs including information such as the human body region feature, the start time Sn, and the waiting time length Wn in descending order of the waiting time length Wn.

In this embodiment, the waiting time of the preset person exceeds a preset duration, and the method further includes: the preset person appears in a video image of a last frame of the video image information.

In the implementation, customers who wait longer than the preset duration should meet but not be limited to the following conditions:

1) the client exists in the video image of the current frame;

2) The waiting time is the longest, or exceeds the preset duration (for example, the preset duration is set to 10 minutes).

In this embodiment, the timeout waiting information includes behavior abnormality information of the preset personnel, and before the sending the timeout waiting information of the preset personnel, the method further includes: determining that the preset personnel behavior is abnormal.

In this embodiment, the determining that the preset personnel behavior is abnormal includes:

If the historical coordinates of the preset person appear in a plurality of preset regions, it is determined that the preset person behavior is abnormal.

In the implementation, if the algorithm server determines that the person appearing in the video image is a customer, the waiting time of the customer is counted, the behavior of the customer is identified, and the human body area of the customer waiting for a long time and/or abnormal behavior is captured and corresponding. The wait time and/or behavior exception information is sent to the dispatch server. Specifically:

1) If the algorithm server determines that the human body area feature of a certain person is not in the preset staff feature database, it is determined that the person appearing in the video image is the customer, and the current time T is taken as the starting time Sn of the customer, Sn= T, and set the current waiting time Wn of the customer to be 0, and record the coordinates of the center point of the customer's human body region at the current time T as (x _T , y _T ); if the algorithm server determines the human body region of a certain person The feature is in the preset customer feature database, and the human body region feature in the customer feature database is updated to the current human body region feature, and the coordinate of the customer in the customer feature database is updated as the coordinate of the current time T (x _T , y _T ), and updating the waiting time length Wn corresponding to the customer according to the starting time Sn and the current time T, Wn=T-Sn;

2) According to the waiting time and historical coordinates of the customer, determine whether there is at least one of the following abnormal behaviors:

a) judging whether it is a timeout due to the waiting time (for example, the waiting time exceeds the preset duration);

b) Since the historical coordinates can draw the behavior track of the customer, it is judged whether the customer is behaving abnormally according to the historical coordinates of the customer, for example, the customer's historical coordinate track has multiple accesses to the waiting area, or moving back and forth; or based on CNN to the human body The regional characteristics are identified to determine if there is an abnormality in the behavior of the customer at the current time. Preferably, the abnormal behavior may predefine a plurality of abnormal behavior patterns according to actual conditions, specifically, identifying a behavior pattern of the customer according to the trajectory of the historical coordinates, the range of activities, and the like, for example, by setting a plurality of regions, if the behavior of the client If the trajectory spans multiple regions for a long time, it is determined that the customer has abnormal behavior.

If the timeout staying or abnormal behavior exists, the client's human body area screenshot, the corresponding waiting time, and the abnormal behavior mode are sent to the scheduling server.

Using the human key point detection model, obtaining key information of the human body of the preset person according to the video image information;

If the human key point information of the preset person matches the preset abnormal behavior template, it is determined that the preset person behavior is abnormal.

In the implementation, the deep learning model based on the detection of the key point of the human body is used to obtain the key point information of the preset human body according to the video image information, and the key information of the obtained human body of the preset person is matched with the preset abnormal behavior template. If the match is successful, it is determined that the preset person behavior is abnormal, and the abnormal behavior corresponding to the preset person is identified.

The present application provides a detailed description of Embodiment 1 of the present application by taking a specific scenario as an example.

The application scope of the embodiments of the present application includes, but is not limited to, the business processing of the banking business hall, and the business processing of the banking business hall is taken as an example. The specific process is as follows:

Step 201: The camera device acquires a video image in the waiting area of the business hall in real time, and sends the video image to the algorithm server.

Step 202: The algorithm server uses the target detection method to identify all the persons appearing in the video image.

Step 203: Extract human body regional characteristics of all personnel, determine whether the customer is a customer by using a preset staff feature database and a customer feature database, update the customer feature database, count the customer waiting time, and identify the customer behavior if the customer is a customer.

Step 204: Steps 202 and 203 are repeated. After the set information transmission time interval is reached, the screenshot of the human body area, the corresponding waiting time, and the behavior abnormality of the client having the longest waiting time (or exceeding the preset duration) are obtained. Information is sent to the dispatch server.

Step 205: The scheduling server distributes the screenshot of the human body area of the customer, the corresponding waiting time and behavior abnormality information to the handheld mobile terminal of the business hall staff.

Step 206: The handheld mobile terminal of the staff member of the business hall displays the screenshot of the human body area of the customer, the corresponding waiting time and behavior abnormal information, and manually judges whether it is a staff member according to the human body area screenshot, and if the staff member is the staff member, the human body of the staff member The area screenshot is added to the staff feature database described in step 203 to avoid the staff being mistakenly recognized as a customer again. If it is not a staff member, the corresponding customer is searched in the waiting area according to the human body area screenshot, and according to The corresponding waiting time and behavior abnormal information provide corresponding services.

Example 2

Based on the same inventive concept, the present application also provides a prompt cloud system for timeout waiting. Since the principle of solving the problem of these devices is similar to the prompting method for timeout waiting, the implementation of these devices can refer to the implementation of the method. The repetitions are not repeated here.

FIG. 3 is a structural diagram of a cloud system in which the timeout waiting prompt is displayed in the second embodiment of the present application. As shown in FIG. 3, the prompting cloud system 300 may include: an imaging device 301, an algorithm server 302, a scheduling server 303, and a mobile terminal. 304.

The imaging device 301 is configured to collect video image information.

An algorithm server 302, configured to acquire video image information; and,

When the waiting time of the preset person exceeds the preset duration, the timeout waiting information of the preset person is sent to the mobile terminal 304.

In this embodiment, the determining the waiting time of the client includes:

In this embodiment, the algorithm server 302 further includes: the preset person appears in a last frame video image of the video image information.

In this embodiment, the timeout waiting information includes behavior abnormality information of the preset personnel, and the algorithm server 302 further includes: determining that the preset personnel behavior is abnormal.

Example 3

Based on the same inventive concept, an electronic device is also provided in the embodiment of the present application. Since the principle is similar to the prompting method for timeout waiting, the implementation of the method may refer to the implementation of the method, and the repeated description is not repeated.

4 is a schematic structural diagram of an electronic device in Embodiment 3 of the present application. As shown in FIG. 4, the electronic device includes: a transceiver device 401, a memory 402, one or more processors 403, and one or more modules. The one or more modules are stored in the memory and configured to be executed by the one or more processors, the one or more modules including steps for performing the steps of any of the above methods instruction.

Example 4

Based on the same inventive concept, the embodiment of the present application further provides a computer program product for use in combination with an electronic device. Since the principle is similar to a prompt method for waiting for a timeout, the implementation may refer to the implementation of the method, and the repetition is not Let me repeat. The computer program product comprises a computer readable storage medium and a computer program mechanism embodied therein, the computer program mechanism comprising instructions for performing the various steps of any of the above methods.

For the convenience of description, the various parts of the above-described apparatus are separately described by functions into various modules. Of course, the functions of each module or unit may be implemented in the same software or hardware in the implementation of the present application.

Those skilled in the art will appreciate that embodiments of the present application can be provided as a method, system, or computer program product. Thus, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment in combination of software and hardware. Moreover, the application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) including computer usable program code.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (system), and computer program products according to embodiments of the present application. It will be understood that each flow and/or block of the flowchart illustrations and/or FIG. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing device to produce a machine for the execution of instructions for execution by a processor of a computer or other programmable data processing device. Means for implementing the functions specified in one or more of the flow or in a block or blocks of the flow chart.

The computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture comprising the instruction device. The apparatus implements the functions specified in one or more blocks of a flow or a flow and/or block diagram of the flowchart.

These computer program instructions can also be loaded onto a computer or other programmable data processing device such that a series of operational steps are performed on a computer or other programmable device to produce computer-implemented processing for execution on a computer or other programmable device. The instructions provide steps for implementing the functions specified in one or more of the flow or in a block or blocks of a flow diagram.

While the preferred embodiment of the present application has been described, it will be apparent that those skilled in the art can make further changes and modifications to the embodiments. Therefore, it is intended that the appended claims be interpreted as

Claims

A prompting method for waiting for a timeout, characterized in that it comprises:

Obtaining video image information;

Determining a waiting time of a preset person in the video image information;

When the waiting time of the preset person exceeds the preset duration, the timeout waiting information of the preset person is sent.
The method according to claim 1, wherein the preset person is a client, and determining a waiting time of the client in the video image information comprises:

Identifying whether the person is a customer according to a human body region characteristic of a person in the video image information;

If the person is identified as a customer, the waiting time of the customer is determined.
The method of claim 2 wherein said person comprises a staff member and a customer, and identifying whether said person is a customer comprises:

Finding a human body region feature of the person from a preset staff feature database;

If the human body area feature of the person is in a preset staff feature database, determining that the person is a staff member;

If the human body area feature of the person is not in the preset staff feature database, it is determined that the person is a customer.
The method according to claim 2 or 3, wherein the determining the waiting time of the client comprises:

If the customer's human body area feature is not in the preset customer feature database, the customer's human body area feature is stored in the preset customer feature database, and the waiting time of the customer is set to 0;

If the customer's human body area feature is in the preset customer feature database, the waiting time of the customer is updated.
The method of claim 1, wherein the waiting time of the preset person exceeds a preset duration, further comprising: the preset person appearing in a video image of a last frame of the video image information.
The method according to claim 1, wherein the timeout waiting information includes behavior abnormality information of the preset person, and before the sending the timeout waiting information of the preset person, the method further includes: determining the Set personnel behavior abnormally.
The method of claim 6, wherein the determining the abnormal behavior of the preset person comprises:

If the historical coordinates of the preset person appear in a plurality of preset regions, it is determined that the preset person behavior is abnormal.
The method of claim 6, wherein the determining the abnormal behavior of the preset person comprises:

Using the human key point detection model, obtaining key information of the human body of the preset person according to the video image information;

If the human key point information of the preset person matches the preset abnormal behavior template, it is determined that the preset person behavior is abnormal.
A prompt cloud system for waiting for a timeout, comprising:

a camera device for collecting video image information;

An algorithm server for acquiring video image information; and,

Determining a waiting time of a preset person in the video image information; and,

When the waiting time of the preset person exceeds the preset duration, the timeout waiting information of the preset person is sent.
The cloud system according to claim 9, wherein the preset person is a client, and determining a waiting time of the client in the video image information comprises:

Identifying whether the person is a customer according to a human body area characteristic of a person in the video image information;

If the person is identified as a customer, the waiting time of the customer is determined.
The cloud system of claim 10, wherein the person comprises a staff member and a customer, and identifying whether the person is a customer comprises:

Finding a human body region feature of the person from a preset staff feature database;

If the human body area feature of the person is in a preset staff feature database, determining that the person is a staff member;

If the human body area feature of the person is not in the preset staff feature database, it is determined that the person is a customer.
The cloud system according to claim 10 or 11, wherein the determining the waiting time of the client comprises:

If the customer's human body area feature is not in the preset customer feature database, the customer's human body area feature is stored in the preset customer feature database, and the waiting time of the customer is set to 0;

If the customer's human body area feature is in the preset customer feature database, the waiting time of the customer is updated.
The cloud system according to claim 9, wherein the algorithm server further comprises: the preset person appearing in a last frame video image of the video image information.
The cloud system according to claim 9, wherein the timeout waiting information includes behavior abnormality information of the preset person, and the algorithm server further comprises: determining that the preset personnel behavior is abnormal.
The cloud system according to claim 14, wherein the determining the abnormal behavior of the preset person comprises:

If the historical coordinates of the preset person appear in a plurality of preset regions, it is determined that the preset person behavior is abnormal.
The cloud system according to claim 14, wherein the determining the abnormal behavior of the preset person comprises:

Using the human key point detection model, obtaining key information of the human body of the preset person according to the video image information;

If the human key point information of the preset person matches the preset abnormal behavior template, it is determined that the preset person behavior is abnormal.
An electronic device, comprising:

Transceiver, memory, one or more processors;

One or more modules stored in the memory and configured to be executed by the one or more processors, the one or more modules including for performing claim 1 The instructions of the various steps in any of the methods of -8.
A computer program product for use in conjunction with an electronic device, the computer program product comprising a computer readable storage medium and a computer program mechanism embedded therein, the computer program mechanism comprising means for performing any of claims 1-8 An instruction for each step in the method.