WO2021135196A1 - Status recognition method and device for display region, electronic apparatus, and storage medium - Google Patents

Abstract

Provided are a status recognition method and device for a display region, an electronic apparatus, and a storage medium. In an embodiment, the method comprises: acquiring a video image captured in a display region; determining, on the basis of the captured video image, customer flow data of the display region within a first time period, the customer flow data comprising at least two of the following: a dwell time of each customer in the display region, an attention duration of each customer, attribute information of each customer, a customer footfall count, and a customer repeat-visit count; using the customer flow data to determine crowd status data of the display region within the first time period; and using the crowd status data to control a corresponding display status of the display region on a display interface.

Description

State recognition method, device, electronic equipment and storage medium of display area

Cross-references to related applications

This patent application claims the priority of the Chinese patent application with application number 2019114163365 filed on December 31, 2019, the full text of which is incorporated herein by reference.

Technical field

The present disclosure relates to the field of computer technology, and in particular, to a state recognition method, device, electronic device, and storage medium of a display area.

Background technique

In exhibition venues such as supermarkets and other shopping stores, exhibition halls, etc., various items are usually displayed in various display areas for users to purchase or visit. The degree of user interest in the items in each display area is one of the concerns of many stores or exhibitions. However, because the characteristics of the flow of people in stores or exhibitions are mostly reflected in the dense crowds and the trajectories of users in an irregular state, this makes it more difficult to analyze the degree of user interest in the items in each display area.

Summary of the invention

In view of this, the present disclosure provides at least a state recognition solution for the display area.

In the first aspect, the present disclosure provides a method for identifying the status of a display area, including:

Obtain the video images collected in the display area;

Based on the collected video images, determine the passenger flow data of the display area in the first time period. The passenger flow data includes the stay time of each user appearing in the display area, the attention time of each user, and the user's At least two of attribute information, number of passengers, and number of passengers;

Using the passenger flow data to determine crowd state data in the display area in the first time period;

Using the crowd state data, the display state corresponding to the display area in the display interface is controlled.

In the above method, the passenger flow data may be determined based on the collected video images, the passenger flow data may be used to determine the crowd state data of the display area in the first time period, and the crowd state data may be used to control the display state of the display area in the display interface. The display status of the display area can be used to prompt the crowd status data in the display area, so that dynamic monitoring of the crowd status in the display area can be realized, and the people of interest in the display area can be analyzed.

In a possible implementation manner, the crowd state data includes crowd popularity levels; the using the crowd state data to control the display state corresponding to the display area in the display interface includes: controlling the display state in the display interface The layer of the display area corresponding to the display area presents a color level that matches the popularity level of the crowd.

Through different color levels, different people's popularity levels are displayed, which enriches the way of presenting the display status of the display area, and can more intuitively display the crowd's popularity information in different display areas.

In a possible implementation manner, the crowd popularity level is determined according to the following manner: determining the crowd popularity value based on the respective weight coefficients of at least two types of passenger flow data in the passenger flow data and the values of the at least two types of passenger flow data , And determine the crowd's popularity level corresponding to the range of the crowd's popularity value.

The crowd popularity value determined in this way combines the characteristics of at least two kinds of passenger flow data, so the determined crowd popularity value can better reflect the passenger flow characteristics of the display area, and the determined crowd popularity level is more accurate.

In a possible implementation manner, in a case where the passenger flow data includes the number of passenger flow and the number of passenger flow, the use of the passenger flow data to determine the state of the crowd in the display area in the first time period The data includes: using the number of passengers and the number of passengers to obtain the number of passengers and the number of passengers at multiple time nodes in the first time period; and using the crowd status data to control the number of passengers in the display interface The corresponding display state includes: based on the number of passengers and the number of passengers at multiple time nodes in the first time period, displaying the change in the number of passengers in the display area in the first time period on the display interface Trends and trends in the number of passenger flows.

In the foregoing embodiment, the number of passenger flows and the change of the number of passenger flows in the display area can be displayed more intuitively.

In a possible implementation manner, the using the passenger flow data to determine the crowd status data of the display area in the first time period includes: determining crowd distribution data with different attributes based on the attribute information of each user , The population distribution data includes distribution data having at least one attribute of age, gender, charm value, and emotion;

The using the crowd state data to control the display state corresponding to the display area in the display interface includes: using the crowd distribution data with the different attributes to control the display interface to separately display the crowd with each attribute Distribution.

In the foregoing embodiment, the population distribution map of each attribute may be displayed according to the population distribution data of different attributes, so as to display a more detailed population distribution situation for each display area.

In a possible implementation manner, in the case that the passenger flow data includes the attention duration of each user and the attribute information of each user, the passenger flow data is used to determine the display area of the display area in the first time period. The crowd status data includes: dividing each user into a crowd set of different attributes based on the attribute information of each user appearing in the display area; obtaining the status of each user in the display area in the crowd set of each type of attribute Attention duration; based on the attention duration corresponding to each user in the crowd set of each type of attribute, determine the total attention duration of the crowd set of each type of attribute;

The use of the crowd state data to control the display state corresponding to the display area in the display interface includes: controlling the crowd who presents each type of attribute in the display interface based on the total attention time of the crowd set of each type of attribute A collection of state-of-interest renderings.

Based on the foregoing implementation manner, the attention state effect diagram of the crowd set of each type of attribute presented on the display interface can be used to indicate the degree of attention of the crowd of different types of attributes to the display area.

In a possible implementation manner, the users in the population set of each type of attribute meet at least two attributes of age, gender, charm value, and emotion.

In a possible implementation manner, the attention state effect diagram is displayed by a specific graphic; the total attention time of the group of people of each type of attribute is different, and the display effect of the corresponding specific image is also different.

In a possible implementation manner, the length of attention is determined according to the following manner: for each user, identify the user’s face orientation data in the collected video frame; In the case where the user pays attention to the display object in the display area, record the time when the user first paid attention to the display object; based on the recorded times when the user first paid attention to the display object, it is determined that the display object is in the first The length of time the user pays attention to the display object within the time period, and the time period of attention does not exceed the first time period.

In a possible implementation manner, the face orientation data includes a pitch angle and a yaw angle of the face; when the pitch angle is within a first angle range and the yaw angle is within a second angle range In this case, it is determined that the face orientation data indicates that the user pays attention to the display object in the display area.

In the above-mentioned embodiment, the user's attention time of the displayed object in the first time period can be determined only based on the recorded time when the user first paid attention to the displayed object. Therefore, it is possible to prevent the user from paying attention to the displayed object for the first time. , Resulting in frequent statistics of the attention time, also makes the way of calculating the attention time more reasonable.

In a possible implementation manner, the number of visitors in the display area in the first time period is determined according to the following manner: the total number of appearances of each user in the display area in the first time period is determined; For each user, record the start time each time the user enters the display area; if the interval between the two recorded start times exceeds the first duration, it is determined that the user appears once; the first duration is not Beyond the first time period.

In the foregoing implementation manner, repeated statistics of the number of user appearances in a relatively short period of time can be avoided, and the accuracy of passenger flow statistics is improved.

In a possible implementation manner, the length of stay of users in the display area in the first time period is determined according to the following method: the total length of stay of each user during each appearance in the first time period is determined; For each user, when it is determined that the interval between the start time when the user enters the display area twice exceeds the first duration, the interval is used as the duration of the user's one stay.

In the foregoing embodiment, it is possible to avoid multiple calculations of the stay time when the user appears multiple times in a short period of time, which reduces the calculation amount of the stay time and also makes the method of calculating the stay time more reasonable.

In a possible implementation manner, the method further includes: before using the crowd state data to control the display state corresponding to the display area in the display interface, acquiring a trigger operation on the display area in the display interface.

In a possible implementation manner, the acquiring the trigger operation for the display area in the display interface includes: controlling the display interface to display a gesture prompt box, the gesture prompt box including gesture description content; The user posture in the collected video frame; in the case that the user posture is consistent with the posture recorded in the posture description content, confirm that the trigger operation for the display area in the display interface is acquired.

In the above embodiment, when acquiring the trigger operation for the display area in the display interface, the user gesture in the video screen may also be collected to enrich the trigger mode of the trigger operation and increase the interaction between the user and the device.

In a second aspect, embodiments of the present disclosure provide a state recognition device for a display area, including:

The acquisition module is used to acquire the video images collected in the display area;

The first determining module is configured to determine the passenger flow data of the display area in the first time period based on the collected video images, where the passenger flow data includes the stay time of each user appearing in the display area, and each user At least two of the following time, the attribute information of each user, the number of visitors, and the number of visitors;

A second determining module, configured to use the passenger flow data to determine crowd status data in the display area in the first time period;

The control module is configured to use the crowd state data to control the display state corresponding to the display area in the display interface.

In a third aspect, the present disclosure provides an electronic device including: a processor and a memory, the memory stores machine-readable instructions executable by the processor, and when the machine-readable instructions are executed by the processor, The processor is caused to execute the method for recognizing the state of the display area as described in the first aspect or any one of the implementation manners.

In a fourth aspect, the present disclosure provides a computer-readable storage medium having a computer program stored on the computer-readable storage medium, and when the computer program is run by a processor, the processor executes the above-mentioned first aspect or any one of The state recognition method of the display area described in the implementation mode.

In order to make the above objectives, features and advantages of the present disclosure more comprehensible, preferred embodiments accompanied with accompanying drawings are described in detail as follows.

Description of the drawings

In order to explain the technical solutions of the embodiments of the present disclosure more clearly, the following will briefly introduce the drawings that need to be used in the embodiments. The drawings here are incorporated into the specification and constitute a part of the specification. The figure shows an embodiment conforming to the present disclosure, and is used together with the description to explain the technical solution of the present disclosure. It should be understood that the following drawings only show certain embodiments of the present disclosure, and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, they can also Obtain other related drawings based on these drawings.

FIG. 1 shows a schematic flowchart of a method for recognizing the status of a display area provided by an embodiment of the present disclosure;

FIG. 2 shows a schematic diagram showing display areas of different color levels in a display interface provided by an embodiment of the present disclosure;

FIG. 3 shows a distribution diagram of people under each attribute in a display interface provided by an embodiment of the present disclosure;

FIG. 4 shows a schematic diagram of a comparison result between the change trend of the number of passenger flow and the change trend of the number of passenger flow provided by an embodiment of the present disclosure;

FIG. 5 shows a schematic flowchart of a method for determining a user's attention duration provided by an embodiment of the present disclosure;

FIG. 6 shows an effect diagram of a state of attention provided by an embodiment of the present disclosure;

FIG. 7 shows a schematic diagram of a gesture prompt box provided by an embodiment of the present disclosure;

FIG. 8A shows a schematic diagram of another display interface provided by an embodiment of the present disclosure;

FIG. 8B shows a schematic diagram of another display interface provided by an embodiment of the present disclosure;

FIG. 9 shows a schematic structural diagram of a state recognition device for a display area provided by an embodiment of the present disclosure;

FIG. 10 shows a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure.

Detailed ways

In order to make the objectives, technical solutions, and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be described clearly and completely in conjunction with the accompanying drawings in the embodiments of the present disclosure. Obviously, the described embodiments It is only a part of the embodiments of the present disclosure, but not all the embodiments. The components of the embodiments of the present disclosure generally described and illustrated in the drawings herein may be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the accompanying drawings is not intended to limit the scope of the claimed present disclosure, but merely represents selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative work shall fall within the protection scope of the present disclosure.

In related technologies, when monitoring each display area, the method of monitoring the crowd density in the current display area is generally adopted. However, because only the statistical crowd density is used, a specific analysis of the crowd in the display area cannot be realized, for example, it is impossible to analyze the crowd density in the display area. Realize the analysis of the attention time of the crowds in the display area, the attribute information of the users in the crowds, and so on, and then it is impossible to realize the analysis of the user's interest in the display area.

Based on this, in the method for identifying the status of the display area provided by the present disclosure, the passenger flow data of the display area in the first time period can be determined based on the collected video images, and the determined passenger flow data includes each user appearing in the display area At least two of the length of stay of the user, the length of attention of each user, the attribute information of each user, the number of passengers, and the number of passengers, which can be combined with at least two characteristics in the passenger flow data to determine the crowd state data. Therefore, the crowd state data is determined based on the crowd state Data, when controlling the display status of the display area in the display interface, the characteristics of the crowd in the display area can be displayed more accurately and intuitively.

The method for identifying the state of the display area provided by the embodiments of the present disclosure can be applied to a server or to a terminal device that supports a display function. The server may be a local server or a cloud server, etc. The terminal device may be a smart phone, a tablet computer, a personal digital assistant (PDA), a smart TV, etc., which is not limited in this application.

When the embodiments of the present disclosure are applied to a server, the data in the present disclosure can be obtained from a terminal device and/or a camera device, and the display state corresponding to the display area in the display interface can be controlled by issuing a control command to the terminal device. , To realize the control of the display state corresponding to the display area in the display interface of the terminal device.

To facilitate the understanding of the embodiments of the present disclosure, a method for identifying the state of the display area is first introduced in detail.

FIG. 1 is a schematic flowchart of a method for recognizing the status of a display area provided by an embodiment of the present disclosure. The method includes steps S101 to S104.

S101. Obtain a video image collected in the display area.

S102. Based on the collected video images, determine the passenger flow data of the display area in the first time period. The passenger flow data includes the stay time of each user appearing in the display area, the attention time of each user, the attribute information of each user, the number of visitors, At least two of the passenger flow.

S103. Using the passenger flow data, determine the crowd state data of the display area in the first time period.

S104. Using the crowd state data, control the display state corresponding to the display area in the display interface.

The method provided in this embodiment can determine the passenger flow data based on the collected video images, use the passenger flow data to determine the crowd state data in the display area within the first time period, and use the crowd state data to control the display corresponding to the display area in the display interface status. The display state corresponding to the display area can be used to indicate the state of the crowd in the display area, thereby realizing dynamic monitoring of the state of the crowd in the display area, and then analyzing the people of interest in the display area.

When the video screen includes multiple display areas, according to the crowd status data of different display areas, different display areas corresponding to different display states can be controlled to be presented in the display interface, and then the crowd status of different display areas can be compared and analyzed.

In an embodiment of the present disclosure, the crowd status data includes crowd popularity levels. The crowd popularity level can be determined according to the following methods: based on the weight coefficients corresponding to at least two types of passenger flow data in the passenger flow data, and the values of at least two types of passenger flow data, determine the crowd popularity value, and then determine the crowd popularity rating corresponding to the range of the crowd popularity value. .

Specifically, the crowd popularity value range corresponding to different crowd popularity levels can be preset. For example, the crowd popularity value range corresponding to the first-level crowd popularity is 0-10, and correspondingly, the crowd popularity value within the first-level crowd popularity Possible values are 0, 5, and 10; the range of crowd popularity values corresponding to the second-level crowd popularity is 11-20. Correspondingly, the crowd popularity values within the second-level crowd popularity may take the values 11, 15, and 20; The crowd fever value range corresponding to the third-level crowd fever is 21-30. Correspondingly, the crowd fever value within the third-level crowd fever can take values of 21, 25, and 30. After the crowd popularity value is determined according to the value of the passenger flow data and the weight coefficient of each type of passenger flow data, the range to which the crowd popularity value belongs can be determined, and then the corresponding crowd popularity level; for example, if the determined crowd popularity value is 25, you can Determine the corresponding crowd popularity level as the third-level crowd popularity.

Exemplarily, the crowd popularity value calculated based on the number of passengers and the number of passengers can be as follows:

p=ax ₁ +bx ₂ formula (1)

Among them, p represents the popularity value of the crowd, x ₁ represents the value of passenger flow, x ₂ represents the value of passenger flow, a represents the weight of passenger flow, and b represents the weight of passenger flow.

Exemplarily, the crowd popularity value may also be calculated based on the combination of the number of passengers and the length of stay, or the crowd popularity value may be calculated based on the combination of the number of passengers and the length of attention, etc., which is not limited in this application.

When using the crowd state data to control the display state of the display area in the display interface, it can be used to control the display area of the display area corresponding to the display area in the display interface to present a color level that matches the crowd popularity level.

The heat level of different people matches different color levels. For example, the color level matching the popularity of the first-level crowd may be green, the color matching the popularity of the second-level crowd may be yellow, and the color matching the popularity of the third-level crowd may be red. Or, different color levels can present different gray values. Among them, the color levels matched by the heat levels of different groups of people may be preset.

After determining the popularity level of the crowd, the color level matching the popularity level of the crowd can be determined, and then the layer of the display area corresponding to the display area in the display interface is controlled to present the color of the color level. Exemplarily, the display schematic diagram of the display area of different color levels in the display interface may be as shown in FIG. 2. In FIG. 2, different gray areas represent different display areas, and the gray value represents the popularity of the crowd in the display area, and the gray value The higher, the higher the popularity of the crowd.

When the display interface includes the display area corresponding to each display area, according to the color of the layer of each display area, the crowd popularity information of different display areas can be displayed, and the display mode is more intuitive.

In another embodiment of the present disclosure, the display area in the display interface can also be controlled to display display states related to passenger flow data. For example, according to different types of passenger flow data, the display interface can be controlled to present displays related to different types of passenger flow data. status.

In a possible implementation manner, using passenger flow data to determine the crowd status data of the display area in the first time period includes: determining crowd distribution data with different attributes based on the attribute information of each user, and the crowd distribution data includes Distribution data of at least one attribute of age, gender, charm value, and emotion. Exemplarily, the population statistics of the population distribution data may be calculated based on the number of passenger flow. In this case, the passenger flow data includes at least the attribute information of each user appearing in the display area and the number of passenger flow.

Correspondingly, the use of crowd state data to control the display state of the display area in the display interface includes: using crowd distribution data with different attributes to control the display interface to separately display the crowd distribution map under each attribute.

Among them, the population distribution graph under each attribute displayed in the display interface can be a pie graph, a ring graph, a bar graph, etc. Exemplarily, the population distribution diagram under each attribute in the display interface can be shown in Figure 3. The attributes shown in Figure 3 are gender, age, charm value, emotion, and the legend next to the distribution of each attribute is used to indicate The proportion of the quantity in different intervals. For example, "XXXX" next to the age distribution of passenger flow can indicate different age ranges, and "XXX" can indicate the proportion of the number of people in the corresponding age range to the total number of people (or it can indicate the number of people in the corresponding age range); next to the gender distribution of passenger flow The "XXX" can indicate the proportion of users of different genders (or the number of people corresponding to the gender); the "XXXX" next to the charm value distribution can indicate different charm value intervals, and "XXX" can indicate the corresponding charm value interval The proportion of the number of people in the total number of people (or the number of people in the corresponding charm value interval); "XXXX" next to the emotional distribution can indicate different types of emotions, and "XXX" can indicate the number of people with corresponding types of emotions in the total number of people Proportion (or the number of people that can represent the corresponding type of emotion).

In another embodiment of the present disclosure, the crowd state data includes the number of passengers and the number of passengers at multiple time nodes in the first time period. When determining the number of passenger flow and passenger flow at multiple time nodes in the first time period, the passenger flow number and passenger flow may be obtained from the passenger flow data.

In order to reduce the amount of calculation, when determining the number of passenger flow and the number of passenger flow in the display area in the first time period, the number of passenger flow and the number of passenger flow may be determined every preset time period, for example, the number of passenger flow and the number of passenger flow may be determined every 1 minute.

Specifically, when determining the number of passenger flow, the total number of appearances of each user in the display area in the first time period can be determined. Among them, for each user, the start time of each user entering the display area can be recorded. If the interval between the two recorded start times exceeds the first duration, it is determined that the user appears once, and the first duration does not exceed the first duration. segment.

Exemplarily, if the starting time of recording the user entering the display area A is 10:00, 10:08, 10:11, the first time length is 10 minutes, and the time length of the interval between 10:00-10:08 is less than The first time length, it is determined that the user appears zero times, and the time length of the interval between 10:00-10:11 is greater than the first time length, it is determined that the user appears once.

In the above method, when the interval between the two start times of the recording exceeds the first time length, it is determined that the user appears once. In this way, the repeated statistics of the number of user appearances in a relatively short period of time can be avoided, and the passenger flow statistics can be improved. Accuracy.

When determining the number of passengers, the number of face images contained in the captured video images within the first preset time period can be acquired, and then the acquired face images can be de-duplicated, that is, the unified user face It appears in the video images at different times, it is determined that the number of visitors corresponding to the user is 1, and the number of facial images after deduplication is determined as the number of visitors.

Considering that both the number of passenger flow and the number of passenger flow change in real time, in order to reduce the amount of calculation, the number of passenger flow and the number of passenger flow at multiple time nodes in the first time period can be obtained. Among them, the time between any two adjacent time nodes The interval can be the same.

In a possible implementation, the number of passengers and the number of passengers at multiple time nodes in the first time period can be displayed in the same coordinate system. For example, the time node can be used as the abscissa, and each time node can be used as the abscissa. The corresponding number of passengers is the ordinate, and a line graph is drawn. Then, in the same coordinate system, the same time node is the abscissa, and the number of passengers corresponding to each time node is the ordinate. The line graph is drawn, and the line graph is displayed. The changing trend of the number of passenger flow and the changing trend of the number of passenger flow in the display area within a period of time; in another example of the present disclosure, the passenger flow of the display area in the first period of time can also be displayed by drawing a bar graph, a pie chart, etc. The changing trend of the number of people and the changing trend of the number of passenger flows.

Exemplarily, the change trend of the number of passenger flow and the change trend of the number of passenger flow displayed on the display interface and the comparison result between the two may be shown in FIG. 4.

In the foregoing embodiment, the change trend of the number of passenger flow and the change trend of the number of passenger flow in the display area is displayed through the display interface, so that the change of the number of passenger flow and the number of passenger flow in the display area can be displayed more intuitively.

In another embodiment of the present disclosure, the crowd status data includes the total attention duration of the crowd collection of each type of attribute. For example, the customer flow data includes the attention duration of the user and the attribute information of each user. Among them, the attribute information of the user includes at least two attributes of age, gender, charm value, and emotion.

When determining the user's attention time, the method shown in FIG. 5 can be used, including the following steps.

S501: For each user, identify the face orientation data of the user in the collected video frame.

Among them, the face orientation data includes the pitch angle and the yaw angle of the face. By judging whether the pitch angle is within the first angle range and the yaw angle is within the second angle range, it can be determined whether the user pays attention to the displayed object in the display area.

S502: In a case where it is detected that the face orientation data indicates that the user pays attention to the display object in the display area, record the time when the user pays attention to the display object for the first time.

In the case where the pitch angle in the face orientation data is within the first angle range and the yaw angle in the face orientation data is within the second angle range, it can be determined that the face orientation data indicates that the user pays attention to the display of the display area object.

S503: Based on the recorded times when the user first paid attention to the display object, determine the time period during which the user paid attention to the display object in the first time period, and the attention time period does not exceed the first time period. period.

Specifically, when determining the length of time the user pays attention to the displayed object in the first time period, you can first determine the start time when the user starts to pay attention to the displayed object in the first time period, and then determine the time between the start time when the user pays attention to the displayed object twice. If the interval exceeds the set duration, the interval is taken as the time duration for the user to pay attention to the displayed object at one time.

Exemplarily, if the first time period is from 10:00 to 10:30, the start time when the user pays attention to the displayed object is 10:02, 10:05, 10:13, 10:15, 10:25, and the first time period is 10:02, 10:05, 10:13, 10:15, and 10:25 respectively. If it is 10 minutes, the first time of attention is 10:13-10:02=11 minutes, and the second time of attention is 10:25-10:13=12 minutes.

In the above method, it is possible to determine the length of time that the user pays attention to the displayed object in the first time period only according to the recorded time when the user pays attention to the displayed object for the first time. Therefore, the interval between the time when the user first pays attention to the displayed object can be prevented from being too short. The resulting frequent statistics of the attention time reduces the amount of calculation when calculating the attention time, and also makes the calculated attention time more reasonable.

Specifically, when using the attention time of each user and the attribute information of each user to determine the total attention time of the group of people of each type of attribute in the first time period, the attribute information of each user appearing in the display area can be used to determine the total attention time of each user. Users are divided into groups of different types of attributes; then, the attention time of each user in the display area of each type of attribute is obtained, and each type of attention is determined based on the corresponding length of attention of each user in the group of each attribute. The total attention time of the attribute's crowd collection.

In a possible implementation manner, the total attention time of the crowd collection of each type of attribute in the first time period may be used to display the attention state effect diagram of the crowd collection of each type of attribute on the display interface.

Among them, the attention effect map can be displayed by specific graphics; the total attention time of the group of people of each type of attribute is different, and the display effect of the corresponding specific image is also different. Exemplarily, the following state effect diagram can be shown in Figure 6. Figure 6 shows the distribution of users with different types of attributes whose total time is in the same interval. Taking the attributes as gender and age as an example, you can first Determine the subset of the population with male gender and the subset of population with female gender in the population set, and then determine the age subsets of each age group in the gender subset of the male population, and determine the age subsets of the gender subset of the female population. According to the age subsets of the segments, the diameters of the corresponding circles are determined according to the determined age subsets. The "XXXX" in the circles indicate different age ranges. Among them, the divided subsets of different genders and different age groups can be regarded as a collection of people with different types of attributes.

Further, according to the calculation method of the above-mentioned attention time length, the attention time of each user in the crowd set of different types of attributes can be counted, and then based on the total attention time of each user under the crowd set of different types of attributes, the crowd set of various attributes can be displayed. The rendering of the state of attention. For example, as shown in Figure 6, the longer the total time of attention is for the group of people, the larger the size of the corresponding attention state effect map.

In the foregoing embodiment, the population distribution map under each attribute can be displayed according to the population distribution data under different attributes, so as to realize the analysis of people with different attributes with different total time of attention.

In another embodiment of the present disclosure, the crowd status data includes the total stay duration of the crowd set of each type of attribute. At this time, the passenger flow data includes the stay duration and the stay duration of each user appearing in the display area within the first time period. User attribute information.

Among them, when determining the stay time of each user appearing in the display area, it may be determining the total stay time of each user each time they appear in the first time period; among them, for each user, when it is determined that the user enters the display twice In the case where the interval between the start times of the regions exceeds the first duration, the interval is taken as the duration of one stay of the user.

Exemplarily, if the first time period is from 10:00 to 10:30, the start time of the user entering the display area is 10:02, 10:05, 10:13, 10:15, 10:25, the first time period If it is 10 minutes, the length of the first stay is 10:13-10:02=11 minutes, and the length of the second stay is 10:25-10:13=12 minutes.

In a possible implementation manner, the staying time of each user appearing in the display area during the first time period and the attribute information of each user may be used to determine the total staying time of each type of attribute crowd set, and then use The total stay time of the crowd collection of each type of attribute, and the effect diagram of the stay state of the crowd collection of each type of attribute is displayed on the display interface.

Specifically, based on the attribute information of each user appearing in the display area, each user can be divided into a crowd set of different attributes; then the stay time of each user in the display area in the crowd set of each type of attribute can be obtained; The stay time corresponding to each user in the crowd set of the category attribute determines the total stay time of the crowd set of each category attribute.

The staying state renderings are displayed by specific images. The total stay time of the crowd set of each type of attribute is different, and the corresponding specific image display effects are also different. Exemplarily, the staying state effect diagram may also be as shown in FIG. 3, but at this time, what is shown in FIG. 3 is the distribution of staying time of users with different attributes within the same interval.

In the foregoing embodiment, the population distribution map under each attribute can be displayed according to the population distribution data under different attributes, so as to realize the analysis of the population of different attributes with different total length of stay.

In an example of the present disclosure, after the trigger operation for the display area in the display interface is obtained, the crowd state data can be used to control the display state of the display area in the display interface.

Wherein, the acquired trigger operation for the display area in the display interface may be to control the display interface to display a gesture prompt box, which includes gesture description content, and then detects the user gesture in the video screen captured by the camera device, and then detects the user gesture If the user's posture is consistent with the posture recorded in the posture description content, it is confirmed that the trigger operation for the display area in the display interface is obtained.

Exemplarily, the posture prompt box can be as shown in Figure 7. The posture prompt box includes posture description information of "please be in the viewfinder frame, raise your hand to confirm", and the human posture shown in the line part is used to prompt the user to be in the corresponding position Make this gesture. When it is detected that the line part of the user's framing frame makes a hand-raising action, it is determined that the trigger operation for the display area in the display interface is obtained.

In another example of the present disclosure, the trigger operation for the display area in the display interface is obtained, and it can also be detected that the position corresponding to the display area on the screen is clicked. The click methods include but are not limited to single click, double click, long press, and double click. Press etc.; or, to obtain the trigger operation for the display area in the display interface, it can also be the voice command input by the user is received, and then the voice instruction is parsed to determine the trigger operation for the display area in the display interface.

In combination with the foregoing implementation manners, the following specific examples are given:

When the passenger flow data appears in the display area for the stay time of each user, the attention time of each user, the attribute information of each user, the number of visitors, and the number of visitors, the initial display interface may be as shown in Figure 8A, in Figure 8A The gray values of different heat levels in the left display area of the display area are different, and the right display area displays the comparison between the change trend of the number of visitors and the change trend of the number of visitors in the display area during the first time period Results, the population distribution diagram under each attribute, and the statistical results of passenger flow data. Among them, the statistical results of passenger flow data include the number of today's passenger flow, today's passenger flow, total stay time today, the number of historical passenger flow, historical passenger flow, and per capita stay time, etc.; When it is detected that any one of the display areas in the left display area is triggered, the display interface can display as shown in Figure 8B. In Figure 8B, the effect of the state of interest displayed in the right display area in Figure 8A is replaced by The interface diagram corresponding to the triggered display area. The interface diagram can display the passenger flow data statistics of the triggered display area (including the number of passengers, the number of passengers, the length of stay, etc.), the number of passengers/the changing trend of the visitors, and the analysis of the people of interest (such as Pay attention to the status renderings) and so on. Among them, in the passenger flow data statistics results shown in FIG. 8A and FIG. 8B, various values can be changed in real time, and the passenger flow data statistics results in a period of time, such as today, 7 days, and 30 days, can be counted. The "today" refers to counting from today to the current time.

Those skilled in the art can understand that in the above-mentioned methods of the specific implementation, the writing order of the steps does not mean a strict execution order but constitutes any limitation on the implementation process. The specific execution order of each step should be based on its function and possibility. The inner logic is determined.

Based on the same concept, the embodiment of the present disclosure also provides a state recognition device for the display area. FIG. 9 is a schematic structural diagram of a state recognition device for a display area provided by an embodiment of the disclosure. The device includes an acquisition module 901, a first determination module 902, a second determination module 903, and a control module 904.

The obtaining module 901 is used for obtaining the video images collected in the display area.

The first determining module 902 is configured to determine the passenger flow data of the display area in the first time period based on the collected video images. The passenger flow data includes the stay time of each user appearing in the display area, and each user At least two of the following time, the attribute information of each user, the number of visitors, and the number of visitors.

The second determining module 903 is configured to use the passenger flow data to determine the crowd state data of the display area in the first time period.

The control module 904 is configured to use the crowd state data to control the display state corresponding to the display area in the display interface.

In a possible implementation manner, the crowd state data includes crowd popularity levels; the control module 904, when using the crowd state data to control the display state corresponding to the display area in the display interface, is used to: control The layer of the display area corresponding to the display area in the display interface presents a color level matching the popularity level of the crowd.

In a possible implementation manner, the second determining module 903 is further configured to determine the popularity level of the crowd according to the following method: based on the weight coefficients corresponding to at least two kinds of passenger flow data in the passenger flow data, and the at least The two kinds of passenger flow data values determine the crowd popularity value, and determine the crowd popularity level corresponding to the range of the crowd popularity value.

In a possible implementation manner, in a case where the passenger flow data includes the number of passenger flow and the number of passenger flow, the second determining module 903 uses the passenger flow data to determine that within the first time period When the crowd status data in the display area is used, it is used to: use the number of passengers and the number of passengers to obtain the number of passengers and the number of passengers at multiple time nodes in the first time period;

The control module 904 uses the crowd state data to control the display state corresponding to the display area in the display interface for: based on the number of passengers and the number of passengers at multiple time nodes in the first time period , Displaying the changing trend of the number of passenger flows and the changing trend of the number of passenger flows in the display area in the first time period on the display interface.

In a possible implementation manner, the second determining module 903, when using the passenger flow data to determine the crowd status data of the display area in the first time period, is configured to: based on the attribute information of each user , Determining population distribution data with different attributes, where the population distribution data includes distribution data having at least one attribute of age, gender, charm value, and emotion;

The control module 904, when using the crowd state data to control the display state corresponding to the display area in the display interface, is used to: use the crowd distribution data with the different attributes to control the respective display in the display interface Distribution map of the population with each attribute.

In a possible implementation manner, in a case where the passenger flow data includes the attention duration of each user and the attribute information of each user, the second determining module 903 determines the first time by using the passenger flow data. When the crowd status data of the display area in the paragraph is used: based on the attribute information of each user appearing in the display area, divide each user into a crowd set of different types of attributes; obtain the crowd set of each type of attribute The attention time of each user in the display area; based on the attention time corresponding to each user in the crowd set of each type of attribute, determine the total attention time of the crowd set of each type of attribute;

The control module 904, when using the crowd state data to control the display state corresponding to the display area in the display interface, is used to: control the display on the display interface based on the total attention time of the crowd set of each type of attribute The effect diagram of the attention state of the crowd collection of each type of attribute.

In a possible implementation manner, the users in the population set of each type of attribute meet at least two attributes of age, gender, charm value, and emotion.

In a possible implementation manner, the attention state effect diagram is displayed by a specific graphic; the total attention time of the group of people of each type of attribute is different, and the display effect of the corresponding specific image is also different.

In a possible implementation manner, the first determining module 902 is further configured to determine the attention duration according to the following manner: for each user, identifying the face orientation data of the user in the collected video frame;

In the case where it is detected that the face orientation data indicates that the user pays attention to the display object in the display area, record the time when the user pays attention to the display object for the first time; The time for displaying the object, and determining the attention time period during which the user pays attention to the displayed object in the first time period, and the attention time period does not exceed the first time period.

In a possible implementation manner, the face orientation data includes the pitch angle and the yaw angle of the face; the first determining module 902 is configured to: when the pitch angle is within a first angle range, and the pitch angle When the yaw angle is within the second angle range, it is determined that the face orientation data indicates that the user pays attention to the display object in the display area.

In a possible implementation manner, the first determining module 902 is further configured to determine the number of visitors in the display area in the first time period according to the following method: determining the display area in the first time period The total number of appearances of each user in each user; among them, for each user, record the start time of each user entering the display area; if the interval between the two recorded start times exceeds the first length of time, determine all The user appears once; the first time period does not exceed the first time period.

In a possible implementation manner, the first determining module 902 is further configured to determine the length of stay of the user in the display area in the first time period according to the following method: determining that each user stays in the display area during the first time period. The total length of stay at the second occurrence; wherein, for each user, if the interval between the start time when the user enters the display area is determined to exceed the first duration, the interval is taken as the user The duration of a stay.

In a possible implementation manner, the obtaining module 901 is further configured to obtain information about the display area in the display interface before using the crowd state data to control the display state corresponding to the display area in the display interface. Trigger action.

In a possible implementation manner, the acquiring module 901, when acquiring a trigger operation on the display area in the display interface, is used to: control the display interface to display a gesture prompt box, the gesture prompt box Include posture description content; detect the user posture in the collected video screen; in the case that the user posture is consistent with the posture recorded by the posture description content, confirm that the display interface is acquired for the display area Trigger action.

In some embodiments, the functions or templates contained in the device provided in the embodiments of the present disclosure can be used to execute the methods described in the above method embodiments, and the specific implementation can refer to the description of the above method embodiments.

Based on the same technical concept, the embodiments of the present disclosure also provide an electronic device. FIG. 10 is a schematic structural diagram of an electronic device provided by an embodiment of the disclosure. The electronic device includes a processor 1001, a memory 1002, and a bus 1003. Among them, the memory 1002 is used to store execution instructions, including a memory 10021 and an external memory 10022; the memory 10021 here is also called an internal memory, which is used to temporarily store the calculation data in the processor 1001 and the data exchanged with the external memory 10022 such as a hard disk. The processor 1001 exchanges data with the external memory 10022 through the memory 10021. When the electronic device 1000 is running, the processor 1001 and the memory 1002 communicate through the bus 1003, so that the processor 1001 executes the following instructions:

Obtain the video images collected in the display area;

Based on the collected video images, determine the passenger flow data of the display area in the first time period. The passenger flow data includes the stay time of each user appearing in the display area, the attention time of each user, and the user's At least two of attribute information, number of passengers, and number of passengers;

Using the passenger flow data to determine crowd state data in the display area in the first time period;

Using the crowd state data, the display state corresponding to the display area in the display interface is controlled.

In addition, the embodiments of the present disclosure also provide a computer-readable storage medium having a computer program stored on the computer-readable storage medium, and when the computer program is run by a processor, the state of the display area described in the above method embodiment is recognized. Method steps.

The computer program product of the method for identifying the status of the display area provided by the embodiment of the present disclosure includes a computer-readable storage medium storing program code, and the instructions included in the program code can be used to execute the display area described in the above method embodiment The steps of the state recognition method.

Those skilled in the art can clearly understand that, for the convenience and conciseness of the description, the specific working process of the system and device described above can refer to the corresponding process in the foregoing method embodiment, which will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed system, device, and method may be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division, and there may be other divisions in actual implementation. For example, multiple units or components may be combined or It can be integrated into another system, or some features can be ignored or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some communication interfaces, indirect coupling or communication connection of devices or units, and may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.

In addition, the functional units in the various embodiments of the present disclosure may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit.

If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a non-volatile computer readable storage medium executable by a processor. Based on this understanding, the technical solution of the present disclosure essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including Several instructions are used to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in the various embodiments of the present disclosure. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disks or optical disks and other media that can store program codes. .

The above are only specific implementations of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present disclosure, and they shall be covered Within the protection scope of this disclosure. Therefore, the protection scope of the present disclosure should be subject to the protection scope of the claims.

Claims (17)

1. A method for identifying the status of a display area, which is characterized in that it includes:

   Acquiring a video screen collected in the display area;

   Based on the collected video images, determine the passenger flow data of the display area in the first time period. The passenger flow data includes the stay time of each user appearing in the display area, the attention time of each user, and the user's At least two of attribute information, number of passengers, and number of passengers;

   Using the passenger flow data to determine crowd state data in the display area in the first time period;

   Using the crowd state data, the display state corresponding to the display area in the display interface is controlled.
2. The method according to claim 1, wherein the crowd state data includes crowd popularity levels; and the using the crowd state data to control the display state corresponding to the display area in the display interface includes:

   Controlling the layer of the display area corresponding to the display area in the display interface to present a color level that matches the popularity level of the crowd.
3. The method according to claim 2, wherein the popularity level of the crowd is determined according to the following method:

   Determining the crowd popularity value based on the respective weight coefficients corresponding to the at least two types of passenger flow data in the passenger flow data and the values of the at least two types of passenger flow data;

   Determine the crowd's popularity level corresponding to the range of the crowd's popularity value.
4. The method according to claim 1, wherein in the case that the passenger flow data includes the number of passenger flow and the number of passenger flow, the passenger flow data is used to determine the passenger flow data in the first time period. Crowd status data in the display area, including:

   Using the number of passenger flow and the number of passenger flow to obtain the number of passenger flow and the number of passenger flow at multiple time nodes in the first time period;

   The using the crowd state data to control the display state corresponding to the display area in the display interface includes:

   Based on the number of passenger flow and the number of passenger flow at multiple time nodes in the first time period, the change trend of the number of passenger flow and the change trend of the number of passenger flow in the display area in the first time period are displayed on the display interface .
5. The method according to claim 1, wherein the using the passenger flow data to determine the crowd status data of the display area in the first time period comprises:

   Determine population distribution data with different attributes based on the attribute information of each user, where the population distribution data includes distribution data with at least one attribute of age, gender, charm value, and emotion;

   The using the crowd state data to control the display state corresponding to the display area in the display interface includes:

   Using the crowd distribution data with the different attributes, control the display interface to separately display the crowd distribution map with each attribute.
6. The method according to claim 1, characterized in that, in the case that the passenger flow data includes the attention duration of each user and the attribute information of each user, the passenger flow data is used to determine that the time period is within the first time period. The crowd status data in the display area includes:

   Based on the attribute information of each user appearing in the display area, classify each user into a group of people with different attributes;

   Acquiring the attention time of each user in the display area in the crowd set of each type of attribute;

   Determine the total attention time of the crowd set of each type of attribute based on the attention time corresponding to each user in the crowd set of each type of attribute;

   The using the crowd state data to control the display state corresponding to the display area in the display interface includes:

   Based on the total attention time length of the crowd collection of each type of attribute, the display interface is controlled to present the attention state effect diagram of the crowd collection of each type of attribute.
7. The method according to claim 6, wherein the users in the population set of each type of attribute meet at least two attributes of age, gender, charm value, and emotion.
8. The method according to claim 6, wherein the attention state effect diagram is displayed by a specific graphic; the total attention time of the crowd set of each type of attribute is different, and the display effect of the corresponding specific image is also different.
9. The method according to any one of claims 1, 6 to 8, wherein the attention duration is determined according to the following manner:

   For each user, identifying the face orientation data of the user in the collected video frame;

   In a case where it is detected that the face orientation data indicates that the user pays attention to the display object in the display area, record the time when the user first pays attention to the display object;

   Based on the recorded times when the users first paid attention to the display object, determine the time period during which the user paid attention to the display object in the first time period, and the attention time period does not exceed the first time period .
10. The method according to claim 9, wherein the face orientation data includes a pitch angle and a yaw angle of the face;

    When the pitch angle is within a first angle range and the yaw angle is within a second angle range, it is determined that the face orientation data indicates that the user pays attention to the display object in the display area.
11. The method according to claim 1 or 4, wherein the number of passengers in the display area in the first time period is determined according to the following method:

    Determining the total number of appearances of each user in the display area in the first time period;

    Wherein, for each user, record the start time each time the user enters the display area; if the interval between the two recorded start times exceeds the first time length, it is determined that the user appears once; The duration of one time does not exceed the first time period.
12. The method according to claim 1, characterized in that the length of time the user stays in the display area in the first time period is determined according to the following manner:

    Determine the total length of stay of each user each time they appear in the first time period;

    Wherein, for each user, in a case where it is determined that the interval between the start time when the user enters the display area twice exceeds the first duration, the interval is taken as the duration of one stay of the user.
13. The method according to any one of claims 1 to 12, further comprising: before using the crowd state data to control the display state corresponding to the display area in the display interface,

    Acquire the trigger operation for the display area in the display interface.
14. The method according to claim 13, wherein said acquiring a trigger operation for said display area in said display interface comprises:

    Controlling the display interface to display a gesture prompt box, the gesture prompt box including gesture description content;

    Detecting the user gesture in the collected video frame;

    In the case that the user posture is consistent with the posture recorded in the posture description content, it is confirmed that the trigger operation for the display area in the display interface is acquired.
15. A state recognition device for a display area, which is characterized in that it comprises:

    The acquisition module is used to acquire the video images collected in the display area;

    The first determining module is configured to determine the passenger flow data of the display area in the first time period based on the collected video images, where the passenger flow data includes the stay time of each user appearing in the display area, and each user At least two of the following time, the attribute information of each user, the number of visitors, and the number of visitors;

    A second determining module, configured to use the passenger flow data to determine crowd status data in the display area in the first time period;

    The control module is configured to use the crowd state data to control the display state corresponding to the display area in the display interface.
16. An electronic device, characterized by comprising: a processor and a memory, the memory storing machine-readable instructions executable by the processor, and when the machine-readable instructions are executed by the processor, the The processor executes the method for identifying the status of the display area according to any one of claims 1 to 14.
17. A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is run by a processor, the processor executes any one of claims 1 to 14 The state recognition method of the display area described above.

Images