KR20210088600A - Exhibition area state recognition method, apparatus, electronic device and recording medium - Google Patents

Abstract

The present invention provides a display area state recognition method, apparatus, electronic device and recording medium. In one example, the method includes: acquiring a video screen collected in an exhibition area; Based on the collected video screen, determining customer flow data of the exhibition area within a first time period, wherein the customer flow data includes the duration of stay of each user who appeared in the exhibition area, the continuation of attention of each user. including at least two of time, attribute information of each user, the number of customer flows, and the number of customer flows; determining crowd state data of the exhibition area within the first time period using the customer flow data; and controlling a display state corresponding to the exhibition area in a display interface by using the crowd state data.

Description

Exhibition area state recognition method, apparatus, electronic device and recording medium

[Citation of cross-reference to related application]

The present invention claims the priority of the Chinese patent application with the filing date of December 31, 2019 and the application number of 2019114163365, and all contents of the Chinese patent application are incorporated into this application by citing it.

[Technology]

The present invention relates to the field of computer technology, and more particularly, to a method, an apparatus, an electronic device, and a recording medium for recognizing an exhibition area state.

In exhibition places such as supermarkets, shopping stores, and exhibition halls, various items are usually displayed in each exhibition area so that users can purchase or observe them. A user's level of interest in an item in each exhibition area is one of the issues that many stores or exhibitions are paying attention to. However, it is very difficult to analyze the user's interest in articles in each exhibition area because most of the characteristics of the flow of people in a store or exhibition are reflected by the trajectory of the user's activity in a dense crowd or in an irregular state.

In view of this, the present invention at least provides a solution of exhibition area state recognition.

According to a first aspect, the present invention provides a method for recognizing a state of an exhibition area, the method comprising:

acquiring video footage collected in the exhibition area;

Based on the collected video screen, determining customer flow data of the exhibition area within a first time period, wherein the customer flow data includes the duration of stay of each user who appeared in the exhibition area, the continuation of attention of each user. including at least two of time, attribute information of each user, the number of customer flows, and the number of customer flows;

determining crowd state data of the exhibition area within the first time period using the customer flow data; and

and controlling a display state corresponding to the exhibition area in a display interface by using the crowd state data.

In the above method, the customer flow data is determined based on the collected video screen, the crowd status data of the exhibition area in the first time zone is determined using the customer flow data, and the display interface is used by using the crowd status data. It is possible to control the display state of the exhibition area within. Since the display state of the exhibition area can prompt the crowd state data in the exhibition area, dynamic monitoring of the crowd state of the exhibition area can be realized, and also the attention crowd of the exhibition area can be analyzed.

In one possible embodiment, the crowd state data includes a crowd popularity level, and using the crowd state data to control a display state corresponding to the display area in the display interface is to be performed in the display area in the display interface. and controlling a layer of a corresponding display area to display a color level matching the crowd popularity level.

By displaying different crowd popularity levels using different color levels, it is possible to enrich the way of indicating the display state of the exhibition area, while at the same time display the crowd popularity information in different exhibition areas more intuitively.

In one possible embodiment, the determining of the crowd popularity level comprises: a crowd popularity value based on a weighting coefficient respectively corresponding to at least two customer flow data of the customer flow data and a numerical value of the at least two customer flow data to confirm; and determining a crowd popularity level corresponding to a range to which the crowd popularity value belongs.

Since the crowd popularity value determined in this way combines the characteristics of at least two customer flow data, the confirmed crowd popularity value can relatively better reflect the customer flow characteristics of the exhibition area, and the confirmed crowd popularity value The level is relatively accurate.

In one possible embodiment, when the customer flow data includes the customer flow number of people and the customer flow number of people, the customer flow data is used to determine the crowd state data of the exhibition area within the first time zone. doing includes using the number of customer flow and the number of customer flow to obtain the number of customer flow and number of customer flow in a plurality of time nodes in the first time zone, and display using the crowd state data. Controlling the corresponding display state in the interface is based on the number of customer flows and the number of customer flows in the plurality of time nodes in the first time zone, according to the display interface, customers of the exhibition area in the first time zone. and displaying trends in the number of flow heads and trends in the number of customer flows.

In the above embodiment, the change status of the number of customer flows and the number of customer flows in the exhibition area can be displayed more intuitively.

In one possible embodiment, the determining of the crowd state data of the exhibition area within the first time period using the customer flow data determines the crowd distribution data having different attributes based on the attribute information of each user. Including, wherein the crowd distribution data includes distribution data of at least one attribute of age, gender, attractiveness value, and emotion,

Controlling the display state corresponding to the exhibition area in the display interface using the crowd state data may include displaying a crowd distribution map having each type of attribute in the display interface using the crowd distribution data having different attributes. including control to do so.

In the above embodiment, in order to display a more detailed crowd distribution situation of each exhibition area, the crowd distribution diagram of each type of attribute may be displayed based on the crowd distribution data of different attributes.

In one possible embodiment, when the customer flow data includes each user's attention duration and each user's attribute information, the crowd status data of the exhibition area within the first time period using the customer flow data The determining may include: classifying each user into a crowd group having different types of attributes based on attribute information of each user appearing in the exhibition area; acquiring attention duration within the display area of each user in the crowd group of each kind of attribute; and determining the total attention time of the crowd group of each kind attribute based on the attention duration corresponding to each user in the crowd group of each kind attribute,

Controlling the display state corresponding to the exhibition area in the display interface using the crowd state data is based on the total attention time of the crowd group of each kind of attribute, in the display interface, of the crowd group of each kind of attribute. and controlling to display the attention state effect diagram.

Based on the above embodiment, the attention status effect diagram of the crowd group of each type of attribute shown in the display interface may be used to indicate the degree of attention of the crowd of different types of attribute to the display area.

In one possible embodiment, the user in the crowd group of each kind of attribute satisfies at least two attributes of age, gender, attractiveness value and emotion.

In one possible embodiment, the attention state effect diagram is displayed in a specific figure, the total attention time of the crowd group of each type of attribute is different from each other, and the display effect of the corresponding specific image is also different from each other.

In one possible embodiment, the determining of the attention duration includes, for each user, recognizing, for each user, face direction data of the user in the collected video screen; when it is detected that the face orientation data indicates that the user is paying attention to the exhibition object in the exhibition area, recording a time when the user first paid attention to the exhibition object; and determining, based on the recorded times when a plurality of the users first paid attention to the exhibition object, a attention duration in which the user paid attention to the exhibition object within the first time period, wherein the attention duration is the second time period. 1 time zone not exceeded - inclusive.

In one possible embodiment, the face orientation data includes a pitch angle and a yaw angle of a face, and when the pitch angle is within a first angular range and the yaw angle is within a second angular range, the face orientation It is determined that the data indicates that the user paid attention to the exhibition object in the exhibition area.

In the above embodiment, since it is possible to determine the attention duration in which the user paid attention to the exhibition object within the first time period based on the recorded time when the user first paid attention to the exhibition object, the user can select the exhibition object. It is possible to avoid frequent statistics of the attention duration due to the interval with the first attention time being too short, and also, the method of calculating the attention duration time is relatively reasonable.

In one possible embodiment, the determining of the number of customers flowing in the exhibition area within the first time period comprises determining the total number of appearances of each user in the exhibition area within the first time period; Here, for each user, the start time when the user enters the exhibition area each time is recorded, and when the interval between the recorded two start times exceeds the first time, it is determined that the user appears once and, the first time does not exceed the first time period.

In the above embodiment, it is possible to avoid repetitive statistics of the number of user appearances in a relatively short time, and it is possible to improve the statistical precision of the number of customer flows.

In one possible embodiment, the determining of the duration of stay of the user in the exhibition area within the first time zone comprises determining the total duration of stay of each user each time they appear within the first time zone; , for each user, if it is determined that the interval between the start times when the user entered the exhibition area twice has exceeded the first time, the duration of the interval the user stayed when the user appeared once set to time

In the above embodiment, it is possible to avoid multiple calculations of the stay duration when the user appears multiple times in a relatively short time, the amount of calculation of the stay duration can be reduced, and the method of calculating the stay duration is also more reasonable.

In one possible embodiment, before using the crowd state data to control a display state corresponding to the exhibition area in the display interface, the method further comprises: acquiring a trigger operation for the exhibition area in the display interface.

In one possible embodiment, acquiring a trigger operation for the display area in the display interface includes: controlling the display interface to display a posture prompt frame including a posture description content; detecting a user's posture in the collected video screen; and when the user posture satisfies the posture recorded in the posture description content, confirming that a trigger operation for the display area in the display interface is acquired.

In the above embodiment, when acquiring the trigger operation on the display area of the display interface, it can be realized through the method of collecting the user posture in the video screen, so the trigger method of the trigger operation is enriched, and the interaction between the user and the device increased conversation.

According to a second aspect, an embodiment of the present invention provides an apparatus for recognizing a display area state, the apparatus comprising:

an acquisition module for acquiring the video screen collected in the exhibition area;

a first determining module for determining customer flow data of the exhibition area within a first time zone, based on the collected video screen, wherein the customer flow data includes a residence duration of each user who appears in the exhibition area, each including at least two of the user's attention duration, attribute information of each user, the number of customer flows, and the number of customer flows;

a second determining module for determining crowd state data of the exhibition area within the first time period by using the customer flow data; and

and a control module for controlling a display state corresponding to the exhibition area in a display interface by using the crowd state data.

According to a third aspect, the present invention provides an electronic device, wherein the electronic device includes a processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, the machine-readable instructions when executed by the processor, causes the processor to execute the display area state recognition method described in the first aspect or any embodiment above.

According to a fourth aspect, the present invention provides a computer readable recording medium having a computer program recorded thereon, and when the computer program is executed by a processor, the processor causes the processor to perform the above first aspect or any embodiment. Let the described exhibition area state recognition method be executed.

Hereinafter, in order to make the above objects, features and advantages of the present invention clearer and easier to understand, preferred embodiments will be described in detail with reference to the drawings as follows.

Hereinafter, in order to more clearly explain the technical solutions of the embodiments of the present invention, drawings necessary for the embodiments are briefly introduced. The drawings herein are incorporated in and constitute a part of this specification. These drawings present an embodiment of the present invention, and together with the specification are used to explain the technical solution of the present invention. It is to be understood that the following drawings only present specific embodiments of the present invention and are therefore not to be regarded as limiting the scope thereof. Other relevant drawings may be obtained from these drawings without any inventive effort to those skilled in the art.
1 is a schematic diagram showing a flow of a display area state recognition method provided by an embodiment of the present invention.
Fig. 2 is a schematic diagram showing the display of display areas of different color levels in the display interface provided by the embodiment of the present invention.
3 is a crowd distribution diagram of each kind of attribute in a display interface provided by an embodiment of the present invention;
Fig. 4 is a schematic diagram showing a comparison result between a change trend in the number of customer flows and a change tendency in the number of customer flows provided by the embodiment of the present invention.
5 is a schematic diagram illustrating a flow of a method for determining a user's attention duration provided by an embodiment of the present invention.
6 is an effect diagram of a state of attention provided by an embodiment of the present invention;
7 is a schematic diagram illustrating a posture prompt frame provided by an embodiment of the present invention.
8A is a schematic diagram illustrating another display interface provided by an embodiment of the present invention.
8B is a schematic diagram illustrating another display interface provided by an embodiment of the present invention.
9 is a schematic diagram showing the configuration of an apparatus for recognizing a display area state provided by an embodiment of the present invention.
10 is a schematic diagram showing the configuration of an electronic device provided by an embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be clearly and completely described with reference to the drawings in order to further clarify the purpose, technical solutions and advantages of the embodiments of the present invention. Naturally, the described embodiments are merely some and not all embodiments of the present invention. In general, the components of the embodiments of the present invention described or shown in the drawings herein may be arranged and designed in other configurations. Accordingly, the following detailed description of embodiments of the invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the invention. All other embodiments that can be obtained by those skilled in the art based on the embodiments of the present invention without making an inventive effort are included in the protection scope of the present invention.

In the related art, when monitoring each exhibition area, it generally adopts the method of monitoring the crowd density within the current exhibition area, but based on the statistical crowd density alone, it is possible to realize a detailed analysis of the crowd in the customer flow of the exhibition area. can't For example, analysis of the duration of attention of the crowd in the customer flow to the exhibition area, attribute information of the user in the crowd in the customer flow, etc. cannot be realized, nor can the analysis of the user's interest in the exhibition area be realized.

In view of this, the method for recognizing the state of the exhibition area provided by the present invention determines customer flow data of the exhibition area within the first time zone based on the collected video screen, and the determined customer flow data is stored in the exhibition area. At least two characteristics of the customer flow data because it may include at least two of the appearance duration of each user, the duration of each user's attention, the attribute information of each user, the number of customer flows, and the number of customers in the customer flow. The crowd state data can be determined with reference to, and when the display state of the exhibition area in the display interface is controlled based on the crowd state data, the crowd characteristics in the exhibition area can be displayed more accurately and intuitively.

The display area state recognition method provided by the embodiment of the present invention may be applied to a server or a terminal device supporting a display function. The server may be a local server or a cloud server, and the terminal device may be a smart phone, a tablet computer, a personal digital assistant (PDA), a smart television, or the like, but the present invention is not limited thereto.

When the embodiment of the present invention is applied to a server, the data in the present invention may be obtained from a terminal device and/or a photographing device, and the control of the display state corresponding to the display area in the display interface is performed by sending a control command to the terminal device. , it is possible to realize control of the display state corresponding to the display area in the display interface of the terminal device.

In order to facilitate understanding of the embodiment of the present invention, a method for recognizing the state of the exhibition area will be described in detail.

1 is a schematic diagram showing the flow of a display area state recognition method provided by an embodiment of the present invention, the method including steps S101 to S104.

In S101, the video screen collected in the exhibition area is acquired.

In S102, based on the collected video screen, the customer flow data of the exhibition area within the first time period is determined, wherein the customer flow data includes the duration of stay of each user who appeared in the exhibition area and the duration of attention of each user. , including at least two of attribute information of each user, the number of customer flows, and the number of customer flows.

In S103, the crowd state data of the exhibition area within the first time period is determined by using the customer flow data.

In S104, the display state corresponding to the exhibition area in the display interface is controlled using the crowd state data.

The method provided by this embodiment determines customer flow data based on the collected video screen, uses the customer flow data to determine the crowd status data of the exhibition area within the first time period, and uses the crowd status data Thus, the display state corresponding to the display area in the display interface may be controlled. Since the display state of the exhibition area can prompt the crowd state data in the exhibition area, dynamic monitoring of the crowd state of the exhibition area can be realized, and also the crowd paying attention to the exhibition area can be analyzed.

When a plurality of exhibition areas are included in the video screen, corresponding different display states representing each exhibition area in the display interface can be controlled based on the crowd state data of each exhibition area, and A comparative analysis can be run on crowd conditions.

In one embodiment of the present invention, the crowd status data includes a crowd popularity level. The crowd popularity level may be determined in the following manner, that is, the crowd popularity value is determined based on a weighting coefficient corresponding to at least two customer flow data in the customer flow data, respectively, and numerical values of the at least two customer flow data In addition, the crowd popularity level corresponding to the range to which the crowd popularity value belongs is determined.

Specifically, a range of crowd popularity values corresponding to different crowd popularity levels may be preset, for example, a crowd popularity value range corresponding to the first level crowd popularity is 0 to 10, and correspondingly, the first level crowd popularity value range is 0 to 10. The value of the crowd popularity value within the crowd popularity of may be 0, 5, 10, and the crowd popularity value range corresponding to the crowd popularity of the second level is 11-20, and correspondingly the crowd popularity value within the crowd popularity of the second level is in the range of 11-20. The value of the value may be 11, 15, 20, the crowd popularity value corresponding to the third level of crowd popularity is in the range of 21-30, and correspondingly the value of the crowd popularity value in the third level of crowd popularity is 21, It could be 25 or 30. After determining the crowd popularity value based on the numerical value of the customer flow data and the weighting coefficients of each type of customer flow data, the range to which the crowd popularity value belongs is determined, and then the corresponding crowd popularity level may be determined. For example, if the determined crowd popularity value is 25, it may be determined that the corresponding crowd popularity level is the third level crowd popularity.

Illustratively, the crowd popularity value may be calculated based on the number of customer flows and the number of customer flows according to the following calculation formula.

는 군중 인기 값을 나타내고,

는 고객 흐름 연인원수의 값을 나타내고,

는 고객 흐름 인원수의 값을 나타내고,

는 고객 흐름 연인원수의 가중치를 나타내고,

는 고객 흐름 인원수의 가중치를 나타낸다.here,

represents the crowd popularity value,

represents the value of the number of customer flows,

represents the value of customer flow headcount,

represents the weight of the number of customer flows,

denotes the weight of the customer flow headcount.

Illustratively, the crowd popularity value may be calculated based on a combination of the customer flow headcount and the duration of stay, or the crowd popularity value may be calculated based on the combination of the customer flow headcount and the attention duration. not limited to

When the display state of the exhibition area in the display interface is controlled using the crowd state data, a layer of the display area corresponding to the display area in the display interface may be controlled to display a color level matching the crowd popularity level.

Each different crowd popularity level is matched to a different color level. For example, a color level matching a first level of crowd popularity may be green, a color matching a second level of crowd popularity may be yellow, and a color matching a third level of crowd popularity may be red. have. Alternatively, each color level may represent a different gray value. Here, the color level matching each crowd popularity level may be preset.

After determining the crowd popularity level, a color level matching the crowd popularity level may be determined, and then the layer of the display area corresponding to the display area in the display interface may be controlled to display the color of the color level. For example, a schematic diagram of display areas of different color levels in the display interface may be as shown in FIG. 2 . In FIG. 2 , different gray areas represent different exhibition areas, a gray value represents crowd popularity in the exhibition area, and the higher the gray value, the higher the crowd popularity value.

When a display area corresponding to each exhibition area is included in the display interface, crowd popularity information of each exhibition area can be displayed based on the layer color of each display area, so the display method is relatively intuitive.

In another embodiment of the present invention, the display area in the display interface can be controlled to display a display state related to customer flow data, for example, each type of customer flow data in the display interface according to the type of customer flow data. You can control the display state related to customer flow data.

In one possible embodiment, the determining of the crowd state data of the exhibition area within the first time period using the customer flow data includes determining the crowd distribution data having different attributes based on the attribute information of each user. and the crowd distribution data includes distribution data of at least one attribute of age, gender, attractiveness value, and emotion. Illustratively, the headcount statistics of the crowd distribution data may be calculated based on the customer flow headcount, and in this case, the client flow data includes at least attribute information and the client flow headcount of each user who appeared in the exhibition area.

Accordingly, controlling the display state of the exhibition area in the display interface using the crowd state data can control the display of crowd distribution diagrams of each type of attribute in the display interface using crowd distribution data having different attributes. have.

Here, the crowd distribution of each type of attribute displayed on the display interface may be a pie graph, a ring chart, a histogram, or the like. For example, a crowd distribution diagram of each type of attribute in the display interface may be as shown in FIG. 3 . The attributes displayed in FIG. 3 are gender, age, attractiveness value, and emotion, respectively, and examples of figures next to each type of attribute distribution indicate the ratio of the quantity occupied in each section. For example, "XXXX" next to the customer flow age distribution represents each age segment, and "XXX" represents the percentage of the headcount in the corresponding age range as a percentage of the total headcount (or may represent the headcount value in the corresponding age range). , "XXX" next to the customer flow gender distribution represents the percentage of users of each gender (or may represent the headcount value of the corresponding gender), and "XXXX" next to the distribution of attractiveness values represents each attractiveness value. represents the section, "XXX" represents the ratio of the number of people in the corresponding attractive value section to the total number of people (or may represent the headcount value of the corresponding attractive value section), and "XXXX" next to the emotion distribution represents each type represents the emotion of , and "XXX" represents the ratio (or may represent the value of the number of emotion persons of the corresponding type) to the total number of persons of emotion of the corresponding type.

In another embodiment of the present invention, the crowd state data includes a customer flow headcount and a customer flow headcount at a plurality of time nodes in the first time zone. When determining the number of customer flows and the number of customer flows in the plurality of time nodes in the first time zone, it may be obtained from the number of customer flows and the number of customer flows in the customer flow data.

In order to reduce the amount of calculation, when determining the number of customer flows and the number of customer flows in the exhibition area within the first time period, the number of customer flows may be determined once every predetermined time period, for example, customer flows once per minute. The number of people and the number of customer flows can be determined.

Specifically, when determining the number of customer flows, the total number of appearances of each user in the exhibition area within the first time period may be determined. Here, for each user, the start time when the user enters the exhibition area each time is recorded, and when the interval between the recorded two start times exceeds the first time, it is determined that the user appeared once, One hour does not exceed the first time zone.

Exemplarily, the start times when the user enters the exhibition area A are recorded as 10:00, 10:08, and 10:11, respectively, and when the first time is 10 minutes, between 10:00 and 10:08 Since the interval time is less than the first time, it is determined that the user appears 0 times, and since the interval time between 10:00 and 10:11 is greater than the first time, it is determined that the user appears once.

In the above method, if the interval between the recorded two start times exceeds the first time, it is determined that the user appears once, so through this method, repeated statistics of the number of user appearances in a relatively short time are avoided. and can improve the statistical precision of the number of customer flows.

When determining the number of customer flows, after acquiring the number of face images included in the video screen collected within the first predetermined time period, duplicate exclusion processing may be performed on the acquired face images, that is, user faces are mutually exclusive. The appearances in the video screen at different times are unified, the number of customer flows corresponding to the user is determined to be 1, and the number of face images after duplication exclusion processing is determined as the number of customer flows.

Considering that both the customer flow headcount and the customer flow headcount change in real time, in order to reduce the computational amount, the customer flow headcount and the customer flow headcount in a plurality of time nodes in the first time zone may be obtained, where any adjacent The time interval between the two time nodes may be the same.

In one possible embodiment, it is possible to display the number of customer flows and the number of customer flows at a plurality of time nodes within a first time period in one coordinate system. Illustratively, set the time node as the abscissa, set the number of customer flows corresponding to each time node as the ordinate, draw a line graph, take the same time node as the abscissa in one coordinate system, and each time By drawing a line graph with the number of customer flows corresponding to the node as the ordinate, the trend of change in the number of customer flows and the number of customer flows in the display area within the first time period can be displayed using the line graph. can In another example of the present invention, a change trend in the number of customer flows and a change in the number of customer flows in the display area within the first time period may be displayed through a method of drawing a histogram, a pie graph, or the like.

Illustratively, the trend of change in the number of customer flows displayed on the display interface and the tendency of change in the number of customer flows and the comparison result between the two are shown in FIG. 4 .

In the above embodiment, by using the display interface to display the trend of change in the number of customer flows and the trend of change in the number of customer flows in the exhibition area, the change status of the number of customers and the number of customer flows in the exhibition area is displayed more intuitively can do.

In another embodiment of the present invention, the crowd state data includes a total attention time of a crowd group of each kind of attribute, and illustratively, the customer flow data includes a user's attention duration and attribute information of each user. . Here, the attribute information of the user includes at least two attributes of age, gender, attractiveness value, and emotion.

In ascertaining the duration of the user's attention, the method shown in Fig. 5 includes the following several steps.

In S501, for each user, the face direction data of the user in the collected video screen is recognized.

Here, the face direction data includes a pitch angle and a yaw angle of the face. By determining whether the pitch angle is within the first angular range and whether the yaw angle is within the second angular range, it may be determined whether the user is paying attention to the exhibition object in the exhibition area.

In S502, when it is detected that the face direction data indicates that the user is paying attention to the exhibition object in the exhibition area, the time when the user first paid attention to the exhibition object is recorded.

When the pitch angle in the face direction data is within the first angle range and the yaw angle in the face direction data is within the second angle range, the face direction data indicates that the user is paying attention to the display object in the display area. can be confirmed

In S503, based on the recorded times when a plurality of the users paid attention to the exhibition object for the first time, a duration of attention during which the users paid attention to the exhibition object within the first time period is determined, wherein the attention duration is does not exceed the first time period.

Specifically, when the user's attention to the exhibition object is determined within the first time period, the user first determines the starting time at which the user starts paying attention to the exhibition object within the first time period, and then the user selects the exhibition object 2 When it is determined that the interval between the start times noticed once exceeds the predetermined time, the interval is set as the attention duration that the user paid attention to the exhibition object once.

Illustratively, the first time zone is 10:00 to 10:30, and the start times at which the user paid attention to the exhibition object are 10:02, 10:05, 10:13, 10:15 and 10:25, respectively, If 1 hour is 10 minutes, the duration of the first attention is 10:13-10:02=11 minutes, and the duration of the second attention is 10:25-10:13=12 minutes.

In the above method, based on the recorded time of first paying attention to the exhibition object of the user, it is possible to determine the attention duration in which the user paid attention to the exhibition object within the first time period, so that the user can determine the duration of the user's attention to the exhibition object for the first time. It is possible to avoid the frequent statistics of the attention duration due to the interval with the attention time being too short, and the method of calculating the attention duration time is relatively reasonable.

Specifically, when determining the total attention time of the crowd group of each type of attribute in the first time zone using the attention duration of each user and the attribute information of each user, attribute information of each user appearing in the exhibition area After classifying each user into crowd groups of different kinds of attributes based on The total attention time of the crowd group of each type of attribute may be determined based on the attention duration corresponding to each user.

In one possible embodiment, the attention status effect diagram of the crowd group of each kind of attribute may be displayed on the display interface by using the total attention time of the crowd group of each kind of attribute within the first time period.

Here, the attention effect map may be displayed as a specific figure. The total attention time of the crowd group of each kind of attribute is different, and the display effect of the corresponding specific image is also different. Illustratively, the attention status effect diagram is as shown in FIG. 6 , and FIG. 6 shows different types of attribute user distribution situations in which the total attention time is within the same interval range, and the attribute is gender and age, for example, first After determining the male-gender crowd subgroup and the female-gender crowd subgroup in the crowd group, respectively, the age subgroup of each age section is determined among the male-gender crowd subgroups, and the female gender crowd Among the subgroups, the age subgroups of each age section are respectively determined, and the diameter of a corresponding circle is determined based on each of the determined age subgroups, wherein "XXXX" in the circle represents each age section. Here, each divided subgroup of each gender section and each age section may be regarded as a crowd group of each type of attribute.

In addition, according to the calculation method of the attention duration, the attention duration of each user in the crowd group of different types of attributes is statistic, and then, based on the total attention time of each user in the crowd group of different types of attributes, It is possible to display the attention status effect diagram of the crowd group of each type of attribute. For example, as shown in FIG. 6 , the larger the crowd group with a longer total attention time, the larger the size of the corresponding attention state effect diagram.

In the above embodiment, based on the crowd distribution data of different attributes, it is possible to display the crowd distribution map of each type of attribute, and also realize analysis of crowds of different attributes at different total attention time.

In another embodiment of the present invention, the crowd status data includes the total residence time of the crowd group of each kind of attribute, in this case, the customer flow data includes the customer flow data of each user who appeared in the exhibition area within the first time period. It includes the duration of stay and attribute information of each user.

Here, when determining the duration of stay of each user who appears in the exhibition area, the total duration of stay when each user appears each time within the first time zone may be determined, wherein, for each user, the user When it is determined that the interval between the start times when the user enters the exhibition area twice exceeds the first time, the interval may be used as the duration for which the user stayed when the user appeared once.

Illustratively, the first time zone is 10:00 to 10:30, and the start times when the user enters the exhibition area are 10:02, 10:05, 10:13, 10:15, and 10:25, respectively, If the first time period is 10 minutes, the first dwell duration is 10:13-10:02=11 minutes and the second dwell duration is 10:25-10:13=12 minutes.

In one possible embodiment, after determining the total residence time of the crowd group of each type of attribute by using the residence duration of each user who appeared in the exhibition area within the first time period and the attribute information of each user, each By using the total residence time of the crowd group of the kind attribute, the effect map of the staying state of the crowd group of each kind attribute may be displayed on the display interface.

Specifically, first, based on the attribute information of each user appearing in the exhibition area, each user is classified into a crowd group of different attributes, and then each user's stay in the exhibition area of the crowd group of each type of attribute is continued acquire the time, and determine the total staying time of the crowd group of each kind of attribute, based on the staying duration corresponding to each user in the crowd group of each kind of attribute;

The residence status effect diagram is displayed as a specific image. The total residence time of the crowd group of each kind of attribute is different, and the display effect of the corresponding specific image is also different. Exemplarily, the staying status effect diagram may be shown in FIG. 3 , and in this case, the distribution status of user stay duration times of different attributes within the same section range is displayed in FIG. 3 .

In the above embodiment, based on the crowd distribution data of different attributes, it is possible to display the crowd distribution map of each type of attribute, and also realize analysis of crowds of different attributes with different total residence times.

In an example of the present invention, after acquiring a trigger operation on the display area of the display interface, the display state of the exhibition area in the display interface may be controlled using the crowd state data.

Here, acquiring the trigger operation for the display area of the display interface includes controlling the display interface to display a posture prompt frame including the posture description content, and then detecting the user posture in the video screen collected by the photographing device, When it is detected that the user posture satisfies the posture recorded in the posture description content, it may be confirmed that a trigger operation for the display area of the display interface has been acquired.

Illustratively, as shown in Fig. 7, the posture prompt frame includes posture description information such as “Face the viewfinder frame, raise your hand, please” in the posture prompt frame, and the line part is displayed. The human body posture is used for prompting the user to take the action posture at the corresponding position.When it is detected that the line portion of the user finder border takes the hand-raising action, the trigger operation for the display area of the display interface is obtained. can be confirmed as

In another example of the present invention, acquiring a trigger operation for the display area of the display interface may also be that a position corresponding to the display area in the screen is detected to be clicked, and the click method is a single click or a double click. , long press, double press, and the like. Alternatively, acquiring the trigger operation on the display area of the display interface may further include receiving a voice command input by the user, interpreting the voice command, and determining that the trigger operation on the display area of the display interface has been acquired. have.

The following specific examples are provided by combining each of the above embodiments.

When the customer flow data is reflected in each user's stay duration, each user's attention duration, each user's attribute information, the number of customer flows, and the number of customers in the exhibition area, the initially displayed display interface is shown in FIG. 8A . may be as shown. In the exhibition area on the left of FIG. 8A , the gray values of areas of different popularity levels are different from each other. In the exhibition area on the right, the comparison result between the trend of change in the number of customer flow and the number of customer flow in the exhibition area within the first time period, the crowd distribution chart of each type of attribute, and the statistical result of customer flow data are displayed. . Here, the customer flow data statistical result includes today's customer flow number of people, today's customer flow number of people, today's total stay time, past customer flow number, past customer flow number of people, staying time per person, etc. When it is detected that one display area is triggered, the display interface may be displayed as shown in FIG. 8B . In FIG. 8B , the attention state effect diagram displayed in the displayed area on the right side of FIG. 8A is converted into an interface screen corresponding to the triggered exhibition area. On the interface screen, statistical information of customer flow data in the triggered exhibition area (including customer flow headcount, customer flow headcount, stay duration, etc.), customer flow headcount / total headcount change trend, attention crowd analysis (e.g. For example, attention status effect map), etc. can be displayed. Here, in the statistical results of customer flow data displayed in FIGS. 8A and 8B , each numerical value is changed in real time, and statistical results of customer flow data within a certain period such as today, 7 days, 30 days, etc. can be statistic. The "today" indicates statistics starting from today and up to the present time.

Those skilled in the art will know that the description order of each step in the above method of specific embodiments means a strict execution order and does not constitute a limitation on the implementation process, the specific execution order of each step is determined by its function and possible internal logic You have to understand that it should be

An embodiment of the present invention further provides an apparatus for recognizing a state of an exhibition area based on the same concept. 9 is a schematic diagram showing the configuration of an apparatus for recognizing a display area state provided by an embodiment of the present invention. The apparatus includes: an acquisition module 901; a first determining module 902; a second determining module 903; and a control module 904 .

The acquisition module 901 acquires the video screen collected in the exhibition area.

The first determining module 902 is configured to determine customer flow data of the exhibition area within a first time period based on the collected video screen, wherein the customer flow data is a continuation of stay of each user who appears in the exhibition area. at least two of time, attention duration of each user, attribute information of each user, number of customer flows, and number of customer flows.

The second determining module 903 determines the crowd state data of the exhibition area within the first time period by using the customer flow data.

The control module 904 controls the display state corresponding to the exhibition area in the display interface by using the crowd state data.

In one possible embodiment, the crowd state data includes a crowd popularity level, and when the control module 904 uses the crowd state data to control a display state corresponding to the exhibition area in the display interface, the A layer of the display region corresponding to the display region in the display interface is controlled to display a color level matching the popularity level of the crowd.

In one possible embodiment, the second determining module 903 is further configured to, when determining the crowd popularity level, a weighting coefficient respectively corresponding to at least two customer flow data of the customer flow data and the at least two customer flows A crowd popularity value is determined based on the numerical value of the data, and a crowd popularity level corresponding to a range to which the crowd popularity value belongs is determined.

In one possible embodiment, when the customer flow data includes the number of customer flows and the number of customer flows, the second determining module 903 is configured to use the customer flow data in the first time zone. When determining the crowd state data of the exhibition area, use the number of customer flow and the number of customer flow to obtain the number of customer flow and the number of customer flow at a plurality of time nodes in the first time zone;

When controlling the display state corresponding to the exhibition area in the display interface by using the crowd state data, the control module 904 is configured to determine the number of customer flows and the number of customer flows in a plurality of time nodes in the first time zone. based on the display interface, a change trend in the number of customer flows and a change in the number of customer flows in the exhibition area within the first time period are displayed.

In one possible embodiment, the second determining module 903 uses the customer flow data to determine the crowd state data of the exhibition area within the first time zone, based on the attribute information of each user. Determine crowd distribution data having different attributes, wherein the crowd distribution data includes distribution data of at least one attribute of age, gender, attractiveness value, and emotion;

When the control module 904 controls the display state corresponding to the exhibition area in the display interface by using the crowd state data, each type of attribute in the display interface using the crowd distribution data having the different attributes Control to display each crowd distribution with

In one possible embodiment, when the customer flow data includes each user's attention duration and each user's attribute information, the second determining module 903 uses the customer flow data to be configured within the first time period. When determining the crowd state data of the exhibition area in , each user is classified into a crowd group of different types of attributes based on attribute information of each user appearing in the exhibition area, and crowd groups of each type of attribute acquire the attention duration of each user in the exhibition area of each kind, and determine the total attention time of the crowd group of each kind attribute based on the attention duration corresponding to each user in the crowd group of each kind attribute and

The control module 904 is configured to control the display state corresponding to the exhibition area in the display interface by using the crowd state data, in the display interface based on the total attention time of the crowd group of each type of attribute, the Control to display the attention status effect diagram of the crowd group of each type of attribute.

In one possible embodiment, the user in the crowd group of each kind of attribute satisfies at least two attributes of age, gender, attractiveness value and emotion.

In one possible embodiment, the attention state effect diagram is displayed in a specific figure, the total attention time of the crowd group of each type of attribute is different from each other, and the display effect of the corresponding specific image is also different from each other.

In one possible embodiment, the first determining module 902 is further configured to recognize, for each user, the face direction data of the user in the collected video screen, when determining the attention duration,

When it is detected that the face direction data indicates that the user is paying attention to the exhibition object in the exhibition area, the time when the user first paid attention to the exhibition object is recorded, and a plurality of the users who recorded the display Based on a time at which the object was first paid attention, a attention duration for which the user paid attention to the exhibition object within the first time period is determined, wherein the attention duration does not exceed the first time period.

In one possible embodiment, the face orientation data includes a pitch angle and a yaw angle of a face, and the first determining module 902 is configured to determine that the pitch angle is within a first angle range, and the yaw angle is a second If it is within the angle range, it is determined that the face direction data indicates that the user paid attention to the exhibition object in the exhibition area.

In one possible embodiment, the first determining module 902 is further configured to, when determining the number of customer flows of the exhibition area within the first time period, each user in the exhibition area within the first time period. Determine the total number of appearances, for each user, record the start time when the user enters the exhibition area each time, and if the interval between the recorded two start times exceeds the first time, the user selects 1 It is determined that it has appeared once, and the first time period does not exceed the first time period.

In one possible embodiment, the first determining module 902 is further configured to determine the duration of user stay in the exhibition area within the first time period, the stay when each user appears each time within the first time period. A total duration is determined, but, for each user, when it is determined that the interval between the start times when the user enters the exhibition area twice exceeds the first time, the user sets the interval to 1 It is set to the duration it stayed when it appeared.

In one possible embodiment, the acquisition module 901 is further configured to use the crowd state data to control a display state corresponding to the exhibition area in the display interface, before triggering a trigger operation on the exhibition area in the display interface. to acquire

In one possible embodiment, the acquisition module 901 controls the display interface to display a posture prompt frame including a posture description when acquiring a trigger operation for the display area in the display interface, The user's posture in the collected video screen is detected, and when the user's posture satisfies the posture recorded in the posture description, it is confirmed that the trigger operation for the display area in the display interface is acquired.

In some embodiments, a function or template included in an apparatus provided by an embodiment of the present invention may be used to execute the method described in the embodiment of the method, the specific implementation of which is the method of the above method. Reference may be made to the description of the examples.

An embodiment of the present invention further provides an electronic device based on the same technical concept. 10 is a schematic diagram showing the configuration of an electronic device provided by an embodiment of the present invention. The electronic device includes a processor 1001; memory 1002; and a bus 1003 . Here, the memory 1002 includes an internal memory 10021 and an external memory 10022 . The internal memory 10021 here is also called a memory, and temporarily stores calculation data in the processor 1001 and data to be exchanged with the external memory 10022 such as a hard disk. The processor 1001 performs data exchange with the external memory 10022 using the internal memory 10021 . When the electronic device 1000 is operated, communication between the processor 1001 and the memory 1002 through the bus 1003 causes the processor 1001 to execute the following commands, which are:

acquiring video footage collected in the exhibition area;

Based on the collected video screen, determining customer flow data of the exhibition area within a first time period, wherein the customer flow data includes the duration of stay of each user who appeared in the exhibition area, the continuation of attention of each user. including at least two of time, attribute information of each user, the number of customer flows, and the number of customer flows;

determining crowd state data of the exhibition area within the first time period using the customer flow data; and

and controlling a display state corresponding to the exhibition area in a display interface by using the crowd state data.

On the other hand, an embodiment of the present invention further provides a computer-readable recording medium in which a computer program is recorded. When the computer program is operated by a processor, the display area state recognition method described in the embodiment of the method is performed. step is executed.

The computer program product of the display area state recognition method provided by the embodiment of the present invention includes a computer-readable recording medium having a program code recorded therein, and the instructions included in the program code are described in the embodiment of the method above. It is used to implement the steps of the display area state recognition method.

A person skilled in the art can clearly understand that the specific working process of the above system and apparatus may refer to the process in the embodiment of the method, and for the sake of convenience and conciseness of the description, it is not repeatedly described herein. It should be understood that the systems, apparatuses and methods disclosed in the embodiments provided by the present invention may be implemented in other ways. The embodiment of the above device is only schematic. For example, the division of the unit is merely division of a logical function, and there may be other division methods in actual implementation, and for example, a plurality of units or components may be combined, integrated into a separate system, or some There are cases where the feature of is omitted or not implemented. The couplings, direct couplings, or communication connections between each other, also indicated or discussed, may employ some communication interface, indirect coupling or communication connection of a device or unit, and may be in an electrical, mechanical or other form.

The units described as separated parts may be physically separated or not physically separated, and the parts indicated as units may or may not be physical units. It may be arranged in one place, or it may be distributed in a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the solution objectives of the embodiments of the present invention.

In addition, each functional unit of each embodiment of the present invention may be integrated into one processing unit, each unit may exist alone physically, or two or more units may be integrated into one unit.

When the above function is implemented in the form of a software function unit and sold or used as an independent product, it may be recorded in a nonvolatile computer readable recording medium executable by one processor. Based on this understanding, the embodiment of the present invention may be essentially implemented in the form of a software product, or a part contributing to the prior art or a part of the embodiment may be implemented in the form of a software product. The computer software product is recorded on a recording medium, and includes several instructions for causing a computer device such as a computer, server, or network device to execute all or part of the steps of the method described in the following various embodiments. The above-described recording medium includes a variety of programs capable of recording program codes, such as a U disk, a removable hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk. includes media.

The above are only some embodiments of the present invention, and the protection scope of the present invention is not limited thereto. Changes or substitutions that can be readily devised by those skilled in the art based on the present invention are included within the scope of the present invention. Accordingly, the scope of the present invention is determined by the claims.

Claims (17)

In the display area state recognition method,
acquiring video footage collected in the exhibition area;
determining, based on the collected video screen, customer flow data of the exhibition area within a first time period, wherein the customer flow data includes a residence duration of each user who appeared in the exhibition area, a continuation of each user's attention including at least two of time, attribute information of each user, the number of customer flows, and the number of customer flows;
determining crowd state data of the exhibition area within the first time period using the customer flow data; and
and controlling a display state corresponding to the exhibition area in a display interface using the crowd state data.
A method for recognizing the state of an exhibition area, characterized in that. According to claim 1,
wherein the crowd state data includes a crowd popularity level;
Controlling the display state corresponding to the exhibition area in the display interface using the crowd state data includes:
controlling a layer of a display region corresponding to the display region in the display interface to display a color level matching the crowd popularity level
A method for recognizing the state of an exhibition area, characterized in that. 3. The method of claim 2,
To determine the crowd popularity level,
determining a crowd popularity value based on a weight coefficient corresponding to each of the at least two customer flow data in the customer flow data and numerical values of the at least two customer flow data; and
Comprising determining a crowd popularity level corresponding to a range to which the crowd popularity value belongs
A method for recognizing the state of an exhibition area, characterized in that. According to claim 1,
When the customer flow data includes the customer flow number of people and the customer flow number of people, determining the crowd state data of the exhibition area within the first time period using the customer flow data includes:
using the customer flow headcount and the customer flow headcount to obtain the customer flow headcount and the customer flow headcount at a plurality of time nodes in the first time zone;
Controlling the display state corresponding to the exhibition area in the display interface using the crowd state data includes:
Based on the number of customer flow people and the number of customer flow people in the plurality of time nodes in the first time zone, the display interface shows a change tendency and a change in the number of customer flows in the exhibition area within the first time zone. which includes exhibiting trends
A method for recognizing the state of an exhibition area, characterized in that. According to claim 1,
Using the customer flow data to determine the crowd state data of the exhibition area within the first time period,
determining crowd distribution data with each attribute based on the attribute information of each user;
The crowd distribution data includes distribution data of at least one attribute of age, gender, attractiveness value, and emotion,
Controlling the display state corresponding to the exhibition area in the display interface using the crowd state data includes:
using the crowd distribution data having each attribute to control to display a crowd distribution map having each type of attribute in the display interface, respectively
A method for recognizing the state of an exhibition area, characterized in that. According to claim 1,
When the customer flow data includes each user's attention duration and each user's attribute information, using the customer flow data to determine the crowd state data of the exhibition area within the first time zone includes:
classifying each user into crowd groups of different types of attributes based on attribute information of each user appearing in the exhibition area;
acquiring attention duration within the display area of each user in the crowd group of each kind of attribute; and
determining a total attention time of the crowd group of each kind attribute based on the attention duration corresponding to each user in the crowd group of each kind attribute;
Controlling the display state corresponding to the exhibition area in the display interface using the crowd state data includes:
Controlling to display the attention status effect diagram of the crowd group of each kind of attribute in the display interface based on the total attention time of the crowd group of each kind of attribute
A method for recognizing the state of an exhibition area, characterized in that. 7. The method of claim 6,
The user in the crowd group of each type of attribute satisfies at least two attributes of age, gender, attractiveness value, and emotion.
A method for recognizing the state of an exhibition area, characterized in that. 7. The method of claim 6,
The attention state effect diagram is displayed in a specific figure, the total attention time of the crowd group of each type of attribute is different, and the display effect of the corresponding specific image is also different
A method for recognizing the state of an exhibition area, characterized in that. 9. The method of any one of claims 1 or 6 to 8,
Determining the attention duration is,
for each user, recognizing the user's face orientation data in the collected video screen;
when it is detected that the face orientation data indicates that the user is paying attention to the exhibition object in the exhibition area, recording a time when the user first paid attention to the exhibition object; and
determining, based on the recorded times when a plurality of the users first paid attention to the exhibition object, a attention duration in which the user paid attention to the exhibition object within the first time period, wherein the attention duration is the first do not exceed time zone - including
A method for recognizing the state of an exhibition area, characterized in that. 10. The method of claim 9,
The face direction data includes a pitch angle and a yaw angle of the face,
When the pitch angle is within the first angle range and the yaw angle is within the second angle range, it is determined that the face direction data indicates that the user paid attention to the exhibition object in the exhibition area.
A method for recognizing the state of an exhibition area, characterized in that. 5. The method of claim 1 or 4,
Determining the number of customers flowing in the exhibition area within the first time period includes:
determining the total number of appearances of each user in the exhibition area within the first time period;
for each user, record the start time when the user enters the exhibition area each time, and if the interval between the recorded two start times exceeds the first time, it is determined that the user appears once; In addition, the first time does not exceed the first time period
A method for recognizing the state of an exhibition area, characterized in that. According to claim 1,
Determining the duration of the user's stay in the exhibition area within the first time zone comprises:
determining the total duration of time that each user stayed when each appeared within the first time zone;
For each user, if it is determined that the interval between the start times when the user entered the exhibition area twice has exceeded the first time, the duration of the interval the user stayed when the user appeared once set to
A method for recognizing the state of an exhibition area, characterized in that. 13. The method according to any one of claims 1 to 12,
Before controlling the display state corresponding to the exhibition area in the display interface using the crowd state data,
further comprising acquiring a trigger operation for the exhibition area in the display interface
A method for recognizing the state of an exhibition area, characterized in that. 14. The method of claim 13,
Acquiring a trigger operation for the display area in the display interface comprises:
controlling the display interface to display a posture prompt frame including posture description content;
detecting a user's posture in the collected video screen; and
When the user posture satisfies the posture recorded in the posture description content, confirming that a trigger operation for the display area in the display interface has been acquired
A method for recognizing the state of an exhibition area, characterized in that. In the display area state recognition device,
an acquisition module for acquiring the video screen collected in the exhibition area;
A first determining module for determining customer flow data of the exhibition area within a first time zone, based on the collected video screen, wherein the customer flow data includes a residence duration of each user who appears in the exhibition area, each including at least two of the user's attention duration, attribute information of each user, the number of customer flows, and the number of customer flows;
a second determining module for determining crowd state data of the exhibition area within the first time period by using the customer flow data; and
and a control module for controlling a display state corresponding to the exhibition area in a display interface using the crowd state data.
Display area state recognition device, characterized in that. In an electronic device,
a processor and memory;
Machine-readable instructions executable by the processor are recorded in the memory;
when the machine readable instructions are executed by the processor, cause the processor to execute the method for recognizing a display area state according to any one of claims 1 to 14.
Electronic device, characterized in that. In a computer-readable recording medium on which a computer program is recorded,
15. When the computer program is executed by the processor, causing the processor to execute the display area state recognition method according to any one of claims 1 to 14.
A computer-readable recording medium, characterized in that.

Images