JP2022519149A - Exhibition area State recognition methods, devices, electronic devices, and recording media - Google Patents

Abstract

The present invention provides an exhibition area state recognition method, an apparatus, an electronic device, and a recording medium. As an example, the method is to acquire the video screen collected in the exhibition area and to determine the customer flow data of the exhibition area within the first time zone based on the collected video screen. The customer flow data includes at least two of the stay duration of each user appearing in the exhibition area, the attention duration of each user, the attribute information of each user, the number of customer flows, and the total number of customers. That, the customer flow data is used to determine the crowd state data of the exhibition area within the first time zone, and the crowd state data is used to enter the exhibition area in the display interface. Includes controlling the corresponding display state. [Selection diagram] Fig. 1

Description

<Mutual citation of related applications>
The present invention claims priority for a Chinese patent application with a filing date of December 31, 2019 and an application number of 2019114163365, the entire contents of which are incorporated herein by reference.
The present invention relates to the field of computer technology, and specifically to an exhibition area state recognition method, an apparatus, an electronic device, and a recording medium.

In exhibition halls such as supermarkets, shopping stores, and exhibition halls, various items are usually exhibited in each exhibition area for users to purchase or visit. The degree of interest of the goods in each exhibition area of the user is one of the issues that many stores or exhibitions are paying attention to. However, since the characteristics of the flow of people in a store or exhibition are mostly reflected in the trajectory of user activity in a crowded crowd or in an irregular state, analyze the degree of interest of the user in each exhibition area. Becomes more difficult.

In view of this, the present invention provides at least a solution for exhibit area state recognition.

According to the first aspect, the present invention provides a method for recognizing an exhibition area state, which method is described as.
To get the video screens collected in the exhibition area,
Based on the collected video screen, the customer flow data of the exhibition area within the first time zone is determined, and the customer flow data is the stay duration of each user appearing in the exhibition area. Include at least two of each user's attention duration, each user's attribute information, customer flow number of people, customer flow total number of people,
Using the customer flow data, determining the crowd status data of the exhibition area within the first time zone, and
The crowd state data is used to control the display state corresponding to the exhibition area in the display interface.

In the above method, the customer flow data is determined based on the collected video screen, the customer flow data is used to determine the crowd state data of the exhibition area in the first time zone, and the crowd state data is used. It is possible to control the display state of the exhibition area in the display interface. The display state of the exhibition area can be used to prompt the crowd state data in the exhibition area, so that dynamic monitoring of the crowd state of the exhibition area can be realized, and the attention crowd of the exhibition area can be realized. Can be analyzed.

In one possible embodiment, the crowd state data includes a crowd popularity level, and the crowd state data can be used to control a display state corresponding to the exhibition area in the display interface within the display interface. The layer of the display area corresponding to the exhibition area of is controlled to represent a color level matched to the crowd popularity level.

By exhibiting different crowd popularity levels using different color levels, it is possible to display the crowd popularity information of different exhibition areas more intuitively, while enriching the method of showing the display state of the exhibition area. can.

In one possible embodiment, determining the crowd popularity level is based on the weighting factors corresponding to at least two customer flow data in the customer flow data, respectively, and the numerical values of the at least two customer flow data. It includes determining the crowd popularity value and determining the crowd popularity level corresponding to the range to which the crowd popularity value belongs.

Since the crowd popularity value determined by such a method combines the characteristics of at least two customer flow data, the determined crowd popularity value can better reflect the customer flow characteristics of the exhibition area and is determined. The crowd popularity level is more accurate.

In one possible embodiment, when the customer flow data includes the customer flow number and the customer flow total number, the customer flow data is used to crowd state data of the exhibition area within the first time zone. The determination of the crowd includes the acquisition of the customer flow number and the customer flow total number at the plurality of time nodes in the first time zone by using the customer flow number and the customer flow total number. Using the state data to control the corresponding display state in the display interface is to the display interface based on the number of customer flows and the total number of customer flows in the plurality of time nodes in the first time zone. It includes displaying the change tendency of the number of customer flows and the change tendency of the number of customer flows in the exhibition area within the first time zone.

In the above embodiment, the change status of the number of customer flows and the number of customer flows in the exhibition area can be displayed more intuitively.

In one possible embodiment, using the customer flow data to determine crowd status data for the exhibition area within the first time zone has different attributes based on the attribute information of each user. Containing the determination of crowd distribution data, said crowd distribution data includes distribution data for at least one attribute of age, gender, attractiveness, and emotion.
Using the crowd state data to control the display state corresponding to the exhibition area in the display interface can use the crowd distribution data with the different attributes to control each type of attribute in the display interface. Includes controlling the display of each crowd distribution map with.

In the above embodiment, in order to display a more detailed crowd distribution situation of each exhibition area, a crowd distribution map of each type of attribute can be displayed based on the crowd distribution data of different attributes.

In one possible embodiment, if the customer flow data includes the attention duration of each user and the attribute information of each user, the customer flow data is used to crowd the exhibition area within the first time zone. Determining the state data is to classify each user into a crowd group of different types of attributes based on the attribute information of each user appearing in the exhibition area, and to determine each user in the crowd group of each type of attribute. Determine the total attention time of a crowd group of each type of attribute based on the duration of attention in the exhibition area and the duration of attention corresponding to each user in the crowd group of each type of attribute. Including that
Using the crowd state data to control the display state corresponding to the exhibition area in the display interface is such that each of the above in the display interface is based on the total attention time of the crowd group of each type of attribute. Includes controlling the attention status effect diagram of a crowd group of types of attributes to appear.

Based on the above embodiments, the attention status effect diagram of the crowd group of each type of attribute displayed in the display interface may be used to show the degree of attention to the exhibition area of the crowd of different types of attributes. can.

In one possible embodiment, a user in a crowd group of said attributes of each type satisfies at least two attributes of age, gender, attractiveness, and emotion.

In one possible embodiment, the attention state effect diagram is displayed in a particular graphic, the total attention time of the crowd group of each type of attribute is different from each other, and the display effect of the corresponding specific image is also different from each other.

In one possible embodiment, determining the attention duration means recognizing the user's face orientation data in the collected video screen for each user, and the face orientation data is the user. When it is detected that the user is paying attention to the exhibition object in the exhibition area, the time when the user first pays attention to the exhibition object is recorded, and the recorded users are the display. Based on the time when the object is first focused, the attention duration for which the user pays attention to the exhibited object is determined within the first time zone, and the attention duration exceeds the first time zone. Including that not.

In one possible embodiment, the face orientation data includes a face pitch angle and a yaw angle, the pitch angle is within the first angle range, and the yaw angle is within the second angle range. , The face orientation data is determined as indicating that the user has focused on the exhibited object in the exhibition area.

In the above embodiment, since it is possible to determine the attention duration during which the user has focused on the exhibited object within the first time zone based only on the recorded time when the user's exhibited object is first focused, the user exhibits. Frequent statistics on the duration of attention caused by too short an interval between the time the object was first focused can be avoided, and the method of calculating the duration of attention is more rational.

In one possible embodiment, determining the total number of customer flows in the exhibition area within the first time zone determines the total number of appearances of each user in the exhibition area within the first time zone. Here, for each user, the start time when the user enters the exhibition area each time is recorded, and when the interval between the two recorded start times exceeds the first hour. , It is confirmed that the user has appeared once, and the first time does not exceed the first time zone.

In the above embodiment, it is possible to avoid repeated statistics on the number of user appearances in a relatively short time, and it is possible to improve the accuracy of statistics on the total number of customer flows.

In one possible embodiment, determining the duration of stay of a user in the exhibition area within the first time zone is the total duration of stay of each user each time within the first time zone. Including the determination, here, if the interval between the start times when the user enters the exhibition area twice is determined as exceeding the first time for each user, the interval. Is the duration of stay when the user appears once.

In the above embodiment, it is possible to avoid the calculation of the stay duration multiple times when the user appears multiple times in a relatively short time, the calculation amount of the stay duration is reduced, and the stay duration is calculated. To be more rational.

In one possible embodiment, the crowd state data is further used to acquire a trigger operation on the exhibition area in the display interface before controlling the display state corresponding to the exhibition area in the display interface. include.

In one possible embodiment, acquiring a trigger operation for the exhibition area in the display interface controls the display interface to display the appearance prompt frame, and in the gesture prompt frame, the appearance description content is displayed. Includes; When the user figure in the collected video screen is detected and the user figure satisfies the figure recorded in the figure description content, it is confirmed that the trigger operation for the exhibition area in the display interface has been acquired. Including that.

In the above embodiment, it can be realized by a method of collecting the user's appearance in the video screen when acquiring the trigger operation for the display area in the display interface, and the trigger method of the trigger operation is enriched with the user and the device. Increased dialogue between.

According to the second aspect, an embodiment of the present invention provides an exhibition area state recognition device, which is a device.
An acquisition module for acquiring video screens collected in the exhibition area,
Based on the collected video screen, the first confirmation module for confirming the customer flow data of the exhibition area within the first time zone is, and the customer flow data is each appearing in the exhibition area. A first confirmed module that includes at least two of the user's stay duration, each user's attention duration, each user's attribute information, customer flow number of people, and total customer flow number of people.
Using the customer flow data, a second confirmation module for determining the crowd status data of the exhibition area within the first time zone, and
A control module for controlling a display state corresponding to the exhibition area in the display interface by using the crowd state data is provided.

According to a third aspect, the present invention provides an electronic device, the electronic device comprising a processor and a memory, the memory containing machine-readable instructions that can be executed by the processor, said machine. When the readable instruction is executed by the processor, the processor is made to execute the exhibition area state recognition method according to the first aspect or any embodiment described above.

According to a fourth aspect, the present invention provides a computer-readable recording medium in which a computer program is stored, and when the computer program is operated by a processor, the processor causes the processor according to the first aspect or any embodiment described above. Make sure to implement the exhibition area state recognition method described in.

Hereinafter, in order to make the above-mentioned objects, features, and advantages of the present invention more clear and understandable, preferred embodiments will be given and described in detail as follows with reference to the drawings.

Hereinafter, in order to more clearly explain the surgical solution of the embodiment of the present invention, the drawings required for the embodiment will be briefly introduced. The drawings of this specification are incorporated herein and form part of this specification. These drawings show embodiments of the invention and are used in conjunction with the specification to illustrate the technical solutions of the invention. It should be understood that the drawings below only show specific embodiments of the invention and therefore should not be considered as limiting scope, without any creative work for those of skill in the art. , Other related drawings can be obtained based on these drawings.
It is a schematic diagram which shows the flow of the exhibition area state recognition method provided by the Example of this invention. It is a schematic diagram which shows the exhibition of the exhibition area of a different color level in the display interface provided by the Embodiment of this invention. It is a crowd distribution map of each kind of attribute in the display interface provided by the embodiment of the present invention. It is a schematic diagram which shows the comparison result between the change tendency of the number of customers flow and the change tendency of the number of times of customer flow provided by the embodiment of the present invention. It is a schematic diagram which shows the flow of the attention duration determination method of the user provided by the Example of this invention. FIG. 3 is a status effect diagram provided by an embodiment of the present invention. It is a schematic diagram which shows the appearance prompt frame provided by the Example of this invention. It is a schematic diagram which shows the other display interface provided by the Embodiment of this invention. It is a schematic diagram which shows the other display interface provided by the Embodiment of this invention. It is a schematic diagram which shows the structure of the exhibition area state recognition apparatus provided by the Example of this invention. It is a schematic diagram which shows the structure of the electronic device provided by the Example of this invention.

Hereinafter, in order to further clarify the objectives, technical solutions, and advantages of the embodiments of the present invention, the embodiments of the present invention will be described clearly and completely with reference to the drawings. Of course, the examples described are only a few examples of the invention, not all examples. In general, the components of the embodiments of the invention described and shown in the drawings herein can be arranged and designed in different configurations. Accordingly, the following detailed description of the embodiments of the invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents the selected embodiments of the invention. Based on the examples of the present invention, all other examples obtained without the creative work of those skilled in the art shall be included in the scope of protection of the present invention.

When monitoring each exhibition area with related technology, the method of monitoring the crowd density in the current exhibition area is generally adopted, but depending on the statistical crowd density alone, the customer flow crowd in the exhibition area is concrete. Analysis cannot be realized. For example, it is not possible to analyze the duration of attention of the customer flow crowd to the exhibition area, the attribute information of the user in the customer flow crowd, etc., and further, it is possible to realize the analysis of the degree of interest of the user in the exhibition area. Can not.

In view of this, the exhibition area state recognition method provided by the present invention determines the customer flow data of the exhibition area within the first time zone based on the collected video screen, and the determined customer flow data is used. , Customer flow because it can include at least two of each user's stay duration, each user's attention duration, each user's attribute information, customer flow number of people, and customer flow total number of people appearing in the exhibition area. The crowd state data can be determined by referring to at least two features in the data, and when controlling the display state of the exhibition area in the display interface based on the crowd state data, the crowd characteristics in the exhibition area are better. It can be exhibited accurately and intuitively.

The exhibition area state recognition method provided by the embodiments of the present invention may be applied to a server or a terminal device that supports a display function. The server can be a local server or a cloud server, and the terminal device can be a smartphone, a tablet computer, a personal digital assistant (PDA), a smart TV, or the like, and the present invention is not limited thereto.

When the embodiments of the present invention are applied to a server, the data in the present invention may be obtained from a terminal device and / or a photographing device, and control of the display state corresponding to the exhibition area in the display interface. Can realize control of the display state corresponding to the exhibition area in the display interface of the terminal device by sending a control command to the terminal device.

In order to facilitate the understanding of the embodiments of the present invention, first, the exhibition area state recognition method will be introduced in detail.

FIG. 1 is a schematic diagram showing a flow of an exhibition area state recognition method provided by an embodiment of the present invention, and the method includes steps S101 to S104.

In S101, the video screen collected in the exhibition area is acquired.

In S102, the customer flow data of the exhibition area in the first time zone is determined based on the collected video screen, and the customer flow data is the stay duration of each user appearing in the exhibition area and the attention of each user. Includes at least two of duration, attribute information for each user, number of customer flows, and total number of customer flows.

In S103, the customer flow data is used to determine the crowd status data of the exhibition area within the first time zone.

In S104, the crowd state data is used to control the display state corresponding to the exhibition area in the display interface.

The method provided by this embodiment determines the customer flow data based on the collected video screens and uses the customer flow data to determine the crowd state data of the exhibition area within the first time zone and the crowd state. The data can be used to control the display state corresponding to the exhibition area in the display interface. The display state of the exhibition area can be used to prompt the crowd state data in the exhibition area, so that dynamic monitoring of the crowd state of the exhibition area can be realized, and the attention crowd of the exhibition area can be realized. Can be analyzed.

If the video screen contains multiple exhibition areas, it is possible to control the corresponding different display states that represent different exhibition areas in the display interface, based on the crowd state data of the different exhibition areas. Comparative analysis can be performed on crowd conditions in different exhibition areas.

In one embodiment of the invention, crowd status data includes crowd popularity levels. The crowd popularity level can be determined by the following method, that is, based on the weighting factors corresponding to at least two customer flow data in the customer flow data, and the numerical values of the at least two customer flow data. Determine the popularity value, and then determine the crowd popularity level corresponding to the location range to which the crowd popularity value belongs.

Specifically, the crowd popularity value range corresponding to different crowd popularity levels can be preset, for example, the crowd popularity value range corresponding to the first level crowd popularity is 0 to 10, corresponding to this. Therefore, the value of the crowd popularity value within the first level crowd popularity can be 0, 5, 10 and the crowd popularity value range corresponding to the second level crowd popularity is 11 to 20, which corresponds to this. Thus, the crowd popularity value within the second level crowd popularity can be 11, 15, 20, and the crowd popularity value range corresponding to the third level crowd popularity is 21-30. Correspondingly, the value of the crowd popularity value within the third level crowd popularity can be 21, 25, 30. After determining the crowd popularity value based on the customer flow data numbers and the weighting factor of each type of customer flow data, determine the range to which the crowd popularity value belongs, and then determine the corresponding crowd popularity level. be able to. For example, if the determined crowd popularity value is 25, the corresponding crowd popularity level can be determined to be the third level of crowd popularity.

Illustratively, the crowd popularity value can be calculated based on the total number of customer flows and the number of customer flows according to the following formula.
P = ax ₁ + bx ₂ equation (1)

Here, P represents the crowd popularity value, x ₁ represents the value of the total number of customer flows, x ₂ represents the value of the total number of customer flows, a represents the weight of the total number of customer flows, and b. Represents the weight of the number of customer flows.

Illustratively, the crowd popularity value can be calculated based on the combination of customer flow numbers and stay duration, or the crowd popularity value can be calculated based on the combination of customer flow number and attention duration. , The present invention is not limited to this.

When using crowd state data to control the display state of an exhibition area in a display interface, the layer of the display area corresponding to the exhibition area in the display interface represents the color level that matches the crowd popularity level. As such, it can be controlled.

Different crowd popularity levels are matched to different color levels. For example, the color level that matches the first level crowd popularity can be green, the color that matches the second level crowd popularity can be yellow, and the color that matches the third level crowd popularity. Can be red. Alternatively, different color levels can represent different gray values. Here, the color levels that are matched to different crowd popularity levels can be preset.

After determining the crowd popularity level, determine the color level that matches the crowd popularity level, so that the layer of the display area corresponding to the exhibition area in the display interface reveals the color of that color level. , Can be controlled. Illustratively, an exhibition schematic of display areas of different color levels within a display interface may be as shown in FIG. The different gray areas in FIG. 2 indicate different exhibition areas, the gray value indicates the crowd popularity of the exhibition area, and the higher the gray value, the higher the crowd popularity value.

If the display interface contains a display area corresponding to each exhibition area, it is possible to display crowd popularity information of different exhibition areas based on the color of the layer of each display area, and the exhibition method is more intuitive. Is.

In another embodiment of the invention, the display area can be controlled to display the display state associated with the customer flow data in the display interface, eg, different types in the display interface according to different types of customer flow data. It can be controlled to show the display state related to the customer flow data of.

In one possible embodiment, the customer flow data is used to determine the crowd status data of the exhibition area within the first time zone, which is based on the attribute information of each user and the crowd distribution data having different attributes. The crowd distribution data includes distribution data of at least one attribute within age, gender, attractiveness, and emotion, including determination. Illustratively, the number statistics in the crowd distribution data can be calculated based on the number of customer flows, in which case the customer flow data will contain the attribute information and the number of customer flows for each user appearing in the exhibition area. At least include.

Correspondingly, controlling the display state of the exhibition area in the display interface by using the crowd state data can utilize the crowd distribution data having different attributes to control the display state of each type of attribute in the display interface. The crowd distribution map can be controlled to be displayed individually.

Here, the crowd distribution map of each type of attribute displayed on the display interface can be a pie chart, a ring chart, a histogram, or the like. Illustratively, a crowd distribution map of each type of attribute in the display interface can be as shown in FIG. The attributes exhibited in FIG. 3 are gender, age, attractiveness, and emotion, respectively, and the example figure next to each type of attribute distribution shows the proportion of quantities within different sections. For example, "XXXX" next to the customer flow age distribution indicates a different age segment, and "XXX" indicates the ratio of the total number of people in the corresponding age segment (or the number of people in the corresponding age segment). Can be), and "XXX" next to the customer flow gender distribution indicates the percentage of users of different genders (or can indicate the person value of the corresponding gender), and "XXX" next to the attractiveness value distribution. "XXXX" indicates a different attractive value section, and "XXX" indicates the ratio of the number of people in the corresponding attractive value section to the total number of people (or can indicate the person value of the corresponding attractive value section), and emotions. "XXXX" next to the distribution indicates a different type of emotion, and "XXX" can indicate the ratio of the number of people of the corresponding type of emotion to the total number of people (or the number of people of the corresponding type of emotion). ) Is shown.

In another embodiment of the present invention, the crowd state data includes the number of customer flows and the total number of customer flows at the plurality of time nodes in the first time zone. When determining the number of customer flows and the total number of customer flows in a plurality of time nodes in the first time zone, it can be obtained from the number of customer flows and the total number of customer flows in the customer flow data.

In order to reduce the amount of calculation, when determining the number of customer flows and the total number of customer flows in the exhibition area within the first time zone, it can be determined once every predetermined time zone, for example, every minute. In addition, the number of customer flows and the total number of customer flows can be determined once.

Specifically, when determining the total number of customer flows, it is possible to determine the total number of appearances of each user in the exhibition area within the first time zone. Here, for each user, the start time when the user enters the exhibition area each time is recorded, and when the interval between the recorded two start times exceeds the first hour, the user appears once. The first time does not exceed the first time zone.

Illustratively, the start times when the user enters the exhibition area A are recorded at 10:00, 10:08, and 10:11, respectively, and if the first time is 10 minutes, then 10:00 to 10 :. The time of the interval between 08 is less than the first hour, it is determined that the user has appeared 0 times, the time of the interval between 10:00 and 10:11 is larger than the first hour, and the user is 1 It is confirmed that it has appeared once.

In the above method, when the interval between the two recorded start times exceeds the first time, it is determined that the user has appeared once, and by such a method, the number of times the user appears in a relatively short time. It is possible to avoid the repeated statistics of, and improve the accuracy of the statistics of the total number of customer flows.

When determining the number of customer flows, it is possible to acquire the number of facial images included in the video screen collected within the first predetermined time zone, and then perform deduplication processing on the acquired facial images. That is, it is possible to unify the appearance on the video screen at different times of the user's face, determine the number of customer flow people corresponding to the user as 1, and set the number of face images after deduplication processing as the number of customer flow people. Determine.

Taking into account that both the number of customer flows and the number of customer flows change in real time, in order to reduce the amount of calculation, the number of customer flows and the total number of customer flows at multiple time nodes in the first time zone are acquired. It can, where the time intervals between any two adjacent time nodes can be the same.

In one possible embodiment, the number of customer flows and the total number of customer flows at a plurality of time nodes in the first time zone can be exhibited in the same coordinate system. Illustratively, a time node is set in abscissa, the number of customer flows corresponding to each time node is set in ordinate, and a polyline diagram is drawn. Then, in the same coordinate system, the same time node is set to horizontal coordinates, the total number of customer flows corresponding to each time node is set to vertical coordinates, a polyline diagram is drawn, and the polyline diagram is used within the first time zone. It is possible to display the changing tendency of the number of customer flows and the changing tendency of the number of customer flows in the exhibition area. In another example of the present invention, a change tendency of the number of customer flows in the exhibition area and a change tendency of the number of customer flows in the exhibition area can be exhibited by a method of drawing a histogram, a pie chart, or the like.

Illustratively, the change tendency of the number of customer flows and the change tendency of the number of customer flows displayed on the display interface, and the comparison result between the two are as shown in FIG.

In the above embodiment, by displaying the change tendency of the number of customer flows in the exhibition area and the change tendency of the number of customer flows using the display interface, the change status of the number of customer flows and the number of customer flows in the exhibition area is more intuitive. Can be exhibited as a target.

In another embodiment of the invention, the crowd state data includes the total attention time of the crowd group of each type of attribute, and exemplary the customer flow data includes the user attention duration and each user attribute information. .. Here, the attribute information of the user includes at least two attributes among age, gender, attractiveness value, and emotion.

In determining the user's attention duration, the method shown in FIG. 5 includes several steps:

In S501, each user recognizes the face orientation data of the user in the collected video screen.

Here, the face orientation data includes the pitch angle and the yaw angle of the face. By determining whether the pitch angle is within the first angle range and whether the yaw angle is within the second angle range, it is determined whether the user is paying attention to the exhibited object in the exhibition area. be able to.

When it is detected in S502 that the face orientation data indicates that the user is paying attention to the exhibited object in the exhibition area, the time when the user first pays attention to the exhibited object is recorded.

When the pitch angle in the face orientation data is within the first angle range and the yaw angle in the face orientation data is within the second angle range, the face orientation data allows the user to pay attention to the exhibited object in the exhibition area. If you show that you are doing it, you can confirm it.

In S503, based on the time when the plurality of recorded users first pay attention to the exhibited object, the attention duration during which the user pays attention to the exhibited object is determined within the first time zone, and here, the above-mentioned The duration of attention does not exceed the first time zone.

Specifically, when determining the attention duration during which the user has focused on the exhibited object within the first time zone, first, the start time at which the user has started paying attention to the exhibited object within the first time zone is determined, and then When it is determined that the interval between the start times when the user pays attention to the exhibited object twice exceeds a predetermined time, the interval is used as the attention duration when the user pays attention to the exhibited object once. ..

Illustratively, the first time zone is from 10:00 to 10:30, and the start times when the user pays attention to the exhibited object are 10:02, 10:05, 10:13, 10:15, and 10 :, respectively. If it is 25 and the first time is 10 minutes, the first attention duration is 10: 13-10: 02 = 11 minutes and the second attention duration is 10: 25-10: 13. = 12 minutes.

In the above method, the user can determine the duration of attention that the user has focused on the exhibit object within the first time zone based only on the recorded time when the user's exhibit object is first focused. You can avoid frequent statistics of the duration of attention caused by the interval between the time of first focus being too short, reducing the amount of computation when calculating the duration of attention, and also the calculated attention. Made the duration more rational.

Specifically, it appears in the exhibition area when determining the total attention time of a crowd group of each type of attribute within the first time zone using the attention duration of each user and the attribute information of each user. Based on the attribute information of each user, each user is classified into a crowd group of different types of attributes, and then the attention duration in the exhibition area of each user in the crowd group of each type of attribute is obtained, and each is obtained. The total attention time of a crowd group of each type of attribute can be determined based on the duration of attention corresponding to each user in the crowd group of each type of attribute.

In one possible embodiment, the total attention time of the crowd group of each type of attribute within the first time zone can be used to display the attention status effect diagram of the crowd group of each type of attribute on the display interface. can.

Here, the attention effect diagram can be displayed as a specific figure. The total attention times of the crowd groups of each type of attribute are different from each other, and the display effect of the corresponding specific image is also different from each other. Illustratively, the status effect diagram is as shown in FIG. 6, which shows the distribution of users of different types of attributes whose total attention time is within the same interval range, where the attributes are gender and age. For example, first determine the crowd subgroup of male gender in the crowd group, and then determine the crowd subgroup of female gender, and then each of the crowd subgroups of male gender. The age subgroups for each age segment are finalized, the age subgroups for each age segment in the crowd subgroup with female gender are finalized, and the corresponding circles are based on the fixed age subgroups. The diameter is fixed and "XXXX" in the circle indicates different age intervals. Here, subgroups of different genders and different age segments that are divided can be considered as crowd groups of different types of attributes.

In addition, according to the attention duration calculation method described above, the attention duration of each user in a crowd group of different types of attributes is statistic, followed by the total attention time of each user in a crowd group of different types of attributes. Based on this, it is possible to display an attention status effect diagram for a crowd group of each type of attribute. For example, as shown in FIG. 6, the longer the total attention time, the larger the size of the corresponding attention state effect diagram.

In the above embodiment, it is possible to display a crowd distribution map of each type of attribute based on the crowd distribution data of different attributes, and further realize an analysis for a crowd of different attributes with different total attention times.

In another embodiment of the invention, the crowd state data includes the total time spent in a crowd group of each type of attribute, in which case the customer flow data is for each user appearing in the exhibition area within the first time zone. Includes the duration of stay and attribute information for each user.

Here, when determining the stay duration of each user appearing in the exhibition area, it is possible to determine the total duration of stay when each user appears each time within the first time zone, and here, For each user, if it is determined that the interval between the start times when the user enters the exhibition area twice exceeds the first hour, the stay when the user appears once in the interval. Can be used as a duration.

Illustratively, the first time zone is from 10:00 to 10:30, and the start times when the user enters the exhibition area are 10:02, 10:05, 10:13, 10:15, and 10 respectively. : 25, if the first time is 10 minutes, the first stay duration is 10: 13-10: 02 = 11 minutes and the second stay duration is 10: 25-10: 13 = 12 minutes.

In one possible embodiment, the duration of stay of each user appearing in the exhibition area within the first time zone and the attribute information of each user are used to determine the total stay time of the crowd group of each type of attribute. Subsequently, the total time spent in the crowd group of each type of attribute can be used to display the status effect diagram of the crowd group of each type of attribute in the display interface.

Specifically, first, each user is classified into a crowd group with different attributes based on the attribute information of each user appearing in the exhibition area, and then the exhibition area of each user in the crowd group of each type of attribute. The duration of stay in each type of attribute can be obtained and the total length of stay of the crowd group of each type of attribute can be determined based on the duration of stay corresponding to each user in the crowd group of each type of attribute.

The status effect diagram is displayed as a specific image. The total length of stay of a crowd group of each type of attribute is different, and the display effect of the corresponding specific image is also different from each other. Illustratively, the status effect diagram may be as shown in FIG. 3, but at this time, what is exhibited in FIG. 3 is the stay duration of users with different attributes within the same section range. It is a distribution situation.

In the above embodiment, it is possible to display a crowd distribution map of each type of attribute based on the crowd distribution data of different attributes, and further realize an analysis for different attribute crowds with different total lengths of stay.

In one example of the present invention, it is possible to further control the display state of the exhibition area in the display interface by using the crowd state data after acquiring the trigger operation for the exhibition area in the display interface.

Here, acquiring the trigger operation for the exhibition area in the display interface controls the display interface to display the appearance prompt frame containing the appearance description content, and subsequently, the user in the video screen collected by the shooting device. When the appearance is detected and it is detected that the user appearance satisfies the appearance recorded in the appearance description content, it may be confirmed that the trigger operation for the exhibition area in the display interface has been acquired.

Illustratively, as shown in FIG. 7, the figure prompt frame contains the figure description information of "Please enter the front of the finder frame and raise your hand to check". The human figure displayed by the line part is used to prompt the user to take the motion figure at the corresponding position. When it is detected that the line portion of the user finder frame has raised the hand, it can be confirmed that the trigger operation for the exhibition area in the display interface has been acquired.

In another example of the present invention, acquiring a trigger operation for an exhibition area in a display interface may further mean that it has been detected that a position corresponding to the exhibition area on the screen has been clicked. Includes, but is not limited to, single-click, double-click, long-press, double-press, etc. Alternatively, acquiring the trigger operation for the exhibition area in the display interface further receives the voice instruction input by the user, analyzes the voice instruction, and determines that the trigger operation for the exhibition area in the display interface has been acquired. It can be.

The following specific examples are provided by combining the above embodiments.

Display that is initially displayed when customer flow data is reflected in the stay duration of each user in the exhibition area, the attention duration of each user, the attribute information of each user, the number of customer flows, and the total number of customer flows. The interface can be as shown in FIG. 8A. In the exhibition area on the left side of FIG. 8A, the gray values of the areas of different popularity levels are different. In the exhibition area on the right side, the comparison result between the change tendency of the number of customer flows and the change tendency of the number of customer flows in the exhibition area during the first time zone, the crowd distribution map of each type of attribute, and the customer Flow data statistical results are on display. Here, the customer flow data statistical results include the number of customers flow today, the total number of customers flow today, the total time stayed today, the number of past customer flows, the total number of past customer flows, the time spent per person, etc., and the exhibition area on the left side. When it is detected that any one of the exhibition areas has been triggered, the display interface can be exhibited as shown in FIG. 8B. In FIG. 8B, the attention status effect diagram displayed in the area displayed on the right side of FIG. 8A is replaced with the interface screen corresponding to the triggered exhibition area. On the interface screen, the customer flow data statistics of the triggered exhibition area (including customer flow number of people, total customer flow number of people, stay duration, etc.), customer flow number of people / total number of people change trend, attention level crowd analysis (for example, attention) State effect diagram) etc. can be exhibited. Here, in the customer flow data statistical results displayed in FIGS. 8A and 8B, each numerical value is changed in real time, and the customer flow data within a certain period such as today, 7 days, 30 days, etc. Statistical results can be statistic. The above-mentioned "today" means that statistics have been made from today to the present time.

Those skilled in the art will appreciate that in the above method of a specific embodiment, the description order of each step does not constitute a restriction on the implementation process, which means a strict execution order, but the specific execution order of each step. , Understand that it should be determined by its function and possible internal logic.

Embodiments of the present invention further provide an exhibition area state recognition device based on the same concept. FIG. 9 is a schematic diagram showing the configuration of the exhibition area state recognition device provided by the embodiment of the present invention. The device may include an acquisition module 901, a first deterministic module 902, a second deterministic module 903, and a control module 904.

The acquisition module 901 acquires the video screen collected in the exhibition area.

The first confirmation module 902 determines the customer flow data of the exhibition area in the first time zone based on the collected video screen, and here, the customer flow data is each appearing in the exhibition area. It includes at least two of the user's stay duration, each user's attention duration, each user's attribute information, the number of customer flows, and the total number of customer flows.

The second confirmation module 903 uses the customer flow data to determine the crowd status data of the exhibition area within the first time zone.

The control module 904 uses the crowd state data to control the display state corresponding to the exhibition area in the display interface.

In one possible embodiment, when the crowd state data includes a crowd popularity level and the control module 904 uses the crowd state data to control a display state corresponding to the exhibition area in the display interface. In addition, the layer of the display area corresponding to the exhibition area in the display interface is controlled to represent a color level matched to the crowd popularity level.

In one possible embodiment, the second determination module 903 further has a weighting factor corresponding to at least two customer flow data in the customer flow data when determining the crowd popularity level, and said. Based on the numerical values of at least two customer flow data, the crowd popularity value is determined, and the crowd popularity level corresponding to the range to which the crowd popularity value belongs is determined.

In one possible embodiment, when the customer flow data includes the customer flow number of people and the customer flow total number of people, the second confirmation module 903 utilizes the customer flow data within the first time zone. When determining the crowd status data of the exhibition area, the customer flow number and the customer flow total number are used to determine the customer flow number and the customer flow total number at the plurality of time nodes in the first time zone. Acquired,
When the control module 904 uses the crowd state data to control the display state corresponding to the exhibition area in the display interface, the customer flow number and customers at the plurality of time nodes in the first time zone. Based on the total number of flows, the display interface displays the change tendency of the number of customer flows and the change tendency of the number of customer flows in the exhibition area within the first time zone.

In one possible embodiment, the second determination module 903 utilizes the customer flow data to determine the attribute information of each user when determining the crowd state data of the exhibition area within the first time zone. Based on, the crowd distribution data having different attributes is determined, and the crowd distribution data includes the distribution data of at least one attribute among age, gender, attractiveness value, and emotion.
When the control module 904 uses the crowd state data to control the display state corresponding to the exhibition area in the display interface, the control module 904 utilizes the crowd distribution data having the different attributes in the display interface. It is controlled so that the crowd distribution map with each kind of attribute is displayed.

In one possible embodiment, when the customer flow data includes the attention duration of each user and the attribute information of each user, the second confirmation module 903 utilizes the customer flow data to perform the first time zone. When determining the crowd status data of the exhibition area within, each user is classified into a crowd group of different types of attributes based on the attribute information of each user appearing in the exhibition area, and of each type of attribute. The total attention of the crowd group of each type of attribute is obtained based on the attention duration of each user in the crowd group in the exhibition area and the attention duration corresponding to each user in the crowd group of each type of attribute. Set the time,
The control module 904 uses the crowd state data to control the display state corresponding to the exhibition area in the display interface, based on the total attention time of the crowd group of each type of attribute. It is controlled so that the attention state effect diagram of the crowd group of each type of attribute appears in the interface.

In one possible embodiment, a user in a crowd group of said attributes of each type satisfies at least two attributes of age, gender, attractiveness, and emotion.

In one possible embodiment, the attention state effect diagram is displayed in a particular graphic, the total attention time of the crowd group of each type of attribute is different from each other, and the display effect of the corresponding specific image is also different from each other.

In one possible embodiment, the first determination module 902 further recognizes, for each user, the face orientation data of the user in the collected video screen when determining the attention duration.
When it is detected that the face orientation data indicates that the user is paying attention to the exhibited object in the exhibition area, a plurality of times when the user first pays attention to the exhibited object are recorded and recorded. Based on the time when the user first focused on the exhibited object, the attention duration during which the user focused on the exhibited object is determined within the first time zone, and the attention duration is the first time. Do not exceed one hour.

In one possible embodiment, the face orientation data includes a face pitch angle and a yaw angle, and the first determination module 902 has the pitch angle within the first angle range and the yaw angle. When it is within the second angle range, the face orientation data is determined as indicating that the user has focused on the exhibited object in the exhibition area.

In one possible embodiment, the first confirmation module 902 further determines the total number of customer flows in the exhibition area within the first time zone, and the first confirmation module 902 is within the exhibition area within the first time zone. The total number of appearances of each user is determined, and here, for each user, the start time when the user enters the exhibition area is recorded, and the interval between the two recorded start times. Is more than the first hour, it is determined that the user has appeared once, and the first hour does not exceed the first time zone.

In one possible embodiment, the first determination module 902 further determines each user within the first time zone each time when determining the duration of stay of the user in the exhibition area within the first time zone. The total duration of stay at the time of appearance is determined, and here, it is assumed that the interval between the start times when the user enters the exhibition area twice exceeds the first hour for each user. If confirmed, the interval shall be the duration of stay when the user appears once.

In one possible embodiment, the acquisition module 901 also uses the crowd state data to reference the display area in the display interface before controlling the display state corresponding to the display area in the display interface. Get the trigger operation.

In one possible embodiment, the acquisition module 901 controls the display interface to display a form prompt frame containing the form description content when acquiring a trigger operation for the exhibition area in the display interface. When the user figure in the collected video screen is detected and the user figure satisfies the figure recorded in the figure description content, it is confirmed that the trigger operation for the exhibition area in the display interface has been acquired.

In some embodiments, the features or templates included in the apparatus provided by the embodiments of the present invention can be used to perform the methods described in the embodiments of the above methods, the specifics thereof. The implementation can refer to the description of the embodiment of the above method.

The embodiments of the present invention further provide electronic devices based on the same technical concept. FIG. 10 is a schematic diagram showing the configuration of the electronic device provided by the embodiment of the present invention. The electronic device includes a processor 1001, a memory 1002, and a bus 1003. Here, the memory 1002 includes an internal memory 10021 and an external memory 10022. The internal memory 10021 here is also called a memory, and temporarily stores the calculation data in the processor 1001 and the data to be exchanged with the external memory 10022 such as a hard disk. The processor 1001 uses the internal memory 10021 to perform data exchange with the external memory 10022. When the electronic device 1000 is operated, the processor 1001 and the memory 1002 communicate with each other via the bus 1003 so that the processor 1001 executes the following instruction.
To get the video screens collected in the exhibition area,
Based on the collected video screen, the customer flow data of the exhibition area within the first time zone is determined, and the customer flow data is the stay duration of each user appearing in the exhibition area. Include at least two of each user's attention duration, each user's attribute information, customer flow number of people, customer flow total number of people,
Using the customer flow data, determining the crowd status data of the exhibition area within the first time zone, and
The crowd state data is used to control the display state corresponding to the exhibition area in the display interface.

In addition, an embodiment of the present invention further provides a computer-readable recording medium in which a computer program is stored, and when the computer program is operated by a processor, the exhibition area state recognition method according to the embodiment of the above method. Steps are performed.

The computer program product of the exhibition area state recognition method provided by the embodiment of the present invention includes a computer-readable recording medium in which the program code is stored, and the instruction contained in the program code is described in the embodiment of the above method. Used to perform the steps of the exhibition area state recognition method described.

It will be clarified by those skilled in the art that the specific working processes of the above systems and devices can be referred to in the embodiments of the method and will not be repeated herein for convenience and brevity of description. Can be understood. It should be understood that the systems, devices, and methods disclosed in the examples provided by the present invention can be realized by other methods. The embodiment of the above device is merely schematic. For example, the division of the unit is merely a division of a logical function, and in an actual implementation, there may be other division methods, and for example, multiple units or components may be combined or integrated into another system. It may be, or some features may be abbreviated or not implemented. In addition, the display or discussion of mutual coupling or direct coupling or communication connection can be via some communication interface, indirect coupling or communication connection of devices or units, and can be in electrical, mechanical or other form. ..

The unit described above as a separate part may or may not be physically separated, and the part displayed as a unit may or may not be a physical unit. .. It can be located in one location or distributed across multiple network units. Some or all of the units may be selected according to the actual need to achieve the objectives of the solutions in the embodiments of this disclosure.

Also, each functional unit in each embodiment of the invention can be integrated into one processing unit, or each unit can physically exist independently, or two or more units can be combined into one unit. Can be integrated.

When the function is implemented in the form of a software functional unit and sold or used as a stand-alone product, it can be stored in a non-volatile computer-readable recording medium that can be run by one processor. Based on this understanding, the embodiments of the present invention are essentially embodied in the form of software products, or parts that contribute to the prior art or parts of embodiments are embodied in the form of software products. Can be transformed into. The computer software product is stored on a recording medium to allow a computer device, such as a personal computer, server, or network device, to perform all or part of the steps of the methods described in the various embodiments below. Includes some instructions. The recording media described above include U-disks, mobile hard disks, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disks, optical disks, and various other recording media that can store program code. Includes medium.

The above are only a few embodiments of the invention, and the scope of protection of the invention is not limited thereto. Changes or substitutions that can be easily imagined by those skilled in the art based on this disclosure shall be included within the scope of this disclosure. Therefore, the scope of the present disclosure is determined by the scope of claims.

Claims (17)

It is an exhibition area state recognition method.
To get the video screens collected in the exhibition area,
Based on the collected video screen, the customer flow data of the exhibition area within the first time zone is determined, and the customer flow data is the stay duration of each user appearing in the exhibition area. Include at least two of each user's attention duration, each user's attribute information, customer flow number of people, customer flow total number of people,
Using the customer flow data, determining the crowd status data of the exhibition area within the first time zone, and
An exhibition area state recognition method comprising: using the crowd state data to control a display state corresponding to the exhibition area in a display interface. The crowd status data includes crowd popularity levels and
Using the crowd state data to control the display state corresponding to the exhibition area in the display interface
The exhibition according to claim 1, wherein the layer of the display area corresponding to the exhibition area in the display interface is controlled so as to represent a color level matched to the crowd popularity level. Area state recognition method. Determining the crowd popularity level is
Determining the crowd popularity value based on the weighting factors corresponding to at least two customer flow data in the customer flow data and the numerical values of the at least two customer flow data.
The exhibition area state recognition method according to claim 2, further comprising determining a crowd popularity level corresponding to a range to which the crowd popularity value belongs. When the customer flow data includes the number of customers flow and the total number of customers flow, it is possible to use the customer flow data to determine the crowd status data of the exhibition area within the first time zone.
Including using the customer flow number of people and the customer flow total number of people to acquire the customer flow number of customers and the customer flow total number of people at the plurality of time nodes in the first time zone.
Using the crowd state data to control the display state corresponding to the exhibition area in the display interface
Based on the number of customer flows and the total number of customer flows in the plurality of time nodes in the first time zone, the display interface has a change tendency of the number of customer flows in the exhibition area and the customer flow in the first time zone. The exhibition area state recognition method according to claim 1, wherein the display includes a tendency of change in the number of times. Using the customer flow data to determine the crowd status data of the exhibition area within the first time zone is possible.
Includes determining crowd distribution data with each attribute based on each user's attribute information.
The crowd distribution data includes distribution data of at least one attribute among age, gender, attractiveness value, and emotion.
Using the crowd state data to control the display state corresponding to the exhibition area in the display interface
The first aspect of the present invention is characterized in that the crowd distribution data having each of the attributes is used to control the display interface to display the crowd distribution map having each type of attribute. Exhibition area state recognition method. If the customer flow data includes attention duration of each user and attribute information of each user, the customer flow data can be used to determine the crowd status data of the exhibition area within the first time zone. ,
Based on the attribute information of each user appearing in the exhibition area, each user is classified into a crowd group of each type of attribute.
To obtain the attention duration in the exhibition area of each user in a crowd group of each type of attribute,
Including determining the total attention time of a crowd group of each type of attribute based on the attention duration corresponding to each user in the crowd group of each type of attribute.
Using the crowd state data to control the display state corresponding to the exhibition area in the display interface
It is characterized by including controlling the display interface to show the attention status effect diagram of the crowd group of each type of attribute based on the total attention time of the crowd group of each type of attribute. The exhibition area state recognition method according to claim 1. The exhibition area state recognition method according to claim 6, wherein the user in the crowd group of each type of attribute satisfies at least two attributes of age, gender, attractiveness value, and emotion. The sixth aspect of claim 6 is characterized in that the attention state effect diagram is displayed in a specific figure, the total attention time of a crowd group of each type of attribute is different from each other, and the display effect of the corresponding specific image is also different from each other. Exhibition area status recognition method. Determining the duration of attention is
Recognizing the face orientation data of the user in the collected video screen for each user,
When it is detected that the face orientation data indicates that the user is paying attention to the exhibition object in the exhibition area, recording the time when the user first pays attention to the exhibition object.
The attention duration is determined by determining the attention duration during which the user has focused on the exhibition object within the first time zone based on the time when the plurality of recorded users first pay attention to the exhibition object. The exhibition area state recognition method according to any one of claims 1 or 6 to 8, wherein the time does not exceed the first time zone. The face orientation data includes a face pitch angle and a yaw angle.
When the pitch angle is within the first angle range and the yaw angle is within the second angle range, it is determined that the face orientation data indicates that the user has focused on the exhibited object in the exhibition area. The exhibition area state recognition method according to claim 9, wherein the exhibition area state recognition method is performed. Determining the total number of customer flows in the exhibition area within the first time zone is not possible.
Including determining the total number of appearances of each user in the exhibition area within the first time zone.
Here, for each user, the start time when the user enters the exhibition area each time is recorded, and when the interval between the two recorded start times exceeds the first time, the user causes the user. The exhibition area state recognition method according to claim 1 or 4, wherein the first time is determined to have appeared once and the first time does not exceed the first time zone. Determining the duration of stay of a user in the exhibition area within the first time zone is
Including determining the total duration of staying each time each user appears within the first time zone.
Here, if it is determined for each user that the interval between the start times when the user enters the exhibition area twice exceeds the first time, the user once determines the interval. The exhibition area state recognition method according to claim 1, wherein the duration of stay at the time of appearance is used. Before using the crowd state data to control the display state corresponding to the exhibition area in the display interface,
The exhibition area state recognition method according to any one of claims 1 to 12, further comprising acquiring a trigger operation for the exhibition area in the display interface. Acquiring a trigger operation for the exhibition area in the display interface is
Controlling the display interface to display the appearance prompt frame containing the appearance description content,
Detecting the user's appearance in the collected video screen and
The exhibition according to claim 13, wherein when the user figure satisfies the figure recorded in the figure description content, it is confirmed that the trigger operation for the exhibition area in the display interface has been acquired, and the present invention includes. Area state recognition method. It is an exhibition area state recognition device,
An acquisition module for acquiring video screens collected in the exhibition area,
Based on the collected video screen, the first confirmation module for confirming the customer flow data of the exhibition area within the first time zone is, and the customer flow data is each appearing in the exhibition area. A first confirmed module that includes at least two of the user's stay duration, each user's attention duration, each user's attribute information, customer flow number of people, and total customer flow number of people.
Using the customer flow data, a second confirmation module for determining the crowd status data of the exhibition area within the first time zone, and
An exhibition area state recognition device comprising: a control module for controlling a display state corresponding to the exhibition area in the display interface by using the crowd state data. It ’s an electronic device,
Equipped with processor and memory,
The memory stores machine-readable instructions that can be executed by the processor.
An electronic device, characterized in that, when the machine-readable instruction is executed by the processor, the processor executes the exhibition area state recognition method according to any one of claims 1 to 14. .. A computer-readable recording medium that stores computer programs.
A computer-readable recording medium, characterized in that, when the computer program is operated by a processor, the processor executes the exhibition area state recognition method according to any one of claims 1 to 14. ..

Images