WO2022160592A1

WO2022160592A1 - Information processing method and apparatus, and electronic device and storage medium

Info

Publication number: WO2022160592A1
Application number: PCT/CN2021/103592
Authority: WO
Inventors: 范畅
Original assignee: 北京市商汤科技开发有限公司
Priority date: 2021-01-27
Filing date: 2021-06-30
Publication date: 2022-08-04
Also published as: CN112950254A; KR20220123100A; CN112950254B; JP2023514764A; TW202230343A

Abstract

Provided are an information processing method and apparatus, and an electronic device and a storage medium. The information processing method comprises: on the basis of a video stream collected in respect of a target place, determining trajectory data of pedestrians in the video stream in the target place; on the basis of the trajectory data of the pedestrians in the target place in the video stream and location ranges of different article display areas in the target place, determining pedestrian statistical data corresponding to the different article display areas, respectively; and on the basis of the pedestrian statistical data, determining the deployment of at least some of the article display areas in the target place. In this way, the visitor flow of each article display area in the target place can be analyzed by means of the video stream collected in the target place online, such that the article display areas that hinder personnel circulation can be adjusted in a timely manner, so as to improve the deployment rationality of each article display area in the target place.

Description

An information processing method, device, electronic device and storage medium

CROSS-REFERENCE TO RELATED APPLICATIONS

This patent application requires the priority of the Chinese patent application number 202110110576.3 submitted on January 27, 2021, the applicant is Beijing Shangtang Technology Development Co., Ltd., and the application name is "an information processing method, device, electronic device and storage medium" , the entirety of which is incorporated by reference into this application.

technical field

The present application relates to the field of computer vision technology, and in particular, to an information processing method, apparatus, electronic device, and storage medium.

Background technique

For some target places, such as large shopping malls, there are multiple different item display areas, such as multiple different stores. Each store usually adopts some traditional item display methods to attract customers, such as new products or high-selling items. Products are placed at the door of the store. In the related art, the adjustment of the item display area is mainly performed by means of human intervention, which is time-consuming and labor-intensive, the processing method is cumbersome, and the accuracy of the data source is low.

SUMMARY OF THE INVENTION

The embodiments of the present application provide at least one information processing solution.

The embodiment of the present application provides an information processing method, which is executed by an electronic device, including:

determining the trajectory data of pedestrians in the target place in the video stream based on the video stream collected from the target place;

Based on the trajectory data of the pedestrians in the video stream in the target place, and the position ranges of different item display areas in the target place, determine the pedestrian statistics corresponding to the different item display areas respectively;

Based on the pedestrian statistics, a deployment is determined for at least a portion of the item display area within the target venue.

In the embodiment of the present application, the above-mentioned technical means can be used to determine the pedestrian statistical data of different item display areas in combination with the collected video streams of the target place, and further, based on the pedestrian statistical data, the pedestrian statistics for at least part of the item display areas in the target place can be determined. deploy.

In this way, the flow of people in each item display area in the target site can be analyzed through the video stream collected online at the target site, so that the item display area that hinders the circulation of people can be adjusted in time to improve the quality of each item in the target site. The rationality of the deployment of the display area.

In a possible implementation manner, the determining, based on the pedestrian statistics, the deployment for at least part of the item display area in the target place includes at least one of the following:

generating, based on the pedestrian statistical data, deployment recommendations for at least part of the item display area in the target site;

Based on the pedestrian statistics, the deployment of at least a portion of the item display area within the target site is adjusted.

Considering that for the target place, whether the target place is a public place or a non-public place, for safety reasons, video capture components such as cameras are usually deployed. That is to say, the data sources for realizing the above technical solutions usually belong to existing resources, and there is no need to collect additional video streams. In this way, the technical solutions provided by the embodiments of the present application can obtain deployment suggestions suitable for the target site by rationally utilizing the existing resources, thereby realizing the full utilization of the existing resources. In addition, the obtained adjustment scheme of the item display area can be more adapted to the actual needs of the target place.

In a possible implementation manner, the determining, based on the pedestrian statistical data, the deployment for at least part of the item display area in the target site includes:

Based on the pedestrian statistical data corresponding to the different item display areas and the trajectory data in the target place respectively, determining whether there is a crowd congestion in the first target item display area;

When it is determined that there is a crowd congestion in the first target item display area, at least one of the following is performed: generating a deployment suggestion for the first target item display area, and adjusting the deployment of the first target item display area.

In the embodiment of the present application, it is possible to analyze whether there is a crowd congestion situation through the pedestrian statistics data in different item display areas and the trajectory data in the target place. When the flow detects that the pedestrian's trajectory returns to the first item display area, it can be determined that the first item display area is crowded with people. At this time, at least one of the following can be performed: generate a deployment for the first item display area. Suggest and adjust the deployment of the first target item display area to reduce the congestion in the first target item display area.

In a possible implementation manner, when it is determined that there is a congestion of people in the first target item display area, at least one of the following is performed: generating a deployment suggestion for the first target item display area, adjusting the The deployment of the first target item display area, including:

extracting video frames associated with the first target item display area from the video stream;

determining the posture data of the pedestrian in the extracted video frame in the display area of the first target item;

Based on the gesture data, at least one of the following is performed: generating a deployment suggestion for the first target item display area, and adjusting the deployment of the first target item display area.

In the embodiment of the present application, it is proposed that the posture data of pedestrians in the video frame in the area can be determined according to the video frame associated with the display area of the first target item, and the congested part of the display area of the first target item can be detected through the gesture data. Reason, based on this, at least one of the following is performed: a reasonable deployment suggestion for the first target item display area can be provided, and a reasonable deployment of the first target item display area can be made to improve the flow of people caused by the first target item display area. congestion.

In a possible implementation manner, the first target article display area includes at least one article display shelf, and each article display shelf includes multiple layers, and based on the gesture data, at least one of the following is performed: Describe the deployment suggestion of the first target item display area, and adjust the deployment of the first target item display area, including:

Based on the gesture data, when it is determined that there are more than a set number of pedestrians in the video frame whose gestures conform to the first preset attitude, obtain a first position matching the gaze direction of the more than set number of pedestrians a first target layer of a target item display shelf;

Based on the first target layer, at least one of the following is performed: generating a deployment proposal instructing to adjust the items displayed on the first target layer to a second target layer of the first target item display shelf for display; The items displayed on the first target layer are adjusted to the second target layer, and the height of the second target layer from the ground is higher than the height of the first target layer from the ground.

In this embodiment of the present application, it can be detected through gesture data that the reason for the congestion of people in the first target item display area is that the items in the item display shelves in the first target item display area are placed unreasonably, for example, a large number of customers are concerned about Items are placed on the bottom of the shelf, causing a large number of customers to lean over to select products, thus causing traffic congestion. In this way, a rational deployment for the display area of the first target item can be proposed, such as adjusting the items on the bottom shelf that customers pay attention to. To the upper floors, it is convenient for customers to stand and choose products, which can improve the congestion situation.

In a possible implementation manner, the item display racks include multiple ones, and the information processing method further includes:

Based on the extracted trajectory data of the pedestrians in the video frame in the first target item display area, determine that the number of passing pedestrians is less than the first set number of second target item display shelves;

Based on the second target item display rack, at least one of the following is performed: generating a deployment suggestion instructing to adjust part of the items displayed on the first target layer to the second target item display rack; Some of the displayed items are adjusted to the second target item display shelf.

In the embodiment of the present application, it is proposed that the items on the first target item display shelf, which causes congestion of people in the first target item display area, can be adjusted to the second target item display shelf with fewer people, so as to effectively improve pedestrians because they must Congestion caused by picking items at a target item display shelf.

In a possible implementation manner, the first target item display rack is a movable rack, and based on the gesture data, at least one of the following is performed: generating a deployment suggestion for the first target item display area, Adjusting the deployment of the first target item display area further includes:

Based on the gesture data, when it is determined that there are more than a set number of pedestrians in the video frame whose gestures conform to the second preset attitude, obtain a first position matching the gaze direction of the more than set number of pedestrians The current position of a target item display shelf and the location area where the crowd congestion occurs;

Based on the current location and the location area where the crowd congestion occurs, at least one of the following is performed: generating a deployment suggestion for moving the first target item display shelf, and according to the current location and the location where the crowd congestion occurs The area adjusts the display shelf of the first target item.

In the embodiment of the present application, in the case where the first target item display rack is a movable rack, and more than a set number of pedestrians are detected to be standing according to the posture data, the position of the first target item display rack can be checked. The adjustment can effectively improve the blocking situation for other customers caused by more pedestrians at the display shelf of the first target item.

In a possible implementation manner, in the case where it is determined that there is a congestion of people in the target item display area, the determining, based on the pedestrian statistical data, is directed to at least part of the item display area in the target place, including: :

Based on the pedestrian statistical data corresponding to the different item display areas respectively, determining a second target item display area where the number of pedestrians is less than the second set number;

Based on the second target item display area, at least one of the following is performed: generating a deployment suggestion instructing to adjust some items displayed in the first target item display area to the second target item display area; Some items in the display area are adjusted to the second target item display area.

In the embodiment of the present application, it is proposed that some items displayed in the first target item display area with crowded traffic can be adjusted to be displayed in the second target item display area with fewer pedestrians, so as to effectively improve the congestion in the first target item display area. Happening.

In a possible implementation manner, the determining, based on the video stream collected from the target place, determines the trajectory data of the pedestrians in the video stream in the target place, including:

Perform pedestrian detection on the video frames in the video stream to obtain the position indication information of the same pedestrian in the associated video frames;

Based on the position indication information of the same pedestrian in the associated video frame, the trajectory data of the same pedestrian in the target place is determined.

In the embodiment of the present application, by using the pedestrian detection technology, the position indication information used to indicate the position of the pedestrian in the video frame can be quickly determined, and based on this, the trajectory data of the same pedestrian in the target place can be quickly determined.

In a possible implementation manner, the position indication information of the same pedestrian in the associated video frame includes relative position information of the same pedestrian and the preset feature point in the associated video frame; The position indication information of the pedestrian in the associated video frame, to determine the trajectory data of the same pedestrian in the target place, including:

extracting preset feature points in the video frames associated with the same pedestrian;

Based on the preset position information of the preset feature point in the target place and the relative position information of the same pedestrian and the preset feature point, determine the trajectory data of the same pedestrian in the target place .

In the embodiment of the present application, the preset position information of the preset feature points in the video frame in the target place is determined in advance, so that the pedestrian can be quickly determined based on the preset feature points and the relative position information of the pedestrian in the video frame. Trajectory data in the venue.

In a possible implementation manner, the video frames associated with the same pedestrian are determined in the following manner:

Perform head and shoulder detection on the video frame in the video stream, and obtain the pedestrian head and shoulder feature information contained in the video frame;

Determine the feature similarity between pedestrians included in the different video frames based on the pedestrian head and shoulders feature information included in the different video frames;

The video frames with the feature similarity higher than the preset similarity threshold are regarded as the video frames associated with the same pedestrian.

In the embodiments of the present application, it is proposed that the head and shoulders feature information of pedestrians contained in video frames can be obtained through the head and shoulders detection technology, and the information identifying the pedestrian features can be extracted more accurately and comprehensively through the head and shoulders detection technology, so that multiple frames of video can be quickly determined. The same pedestrian in the frame and the video frames associated with the same pedestrian provide convenience for the subsequent tracking of the pedestrian to determine the trajectory data of the pedestrian.

In a possible implementation, the information processing method further includes:

Based on the pedestrian statistical data corresponding to the different item display areas in at least one time period, and the preset rendering methods corresponding to the different pedestrian statistical data, a pedestrian heat map of the target place within a preset time period is generated and output.

In the embodiment of the present application, based on the pedestrian statistical data corresponding to different item display areas in at least one time period, a pedestrian flow heat map that can intuitively reflect the changes in the number of people in each item display area in the target place can be generated, so as to be the number of people in the target place. The item display area provides deployment suggestions.

Determine whether there is at least one item display area that meets the warning condition based on the pedestrian statistical data corresponding to the different item display areas;

When it is determined that there is at least one item display area that meets the warning condition, first warning prompt information is generated.

In the embodiment of the present application, it is proposed to detect whether there is a potential danger based on the pedestrian statistical data corresponding to different item display areas, and generate a warning prompt when it is determined that there is a potential danger, so as to effectively improve the safety in the target place .

In a possible implementation manner, the warning condition includes that the number of pedestrians in the item display area is greater than or equal to a first preset threshold.

In a possible implementation, the warning condition further includes at least one of the following:

The difference between the number of pedestrians in the item display area and the number of pedestrians in any other item display area is greater than the second preset threshold;

The distance between the item display area and the adjacent item display area is less than the preset distance threshold;

The item display area is located in a preset transportation hub area in the target site.

In the embodiment of the present application, by setting up different early warning conditions, the potential danger of the target place is comprehensively predicted, and the safety of the target place is improved.

When it is determined that there is a crowd congestion in the display area of the first target item, obtaining the duration of the crowd congestion in the display area of the first target item;

When the duration reaches a preset duration threshold, second warning prompt information is generated.

In the embodiment of the present application, it is proposed that the duration of the congestion event can be detected, and when it is detected that the duration of the congestion event is long, an early warning prompt can be provided to improve the safety of the target place.

For the third target item display area where there is no crowd congestion, obtain the number of pedestrians corresponding to the third target item display area in multiple time periods;

In the case where it is determined that the number of pedestrians corresponding to the third target item display area in the plurality of time periods is less than a third preset threshold, a deployment suggestion for the third target item display area is generated.

In the embodiment of the present application, in the case where there is no congestion event in the third target item display area, if it is detected that the number of pedestrians in the third target item display area is still small, a rationalization of the third target item display area can be generated. Deploy recommendations so that the number of pedestrians in the display area of the third target item can be increased.

In the case where it is determined that the number of pedestrians corresponding to the third target item display area in at least one time period is greater than the third preset threshold, based on the trajectory data corresponding to each sub-area in the third target item display area, determine Whether there is at least one sub-area in the third target item display area and the number of pedestrians corresponding to the plurality of time periods is smaller than the fourth preset threshold;

When it is determined that the number of pedestrians corresponding to at least one sub-area in the plurality of time periods is less than a fourth preset threshold, a deployment suggestion for the third target item display area is generated.

In the embodiment of the present application, it is proposed that the third target item display area has a large number of pedestrians in one period, but the number of pedestrians in at least one sub-area is always small. At this time, a rationalization of the third target item display area can also be generated. Recommendations are deployed so that the number of pedestrians in each sub-area in the third target item display area can be balanced.

The embodiment of the present application provides an information processing device, including:

a first determining module, configured to determine the trajectory data of pedestrians in the target place in the video stream based on the video stream collected from the target place;

The second determining module is configured to determine the pedestrians corresponding to the different item display areas respectively based on the trajectory data of the pedestrians in the video stream in the target place and the position ranges of the different item display areas in the target place Statistical data;

A processing module configured to determine deployment for at least a portion of the item display area in the target venue based on the pedestrian statistics.

In a possible implementation manner, the processor module is further configured to generate deployment suggestions for at least part of the item display areas in the target place based on the pedestrian statistics;

is configured to adjust the deployment of at least a portion of the item display area within the target venue based on the pedestrian statistics.

An embodiment of the present application provides an electronic device, including: a processor, a memory, and a bus, where the memory stores machine-readable instructions executable by the processor, and when the electronic device runs, the processor and the The memories communicate with each other through a bus, and when the machine-readable instructions are executed by the processor, the steps of the information processing method according to the first aspect are executed.

An embodiment of the present application provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is run by a processor, the steps of the information processing method described in the first aspect are executed.

An embodiment of the present disclosure provides a computer program, including computer-readable codes, and when the computer-readable codes are executed in an electronic device and executed by a processor in the electronic device, the above-mentioned living body detection is realized method.

In order to make the above-mentioned objects, features and advantages of the embodiments of the present application more obvious and easy to understand, the preferred embodiments are exemplified below, and are described in detail as follows in conjunction with the accompanying drawings.

Description of drawings

In order to explain the technical solutions of the embodiments of the present disclosure more clearly, the following briefly introduces the accompanying drawings required in the embodiments, which are incorporated into the specification and constitute a part of the specification. The drawings illustrate embodiments consistent with the present disclosure, and together with the description serve to explain the technical solutions of the present disclosure. It should be understood that the following drawings only show some embodiments of the present disclosure, and therefore should not be regarded as limiting the scope. Other related figures are obtained from these figures.

FIG. 1A shows a flowchart of an information processing method provided by an embodiment of the present application;

FIG. 1B shows a schematic diagram of a system architecture of an information processing method provided by an embodiment of the present application;

FIG. 2 shows a flowchart of a method for determining pedestrian trajectory data provided by an embodiment of the present application;

3 shows a flowchart of a method for determining video frames associated with the same pedestrian provided by an embodiment of the present application;

FIG. 4 shows a flowchart of the first method for adjusting at least part of the display area of items in the target site provided by the embodiment of the present application;

FIG. 5 shows a flowchart of a second method for adjusting at least a part of the display area of items in a target site provided by an embodiment of the present application;

FIG. 6 shows a flowchart of a third method for adjusting at least a part of the display area of items in a target site provided by an embodiment of the present application;

FIG. 7 shows a schematic diagram of a pedestrian heat map provided by an embodiment of the present application;

FIG. 8 shows a flowchart of an early warning prompting method provided by an embodiment of the present application;

FIG. 9 shows a flowchart of a fourth method for adjusting at least a part of the item display area in a target site provided by an embodiment of the present application;

FIG. 10 shows a flowchart of the fifth method for adjusting at least part of the item display area in the target place provided by the embodiment of the present application

FIG. 11 shows a schematic structural diagram of an information processing apparatus provided by an embodiment of the present application;

FIG. 12 shows a schematic diagram of an electronic device provided by an embodiment of the present application.

Detailed ways

In order to make the purposes, technical solutions and advantages of the embodiments of the present disclosure more clear, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only These are some, but not all, embodiments of the present disclosure. The components of the disclosed embodiments generally described and illustrated in the drawings herein may be arranged and designed in a variety of different configurations. Therefore, the following detailed description of the embodiments of the disclosure provided in the accompanying drawings is not intended to limit the scope of the disclosure as claimed, but is merely representative of selected embodiments of the disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the present disclosure.

It should be noted that like numerals and letters refer to like items in the following figures, so once an item is defined in one figure, it does not require further definition and explanation in subsequent figures.

The term "and/or" in this paper only describes an association relationship, which means that there can be three kinds of relationships, for example, A and/or B, which can mean: the existence of A alone, the existence of A and B at the same time, the existence of B alone. a situation. In addition, the term "at least one" herein refers to any combination of any one of the plurality or at least two of the plurality, for example, including at least one of A, B, and C, and may mean including from A, B, and C. Any one or more elements selected from the set of B and C.

Multiple or multiple in the embodiments of the present disclosure may refer to at least two or at least two, respectively.

The traditional way of displaying items may cause uneven distribution of people. For example, people are likely to be distributed in areas with high sales, causing traffic congestion, which is not conducive to the circulation of people in shopping malls, thus affecting sales. Therefore, an adjustment information generation scheme for different item display areas in a shopping mall is urgently needed, so as to improve the problem caused by the uneven distribution of people flow caused by the traditional display item display method.

Based on the above research, the embodiment of the present application provides an information processing method, which can determine the pedestrian statistical data of different item display areas in combination with the collected video streams of the target place, and further can determine the pedestrian statistics based on the pedestrian statistical data. Deployment of at least part of the item display area.

In addition, considering that for a target place, whether the target place is a public place or a non-public place, for safety reasons, video capture components such as cameras are usually deployed. That is to say, the data sources for realizing the above technical solutions usually belong to existing resources, and there is no need to collect additional video streams. In this way, the technical solutions provided by the embodiments of the present application can obtain deployment suggestions suitable for the target site by rationally utilizing the existing resources, thereby realizing the full utilization of the existing resources. In addition, the obtained adjustment scheme of the item display area can be more adapted to the actual needs of the target place.

In order to facilitate the understanding of this embodiment, an information processing method disclosed in the embodiment of this application is first introduced in detail. The execution subject of the information processing method provided by the embodiment of this application is generally a computer device with a certain computing capability. Devices include, for example, terminal devices or servers or other processing devices. In some possible implementations, the information processing method may be implemented by a processor invoking computer-readable instructions stored in a memory.

Referring to FIG. 1A , which is a flowchart of an information processing method provided by an embodiment of the present application, the information processing method includes the following S101 to S103:

S101: Determine, based on the video stream collected from the target place, track data of pedestrians in the video stream in the target place.

Exemplarily, the target place may be a place containing pedestrian distribution statistics, such as an exhibition hall including multiple exhibition halls. The types of displayed items can be adjusted, and it can also be a shopping mall containing multiple stores. Based on the statistics of the number of pedestrians in different stores within a set period, the deployment of each store display can be adjusted.

Exemplarily, a video capture component for capturing video streams may be arranged in multiple areas included in the target site, and the video capture component includes a camera, which may be a three-primary color (Red Green Blue, RGB) camera, or a three-primary color depth (Red Green Blue Depth). , RGBD) cameras, which are not limited here, for example, cameras for video capture of the corresponding article display areas can be set up in multiple item display areas of the target site, so that the cameras set in multiple physical display areas can capture After the video streams are spliced, the video streams of the entire target site can be obtained.

Exemplarily, after obtaining the video stream for the target place, the video stream can be framed, so as to perform pedestrian detection on the video frame in the video stream, so that the position change of the pedestrian in the video stream in the video frame can be determined, Further, the trajectory data of pedestrians in the video stream in the target place can be obtained. For example, for the same pedestrian A as an example, the trajectory data of the pedestrian A can include the pedestrian A at multiple time points or at multiple time periods. Location.

S102 , based on the trajectory data of pedestrians in the video stream in the target place and the position ranges of different item display areas in the target place, determine pedestrian statistical data corresponding to different item display areas respectively.

Exemplarily, the location ranges of different item display areas included in the target site may be determined in advance, for example, a world coordinate system may be established for the target site, so that the world coordinate system of different item display areas in the target site may be determined and saved in advance. Location range.

Exemplarily, the pedestrian statistical data corresponding to the item display area may refer to the number of pedestrians contained in the item display area at a preset time, such as determining the number of pedestrians contained in different item display areas at 9:00, for example, it may be determined based on the video stream. The trajectory data of the pedestrian at the preset time (that is, the position of the pedestrian at the preset time) and the position range of different item display areas are determined; or the pedestrian statistics corresponding to the item display area can refer to the item display area at the set time period. The number of pedestrians in the video stream can be determined based on the trajectory points of the pedestrians in the set period determined by the video stream and the position range of different item display areas. The implementation method will be described in detail later.

S103 , based on the pedestrian statistical data, determine the deployment for at least part of the item display area in the target place.

Exemplarily, based on the pedestrian statistical data, determining the deployment for at least part of the item display area in the target site includes at least one of the following: a deployment suggestion for at least a part of the item display area in the target site; deployment; wherein, the deployment suggestion may include at least one of the following: an adjustment suggestion for the placement position of the displayed items, and an adjustment suggestion for the display shelves used for displaying the items; correspondingly, adjusting the deployment of at least part of the item display area in the target site may It includes at least one of the following: adjustment of the displayed items, and adjustment of the display shelves used for displaying the items.

Pedestrian statistics of different item display areas can be determined in combination with the collected video streams of the target site, and further, based on the pedestrian statistics data, deployment for at least part of the item display areas in the target site can be determined.

FIG. 1B is a schematic diagram of a system architecture to which an information processing method according to an embodiment of the present application can be applied; as shown in FIG. In order to support an exemplary application, the video capture device 2001 and the image capture terminal 2003 can establish a communication connection through the network 2002, the video capture device 2001 transmits the captured video stream to the image capture terminal 2003 through the network 2002, and the image capture terminal 2003 receives the video stream. , and analyze the video stream, and then determine the deployment for the display area of some items in the target place in the video stream.

As an example, the current scene video capture device 2001 may include an image capture device such as a camera, and the camera may be, for example, an RGB camera or an RGBD camera. The image acquisition terminal 2003 may include a computer device with a certain computing capability, for example, the computer device includes a terminal device or a server or other processing devices. The network 2002 can be wired or wireless. Wherein, when the video capture device 2001 is an image capture device and the image capture terminal 2003 is a server, the video capture device can be connected to the image capture terminal through a wired connection, such as data communication through a bus; when the video capture device 2001 is an image capture device Acquisition device, when the image acquisition terminal 2003 is a terminal device, the video acquisition device can communicate with the image acquisition terminal through a wireless connection, thereby performing data communication.

Or, in some scenarios, the image acquisition terminal 2003 may be a vision processing device with a video acquisition module, or a host with a camera. At this time, the information processing method of the embodiment of the present application may be executed by the image acquisition terminal 2003 , and the above-mentioned system architecture may not include the network 2002 and the video acquisition device 2001 .

The above S101 to S103 will be described in detail below with reference to some embodiments.

For the above S101, when determining the trajectory data of pedestrians in the video stream in the target place based on the video stream collected from the target place, as shown in FIG. 2, the following S201 and S202 may be included:

S201: Perform pedestrian detection on a video frame in a video stream to obtain position indication information of the same pedestrian in an associated video frame.

Exemplarily, considering that the pedestrians in the video frame may be occluded, the whole body of the person may not be photographed from the video frame. Therefore, when pedestrian detection is performed on the video frame in the video stream, head and shoulders detection can be considered. algorithm to complete the detection of pedestrians in video frames.

Exemplarily, the position indication information may include information used to indicate the position of the pedestrian in the video frame, for example, may include a detection frame determined based on the pedestrian detection technology and used to represent the position information of the pedestrian in the video frame, and may also include the pedestrian in the video frame. The relative position information between the video frame and the preset feature points. When the position information of the preset feature points in the target site is known, the pedestrian can be determined by combining the relative position information between the preset feature points and the pedestrian. Trajectory data in the target location.

Considering that a video stream can contain multiple pedestrians, when tracking pedestrians to determine the trajectory data of pedestrians, it is necessary to identify the same pedestrian contained in different frames, and determine the position indication information of the same pedestrian in the associated video frame, such as If consecutive n video frames contain the same pedestrian, the n video frames can be regarded as video frames associated with the same pedestrian, and the position indication information of the same pedestrian in the n video frames can be determined.

S202: Determine the trajectory data of the same pedestrian in the target place based on the position indication information of the same pedestrian in the associated video frame.

Exemplarily, the position indication information may include a detection frame for indicating the position of the pedestrian in the video frame, for example, the position information of the center point of the bottom edge of the detection frame in the video frame may be used as the position information of the pedestrian in the video frame, Then, based on the transformation relationship between the image coordinate system and the camera coordinate system, and the transformation relationship between the camera coordinate system and the world coordinate system, the position of the pedestrian in the target place is determined.

Exemplarily, the position indication information can also include the relative position information of the pedestrian in the video frame and the preset feature point, the position information of the preset feature point in the target place can be determined first, and then according to the preset feature point and the preset feature point. The relative position information between pedestrians is used to determine the pedestrian's position in the target place. For example, the preset feature point and the pedestrian's foot are coincident in the video frame, and the position of the preset feature point in the target place can be used as the pedestrian's foot. position in the target site.

Exemplarily, when tracking the same pedestrian, multiple positions of the same pedestrian in the target site can be obtained. For example, the positions of the same pedestrian in the target site include position A, position B, and position C, and these positions are sequentially connected in sequence. Then, the trajectory data of the same pedestrian in the target place can be represented.

In the embodiment of the present application, by using the pedestrian detection technology, the position indication information used to indicate the position of the pedestrian in the video frame can be quickly determined, and based on this, the trajectory data of the same pedestrian can be quickly determined.

The position indication information of the same pedestrian in the associated video frame mentioned above includes the relative position information of the same pedestrian in the associated video frame and the preset feature point; for the above S202, based on the same pedestrian in the associated video frame The location indication information, when determining the trajectory data in the same pedestrian target place, may include the following S2021 and S2022:

S2021, extracting preset feature points in video frames associated with the same pedestrian;

S2022 , based on the preset position information of the preset feature point in the target place and the relative position information of the same pedestrian and the preset feature point, determine the trajectory data of the same pedestrian in the target place.

Exemplarily, a computer-aided design (Computer Aided Design, CAD) diagram corresponding to the target site can be drawn in advance, and the preset position information in the target site of a plurality of preset feature points contained in the target site can be marked in the CAD diagram, The plurality of preset feature points can be pre-selected feature points contained in objects that are stationary for a certain period of time, such as the ground, table, lamp, wall, etc. The preset position information of the preset feature points in the target place can be determined according to the preset position information. The world coordinate system corresponding to the established target location is determined.

Exemplarily, taking a video frame associated with a pedestrian as an example, the preset feature points associated with the position information of the pedestrian can be extracted according to the position information of the pedestrian in the video frame. In this way, the position of the pedestrian in the target place can be determined directly based on the preset position information of the preset feature point in the target place. In this way, the video frames associated with the same pedestrian contain multiple During the frame, multiple positions of the same pedestrian in the target place can be determined, and thus the position change of the same pedestrian in the target place can be determined, and the same pedestrian can be tracked.

For the above-mentioned process of determining the position indication information of the same pedestrian in the associated video frame, for example, the video frame associated with the same pedestrian can be determined in the following manner, as shown in FIG. 3 , including S301 to S303:

S301 , performing head and shoulder detection on a video frame in a video stream, and acquiring the pedestrian head and shoulder feature information contained in the video frame.

Exemplarily, after the video stream is divided into frames, for each frame of video frame, after it is determined that the video frame contains the head and shoulders information of the pedestrian, feature information for describing the head and shoulders of the pedestrian can be extracted. The outline features, shape features, color features, texture features, and motion features of pedestrians’ heads and shoulders can also include pedestrian facial features. Through the combination of multiple features, it can be used to determine whether pedestrians in different video frames are the same pedestrian. Technical Support.

S302 , based on the pedestrian head and shoulder feature information contained in different video frames respectively, determine the feature similarity between pedestrians contained in different video frames.

Exemplarily, feature vectors corresponding to pedestrians contained in different video frames can be generated according to the pedestrian head and shoulder feature information contained in different video frames, and then the feature similarity between pedestrians contained in different video frames can be determined by the cosine formula.

S303, taking the video frames with the feature similarity higher than the preset similarity threshold as the video frames associated with the same pedestrian.

Exemplarily, considering that the similarity between the pedestrian feature information corresponding to the same pedestrian in different video frames of the video stream is relatively high, therefore, the videos associated with the same pedestrian can be screened based on a preset similarity threshold value set in advance. For example, if the feature similarity between a pedestrian contained in the first frame, the second frame and the third frame is higher than the preset similarity threshold, the first frame, the second frame and the third frame can be used as the pedestrian associated with the pedestrian. video frame.

Exemplarily, in the process of performing head and shoulders detection on the video frame, ID encoding can also be performed on the detected pedestrian, for example, the same pedestrian in different video frames can be encoded by the same ID, so that in the process of tracking the pedestrian. , the track point information of the same pedestrian can be tracked through the same ID.

In a possible implementation manner, for the above S103, when determining the deployment for at least part of the item display area in the target place based on pedestrian statistics, as shown in FIG. 4, the following S401 and S402 may be included:

S401 , based on pedestrian statistical data corresponding to different item display areas and trajectory data in the target place, determine whether there is a crowd congestion in the first target item display area.

Exemplarily, it can be determined whether the number of pedestrians in each item display area in the preset time period exceeds the number of pedestrians corresponding to the congestion, according to the pedestrian statistical data corresponding to different item display areas within a preset period of time, such as in advance. When the number of pedestrians corresponding to the item display area with an area of n square meters exceeds m, the item display area may be crowded with people. Further, it can be combined with the trajectory data of the target site to detect whether there are pedestrians arriving at an item with a large number of pedestrians. After the display area, turn back to the original road, or take a detour. If there is, it can be determined that there is a high probability of crowd congestion in the display area of the first target item.

S402, when it is determined that the first target item display area is crowded with people, at least one of the following is performed: generating a deployment suggestion for the first target item display area, and adjusting the deployment of the first target item display area.

Exemplarily, for the first target item display area where it is determined that there is a crowd congestion, it is possible to further determine the cause of the crowd congestion in the first target item display area, and then at least one of the following is performed: Deploy suggestions and adjust the deployment of the first target item display area.

In a possible implementation manner, for the above S402, when it is determined that there is a crowd congestion in the first target item display area, at least one of the following is performed: generating a deployment suggestion for the first target item display area, adjusting the first target item display area During the deployment of the target item display area, as shown in FIG. 5 , the following S501 to S503 may be included:

S501. Extract video frames associated with the display area of the first target item from the video stream.

Exemplarily, multiple cameras can be set in the target place, and each camera can collect video streams corresponding to different areas. The video stream collected by the camera associated with the display area is further processed into frames to obtain the video frame associated with the display area of the first target item.

S502: Determine the posture data of the pedestrian in the first target item display area in the extracted video frame.

Exemplarily, the gesture data of the pedestrian in the video frame in the display area of the first target item may be recognized based on the pre-trained gesture recognition network. For example, the gesture data may include a leaning posture and a standing posture.

S503: Based on the gesture data, at least one of the following is performed: generating a deployment suggestion for the first target item display area, and adjusting the deployment of the first target item display area.

Exemplarily, for a shopping scene, such as a supermarket shopping scene, when most of the commodities that customers pay attention to are arranged on the bottom of the shopping rack, the customer needs to lean over to select such commodities, so that the aisle in front of the shopping rack is easy to see. To cause congestion, for this, at least one of the following is performed: generating a deployment suggestion for the first target item display area, and adjusting the deployment of the first target item display area.

In a possible implementation, the first target article display area includes at least one article display shelf, and each article display shelf includes multiple layers. For the above S503, based on the gesture data, at least one of the following is performed: The deployment suggestion of the target item display area and the adjustment of the deployment of the first target item display area may include the following S5031 and S5032:

S5031, based on the attitude data, in the case where it is determined that there are more than a set number of pedestrians in the video frame whose attitudes conform to the first preset attitude, obtain a first target item display that matches the gaze direction of the more than set number of pedestrians The first target layer of the shelf.

Exemplarily, taking the store in the first target item display area as the target site as an example, the store may include multiple item display shelves, and a passage for customers to pass between two adjacent item display shelves is formed. Display shelves can contain multiple layers from the bottom up.

Exemplarily, the first preset posture may be a leaning-over posture, and when it is detected that the posture of the customers exceeding the set number is the leaning-over posture, it is determined by detecting the gaze direction of these customers that the posture exceeds the set number. On the first target layer of the display shelf for the first target item that the customer pays attention to, it can be determined that the cause of the crowd congestion is that more than the set number of customers lean over to select the goods on the lower layer of the first target article display shelf.

S5032, based on the first target layer, at least one of the following is performed: generating a deployment suggestion instructing to adjust the items displayed on the first target layer to the second target layer of the first target item display shelf for display; The item is adjusted to the second target layer, and the height of the second target layer from the ground is higher than the height of the first target layer from the ground.

Exemplarily, considering that the first target item display area is crowded with people at the first target item display shelf due to the over-set number of customers leaning over to pick the goods on the lower layer of the first target item display shelf, at least the following can be performed: One: generate a deployment proposal that instructs to adjust the items displayed on the lower level to the upper level for display, and adjust the items displayed on the lower level to the upper level, so that customers can stand and select the goods they are interested in, so as to alleviate the problem of most customers leaning over to choose. Traffic congestion caused by goods.

In a possible implementation manner, the item display shelves include a plurality of shelves, and the information processing method provided in this embodiment of the present application further includes the following S5033 and S5034:

S5033, based on the trajectory data of the pedestrians in the extracted video frame in the first target item display area, determine that the number of passing pedestrians is less than the first set number of second target item display shelves;

S5034: Based on the second target item display rack, at least one of the following is performed: generating a deployment suggestion instructing to adjust some items displayed on the first target layer to the second target item display rack, displaying some items on the first target layer Adjusted to the second target item display shelf.

Exemplarily, the first set number may be determined according to the maximum number of pedestrians that can be accommodated between different item display shelves included in the item display area. For example, the maximum number of pedestrians that can be accommodated between different item display shelves is N, then the The first set number may be set to be k times N, where k is greater than 0 and less than 1, and the first set number indicates that the number of pedestrians is small, far less than the maximum number of pedestrians.

Exemplarily, also taking the first target item display area as an example of a store in the target location, for the above-mentioned cause of traffic congestion, more than the set number of customers leaned over to select the first target item on the lower level of the display shelf. In the case of goods caused, a second target item display shelf with fewer customers can be selected in the first target item display area, so that the goods displayed on the first target layer in the first target item shelf can be adjusted to the second target item. Items are placed on display shelves, so that popular goods can be placed in different item display shelves to reduce the probability of crowd congestion.

In another embodiment, the first target item display shelf is a movable shelf, and for the above S503, based on the gesture data, at least one of the following is performed: generating a deployment suggestion for the first target item display area, adjusting the first target During the deployment of the item display area, the following S5035 and S5036 are also included:

S5035, based on the attitude data, in the case where it is determined that there are more than a set number of pedestrians in the video frame whose attitudes conform to the second preset attitude, obtain a first target item display that matches the gaze direction of the more than set number of pedestrians The current position of the shelf and the location area where the traffic congestion occurs;

S5036, based on the current location and the location area where the crowd congestion occurs, at least one of the following is performed: generating a deployment suggestion for moving the first target item display shelf, adjusting the first target item display shelf according to the current location and the location area where the crowd congestion occurs .

Exemplarily, also take the store in the target location as the first target item display area as an example, the plurality of item display shelves included in the store are movable shelves, and the crowded situation occurs in the area where the first target item display shelf is located. In this case, the congestion situation can be improved by adjusting the position of the display shelf of the first target item.

Exemplarily, the second preset posture may be a standing posture, and in the case that the postures of the customers exceeding the set number are detected as standing postures, the attention of the customers exceeding the set number is determined by detecting the gaze directions of these customers. The first target item display shelf, it can be determined that the cause of the crowd congestion is caused by more than the set number of customers standing near the first target item display shelf and picking the goods. At this time, the first target item can be identified according to the video frame. The current position of the display shelf and the location area where the crowd congestion occurs may further perform at least one of the following: generating a deployment suggestion for moving the display shelf of the first target item, adjusting the first target item according to the current position and the location area where the crowd congestion occurs Display shelves.

Exemplarily, when adjusting the first target item display rack according to the current position and the location area where the crowd congestion occurs, at least one of the following may be performed: translating the first target item display rack, and rotating the first target item display rack , wherein the translation distance is to move from the current position of the first target item display shelf to the direction away from the crowd congestion, the translation distance is determined according to the range of the location area where the crowd congestion is located, and the rotation can affect the first target item display shelf. The angle of rotation can be used to adjust the position of the customer when selecting the product through the rotation of the display shelf, and since the flow of people usually moves along the direction of the shelf, this adjustment can also effectively adjust the movement direction of the flow of people. When a target item display shelf is rotated, it can be rotated in a direction away from the crowd congestion, and the rotation angle is determined according to the range of the location area where the crowd congestion is located.

In another possible implementation, for the above S402, when it is determined that there is a crowd congestion in the first target item display area, based on pedestrian statistics, when determining the deployment for at least part of the item display area in the target place, such as As shown in FIG. 6, the following S601 and S602 may be included:

S601, based on pedestrian statistical data corresponding to different item display areas, determine a second target item display area where the number of pedestrians is less than a second set number;

S602, based on the second target item display area, at least one of the following is performed: generating a deployment suggestion instructing to adjust part of the items displayed in the first target item display area to the second target item display area; The item is adjusted to the second target item display area.

Exemplarily, the second set number can be determined according to the maximum number of pedestrians that can be accommodated in the item display area. For example, the maximum number of pedestrians that can be accommodated in different item display areas is P, then the second set quantity can be set as f of P. times, where f is greater than 0 and less than 1, the second set number indicates that the number of pedestrians is small, which is far less than the maximum number of pedestrians that can be accommodated in the item display area.

In an implementation manner, the information processing method provided by the embodiment of the present application further includes:

Based on the pedestrian statistical data corresponding to different item display areas in at least one time period, and the preset rendering methods corresponding to different pedestrian statistical data, a pedestrian heat map of the target place within the preset time period is generated and output.

Exemplarily, the preset duration includes multiple time periods, for example, one time period is 1 hour, and a pedestrian heat map of the target place within 12 hours can be generated and output.

Exemplarily, in order to more vividly grasp the changes in the number of pedestrians corresponding to different item display areas within a preset time period, a pedestrian heat map can be generated according to the preset rendering methods corresponding to different pedestrian statistical data. The area with the corresponding pedestrian statistical data greater than M is marked in red, and the item display area with the corresponding pedestrian statistical data greater than N and less than or equal to M in this period is marked in yellow, and the corresponding pedestrian statistical data in this period is greater than or equal to M. 0, and the item display area less than or equal to N is marked as green, and the item display area with the pedestrian statistical data of 0 in this period is marked as white, so that the pedestrian heat corresponding to the target place at different time periods within the preset time period can be obtained. Changes in the graph.

As shown in FIG. 7 , which is the pedestrian heat map corresponding to the target site in a period, it can be seen that the item display area A contains the largest number of pedestrians in this period, and the item display area C contains the least number of pedestrians in this period.

In the embodiment of the present application, based on the pedestrian statistical data corresponding to different item display areas in at least one time period, a people flow heat map that can intuitively reflect the changes in the number of people in each item display area in the target place can be generated, so as to be the number of people in the target place. The item display area provides deployment suggestions.

In an implementation manner, as shown in FIG. 8 , the information processing method provided by this embodiment of the present application further includes the following S701 and S702:

S701, based on the pedestrian statistical data corresponding to different item display areas, determine whether there is at least one item display area that meets the warning condition;

S702, when it is determined that there is at least one article display area that meets the warning condition, generate first warning prompt information.

In a possible implementation, the warning condition includes that the number of pedestrians in the item display area is greater than or equal to a first preset threshold.

Exemplarily, the first preset threshold may be set according to actual needs, for example, it may be set according to the type of the target place, the position of the item display area in the target place, the statistical time period, and the like.

In another possible implementation, the warning condition further includes at least one of the following:

(1) The difference between the number of pedestrians in the item display area and the number of pedestrians in any other item display area is greater than the second preset threshold.

(2) The distance between the item display area and the adjacent item display area is less than the preset distance threshold;

(3) The item display area is located in the preset transportation hub area in the target site.

For item (1), the second preset threshold is used to measure the maximum difference between the number of pedestrians contained in any two item display areas in the target site. For example, for the case where the target site is the same store, the store contains multiple items In the display area, if there is a large difference in the number of pedestrians between an item display area and any other item display area, a first warning prompt message can be generated to facilitate timely inspection of problems in the item display area.

For item (2), considering that the time period with large passenger flow is easy to cause drainage congestion, the distance between adjacent item display areas needs to meet certain conditions, such as not less than the preset distance threshold, so when the item is detected When the distance between the display area and the adjacent item display area is smaller than the preset distance threshold, first warning prompt information may be generated, which facilitates timely adjustment.

For item (3), multiple traffic roads of the target site can be pre-planned, and there are some areas that are intersection areas of multiple traffic roads, that is, the preset traffic hub area. In the case where the item display area is located in the preset traffic hub area , the first warning prompt information can be generated, so that the position of the item display area can be adjusted in time, and the occurrence of traffic jams can be reduced.

In a possible implementation, the duration of the congestion event can be detected, and the second alarm information can be output in time. That is, first, when it is determined that there is a crowd congestion in the display area of the first target item, the first target object is obtained. The duration of the crowd congestion in the item display area; then, when the duration reaches a preset duration threshold, a second warning prompt message is generated. In this way, the duration of the congestion event is detected, and when it is detected that the duration of the congestion event is long, an early warning prompt is provided, which can improve the safety of the target site.

Exemplarily, the generated first warning prompt information may be information in the format of text, voice, video, etc. For example, the generated first early warning prompt information may be used to prompt pedestrians, for example, it may be "Attention, where are you currently? There are a lot of pedestrians in the area, please pay attention to safety.”

In an implementation manner, as shown in FIG. 9 , the information processing method provided by this embodiment of the present application further includes the following S801 and S802:

S801, for a third target item display area where there is no crowd congestion, obtain the number of pedestrians corresponding to the third target item display area in multiple time periods;

S802 , when it is determined that the number of pedestrians corresponding to the third target item display area in multiple time periods is less than a third preset threshold, generate a deployment suggestion for the third target item display area.

Exemplarily, the third preset threshold may be used to indicate that the number of pedestrians contained in the item display area is small. When it is determined that the third target item display area is not crowded with people, if the third target item display area is The number of pedestrians contained in a long period of time is small, indicating that the third target item display area may be remote in the target site, or the displayed items are less attractive, and you can consider generating a deployment for the third target item display area Recommendations, such as placing popular items in the store front area to attract customers into the store.

In an implementation manner, as shown in FIG. 10 , the information processing method provided by this embodiment of the present application further includes the following S803 and S804:

S803, when it is determined that the number of pedestrians corresponding to the third target item display area in at least one time period is greater than the third preset threshold, based on the trajectory data corresponding to each sub-area in the third target item display area, determine the third target item display area Whether there is at least one sub-area in the area where the number of pedestrians corresponding to multiple time periods is less than the fourth preset threshold;

S804, when it is determined that the number of pedestrians corresponding to at least one sub-area in multiple time periods is less than a fourth preset threshold, generate a deployment suggestion for the third target item display area.

Exemplarily, the fourth preset threshold is smaller than the third preset threshold, and the third target item display area may include multiple sub-areas. If the third target item display area contains a number of pedestrians that is greater than the third preset threshold, It shows that the third target item display area can attract customers, but there is at least one sub-area that contains a small number of pedestrians for a long period of time, indicating that the sub-area may be remote in the third target item display area, or the sub-area The items displayed in the area are less attractive, and you can consider generating deployment suggestions that are adjusted for the items displayed in the sub-area.

Those skilled in the art can understand that, in the above-mentioned method of the specific embodiment, the writing order of each step does not mean a strict execution order but constitutes any limitation on the implementation process, and the execution order of each step should be based on its function and possible intrinsic Logical OK.

Based on the same technical concept, the embodiment of the present application also provides an information processing device corresponding to the information processing method. Reference may be made to the implementation of the method, and repeated descriptions will not be repeated.

Referring to FIG. 11, which is a schematic diagram of an information processing apparatus 900 provided by an embodiment of the present application, the information processing apparatus includes:

The first determining module 901 is configured to determine the trajectory data of pedestrians in the target place in the video stream based on the video stream collected from the target place;

The second determination module 902 is configured to determine pedestrian statistics corresponding to different item display areas respectively based on the trajectory data of pedestrians in the video stream in the target site and the position ranges of different item display areas in the target site;

The processing module 903 is configured to determine, based on the statistical data of pedestrians, the deployment for at least part of the item display area in the target place.

In a possible implementation manner, the processing module 903 is further configured to generate deployment suggestions for at least part of the item display areas in the target place based on the pedestrian statistical data;

is configured to adjust the deployment of at least a portion of the item display area within the target venue based on pedestrian statistics.

In a possible implementation manner, when the processing module 903 is configured to determine the deployment for at least part of the item display area in the target place based on pedestrian statistics, the process includes:

Based on the pedestrian statistics data corresponding to different item display areas and the trajectory data in the target place, determine whether there is a crowd congestion in the first target item display area;

When it is determined that the first target item display area is crowded with people, at least one of the following is performed: generating a deployment suggestion for the first target item display area, and adjusting the deployment of the first target item display area.

In a possible implementation manner, the processing module 903 is configured to perform at least one of the following when it is determined that there is a congestion of people in the display area of the first target item: generating a deployment suggestion for the display area of the first target item, adjusting The deployment of the first target item display area includes:

extracting video frames associated with the display area of the first target item from the video stream;

Determine the posture data of the pedestrian in the extracted video frame in the display area of the first target item;

In a possible implementation, the first target article display area includes at least one article display shelf, and each article display shelf includes multiple layers, and the processing module 903 is configured to perform at least one of the following based on the gesture data: The deployment suggestion of a target item display area, and when adjusting the deployment of the first target item display area, include:

Based on the gesture data, in the case where it is determined that there are more than a set number of pedestrians in the video frame whose gestures conform to the first preset attitude, obtain the first target item display shelf that matches the gaze direction of the more than set number of pedestrians the first target layer;

Based on the first target layer, at least one of the following is performed: generating a deployment suggestion instructing to adjust the items displayed on the first target layer to the second target layer of the first target item display shelf for display, adjusting the items displayed on the first target layer To the second target layer, the height of the second target layer from the ground is higher than the height of the first target layer from the ground.

In a possible implementation manner, the item display shelves include a plurality of shelves, and the processing module 903 is further configured to:

Based on the trajectory data of the pedestrians in the extracted video frame in the display area of the first target item, determine that the number of passing pedestrians is less than the first set quantity of the second target item display shelf;

Based on the second target item display rack, at least one of the following is performed: generating a deployment proposal instructing to adjust some items displayed on the first target layer to the second target item display rack, adjusting some items displayed on the first target layer to the second target item display rack Target item display shelf.

In a possible implementation manner, the first target item display shelf is a movable shelf, and the processing module 903 is configured to perform at least one of the following based on the gesture data: generate a deployment suggestion for the first target item display area, adjust the first target item display area When deploying a target item display area, it also includes:

Based on the gesture data, in the case where it is determined that there are more than a set number of pedestrians in the video frame whose gestures conform to the second preset attitude, obtain the first target item display shelf that matches the gaze direction of the more than set number of pedestrians The current location and the location area where crowd congestion occurs;

Based on the current location and the location area where crowd congestion occurs, at least one of the following is performed: generating a deployment suggestion for moving the first target item display rack, and adjusting the first target item display rack according to the current location and the location area where crowd congestion occurs.

In a possible implementation manner, when the processing module 903 is configured to, when it is determined that there is a crowd congestion in the first target item display area, based on pedestrian statistics, when determining the deployment for at least part of the item display area in the target place, include:

Based on the pedestrian statistical data corresponding to different item display areas, determine a second target item display area where the number of pedestrians is less than the second set number;

Based on the second target item display area, at least one of the following is performed: generating a deployment suggestion instructing to adjust some items displayed in the first target item display area to the second target item display area, adjusting some items in the first target item display area to the second target item display area.

In a possible implementation manner, when the first determining module 901 is configured to determine the trajectory data of pedestrians in the video stream in the target place based on the video stream collected from the target place, the method includes:

Perform pedestrian detection on the video frames in the video stream to obtain the position indication information of the same pedestrian in the associated video frame;

In a possible implementation manner, the position indication information of the same pedestrian in the associated video frame includes the relative position information of the same pedestrian in the associated video frame and the preset feature point; the first determining module 901 is configured to be based on the same Pedestrian position indication information in the associated video frame, when determining the trajectory data in the same pedestrian target location, including:

Extract preset feature points in video frames associated with the same pedestrian;

Based on the preset position information of the preset feature point in the target place and the relative position information of the same pedestrian and the preset feature point, the trajectory data of the same pedestrian in the target place is determined.

In a possible implementation manner, the first determining module 901 is configured to determine video frames associated with the same pedestrian in the following manner:

Perform head and shoulder detection on the video frame in the video stream to obtain the pedestrian head and shoulder feature information contained in the video frame;

Determine the feature similarity between pedestrians contained in different video frames based on the pedestrian head and shoulder feature information contained in different video frames;

The video frames whose feature similarity is higher than the preset similarity threshold are regarded as the video frames associated with the same pedestrian.

In a possible implementation manner, the processing module 903 is further configured to:

Based on the pedestrian statistics corresponding to different item display areas, determine whether there is at least one item display area that meets the warning condition;

The item display area is located in a preset traffic hub area in the target site.

When it is determined that there is a crowd congestion in the display area of the first target item, obtain the duration of the crowd congestion in the display area of the first target item;

When the duration reaches the preset duration threshold, second warning prompt information is generated.

When it is determined that the number of pedestrians corresponding to the third target item display area in multiple time periods is all smaller than the third preset threshold, a deployment suggestion for the third target item display area is generated.

In the case where it is determined that the number of pedestrians corresponding to the third target item display area in at least one time period is greater than the third preset threshold, determine the number of pedestrians in the third target item display area based on the trajectory data corresponding to each sub-area in the third target item display area Whether there is at least one sub-area in which the number of pedestrians corresponding to multiple time periods is less than the fourth preset threshold;

When it is determined that the number of pedestrians corresponding to at least one sub-area in multiple time periods is less than the fourth preset threshold, a deployment suggestion for the third target item display area is generated.

For the description of the processing flow of each module in the apparatus and the interaction flow between the modules, reference may be made to the relevant descriptions in the foregoing method embodiments, which will not be described in detail here.

Corresponding to the information processing method in FIG. 1 , the embodiment of the present application further provides an electronic device 1000 . As shown in FIG. 12 , the schematic structural diagram of the electronic device 1000 provided by the embodiment of the present application includes:

The processor 101, the memory 102, and the bus 103; the memory 102 is configured to store execution instructions, including the memory 1021 and the external memory 1022; the memory 1021 here is also called internal memory, and is configured to temporarily store the operation data in the processor 101, and For the data exchanged by the external memory 1022 such as the hard disk, the processor 101 exchanges data with the external memory 1022 through the memory 1021. When the electronic device 1000 is running, the processor 101 and the memory 102 communicate through the bus 103, so that the processor 101 executes the following instructions : Based on the video stream collected from the target place, determine the trajectory data of pedestrians in the video stream in the target place; Determine the pedestrian statistics corresponding to different item display areas; based on the pedestrian statistics, perform at least one of the following: generate deployment suggestions for at least part of the item display areas in the target site, and adjust the deployment of at least some item display areas in the target site.

Embodiments of the present application further provide a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is run by a processor, the steps of the information processing method in the foregoing method embodiments are executed. Wherein, the storage medium may be a volatile or non-volatile computer-readable storage medium.

Embodiments of the present application further provide a computer program product, where the computer program product carries program codes, and the instructions included in the program codes can be configured to execute the steps of the information processing methods described in the foregoing method embodiments, and reference may be made to the foregoing methods. Example.

Wherein, the above-mentioned computer program product can be realized by means of hardware, software or a combination thereof, for example. In an optional embodiment, the computer program product is embodied as a computer storage medium, for example, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK), etc. Wait.

Those skilled in the art can clearly understand that, for the convenience and brevity of description, for the working process of the system and device described above, reference may be made to the corresponding process in the foregoing method embodiments, and details are not repeated here. In the several embodiments provided in this application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. The apparatus embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some communication interfaces, indirect coupling or communication connection of devices or units, which may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and components shown as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit.

The functions, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a processor-executable non-volatile computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present application can be embodied in the form of software products in essence, or the parts that make contributions to the prior art or the parts of the technical solutions, and the computer software products are stored in a storage medium , including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program codes .

Lastly, it should be noted that the above-mentioned embodiments are only implementations of the embodiments of the present application, and are used to illustrate the technical solutions of the embodiments of the present application, but not to limit them, and the protection scope of the embodiments of the present application is not limited to Therefore, although the embodiments of the present application have been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that any person skilled in the art who is familiar with the technical field within the technical scope disclosed in the embodiments of the present application can still The technical solutions recorded in the embodiments are modified or easily thought of to change, or equivalent replacements are made to some of the technical features; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit of the technical solutions in the embodiments of the present application and scope, all should be covered within the protection scope of the embodiments of the present application. Therefore, the protection scope of the embodiments of the present application should be based on the protection scope of the claims.

Industrial Applicability

The embodiments of the present application disclose an information processing method, device, electronic device, and storage medium, wherein the information processing method includes: determining the trajectory of pedestrians in the video stream in the target place based on a video stream collected from a target place data; based on the trajectory data of pedestrians in the video stream in the target site, and the location ranges of different item display areas in the target site, determine the pedestrian statistics corresponding to different item display areas; Describe the deployment of at least part of the item display area in the target site.

Claims

An information processing method, the method is executed by an electronic device, comprising:

determining the trajectory data of pedestrians in the target place in the video stream based on the video stream collected from the target place;

Based on the trajectory data of the pedestrians in the video stream in the target place, and the position ranges of different item display areas in the target place, determine the pedestrian statistics corresponding to the different item display areas respectively;

Based on the pedestrian statistics, a deployment is determined for at least a portion of the item display area within the target venue.
The information processing method according to claim 1, wherein, based on the pedestrian statistical data, determining the deployment for at least part of the item display area in the target place includes at least one of the following:

generating, based on the pedestrian statistical data, deployment recommendations for at least part of the item display area in the target site;

Based on the pedestrian statistics, the deployment of at least a portion of the item display area within the target site is adjusted.
The information processing method according to claim 2, wherein, based on the pedestrian statistical data, determining the deployment for at least part of the item display area in the target place comprises:

Based on the pedestrian statistical data corresponding to the different item display areas and the trajectory data in the target place respectively, determining whether there is a crowd congestion in the first target item display area;

When it is determined that there is a crowd congestion in the first target item display area, at least one of the following is performed: generating a deployment suggestion for the first target item display area, and adjusting the deployment of the first target item display area.
The information processing method according to claim 3, wherein when it is determined that there is a crowd congestion in the first target item display area, at least one of the following is performed: generating a deployment suggestion for the first target item display area . Adjust the deployment of the first target item display area, including:

extracting video frames associated with the first target item display area from the video stream;

determining the posture data of the pedestrian in the extracted video frame in the display area of the first target item;

Based on the gesture data, at least one of the following is performed: generating a deployment suggestion for the first target item display area, and adjusting the deployment of the first target item display area.
The information processing method according to claim 4, wherein the first target article display area includes at least one article display shelf, each article display shelf includes multiple layers, and based on the gesture data, at least one of the following is performed : generating deployment suggestions for the first target item display area and adjusting the deployment of the first target item display area, including:

Based on the gesture data, when it is determined that there are more than a set number of pedestrians in the video frame whose gestures conform to the first preset attitude, obtain a first position matching the gaze direction of the more than set number of pedestrians a first target layer of a target item display shelf;

Based on the first target layer, at least one of the following is performed: generating a deployment proposal instructing to adjust the items displayed on the first target layer to a second target layer of the first target item display shelf for display; The items displayed on the first target layer are adjusted to the second target layer, and the height of the second target layer from the ground is higher than the height of the first target layer from the ground.
The information processing method according to claim 5, wherein the item display racks comprise a plurality of shelves, and the information processing method further comprises:

Based on the extracted trajectory data of the pedestrians in the video frame in the first target item display area, determine that the number of passing pedestrians is less than the first set number of second target item display shelves;

Based on the second target item display rack, at least one of the following is performed: generating a deployment suggestion instructing to adjust part of the items displayed on the first target layer to the second target item display rack; Some of the displayed items are adjusted to the second target item display shelf.
The information processing method according to claim 5 or 6, wherein the first target item display rack is a movable rack, and based on the gesture data, at least one of the following is performed: generating a target item for the first target item The deployment suggestion of the display area and the adjustment of the deployment of the first target item display area also include:

Based on the gesture data, when it is determined that there are more than a set number of pedestrians in the video frame whose gestures conform to the second preset attitude, obtain a first position matching the gaze direction of the more than set number of pedestrians The current position of a target item display shelf and the location area where the crowd congestion occurs;

Based on the current location and the location area where the crowd congestion occurs, at least one of the following is performed: generating a deployment suggestion for moving the first target item display shelf, and according to the current location and the location where the crowd congestion occurs The area adjusts the display shelf of the first target item.
The information processing method according to claim 3, wherein, when it is determined that there is a congestion of people in the target article display area, the determination is based on the pedestrian statistical data for at least part of the article display area in the target place. deployment, including:

Based on the pedestrian statistical data corresponding to the different item display areas respectively, determining a second target item display area where the number of pedestrians is less than the second set number;

Based on the second target item display area, at least one of the following is performed: generating a deployment suggestion instructing to adjust some items displayed in the first target item display area to the second target item display area; Some items in the display area are adjusted to the second target item display area.
The information processing method according to any one of claims 1 to 8, wherein the determining, based on the video stream collected from the target place, determines the trajectory data of pedestrians in the video stream in the target place, comprising:

Perform pedestrian detection on the video frames in the video stream to obtain the position indication information of the same pedestrian in the associated video frames;

Based on the position indication information of the same pedestrian in the associated video frame, the trajectory data of the same pedestrian in the target place is determined.
The information processing method according to claim 9, wherein the position indication information of the same pedestrian in the associated video frame includes relative position information of the same pedestrian and a preset feature point in the associated video frame; the Based on the position indication information of the same pedestrian in the associated video frame, determining the trajectory data of the same pedestrian in the target place, including:

extracting preset feature points in the video frames associated with the same pedestrian;

Based on the preset position information of the preset feature point in the target place and the relative position information of the same pedestrian and the preset feature point, determine the trajectory data of the same pedestrian in the target place .
The information processing method according to claim 9 or 10, wherein the video frames associated with the same pedestrian are determined in the following manner:

Perform head and shoulder detection on the video frame in the video stream, and obtain the pedestrian head and shoulder feature information contained in the video frame;

Determine the feature similarity between pedestrians included in the different video frames based on the pedestrian head and shoulders feature information included in the different video frames;

The video frames with the feature similarity higher than the preset similarity threshold are regarded as the video frames associated with the same pedestrian.
The information processing method according to any one of claims 1 to 11, wherein the information processing method further comprises:

Based on the pedestrian statistical data corresponding to the different item display areas in at least one time period, and the preset rendering methods corresponding to the different pedestrian statistical data, a pedestrian heat map of the target place within a preset time period is generated and output.
The information processing method according to any one of claims 1 to 12, wherein the information processing method further comprises:

Determine whether there is at least one item display area that meets the warning condition based on the pedestrian statistical data corresponding to the different item display areas;

When it is determined that there is at least one item display area that meets the warning condition, first warning prompt information is generated.
The information processing method according to claim 13, wherein the warning condition includes that the number of pedestrians in the item display area is greater than or equal to a first preset threshold.
The information processing method according to claim 14, wherein the warning condition further comprises at least one of the following:

The difference between the number of pedestrians in the item display area and the number of pedestrians in any other item display area is greater than the second preset threshold;

The distance between the item display area and the adjacent item display area is less than the preset distance threshold;

The item display area is located in a preset transportation hub area in the target site.
The information processing method according to any one of claims 2 to 15, wherein the information processing method further comprises:

In the case where it is determined that there is crowd congestion in the display area of the first target item, acquiring the duration of crowd congestion in the display area of the first target item;

When the duration reaches a preset duration threshold, second warning prompt information is generated.
The information processing method according to any one of claims 2 to 16, wherein the information processing method further comprises:

For the third target item display area where there is no crowd congestion, obtain the number of pedestrians corresponding to the third target item display area in multiple time periods;

In the case where it is determined that the number of pedestrians corresponding to the third target item display area in the plurality of time periods is less than a third preset threshold, a deployment suggestion for the third target item display area is generated.
The information processing method according to claim 17, wherein the information processing method further comprises:

In the case where it is determined that the number of pedestrians corresponding to the third target item display area in at least one time period is greater than the third preset threshold, based on the trajectory data corresponding to each sub-area in the third target item display area, determine Whether there is at least one sub-area in the third target item display area and the number of pedestrians corresponding to the plurality of time periods is smaller than a fourth preset threshold;

When it is determined that the number of pedestrians corresponding to at least one sub-area in the plurality of time periods is less than a fourth preset threshold, a deployment suggestion for the third target item display area is generated.
An information processing device, comprising:

a first determining module, configured to determine the trajectory data of pedestrians in the target place in the video stream based on the video stream collected from the target place;

The second determining module is configured to determine the pedestrians corresponding to the different item display areas respectively based on the trajectory data of the pedestrians in the video stream in the target place and the position ranges of the different item display areas in the target place Statistical data;

A processing module configured to determine deployment for at least a portion of the item display area in the target venue based on the pedestrian statistics.
An electronic device, comprising: a processor, a memory and a bus, the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus , the machine-readable instructions execute the steps of the information processing method according to any one of claims 1 to 18 when the machine-readable instructions are executed by the processor.
A computer-readable storage medium having a computer program stored on the computer-readable storage medium, the computer program executes the steps of the information processing method according to any one of claims 1 to 18 when the computer program is run by a processor.
A computer program, comprising computer-readable codes, in the case that the computer-readable codes are executed in an electronic device and executed by a processor in the electronic device, to implement the method described in any one of claims 1 to 18 method.