CN111008859B

CN111008859B - Information presentation method and device in virtual store, electronic equipment and storage medium

Info

Publication number: CN111008859B
Application number: CN201911097190.2A
Authority: CN
Inventors: 李晓彤
Original assignee: Beijing Megvii Technology Co Ltd
Current assignee: Beijing Megvii Technology Co Ltd
Priority date: 2019-11-11
Filing date: 2019-11-11
Publication date: 2024-04-09
Anticipated expiration: 2039-11-11
Also published as: CN111008859A

Abstract

The application provides a method and device for presenting information in a virtual store, electronic equipment and a storage medium. The method comprises the following steps: acquiring data acquired by data acquisition equipment arranged in an entity store; determining target virtual equipment in a virtual store, wherein a mapping relationship exists between the target virtual equipment and entity equipment in the acquisition range of the data acquisition equipment; and obtaining display information according to the data, and displaying the display information around the target virtual equipment. The virtual store dynamically reflects the operation state of the physical store in real time by establishing the data mapping relation between the virtual store on the line and the physical store under the line, and compared with the prior art, the virtual store can collect and analyze various information in the physical store manually, can automatically collect and present various information in the physical store to the virtual store, improves the management efficiency of the physical store, avoids errors in manual operation and improves the quality of store management.

Description

Information presentation method and device in virtual store, electronic equipment and storage medium

Technical Field

The present invention relates to the field of information processing technologies, and in particular, to a method and apparatus for presenting information in a virtual store, an electronic device, and a storage medium.

Background

Nowadays, online electronic commerce is rapidly developed, great convenience is brought to daily life of people, meanwhile, offline physical stores are subjected to strong impact, and lack of an efficient operation mode for the physical stores is one of important reasons for causing the offline physical stores to suffer from the bottleneck.

Taking convenience stores in retail field as an example, the related art mainly manages the convenience stores through experience and subjective deduction of operators, however, there are a plurality of defects in this management manner: firstly, management efficiency is low, for example, operators can actively visit the shops to check possible faults of equipment, so that the faults can be removed, the shops can be managed, and although some physical shops are internally provided with data acquisition equipment for acquiring data, the acquired data are manually processed, for example, the operators can manually derive the data from the data acquisition equipment and carry out arrangement to form various forms or two-dimensional drawings and the like; secondly, the management quality is low, for example, when an operator analyzes various forms or two-dimensional drawings obtained by arrangement, required information is required to be obtained from massive forms or two-dimensional drawings, the operation is complex, errors are easy to occur, and objective and effective operation decisions cannot be finally obtained. Therefore, there is an urgent need in the related art for a way to more effectively manage physical stores.

Disclosure of Invention

The embodiment of the application provides a method, a device, electronic equipment and a storage medium for presenting information in a virtual store, and aims to provide a more effective way for managing an entity store.

An embodiment of the present application provides a method for presenting information in a virtual store, where the method includes:

acquiring data acquired by data acquisition equipment arranged in an entity store;

determining target virtual equipment in a virtual store, wherein a mapping relationship exists between the target virtual equipment and entity equipment in the acquisition range of the data acquisition equipment;

and obtaining display information according to the data, and displaying the display information around the target virtual equipment.

Optionally, after displaying the display information around the target virtual device, the method further comprises:

detecting a switching operation of a user for the display;

and when the switching operation is detected, displaying the real-time monitoring video stream of the entity equipment mapped by the target virtual equipment.

judging whether the data accords with the alarm condition of the entity equipment mapped by the target virtual equipment or not;

Generating an alarm message when the data accords with the alarm condition;

and displaying the alarm message around the target virtual equipment and/or sending the alarm message to a message display terminal arranged in the entity store.

Optionally, the method further comprises:

storing historical data acquired by the data acquisition equipment within a preset time period;

obtaining display information according to the data and displaying the display information around the target virtual device, wherein the method comprises the following steps:

analyzing the historical data to obtain customer information and goods information;

displaying the customer information and the item information around the target virtual device;

wherein the item information includes at least one of: goods placement information, goods transaction information, man-goods interaction information and goods historical sales information; the customer information includes at least one of: store customer portrait, store information, in-store passenger flow movement track information, and in-store passenger flow thermodynamic diagrams.

Optionally, the method further comprises: constructing a knowledge graph based on the customer information and the goods information;

generating an operation prompt according to the knowledge graph and the real-time data acquired by the data acquisition equipment at the current moment;

And displaying the operation prompt around a virtual shelf corresponding to the physical shelf in the physical store, and/or sending the operation prompt to a message display terminal arranged in the physical store.

Optionally, the types of the operation prompts are multiple, and when the data meets the alarm condition, generating an alarm message includes:

determining the type of the operation prompt needing alarm processing in a plurality of operation prompts in advance;

when an operation prompt conforming to the type is generated, an alarm message is generated for the operation prompt.

A second aspect of the embodiments of the present application provides an information presentation apparatus in a virtual store, the apparatus including:

the acquisition module is used for acquiring data acquired by data acquisition equipment arranged in the physical store;

the determining module is used for determining target virtual equipment in the virtual store, wherein a mapping relation exists between the target virtual equipment and entity equipment in the acquisition range of the data acquisition equipment;

and the data display module is used for obtaining display information according to the data and displaying the display information around the target virtual equipment.

Optionally, the apparatus further comprises:

The detection module is used for detecting switching operation of a user on the display information;

and the video display module is used for displaying the real-time monitoring video stream of the entity equipment mapped by the target virtual equipment when the switching operation is detected.

Optionally, the apparatus further comprises:

the judging module is used for judging whether the data accords with the alarm condition of the entity equipment mapped by the target virtual equipment;

the alarm generation module is used for generating an alarm message when the data accords with the alarm condition;

and the alarm display module is used for displaying the alarm message around the target virtual equipment and/or sending the alarm message to a message display terminal arranged in the physical store.

Optionally, the apparatus further comprises:

the storage module is used for storing the historical data acquired by the data acquisition equipment in a preset time period;

the data display module includes:

the analysis module is used for analyzing the historical data to obtain customer information and goods information;

a data display sub-module for displaying the customer information and the item information around the target virtual device;

Optionally, the apparatus further comprises:

the construction module is used for constructing a knowledge graph based on the customer information and the goods information;

the prompt generation module is used for generating an operation prompt according to the knowledge graph and the real-time data acquired by the data acquisition equipment at the current moment;

and the prompt display module is used for displaying the operation prompt around a virtual shelf corresponding to the physical shelf in the physical store and/or sending the operation prompt to a message display terminal arranged in the physical store.

Optionally, the alarm generating module includes:

the alarm generation sub-module is used for determining the type of the operation prompt needing alarm processing in various operation prompts in advance; and

A third aspect of the embodiments of the present application provides an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, which when executed implements the steps of the method described in the first aspect of the present application.

A fourth aspect of the embodiments provides a computer readable storage medium having stored thereon a computer program which when executed by a processor performs steps in a method as described in the first aspect of the application.

The embodiment of the application provides a method for presenting information in a virtual store, which comprises the steps of firstly acquiring data through data acquisition equipment in an off-line physical store, acquiring data display information according to the acquired data after a user determines a target virtual device in the virtual store, and displaying the display information around the target virtual device. According to the method, the data mapping relation between the online virtual store and the offline physical store is established, so that the virtual store can dynamically reflect the operation state of the physical store in real time, further, an online manager can make operation decisions in time and instruct operators to process the information, compared with the method that multiple information in the physical store is manually collected and analyzed in the related art, the method can automatically collect multiple information in the physical store and present the information in the virtual store, management efficiency of the physical store is improved, errors occurring during manual operation are avoided, quality of store management is improved, and in addition, the method does not need to be used for on-site supervision and guidance of the physical store in person, so that convenience of store management is improved.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the description of the embodiments of the present application will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a schematic diagram of an implementation environment shown in an embodiment of the present application;

FIG. 2 is a flow chart of a method of information presentation in a virtual store according to an embodiment of the present application;

FIG. 3 is a flow chart illustrating a method of video display according to an embodiment of the present application;

FIG. 4 is a flow chart illustrating a method of generating an alert prompt according to an embodiment of the present application;

FIG. 5 is a flow chart illustrating a method of generating an operation hint according to an embodiment of the present application;

fig. 6 is a block diagram showing a configuration of an information presentation apparatus in a virtual store according to an embodiment of the present application;

fig. 7 is a schematic diagram of an electronic device according to an embodiment of the present application.

Detailed Description

The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all, of the embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments herein without making any inventive effort, are intended to be within the scope of the present application.

FIG. 1 is a schematic diagram of an implementation environment according to an embodiment of the present application. Referring to fig. 1, a server is communicatively connected to a data collection device and a terminal, respectively, wherein the data collection device is disposed in an off-line physical store, and the terminal is provided with an application interface for displaying a virtual store to receive an operation command input by an on-line manager.

To overcome the problems in the related art, the present application provides an information presentation method, which is applied to the server in fig. 1. Fig. 2 is a flowchart of a method for presenting information in a virtual store according to an embodiment of the present application. Referring to fig. 2, the information presentation method of the present application includes the steps of:

step S11: and obtaining data acquired by data acquisition equipment arranged in the physical store.

In this embodiment, the data acquisition device in the physical store acquires various data in the store in real time, and transmits the data to the server through the network, and the server is used for storing the data transmitted by the data acquisition device, maintaining the virtual store corresponding to the physical store, and responding to the operation command transmitted by the online manager through the virtual store.

The virtual store is a digital store corresponding to an off-line physical store generated by a digital twin technique (digital twin technique is a technique of reconstructing all elements of parameters acquired in a physical world in an information space to form a twin world having a perceptive ability, an analysis ability, and a predictive ability).

In each embodiment of the application, each virtual store is obtained by performing model mapping on an offline physical store, and a data acquisition device is arranged in the offline physical store, and the virtual store obtained by mapping the physical store model is completely consistent with the internal structure of the physical store, the layout of the device and the layout of the device. Wherein, data acquisition equipment includes: sensors (such as temperature sensors, humidity sensors, sound sensors, power sensors and other sensors with various detection functions), cameras (cameras with various functions such as monitoring cameras and face recognition cameras) and other IOT (Internet of Things ) devices which are set according to actual service requirements. Therefore, the data that can be acquired by the data acquisition device of the present application includes: environmental parameters (e.g., air temperature, humidity, noise decibel values, etc.), equipment parameters (equipment temperature, equipment power consumption, etc.), surveillance video streams, etc. during operation of the physical store. Other IOT devices besides the above-mentioned sensors and cameras may also be deployed in the physical store to collect data required in the actual business, which is not particularly limited in this application.

Step S12: and determining target virtual equipment in the virtual store, wherein a mapping relation exists between the target virtual equipment and entity equipment in the acquisition range of the data acquisition equipment.

In this embodiment, the virtual store is a spatial model, and an online manager can view and operate the virtual store in 2D or 3D mode through a terminal-installed application that exposes the virtual store. Virtual shops are built from various types of physical models, including, for example: store hardware, store equipment, and IOT equipment, wherein the store hardware may be: walls, windows, stairs, steps, beams, etc.; the store facility may be: goods shelves, tables, chairs, sofas, counters, air conditioners, refrigerators, etc.; IOT devices may be: sensors, cameras, etc. (IOT devices are described with particular reference to the foregoing).

In this embodiment, the mapping relationship between the target virtual device and the entity device within the collection range of the data collection device is a data mapping relationship, where the data mapping is different from the model mapping described above. For example, if only the sensor a model in the physical store is mapped into the virtual store, the sensor a ' in the virtual store cannot display the data detected by the sensor a in the physical store, but if on the basis, the sensor a data in the physical store is mapped into the virtual store, that is, the data mapping relationship between the sensor a in the physical store and the sensor a ' in the virtual store is established, the sensor a ' in the virtual store may also display the data detected by the sensor a in the physical store in real time.

In an actual implementation scenario, the acquisition range of the data acquisition device may cover both part of the entity devices and all of the entity devices. For example, taking the physical equipment in the physical store as a shelf and taking the data acquisition equipment as a monitoring camera as an example, the monitoring camera can be arranged in a designated area to acquire a monitoring video stream in an area where part of the shelves are located or a monitoring video stream in an area where all the shelves are located, and in the case, the shelves in the area which are not monitored by the monitoring camera cannot complete data mapping with the virtual equipment in the virtual store.

The target virtual device is any one of the virtual devices which establish a data mapping relation with the entity device. For example, when the shelf a in the virtual shop is in the area monitored by the monitoring camera X, the shelf X may be the target virtual device, and when the shelf a is not in the area monitored by the monitoring camera X, the shelf X may not be the target virtual device.

In this embodiment, the user may load the target virtual store through the application installed at the terminal that displays the virtual store, and operate the target virtual store through the application interface provided by the application. In various embodiments of the present application, the user is an online manager on the terminal side that operates the virtual store, and is configured to issue a specific command to an offline operator to instruct the operation of the online manager, which is opposite to the concept of the offline operator in the physical store, and for convenience in describing various embodiments of the present application, all users mentioned in the present application refer to the online manager. Of course, the user mentioned in the present application may be an off-line operator, and the off-line operator may better manage the physical store through the virtual store, which is not specifically limited in the present application.

The application is to determine a target virtual device in a virtual store according to a user operation. The user operation may be triggered by a mouse, a gesture, or other manners, which is not particularly limited in this application. By way of example, the manner of triggering by the mouse may be: when the mouse is moved, the virtual device pointed by hovering the cursor is determined to be the target virtual device; or when the mouse is clicked, determining the virtual device pointed by the clicking as the target virtual device. As another example, the manner triggered by the gesture may be: when the screen is pressed for a long time, determining virtual equipment pointed by the long-time pressing area of the finger as target virtual equipment; or when the screen is double-clicked, the virtual device pointed by the double-clicked finger is determined as the target virtual device.

Step S13: and obtaining display information according to the data, and displaying the display information around the target virtual equipment.

In this embodiment, the display information includes collected data and/or data obtained by processing the collected data, where the processed data may be, for example: and analyzing or predicting the acquired data. When a user operates the virtual store through the terminal application interface, the terminal automatically detects the triggered operation, generates an operation command according to the operation, and then sends the operation command to the server. After receiving an operation command sent by a terminal, the server stores data acquired by the data acquisition equipment in real time, if the data requested by the operation command is the acquired data, the acquired data is returned to the terminal, and if the data requested by the operation command is the processed data, the processed data of the acquired data is returned to the terminal, so that the terminal displays display information obtained according to the acquired data to a user, so that the user can make an operation decision according to the returned display information, and further an operator under a guide line can conduct corresponding operation on an entity shop.

The specific process of the above step S13 may be, for example: the user determines a target virtual device M through a preset operation; the terminal displays a menu list around the target virtual device M, wherein the menu list can comprise a plurality of menu items such as detailed information viewing, real-time monitoring video stream calling, returning, helping and the like; when the user selects a menu item for viewing detailed information, the terminal generates an operation command corresponding to the operation (the operation command includes detailed event information of the operation, such as time, user name, operation content, etc.), and then transmits the operation command to the server; after receiving the operation command, the server analyzes the operation content in the operation command, acquires the data required by the operation content, and returns the data to the terminal; the terminal presents the returned data around the target virtual device M for viewing by the user.

In one embodiment, the user checks the temperature of the shelf a through the virtual shop, and then according to the above steps S11-S13, the specific procedure of this operation may be: the user loads a virtual shop in the terminal, selects the shelf A through a mouse (or a gesture), selects a menu item for checking detailed information in the pop-up menu list, waits for a response of the server, and checks the detailed information about the shelf A around the shelf A if the response of the server is received, wherein the detailed information can be: "shelf A: the current temperature is 20 degrees celsius. If the user wants to view the real-time monitoring video stream about the shelf A, the user only needs to choose to call the real-time monitoring video stream in the menu list, and the principle is similar to that of viewing the detailed information.

The embodiment of the application provides an information presentation method in a virtual store, which comprises the steps of firstly acquiring data through data acquisition equipment in an off-line physical store, and displaying display information around a target virtual device after a user determines the target virtual device in the virtual store. According to the information presentation method, the data mapping relation between the virtual store on the line and the physical store below the line is established, so that the virtual store can dynamically reflect the operation state of the physical store in real time, further, users can make operation decisions in time and instruct operators to process the information, compared with the method that multiple information in the physical store is manually collected and analyzed in the related art, the information presentation method can automatically collect multiple information in the physical store and present the multiple information in the virtual store, management efficiency of the physical store is improved, errors occurring during manual operation are avoided, quality of store management is improved, and in addition, users do not need to personally monitor and guide the physical store on site, so that convenience of store management is improved.

The information presentation method provided by the application not only can display the detailed information of the target virtual equipment to the user according to the operation of the user, but also can display the video information corresponding to the detailed information to the user, as shown in fig. 3. Fig. 3 is a flow chart illustrating a method of video display according to an embodiment of the present application. Referring to fig. 3, the method for displaying video provided in the present application includes the following steps:

Step S21: detecting switching operation of a user on the number display information;

step S22: and when the switching operation is detected, displaying the real-time monitoring video stream of the entity equipment mapped by the target virtual equipment.

In combination with the above embodiment, in another implementation, the terminal further provides a video stream switching function to the user when presenting the detailed information of the target virtual device to the user. Through the video stream switching function, a user can make an operation decision by combining the real-time monitoring video stream corresponding to the detailed information while viewing the detailed information of the target virtual device.

For example, when a piece of detailed information about the shelf a displayed to the user in the display page is "shelf a, and the current temperature is 38 degrees celsius", a switch button for viewing the real-time monitoring video stream is also provided in the display page, after the user views the detailed information, the user considers that the temperature of the shelf a is abnormal and may have a fault, so that the switch button for viewing the real-time monitoring video stream in the display page can be clicked, if the clicking operation of the switch button by the user is detected by the terminal, a window is additionally added in the current display page to display the real-time monitoring video stream corresponding to the shelf a, so that the user can further determine whether the shelf a has a fault in combination with the actual scene of the store under the line, and if the fault is finally determined, the user further instructs the operator under the line to process the shelf a.

According to the embodiment of the application, the function of checking the real-time monitoring video stream is set for each target virtual device, so that a user can make a decision according to displayed digital information, and the digital information and the real-time monitoring video stream can be combined, analyzed and made, so that the quality of management of an entity store is further improved, and the use experience of the user is optimized.

The information presentation method provided by the application not only can display the detailed information of the target virtual equipment and the corresponding real-time monitoring video stream to the user according to the operation of the user, but also can generate an alarm message according to the acquired data, as shown in fig. 4. FIG. 4 is a flow chart illustrating a method of generating an alert prompt in accordance with an embodiment of the present application. Referring to fig. 4, the method for generating an alarm prompt of the present application includes the following steps:

step S31: and judging whether the data accords with the alarm condition of the entity equipment mapped by the target virtual equipment.

In this embodiment, the data mainly refers to data directly collected by the sensor. The server sets alarm conditions for the data collected by each sensor in advance, for example, for a temperature sensor, the alarm conditions can be that alarm information is sent when the temperature is greater than (or less than) a preset threshold value; for another example, for an acoustic sensor, the alarm condition may be an alarm message when the decibel value is greater than (or less than) a preset threshold.

Of course, the data may also be a surveillance video stream collected by a camera, and the alarm message may be obtained by processing an image extracted from the surveillance video stream, and the process of processing the surveillance video stream and obtaining the alarm message will be described in detail below.

Step S32: and generating an alarm message when the data accords with the alarm condition.

Step S33: and displaying the alarm message around the target virtual equipment and/or sending the alarm message to a message display terminal arranged in the entity store.

In this embodiment, the alarm message belongs to one of the data types obtained after the collected data is processed. After the server acquires the data acquired by the data acquisition equipment, the server stores the data on one hand, analyzes the data on the other hand, judges whether the data meets preset alarm conditions, generates alarm information if the data meets the preset alarm conditions, displays the alarm information to the periphery of target virtual equipment aimed by the alarm information, and/or displays the alarm information to a message display terminal specially arranged in an off-line entity store for timely processing by an off-line operator.

For example, when the server obtains the current temperature of 45 ℃ of the shelf A, comparing the temperature with the preset normal temperature of 25 ℃, and determining that the temperature of the shelf A exceeds the preset normal temperature, thereby determining that the shelf A is abnormal; the server generates alarm information, for example, the alarm information can be "the current temperature of the shelf A is 45 ℃ and the temperature is too high, please process-! "; after the alarm information is generated, the server can send the alarm information to the terminal so as to be displayed around the goods shelf A, can send the alarm information to a message display terminal specially arranged in an off-line physical store, and can also send the alarm information to the terminal and the message display terminal in the physical store at the same time so as to facilitate off-line operators to check whether the goods shelf A is abnormal or not in time.

In the application, when the alarm information is displayed around the target virtual device or on the message display terminal, various other functions for assisting the user in viewing can be further set, for example, a mark is set in an area corresponding to the alarm information. To show that "the current temperature of the shelf A is 45 ℃ and the temperature is too high, please process-! For example, the storage rack a may be marked in a virtual store, so that the user can see the storage rack in time, and the marking modes may be various, for example: text marks, graphic marks, color marks, etc., are added, which are not particularly limited in this application.

According to the method and the device for analyzing the data in the physical store, the data collected in the physical store can be automatically analyzed, when an analysis result shows that an abnormal condition occurs in the physical store, an alarm message is generated for the abnormal condition, and the alarm message is actively displayed to the terminal or a message display terminal in the physical store, so that an online manager or an offline operator is reminded of timely taking corresponding measures to process possible abnormal events, possible loss caused by negligence of the operator is avoided, and the capability of the offline physical store for countering risks is remarkably improved.

The information presentation method provided by the application can also conduct prediction analysis on the collected data to generate customer information and goods information, help users make decisions, and help the users manage physical stores. The information presentation method provided by the application further comprises the following steps:

and storing historical data acquired by the data acquisition equipment within a preset time period.

On this basis, the step S13 may include: analyzing the historical data to obtain customer information and goods information;

In this embodiment, the customer information and the item information also belong to the data type obtained by processing the collected data. Historical data is data acquired by data acquisition equipment. The data collected by the data collection device can be data detected by a sensor, a monitoring video stream collected by a camera and the like.

In this embodiment, the analysis of the history data to obtain customer information and goods information is specifically: and analyzing the monitoring video stream in the historical data and the historical operation data acquired by the non-data acquisition equipment to obtain customer information and goods information.

By analyzing the historical operation data, the commodity transaction information (such as the total annual/monthly/daily commodity transaction amount, the annual/monthly/daily commodity transaction amount of each type of commodity) and the commodity historical sales information (such as the annual/monthly/daily commodity sales total amount, the annual/monthly/daily commodity sales amount of each type of commodity) can be obtained, and the commodity placement information, the man-cargo interaction information, the store customer image, the store information, the store movement track information, the store passenger flow thermodynamic diagram and the like can be obtained by analyzing the monitoring video stream acquired by the monitoring camera. Specifically, the article placing information includes: the goods placing rate, whether the goods are placed neatly, whether the goods placing positions are reasonable, and the like. The goods placing rate refers to: the ratio of the number of the goods placed on the same layer of goods shelf to the number of the goods which can be placed; whether goods are put neatly refers to: whether the goods are placed according to a specified style, such as whether a side placing phenomenon exists for bottled or canned goods, etc.; whether the goods placement position is reasonable refers to: whether the category to which the goods belong should be placed in the shelf to which the goods currently belong.

The goods placement information can be obtained through analysis of the monitoring video stream, specifically: taking the article M placed on the 3 rd layer of the article M as an example, firstly acquiring a monitoring video stream corresponding to the article M placed on the 3 rd layer of the article a, extracting a plurality of frames of video images, inputting the video images into a feature extraction model, obtaining image features (the image features comprise at least one sub-image feature, in the application, one sub-image feature corresponds to one article), comparing the image features with the image features corresponding to the article M, determining the number of sub-image features with the similarity of the image features corresponding to the article M being greater than a preset threshold, wherein the number is the number of the articles M placed on the 3 rd layer of the article a, and comparing the number with the number of the articles placed on the 3 rd layer of the article a stored in a database, thus obtaining the article placing rate of the 3 rd layer of the article a. Taking the article M placed on the 3 rd layer of the goods shelf A as an example, firstly acquiring a monitoring video stream corresponding to the article M on the 3 rd layer of the goods shelf A, extracting a plurality of frames of video images, inputting the video images into a feature extraction model to obtain image features, comparing the image features with the image features corresponding to the article M, identifying sub-image features with similarity larger than a preset threshold value, and comparing the identified sub-image features with the image features corresponding to the article M with the similarity not larger than the preset threshold value to obtain the article with irregular placement. For whether the goods placement position is reasonable or not, taking the goods M placed on the 3 rd layer of the goods shelf A as an example, firstly acquiring a monitoring video stream corresponding to the goods M on the 3 rd layer of the goods shelf A, extracting a plurality of frames of video images, inputting the video images into a feature extraction model to obtain image features, comparing the image features with the image features corresponding to the goods M, and if sub-image features with the similarity of the image features corresponding to the goods M not larger than a preset threshold value exist, and the similarity of the sub-image features corresponding to the goods N (the goods N and the goods M are different types of goods) is not larger than the preset threshold value, determining that other types of goods exist in the goods M placed on the 3 rd layer of the goods shelf A, namely, the goods placement position is unreasonable in the 3 rd layer of the goods shelf A.

The man-cargo interaction information refers to: the customer removes the goods from the shelves and information about the goods involved in the process of returning the goods to the shelves. For example: the type of the article removed, whether the article is returned, the length of time the article is checked if returned, and the like. The specific implementation process for extracting the man-cargo interaction information through the monitoring video stream comprises the following steps: and periodically extracting multi-frame images of the goods shelf, identifying the missing goods through the comparison of the multi-frame images, determining the missing goods as the goods taken down by the customer, determining that the goods exist in the images again through the comparison of the images if the goods are within the specified time period, determining that the goods are put back into the goods shelf by the customer, and determining that the customer purchases the goods if the goods are not detected to be put back into the goods shelf by the customer beyond the specified time period.

The store customer portrait refers to: according to the historical consumption record of the customer in the physical store, the label added to the physical store can be: the preferred price range, the type of items purchased more frequently, the frequency of arrival at the store, etc. The concrete implementation process of extracting the store customer portrait comprises the following steps: and arranging a face recognition camera at an entrance or a cash desk of the physical store, and recognizing face information of a customer through the face recognition camera, so as to acquire a historical consumption record of the customer in the physical store and further acquire a label of the customer.

The customer flow-through store information refers to: information of customers passing through the physical store, for example: the number of customers in the year/month/day, the sex of the customers, whether it is an adult or a child, etc. The specific implementation process for acquiring the information of the passenger flow through the store is as follows: and installing a monitoring camera outside the physical store, extracting multi-frame images from a monitoring video stream acquired by the monitoring camera, and analyzing the character images in the multi-frame images to obtain the information of the passenger flowing through the store.

The customer flow to store information refers to: information of customers entering into the physical store, for example: the number of customers entering the store in the year/month/day, the sex of the customers, whether they are adults or children, the ratio of the number of customers actually arriving at the store to the number of customers passing through the store, and the like. The specific implementation process for acquiring the information of the passenger flow to the store is as follows: and installing a monitoring camera at the entrance of the physical store, extracting multi-frame images from a monitoring video stream acquired by the monitoring camera, and analyzing the character images in the multi-frame images to obtain the information of the passenger flowing through the store.

The in-store passenger flow movement track information refers to: a map of the movement of a customer within a physical store. The specific implementation process for extracting the passenger flow movement track information in the store through the monitoring video flow comprises the following steps: and extracting a plurality of images from the monitoring video stream according to the time sequence, analyzing the character images in the plurality of images, determining the actual coordinates of each customer in the physical store (the application establishes a coordinate system for the physical store in advance), and finally generating an in-store movement track graph of each customer according to the change of the actual coordinates of each customer.

In-store passenger flow thermodynamic diagrams refer to: in the physical store, when the number of customers with residence time exceeding the preset time in a certain area is larger than the preset number, the area is determined as a hot area. The specific implementation process for acquiring the in-store passenger flow thermodynamic diagram comprises the following steps: face recognition cameras are arranged in all areas in a store, the number of times of faces recognized by all areas in a specified time period is counted, the area with the large number of times of faces recognized can be usually determined as a hot area, and the face recognition times of all areas are integrated to obtain a passenger flow thermodynamic diagram in the store.

According to the embodiment of the application, the collected data are further integrated to obtain the customer information and the goods information, the problem that the collected data are lack of effective analysis, association and reasoning when the data collection equipment is only used for monitoring or displaying in the related technology is solved, the deep mining of the data is realized, the integration degree of the information is improved, and effective technical support is provided for realizing efficient operation of the entity store.

In this embodiment, customer information and goods information may be further predicted and analyzed to obtain an operation prompt. Fig. 5 is a flowchart illustrating a method of generating an operation hint according to an embodiment of the present application. Referring to fig. 5, specifically, after obtaining the customer information and the item information, the following steps may be further performed:

Step S41: and constructing a knowledge graph based on the customer information and the goods information.

According to the embodiment of the application, the knowledge graph technology is adopted, the integrally obtained customer information and goods information are learned, the characteristics of the knowledge graph are fully utilized, and a reasonable operation prompt is obtained for the prediction of the physical store.

Step S42: and generating an operation prompt according to the knowledge graph and the real-time data acquired by the data acquisition equipment at the current moment.

Through the constructed knowledge graph, the server can generate operation prompts according to the real-time data acquired by the data acquisition equipment at the current moment and the customer information and the goods information which are obtained by integrating the acquired real-time data, and the types of the operation prompts are various, and the operation prompts of various types are described below.

First, the backorder suggestion, the acquisition mode is: acquiring a monitoring video stream of a goods shelf, extracting multi-frame images from the monitoring video stream, identifying the number of goods in the images, solving the ratio of the number to the preset number, and generating a backorder prompt when the ratio is larger than the preset ratio. In particular implementations, it may be determined whether to generate an out-of-stock cue based on the item placement rate described above, for example: setting the preset threshold value to be 80%, when the goods placement rate of a certain layer of goods shelf is greater than 80%, determining that the out-of-stock prompt for the layer of goods shelf should be generated, and when the goods placement rate of the layer of goods shelf is not greater than 80%, determining that the out-of-stock prompt for the layer of goods shelf should not be generated.

Secondly, the cargo handling prompt is obtained by the following steps: and acquiring a monitoring video stream of the goods shelf, extracting a multi-frame image from the monitoring video stream, identifying the surface arrangement irregularity of goods in the image, and generating a goods sorting prompt when the surface arrangement irregularity is larger than a preset ratio. In a specific implementation process, whether to generate the tally prompt may be determined according to the related information about whether the goods are placed in order and/or whether the goods are placed in a reasonable position, for example: the goods sorting prompt can be generated when the phenomenon that goods are not placed in a certain layer of goods shelves is determined, and/or the goods sorting prompt can be generated when the phenomenon that the categories of the goods in the certain layer of goods shelves are not in the categories of the goods which should be placed in the layer of goods shelves is determined.

Thirdly, a shelf layout prompt is obtained in the following way: and analyzing the in-store passenger flow movement track information and the in-store passenger flow thermodynamic diagram, and automatically planning a path by adopting an algorithm to obtain various shelf layout diagrams.

Fourth, recommending goods prompts, the obtaining mode is: the recommended item prompt is generated by analyzing one or more of a plurality of information such as item delivery information, person-to-item interaction information, item history sales information, store customer image, customer flow-through store information, customer flow-to-store information, and the like.

Of course, other types of operation prompts can be generated by using the customer information and the goods information, so that the purpose of providing operation suggestions for users by fully using the customer information and the goods information is achieved, and the operation suggestions are not listed here.

Step S43: and displaying the operation prompt around a virtual shelf corresponding to the physical shelf in the physical store, and/or sending the operation prompt to a message display terminal arranged in the physical store.

In this application, the operation prompt also belongs to the data type obtained after the collected data is processed. Various other functions for assisting the user in viewing can also be set when the operation prompt is displayed, for example: and adding a mark in the region corresponding to the operation prompt. Taking the example of displaying a tally cue around a virtual shelf, a label may be added to a specific location in the virtual shelf where a tally is required, for example: adding a red exclamation mark at a specific position of the 3 rd layer of the virtual shelf, which is required to be tidied, so as to prompt a user that the position is required to be tidied; taking the example of displaying out-of-stock cues around a virtual shelf, markers may be added to a partial area of the virtual shelf, such as: the out-of-stock area of layer 3 of the virtual shelf is set to a yellow background to alert the user that the location requires restocking. After various types of operation prompts are generated, the operation prompts can be sent to the terminal, and the operation prompts can be sent to the information display terminal distributed in the physical store, so that an online manager or an offline operator can be reminded of timely responding to the operation prompts. When the operation prompt is displayed in the operation terminal, various functions for assisting the user in viewing can be set, and the above description is specifically referred to.

In the embodiment of the application, under the support of a knowledge graph technology, through the deep analysis of customer information and goods information, a plurality of store operation prompts which cannot be directly obtained manually (such as the types of customers attracted by stores with high probability, the yield of goods, peak time, the goods which are not supplied enough in the peak time, the rationality of goods display, whether the services of operators are round, the attraction of promotion advertisements and the like) can be predicted, the limitation that the operation of the entity stores in the related technology stays in the visible range of the operators all the time (such as the types of goods purchased by the customers, the types of goods with good sales volume, whether the sales volume is increased and the like) is broken through, the optimization is carried out on the entity stores from the fields, the goods and the people, more real and effective operation suggestions are provided for users, the problem that the operation decisions caused by the experience and subjective inference of the operators are not accurate in the related technology is overcome, and the competitive power of the entity stores is remarkably improved.

Accordingly, step S32 may specifically include:

In this embodiment, since the server may learn the historical data to predict multiple types of operation prompts, the present application further provides a function of custom alarm processing. The user can preset operation prompts needing automatic alarm processing through a terminal application interface according to the requirements of an entity store, when the server predicts the operation prompts, the operation prompts are actively displayed in an alarm processing mode so that the user can make decisions in time, in other words, the operation prompts set to be used for automatic alarm processing have higher display priority than the operation prompts not set to be used for automatic alarm (the priority can be shown as prompt sound or lamplight flickering and the like during display), and of course, the user can cancel the automatic alarm function of any operation prompt at any time.

When the collected data represent that the electric equipment in the physical store fails, an equipment maintenance prompt is generated, and meanwhile, the equipment maintenance prompt is actively displayed to a terminal and/or a message display terminal arranged in the physical store in an alarm processing mode.

According to the embodiment, alarm processing can be set for various operation prompts predicted by the server in a targeted manner, so that a user can respond to important operation prompts preferentially, and the flexibility of management of offline entity stores is improved.

The information presentation method in the virtual store can be applied to various types of off-line physical stores. For example, the method is applied to convenience stores, and in specific application, the convenience stores are mapped into virtual stores, and then data acquired by data acquisition equipment (such as monitoring cameras, various sensors and the like) in the convenience stores are acquired in real time and stored in a server. When a user selects a certain target virtual device in a virtual store of a convenience store (the virtual device and corresponding entity devices in the convenience store have a data mapping relation), the server automatically displays information of the target virtual device around the target virtual device, wherein the information can be data (such as air temperature, humidity, noise decibel values, monitoring video streams and the like) acquired by the data acquisition device in real time, or can be data (including alarm information, customer information, goods information, operation prompts and the like) obtained by processing the data, so that the user can be helped to make effective operation decisions, and the management efficiency of the convenience store is further improved.

The application provides a virtual store constructed by a knowledge graph, a model ontology definition (namely mapping an entity store into a virtual store) and a big data analysis system, wherein effective information of a field, goods and people is analyzed based on a digital twin technology, and data in the entity store and an operation prompt obtained by mining according to the data are presented in the virtual store in a virtual and reality combined mode, so that efficient store management is realized, and the method has the following effects:

1. In the presentation of fields and devices

The mapping of the virtual shops in the virtual space fully utilizes the physical models of all the equipment, integrates the multi-disciplinary, multi-physical quantity and multi-scale simulation shops, realizes a plurality of functions such as real-time monitoring, 3D (three-dimensional) tour, prediction judgment, equipment alarm and the like by utilizing the linkage with the real world, and can freely switch the virtual shops from multiple views to the entity shops under the real-time monitoring, so that the information presentation is more visual and the reliability is higher.

2. In the presentation of goods

The method can accurately adjust the goods operation mode of the physical store through the change of the climate environment, the surrounding natural data, the historical operation data and the knowledge graph constructed by the pictures of the store customers, not only can provide goods display suggestions, but also can present different prompts on the corresponding virtual shelves of the 3D virtual store when the goods are out of specification, the goods are out of stock and the products are about to expire.

3. In the presentation of guest groups

Through the cooperation of various sensors and cameras, accurate comprehensive acquisition of passenger flow information, in-store passenger flow movement track information, in-store passenger flow thermodynamic diagrams and the like, the problem of the relation between passenger flow and spatial field and equipment facility is visually represented through spatial presentation, and effective operation suggestions can be provided for goods shelf layout and hot-plug goods display.

It should be noted that, for simplicity of description, the method embodiments are shown as a series of acts, but it should be understood by those skilled in the art that the embodiments are not limited by the order of acts, as some steps may occur in other orders or concurrently in accordance with the embodiments. Further, those skilled in the art will appreciate that the embodiments described in the specification are presently preferred embodiments, and that the acts are not necessarily required by the embodiments of the invention.

Based on the same inventive concept, the present application also provides an information presentation apparatus 600 in a virtual store, as shown in fig. 6. Fig. 6 is a block diagram showing a configuration of an information presentation apparatus according to an embodiment of the present application. Referring to fig. 6, the in-store information presentation apparatus 600 of the present application includes:

an obtaining module 601, configured to obtain data collected by a data collection device deployed in an entity store;

the determining module 602 is configured to determine a target virtual device in a virtual store, where the target virtual device has a mapping relationship with an entity device within a collection range of the data collection device;

and a data display module 603, configured to obtain display information according to the data, and display the display information around the target virtual device.

Optionally, the information presentation apparatus 600 further includes:

the data display module includes:

Optionally, the apparatus further comprises:

Optionally, the alarm generating module includes:

the alarm generation sub-module is used for determining the type of the operation prompt needing alarm processing in various operation prompts in advance; and generating an alarm message for the operation prompt when the operation prompt conforming to the type is generated.

Based on the same inventive concept, another embodiment of the present application provides an electronic device 700, as shown in fig. 7. Fig. 7 is a schematic diagram of an electronic device according to an embodiment of the present application. The electronic device comprises a memory 702, a processor 701 and a computer program stored on the memory and executable on the processor, which when executed implements the steps of the method according to any of the embodiments described herein.

Based on the same inventive concept, another embodiment of the present application provides a computer readable storage medium, on which a computer program is stored, which program, when being executed by a processor, realizes the steps in the method according to any of the embodiments described in the present application.

For the device embodiments, since they are substantially similar to the method embodiments, the description is relatively simple, and reference is made to the description of the method embodiments for relevant points.

In this specification, each embodiment is described in a progressive manner, and each embodiment is mainly described by differences from other embodiments, and identical and similar parts between the embodiments are all enough to be referred to each other.

It will be apparent to those skilled in the art that embodiments of the present invention may be provided as a method, apparatus, or computer program product. Accordingly, embodiments of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the invention may take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

Embodiments of the present invention are described with reference to flowchart illustrations and/or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following claims be interpreted as including the preferred embodiment and all such alterations and modifications as fall within the scope of the embodiments of the invention.

Finally, it is further noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or terminal device comprising the element.

The method, the device, the electronic equipment and the readable storage medium for presenting information in a virtual store provided by the invention are described in detail, and specific examples are applied to the explanation of the principle and the implementation of the invention, and the explanation of the above examples is only used for helping to understand the method and the core idea of the invention; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in accordance with the ideas of the present invention, the present description should not be construed as limiting the present invention in view of the above.

Claims

1. A method of information presentation in a virtual store, the method comprising:

acquiring data acquired in real time by data acquisition equipment arranged in an entity store; the acquired data includes: environmental parameters, equipment parameters, monitoring video streams and data required in actual business in the operation process of the entity store;

determining target virtual equipment in a virtual store, wherein a real-time dynamic data mapping relation exists between the target virtual equipment and entity equipment in the acquisition range of the data acquisition equipment;

display information is obtained according to the data, and is displayed around the target virtual equipment, wherein the display information is used for a user to make an operation decision of an operator under the guide line in time; the display information comprises collected data, customer information and goods information obtained by analyzing the collected data, and store operation prompts obtained by predicting the customer information and the goods information;

The item information includes at least one of: goods placement information, goods transaction information, man-goods interaction information and goods historical sales information; the customer information includes at least one of: store customer portrait, store information, in-store passenger flow movement track information, and in-store passenger flow thermodynamic diagrams.

2. The method of claim 1, wherein after displaying the display information around the target virtual device, the method further comprises:

detecting switching operation of a user on the display information;

3. The method of claim 1, wherein after displaying the display information around the target virtual device, the method further comprises:

generating an alarm message when the data accords with the alarm condition;

4. A method according to any one of claims 1-3, wherein analyzing the collected data comprises:

and analyzing the historical data to obtain customer information and goods information.

5. The method according to claim 4, wherein the method further comprises:

constructing a knowledge graph based on the customer information and the goods information;

6. The method of claim 5, wherein the type of the operation prompt is a plurality of types, and generating an alert message when the data meets the alert condition comprises:

7. An information presentation apparatus in a virtual store, the apparatus comprising:

the acquisition module is used for acquiring data acquired in real time by data acquisition equipment arranged in the physical store; the acquired data includes: environmental parameters, equipment parameters, monitoring video streams and data required in actual business in the operation process of the entity store;

the determining module is used for determining target virtual equipment in the virtual store, wherein a real-time dynamic data mapping relation exists between the target virtual equipment and entity equipment in the acquisition range of the data acquisition equipment;

the data display module is used for obtaining display information according to the data and displaying the display information around the target virtual equipment, and the display information is used for a user to make an operation decision of an operator under the guide line in time; the display information comprises collected data, customer information and goods information obtained by analyzing the collected data, and store operation prompts obtained by predicting the customer information and the goods information;

8. The apparatus of claim 7, wherein the apparatus further comprises:

9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor performs the steps of the method according to any of claims 1-6.

10. A computer readable storage medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the steps of the method according to any of claims 1-6.