CN116471383A

CN116471383A - Display method and device for unattended store monitoring interface

Info

Publication number: CN116471383A
Application number: CN202310732329.6A
Authority: CN
Inventors: 洪泽威
Original assignee: Shenzhen Zewei Information Technology Co ltd
Current assignee: Shenzhen Zewei Information Technology Co ltd
Priority date: 2023-06-20
Filing date: 2023-06-20
Publication date: 2023-07-21
Anticipated expiration: 2043-06-20
Also published as: CN116471383B; CN117424981A

Abstract

The application discloses a display method and a display device for an unattended store monitoring interface; the scheme can be that a plurality of monitoring interfaces, such as a third monitoring interface related to the moving track of the target object in the store, a fourth monitoring interface related to the moving picture of the target object in the monitoring area and the like, are generated according to the payment checking information sent by the local monitoring equipment, wherein the position and the picture shooting angle corresponding to the moving picture currently displayed by the fourth monitoring interface are marked on the moving track in the third monitoring interface, the marking content can be updated in real time along with the switching of the moving picture, and then the interfaces are displayed in a split screen mode so that a user can execute payment checking operation based on the interfaces; the proposal not only can reduce the monitoring data quantity and display the monitoring interface which can be flexibly and friendly operated so as to improve the processing efficiency, but also can monitor the transaction in real time without sense and improve the transaction safety.

Description

Display method and device for unattended store monitoring interface

Technical Field

The application relates to the field of internet, in particular to a display method and device of an unattended store monitoring interface.

Background

In recent years, unattended shops are increasingly being pursued by people due to the advantages of low labor cost, flexibility in shopping, and the like. The unattended shop refers to a shop which does not need staff to carry out unattended operation and shopping buying order and is completely self-service.

Although the unattended store can greatly reduce the labor cost, the current operation mode is generally completely dependent on the trust of the store to the customer, namely, the customer buying order is completely conscious, so that the conditions of commodity loss, mispaying or missed payment often occur, and the property loss of the store is caused. In order to avoid loss of store property, in the prior art, users often need to review the surveillance video to supervise the situation in an unattended store. However, in the research and practice process of the prior art, the inventor of the present application found that, because the data volume of the existing monitoring video is huge, a great deal of effort and time are required for the user to review, the processing efficiency is extremely low, and the monitoring effect is poor because of the post review, and the situation of commodity loss, mispayment or missed payment still occurs.

Disclosure of Invention

The embodiment of the application provides a display method and a display device for an unattended store monitoring interface, which can not only reduce the data volume of monitoring data and display the monitoring interface which can be flexibly and friendly operated so as to improve the processing efficiency, but also monitor commodity transaction in real time without sense, avoid the conditions of commodity loss, mispayment or missed payment and reduce the property loss of the store.

The monitoring system of the unattended stores comprises local monitoring equipment and remote monitoring equipment, wherein the local monitoring equipment is arranged in the unattended stores, and the remote monitoring equipment is connected with the local monitoring equipment of a plurality of unattended stores; the monitoring method comprises the following steps:

receiving a payment check message sent by a local monitoring device and aiming at a target object through a remote monitoring device, wherein the payment check message carries payment information of the target object, a moving track of the target object in an unattended store and a moving picture of the target object in a monitoring area, and the monitoring area is arranged in front of a container of the unattended store;

generating a first monitoring interface related to the unattended store, a second monitoring interface related to payment information, a third monitoring interface related to a moving track and a fourth monitoring interface related to a moving picture according to the payment check information, wherein the moving track in the third monitoring interface is marked with a position corresponding to the moving picture currently displayed by the fourth monitoring interface and a picture shooting angle, and when the moving picture currently displayed by the fourth monitoring interface is switched, the marking content on the moving track in the third monitoring interface is correspondingly updated;

And carrying out split-screen display on the first monitoring interface, the second monitoring interface, the third monitoring interface and the fourth monitoring interface on the remote monitoring equipment so that a user can execute payment checking operation based on the split-screen displayed first monitoring interface, second monitoring interface, third monitoring interface and fourth monitoring interface.

Correspondingly, the embodiment of the application also provides a display device of the unattended store monitoring interface, which comprises a receiving and transmitting unit, a generating unit and a display unit, and is as follows:

the receiving and transmitting unit is used for receiving a payment check message for a target object sent by the local monitoring equipment, wherein the payment check message carries payment information of the target object, a moving track of the target object in an unattended store and a moving picture of the target object in a monitoring area, and the monitoring area is arranged in front of a container of the unattended store;

the generation unit is used for generating a first monitoring interface related to the unattended store, a second monitoring interface related to payment information, a third monitoring interface related to a moving track and a fourth monitoring interface related to a moving picture according to the payment check information, wherein the moving track in the third monitoring interface is marked with the position corresponding to the moving picture currently displayed by the fourth monitoring interface and a picture shooting angle, and when the moving picture currently displayed by the fourth monitoring interface is switched, the identification content corresponding to the moving track in the third monitoring interface is updated;

And the display unit is used for carrying out split screen display on the first monitoring interface, the second monitoring interface, the third monitoring interface and the fourth monitoring interface so that a user can execute payment check operation based on the split screen displayed first monitoring interface, second monitoring interface, third monitoring interface and fourth monitoring interface.

According to the display method of the unattended store monitoring interface, the remote monitoring equipment can receive the payment check message sent by the local monitoring equipment and aiming at the target object, and according to the payment check message, a first monitoring interface related to the unattended store, a second monitoring interface related to the payment information of the target object, a third monitoring interface related to the moving track of the target object in the unattended store and a fourth monitoring interface related to the moving picture of the target object in a monitoring area (arranged in front of a container of the unattended store) are generated, wherein the moving track in the third monitoring interface is marked with the position corresponding to the moving picture currently displayed by the fourth monitoring interface and the picture shooting angle, when the moving picture currently displayed by the fourth monitoring interface is switched, the identification content correspondence in the moving track in the third monitoring interface is updated, and then the first monitoring interface, the second monitoring interface, the third monitoring interface and the fourth monitoring interface are subjected to split-screen display on the remote monitoring equipment so that a user can execute payment operation based on the first monitoring interface, the second monitoring interface, the third monitoring interface and the fourth monitoring interface; the scheme only monitors the video data, such as the moving track, of the target object in a specific area, such as the moving picture, of the target object in the monitoring area, so that the data size of the monitoring data to be checked can be greatly reduced; in addition, different monitoring interfaces which can be flexibly and friendly operated are generated aiming at different monitoring data, and the relevance among the monitoring interfaces is identified when the monitoring interfaces are displayed in a split screen mode, so that a user can more conveniently check the monitoring data, and the processing efficiency is greatly improved. In addition, the scheme can monitor the transactions in the unattended store in real time without sense, so that the conditions of commodity loss, mispayment or missed payment can be greatly reduced, the store property loss is reduced, and the security of the store transactions is improved.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the description of the embodiments will be briefly introduced below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a schematic view of a scenario of an unattended store monitoring system in an embodiment of the application;

FIG. 2 is an example diagram of an unattended store in an embodiment of the application;

FIG. 3 is a flowchart of a method for displaying an unattended store monitoring interface according to an embodiment of the application;

FIG. 4 is a schematic diagram of an unattended store monitoring interface provided in an embodiment of the application;

FIG. 5 is another schematic illustration of an unattended store monitoring interface provided in an embodiment of the application;

FIG. 6 is another flowchart of a method for displaying an unattended store monitoring interface according to an embodiment of the invention;

fig. 7 is a schematic view of a scenario of a display method of an unattended shop monitoring interface according to an embodiment of the application;

FIG. 8 is an exemplary diagram of an activity trajectory in an embodiment of the present application;

FIG. 9 is an exemplary diagram of a monitoring interface and split screen display in an embodiment of the present application;

FIG. 10 is another exemplary diagram of a monitor interface and split screen display according to an embodiment of the present application;

fig. 11 is an exemplary diagram of a direction display of a C-screen switching screen in the embodiment of the present application;

fig. 12 is an exemplary diagram of a D-screen switching screen displaying direction in the embodiment of the present application;

FIG. 13 is an exemplary diagram of a C-screen click trajectory point in an embodiment of the present application;

fig. 14 is a schematic structural diagram of a display device according to an embodiment of the present application.

Detailed Description

The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all, of the embodiments of the present application. All other embodiments, which can be made by those skilled in the art based on the embodiments herein without making any inventive effort, are intended to be within the scope of the present application.

The embodiment of the application provides a display method and a display device for an unattended store monitoring interface, and the method and the device are respectively described in detail below.

As shown in fig. 1, the unattended store monitoring system may include a remote monitoring device and a local monitoring device, wherein the local monitoring device is disposed in the unattended store. The unattended shop (for short, shop) refers to a shop which does not need staff to watch and buy a shopping list completely self-service. The unattended store is typically provided with a store entrance, a store exit and a container, in front of which a monitoring area is provided.

The local monitoring equipment is used for shooting the monitoring area and all areas in the unattended store to obtain monitoring data (such as monitoring video); the remote monitoring device can be deployed at the cloud end and connected with local monitoring devices of a plurality of unattended stores, for example, the remote monitoring device can be connected and communicated through a wired or wireless network.

For example, referring to FIG. 2, an example of an unattended store is shown. The local monitoring device may include a plurality of cameras, such as a camera 1, a camera 2, a camera 3, a camera 4, a camera 5, a camera 6, a camera 7, … …, a camera n, and the like, and these cameras may be installed at various positions of the unattended store according to the actual application requirements, so as to capture all areas, particularly monitoring areas, inside the unattended store, and obtain monitoring data. The monitoring data may be provided to a remote monitoring device.

The type, the number and the installation position of the cameras can be determined according to the actual application requirements, the monitoring area is required to be arranged in front of the container, however, the area size of the monitoring area can be not limited, and the monitoring area can be particularly set according to the actual application requirements, so that the monitoring area can be ensured only when a target object (such as a customer) contacts with a commodity, such as selecting the commodity. For example, the inside of the dashed box (i.e., the shaded portion) in front of the container in FIG. 2 is the monitoring area. That is, when the customer contacts with the commodity, the customer is necessarily located in the monitoring area, so the local monitoring device only needs to monitor the monitoring areas with emphasis, and can know whether the customer takes the commodity and what kind of commodity is taken, and for other areas outside the monitoring area, the customer cannot contact the commodity, so the monitored data such as the monitored video shot by the customer can be taken as auxiliary data without emphasis, thus the data size of the subsequent monitored data analysis can be greatly reduced, and the processing efficiency is improved.

Based on the above-mentioned unattended shop monitoring system, the embodiment of the application provides a display method of an unattended shop monitoring interface, which comprises the following steps: receiving a payment check message sent by a local monitoring device for a target object through a remote monitoring device, wherein the payment check message carries payment information of the target object, a moving track of the target object in an unattended store and a moving picture of the target object in a monitoring area, and the monitoring area is arranged in front of a container of the unattended store; generating a first monitoring interface related to the unattended store, a second monitoring interface related to payment information, a third monitoring interface related to an activity track and a fourth monitoring interface related to a moving picture according to the payment check information, wherein the position corresponding to the moving picture currently displayed by the fourth monitoring interface and a picture shooting angle are marked on the activity track in the third monitoring interface, and when the moving picture currently displayed by the fourth monitoring interface is switched, the marking content corresponding to the moving track in the third monitoring interface is updated; and carrying out split-screen display on the first monitoring interface, the second monitoring interface, the third monitoring interface and the fourth monitoring interface on the remote monitoring equipment so that a user can execute payment checking operation based on the split-screen displayed first monitoring interface, second monitoring interface, third monitoring interface and fourth monitoring interface.

The display method of the unattended shop monitoring interface will be described in detail from the point of view of a display device (simply referred to as a display device) of the unattended shop monitoring interface. The display device can be specifically arranged at the cloud end by an independent entity, and can also be integrated in remote monitoring equipment.

As shown in fig. 3, the specific flow of the method for displaying the monitoring interface of the unattended shop may be as follows:

101. and receiving a payment check message for the target object sent by the local monitoring equipment through the remote monitoring equipment.

The payment checking message carries payment information of the target object, a moving track of the target object in an unattended store and a moving picture of the target object in a monitoring area, wherein the payment information can comprise paid commodity information, payment amount, number of paid commodities and the like. Optionally, the payment check message may also carry other information such as store information of an unattended store where the local monitoring device is located.

For convenience of description, in the application embodiment, the target object is taken as an example of a person, that is, a customer.

Optionally, the payment checking message may be acquired from the local monitoring device when the remote monitoring device needs the payment checking message, or may be provided to the remote monitoring device by the local monitoring device when the local monitoring device determines that the payment checking operation needs to be performed. That is, before the step of receiving the payment check message for the target object sent by the local monitoring device through the remote monitoring device, the display method of the unattended shop monitoring interface may further include:

when a target object enters a store entrance, collecting characteristic information of the target object; when the target object enters the store exit, a video frame corresponding to the target object is called from monitoring data such as monitoring video shot by local monitoring equipment according to the characteristic information; generating a moving track of the target object in the unattended store and a moving picture of the target object in a monitoring area according to the video frame; acquiring payment information corresponding to the target object; allowing the activity tracking object to pass through the store exit when it is determined that the payment checking operation is not required based on the payment information, the activity track and the activity picture; otherwise, when it is determined that the payment checking operation is required based on the payment information, the moving track and the moving picture, a payment checking message for the target object is transmitted to the remote monitoring device through the local monitoring device.

Wherein the characteristic information of the target object refers to essential and basic information which can reflect the characteristic of the target object and can be used for identifying the target object. For example, taking the target object as a human example, the feature information of the target object may include face feature information, and/or body feature information, etc. The step of "retrieving a video frame corresponding to the target object from monitoring data, such as a monitoring video, captured by the local monitoring device according to the feature information" may specifically be:

and according to the face characteristic information and/or the human characteristic information, retrieving a video frame corresponding to the target object from monitoring data such as monitoring video shot by the local monitoring equipment.

For example, taking the case that the feature information includes face feature information and human body feature information, the face feature information and the human body feature information can be fused by using a preset face and human body recognition model to obtain fused feature information, and then a video frame corresponding to the target object is called from monitoring data, such as a monitoring video, shot by the local monitoring device according to the fused feature information.

The facial feature information may include geometric features, such as geometric relationships between facial features including eyes, nose, mouth, and the like, such as distances, areas, angles, and the like, and characterization features, such as global or local features extracted by some algorithms, such as a local binary pattern (LBP,: local BinaryPatterns) algorithm, using gray information of a facial image. The body characteristics may include height, width (emaciation), color (e.g., dressing color and skin tone), texture, depth, behavioral characteristics, and/or direction of movement of the body.

Alternatively, "generating a moving track of the target object in the unattended store from the video frame, and a moving picture of the target object in the monitored area" may be various, for example, specifically as follows:

determining position information and time information of the target object in the store according to the video frame; screening out the position information in the monitoring area to obtain the position information after screening; generating track points of the target object in a preset shop map according to the screened position information; connecting the track points according to the sequence of the time information to obtain the moving track of the target object in the store; and screening out video frames of which the target object is positioned in the monitoring area and the stay time is longer than a set value according to the position information and the time information, and obtaining a moving picture of the target object in the monitoring area.

The setting value may be set according to the actual application requirement, which is not described herein.

Alternatively, this approach may also be used: screening video frames of the target object in the monitoring area from the video frames to obtain screened video frames; determining the position information and time information of the target object in the store according to the screened video frame; generating track points of the target object in a preset store map according to the position information, and connecting the track points according to the sequence of time information to obtain the moving track of the moving tracking object in the store; and extracting key frames from the screened video frames to obtain the moving picture of the moving tracking object in the monitoring area.

Alternatively, there may be various ways of determining whether or not the payment checking operation is required based on the payment information, the moving track and the moving picture, for example, for several cases where a mispayment or a miss-payment is liable to occur, it may be determined whether or not the payment checking operation is required in particular as follows:

first, the commodity is in contact with a lot, but the commodity is too little to purchase

And determining a commodity display area corresponding to the paid commodity information according to the paid commodity information, and acquiring the duty ratio of the commodity display area in the moving track, wherein if the duty ratio is smaller than a first threshold value, the payment checking operation is determined to be needed, otherwise, if the duty ratio is larger than or equal to the first threshold value, the payment checking operation is determined to be not needed.

The first threshold may be set according to the requirements of practical applications, for example, may be set to 5%, 10% or 20%, etc.

Wherein the product display area refers to the area to which a product or a type of product belongs in a container, the product display area is located in the monitoring areas, and each monitoring area can comprise at least one product display area. The commodity display area corresponding to the paid commodity information refers to the area of the commodity corresponding to the paid commodity information in the container. Namely, the step of acquiring the duty ratio of the commodity display area in the moving track may specifically be:

And counting the number of the commodity display areas corresponding to the paid commodity information, determining the total number of the commodity display areas related to the target object according to the moving track, and then calculating the ratio of the number of the commodity display areas corresponding to the paid commodity information to the total number to obtain the duty ratio of the commodity display areas in the moving track.

For example, taking the first threshold value as 20%, if four monitoring areas, such as the monitoring area 1, the monitoring area 7, the monitoring area 3 and the monitoring area 4, are involved in the three active tracks. Wherein, in the monitoring area 1, the number of the commodity display areas related to Zhang three is 3; in the monitoring area 7, the number of merchandise display areas related to Zhang three is 4; in the monitoring area 3, 1 merchandise display area is related to Zhang san, and in the monitoring area 4, 2 merchandise display areas are related to Zhang san; that is, if three items of merchandise are purchased in 1 item of merchandise display area, and the merchandise display area corresponding to the item of merchandise is located in the monitoring area 1, the ratio of the merchandise display area corresponding to the paid-out merchandise information to the moving track can be calculated as: 1:10 =10%

Since 10% is less than 20%, it can be determined that the payment verification operation is required.

For example, in the above example, if only 3 products are purchased, in which the product display areas corresponding to two products are located in the product display area 1 and the product display area 2 of the monitoring area 1, respectively, and the product display areas corresponding to the other products are located in the monitoring area 3, then the ratio of the product display area corresponding to the paid product information in the moving track may be calculated as:

3：10=30%

since 30% is greater than 20%, it can be determined that the payment checking operation is not required, and the target object can be directly allowed to pass through the store exit.

(II) case where the payment amount is too small

If the payment amount in the payment information is less than the second threshold value, determining that a payment verification operation is required (because the payment amount is too small to be a condition of being misplaced or missed); if the payment amount in the payment information is greater than or equal to a second threshold value, determining a commodity display area corresponding to the paid commodity information according to the paid commodity information, acquiring the duty ratio of the commodity display area in the moving track, if the duty ratio is smaller than the first threshold value, determining that payment checking operation is required, otherwise, if the duty ratio is greater than or equal to the first threshold value, determining that payment checking operation is not required.

(III) case of a large number of commodities

If the number of the paid commodities in the payment information is larger than a third threshold value, determining that payment checking operation is needed; if the number of the paid commodities in the payment information is smaller than or equal to a third threshold value, determining a commodity display area corresponding to the paid commodity information, acquiring the duty ratio of the commodity display area in the moving track, if the duty ratio is smaller than the first threshold value, determining that payment checking operation is required, otherwise, if the duty ratio is larger than or equal to the first threshold value, determining that payment checking operation is not required.

(IV) combining the above cases

If the payment amount in the payment information is smaller than the second threshold value or the number of the paid commodities is larger than the third threshold value, determining that the payment checking operation is required; if the payment amount in the payment information is greater than or equal to a second threshold value and the number of the paid commodities is less than or equal to a third threshold value, determining a commodity display area corresponding to the paid commodity information, and acquiring the occupation ratio of the commodity display area in the activity track; if the duty ratio is smaller than a first threshold value, determining that payment checking operation is required; if the duty ratio is equal to or greater than the first threshold, it is determined that the payment verification operation is not required.

(V) shopping only

If the payment information is blank (for example, payment information cannot be acquired or the acquired payment information is blank information), determining the stay time of the movable tracking object in each commodity display area according to the movable track and the movable picture; if the stay time is greater than the fourth threshold, determining that a payment verification operation is required; otherwise, if the stay time is less than or equal to the fourth threshold, determining that the payment checking operation is not required, and allowing the target object to pass through the store exit.

The second threshold, the third threshold and the fourth threshold may be set according to the actual application requirement, which is not described herein.

In addition, in the cases of the modes (two), (three) and (four), the specific calculation mode of the "duty ratio of the merchandise display area in the moving track" may be referred to as the mode (one), and will not be described herein.

102. A first monitoring interface for the unattended store, a second monitoring interface for payment information, a third monitoring interface for an activity track, and a fourth monitoring interface for a moving picture are generated according to the payment check message. The moving track in the third monitoring interface is marked with a position corresponding to a moving picture currently displayed by the fourth monitoring interface and a picture shooting angle; when the currently displayed moving picture of the fourth monitoring interface is switched, the identification content on the moving track in the third monitoring interface is correspondingly updated. For example, the following may be specifically mentioned:

Acquiring store information of an unattended store where a local monitoring device for sending the payment check message is located, and generating a first monitoring interface according to the store information;

generating a second monitoring interface according to the payment information in the payment check message;

generating a third monitoring interface according to the activity track in the payment verification message;

generating a fourth monitoring interface according to the moving picture in the payment check message;

the third monitoring interface may include a thumbnail display control and a shooting angle switching control, as shown in fig. 4. The thumbnail display control is used for generating a thumbnail according to a moving picture currently displayed by the fourth monitoring interface, determining a track point corresponding to the moving picture, and displaying the thumbnail on a moving track in the third monitoring interface according to the track point; the shooting angle switching control is used for determining the picture shooting angle of the moving picture currently displayed by the fourth monitoring interface, and the picture shooting angle is identified on the moving track in the third monitoring interface.

The picture shooting angle refers to an angle at which the moving picture is shot. It should be noted that, since the local monitoring device includes a plurality of cameras, the target object may be photographed from a plurality of angles, and thus, for the same location (such as the same monitoring area or the same merchandise display area), there may be moving pictures of a plurality of picture photographing angles, such as moving pictures photographed from a first camera in the south, moving pictures photographed from a second camera in the west, and so on. The image capturing angle may be represented in various manners, for example, the image capturing angle may be identified by a direction, or may be identified by a position of the camera, a number of the camera, or coordinate information.

Optionally, in order to facilitate the user to view moving pictures with different shooting angles in the same area, the shooting angle switching control may be further configured to obtain a moving picture corresponding to the current position of the switched shooting angle in response to a shooting angle switching instruction triggered by the user based on the shooting angle switching control, obtain a switched moving picture, and update the moving picture currently displayed in the fourth monitoring interface to the switched moving picture.

Optionally, in order to facilitate the user to check the position of the paid commodity in the moving track, as shown in fig. 5, the third monitoring interface may further include a payment information display control, configured to determine a corresponding commodity display area according to the paid commodity information, determine a track point corresponding to the commodity display area, generate thumbnail information according to the paid commodity information, and display the thumbnail information on the moving track in the third monitoring interface according to the determined track point.

Optionally, in order to facilitate the user to view the position of the moving picture in the moving track and switch the moving picture, as shown in fig. 5, the third monitoring interface may further include a picture switching control, configured to, in response to a moving picture switching instruction triggered by the user clicking on the track point, obtain, through the picture switching control, a moving picture corresponding to the track point clicked by the user, obtain a switched moving picture, and update the moving picture currently displayed in the fourth monitoring interface to the switched moving picture.

Optionally, as shown in fig. 5, the first monitoring interface may further include a store switching control, which is configured to respond to a store switching instruction triggered by the store switching control by a user to switch an unattended store to be monitored, so that, for example, a monitoring person can flexibly select or switch a store to be monitored by triggering the store switching control.

Optionally, in order to facilitate viewing the moving track position and/or moving picture corresponding to the paid commodity information in the payment information, as shown in fig. 5, the second monitoring interface may further include a trace selection control, where the trace selection control may be used to identify, on the moving track displayed in the third monitoring interface, a commodity display area where the commodity corresponding to the selected paid commodity information is located (for example, highlight the commodity display area) in response to a paid commodity information selection instruction triggered by the user based on the trace selection control, and display, in the third monitoring interface, the moving picture of the target object in the commodity display area.

Optionally, in order to facilitate the user to display the active track and the active screen, as shown in fig. 5, the second monitoring interface may further include an active track display trigger key, an active screen display trigger key, and the like.

The system comprises an activity track display trigger key, an activity track display control interface and an activity track display control interface, wherein the activity track display trigger key is used for responding to an activity track display instruction triggered by the activity track display trigger key by a user, displaying the activity track in a third monitoring interface, and stopping displaying the activity track in the third monitoring interface in response to an activity track closing instruction triggered by the activity track display trigger key by the user.

A moving picture display trigger key for displaying a moving picture in the fourth monitoring interface in response to a moving picture display instruction triggered by the user based on the moving picture display trigger key, and stopping displaying the moving picture in the fourth monitoring interface in response to a moving picture close instruction triggered by the user based on the moving picture display trigger key.

Optionally, for convenience of user selection and switching of the moving picture, as shown in fig. 5, the fourth monitoring interface may further include a forward switch control and/or a backward switch control.

The forward switch control may be configured to obtain a previous moving picture of a moving picture currently displayed on the fourth monitoring interface in response to a moving picture switch instruction triggered by the user through the forward switch control, and update the moving picture currently displayed on the fourth monitoring interface to the previous moving picture.

The backward switching control can be used for responding to a moving picture switching instruction triggered by a user through the backward switching control, acquiring the next moving picture of the moving picture currently displayed in the fourth monitoring interface, and updating the moving picture currently displayed in the fourth monitoring interface to the next moving picture.

103. And carrying out split-screen display on the first monitoring interface, the second monitoring interface, the third monitoring interface and the fourth monitoring interface on the remote monitoring equipment so that a user can execute payment checking operation based on the monitoring interfaces.

The split display mode may be set according to the actual application requirement, for example, the split display mode may be displayed side by side, or, as shown in fig. 4 and fig. 5, the split display mode may be used to divide the screen into four boxes, for example, the display mode is displayed in the upper left, lower left, upper right and lower right directions, and so on.

Optionally, if the third monitoring interface includes a thumbnail display control, after "performing split screen display on the first monitoring interface, the second monitoring interface, the third monitoring interface, and the fourth monitoring interface on the remote monitoring device" in step, the method for displaying the monitoring interface of the unattended store may further include:

Determining a track point corresponding to the moving picture currently displayed on the fourth monitoring interface, generating a thumbnail of the moving picture currently displayed by using the thumbnail display control, and displaying the thumbnail of the moving picture currently displayed on the corresponding position on the moving track in the third monitoring interface by using the thumbnail display control according to the determined track point.

For example, if the determined track point is track point a, at this time, a thumbnail of the currently displayed moving picture may be displayed by the thumbnail display control at a blank place near track point a of the moving track in the third monitoring interface, and so on.

Optionally, if the third monitoring interface includes a shooting angle switching control, after "performing split screen display on the first monitoring interface, the second monitoring interface, the third monitoring interface, and the fourth monitoring interface on the remote monitoring device" in step, the method for displaying the monitoring interface of the unattended store may further include:

and responding to a shooting angle switching instruction triggered by the shooting angle switching control by a user, acquiring a moving picture corresponding to the current position of the shooting angle after switching, obtaining a moving picture after switching, and updating the moving picture currently displayed in the fourth monitoring interface into the moving picture after switching.

Optionally, if the third monitoring interface includes a payment information display control, after "performing split screen display on the first monitoring interface, the second monitoring interface, the third monitoring interface, and the fourth monitoring interface on the remote monitoring device", the method for displaying the monitoring interface of the unattended store may further include:

and generating thumbnail information of the paid commodity information, determining a corresponding commodity display area according to the paid commodity information, determining a track point corresponding to the commodity display area, and then displaying the thumbnail information on an active track in a third monitoring interface by using a payment information display control according to the determined track point.

For example, if the determined track point is track point a, at this time, the payment information display control may be used to display the thumbnail information in a blank space near track point a of the active track in the third monitoring interface, and so on.

Optionally, if the third monitoring interface further includes a screen switching control associated with each track point, after "performing split screen display on the first monitoring interface, the second monitoring interface, the third monitoring interface, and the fourth monitoring interface on the remote monitoring device", the method for displaying the monitoring interface of the unattended store may further include:

Responding to a moving picture switching instruction triggered by clicking the track point by a user, and acquiring a moving picture corresponding to the track point clicked by the user through the picture switching control to obtain a switched moving picture; and updating the moving picture currently displayed in the fourth monitoring interface to the switched moving picture.

For example, if the moving picture currently displayed in the fourth monitoring interface is the moving picture corresponding to the track point a, when the user clicks the track point B in the third monitoring interface, a moving picture switching instruction about the track point B is generated, then, according to the moving picture switching instruction about the track point B, the moving picture corresponding to the track point B is obtained through the picture switching control, and then, the moving picture corresponding to the track point a currently displayed in the fourth monitoring interface can be updated to the moving picture corresponding to the track point B, and so on.

Optionally, if the second monitoring interface includes a trace selection control, after "performing split screen display on the first monitoring interface, the second monitoring interface, the third monitoring interface, and the fourth monitoring interface on the remote monitoring device" in step, the method for displaying a monitoring interface of the unattended store may further include:

And responding to a paid commodity information selection instruction triggered by the trace selection control by a user, identifying a commodity display area where the commodity corresponding to the selected paid commodity information is located on a moving track displayed in a third monitoring interface, and displaying a moving picture of a target object in the commodity display area in the third monitoring interface.

Optionally, if the second monitoring interface includes an active track display trigger key, after "performing split screen display on the first monitoring interface, the second monitoring interface, the third monitoring interface, and the fourth monitoring interface on the remote monitoring device" in step, the method for displaying a monitoring interface of the unattended store may further include:

and displaying the activity track in the third monitoring interface in response to an activity track display instruction triggered by the user based on the activity track display trigger key, and stopping displaying (such as hiding or not displaying) the activity track in the third monitoring interface in response to an activity track closing instruction triggered by the user based on the activity track display trigger key.

Optionally, if the second monitoring interface includes a moving picture display trigger key, after "performing split screen display on the first monitoring interface, the second monitoring interface, the third monitoring interface, and the fourth monitoring interface on the remote monitoring device", the method for displaying the monitoring interface of the unattended shop may further include:

The moving picture in the fourth monitoring interface is displayed in response to a moving picture display instruction triggered by the user based on the moving picture display trigger key, and the moving picture in the fourth monitoring interface is stopped from being displayed in response to a moving picture closing instruction triggered by the user based on the moving picture display trigger key.

Optionally, if the fourth monitoring interface includes a forward switch control, after "performing split screen display on the first monitoring interface, the second monitoring interface, the third monitoring interface, and the fourth monitoring interface on the remote monitoring device" in step, the method for displaying the monitoring interface of the unattended store may further include:

responding to a moving picture switching instruction triggered by a user through the forward switching control, acquiring a previous moving picture of a moving picture currently displayed in a fourth monitoring interface, and updating the moving picture currently displayed in the fourth monitoring interface into the previous moving picture; generating a thumbnail of the previous moving picture by using the thumbnail display control, determining a track point corresponding to the previous moving picture, displaying the thumbnail of the previous moving picture on the moving track in the third monitoring interface according to the determined track point, and stopping displaying the thumbnail originally displayed; and determining the picture shooting angle of the last moving picture by using the shooting angle switching control, and marking the picture shooting angle on the moving track in the third monitoring interface according to the determined track point.

Optionally, if the fourth monitoring interface includes a backward switching control, after "performing split screen display on the first monitoring interface, the second monitoring interface, the third monitoring interface, and the fourth monitoring interface on the remote monitoring device" in step, the method for displaying the monitoring interface of the unattended store may further include:

responding to a moving picture switching instruction triggered by a user through the backward switching control, acquiring a next moving picture of a moving picture currently displayed in a fourth monitoring interface, and updating the moving picture currently displayed in the fourth monitoring interface into the next moving picture; generating a thumbnail of the next moving picture by using the thumbnail display control, determining a track point corresponding to the next moving picture, displaying the thumbnail of the next moving picture on the moving track in the third monitoring interface according to the determined track point, and stopping displaying the originally displayed thumbnail; and determining the picture shooting angle of the next moving picture by using the shooting angle switching control, and marking the picture shooting angle on the moving track in the third monitoring interface according to the determined track point.

Optionally, if the user, such as a monitor, determines that there is no problem after checking the payment, a response message that the user passes the check may be sent to the local monitoring device, so that the local monitoring device operates the target object to pass through the store exit, otherwise, if it is determined that there is a problem with the payment, a corresponding measure may be taken, such as sending a response message that the user does not pass the check to the local monitoring device. That is, after the step of "performing split screen display on the first monitoring interface, the second monitoring interface, the third monitoring interface, and the fourth monitoring interface on the remote monitoring device", the method for displaying the monitoring interface of the unattended shop may further include:

If the user determines that there is no problem after checking the payment, the user can send a response message that the user passes the check to the local monitoring device, and the local monitoring device allows the activity tracking object to pass through the outlet of the store after receiving the response message that the user passes the check, for example, can control the entrance guard of the outlet of the store to open, or else, if the user determines that the payment has the problem, the user sends a response message that the user does not pass the check to the local monitoring device, and after receiving the response message that the user does not pass the check, the local monitoring device can execute corresponding operations, for example, can play reminding information, so that the target object can pay the payment again or alarm.

The preset policy may be set according to the actual application requirement, which is not described herein.

As can be seen from the foregoing, in the method for displaying an unattended store monitoring interface provided by the embodiment of the present application, a payment check message sent by a local monitoring device for a target object may be received by a remote monitoring device, and a first monitoring interface related to the unattended store, a second monitoring interface related to payment information of the target object, a third monitoring interface related to a moving track of the target object in the unattended store, and a fourth monitoring interface related to a moving picture of the target object in a monitoring area are generated according to the payment check message, where a position corresponding to the moving picture currently displayed by the fourth monitoring interface and a picture shooting angle are identified on the moving track in the third monitoring interface, and when the moving picture currently displayed by the fourth monitoring interface is switched, identification content correspondence on the moving track in the third monitoring interface is updated, and then, split-screen display is performed on the remote monitoring device for the interfaces, so that a user performs a payment check operation based on the interfaces; the scheme only monitors the video data, such as the moving track, of the target object in a specific area, such as the moving picture, of the target object in the monitoring area, so that the data size of the monitoring data to be checked can be greatly reduced; in addition, different monitoring interfaces which can be flexibly and friendly operated are generated aiming at different monitoring data, and the relevance among the monitoring interfaces is identified when the monitoring interfaces are displayed in a split screen mode, so that a user, such as a monitoring person, can more conveniently check the monitoring data, and the processing efficiency is greatly improved. In addition, the scheme can monitor the transactions in the unattended store in real time without sense, so that the conditions of commodity loss, mispayment or missed payment can be greatly reduced, the store property loss is reduced, and the security of the store transactions is improved.

Based on the embodiments described above, examples will be described in further detail below.

In this embodiment, an example will be described in which the display device is specifically integrated in a remote monitoring apparatus.

As shown in fig. 6, a method for displaying a monitoring interface of an unattended shop specifically includes the following steps:

201. when a target object enters a store entrance, the local monitoring equipment collects characteristic information of the target object, and when the target object enters the store entrance, a video frame corresponding to the target object is called from a monitoring video shot by the local monitoring equipment according to the characteristic information, and a moving track of the target object in the store and a moving picture of the target object in a monitoring area are generated according to the video frame.

For example, when a target object enters the store entrance, the local monitoring device collects face feature information and body feature information of the target object, fuses the face feature information and the body feature information to obtain fused feature information, then, when the target object enters the store entrance, the local monitoring device can call a video frame matched with the fused feature information from a monitoring video to obtain a video frame corresponding to the target object, then, position information and time information of the target object in the store are determined according to the called video frame, position information in a monitoring area is screened out to obtain screened position information, track points of the target object are generated in a preset store map according to the screened position information, and the track points are connected according to the sequence of the time information to obtain a moving track of the target object in the store, wherein the starting point of the moving track is the store entrance, and the end point of the moving track is the store exit. In addition, the local monitoring equipment can screen out the video frame of the target object in the monitoring area according to the position information and the time information, wherein the stay time of the video frame is longer than the set value, and the moving picture of the target object in the monitoring area can be obtained.

For example, referring to fig. 7, if the target object enters the store three times, the entered monitoring areas are the monitoring area 1, the monitoring area 7, the monitoring area 3 and the monitoring area 4 in sequence, at this time, the local monitoring device may screen out the position information of the three times located in the monitoring area 1, the monitoring area 7, the monitoring area 3 and the monitoring area 4 to obtain screened position information, and then generate three track points in a preset store map according to the screened position information, for example, as shown in fig. 8, the track point a of the store entrance, the track point B in the monitoring area 1, the track point C in the monitoring area 7, the track point D in the monitoring area 3, the track point E in the monitoring area 4 and the track point F at the store exit are sequentially generated, and then the track points are connected according to the sequence of time information, so as to obtain the active track of the three times in the store: a- (B-) C- (D-) E- (F).

Note that the order of generation of the moving track and the moving picture may be different, and the product display area is simply referred to as "display area" in fig. 7 to 11 for convenience of drawing.

The surveillance video may be obtained by capturing, by a local surveillance device, for example, in fig. 7, all areas and surveillance areas in the store, such as surveillance area 1, surveillance area 2, surveillance area 3, surveillance area 4, surveillance area 5, surveillance area 6, surveillance area 7, and surveillance area 5, by using camera 1, camera 2, camera 3, camera 4, camera 5, camera 6, camera 7, camera … …, camera n, and the like.

202. The local monitoring device obtains payment information corresponding to the target object, determines whether a payment check operation is required based on the payment information, the moving track and the moving picture, if yes, executes step 203, and if not, allows the target object to pass through the store exit.

Optionally, in order to better remind and guide the customer, when the target object is allowed to pass through the outlet of the store, a corresponding voice prompt message and/or a light shadow prompt message may also be set, for example, when the entrance guard is opened, the local monitoring device may play the prompt message: please pass, welcome next visit, and at the same time, the gate indicator light is green, etc.

The payment information may include information such as information of the paid goods, payment amount, and number of paid goods, and the method for determining whether the payment checking operation is needed may be found in the previous embodiments, which are not described herein.

203. The local monitoring device sends a payment check message for the target object to the remote monitoring device, wherein the payment check message carries payment information of the target object, a moving track of the target object in an unattended store, and a moving picture of the target object in a monitoring area.

204. The remote monitoring device generates a first monitoring interface for the unattended shop, a second monitoring interface for payment information, a third monitoring interface for a moving track, and a fourth monitoring interface for a moving picture according to the payment check message after receiving the payment check message.

The styles of the first monitoring interface, the second monitoring interface, the third monitoring interface and the fourth monitoring interface can be determined according to the actual application requirements. For example, referring to fig. 9, the following may be specifically mentioned:

the remote monitoring device generates a first monitoring interface 301 according to the store information of the unattended store where the local monitoring device sending the payment check message is located; generating a second monitoring interface 302 based on the payment information in the payment verification message; generating a third monitoring interface 303 according to the activity track in the payment verification message; and generating a fourth monitoring interface 304 based on the moving picture in the payment check message; the monitoring interfaces specifically may be as follows:

(1) A first monitoring interface 301;

the first monitoring interface 301 is used to display store information such as a store number, a store address, a contact phone, responsible person information, and photographing dates and times of the moving track and moving picture, etc.

Optionally, the first monitoring interface 301 may further include a store switching control 3011, where the switching control 3011 is configured to switch an unattended store that needs to be monitored in response to a store switching instruction triggered by the user based on the store switching control 3011.

The style of the shop switching control 3011 may be set according to the requirements of the practical application, for example, as shown in fig. 9, the style of the shop switching control 3011 may be set to a "search box", and when the user inputs a shop number to be searched in the search box and clicks a "magnifier" icon beside the search box to perform searching, the shop switching control 3011 may be triggered.

(2) A second monitoring interface 302;

the second monitoring interface 302 is used to display payment information and the like.

Optionally, as shown in fig. 9, the second monitoring interface 302 may further include a trace selection control 3201, an active trace display trigger key 3022, and an active screen display trigger key 3023, specifically as follows:

The trace selection control 3021 may be used to identify, on a moving track displayed in the third monitoring interface 303, a merchandise display area where a merchandise corresponding to the selected paid merchandise information is located (for example, to highlight the merchandise display area) in response to a paid merchandise information selection instruction triggered by the trace selection control 3021 by a user, and to display a moving picture of a target object in the merchandise display area in the third monitoring interface 303.

The moving track display trigger key 3022 is configured to display the moving track in the third monitoring interface 303 in response to an moving track display instruction triggered by the user based on the moving track display trigger key 3022, and stop displaying the moving track in the third monitoring interface 303 in response to an moving track closing instruction triggered by the user based on the moving track display trigger key 3022.

A moving picture display trigger key 3023 for displaying a moving picture in the fourth monitoring interface 304 in response to a moving picture display instruction triggered by the user based on the moving picture display trigger key 3023, and stopping displaying the moving picture in the fourth monitoring interface 304 in response to a moving picture closing instruction triggered by the user based on the moving picture display trigger key 3023.

The patterns of the trace selection control 3021, the moving trace display trigger key 3022 and the moving picture display trigger key 3023 may be set according to the requirements of the practical application, for example, as shown in fig. 9, the trace selection control 3021 may be set on "payment information", so that when the user clicks a piece of paid commodity information on the payment information, the trace selection control 3021 is triggered and generates a paid commodity information selection instruction, where the paid commodity information selection instruction carries the clicked piece of paid commodity information. The style of the moving track display trigger key 3022 may be set to a trigger key "moving track", and similarly, the style of the moving picture display trigger key 3023 may be set to a trigger key "moving picture", and so on.

(3) A third monitoring interface 303;

the third monitoring interface 303 is configured to display contents such as an activity track, and in addition, the third monitoring interface 303 further includes a thumbnail display control 3031 and a shooting angle switching control 3032, which are specifically as follows:

a thumbnail display control 3031 is configured to generate a thumbnail according to a moving picture currently displayed by the fourth monitoring interface 304, determine a track point corresponding to the moving picture, display the thumbnail according to the track point on the moving track in the third monitoring interface 303, for example, may display a thumbnail of the moving picture near the track point, such as a blank of the track point B, and so on.

The shooting angle switching control 3032 is used for determining the picture shooting angle of the moving picture currently displayed by the fourth monitoring interface 304, and identifying the picture shooting angle on the moving track in the third monitoring interface 303. The shooting angle switching control 3032 may be further configured to, in response to a shooting angle switching instruction triggered by the user based on the shooting angle switching control 3032, obtain a moving picture corresponding to the switched shooting angle at the current position, obtain a switched moving picture, update the moving picture currently displayed in the fourth monitoring interface 4 to the switched moving picture, and update the thumbnail displayed in the third monitoring interface 303 to the thumbnail of the switched moving picture.

The style of the shooting angle switching control 3032 may be determined according to the requirements of practical applications, for example, taking shooting angles as "east (E), west (W), south (S) or north (N)" as an example, the shooting angle switching control 3032 may be set as a direction identifier shown in fig. 9, in which a shooting angle corresponding to a currently displayed moving picture may be highlighted, for example, referring to fig. 9, if the moving picture is shot by a camera from the south direction, the direction identifier may be highlighted at this time to the south direction (S). In addition, when the user clicks a certain direction, such as the western direction (W), in the direction identifier, the direction identifier is triggered, and a corresponding shooting angle switching instruction is generated, where the shooting angle after switching carried by the shooting angle switching instruction is the clicked direction, such as the western direction (W), so that a moving picture corresponding to the shooting angle after switching in the same merchandise display area can be obtained as a moving picture after switching according to the shooting angle switching instruction.

Optionally, as shown in fig. 9, the third monitoring interface 303 may further include a payment information display control 3033 and a screen switching control, which may specifically be as follows:

the payment information display control 3033 is configured to determine a corresponding merchandise display area according to the paid merchandise information, determine a track point corresponding to the merchandise display area, generate thumbnail information according to the paid merchandise information, and display the thumbnail information on the active track in the third monitoring interface 303 according to the determined track point, for example, specifically, may display the thumbnail information of the paid merchandise information in a blank position close to the merchandise display area corresponding to the paid merchandise information.

For example, if three products purchased in the product display area Z1 (trace point B) are a bottle of X bottled water having a price of 3 yuan, and one product purchased in the product display area Z2 (trace point D) is a package of Y biscuits having a price of 8 yuan, as shown in fig. 9 and 10, "X bottled water, 1-" 3.0 "can be displayed in the blank of trace point B, and" Y biscuits, 1- "8.0" can be displayed in the blank of trace point D. Therefore, when the user sees the information, the user can learn that the third person stays at the two places to purchase a bottle of X bottled water and a bag of Y biscuits, and the two goods are paid, so that the user can quickly check or skip the moving pictures at the two places and directly check the moving pictures in other monitoring fields, and the efficiency of payment checking operation is improved.

And the screen switching control is used for responding to a moving screen switching instruction triggered by clicking the track point by a user, acquiring a moving screen corresponding to the track point clicked by the user through the screen switching control, obtaining a switched moving screen, and updating the moving screen currently displayed in the fourth monitoring interface 304 into the switched moving screen.

The style of the screen switching control may be determined according to the actual application requirement, for example, the screen switching control may be set on each track point (the track points are more and are not shown in fig. 9), and when a user clicks a certain track point, the screen switching control may be triggered to obtain a moving picture corresponding to the track point clicked by the user, so as to obtain a moving picture after switching.

(4) A fourth monitoring interface 304;

the fourth monitoring interface 304 is used to display contents such as a moving picture.

Optionally, as shown in fig. 9, the fourth monitoring interface 304 may further include a forward switch control 3041 and a backward switch control 3042, which may specifically be as follows:

a forward switch control 3041, configured to obtain a previous moving picture of a moving picture currently displayed on the fourth monitoring interface 304 in response to a moving picture switch instruction triggered by the user through the forward switch control 3041, and update the moving picture currently displayed on the fourth monitoring interface 304 to the previous moving picture.

A backward switching control 3042 is configured to obtain a next moving picture of the moving picture currently displayed in the fourth monitoring interface 304 in response to a moving picture switching instruction triggered by the user through the backward switching control 3042, and update the moving picture currently displayed in the fourth monitoring interface 304 to the next moving picture.

The styles of the toggle forward control 3041 and the toggle backward control 3042 may be determined according to the requirements of the practical application, for example, as shown in fig. 9, the style of the toggle forward control 3041 may be set to a trigger key "previous page", the style of the toggle forward control 3041 may be set to a trigger key "next page", and so on.

205. And the remote monitoring equipment performs split-screen display on the first monitoring interface, the second monitoring interface, the third monitoring interface and the fourth monitoring interface so that a user can execute payment checking operation based on the interfaces.

The split screen mode can be determined according to the actual application requirements and the preference of the user.

For example, as shown in fig. 9 and 10, four screens can be divided: screen a, screen B, screen C, and screen D, wherein screen a is for the first monitoring interface 301; the screen B can display a second monitoring interface; the C screen is used for displaying a third monitoring interface; and the D screen is used to display the fourth monitoring interface 304.

Alternatively, when displaying a moving picture, only one moving picture may be displayed, for example, see fig. 9, or a plurality of moving pictures may be simultaneously displayed, for example, a D screen in fig. 10, that is, 4 moving pictures photographed by a camera from different angles when three moving pictures are simultaneously displayed at the locus point B.

Optionally, for the displayed moving track and moving picture, the user, such as a monitor, can trigger various controls or trigger keys to execute corresponding operations, so as to better check payment.

For example, taking the setting of the style of the shooting angle switching control 3032 as the direction identifier shown in fig. 11 as an example, when the user clicks the west direction (W) in the direction identifier corresponding to the track point B, the shooting angle switching control 3032 is triggered and generates a corresponding shooting angle switching instruction, where the shooting angle after switching carried by the shooting angle switching instruction is the clicked direction—the west direction (W), and then, according to the shooting angle switching instruction, the remote monitoring device acquires, from the multiple moving pictures corresponding to the track point B, the moving picture corresponding to the west direction (W) as a moving picture after switching, and then, updates the moving picture currently displayed in the fourth monitoring interface 304 to the moving picture after switching, for example, see fig. 12. In addition, the remote monitoring device may generate a thumbnail of the switched moving picture by using the thumbnail display control 3031, and update the thumbnail originally displayed on the track point B in the third monitoring interface 303 to the thumbnail of the switched moving picture.

For another example, taking a screen switching control set on each track point as shown in fig. 13, when a user clicks a certain track point, for example, clicks a track point E, the screen switching control is triggered to generate a moving screen switching instruction, then a moving screen corresponding to the clicked track point, namely, the track point E, is obtained according to the moving screen switching instruction, a switched moving screen is obtained, and then the moving screen currently displayed in the fourth monitoring interface 304 is updated to the switched moving screen. In addition, the remote monitoring device may also generate a thumbnail of the switched moving picture using the thumbnail display control 3031, determine the picture photographing angle of the switched moving picture using the photographing angle switching control 3032, then stop displaying the thumbnail originally displayed on the track point B and the identification related to the picture photographing angle in the third monitoring interface 303 (i.e., restore the track point B to the original state, see fig. 13), and display the thumbnail of the switched moving picture in the blank place near the track point E in the third monitoring interface 303, and identify the picture photographing angle, e.g., if the picture photographing angle is the south direction (S), the south direction (S) may be highlighted on the displayed direction identification, see fig. 11, and so on.

For another example, taking the trace selection control 3021 set on payment information as an example, when a user clicks on a piece of paid commodity information such as "X bottled water" on the payment information, the trace selection control 3021 is triggered and generates a paid commodity information selection instruction, wherein the paid commodity information selection instruction carries the clicked paid commodity information, then determines a commodity display area where a commodity corresponding to the paid commodity information is located and a moving picture of an acquisition target object in the commodity display area according to the paid commodity information selection instruction, and then highlights the commodity display area in the third monitoring interface 303, for example, the commodity display area may be highlighted with a vivid color, and the acquired moving picture is displayed in the fourth monitoring interface 304. Of course, it is also possible to generate a corresponding thumbnail and a corresponding picture shooting angle based on the moving picture displayed in the fourth monitoring interface 304, and display the thumbnail and the picture shooting angle in the third monitoring interface 303, and the method for generating and displaying the thumbnail and the picture shooting angle may be found in the previous embodiments, which are not described herein.

For another example, taking the style of the forward switch control 3041 as the trigger key "previous page", when the user clicks the trigger key "previous page", the forward switch control 3041 is triggered to generate a moving picture switching instruction, and then, the previous moving picture of the moving picture currently displayed in the fourth monitoring interface 304 is acquired according to the moving picture switching instruction, and the moving picture currently displayed in the fourth monitoring interface 304 is updated to the previous moving picture. In addition, the thumbnail display control 3031 may also be used to generate a thumbnail of the previous moving picture, determine a track point corresponding to the previous moving picture, display the thumbnail of the previous moving picture on the moving track in the third monitoring interface 303 according to the determined track point, and stop displaying the thumbnail originally displayed; at the same time, the shooting angle switching control 3032 may also be used to determine the picture shooting angle of the previous moving picture, and the picture shooting angle is identified on the moving track in the third monitoring interface 303 according to the determined track point.

Similarly, taking the example that the style of the backward switching control 3042 is set as the trigger key "next page", when the user clicks the trigger key "next page", the backward switching control 3042 is triggered to generate a moving picture switching instruction, and then, the next moving picture of the moving picture currently displayed in the fourth monitoring interface 304 is acquired according to the moving picture switching instruction, and the moving picture currently displayed in the fourth monitoring interface 304 is updated to the next moving picture. In addition, the thumbnail display control 3031 may also be used to generate a thumbnail of the next moving picture, determine a track point corresponding to the next moving picture, display the thumbnail of the next moving picture on the moving track in the third monitoring interface 303 according to the determined track point, and stop displaying the originally displayed thumbnail; at the same time, the shooting angle switching control 3032 may also be used to determine the picture shooting angle of the next moving picture, and the picture shooting angle is identified on the moving track in the third monitoring interface 303 according to the determined track point.

Optionally, if the user, such as a monitor, determines that there is no problem after checking by payment, the user may send a response message for checking to pass to the local monitoring device, and after receiving the response message for checking to pass, the local monitoring device allows the activity tracking object to pass through the outlet of the store, for example, may control the entrance guard of the outlet of the store to open, etc.; otherwise, if the payment is determined to be problematic, sending a response message that the verification fails to pass to the local monitoring device, and after receiving the response message that the verification fails, the local monitoring device may perform a corresponding operation according to a preset policy, for example, reminding the target object to make payment again.

Optionally, in order to better remind and guide the customer, a corresponding voice prompt message and/or a light and shadow prompt message may be further set, for example, when the entrance guard is opened, the local monitoring device may play the prompt message: please pass, welcome next visit ", at the same time, the gate indicator light is" green ", and when determining that payment is problematic, the local monitoring device can play the reminding information: "please check whether the goods are paid for" and at the same time, the gate indicator light is "red", etc.

Therefore, the scheme can adopt the traditional goods shelves to list goods for customers to select freely, so that better shopping experience can be ensured. Meanwhile, the scheme can automatically judge whether the payment checking operation is needed or not based on the payment information of the customer and the moving track and moving picture of the customer in the store, and the payment checking operation can be performed only under abnormal conditions, and the moving track and moving picture carried in the payment checking information sent to the remote monitoring equipment are generated based on the monitoring area where the customer is most likely to contact with the commodity instead of all monitoring videos, so that the data processing amount can be greatly reduced, the processing efficiency of the monitoring system is greatly improved, the monitoring system can make quick response during monitoring, and therefore, the monitoring is noninductive and does not influence the shopping flow for the customer, that is, the scheme can realize noninductive monitoring while ensuring quick and flexible shopping of the customer, avoid the condition of commodity loss, mispaying or missing payment as much as possible, and reduce the property loss of the store.

In addition, the scheme can generate a plurality of different types of monitoring interfaces aiming at different monitoring data, and the relevance among the monitoring interfaces is marked while the monitoring interfaces are displayed in a split screen mode, and a plurality of controls and trigger keys are arranged for a user to operate, so that the user can more conveniently check the monitoring data, the flexibility and the friendliness of the operation are greatly improved, and the processing efficiency of monitoring personnel in executing payment checking operation is further improved.

Correspondingly, in order to better implement the method, the embodiment of the application also provides a display device (simply called a display device) of the monitoring interface of the unattended shop, and the display device can be implemented as an independent entity or can be integrated in a remote monitoring device in a monitoring system of the unattended shop (simply called a monitoring system). The monitoring system may include a remote monitoring device and a local monitoring device disposed in an unattended store, where the unattended store is provided with a store entrance, a store exit and a container, and a monitoring area is disposed in front of the container, and the foregoing method embodiments may be referred to specifically, and will not be described herein.

As shown in fig. 14, the display device may include a transceiver unit 401, a generating unit 402, and a display unit 403, specifically as follows:

and a transceiver unit 401, configured to receive a payment verification message for a target object sent by a local monitoring device, where the payment verification message carries payment information of the target object, a moving track of the target object in an unattended store, and a moving picture of the target object in a monitoring area.

A generation unit 402 for generating a first monitoring interface regarding the unattended store, a second monitoring interface regarding payment information, a third monitoring interface regarding an activity track, and a fourth monitoring interface regarding a moving picture according to the payment check message; and when the moving picture currently displayed by the fourth monitoring interface is switched, the identification content on the moving track in the third monitoring interface is correspondingly updated.

And the display unit 403 is configured to perform split-screen display on the first monitoring interface, the second monitoring interface, the third monitoring interface, and the fourth monitoring interface, so that the user performs payment checking operation based on the split-screen displayed first monitoring interface, second monitoring interface, third monitoring interface, and fourth monitoring interface.

Optionally, the first monitoring interface, the second monitoring interface, the third monitoring interface, and the fourth monitoring interface generated by the generating unit 402 may further include various controls and/or trigger keys, and the foregoing method embodiments may be referred to specifically, which are not described herein.

If the monitoring interfaces generated by the generating unit 403 further include other controls and/or trigger keys, the displaying unit 403 may further execute corresponding operations in response to the instructions triggered by the controls and/or trigger keys by the user after the monitoring interfaces are displayed in a split screen manner, and the foregoing method embodiments will not be described herein.

Furthermore, the transceiver unit 401 may be further configured to send a response message of the check passing to the local monitoring device when the payment check passes is determined, so that the local monitoring device operates the target object to pass through the store exit, otherwise, if the payment check does not pass, send a response message of the check failing to pass to the local monitoring device.

The implementation of each unit above may be specifically referred to the foregoing method embodiments, and will not be described herein.

It should be noted that, in the implementation, each unit may be combined arbitrarily, integrated in one or several modules, or may be implemented as a separate entity. In addition, the above units may be implemented in hardware or in software functional modules.

The display device not only can reduce the data volume of the monitoring data and display the monitoring interface which can be flexibly and friendly operated so as to improve the processing efficiency, but also can monitor commodity transaction in real time without sense, avoid the conditions of commodity loss, mispaying or missed payment and reduce the property loss of stores.

The foregoing has described in detail the method and apparatus for displaying an unattended shop monitoring interface provided by the embodiments of the present application, and specific examples have been applied to illustrate the principles and embodiments of the present application, where the foregoing description of the embodiments is only for helping to understand the method and core ideas of the present application; meanwhile, those skilled in the art will have variations in the specific embodiments and application scope in light of the ideas of the present application, and the present description should not be construed as limiting the present application in view of the above.

Claims

1. The method is characterized by being applied to a monitoring system of an unattended store, wherein the monitoring system of the unattended store comprises local monitoring equipment and remote monitoring equipment, the local monitoring equipment is arranged in the unattended store, and the remote monitoring equipment is connected with the local monitoring equipment of a plurality of unattended stores; the monitoring method comprises the following steps:

2. The method according to claim 1, wherein the generating, based on the payment check message, a first monitoring interface regarding the unattended store, a second monitoring interface regarding payment information, a third monitoring interface regarding an activity track, and a fourth monitoring interface regarding an activity screen, the activity track in the third monitoring interface identifying a position corresponding to the activity screen currently displayed by the fourth monitoring interface, and a screen capturing angle, the updating of the identification content correspondence on the activity track in the third monitoring interface when the activity screen currently displayed by the fourth monitoring interface is switched, includes:

acquiring store information of an unattended store where a local monitoring device sending the payment check message is located, and generating a first monitoring interface according to the store information;

the third monitoring interface comprises a thumbnail display control and a shooting angle switching control, wherein the thumbnail display control is used for generating a thumbnail according to a moving picture currently displayed by the fourth monitoring interface, determining a track point corresponding to the moving picture and displaying the thumbnail on a moving track in the third monitoring interface according to the track point; and the shooting angle switching control is used for determining the picture shooting angle of the moving picture currently displayed by the fourth monitoring interface, and identifying the picture shooting angle on the moving track in the third monitoring interface.

3. The method of claim 2, wherein after the first, second, third, and fourth monitoring interfaces are displayed on the remote monitoring device in a split screen manner, further comprising:

responding to a shooting angle switching instruction triggered by a user based on the shooting angle switching control, acquiring a moving picture corresponding to the switched shooting angle at the current position, and acquiring a switched moving picture;

and updating the moving picture currently displayed in the fourth monitoring interface to the moving picture after switching.

4. The method of claim 2, wherein the third monitoring interface further comprises a screen switch control associated with each track point, and wherein after the first, second, third, and fourth monitoring interfaces are displayed on the remote monitoring device in a split screen, further comprising:

responding to a moving picture switching instruction triggered by clicking the track point by a user, and acquiring a moving picture corresponding to the track point clicked by the user through the picture switching control to obtain a switched moving picture;

5. The method according to any one of claims 2 to 4, wherein the fourth monitoring interface comprises a forward-switch control and/or a backward-switch control, and wherein after the first, second, third and fourth monitoring interfaces are displayed on the remote monitoring device in a split screen, further comprising:

responding to a moving picture switching instruction triggered by a user through the forward switching control, acquiring a previous moving picture of a moving picture currently displayed in a fourth monitoring interface, and updating the moving picture currently displayed in the fourth monitoring interface into the previous moving picture; generating a thumbnail of the previous moving picture by using the thumbnail display control, determining a track point corresponding to the previous moving picture, displaying the thumbnail of the previous moving picture on a moving track in a third monitoring interface according to the determined track point, and stopping displaying the originally displayed thumbnail; determining a picture shooting angle of the last moving picture by using the shooting angle switching control, and marking the picture shooting angle on a moving track in a third monitoring interface according to the determined track point; and/or the number of the groups of groups,

6. The method of any of claims 1-4, wherein the payment information comprises paid merchandise information and the third monitoring interface comprises a payment information display control, the method further comprising, after the first, second, third, and fourth monitoring interfaces are displayed on the remote monitoring device in a split screen:

Generating thumbnail information of the paid commodity information;

determining a corresponding commodity display area according to paid commodity information, and determining a track point corresponding to the commodity display area, wherein the commodity display area is positioned in a monitoring area;

and displaying the thumbnail information on the active track in the third monitoring interface by utilizing the payment information display control according to the determined track point.

7. The method according to any one of claims 1 to 4, wherein the unattended store is provided with a store entrance and a store exit, and further comprising, before receiving the payment check message for the target object sent by the local monitoring device through the remote monitoring device:

when a target object enters the store entrance, collecting characteristic information of the target object;

when the target object enters the store exit, a video frame corresponding to the target object is called from a monitoring video shot by local monitoring equipment according to the characteristic information;

generating a moving track of the target object in the unattended store and a moving picture of the target object in a monitoring area according to the video frame;

acquiring payment information corresponding to the target object;

Allowing the activity tracking object to pass through the store exit when it is determined that a payment check operation is not required based on the payment information, activity trajectory, and activity screen;

and when the payment checking operation is determined to be required based on the payment information, the moving track and the moving picture, sending a payment checking message aiming at the target object to a remote monitoring device through a local monitoring device.

8. The method of claim 7, wherein generating an activity trajectory of the target object in the unattended store and an activity screen of the target object within a monitored area from the video frame comprises:

determining position information and time information of the target object in a store according to the video frame;

screening out the position information in the monitoring area to obtain the screened position information;

generating track points of the target object in a preset shop map according to the screened position information;

connecting the track points according to the sequence of the time information to obtain the moving track of the target object in the store;

and screening out video frames of which the target object is positioned in the monitoring area and the stay time is longer than a set value according to the position information and the time information, and obtaining a moving picture of the target object in the monitoring area.

9. The method of claim 7, wherein the payment information includes paid article information, a payment amount, and a number of paid articles, and determining whether a payment checking operation is required based on the payment information, the movement track, and the movement picture comprises:

if the payment amount in the payment information is smaller than the second threshold value or the number of the paid commodities is larger than the third threshold value, determining that payment checking operation is needed;

if the payment amount in the payment information is greater than or equal to a second threshold value and the number of the paid commodities is less than or equal to a third threshold value, determining a corresponding commodity display area according to the paid commodity information, wherein the commodity display area is positioned in the monitoring area;

acquiring the duty ratio of the commodity display area in the movable track;

if the duty ratio is smaller than a first threshold value, determining that payment checking operation is required;

and if the duty ratio is larger than or equal to a first threshold value, determining that the payment checking operation is not needed.

10. A display device for an unattended store monitoring interface, comprising: