WO2022074069A2 - Dispositif d'affichage de commerce et procédé de traitement d'images capturées par ce dernier - Google Patents

Dispositif d'affichage de commerce et procédé de traitement d'images capturées par ce dernier Download PDF

Info

Publication number
WO2022074069A2
WO2022074069A2 PCT/EP2021/077603 EP2021077603W WO2022074069A2 WO 2022074069 A2 WO2022074069 A2 WO 2022074069A2 EP 2021077603 W EP2021077603 W EP 2021077603W WO 2022074069 A2 WO2022074069 A2 WO 2022074069A2
Authority
WO
WIPO (PCT)
Prior art keywords
display device
commerce
customer
product
multimedia content
Prior art date
Application number
PCT/EP2021/077603
Other languages
English (en)
Other versions
WO2022074069A3 (fr
Inventor
Jaime Guixeres Provinciale
Mariano Luis ALCAÑIZ RAYA
Javier MARÍN MORALES
José Enrique Bigné Alcañiz
Original Assignee
Quatechnion S.L.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Quatechnion S.L. filed Critical Quatechnion S.L.
Publication of WO2022074069A2 publication Critical patent/WO2022074069A2/fr
Publication of WO2022074069A3 publication Critical patent/WO2022074069A3/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/02—Marketing; Price estimation or determination; Fundraising

Definitions

  • the present patent application has as its object a display device for commerce, according to claim 1, as well as a method of processing captured images by it according to claim 14, both incorporating notable innovations and advantages.
  • multimedia content may be generic, in relation to the products on display, or present a certain degree of customization, depending on specific types of the product displayed on said shelves, and may also be based on the interaction of the customer with the device, depending on the interface that includes.
  • a system includes one or more display units configured to couple to the shelves of a modular shelving system, one or more processors communicatively coupled to the one or more display units, one or more memory components communicatively coupled to one or more processors , and a series of instructions stored in one or more memory components.
  • the instructions When executed by one or more processors, the instructions cause the system to determine a product identifier corresponding to a product, identify one or more display units to update in response to determining the product identifier corresponding to the product, and update a graph output of one or more display units identified to update in response to the identification of one or more display units to update.
  • a display rack can have a projector and a computer.
  • the projector projects the luminous flux of the projector image.
  • the screen shelf may have a reflective mirror that conducts the image flow from the projector to the screens.
  • the present invention consists of a display device for commerce and for tourist spaces, and an associated method of processing captured images by it.
  • Said display device integrates a layer or level of stimulation processing and a layer or level of sensorization processing, or detection by means of sensors, so that the information presented on said smart display can be personalized in real time in order to maximize the interest of consumers in the exposed products, additionally using the previous data obtained from the behaviour of those or other customers.
  • the display device for commerce of the present invention that comprises emission means of multimedia content associated with a product offered by the commerce, towards at least one customer of said commerce; acquisition means of captured images of the physical environment of said commerce; processing means of said captured images of said physical environment; additionally comprising behavioural recognition means, or behaviour, of at least one customer present in said commerce, configured to monitor their spatial and/or gestural behaviour in the physical environment of the display device, as well as for the recognition of their demographic attributes and/or your emotional response.
  • the information about the product offered to said customer may include advertising multimedia content, recommendation of the products of interest and detailed personalization of the product information.
  • the emission means of multimedia content are specifically projection means of emitted images, alternatively being emission means of sound.
  • tracking is understood a tracking of your physical position, and/or of your movements, and therefore of whether or not you make a stop before the display device, or of your body movements, specifically the orientation of your head and look, or facial expression, particularly and preferentially in the physical environment of the display device.
  • demographic attributes should be understood, in a general and non-limiting way, age, gender and race.
  • the processing means of said captured images optionally comprise selection means of captured images, from among the captured images of said physical environment.
  • the acquisition means of captured images can be unique, both to detect an interaction with the display device, as well as to perform facial recognition, such as presenting some first acquisition means, specific to detect an interaction with the display device, and on the other hand a second acquisition means to perform facial recognition.
  • the display device for commerce specifically includes interaction detection means of the customer with said display device, so that the display device reacts by showing one or the other multimedia content based on the customer's indications, personalizing the information to the specific concerns of the customer activating the display device commands.
  • the acquisition means of captured images comprise interaction detection means such as a first camera to detect interaction, comprising preferably, on the other hand, a second camera to detect and record the behaviour or conduct of the customer, performing a behavioural recognition, or of the behaviour itself.
  • interaction detection means such as a first camera to detect interaction, comprising preferably, on the other hand, a second camera to detect and record the behaviour or conduct of the customer, performing a behavioural recognition, or of the behaviour itself.
  • the behavioural recognition means of at least one customer, present in said commerce are configured to monitor and/or to track the interaction of the customer with the objects or products arranged in the display device, that is, whether it takes or leave a product, and the time he has it in his hand, being also and/or alternatively configured to monitor and/or to track the user interaction with touch projection means integrated into the display device.
  • the multimedia content emission means of the display device are a touch screen, said touch screen being tactilely interactive by the customer, so that the customer can select the information options that are most of interest to him, in view of the exposed products.
  • the display device can optionally comprise at least one infrared frame configured to detect the passage of some physical element through it.
  • the display device is enabled to recognize the action of the customer to bring his hand, or another object, to the exposed product, or to the screen of the display device where information about it is presented.
  • Said infrared frame is, optionally, capable of detecting the position or coordinates, by which the infrared frame has been crossed.
  • said infrared frame can be placed adjacent to a glazed surface or to a screen, providing it with tactility, in the sense of having the capability to detect an object that approaches until it touches the glazed surface.
  • the emission means of multimedia content and/or the acquisition means of captured images and/or the processing means of said captured images are connected to control means, so that it can be done a regulated management of said resources, under some parameters programmed in advance.
  • control means are optionally located on a remote server.
  • emission means of multimedia content and/or the acquisition means of captured images and/or the processing means and/or control means are optionally connected to a database, either locally or remotely.
  • the connection to the control means is preferably through a 4G or 5G router or modem, establishing the connection through the internet, benefiting from the benefits and advantages of said infrastructure.
  • the multimedia content emission means associated with a product are, optionally, at least one rendering device of virtual reality, so that the customer can receive information about the product in a more complete and detailed way, recreating before their eyes a physical environment different from the real one.
  • the rendering device of virtual reality shows information associated with the selected product, either through an external touch interface, or directly by picking up the product.
  • image acquisition means, or internal optical-type sensors, of virtual reality glasses are used, which have the capacity to detect movement, to collect information on the visual attention, or direction of visual focus, of the customer, and his interaction with the virtual reality content shown in the virtual reality glasses.
  • the customer can interact through gestures, for example with their hands, when captured by image acquisition means, without having to touch the screen, thus being optional a touch screen.
  • the display device for commerce comprises at least one support structure that is configured to support at least one product of said commerce, so that said product is placed safely in front of the customer, who can, optionally, take it and return it to its place.
  • the display device optionally comprises at least one RFID antenna configured to establish a wireless connection with at least one RFID tag located on the product, so the display device can detect the action of taking the product by the customer, optionally triggering the specific presentation of multimedia content.
  • the RFID antenna is located in a support structure of the display device, preferably in a support zone for said product.
  • it comprises a common RFID antenna for all the products of the display device, and alternatively it comprises an individual RFID antenna for each support zone of each product.
  • the display device includes lighting means of the product in its support zone on the support structure, so that the visibility of the details of the product by the customer is greater.
  • the lighting means are RGB LED diodes configured to be controlled by control means, so that it is possible to select and vary the colour of the light emitted over the product.
  • the display device comprises a support structure in a substantially horizontal position, and emission means of multimedia content in a substantially vertical position, so that the multimedia content is shown in the most appropriate orientation, according to the direction from a substantially horizontal view of the customer towards the product, usually placed vertically, on the support structure.
  • emission means of multimedia content are specifically projection means of emitted images, preferably located behind the support structure, being perceived as behind, as in a back plane, according to the customer's observation direction.
  • the display device comprises a support structure in a substantially horizontal position, and emission means of multimedia content in a substantially horizontal position, so that the multimedia content is shown in the most appropriate orientation, according to the direction from a substantially vertical look from top to bottom of the customer towards the placed product, on the support structure.
  • the support structure can be substantially transparent, specifically a glass, and the emission means of multimedia content, preferably projection means of emitted images, are located below the support structure.
  • the support structure is the surface of the projection means of emitted images.
  • the display device comprises emission means of multimedia content associated with a product, in a substantially vertical position, and at least one touch screen configured for interaction with the customer, so that the customer can operate with the touch screen, influencing the multimedia content received, in relation to the product of his choice.
  • said touch screen is located below of the emission means of the multimedia content, and in an inclined position, substantially perpendicular to the direction of the gaze of a customer located in front of the display device.
  • the menu of options towards the customer and the multimedia content presented can be designed autonomously, offering greater possibilities of interaction.
  • the display device comprises collection means of environmental data in the physical environment of a display device, where the environmental data are related to at least one variable from the group of temperature, humidity, ambient noise and/or lighting, so that it is possible to further customize the multimedia content presented to the customer, while environmental data information in the physical environment of the display device can be collected to process them, eventually, centrally, before a multiple network of display devices, thus establishing a platform for collecting environmental data in the physical store environment, with a variable number of display devices, connected to each other, for example through the internet.
  • Another object of the present invention is an associated method of processing captured images by a display device for commerce, which comprises the steps of i) capturing captured images from the physical environment of said commerce; ii) detection of a customer of said commerce; iii) monitoring the physical position and/or movements and/or body movements and/or facial expression of said customer of the commerce; iv) obtaining a plurality of metrics associated with the physical position and/or the movements and/or the body movements and/or the facial expression of said customer of the commerce; v) determination of the interest degree of the customer in at least one product presented in the display device based on said metrics.
  • the plurality of metrics can be associated with interaction and visual attention, or direction of visual focus, in virtual reality contents, as well as interest degree and focus in the case of images and/or stimuli presented in virtual reality glasses.
  • monitoring the body movements and/or facial expression of the customer comprises monitoring at least one variable from the group of yaw, pitch, roll, proximity and time looking at acquisition means of captured images. In this way, it is possible to more accurately calibrate the customer interest with respect to a certain exposed product.
  • the determination of the customer's interest degree in at least one product presented on the display device is based on a linear combination of the number of stops with a glance in front of the display device, with the number of interactions with the product, and with the number of touches on a touch screen and/or with the number of interactions, and optionally with the degree of visual attention, or measurement of visual focus, in a rendering device of virtual reality.
  • the quantitative value of the variable interest degree of the customer is determined with high precision, by considering a plurality of factors, all of them directly related to a greater probability that the customer acquires the exposed product.
  • a preferred linear combination of stops with a glance, 'picks', or interactions with the product, and 'clicks', or number of touches on a touch screen and/or number of interactions on a rendering device of virtual reality is specifically the sum of stops with a gaze, plus the number of 'picks' multiplied by a factor of 1.5, plus the number of 'clicks' multiplied by a factor of 2.
  • the determination of the interest degree of the customer in at least one product presented in the display device is a function of the location of the display device, the environmental and temporal conditions, calculated from the training of supervised 'machine learning' algorithms in view of the historical data of said interest degree.
  • the interest degree of the customer is estimated based on conditions such as type of audience, location, based on what happened in the past.
  • the determination of the interest degree of each audience group, such as a plurality of customers, in at least one product presented on the display device is a function of the location of the display device, the environmental and temporal conditions, being calculated from the training of supervised 'machine learning' algorithms in view of the historical data of said interest degree.
  • interactions with the product can take place through a rendering device of virtual reality, specifically VR virtual reality glasses, which can display information associated with the selected product, either through an external touch interface or directly by picking up the product.
  • a rendering device of virtual reality specifically VR virtual reality glasses, which can display information associated with the selected product, either through an external touch interface or directly by picking up the product.
  • the indicator of degree of interest with virtual reality content is obtained by analyzing the content previously segmented into interest areas.
  • a numerical indicator updated minute by minute is obtained.
  • said first and second weighting factors are readjusted between values that vary from 0.5 to 1 depending on the environment in which the display device is located, adapting specifically when, for example, tourist-type content or advertising-type content. 360° are shown.
  • the visual focus indicator with the virtual reality content it should be noted that it is obtained through minute by minute obtaining of the linear combination of the total time of the visual focus on the interest areas, in which the surrounding space is segmented, that are located in the 25% or first quartile of areas with greater visual attention or visual focus, as the first term, multiplied by a first weighting factor, plus, as the second term, the number of interactions of the interest areas, in which It segments the surrounding space, which is located in the 25% or first quartile of areas with the greatest interaction, multiplied by a second weighting factor.
  • Figure 1 Schematic view of a plurality of 'Phygi' display devices connected to a remote server, equipped with artificial intelligence, according to the present invention
  • Figure 2. Schematic view of a block diagram with a display device, connected to a server and a database, including means control and processing of captured images, according to the present invention
  • FIG. 1 Schematic view of RFID tags in the vicinity of RFID antennas in connection with control means, according to the present invention.
  • Figure 4 Perspective view of a first model of the display device, as an interactive shelf, according to the present invention.
  • Figure 5 Perspective view of a second model of the display device, as an interactive shelf, according to the present invention.
  • Figure 6. Perspective view of a third model of the display device, like a showcase, according to the present invention.
  • Figure 7. Perspective view of a fourth model of the display device, as a virtual reality totem, according to the present invention.
  • Figure 8. Perspective view of a first implementation of the first model of the display device, as an interactive shelf, according to the present invention;
  • Figure 9. Perspective view of a second implementation of the first model of the display device, as an interactive shelf, according to the present invention.
  • Figure 11. Perspective view of a customer interacting with interest areas with virtual reality content, according to the present invention.
  • the invention can consist of a plurality of 'Phygi' display devices, each with its physical environment (3), connected to a remote server (8), optionally in the cloud, optionally equipped artificial intelligence.
  • the connection (82) is usually via the internet (83).
  • each unit or display device being connected by 4G, or successive generations of mobile telephony, to a server (8) in the cloud, which indicates what to show in every moment, and where all the information captured is sent to be analyzed.
  • the invention comprises a display device of at least one product (1), connected to a server (8) and a database (81), including acquisition means (41) of captured images (41a), as well as control means (47), processing means (42), selection means (43) and behavioural recognition means (44) and interaction detection means (45), through which a series is reached of metrics (91) of the interest degree (92) of the customer (2) with respect to the product (1), in particular as a result of the calculation of a linear combination (93) of detected variables related to the movements and behaviour of the customer (2 ) in the physical environment (3) of the display device.
  • the display device of the invention comprises at least one RFID tag (73) in the vicinity of at least one RFID antenna (72) in connection with control means (47), including the controllers and drivers of said RFID antenna (72).
  • Figure 4 shows a perspective view of a first model of the display device, as an interactive shelf, optionally including at least one light source, preferably LED, on the screen or emission means (40) of multimedia content (40a).
  • the objective in this first model is to highlight high- end products (1) and provide increased information on them, in addition to improving the shopping experience.
  • This embodiment includes a 43" capacitive screen, an RGB-D camera to detect the interaction, an RGB camera for facial recognition and a computer to process the information. It is designed to place a series of products (1), depending on their size, while multimedia content (40a) is emitted through the emission means (40) or screen.
  • the interaction is detected by means of a camera with a computer vision algorithm, or artificial vision, located in an overhead manner, and multimedia content is launched (40a) particular in addition to increased product information. It includes a touch screen (40b) that can be interacted due to its tactility.
  • the facial recognition camera allows to detect and characterize the audience that interacts with the display device.
  • the operation is controlled by an application developed in a graphics engine that connects to the management platform and shows the information of the products (1) exposed when they are picked up by customers (2), recognizing this interaction through the interaction of a RFID antenna (72) and at least one RFID tag (73). All the information of the products (1) and the multimedia content (40a) are in the database (81), and can be updated at any time in a centralized manner.
  • Figure 5 shows a perspective view of a second model of the display device, also as an interactive shelf, in which one or more acquisition means (41) can be located per display device, in the upper position, in one side, or another that is deemed convenient.
  • it comprises some lighting means (48) either in the central space, where the emission means (40) of multimedia content (40a) are found, although it is more common for said lighting means (48) to be located in the adjoining cells, also with products (1), but without emission means (40) of multimedia content (40a), as a substitute for them.
  • it comprises an infrared frame (5) at the outer edge of the central space with the emission means (40) of multimedia content (40a).
  • said second model of display device is preferably specified in a multiple vertical shelf, where a wide range of products (1) can be highlighted in a sales space. It also has the objective of encouraging consumption, improving the customer experience (2) and obtaining information on their behaviour.
  • Some emission means (40) of multimedia content (40a) are included, specifically a 65" screen, placed in the background in a vertical orientation, on which the products (1) are placed in front on shelves. It is optionally placed, in front of the screen and the products (1), an infrared frame (5), as an alternative to RFID detection technology, in order to detect when a customer (2) introduces their hand, or physical element (51), to take a product (1). This interaction produces a change in the content, giving information about the product (1) with which it has interacted and creating an experience around the purchase.
  • acquisition means (41) specifically a facial recognition camera to detect the audience and characterize it.
  • the operation of the display device mode as multiple vertical shelving is controlled, like the embodiment of figure 4, by an application developed in a graphics engine with similar features.
  • Figure 6 shows a perspective view of a third model of the display device, like a showcase, which can be horizontal or vertical, although it is the horizontal case which is shown, as a preferred embodiment.
  • the difference with the models in figures 4 and 5 is that it uses an external glass surface that protects the product (1) and makes direct access by the customer (2) impossible. Said exterior glass surface can be endowed with interactive features, by placing an infrared frame (5) located on its perimeter.
  • said display device model can be classified as an interactive digital display case.
  • an emission means (40) of multimedia content (40a) or a 43" screen that improves the shopping experience of the product (1).
  • Products (1) that want to be displayed are placed on top of said screen.
  • a clear example is expensive watches or jewellery, which is not directly accessible by the consumer or customer (2).
  • an infrared frame (5) is placed on the glass located in the upper part of the showcase, giving it tactility, thus making the screen interactive through the upper glass, creating an positive experiential effect for the customer (2).
  • This model is more oriented towards luxury items, exhibitors at fairs, events, etc.
  • it has a facial recognition camera to detect the audience and characterize it.
  • the operation of the display device mode as multiple vertical shelving is controlled, like the embodiment of figures 4 and 5, by an application developed in a graphics engine with similar features, connecting with the management platform and displaying the information of the products (1) exposed when picked up by customers (2).
  • the interaction is achieved by tapping on the top of the display case, since the products cannot be picked up.
  • All the information of the products (1) and the multimedia content (40a) can be in the same way in the database (81) and can be updated at any time in a centralized way.
  • Figure 7 shows a perspective view of a fourth model of the display device, as a virtual reality totem, optionally including one or more acquisition means (41).
  • a rendering device of virtual reality (6) is included, specifically virtual reality glasses (61), supported on a base that enables their electrical charge by contact, while holding them. Additionally it has an alarm that activates if the customer (2) is separated for example more than 3 meters from the display device, warning for example previously at 2 meters.
  • a touch screen (40b) is included to select the experiences, and preferably several virtual reality representation devices (6), or virtual reality glasses (61), to be used by customers (2).
  • Said virtual reality glasses (61) are designed to show images or video, optionally at 360°, and/or showing 3D scenes.
  • the functionality is managed by an application developed on a graphics engine that connects to the management platform, optionally located remotely on a server (8) and in a database (81), establishing communication through a connection (82) via the internet (83).
  • the information of the multimedia content (40a) and of virtual reality are uploaded through said platform, such as tourist videos depending on the place, images of cars or any other product (1) or service to be exposed. Similarly, all the information is in the database (81) and can be updated centrally at any time.
  • said display device model like a totem preferentially presents an upper screen, such as emission means (40) of multimedia content (40a), and a lower touch screen intended for interaction with the customer (2), which shows the products (1) and/or experiences to be visualized. Specifically, the experiences have to be dragged to the representative icon of each virtual reality glasses (61), leaving said virtual reality glasses (61) ready to view the experience.
  • an upper screen such as emission means (40) of multimedia content (40a)
  • a lower touch screen intended for interaction with the customer (2), which shows the products (1) and/or experiences to be visualized.
  • the experiences have to be dragged to the representative icon of each virtual reality glasses (61), leaving said virtual reality glasses (61) ready to view the experience.
  • Figure 8 shows a first example of implementation of the first model of the display device, as an interactive shelf, specifically presented wine products (1).
  • FIG 9 a second example of implementation of the first model of the display device, as an interactive shelf, specifically presenting sports footwear products (1), with a customer (2) taking a product (1), while the acquisition means (41) of captured images (41a) corresponding to a customer (1) perform their function, thus making it possible to improve the shopping experience.
  • FIG 10 it can be seen an example of implementation of the second model of the display device, as an interactive shelf, optionally without support structures (7), specifically presenting interior design products (1), with a customer (2) taking a product (1), while the acquisition means (41) of captured images (41a) corresponding to a customer (1) perform their function, thus making it possible to improve the shopping experience.
  • figure 11 it can be seen, a customer (2) interacting with the interest areas (95) with virtual reality content (94), said customer (2) wearing virtual reality glasses (61) that virtually present said virtual reality content (94), while they can detect through some acquisition means (41), optionally located in the virtual reality glasses (61) themselves, some captured images (41a) and the successive directions of the visual focus (96) of said customer (2), being able to establish the number of interactions (97) with the different virtual reality content (94), and thus the interest degree (92) in the various products (1) shown through the display device, in its various realizations, said figure 11 representing a 360° virtual spatial physical environment (3).
  • said display device integrates a layer or level of sensorization processing, or detection by means of sensors, focused first to the control of the audience, by means of vision by computer, and secondly to the control of the interaction, by means of RFID identification and interactions with the screens of the display device.
  • a series of metrics (91) are obtained from the computer vision for each user customer (2).
  • the system comprises several serial processes: detection, tracking, and obtaining metrics (91).
  • Detection is preferably carried out with a library called OpenVINO, developed by Intel Corporation. Mention that facial detection models will be used, for example those related to gender, age and/or emotional state.
  • tracking a unique and consistent identifier is assigned to each customer user (2).
  • the temporal processing of the different variables obtained previously or metrics (91) are associated with a subject identified as customer user (2), for their follow-up.
  • the tracking algorithms are built sequentially in order not to lose track of the subject identified as user customer (2).
  • metrics (91) as variables corresponding to each subject of the audience, or group of users customers (2) in a given physical environment (3), they preferably comprise a) three different angles of facial orientation, such as yaw, pitch and roll; b) proximity to the display device; and c) if user customer (2) has been looking at the camera.
  • the metrics (91) per user customer (2) are registered in a database (81)
  • the detection is carried out with a computer vision algorithm, whose input or input is an RGB camera, or acquisition means (41), and whose output or output is called Raw Data 1, or raw data 1 , as shown in figure 2.
  • the Python Script 1 , or SmartShelf algorithm of the display device generates Raw Data 2, or raw data 2, from Raw Data 1.
  • the Python 2 Script, or Server algorithm present in the server (8), processes the uploaded data and generates a set of metrics (91) processed and classified. Additionally, it uploads the processed data to the database (81), preferably of the SQL type.
  • a layer of analysis of the audience of the display device, or smart shelf has been developed using computer vision. This has been developed to analyze videos in real time, although it allows its application in post-processing to analyze videos already captured, in order to diagnose possible errors and improve the solution.
  • the analysis consists of obtaining different metrics (91) for each user or customer (2) that has appeared in the video in a specific space of time.
  • the developed system has serial processes, specifically those of detection, tracking or monitoring and obtaining metrics. More specifically regarding the detection process, different Machine Learning models, or automatic learning models, are used, specifically Deep Learning, among others that can perform car detection and automatic license plate reading, people detection or face detection. These models work in real time, both for a video and for an image.
  • facial detection models will be used, such as the detection of age and gender, the mood of a person or the orientation of their head.
  • a series of measures related to gender, age, emotional state (happy, neutral, sad, angry or surprised) and the orientation of the head of each subject or customer (2) are obtained.
  • a model has also been incorporated which detects the size and position of a human face, on the studied video, according to a square.
  • face detection models analyze each image independently, so they cannot track people or customers (2). By detecting faces at a given moment, but without an identifier allowing the same face to be identified between frames, we can know how many faces, and by extension people or customers (2), there are at any given moment, but not how many different faces have passed during a period. Therefore, it is necessary to implement an algorithm that performs a tracking or follow-up, that is, that allows assigning a unique and consistent identifier to people or customers (2) over time.
  • the detection frequency is approximately 25Hz-20Hz, the variation depends on the number of subjects or customers (2) in the scene or physical environment (3). So the more subjects or customers (2), the more processing and therefore the frequency decreases.
  • the detections are saved in real time in a table format.
  • the continuous capture from the camera or acquisition means (41) is segmented into sessions, which are defined from since a person or customer (2) enters the scene, until none face is detected for 1 second.
  • the detection during this session is exported, preferably in Excel format, for further processing.
  • the detection is cut by saving the data of the three-minute session and restarting the detection algorithm automatically.
  • Said data is stored in a temporary format where to each piece of data it is assigned the time in which it was obtained, without having a coherence with the subject or customer (2) on which a specific measurement has been made.
  • the second processing algorithm is used: the one of tracking.
  • the second algorithm or tracking algorithm it deals with the temporal processing of the different variables obtained previously, associating them with a subject or customer (2) identified with a unique integer number in the session.
  • a series of concatenated algorithms for tracking the subject or customer (2) are used, in order to associate the variables obtained from the first algorithm to the subjects or customers (2) in the scene or physical environment (3). These algorithms basically study the evolution of the position of the subjects as they move.
  • a unique identifier is associated with each initial subject throughout the session. By comparing two temporally-followed data sets, the initial identifiers are associated with the closest identifiers in the next data set. This association of identifiers is not only done through proximity algorithms; Others are used that take into account, for example, the speed of the subjects or customers (2), and the optical flow.
  • these tracking algorithms are built sequentially, that is, first the data goes through a first algorithm, these results are processed by a second algorithm, etc.
  • Regression machine learning models have also been implemented that base their knowledge on the movement of the subject or customer (2), during the session and then predict their position a certain time later in order not to lose track of this person. This allows, for example, that, if a subject is not detected for a certain time in the scene and then it is detected again, the tracking of it is not lost. That is, the future position of the subject or customer (2) is interpolated to continue with its monitoring.
  • two different subjects may have the same identifier, for example as a result of two subjects or customers (2) crossing each other during a long time interval, which may cause the algorithm to assign the same identifier to two subjects or customers (2).
  • the subjects or customers (2) with the same identifier are reviewed, assigning the repeated identifier to that subject or customer (2) who has had a trajectory as similar as possible to the prediction made.
  • the data of the orientation of the subject's head or customer (2) is processed, which determine three different angles called yaw, pitch and roll. These angles are calibrated by studying different videos and with this a threshold is set for when a subject or customer (2) is looking within a certain spatial range as an area of interest of the physical environment (3). The same is done for the distance to which the subject or customer (2) is.
  • the camera or acquisition means (41) that achieves the recording of videos does not have a depth component, which returns us some data that is on a really flat image, without a sense of depth. Despite this, it is possible to establish a parameter that marks the proximity of a subject or customer (2) with respect to the camera or acquisition means (41), thus segmenting the position of the subject or customer (2) in near and far.
  • an additional variable can be established that evaluates whether a person has been looking for a certain time within a viewing angle during the session. If he has been doing it, the metrics (91) related to this subject or customer (2) are obtained during the viewing time, eliminating the records where she has not been looking. Thus, the information derived from these subjects or customers (2) will be obtained only from instants of time where said subject or customer (2) has been looking at the camera or acquisition means (41), and therefore, these variables are much more robust than those that can be obtained by averaging over the entire session. More complex metrics (91) have also been studied, such as the sequence of movements of a subject or customer (2). This sequence determines the order of actions carried out by said subject or customer (2), differentiating between approaching, moving away, left and right, and looking or not looking. This allows the subjects or customers (2) of the same session to be grouped according to movement patterns.
  • the set of variables described above is grouped and summarized in a table that is automatically saved in a database (81), registering as an analysis by subject or customer (2) of a session.
  • a third data processing within the same algorithm, performs the analysis per session, thus reporting, for example, the number of men and women who have participated in it.
  • the various algorithms work in parallel so that the data collection of the first algorithm and the data processing by the second algorithm of already saved data are achieved at the same time. Since the second processing algorithm normally takes more time than the first, all the data to be processed is saved and recorded in a history in the database (81).
  • the data belonging to the metrics (91) by user or customer (2) are recorded in the database (81) in order to be able to graphically display the results achieved, and to generate automated reports.
  • these algorithms are a single program that can run on a single computer core, making it possible for the algorithm to run on as many cores as the computer has, doubling or quadrupling its speed.
  • the interactions with the touch screens (40b) are considered first, by means of light pulsations or touches, relating it to each product (1), and giving rise to the generation of metrics (91), for example with the number of touches.
  • radio frequency or RFID identification technology is considered, optionally with RFID HF sensors, and alternatively with RFID UHF sensors, integrated in the display device:
  • the products (1) comprise a tag or RFID tags (73) that make possible to know via the sensor if a tag is removed from the accessible field of the RFID antenna (72) or if it is returned to the initial position.
  • an antenna controller optionally of the type (XN), connected to control means (47) or control unit (Raspberry) which is the device that stores the algorithm to translate the signals and send the corresponding events to the computer that stores the application in charge of displaying the actions on the screen or emission means (40).
  • control means (47) or control unit (Raspberry) which is the device that stores the algorithm to translate the signals and send the corresponding events to the computer that stores the application in charge of displaying the actions on the screen or emission means (40).
  • it has tags in the products, or RFID tag (73), and RFID antennas (72), with a specific driver or controller, connected to the control means (47) or control unit (Raspberry).
  • it is preferably connected to a 4G router through which the containing computer in charge of launching the application is connected.
  • the screens used to capture the interaction with the user or customer (2) are tactile, in order to receive inputs based on the interaction of each user or customer (2) with the touch screen (40b), relating it to each product (1). All this is collected in the application, generating metrics (91) representative of the interest degree, such as the number of times each information tab relative to each product (1) is clicked or pressed.
  • RFID antennas (72) there must be a minimum distance between RFID antennas (72) to avoid interference and erroneous detections. Said minimum distance must be 8.5 cm horizontally and 6 cm vertically so as not to cause interference either with the metal that is part of the display device.
  • each RFID antenna (72) has a specific detection range, therefore, when installing the components, it is necessary to consider that the distance from the tag or RFID tag (73) to the RFID antenna (72) is within its detection range. Specifically the maximum distance is 2.2 cm.
  • the RFID antenna controllers (72) are directly connected to the control means (47), which is the device that stores the algorithm to translate the signals and send the corresponding events to the computer that stores the application in charge of displaying the actions by the emission means (40) or screen.
  • the delivery system is carried out through MQTT (Message Queing Telemetry Transport, or Message Queue Telemetry Transport), which is an M2M (machine to machine) communication protocol. This protocol is based on a TCP I IP stack and each connection is kept open.
  • Said system is connected to a 4G router through which the containing computer in charge of launching the application that will interpret the MQTT messages received by the control means (47), is connected to use them.
  • the application is developed in a graphics engine, which will receive messages through MQTT as the interactions are recorded, and will translate them into different situations on the emission means (40) or screen.
  • the report is generated in an advanced data visualization service, it is published within a workspace, where reports, dashboards, data sets, etc. can be shared, with end users or customers (2), by connecting to SQL Server.
  • the import or load of the entire data model in memory is implemented, as it offers greater flexibility, both in data manipulation and in the implementation of calculations.
  • a higher performance is obtained since when performing an update of the data, these are stored in the memory of the capacity of the advanced data visualization service, allowing queries to be carried out more quickly; and finally, a single data source is used.
  • the set of represented data is not updated in real time although multiple updates can be scheduled to run throughout the day.
  • a filter panel applicable to the current tab This area allows segmenting the information displayed, limiting the set of data that is represented in the viewing area; b) an area for displaying the information extracted from a data set, which may contain one or more visual objects. And c) a corporate logo.
  • the report comprises different tabs, the main ones being the following: I) ‘Location’ tab: Shows the main indicators, as well as the geolocation of the different display devices. It visually includes different informative objects such as i) the date, allowing to limit the data displayed within a range of time; ii) the location, limiting the set of data represented according to the location of the display devices that are part of the commercial campaign, presenting multiple information fields, such as autonomous community, province, municipality, establishment or surface where the device is located ; iii) display of metrics (91), as indicators related to the efficiency and productivity of the display device; iv) location map, showing the geolocation of the establishment where the different display devices are located.
  • I ‘Location’ tab: Shows the main indicators, as well as the geolocation of the different display devices. It visually includes different informative objects such as i) the date, allowing to limit the data displayed within a range of time; ii) the location, limiting the set of data represented according to the location of the display devices that are part of
  • a graphic resource is the size of the circle, which specifies its geographical location on the map, and represents the average daily potential audience, that is, the average number of daily visits in the selected time range; v) metrics (91) comparators in the form of tables, allowing to obtain a view of the status of the current day in comparison with the previous results.
  • ‘Traffic’ tab Shows the volume of traffic, as well as other characteristics, which can be extracted from the audience analysis. Visually includes different informational objects such as i) Date; ii) Location; iii) Gender; iv) Age, filtering the information represented in the metrics (91) display area according to the age range of the subjects, preferably ⁇ 18; 19-30; 31-45; 46-55; 56-65; or >65; v) Distance; vi) Purchase opportunities, graphically showing the behaviour of potential buyers or customers (2).
  • informational objects such as i) Date; ii) Location; iii) Gender; iv) Age, filtering the information represented in the metrics (91) display area according to the age range of the subjects, preferably ⁇ 18; 19-30; 31-45; 46-55; 56-65; or >65; v) Distance; vi) Purchase opportunities, graphically showing the behaviour of potential buyers or customers (2).
  • ‘Buyer profile’ tab Shows information on demographic characteristics of potential buyers or customers (2) allowing to specify objectives. Visually includes different informational objects such as i) Age; ii) Gender; iii) Population pyramid, making possible to jointly analyze the demographic characteristics of both gender and age.
  • Emotions tab Shows, in addition to demographic traits, the emotional response. Based on the probability that one of the registered emotions occurs, it establishes the percentage of time in which a buyer has mostly experienced an emotion of the type of happiness, surprise, sadness, anger or neutral... Visually includes different informational objects such as i) Emotional response, presenting the percentage of time in which an emotional response, either positive or negative, has occurred with respect to a neutral response; ii) Type of emotional response, indicating as a percentage, the total duration in which one of the nonneutral emotions has been registered; iii) Emotional response according to the day of the week.
  • VI) ‘TouchOnScreen’ tab Shows a perspective on interactivity on the touch screen (40b) of customers (2) with the display device. Visually includes different informational objects such as i) Dates; ii) Location; iii) Product (1), limiting the set of data represented according to the product (1) to which the touch is associated on the touch screen (40b), allowing a multiple selection; iv) Type of display device, not allowing multiple selection; v) Daily touches according to hour, vi) Touches on the different tabs; vii) Flow through the different tabs, allowing a quick reading of how the customers (2) have moved through the different tabs.
  • different informational objects such as i) Dates; ii) Location; iii) Product (1), limiting the set of data represented according to the product (1) to which the touch is associated on the touch screen (40b), allowing a multiple selection; iv) Type of display device, not allowing multiple selection; v) Daily touches according to hour, vi) Touches on the
  • ‘Content’ tab Shows information regarding the reproduction of different visual audiovisual content on display devices. Visually includes different informative objects such as i) Summary of indicators with the most significant metrics related to the content reproduced, specifically the potential audience per hour, the attraction rate per hour, interactions per hour or the time of reproduction per day.
  • the access of the end users or customers (2) to the reports specifically the access of the end users or customers (2) to the reports, the requirement that the end users or customers (2) have access only to the information related to the campaigns they are owners of, including a mechanism called row-level security, consists of the creation of roles that have associated rules and that restrict the access of end users or customers (2) to fragments of the data sets displayed in panels, reports, icons, etc. to which they have access.
  • the audience data that is, the interactions through the RFID system and the clicks on the touch screen (40b) are sent to a server (8) to be managed.
  • a server (8) to be managed Specifically, you can use the IIS (Internet Information Services) server hosted by one of the layers offered by the AWS cloud infrastructure (Amazon Web Services) and specifically in this case, the Elastic Beanstalk layer.
  • IIS Internet Information Services
  • AWS cloud infrastructure Amazon Web Services
  • SQL Server 2016 database there is a relational database (81) for use in applications, specifically a SQL Server 2016 database.
  • This database (81) is the one that stores all the information records and through which the information of the inputs and outputs. In turn, it contains tables distributed in different areas that are interrelated according to the purpose and nature of the data that are housed in them.
  • the information to be displayed will depend on the position and type of product (1), following some guidelines for managing RFID tags (73 ).
  • the information will disappear when it is left in the RFID antenna (72) where it was, or when a defined time has elapsed, which can be 5 minutes for example.
  • the information to be shown in the meantime would be that corresponding to the last product raised, that is, separated from its support position, in the vicinity of the RFID antenna (72).
  • the multimedia content (40a) to be shown can be based on the audience through modular rules that allow segmenting said multimedia content (40a).
  • the general functionalities with which the display devices can be supplied, once they are connected with the management platform, can be, in a non-limiting way: i) Connection with the platform to recognize the templates and multimedia content (40a) to be displayed.
  • the database (81) provides the information necessary for the operation of the display device, including modes, a list of associated reproductions, products (1) and the associated audiovisual multimedia content (40a); ii) RFID tag (73) management, including in the defined products (1) the RFID tag (73) number associated with the product (1); iii) Management of the list of associated reproductions, adapting the background to what is defined in the platform, with all the dynamic audiovisual multimedia content (40a).
  • the following audiovisual multimedia content (40a) are displayed:
  • the brand logo for example, top left of the screen, languages with flags icon for language selection, title, subtitle.
  • the videos associated with the chosen playlist contained in the multimedia content template (40a) appear in the background.
  • the information to be displayed is defined by templates, which contain tabs, and each one has a design and multimedia content information (40a).
  • templates which contain tabs, and each one has a design and multimedia content information (40a).
  • various informations will be displayed, corresponding to each of the products (1), sequentially, ending with the information of the last product (1) raised. And if all the products (1) are left in their position, the rest state is returned.
  • the visual functionalities can be intelligent, that is, based on the particular audience at every moment.
  • a connection has been made between the audience metrics (91) with the emission means (40).
  • a series of modular rules have been developed that allow segmenting multimedia content (40a) based on the estimated age and gender of the consumers or customers (2) who are in front of the display device.
  • this last layer or level of rule-based processing provides the display device with an intelligence that allows adapting the multimedia content (40a) stored in the platform for a specific brand or product (1) depending on the audience it has in every moment.
  • a first typology of Artificial Intelligence models would be the one that would establish a generic priority, based on the day and hour for the existing multimedia content (40a).
  • a second type of Artificial Intelligence models would be applied in real time, recognizing the dynamics of the audience at all times.
  • the concept of the interest rate, or Interest score is relevant, calculated as a linear combination of stops with glance, 'picks', or interactions with the product, and 'clicks', or number of touches.
  • supervised machine learning algorithms are used to create artificial intelligence models.
  • the objective is to predict which multimedia content (40a) is executed when no customer (2) interacts, given a range of time of a specific day.
  • the predictive model given prior information on the operation of multimedia content (40a) in a specific time frame, learns which ones have been the ones that maximize interest.
  • Said model can be run several times a day to change the probability of each of the available multimedia content (40a) being shown, thus emitting the ones that maximize interest more times.
  • the main inputs can be considered those of i) multimedia content (40a), with the identifier data, keywords, authors, influencers, etc .; ii) location, with city, region and country data; iii) Environmental information, with data on temperature, humidity and rainfall; iv) temporary, with the data of time slot, day of the week, month.
  • the value of the interest rate, or Interest score can be issued as output, calculated according to the formula (# StopswithGlance + 1.5 * picks + 2 * clicks]
  • the predictive model takes into consideration the previous information on the operation of multimedia content (40a) with a specific audience, learning which ones have been the ones that maximize interest. This model is executed several times a day to change the probability of each of the multimedia content (40a) available to be shown, depending on the audience that the display device may have in front of, for a given time frame. Once the results of the model have been pre-calculated, the probability of displaying multimedia content (40a) for a specific audience will change each time that audience standing in front of the display device is recognized, changing the multimedia content (40a) shown to adapt them to the audience.
  • the main inputs can be considered those of i) multimedia content (40a), with the identifier data, keywords, authors, influencers, etc .; ii) location, with city, region and country data; iii) environmental information, with data on temperature, humidity and rainfall; iv) storms, with the data of time slot, day of the week, month; v) type audience.
  • the value of the interest rate can be emitted, calculated according to the formula (# StopswithGlance + 1.5 * picks + 2 * clicks]
  • the predictive model Given prior information on the operation of content variants for the same product (1) that is shown to the user customer (2) and that is selectable by him, learns which ones have received the most interest from the subject customer (2).
  • This model is executed with the different variants of multimedia content (40a), and several times a day to change the probability of each of the available multimedia content (40a) being shown, depending on the audience that the display device may have in front of and a time slot.
  • the probability of displaying a type of variant of product information (1) for a specific audience will change each time that audience standing in front of the display device is recognized, changing the multimedia content (40a) that are displayed to suit the audience.
  • the different probabilities associated with audience combinations are calculated (young men, middle-aged men, older men, young women, middle-aged women, older women, young mixed gender, middle-aged mixed gender and older mixed gender), and also taking into account the parameters of that afternoon (Tuesday in February, from 5:00 p.m. to 9:00 p.m., cold, humid and rainy day, in a specific city).
  • the models do not have to be calculated at every moment, but the results are preloaded.
  • the display device detects that there are 3 middle-aged men standing in front of said display device, so it changes the probability of the contents associated with the product (1) to those that maximize the interest rate for that range of hearing that afternoon.
  • the information tab associated with product (1) is only one and the variants are produced in an emitted image (40c) that is shown of the product (1).
  • the model will learn which of the emitted images (40c) is the ideal one, based on the interest that could be measured in this case, by the positive/negative emotional facial response of the user customer (2) interacting with product (1).
  • the main inputs can be considered those of i) multimedia content (40a), with the identifier data, keywords; ii) location, with city, region and country data; iii) environmental information, with data on temperature, humidity and rainfall; iv) temporal, with the data of time slot, day of the week, month; v) type audience.
  • the value of the interest rate can be emitted, calculated according to the formula (# StopswithGlance + 1.5 * picks + 2 * clicks]
  • the display device for commerce that comprises emission means (40) of multimedia content (40a) associated with a product (1) offered by the commerce, towards at least one customer (2) of said commerce; acquisition means (41) of captured images (41a) of the physical environment (3) of said commerce; processing means (42) of said captured images (41a) of said physical environment (3); additionally comprising behavioural recognition means (44) of at least one customer (2) present in said commerce, configured to monitor their spatial and/or gestural behaviour in the physical environment (3) of the display device, as well as for the recognition of their demographic attributes and/or their emotional response.
  • the display device for commerce comprises interaction detection means (45) of the customer (2) with the display device. More in detail, as seen in figure 10, the emission means (40) of multimedia content (40a) are a touch screen (40b).
  • the display device optionally comprises at least one infrared frame (5) configured to detect the passage of some physical element (51) through it.
  • physical element (51) cite the hand of a customer (2).
  • the emission means (40) of multimedia content (40a) and/or the acquisition means (41) of captured images (41a) and/or the processing means (42) of said captured images (41a), are connected to control means (47). Additionally the display device is connected to the internet through a router.
  • the emission means (40) of multimedia content (40a) associated with a product (1) are at least one rendering device of virtual reality (6).
  • the display device comprises at least one support structure (7) configured to support at least one product (1) of said commerce.
  • the display device optionally comprises at least one RFID antenna (72) configured to establish a wireless connection (82) with at least one RFID tag (73) located on the product (1).
  • the display device comprises lighting means (48) of the product (1) in its support zone (71) on the support structure (7).
  • the display device for commerce comprises a support structure (7) in a substantially horizontal position, and emission means (40) of multimedia content (40a) in a substantially vertical position.
  • the display device for commerce comprises a support structure (7) in a substantially horizontal position, and emission means (40) of multimedia content (40a) in a substantially horizontal position.
  • the display device for commerce comprises emission means (40) of multimedia content (40a) associated with a product (1), in substantially position vertical and at least one touch screen (40b) configured for interaction with the customer (2).
  • the display device for commerce comprises collection means (49) of environmental data (49a) in the physical environment (3) of a display device, where the environmental data (49a) are relative to at least one variable from the group of temperature, humidity, ambient noise and/or lighting.
  • the environmental data (49a) are relative to at least one variable from the group of temperature, humidity, ambient noise and/or lighting. Its objective is to create a platform for collecting environmental data in the physical environment (3) of the store, which contemplates the characteristics of being scalable, modular, and distributed. It must be scalable, since the initial prototype can consist of a variable number of devices, data collectors and receivers. It has to be modular, since the task of collecting data in a location can be carried out in a single device or node, or separated into several according to the conditions of the physical environment (3).
  • the system will consist of a platform connected through the network, with no other requirement than internet access.
  • the part of the data receiving platform must have the capability to record and send the data to a remote server (8) or database (81) for subsequent analysis and obtaining metrics (91).
  • the basic approach is to obtain reads for temperature, humidity, ambient noise, and lighting.
  • Another object of the present invention is a method of processing captured images (41a) by a display device for commerce, which comprises the steps of i) capturing captured images (41a) from the physical environment (3) of said commerce; ii) detection of a customer (2) of said commerce; iii) monitoring the physical position and/or movements and/or body movements and/or facial expression of said customer (2) of the commerce; iv) obtaining a plurality of metrics (91) associated with the physical position and/or the movements and/or the body movements and/or the facial expression of said customer (2) of the commerce; v) determination of the interest degree (92) of the customer (2) in at least one product (1) presented in the display device based on said metrics (91).
  • the monitoring of the customer (2) it is tracked their physical position, their movements and, therefore, whether they make at least one stop, the orientation of their head and therefore their glance, of the interaction of said user customer (2) with the products (1) placed on the display device, and therefore of whether he takes or leaves a product (1) and the time he has it in his hand.
  • the interaction of the user customer (2) with the emission means (40) located on the display device and the interaction with a rendering device of virtual reality (6), specifically virtual reality glasses (61) of the type of Head mounted display, or head-mounted screen, located on the display device.
  • the behaviour information of the user customer (2) is processed during the visualization of the virtual reality content (94), optionally content multimedia (40a), which includes the monitoring of the head and the pupil, of the customer user (2), in order to determine the direction of his visual focus (96), that is, his visual attention, on the immersive stage posed, as a virtual physical environment (3), either static, with at least one image, or dynamic, with a video projection, or succession of images. From there, values are obtained for the complete viewing time of the exposed stimulus, or virtual reality content (94), detecting the time that the virtual reality glasses (61) are on the head.
  • Visual focus (96) is obtained by monitoring the orientation of the head, or by monitoring the pupil, determining the viewing time by spatial sectors, or interest areas (95), into which the stimulus is divided. This makes it possible to generate a attention map relative to the visual attention of each stimulus presented.
  • the number of touches or interactions (97) in internal interactive elements, or interest areas (95), of the virtual space is also collected.
  • Said stimulus processing method in the rendering device of virtual reality (6, 61) comprises the determination of the interest degree based on a linear combination of the number of different interest areas (95) visually explored per minute, and/or the number of interactions (97) per minute, and includes the determination of a focus indicator or metric (91) as a linear combination of the viewing times of the most viewed and focused interest areas (95), taking into account the attention in the 25% or first quartile of interest areas (95) of greatest attention.
  • the processing method of captured images (41a) comprises monitoring the body movements and/or facial expression of the customer (2) and includes monitoring at least one variable from the group of yaw, pitch, roll, proximity and time looking at some acquisition means (41) from captured images (41a).
  • said processing method of captured images (41a) comprises the determination of the interest degree (92) of the customer (2) in at least one product (1) presented on the display device, and is based on to a linear combination (93) of the number of stops with glance in front of the display device, with the number of interactions with the product (1), and with the number of touches on a touch screen (40b) and/or with the number of interactions in a rendering device of virtual reality (6).

Landscapes

  • Business, Economics & Management (AREA)
  • Strategic Management (AREA)
  • Engineering & Computer Science (AREA)
  • Accounting & Taxation (AREA)
  • Development Economics (AREA)
  • Finance (AREA)
  • Economics (AREA)
  • Game Theory and Decision Science (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Marketing (AREA)
  • Physics & Mathematics (AREA)
  • General Business, Economics & Management (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • User Interface Of Digital Computer (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
  • Controls And Circuits For Display Device (AREA)

Abstract

L'invention concerne un dispositif d'affichage de commerce qui comprend un moyen d'émission (40) de contenu multimédia (40a) associé à un produit (1) proposé par le commerce, à destination d'au moins un client (2) ; un moyen d'acquisition (41) d'images capturées (41a) de l'environnement physique (3) dudit commerce ; un moyen de traitement (42) desdites images capturées (41a) dudit environnement physique (3) ; et un moyen de reconnaissance de comportement (44) d'au moins un client (2), configuré pour en surveiller le comportement spatial et/ou gestuel dans l'environnement physique (3) du dispositif d'affichage, ainsi que pour la reconnaissance de ses attributs démographiques et/ou de sa réponse émotionnelle. L'invention concerne également un procédé de traitement associé d'images capturées (41a) par ledit dispositif d'affichage, de sorte qu'il capture des informations à partir de son environnement physique et les traite afin d'optimiser les informations à afficher de façon à élever au maximum l'intérêt du client pour le produit exposé (1).
PCT/EP2021/077603 2020-10-08 2021-10-06 Dispositif d'affichage de commerce et procédé de traitement d'images capturées par ce dernier WO2022074069A2 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
ESP202031022 2020-10-08
ES202031022A ES2818423A1 (es) 2020-10-08 2020-10-08 Dispositivo expositor para comercio y metodo de procesamiento de imagenes captadas por el mismo

Publications (2)

Publication Number Publication Date
WO2022074069A2 true WO2022074069A2 (fr) 2022-04-14
WO2022074069A3 WO2022074069A3 (fr) 2022-06-02

Family

ID=75375323

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2021/077603 WO2022074069A2 (fr) 2020-10-08 2021-10-06 Dispositif d'affichage de commerce et procédé de traitement d'images capturées par ce dernier

Country Status (2)

Country Link
ES (1) ES2818423A1 (fr)
WO (1) WO2022074069A2 (fr)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7798652B2 (en) 2006-03-28 2010-09-21 Toshiba Tec Kabushiki Kaisha Display shelf and display shelf system for projecting projector images displaying information about articles
US10580052B2 (en) 2015-05-04 2020-03-03 Sunrise R&D Holdings, Llc Systems and methods for controlling shelf display units and for graphically presenting information on shelf display units

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7921036B1 (en) * 2002-04-30 2011-04-05 Videomining Corporation Method and system for dynamically targeting content based on automatic demographics and behavior analysis
US20120200601A1 (en) * 2010-02-28 2012-08-09 Osterhout Group, Inc. Ar glasses with state triggered eye control interaction with advertising facility
US20140365272A1 (en) * 2013-06-07 2014-12-11 Bby Solutions, Inc. Product display with emotion prediction analytics
US20140363059A1 (en) * 2013-06-07 2014-12-11 Bby Solutions, Inc. Retail customer service interaction system and method
US9990438B2 (en) * 2014-03-13 2018-06-05 Ebay Inc. Customized fitting room environment

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7798652B2 (en) 2006-03-28 2010-09-21 Toshiba Tec Kabushiki Kaisha Display shelf and display shelf system for projecting projector images displaying information about articles
US10580052B2 (en) 2015-05-04 2020-03-03 Sunrise R&D Holdings, Llc Systems and methods for controlling shelf display units and for graphically presenting information on shelf display units

Also Published As

Publication number Publication date
ES2818423A1 (es) 2021-04-12
WO2022074069A3 (fr) 2022-06-02

Similar Documents

Publication Publication Date Title
US10163115B2 (en) Control method for displaying merchandising information on information terminal
CN103314391B (zh) 基于自然姿势的用户接口方法及系统
CN102982059B (zh) 信息处理装置、信息处理方法
US8295542B2 (en) Adjusting a consumer experience based on a 3D captured image stream of a consumer response
US20190108561A1 (en) Purchase Intent Determination And Real Time In-store Shopper Assistance
CN110246569A (zh) 可穿戴装置和从先前会话检索并显示关键词的方法及介质
US11756095B2 (en) Facilitating camera installation and maintenance using extended reality
US10223998B2 (en) Dynamic content analysis and engagement system for shared displays
CN107924548A (zh) 使用可穿戴设备自动监视一位置处的实时活动以确定等待时间的系统和方法
CN110942055A (zh) 展示区域的状态识别方法、装置、设备及存储介质
CN111008859B (zh) 虚拟店铺中信息呈现方法、装置、电子设备及存储介质
JP2008287690A (ja) グループ可視化システム及びセンサネットワークシステム
CN110895768B (zh) 一种数据处理方法、设备、系统及存储介质
US20180349818A1 (en) Methods and Systems for Evaluating Performance of a Physical Space
EP2136329A2 (fr) Système informatisé complet et procédé d'adaptation du contenu des affichages de signes numériques
US20150365390A1 (en) Method of creating preference image identification code, method of diagnosing preference image identification code, and method of providing information using preference image identification code
CN114746882A (zh) 交互感知和内容呈现的系统和方法
WO2022074069A2 (fr) Dispositif d'affichage de commerce et procédé de traitement d'images capturées par ce dernier
US20240108985A1 (en) Managing virtual collisions between moving virtual objects
US20230138204A1 (en) Augmented reality object interaction and notification
CN115119004B (zh) 数据处理方法、信息显示方法、装置、服务器及终端设备
Bianco et al. Who Is in the Crowd? Deep Face Analysis for Crowd Understanding
EP2544103A1 (fr) Procédé et système pour interagir avec des objets
CN111935229A (zh) 智能货柜推送方法、装置和电子设备
CN110941194A (zh) 一种物品管理方法和系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21787441

Country of ref document: EP

Kind code of ref document: A2

DPE1 Request for preliminary examination filed after expiration of 19th month from priority date (pct application filed from 20040101)
NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21787441

Country of ref document: EP

Kind code of ref document: A2