ES2741377A1 - ANALYTICAL PROCEDURE FOR ATTRACTION OF PRODUCTS IN SHIELDS BASED ON AN ARTIFICIAL INTELLIGENCE SYSTEM AND EQUIPMENT TO CARRY OUT THE SAID PROCEDURE (Machine-translation by Google Translate, not legally binding) - Google Patents

Abstract

Analytical procedure of attracting products in shop windows based on an artificial intelligence system and equipment to carry out said procedure, which includes: - Place in the shop window (3) sensors (2) with video camera and autonomous processing board whose computer program is based on algorithms for counting the attraction of the pedestrian gaze (6) on the products (7) and generate data, - send the generated information to a central processing unit (5) that centralizes the information of all the sensors (2) of the showcase (3), - and to a remote server (8) where it is stored in a database, for real-time or deferred analysis. The equipment for carrying out the procedure comprises one or more sensors (2) each constituting an assembly consisting of a video camera and an autonomous processing board; a central unit (5) of a local computer processor, commercially installed (4) and connected to said sensors (2); a remote server (8) that receives the information generated and sent to a computer (9). (Machine-translation by Google Translate, not legally binding)

Description

DESCRIPTION

ANALYTICAL PROCEDURE FOR ATTRACTION OF PRODUCTS IN ESCAPARATES BASED ON AN INTELLIGENCE SYSTEM

ARTIFICIAL AND EQUIPMENT TO CARRY OUT SAID

PROCESS

OBJECT OF THE INVENTION

The invention, as stated in the present specification, refers to an analytical process of attracting products in shop windows based on an artificial intelligence system and the equipment for carrying out said procedure contributing, to the function to which destin, advantages and characteristics described in detail below.

Specifically, the object of the present invention lies in an analytical procedure of attracting the look of the products of the showcases, based on the analysis of the positioning and the look of the passersby, captured by static cameras installed in the showcase and processed by a artificial intelligence system, which will be of great commercial use both for shops and shop windows, and for the manufacturers of the products displayed in the shop windows.

FIELD OF APPLICATION OF THE INVENTION

The field of application of the present invention is part of the industry sector dedicated to the development of marketing studies, specific for physical businesses, while covering the scope of the industry dedicated to the manufacture and installation of digital imaging equipment .

BACKGROUND OF THE INVENTION

As is known, information about the behavior of potential customers of a product is very important for commercial teams and manufacturers.

On the other hand, shop windows are one of the main elements that allow stores and businesses to exhibit the products they sell so that passers-by can see them and be attracted to them to enter to acquire them.

Therefore, it would be very helpful, both for merchants and manufacturers, to know the degree of interest that the products of their shop windows raise on passers-by when they pass and stop in front of them and, from the information obtained objectively, analyze and decide strategies based on the behavior of these passers-by against what is exposed in the shop windows.

The objective of the present invention is, therefore, to provide the market with an objective analysis tool of the attraction count of passers-by look at the products displayed in the shop windows, and analyzed individually by "user sessions", that is, by analyzing individually all those products that each person looks at while they are within the reach of the local network of static cameras.This management is carried out from the capture and digital treatment of images, which generates data of interest that will serve for its subsequent analysis and decision making .

On the other hand, and as a reference to the current state of the art, it should be noted that there are digital image processing equipment and programs with the capacity to perform computer processes from the recognition of images, such as people counting, the flow within shops and varied security options. There are also applications of gaze analysis through the use of specialized glasses for the analysis of the pupil that all users subject to the test or experiment must wear, but which differs substantially from the characteristics of the present invention, both in the purpose and in the technique.

In conclusion, at least on the part of the applicant, it is unknown the existence of any that is applicable in the field of commercial information related to the products displayed in shop windows based on the stop, pose of the face and look of the passers-by, as well as neither any procedure nor any similar application equipment that has the same or similar technical and structural characteristics as those presented in the procedure as claimed herein.

EXPLANATION OF THE INVENTION

The analytical procedure of attracting products in shop windows based on an artificial intelligence system and the equipment to carry out said procedure proposed by the invention are thus configured as a practical solution to the problem described, the characterizing details that make them possible and that they are conveniently distinguished in the final claims that accompany the present description.

More specifically, what the invention proposes, as indicated above, is a procedure for the elaboration of analytical studies of shop windows and their products that, based on the count of the attraction of the gaze of pedestrians traveling in front of a shop window associated with a "user session", allows to obtain information intended to serve as an objective tool for commercial analytics for both merchants and product manufacturers, being a Second aspect of the invention is the provision of camera equipment and other technical and computer elements for digital image processing that allow said procedure to be carried out.

Thus, said procedure essentially includes the following:

- Place several cameras inside the window to be analyzed, located at the same height and distance between them, forming a row. A minimum of HD cameras with a field of view between 60 and 90 degrees, placed at a distance of 1.5 or 1.6 meters from the street floor, is recommended.

Each camera has associated a plate with autonomous computer processor, and as a whole they form what we will call here “sensor”. Each sensor is conveniently equipped with a computer program based on algorithms for the image processing that the camera captures for this purpose. These algorithms are updated periodically.

These algorithms must detect pedestrians and their relative location, as well as characteristics of their height and clothing to facilitate the monitoring of the person; pose of the face and orientation of the eyes, taking as reference the position of the pupils for the estimation of the look.

The information generated by the sensors is communicated to a central local processing unit that centralizes the information of all the sensors. This information is analyzed in real time, which allows grouping the information individually by person through what is called here "user session" and uniquely despite having windows and / or cameras installed, recording the order and time of all The products observed. It is considered the end of a user session when the person it remains several seconds outside the visual field of the network of cameras of the commerce.

At the end of each user session, the information is sent to the remote server where it is stored in a database, which allows its analysis in real time or in deferred time, which represents a high value for shops with shop windows and for manufacturers of the products, to analyze the performance of their windows and thus their eventual improvement.

The computer program works from known data and the data it can calculate.

Data you know:

- Position of the X, Y and Z cameras.

- Characteristics of the cameras, such as the field of view and focus length.

- Identifiers of the products shown.

- Volume of the form of the products shown.

Data calculated / estimated during the process:

- Pedestrian detection and X, Y, Z coordinates.

- Position and pose of the pedestrian's head.

- Analysis of the eyes, specifically of the pupils.

The set of known data combined with the estimated data, during the process, allows you to finally assess the product that each person is looking at at any time

The procedure, in general, contemplates the following:

Any of the various processing plates of the sensor cameras that are incorporated into the window processes several frames per second in the video captures.

For each frame, an algorithm for the detection of "objects" is applied, based on a convolutional neural network (CNN) with a pre-trained model for human detection.

As is known, a convolutional neural network is a type of artificial neural network where neurons correspond to receptive fields in a manner very similar to neurons in the primary visual cortex of a biological brain.

Based on this, faces are similarly detected and a bounding frame is generated for each face. These small areas are analyzed to find the edges of the face by applying several filters and taking some reference points to understand where the limits of the eyes, mouth, chin and face are.

Then, the head distance and the camera angle at the X, Y and Z coordinates are estimated (the estimated proportional size of the head height is known compared to the rest of the body and the size / distance curve of the head for a particular camera configuration).

Some filters are applied to the eyes and the pupils are estimated with any of the standard techniques, such as taking the largest circle inside the eyes with the highest black density and their location within the eye.

By combining the pose of the face, the position of the pupil, the distance and the rest of the coordinates, it is possible to estimate the pedestrian's position and the direction of the gaze.

It is also known the location and approximate volumetry of the products, so you can calculate the perspective of the products of the showcase and the obstruction between them according to the position of each pedestrian to generate the estimated area visible per product for each pedestrian at each moment, which provides great precision.

Finally, since all distances are relative to the cameras, it can be estimated if the pedestrians are looking at the window and which of the products are looking for the current frame.

To handle the information and ensure that data is generated only once per pedestrian, the "user session" is created, so that statistics can be accumulated in each session:

- If the pedestrian looked at a store window and in what order,

- What products you looked at, how long and in what order,

- Address and time of the user session, etc.

As the references of each product and shop window are known, all this numerical data is accumulated on a server to allow the stores whose shop windows incorporate the system and / or the manufacturers of the products to analyze the performance of their shop windows.

Special treatment to get unique "pedestrian sessions":

It is important to emphasize that the information has to be accumulated in a unique way for each pedestrian and for this reason the identification and monitoring management by pedestrians is done centrally based on the data collected by all the cameras of the commercial sensors.

On the other hand, for the solution to comply with the current data protection law, information that could identify people cannot be registered. To do this, each camera shares all the characteristics of the clothing of each of the pedestrians detected with the central computer of the local network, without taking any facial feature that could identify you. This compares in real time the characteristics received from each camera with those accumulated in the previous seconds, and associates the data on look, as well as the new dress data, to the best of coincidences, assuming that it is the same pedestrian. When the pedestrian session is terminated, the data accumulated during the second life of the session is deleted.

Data treatment:

The final data is sent to the remote server to show the results to the merchant, shop window or manufacturer through a private website and / or application accessible from your desktop computer and / or mobile device.

Communication between server and client, local processing unit and / or sensors can be done with different protocols and formats, tokenization, encryption, etc., about what is not considered necessary to go into more detail because it is considered irrelevant.

Finally, it should be reiterated that no video, human patterns, hashes, tuples or data that allow pedestrians to be recognized after each session are recorded. The images are processed and destroyed in real time. In the course of the session for the follow-up of the pedestrian neither people nor facial features are registered or identified, shapes, colors and vertices of the clothes of the people are identified, that is, from the neck down, this process is sufficient to distinguish the different pedestrians in a short space of time, while avoiding conflicts with data protection regulations.

As noted, the process unit that centralizes the data In the management of sessions, it is installed in the shop premises, whose shop window is intended to be analyzed with the system. With optimal computer hardware, the system can process around 10 frames per second, including several people per frame and several cameras per installation.

DESCRIPTION OF THE DRAWINGS

To complement the description that is being made and in order to help a better understanding of the characteristics of the invention, a descriptive plan is attached as an integral part thereof, with an illustrative and non-limiting character The following has been represented:

The number 1 and only figure. - It shows a plan view of a schematic representation of an example of the equipment to carry out the analytical procedure, according to the invention, implemented in the window of a shop, appreciating the main elements it comprises and its provision.

PREFERRED EMBODIMENT OF THE INVENTION

In view of the described figure 1 and only, and according to the numbering adopted therein, an example of non-limiting embodiment of the equipment of the invention can be observed, which comprises what is described in detail below.

Thus, as can be seen in said figure, the equipment (1) of the invention, in order to carry out the described analytical showcase procedure, is essentially configured from:

- One or more sensors (2) that constitute, each one, a set consisting of a video camera and an autonomous processing board, ability to process algorithms on the video images captured for the abstraction of relevant data of pedestrians (6) that pass in front of shop windows (3) and their products (7). The sensors (2) are installed in the shop windows (3) of the store (4) and are arranged in a row, so that they are all located at the same height and the same separation distance between them;

- a central unit (5) of local computer processor (computer normally), installed in the store itself (4) and connected to said sensors (2), which centralizes data and manages user sessions, that is, pedestrian images ( 6) unique ones that are located in front of the shop windows (3) of the shop (4), as well as the communication with a remote server (8); Y

- a remote server (8) that, through the communication module with which the central unit (5) of the processor is equipped, preferably via the Internet (I), receives the generated information and, in turn, generates a database which, also preferably via the Internet, can be sent or consulted to / from the computer (9) or other device, portable, mobile, etc., of who manages the store (4) whose storefront (3) is analyzed and / or manufacturer or manufacturers of the products (7) exposed in said showcase (3).

Preferably, the equipment (1) comprises a minimum number of sensors (2) with video camera such that it is sufficient to cover the entire width of the window (10) of the showcase (3) that is to be analyzed with an approximate separation between they from 2 to 3 meters, standing at a height of 1.5 or 1.6 meters from the street floor. Preferably the sensor cameras (2) are high definition HD with a field of view between 60 and 90 degrees.

The autonomous processing plate of each sensor (2) is responsible for interpreting, in the images captured by the camera, the people it finds, location, characteristics of clothing, look, etc. for each video frame. Said data is sent to the central processing unit (5) by the local network after the processing of each frame.

The central processing unit (5) manages user sessions trying to eliminate duplicate pedestrian data that has appeared on two or more sensors (2) at the same time.

When the central unit (5) terminates the pedestrian session, it will be sent to the data queue to be sent to the server (8), which will be dispatched in real time or periodically, depending on the configuration of the application at that time.

In the event that the shop (4) only has one sensor (2), optionally, the camera's own autonomous processing board can act as the central processing unit (5).

Optionally, the same equipment (1) may comprise the installation of sets or groups of sensors (2) in more than one shop window (3) of the same store (4) connected to the same central processing unit (5). Alternatively, each showcase (3) can have a central processing unit (5) independent of the other central units (5) processing the other showcases.

In any case, said central unit (5) is connected to said sensors (2) by physical wiring (11) or wireless mode.

Describe sufficiently the nature of the present invention, thus As a way of putting it into practice, it is not considered necessary to extend its explanation so that any expert in the field understands its scope and the advantages that derive from it, stating that, within its essentiality, it can be taken to the practice in other embodiments that differ in detail from that indicated by way of example, and which will also achieve the protection that is sought as long as it does not alter, change or modify its fundamental principle.

Claims (15)

1. - Analytical procedure of attracting products in shop windows based on an artificial intelligence system that, destined to the elaboration of analytical studies of the behavior of pedestrians (6) that stop in front of a shop window (3), is characterized by understanding the performances: - Place several cameras inside the shop window (3) to be analyzed, each associated with a plate with autonomous computer processor, said set forming a sensor (2) whose computer program is based on algorithms for the image processing that the camera captures for count the attraction of the look of the pedestrians (6) that pass in front of the shop window (3) on the products (7) incorporated in it associated with a "user session", ensuring that data is generated only once per pedestrian, so that statistics can be accumulated in each session, - send the information generated by the sensors (2) to a central unit (5) of local processing that centralizes the information of all the sensors (2) of the showcase (3), where it is analyzed in real time, and grouped individually by person or "user session" uniquely, and - send the information to a remote server (8) where it is stored in a database, for real-time or deferred analysis. 2. - Analytical procedure for attracting products in shop windows based on an artificial intelligence system, according to claim 1, characterized in that the algorithms are updated periodically. 3. - Analytical method of attracting products in shop windows based on an artificial intelligence system, according to claim 1, characterized in that the algorithms detect pedestrians and their relative location, as well as characteristics of their height and clothing to facilitate the monitoring of the person; pose of the face and orientation of the eyes, taking as reference the position of the pupils for the estimation of the look. 4. - Analytical procedure for attracting products in shop windows based on an artificial intelligence system, according to claims 1 to 3, characterized in that any of the sensors (2) processes several frames per second in the video captures; applying for each frame, an algorithm for the detection of "objects" is applied, based on a convolutional neural network with a pre-trained model for human detection, based on it, faces similarly and generating a delimiter frame for each face, analyzed below these small areas to find the edges of the face by applying several filters and taking some reference points to understand where the limits of the eyes, mouth, chin and face are, then estimating the distance of the head and angle of the cameras in the X, Y and Z coordinates, applying filters to the eyes and estimating the pupils, then combining the pose of the face, the position of the pupil, the distance and other coordinates with which the position of the pedestrian and the direction of the gaze are estimated. 5. - Analytical procedure of attracting products in shop windows based on an artificial intelligence system, according to claim 4, characterized in that by means of the approximate location and volumetry of the products, the perspective of the products of the shop window and the obstruction between them are calculated according to the position of each pedestrian to generate an estimated visible area per product for each pedestrian at each moment. 6. - Equipment to carry out an analytical procedure to attract products in shop windows based on an intelligence system artificial, as described in claims 1 to 5, characterized by comprising: - one or more sensors (2) that constitute, each one, a set consisting of a video camera and an autonomous processing board, which are installed in the shop window (3) of a shop (4) so that they capture images of the pedestrians (6) passing in front of said shop window (3), said plate being provided with the capacity to process algorithms on the captured video images to extract relevant data from the pedestrians (6); - a central unit (5) of local computer processor, installed in the store itself (4) and connected to said sensors (2), which centralizes data and manages the images of unique pedestrians (6) that are located in front of shop windows (3) of commerce (4), as well as communication with a remote server (8); Y - a remote server (8) that, through the communication module with which the central unit (5) of the processor is equipped, receives the generated information and, in turn, generates a database that, in turn, can be sent or consulted to / from the computer (9) or other device, portable, mobile or similar, of who manages the trade (4) whose shop window (3) is analyzed and / or the manufacturer or manufacturers of the products (7) exposed in said showcase (3). 7. - Equipment according to claim 6, characterized in that the communication between the central unit (5) and the remote server (8) and the communication between it and the computer (9) or other device, portable, mobile or similar, of whom manages the trade (4) whose showcase (3) is analyzed and / or the manufacturer or manufacturers of the products (7) exposed in said showcase (3), is carried out via the Internet. 8. - Equipment according to claim 6 or 7, characterized in that, when the showcase (3) incorporates more than one sensor (2), they are arranged in a row, so that they are all located at the same height and the same separation distance between them. 9. - Equipment according to claim 8, characterized in that the number of sensors (2) is such that it covers the entire width of the window (10) of the showcase (3) with an approximate separation between them of 2 to 3 meters. 10. - Equipment according to any of claims 6 to 9, characterized in that the sensors (2) are located at a height of 1.5 or 1.6 meters from the street floor 11. - Equipment according to any of claims 2 to 10, characterized in that the sensor cameras (2) are high definition HD with a field of view between 60 and 90 degrees. 12. - Equipment according to claim 2, 10 or 11, characterized in that, when there is only one sensor (2), the autonomous processing board of the chamber acts as the central processing unit (5). 13. - Equipment according to any of claims 6 to 12, characterized in that it comprises sets of sensors (2) installed in more than one shop window (3) of the same store (4). 14. - Equipment according to claim 13, characterized in that all the sensors (2) of the different windows (3) of the same store (4) are connected to the same central processing unit (5). 15. - Equipment according to claim 13, characterized in that the sensor assemblies (2) of each showcase (3) of the same store (4) are connected to a central processing unit (5) independent of the central units (5) processors of the other shop windows (3) of the trade (4).