WO2019025790A1 - Smart display system - Google Patents

Smart display system Download PDF

Info

Publication number
WO2019025790A1
WO2019025790A1 PCT/GB2018/052190 GB2018052190W WO2019025790A1 WO 2019025790 A1 WO2019025790 A1 WO 2019025790A1 GB 2018052190 W GB2018052190 W GB 2018052190W WO 2019025790 A1 WO2019025790 A1 WO 2019025790A1
Authority
WO
WIPO (PCT)
Prior art keywords
content
item
image
operable
display system
Prior art date
Application number
PCT/GB2018/052190
Other languages
French (fr)
Inventor
Mohammed Shahzad MUGHAL
Garry Partington
Mohammed Abdul ALIM
Original Assignee
Admoments Holdings Limited
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Admoments Holdings Limited filed Critical Admoments Holdings Limited
Publication of WO2019025790A1 publication Critical patent/WO2019025790A1/en

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • H04N7/188—Capturing isolated or intermittent images triggered by the occurrence of a predetermined event, e.g. an object reaching a predetermined position
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/02—Marketing; Price estimation or determination; Fundraising
    • G06Q30/0241—Advertisements
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/02—Marketing; Price estimation or determination; Fundraising
    • G06Q30/0241—Advertisements
    • G06Q30/0251—Targeted advertisements
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/02—Marketing; Price estimation or determination; Fundraising
    • G06Q30/0241—Advertisements
    • G06Q30/0251—Targeted advertisements
    • G06Q30/0269—Targeted advertisements based on user profile or attribute
    • G06Q30/0271—Personalized advertisement
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/02—Marketing; Price estimation or determination; Fundraising
    • G06Q30/0241—Advertisements
    • G06Q30/0273—Determination of fees for advertising
    • G06Q30/0275—Auctions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/02—Marketing; Price estimation or determination; Fundraising
    • G06Q30/0241—Advertisements
    • G06Q30/0276—Advertisement creation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/174—Facial expression recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/414—Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance
    • H04N21/41415—Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance involving a public display, viewable by several users in a public space outside their home, e.g. movie theatre, information kiosk
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/441—Acquiring end-user identification, e.g. using personal code sent by the remote control or by inserting a card
    • H04N21/4415—Acquiring end-user identification, e.g. using personal code sent by the remote control or by inserting a card using biometric characteristics of the user, e.g. by voice recognition or fingerprint scanning
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/442—Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk
    • H04N21/44213—Monitoring of end-user related data
    • H04N21/44218—Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/45—Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
    • H04N21/458—Scheduling content for creating a personalised stream, e.g. by combining a locally stored advertisement with an incoming stream; Updating operations, e.g. for OS modules ; time-related management operations
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/45—Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
    • H04N21/466—Learning process for intelligent management, e.g. learning user preferences for recommending movies
    • H04N21/4668—Learning process for intelligent management, e.g. learning user preferences for recommending movies for recommending content, e.g. movies

Definitions

  • the present invention relates to a smart display system and a method of displaying an item of content.
  • An advert is considered to be more successful if it is more likely to draw the publics' attention. This is the case if the advert chosen to be displayed on the screen is relevant to a person or persons in the vicinity.
  • current smart display systems lack the means to determine what adverts people in the vicinity would find relevant.
  • an advert which is considered to be relevant to people in the vicinity of the screen, is displayed it is also advantageous to determine whether said people did in fact find the advert to be relevant.
  • One method for measuring relevance is to measure whether there is an increase in sales of the advertised product or service. However, it can be difficult to impossible to determine whether there is a direct link between an increase in sales and the advert. Therefore a more direct means of measuring relevance would be beneficial.
  • a smart display system comprising an imaging device operable to capture an image, a multi- core image processing unit operable to process the captured image so as to determine features of the image and a display screen operable to display an item of content relevant to the determined features of the image.
  • the system has the advantage that the content displayed can be tailored to be relevant to the features of the captured image.
  • the captured image shows people within the vicinity of the display screen and the features determined include features of these people the content can be tailored to the people in the vicinity. This allows for the content displayed on the smart display to be targeted towards a person or other features of an area, and therefore be more likely to be successful.
  • the use of a multi-core image processing unit speeds up processing of the image, so that targeted advertising can occur in real-time. This means that the image will be processed and content displayed while a person is still within sight of the display screen.
  • the smart display system may comprise a registration unit operable to register a user.
  • the smart display system may comprise an input operable to receive registration information.
  • the input may comprise a link to an application and/or web browser.
  • the registration information may include any of the following: identity of user, address, telephone number, email address and/or details of a method of payment.
  • the method of payment may comprise any of the following: a credit card, a debit card and/or an online payment system.
  • Registering a user allows said user to add items of content to the system.
  • the system can associate items of content to a particular user. This therefore allows the system to keep a record of adverts for the user.
  • the inclusion of details of a method of payment allows the system to automatically charge a registered user each time one of their adverts are shown.
  • the smart display system may comprise a memory storage unit operable to store one or more items of content.
  • the memory storage unit may store the or each item of content in a list of stored items of content.
  • the memory storage unit may be operable to store an entry for the or each registered user, comprising the registration information.
  • the memory storage unit may be operable to store image libraries, each image library containing one or more objects with an associated description.
  • the multi-core image processing unit may be operable to store a link to said image libraries.
  • the memory storage unit may be operable to store the or each item of content associated with a registered user in a list of the registered user's stored items of content.
  • the input may be operable to view a registered user's entry and/or items of content. Additionally, the input may be operable to edit a registered user's entry and/or items of content.
  • the memory storage unit allows a user to access their list of adverts and review details associated with them. Additionally, a user can remove and/or modify their adverts.
  • the imaging device may capture an image or images comprising a view of an area.
  • the area may comprise the vicinity of the display screen.
  • the area may comprise an area within line of sight of the display screen.
  • the or each image may comprise a view of a person.
  • the imaging device may comprise a camera.
  • the area comprising the vicinity of the display screen allows the displayed advert to target people around the display screen.
  • the smart display system may comprise a management unit operable to manage the camera and/or display screen.
  • the smart display system may comprise a transmitter operable to transmit the image to the multi-core image processing unit. Additionally, the smart display system may comprise a receiver operable to receive an item of content for display on the display screen.
  • the transmitter and receiver allow the display screen and camera to be physically separate from the rest of the smart display system. This allows for these components to form part of a billboard, which is connected to a server which runs the rest of the system.
  • the multi-core image processing unit may be operable to process the captured image by applying computer vision algorithms to the image.
  • the computer vision algorithms may comprise pattern matching algorithms, wherein the pattern matching algorithms scans at least one image library to match objects in the or each image library to features of the image.
  • the multi-core image processing unit may be operable to apply one or more metadata tags to the image, the or each metadata tag comprising a description of a feature determined to be in the image.
  • the description of the determined feature may be the description of an object in the or each image library which matched the feature.
  • the or each metadata tag may comprise any one of the following: gender, race, age ranges, clothing types, clothing colours, clothing brands, clothing accessories, car makes, car models, groups of people, types of groups of people, size of groups of people, sentiments, moods, faces and/or a particular person.
  • the multi- core image processing unit may be operable to apply metadata tags to the or each image by adding them to code representing the image. Additionally, each metadata tag may be applied to the code which represents the relevant determined feature.
  • the multi-core image processing unit may be operable to determine if a feature is a person. Even more preferably, the multi-core image processing unit may be operable to determine features which are associated with a person.
  • Determining if a feature is a person and then associating determined features allows a profile to be built for said person, which allows future adverts to be even more targeted.
  • the multi-core image processing unit may be operable to determine features which are associated with a group of people. Additionally, the multi-core image processing unit may be operable to determine the type of group of people based on the features associated with the group.
  • the multi-core processing unit may processes the image to determine the proximity of the determined features.
  • the multi-core processing unit may be operable to ignore determined features which are outside a set proximity range.
  • the multi-core processing unit may be operable to determine the most prominent person.
  • the smart display system may comprise a database unit operable to store sections of images comprising a feature determined to be a face as an entry on a list of faces.
  • the database unit may be operable to store metadata tags associated with the person associated with a determined face against the respective faces' entry.
  • the multi-core image processing unit may be operable to apply facial recognition algorithms to the image, wherein the list of faces is used as an image library.
  • the multi-core image processing unit may be operable to store sections of the image comprising features determined to match faces stored on the list of faces against the same entry.
  • the input may be operable to view the list of faces.
  • the database unit allows profiles of particular people to be built. Multiple views of the person's face allow the individual to be more easily identified in the future by the facial recognition algorithms, while the profile allows adverts to target the person based on past encounters with the smart display system. A registered user can view the list of faces, so as to determine how best to target current and future adverts.
  • the smart display system may comprise a content creation unit operable to create one or more items of content.
  • the or each item of content may comprise an advert.
  • the advert may comprise any combination of the following: images, text and/or videos. Additionally, the or each item of content may comprise information associated with the advert.
  • the associated information may comprise any combination of the following: the identity of the registered user, one or more metadata tags, one or more maximum bid amounts, a continuous daily budget, a total budget for a set number of days and/or dates for showing the advert.
  • the or each maximum bid amount may be associated with the or each metadata or a combination of metadata tags.
  • the input may be operable to receive the advert and/or associated information.
  • the input may be operable to edit the associated information stored on the memory storage unit.
  • the associated information allows a registered user to provide detailed instructions for how an advert should operate.
  • the maximum bid amounts being associated with metadata tags or a combination of metadata tags allows a registered user to prioritise certain metadata tags over others.
  • the smart display system may comprise a content selection unit operable to select the relevant item of content.
  • the content selection unit may be operable to select the relevant item of content from the list of stored items of content.
  • the content selection unit may be operable match one or more items of content to the image by comparing the metadata tags of the image with metadata tags of the or each item of content, wherein an item of content with one or more metadata tags which match one or more metadata tags of the image is matched.
  • the content selection unit may be operable to select the relevant items of content by ranking maximum bid amounts by largest value, wherein the item of content associated with the top ranked maximum bid amount is selected.
  • the content selection unit is operable to rank only maximum bid amounts which are associated with matched metadata tags or combinations of matched metadata tags.
  • the display screen may be operable to display the advert of the selected item of content.
  • the display screen may be operable to display the item of content for a set time.
  • the set time may be determined by the management unit.
  • the associated information of each item of content may comprise the set time.
  • the management unit may be operable to record the time, date and length of display of an item of content against the items of contents entries on the memory storage unit.
  • the smart display system may comprise a payment unit operable to charge the maximum bid amount ranked second plus one unit of monetary value to the registered user associated with the selected item of content.
  • the payment unit may be operable to charge the registered user's method of payment via the details of the method of payment.
  • the imaging device may be operable to capture a period of video.
  • the imaging device may be operable to capture a set of periods of video encompassing the entire time an item of content is on display.
  • the multi-core image processing unit may be operable to process the or each period of video by applying eye tracking algorithms. Additionally, the multi-core image processing unit may be operable to determine dwell times of one or more determined persons in the image.
  • the multi- core image processing unit may be operable to record a determined persons' dwell time and associated metadata tags on the entry for the displayed item of content on the list of the registered users' stored items of content. Additionally, the multi-core image processing unit may be operable to record a determined person's dwell time on the associated entry on the list of faces.
  • the multi-core image processing unit may be operable to process a period of video so as to determine if one or more features are gestures.
  • the multi- core image processing unit may be operable to determine if a feature is a gesture only if the feature is associated with the most prominent person.
  • the most prominent person is either the person who comprises the largest feature determined to be a person in the image, or is the person with the best line of sight to the display screen 2.
  • the management unit may comprise a list of known gestures, each entry comprising a metadata tag of a known gesture and a response.
  • the multi-core image processing unit may be operable to transmit metadata tags for gestures of determined features to the management unit, which is operable on to then carry out the response associated with said gesture.
  • the receiver may be operable to receive the metadata tags for gestures.
  • the response can comprise either end the display of the current advert or display the current advert for an additional set amount of time.
  • the additional set amount of time may be 5 seconds. Determining gestures and having responses to said gestures allows a person to interact with the smart display system. Restricting the determination of gestures to the most prominent person prevents the smart display system accidently interacting with multiple people at the same time. The person interacting with the smart display system can then skip through adverts at will, or stop on one of particular interest.
  • the smart display system may comprise a plurality of imaging devices and display, each imaging device being associated with a display screen. Additionally, the smart display system may comprise a plurality of management units. Additionally, the associated information may comprise a list of display screens on which to display the item of content.
  • the smart display system may comprise a server comprising a platform.
  • the platform may comprise the multi-core image processing unit. Additionally, the platform may comprise any of the following: the memory storage unit, the content creation unit, the content selection unit, the registration unit, input and/or the database unit.
  • the smart display system may comprise one or more billboards. Additionally or alternatively, the smart display system may comprise one or more wall displays.
  • the or each billboard and/or wall display may comprise a display screen. Additionally, the or each billboard and/or wall display may comprise the associated imaging device.
  • the or each management unit may comprise part of a billboard or wall display and/or the platform.
  • the or each management unit may comprise a computer.
  • a method of displaying an item of content comprising the steps of providing a smart display system according to the first aspect of the present invention, capturing an image, processing the captured image so as to determine features of the image and displaying an item of content relevant to the determined features of the image.
  • the method of the second aspect of the present invention may incorporate any or all features of the system of the first aspect of the present invention, as desired or as appropriate.
  • Figure 1 shows a billboard which forms part of a smart display system according to an embodiment of the present invention.
  • Figure 2 shows a block diagram of a smart display system according to an embodiment of the present invention.
  • a billboard 1 which comprises a display screen 2, a management unit 4 and a camera 3.
  • the camera 3 is positioned so as to take images of the area 5 in front of the billboard 1.
  • the camera 3 is set on a timer so that images are only captured at set times, for example every second.
  • the management unit 4 directs the camera 3 to take an image when it is time to change the currently displayed advert.
  • the camera 3 is operable to record video, and the image may comprise a frame of a video.
  • the smart display system 6 comprises the billboard 1 and a platform 7 on a server (not shown), wherein the platform 7 manages the billboard 1.
  • the server will be a cloud server.
  • a user can access the platform 7 either via an input 8 which comprises either a link to an application 9 (on, for example, a mobile phone or tablet) or a web browser 10.
  • the platform 7 comprises a registration unit 11, which can register the or each user.
  • the or each user can input their details via the input 8, comprising their identity (e.g. their name) and contact details (e.g. their address, telephone number and/or email address), which the registration unit 11 then registers.
  • the or each user's details also comprise their method of payment, which can be, for example, a credit card, debit card and/or an online payments system.
  • the platform comprises a memory storage unit 12, to which the or each user's registered details are saved.
  • the platform 7 comprises a content creation unit 13 which registered users use to generate their item of content, which comprises an advert.
  • the advert can comprise a template which comprises text, one or more images and/or a video. Such videos can be up to 10 seconds in length. Users can either use a standard template on the content creation unit 13, or upload a different template. The users can also upload images and/or video for the advert, via the input 8.
  • the or each item of content can also comprise associated information. Once the or each advert is generated information can be associated with the advert. This information can comprise metadata tags, which describe the audiences the user is interested in targeting.
  • Such metadata tags can describe various features, including demographics such as gender, race, age ranges, clothing types, clothing colours, clothing brands, clothing accessories, car makes, car models, groups of people, types of groups of people, size of groups of people, sentiments, moods, faces and/or people.
  • the metadata tags can be selected via the input 8 from a dropdown menu and/or inputted manually.
  • the associated information also comprises one or more maximum bid amounts.
  • the or each maximum bid amount lists a monetary value. In a preferred embodiment of the invention, one unit of the monetary value may be lp.
  • Each metadata tag within the associated information has an associated maximum bid amount.
  • the associated information can also comprise maximum bid amounts for selected combinations of tags.
  • the associated information also comprises a continuous daily budget and/or a total budget for a set number of days, and/or the dates for which the advert should be shown.
  • the smart display system 6 comprises a plurality of billboards 1.
  • the associated information also comprises the places the advert should be shown.
  • the content creation unit 13 saves the item of content to the memory storage unit 12.
  • Each item of content is saved in a list of the stored items of content, and to a list of the registered user's stored items of content.
  • the registered user can access the memory storage unit 12 via the input 8 to view the list of registered user's stored items of content. Additionally, the registered user can generate new items of content via the content creation unit 13 by copying and then modifying an item of content saved on the memory storage unit 12, or delete items of content associated with the user from the memory storage unit 12.
  • deleted items of content are removed from the list of stored items of content, but not the list of the registered user's stored items of content.
  • a note is added to the entry for the deleted item on the list of the registered user's stored items of content stating that the item of content is removed. The registered user can then reinstate the item of content, thereby removing the note and re-adding the item to the list of stored items of content.
  • the management unit 4 uploads the image to the platform 7. This upload is via a transmitter 14.
  • the management unit 4 also records the time and date of the image, and the billboard 1 with which the image is associated, and uploads these details to platform 7 in conjunction with the image.
  • the platform comprises a multi-core image processing unit 15, which applies computer vision algorithms to the image so as to process it.
  • the computer vision algorithms can include optical character recognition algorithms and facial recognition algorithms.
  • the multi-core image processing unit 15 comprises at least one multi-core processor, so as to allow the image to be processed in real time.
  • the computer vision algorithms use pattern matching to identify features of the image, by scanning image libraries for objects which match a feature within the image. The or each feature can match a plurality of different objects (for example, a feature can be both a 'vehicle' and a 'car').
  • the multi-core image processing unit 15 also processes the image to determine the proximity of the determined features. Determined features which are outside a set proximity range are ignored.
  • the multi-core image processing unit 15 associates metadata tags with the image.
  • the metadata tags are associated with an image by incorporating the tags into the code defining the image.
  • Each metadata tag describes a feature determined to be in the photo, wherein the description is captured from the description of the matched object in the scanned image library.
  • the image libraries can include the DVLA vehicle database.
  • the registration plate of a vehicle in the image can be determined, and the make and model metadata tagged to the image based on the listed information in the DVLA vehicle database.
  • the multi-core image processing unit 15 can also determines features in the image which are associated with the person.
  • the multi-core processing unit 15 When a feature is determined to be a group of people, the multi-core processing unit 15 also determines features in the image which are associated with the group. The multi-core image processing unit 15 then uses the features in the image which are associated with the group to determine the type of the group of people. For example, a group of people with a majority of females would be determined to be a female dominated group, or a group with a mixture of adults and children would be determined to be a family.
  • the smart display system 6 comprises a database unit 16.
  • the multi-core image processing unit 15 saves sections of the image comprising features determined to be faces in an entry on a list of faces on the database unit 16, the entry also comprising the determined demographics of the face, the date and time of the image and the billboard with which the image is associated.
  • the multi-core image processing unit 15 also saves metadata tags associated with a person, including clothing types, brands and colours, clothing accessories and car makes and models, against the entry for the face of the person in the database unit 16.
  • the multi-core image processing unit 15 can then use facial recognition algorithms to determine if features of an image match any faces saved to the database unit 16. Matched faces are saved to the same entry in the database unit 16.
  • a registered user can access and search the database unit 16 via the input 8, to determine the demographics which frequent each billboard, and when they frequent it.
  • the smart display system 6 also comprises a content selection unit 17, to which images with metadata tags are sent.
  • the content selection unit 17 accesses the memory storage unit 12 so as to obtain the list of the stored items of content.
  • the content selection unit 17 uses machine learning algorithms to match items of content with the image, via determining if each item of content has at least one metadata tag which matches a metadata tag of the image.
  • the content selection unit 17 narrows down the matched items of content to those which should be displayed on the billboard 1 from which the image was transmitted.
  • the content selection unit 17 then ranks maximum bid amounts of the matched items of content, the highest maximum bid amount being ranked top.
  • the content selection unit 17 only considers the maximum bid amounts which are associated with a matched metadata tag, or a combination of matched metadata tags.
  • the item of content associated with the top ranked maximum bid amount is selected.
  • the selected item of content is then sent to the billboard 1 to be received by receiver 18, for the advert to be displayed on the display screen 2.
  • Each advert can be displayed on the billboard 1 for the same amount of time, which is determined by the management unit 4. In some embodiments, the amount of time is 10 seconds. In alternative embodiments, the associated information can comprise a set time for which the advert should be displayed.
  • the set time is selected by the registered user from a range of times, ranging from a minimum time to a maximum time.
  • the content creation unit 13 can set a minimum maximum bid amount for each item of content, the minimum being higher for items of content with higher set times.
  • the platform comprises a payment unit 19, which receives an indication from the content selection unit 17 that an item of content is to be displayed.
  • the payment unit accesses the memory storage unit 12 to determine the method of payment of the user whose item of content was selected for display.
  • the payment unit 19 then charges the maximum bid amount ranked second, plus one unit of monetary value, to the method of payment.
  • the payment unit 19 comprises details of a bank account, into which the method of payment is directed to pay.
  • the payment unit 19 then stores a record of the charge, comprising the date, time and amount, on the memory storage unit 12, against the entry for the selected item of content on the list of the registered user's stored items of content. Once an advert has been displayed on the display screen, the management unit
  • the 4 adds a value to the associated item of content's entry on the list of stored items of content.
  • the value comprises a number which is non-zero.
  • the content selection unit 17 cannot select an advert for display which has a note.
  • the management unit 4 reduces all values on the memory storage unit 12 by 1. Once a value reaches zero it is deleted by the management unit 4.
  • the content selection unit 17 considers the charges against the entry for the selected item of content on the list of the registered user's stored items of content. If the maximum bid amount ranked second plus one unit of monetary value exceeds the difference between the daily budget and the total of the day's charges, or the difference between the total budget and the total of the charges for the set number of days, then the content selection unit removes the maximum bid amount ranked top from the rankings. This process is repeated until a maximum bid amount ranked second plus one unit of monetary value does not exceed the difference.
  • the management unit 4 is operable to check the list of stored items of content once a day. If and when the current day does not match any of the dates for showing the advert, the management unit 4 is operable to remove the associated item of content from the list of stored items of content.
  • the management unit 4 can transmit a record of the display to the platform 7.
  • the platform 7 can then save the record of the display to the memory storage unit 12, against the entry for the relevant item of content on the list of the registered user's stored items of content.
  • the record can comprise the date, time and length of display.
  • the registered user can access this list to examine the record of displays for each item of content.
  • the billboard 1 can operate via the
  • the management unit 4 while an advert is being displayed on the display screen 2, the management unit 4 directs the camera 3 to record periods of video. Each period of video extends for a period of time set by the management unit 4 (for example, 2 seconds) and a set of periods of video cover the entire time for which an advert is displayed. Each period of video, along with information detailing the date and time of recording and the identity of the associated billboard, is uploaded by the management unit 4 to the platform 7.
  • the multi-core image processing unit 15 determines features of period of video in the same way as described above for images.
  • the multi-core image processing unit 15 also processes the period of video by applying eye tracking algorithms, to determine when the eyes of a feature determined to be a person are looking at the display screen.
  • the dwell time of each person (which is the time spent looking at the display screen divided by the total time in which the display screen is within line of sight) is determined by the multi-core image processing unit 15.
  • the dwell time of each person is then saved on the entry for the displayed advert on the list of the registered users' stored items of content, along with the metadata tags associated with the person. The dwell times can then be reviewed by the user to determine the effectiveness of the advert.
  • the multi-core image processing unit 15 When processing a period of video one of the features which can be determined by the multi-core image processing unit 15 are gestures.
  • the multi-core image processing unit 15 sends the metadata tag of a gesture associated with the person determined to be closest to the display screen to the management unit 4.
  • the management unit 4 comprises a list of known gestures, each entry on the list comprising the metadata tag of a gesture and a response to carry out.
  • the responses to carry out include, for example, to stop showing the displayed advert prematurely and select a new advert or to add a set amount of time to the display time for the current advert.

Landscapes

  • Engineering & Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • Strategic Management (AREA)
  • Finance (AREA)
  • Development Economics (AREA)
  • Multimedia (AREA)
  • Accounting & Taxation (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Signal Processing (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Game Theory and Decision Science (AREA)
  • General Business, Economics & Management (AREA)
  • Marketing (AREA)
  • Economics (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Human Computer Interaction (AREA)
  • Databases & Information Systems (AREA)
  • Social Psychology (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Psychiatry (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Biomedical Technology (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

A smart display system (6) comprising an imaging device (3) operable to capture an image, a multi-core image processing unit (15) operable to process the captured image so as to determine features of the image and a display screen (1) operable to display an item of content relevant to the determined features of the image. The content displayed can be tailored to be relevant to the features of the captured image. When the captured image shows people within the vicinity of the display screen and the features determined include features of these people the content can be tailored to the people in the vicinity. This allows for the content displayed on the smart display to be targeted towards a person or other features of an area, and therefore be more likely to be a successful advertisement.

Description

Smart Display System
Technical Field of the Invention
The present invention relates to a smart display system and a method of displaying an item of content. Background to the Invention
In many public spaces advertising billboards and displays have been or are being replaced by digital advertising screens. These digital advertising screens allow for multiple adverts to be displayed on the screen during the same advertising period, the screen cycling through the various adverts in a slide show fashion. The screens also allow for adverts to be removed from or added to the cycle relatively quickly and easily once one advertising campaign ends and another begins. The nature of the medium also allows for adverts to have video and/or audio elements, rather than just comprising a still image.
An advert is considered to be more successful if it is more likely to draw the publics' attention. This is the case if the advert chosen to be displayed on the screen is relevant to a person or persons in the vicinity. However, current smart display systems lack the means to determine what adverts people in the vicinity would find relevant.
Additionally, once an advert, which is considered to be relevant to people in the vicinity of the screen, is displayed it is also advantageous to determine whether said people did in fact find the advert to be relevant. One method for measuring relevance is to measure whether there is an increase in sales of the advertised product or service. However, it can be difficult to impossible to determine whether there is a direct link between an increase in sales and the advert. Therefore a more direct means of measuring relevance would be beneficial.
It is therefore an object of the present invention to provide a smart display system that at least partially overcomes or alleviates the above problems. Summary of the Invention
According to a first aspect of the present invention there is provided a smart display system comprising an imaging device operable to capture an image, a multi- core image processing unit operable to process the captured image so as to determine features of the image and a display screen operable to display an item of content relevant to the determined features of the image.
The system has the advantage that the content displayed can be tailored to be relevant to the features of the captured image. When the captured image shows people within the vicinity of the display screen and the features determined include features of these people the content can be tailored to the people in the vicinity. This allows for the content displayed on the smart display to be targeted towards a person or other features of an area, and therefore be more likely to be successful.
In addition, the use of a multi-core image processing unit speeds up processing of the image, so that targeted advertising can occur in real-time. This means that the image will be processed and content displayed while a person is still within sight of the display screen.
The smart display system may comprise a registration unit operable to register a user. The smart display system may comprise an input operable to receive registration information. The input may comprise a link to an application and/or web browser. The registration information may include any of the following: identity of user, address, telephone number, email address and/or details of a method of payment. The method of payment may comprise any of the following: a credit card, a debit card and/or an online payment system. Registering a user allows said user to add items of content to the system. The system can associate items of content to a particular user. This therefore allows the system to keep a record of adverts for the user. The inclusion of details of a method of payment allows the system to automatically charge a registered user each time one of their adverts are shown. The smart display system may comprise a memory storage unit operable to store one or more items of content. The memory storage unit may store the or each item of content in a list of stored items of content. The memory storage unit may be operable to store an entry for the or each registered user, comprising the registration information. The memory storage unit may be operable to store image libraries, each image library containing one or more objects with an associated description. Alternatively, the multi-core image processing unit may be operable to store a link to said image libraries. The memory storage unit may be operable to store the or each item of content associated with a registered user in a list of the registered user's stored items of content. The input may be operable to view a registered user's entry and/or items of content. Additionally, the input may be operable to edit a registered user's entry and/or items of content. The memory storage unit allows a user to access their list of adverts and review details associated with them. Additionally, a user can remove and/or modify their adverts.
The imaging device may capture an image or images comprising a view of an area. The area may comprise the vicinity of the display screen. The area may comprise an area within line of sight of the display screen. In some implementations, the or each image may comprise a view of a person. The imaging device may comprise a camera.
The area comprising the vicinity of the display screen allows the displayed advert to target people around the display screen.
The smart display system may comprise a management unit operable to manage the camera and/or display screen. The smart display system may comprise a transmitter operable to transmit the image to the multi-core image processing unit. Additionally, the smart display system may comprise a receiver operable to receive an item of content for display on the display screen.
The transmitter and receiver allow the display screen and camera to be physically separate from the rest of the smart display system. This allows for these components to form part of a billboard, which is connected to a server which runs the rest of the system. The multi-core image processing unit may be operable to process the captured image by applying computer vision algorithms to the image. The computer vision algorithms may comprise pattern matching algorithms, wherein the pattern matching algorithms scans at least one image library to match objects in the or each image library to features of the image. The multi-core image processing unit may be operable to apply one or more metadata tags to the image, the or each metadata tag comprising a description of a feature determined to be in the image. The description of the determined feature may be the description of an object in the or each image library which matched the feature.
The or each metadata tag may comprise any one of the following: gender, race, age ranges, clothing types, clothing colours, clothing brands, clothing accessories, car makes, car models, groups of people, types of groups of people, size of groups of people, sentiments, moods, faces and/or a particular person. The multi- core image processing unit may be operable to apply metadata tags to the or each image by adding them to code representing the image. Additionally, each metadata tag may be applied to the code which represents the relevant determined feature.
Applying metadata tags to images allows adverts to target a wide variety of different features, allowing multiple adverts with different target markets to be run from the same billboard.
Preferably, the multi-core image processing unit may be operable to determine if a feature is a person. Even more preferably, the multi-core image processing unit may be operable to determine features which are associated with a person.
Determining if a feature is a person and then associating determined features allows a profile to be built for said person, which allows future adverts to be even more targeted.
The multi-core image processing unit may be operable to determine features which are associated with a group of people. Additionally, the multi-core image processing unit may be operable to determine the type of group of people based on the features associated with the group.
Determining the type of a group of people allows adverts to target a group as a whole, instead of just individual members of the group. The multi-core processing unit may processes the image to determine the proximity of the determined features. The multi-core processing unit may be operable to ignore determined features which are outside a set proximity range. The multi-core processing unit may be operable to determine the most prominent person.
Setting a proximity range prevents the adverts from targeting people which are not close enough to appreciate the advert, and instead prioritise closer people. Determining the most prominent person results in it being possible for said person to interact with the smart display system.
The smart display system may comprise a database unit operable to store sections of images comprising a feature determined to be a face as an entry on a list of faces. The database unit may be operable to store metadata tags associated with the person associated with a determined face against the respective faces' entry. The multi-core image processing unit may be operable to apply facial recognition algorithms to the image, wherein the list of faces is used as an image library. The multi-core image processing unit may be operable to store sections of the image comprising features determined to match faces stored on the list of faces against the same entry. The input may be operable to view the list of faces.
The database unit allows profiles of particular people to be built. Multiple views of the person's face allow the individual to be more easily identified in the future by the facial recognition algorithms, while the profile allows adverts to target the person based on past encounters with the smart display system. A registered user can view the list of faces, so as to determine how best to target current and future adverts. The smart display system may comprise a content creation unit operable to create one or more items of content. The or each item of content may comprise an advert. The advert may comprise any combination of the following: images, text and/or videos. Additionally, the or each item of content may comprise information associated with the advert. The associated information may comprise any combination of the following: the identity of the registered user, one or more metadata tags, one or more maximum bid amounts, a continuous daily budget, a total budget for a set number of days and/or dates for showing the advert. The or each maximum bid amount may be associated with the or each metadata or a combination of metadata tags. The input may be operable to receive the advert and/or associated information. The input may be operable to edit the associated information stored on the memory storage unit.
The associated information allows a registered user to provide detailed instructions for how an advert should operate. The maximum bid amounts being associated with metadata tags or a combination of metadata tags allows a registered user to prioritise certain metadata tags over others.
The smart display system may comprise a content selection unit operable to select the relevant item of content. The content selection unit may be operable to select the relevant item of content from the list of stored items of content. The content selection unit may be operable match one or more items of content to the image by comparing the metadata tags of the image with metadata tags of the or each item of content, wherein an item of content with one or more metadata tags which match one or more metadata tags of the image is matched. Additionally, the content selection unit may be operable to select the relevant items of content by ranking maximum bid amounts by largest value, wherein the item of content associated with the top ranked maximum bid amount is selected. Preferably the content selection unit is operable to rank only maximum bid amounts which are associated with matched metadata tags or combinations of matched metadata tags. The display screen may be operable to display the advert of the selected item of content. The display screen may be operable to display the item of content for a set time. The set time may be determined by the management unit. Alternatively, the associated information of each item of content may comprise the set time. The management unit may be operable to record the time, date and length of display of an item of content against the items of contents entries on the memory storage unit.
The smart display system may comprise a payment unit operable to charge the maximum bid amount ranked second plus one unit of monetary value to the registered user associated with the selected item of content. The payment unit may be operable to charge the registered user's method of payment via the details of the method of payment.
Using the details of the method of payment allows the smart display system to charge the registered user automatically, and so results in a more automated system. The imaging device may be operable to capture a period of video. The imaging device may be operable to capture a set of periods of video encompassing the entire time an item of content is on display. The multi-core image processing unit may be operable to process the or each period of video by applying eye tracking algorithms. Additionally, the multi-core image processing unit may be operable to determine dwell times of one or more determined persons in the image. The multi- core image processing unit may be operable to record a determined persons' dwell time and associated metadata tags on the entry for the displayed item of content on the list of the registered users' stored items of content. Additionally, the multi-core image processing unit may be operable to record a determined person's dwell time on the associated entry on the list of faces.
Dwell times give registered users a metric by which the success of an advert can be measured. The dwell times can be reviewed, and adverts adjusted according to the results. The multi-core image processing unit may be operable to process a period of video so as to determine if one or more features are gestures. Preferably, the multi- core image processing unit may be operable to determine if a feature is a gesture only if the feature is associated with the most prominent person. The most prominent person is either the person who comprises the largest feature determined to be a person in the image, or is the person with the best line of sight to the display screen 2. The management unit may comprise a list of known gestures, each entry comprising a metadata tag of a known gesture and a response. The multi-core image processing unit may be operable to transmit metadata tags for gestures of determined features to the management unit, which is operable on to then carry out the response associated with said gesture. The receiver may be operable to receive the metadata tags for gestures. The response can comprise either end the display of the current advert or display the current advert for an additional set amount of time. The additional set amount of time may be 5 seconds. Determining gestures and having responses to said gestures allows a person to interact with the smart display system. Restricting the determination of gestures to the most prominent person prevents the smart display system accidently interacting with multiple people at the same time. The person interacting with the smart display system can then skip through adverts at will, or stop on one of particular interest. The smart display system may comprise a plurality of imaging devices and display, each imaging device being associated with a display screen. Additionally, the smart display system may comprise a plurality of management units. Additionally, the associated information may comprise a list of display screens on which to display the item of content. The smart display system may comprise a server comprising a platform. The platform may comprise the multi-core image processing unit. Additionally, the platform may comprise any of the following: the memory storage unit, the content creation unit, the content selection unit, the registration unit, input and/or the database unit. The smart display system may comprise one or more billboards. Additionally or alternatively, the smart display system may comprise one or more wall displays. The or each billboard and/or wall display may comprise a display screen. Additionally, the or each billboard and/or wall display may comprise the associated imaging device.
The or each management unit may comprise part of a billboard or wall display and/or the platform. The or each management unit may comprise a computer. According to a second aspect of the present invention there is provided a method of displaying an item of content, comprising the steps of providing a smart display system according to the first aspect of the present invention, capturing an image, processing the captured image so as to determine features of the image and displaying an item of content relevant to the determined features of the image. The method of the second aspect of the present invention may incorporate any or all features of the system of the first aspect of the present invention, as desired or as appropriate.
Detailed Description of the Invention
In order that the invention may be more clearly understood one or more embodiments thereof will now be described, by way of example only, with reference to the accompanying drawings, of which:
Figure 1 shows a billboard which forms part of a smart display system according to an embodiment of the present invention; and
Figure 2 shows a block diagram of a smart display system according to an embodiment of the present invention.
As shown in figure 1, there is provided a billboard 1 which comprises a display screen 2, a management unit 4 and a camera 3. The camera 3 is positioned so as to take images of the area 5 in front of the billboard 1. The camera 3 is set on a timer so that images are only captured at set times, for example every second. In alternative embodiments the management unit 4 directs the camera 3 to take an image when it is time to change the currently displayed advert. In some embodiments the camera 3 is operable to record video, and the image may comprise a frame of a video.
As shown in figure 2, the smart display system 6 comprises the billboard 1 and a platform 7 on a server (not shown), wherein the platform 7 manages the billboard 1. Preferably the server will be a cloud server. A user can access the platform 7 either via an input 8 which comprises either a link to an application 9 (on, for example, a mobile phone or tablet) or a web browser 10. The platform 7 comprises a registration unit 11, which can register the or each user. The or each user can input their details via the input 8, comprising their identity (e.g. their name) and contact details (e.g. their address, telephone number and/or email address), which the registration unit 11 then registers. The or each user's details also comprise their method of payment, which can be, for example, a credit card, debit card and/or an online payments system. The platform comprises a memory storage unit 12, to which the or each user's registered details are saved.
The platform 7 comprises a content creation unit 13 which registered users use to generate their item of content, which comprises an advert. The advert can comprise a template which comprises text, one or more images and/or a video. Such videos can be up to 10 seconds in length. Users can either use a standard template on the content creation unit 13, or upload a different template. The users can also upload images and/or video for the advert, via the input 8. The or each item of content can also comprise associated information. Once the or each advert is generated information can be associated with the advert. This information can comprise metadata tags, which describe the audiences the user is interested in targeting. Such metadata tags can describe various features, including demographics such as gender, race, age ranges, clothing types, clothing colours, clothing brands, clothing accessories, car makes, car models, groups of people, types of groups of people, size of groups of people, sentiments, moods, faces and/or people. The metadata tags can be selected via the input 8 from a dropdown menu and/or inputted manually. The associated information also comprises one or more maximum bid amounts. The or each maximum bid amount lists a monetary value. In a preferred embodiment of the invention, one unit of the monetary value may be lp. Each metadata tag within the associated information has an associated maximum bid amount. In some embodiments the associated information can also comprise maximum bid amounts for selected combinations of tags. In preferred embodiments, the associated information also comprises a continuous daily budget and/or a total budget for a set number of days, and/or the dates for which the advert should be shown.
In some embodiments, the smart display system 6 comprises a plurality of billboards 1. In such embodiments, the associated information also comprises the places the advert should be shown.
Once an advert has been made and the associated information entered, the content creation unit 13 saves the item of content to the memory storage unit 12. Each item of content is saved in a list of the stored items of content, and to a list of the registered user's stored items of content. The registered user can access the memory storage unit 12 via the input 8 to view the list of registered user's stored items of content. Additionally, the registered user can generate new items of content via the content creation unit 13 by copying and then modifying an item of content saved on the memory storage unit 12, or delete items of content associated with the user from the memory storage unit 12.
In some embodiments of the invention, deleted items of content are removed from the list of stored items of content, but not the list of the registered user's stored items of content. In such embodiments, a note is added to the entry for the deleted item on the list of the registered user's stored items of content stating that the item of content is removed. The registered user can then reinstate the item of content, thereby removing the note and re-adding the item to the list of stored items of content.
Once an image has been captured by the camera 3 the management unit 4 uploads the image to the platform 7. This upload is via a transmitter 14. The management unit 4 also records the time and date of the image, and the billboard 1 with which the image is associated, and uploads these details to platform 7 in conjunction with the image.
The platform comprises a multi-core image processing unit 15, which applies computer vision algorithms to the image so as to process it. The computer vision algorithms can include optical character recognition algorithms and facial recognition algorithms. The multi-core image processing unit 15 comprises at least one multi-core processor, so as to allow the image to be processed in real time. Preferably the computer vision algorithms use pattern matching to identify features of the image, by scanning image libraries for objects which match a feature within the image. The or each feature can match a plurality of different objects (for example, a feature can be both a 'vehicle' and a 'car'). The multi-core image processing unit 15 also processes the image to determine the proximity of the determined features. Determined features which are outside a set proximity range are ignored.
Once the features of an image have been determined, the multi-core image processing unit 15 associates metadata tags with the image. In some embodiments the metadata tags are associated with an image by incorporating the tags into the code defining the image. Each metadata tag describes a feature determined to be in the photo, wherein the description is captured from the description of the matched object in the scanned image library.
In some embodiments the image libraries can include the DVLA vehicle database. In such embodiments the registration plate of a vehicle in the image can be determined, and the make and model metadata tagged to the image based on the listed information in the DVLA vehicle database.
When a feature is determined to be a person, the multi-core image processing unit 15 can also determines features in the image which are associated with the person.
When a feature is determined to be a group of people, the multi-core processing unit 15 also determines features in the image which are associated with the group. The multi-core image processing unit 15 then uses the features in the image which are associated with the group to determine the type of the group of people. For example, a group of people with a majority of females would be determined to be a female dominated group, or a group with a mixture of adults and children would be determined to be a family. The smart display system 6 comprises a database unit 16. The multi-core image processing unit 15 saves sections of the image comprising features determined to be faces in an entry on a list of faces on the database unit 16, the entry also comprising the determined demographics of the face, the date and time of the image and the billboard with which the image is associated. The multi-core image processing unit 15 also saves metadata tags associated with a person, including clothing types, brands and colours, clothing accessories and car makes and models, against the entry for the face of the person in the database unit 16. The multi-core image processing unit 15 can then use facial recognition algorithms to determine if features of an image match any faces saved to the database unit 16. Matched faces are saved to the same entry in the database unit 16. A registered user can access and search the database unit 16 via the input 8, to determine the demographics which frequent each billboard, and when they frequent it.
The smart display system 6 also comprises a content selection unit 17, to which images with metadata tags are sent. The content selection unit 17 accesses the memory storage unit 12 so as to obtain the list of the stored items of content. The content selection unit 17 then uses machine learning algorithms to match items of content with the image, via determining if each item of content has at least one metadata tag which matches a metadata tag of the image. In embodiments of the invention which have a plurality of billboards 1, the content selection unit 17 narrows down the matched items of content to those which should be displayed on the billboard 1 from which the image was transmitted.
The content selection unit 17 then ranks maximum bid amounts of the matched items of content, the highest maximum bid amount being ranked top. The content selection unit 17 only considers the maximum bid amounts which are associated with a matched metadata tag, or a combination of matched metadata tags. The item of content associated with the top ranked maximum bid amount is selected. The selected item of content is then sent to the billboard 1 to be received by receiver 18, for the advert to be displayed on the display screen 2. Each advert can be displayed on the billboard 1 for the same amount of time, which is determined by the management unit 4. In some embodiments, the amount of time is 10 seconds. In alternative embodiments, the associated information can comprise a set time for which the advert should be displayed. The set time is selected by the registered user from a range of times, ranging from a minimum time to a maximum time. In such embodiments the content creation unit 13 can set a minimum maximum bid amount for each item of content, the minimum being higher for items of content with higher set times.
The platform comprises a payment unit 19, which receives an indication from the content selection unit 17 that an item of content is to be displayed. The payment unit accesses the memory storage unit 12 to determine the method of payment of the user whose item of content was selected for display. The payment unit 19 then charges the maximum bid amount ranked second, plus one unit of monetary value, to the method of payment. The payment unit 19 comprises details of a bank account, into which the method of payment is directed to pay. The payment unit 19 then stores a record of the charge, comprising the date, time and amount, on the memory storage unit 12, against the entry for the selected item of content on the list of the registered user's stored items of content. Once an advert has been displayed on the display screen, the management unit
4 adds a value to the associated item of content's entry on the list of stored items of content. The value comprises a number which is non-zero. The content selection unit 17 cannot select an advert for display which has a note. For each subsequent advert which is displayed, the management unit 4 reduces all values on the memory storage unit 12 by 1. Once a value reaches zero it is deleted by the management unit 4.
In embodiments of the invention in which the associated information comprises either a continuous daily budget and/or total budget for a set number of days, the content selection unit 17 considers the charges against the entry for the selected item of content on the list of the registered user's stored items of content. If the maximum bid amount ranked second plus one unit of monetary value exceeds the difference between the daily budget and the total of the day's charges, or the difference between the total budget and the total of the charges for the set number of days, then the content selection unit removes the maximum bid amount ranked top from the rankings. This process is repeated until a maximum bid amount ranked second plus one unit of monetary value does not exceed the difference.
In embodiments of the invention in which the associated information comprises dates for showing the advert, the management unit 4 is operable to check the list of stored items of content once a day. If and when the current day does not match any of the dates for showing the advert, the management unit 4 is operable to remove the associated item of content from the list of stored items of content.
In some embodiments of the invention, if and when an item of content is displayed on a billboard 1, the management unit 4 can transmit a record of the display to the platform 7. The platform 7 can then save the record of the display to the memory storage unit 12, against the entry for the relevant item of content on the list of the registered user's stored items of content. The record can comprise the date, time and length of display. In such embodiments, the registered user can access this list to examine the record of displays for each item of content. In some embodiments of the invention the billboard 1 can operate via the
Android operating system.
In a preferred embodiment of the invention, while an advert is being displayed on the display screen 2, the management unit 4 directs the camera 3 to record periods of video. Each period of video extends for a period of time set by the management unit 4 (for example, 2 seconds) and a set of periods of video cover the entire time for which an advert is displayed. Each period of video, along with information detailing the date and time of recording and the identity of the associated billboard, is uploaded by the management unit 4 to the platform 7.
The multi-core image processing unit 15 determines features of period of video in the same way as described above for images. The multi-core image processing unit 15 also processes the period of video by applying eye tracking algorithms, to determine when the eyes of a feature determined to be a person are looking at the display screen. The dwell time of each person (which is the time spent looking at the display screen divided by the total time in which the display screen is within line of sight) is determined by the multi-core image processing unit 15. The dwell time of each person is then saved on the entry for the displayed advert on the list of the registered users' stored items of content, along with the metadata tags associated with the person. The dwell times can then be reviewed by the user to determine the effectiveness of the advert.
When processing a period of video one of the features which can be determined by the multi-core image processing unit 15 are gestures. The multi-core image processing unit 15 sends the metadata tag of a gesture associated with the person determined to be closest to the display screen to the management unit 4. The management unit 4 comprises a list of known gestures, each entry on the list comprising the metadata tag of a gesture and a response to carry out. The responses to carry out include, for example, to stop showing the displayed advert prematurely and select a new advert or to add a set amount of time to the display time for the current advert.
The above embodiments are described by way of example only. Many variations are possible without departing from the scope of the invention as defined in the appended claims.

Claims

1. A smart display system comprising an imaging device operable to capture an image, a multi-core image processing unit operable to process the captured image so as to determine features of the image and a display screen operable to display an item of content relevant to the determined features of the image.
2. A smart display system as claimed in claim 1 comprising a registration unit operable to register a user.
3. A smart display system as claimed in either claim 1 or claim 2 comprising a memory storage unit operable to store one or more items of content.
4. A smart display system as claimed in any preceding claim wherein the multi- core image processing unit is operable to process a captured image by applying computer vision algorithms to the image.
5. A smart display system as claimed in claim 4 wherein the computer vision algorithms comprise pattern matching algorithms, the pattern matching algorithms scanning at least one image library to match objects in the or each image library to features of the or each image.
6. A smart display system as claimed in any preceding claim wherein the multi- core processing unit is operable to process the image to determine the proximity of the determined features.
7. A smart display system as claimed in any preceding claim wherein the multi- core image processing unit is operable to apply one or more metadata tags to the image, the or each metadata tag comprising a description of a feature determined to be in the or each image.
8. A smart display system as claimed in claim 7 wherein the or each metadata tag may comprise any one of the following: gender, race, age ranges, clothing types, clothing colours, clothing brands, clothing accessories, car makes, car models, groups of people, types of groups of people, size of groups of people, sentiments, moods, faces, a person and/or a particular person.
9. A smart display system as claimed in claim 8 wherein the multi-core image processing unit is operable to determine features which are associated with a person.
10. A smart display system as claimed in either claim 8 to claim 9 to wherein the multi-core processing unit may be operable to determine the most prominent person.
11. A smart display system as claimed in any of claims 8 to 10 comprising a database unit operable to store sections of images comprising a feature determined to be a face as an entry on a list of faces.
12. A smart display system as claimed in claim 11, when dependent either directly or indirectly on claim 9, wherein the database unit is operable to store metadata tags associated with a face against the respective faces' entry.
13. A smart display system as claimed in either claim 11 or claim 12 wherein the multi-core image processing unit is operable to apply facial recognition algorithms to the image, wherein the list of faces is used as an image library.
14. A smart display system as claimed in any preceding claim comprising a content creation unit operable to create one or more items of content.
15. A smart display system as claimed in claim 14 wherein the or each item of content comprises associated information comprising any combination of the following: the identity of the registered user, one or more metadata tags, one or more maximum bid amounts, a continuous daily budget, a total budget for a set number of days and/or dates for showing the advert.
16. A smart display system as claimed in claim 15 wherein the or each maximum bid amount is associated with the or each metadata or a combination of metadata tags.
17. A smart display system as claimed in any preceding claim comprising a content selection unit operable to select the relevant item of content.
18. A smart display system as claimed in claim 17 wherein the content selection unit is operable match one or more items of content to the image by comparing the metadata tags of the or each image with metadata tags of the or each item of content, wherein an item of content with one or more metadata tags which match one or more metadata tags of the image is matched.
19. A smart display system as claimed in claim 18, when dependent either directly or indirectly on claim 16, wherein the content selection unit is operable to select the relevant items of content by ranking maximum bid amounts by largest value, wherein the item of content associated with the top ranked maximum bid amount is selected.
20. A smart display system as claimed in claim 19 comprising a payment unit operable to charge the maximum bid amount ranked second plus one unit of monetary value to the registered user associated with the selected item of content.
21. A smart display system as claimed in any preceding claim wherein the imaging device is operable to capture a period of video.
22. A smart display system as claimed in claim 21 wherein the imaging device is operable to capture a set of periods of video encompassing the entire time an item of content is on display.
23. A smart display system as claimed in either claim 21 or claim 22 wherein the multi-core image processing unit is operable to process the or each period of video by applying eye tracking algorithms and so determine dwell times of one or more persons in the image.
24. A smart display system as claimed in any of claims 21 to 23 wherein the multi-core image processing unit is operable to process a period of video so as to determine if one or more features are gestures.
25. A smart display system as claimed in claim 24, when dependent either directly or indirectly on claims 9 and 10, wherein the multi-core image processing unit is operable to determine if a feature is a gesture only if the feature is associated with the most prominent person.
26. A smart display system as claimed in either claim 24 or claim 25 comprising a management unit comprising a list of known gestures, each entry comprising a metadata tag of a known gesture and a response.
27. A smart display system as claimed in claim 26 wherein the multi-core image processing unit is operable to transmit metadata tags for gestures of determined features to the management unit, which is operable on receiving the metadata tag to carry out the response associated with said gesture.
28. A method of displaying an item of content, comprising the steps of providing a smart display system as claimed in any of claims 1 to 27, capturing an image, processing the captured image so as to determine features of the image and displaying an item of content relevant to the determined features of the image.
29. A method of displaying an item of content as claimed in claim 28 comprising registering a user.
30. A method of displaying an item of content as claimed in either claim 28 or claim 29 comprising storing one or more items of content.
31. A method of displaying an item of content as claimed in any of claims 28 to 30 wherein a captured image is processed by applying computer vision algorithms to the image.
32. A method of displaying an item of content as claimed in claim 31 wherein the computer vision algorithms comprise pattern matching algorithms, the pattern matching algorithms scanning at least one image library to match objects in the or each image library to features of the or each image.
33. A method of displaying an item of content as claimed in any of claims 28 to 32 comprising applying one or more metadata tags to the image, the or each metadata tag comprising a description of a feature determined to be in the or each image.
34. A method of displaying an item of content as claimed in claim 33 wherein the or each metadata tag may comprise any one of the following: gender, race, age ranges, clothing types, clothing colours, clothing brands, clothing accessories, car makes, car models, groups of people, types of group of people, size of groups of people, sentiments, moods, faces, a person and/or a particular person.
35. A method of displaying an item of content as claimed in claim 34 comprising determining features which are associated with a person.
36. A method of displaying an item of content as claimed in any of claims 28 to 35 comprising determining the proximity of the determined features.
37. A method of displaying an item of content as claimed in claim 36, when dependent either directly or indirectly on claim 34, comprising determining the most prominent person.
38. A method of displaying an item of content as claimed in any of claims 34 to 37 comprising storing a section of the image comprising a feature determined to be a face as an entry on a list of faces.
39. A method of displaying an item of content as claimed in claim 38 comprising storing metadata tags associated with a face against the respective faces' entry.
40. A method of displaying an item of content as claimed in either claim 37 or claim 38 comprising applying facial recognition algorithms to the image, wherein the list of faces is used as an image library.
41. A method of displaying an item of content as claimed in any of claims 28 to 40 comprising creating items of content.
42. A method of displaying an item of content as claimed in claim 41 wherein the or each item of content comprises associated information comprising any combination of the following: the identity of the registered user, one or more metadata tags, one or more maximum bid amounts, a continuous daily budget, a total budget for a set number of days and/or dates for showing the advert.
43. A method of displaying an item of content as claimed in claim 42 wherein the or each maximum bid amount is associated with the or each metadata or a combination of metadata tags.
44. A method of displaying an item of content as claimed in any of claims 28 to
43 comprising selecting the relevant item of content.
45. A method of displaying an item of content as claimed in claim 44 comprising matching one or more items of content to the image by comparing the metadata tags of the image with metadata tags of the or each item of content, wherein an item of content with one or more metadata tags which match one or more metadata tags of the image is matched.
46. A method of displaying an item of content as claimed in claim 45, when dependent either directly or indirectly on claims 42 and 43, comprising ranking maximum bid amounts by largest value, wherein the item of content associated with the top ranked maximum bid amount is selected.
47. A method of displaying an item of content as claimed in claim 46 comprising charging the maximum bid amount ranked second plus one unit of monetary value to the registered user associated with the selected item of content.
48. A method of displaying an item of content as claimed in any of claims 28 to 47 comprising capturing a period of video.
49. A method of displaying an item of content as claimed in claim 48 comprising capturing a set of periods of video encompassing the entire time an item of content is on display.
50. A method of displaying an item of content as claimed in either claim 48 or claim 49 comprising processing the or each period of video by applying eye tracking algorithms and determining dwell times of one or more persons in the image.
51. A method of displaying an item of content as claimed in any of claims 48 to 50 comprising processing a period of video so as to determine if one or more features are gestures.
52. A method of displaying an item of content as claimed in claim 51, when dependent either directly or indirectly on claim 37, wherein determining if a feature is a gesture is done only if the feature is associated with the most prominent person.
53. A method of displaying an item of content as claimed in either claim 51 or claim 52 comprising carrying out a response associated with said gesture.
PCT/GB2018/052190 2017-07-31 2018-07-31 Smart display system WO2019025790A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
GB1712309.2 2017-07-31
GB1712309.2A GB2565087A (en) 2017-07-31 2017-07-31 Smart display system

Publications (1)

Publication Number Publication Date
WO2019025790A1 true WO2019025790A1 (en) 2019-02-07

Family

ID=59778713

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/GB2018/052190 WO2019025790A1 (en) 2017-07-31 2018-07-31 Smart display system

Country Status (2)

Country Link
GB (1) GB2565087A (en)
WO (1) WO2019025790A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110197171A (en) * 2019-06-06 2019-09-03 深圳市汇顶科技股份有限公司 Exchange method, device and the electronic equipment of action message based on user

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130290994A1 (en) * 2012-04-27 2013-10-31 Leonardo Alves Machado Selection of targeted content based on user reactions to content
US20140002342A1 (en) * 2012-06-29 2014-01-02 Elena A. Fedorovskaya System for presenting high-interest-level images

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102129644A (en) * 2011-03-08 2011-07-20 北京理工大学 Intelligent advertising system having functions of audience characteristic perception and counting
US9996150B2 (en) * 2012-12-19 2018-06-12 Qualcomm Incorporated Enabling augmented reality using eye gaze tracking
IN2013MU02083A (en) * 2013-06-19 2015-07-10 Tata Consultancy Services Ltd
US20150262255A1 (en) * 2014-03-12 2015-09-17 Netseer, Inc. Search monetization of images embedded in text
CN204117556U (en) * 2014-08-22 2015-01-21 安徽工程大学 A kind of intelligent advertisement board
CN205406011U (en) * 2016-02-02 2016-07-27 郑州工业应用技术学院 Intelligence advertising tablet
CN106971317A (en) * 2017-03-09 2017-07-21 杨伊迪 The advertisement delivery effect evaluation analyzed based on recognition of face and big data and intelligently pushing decision-making technique

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130290994A1 (en) * 2012-04-27 2013-10-31 Leonardo Alves Machado Selection of targeted content based on user reactions to content
US20140002342A1 (en) * 2012-06-29 2014-01-02 Elena A. Fedorovskaya System for presenting high-interest-level images

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110197171A (en) * 2019-06-06 2019-09-03 深圳市汇顶科技股份有限公司 Exchange method, device and the electronic equipment of action message based on user

Also Published As

Publication number Publication date
GB2565087A (en) 2019-02-06
GB201712309D0 (en) 2017-09-13

Similar Documents

Publication Publication Date Title
JP6511024B2 (en) Consumer-driven advertising system
US11611795B2 (en) Online live video sales management system
CN110033298B (en) Information processing apparatus, control method thereof, system thereof, and storage medium
JP6148948B2 (en) Information processing system, information processing method, and information processing program
CN102542486A (en) Prioritizing advertisements based on user engagement
WO2010053191A1 (en) Electronic advertisement system, electronic advertisement distribution apparatus, electronic advertisement distribution method, and recording medium
JP5002441B2 (en) Marketing data analysis method, marketing data analysis system, data analysis server device, and program
JP7130991B2 (en) ADVERTISING DISPLAY SYSTEM, DISPLAY DEVICE, ADVERTISING OUTPUT DEVICE, PROGRAM AND ADVERTISING DISPLAY METHOD
CN111813986B (en) Intelligent advertisement pushing method, device, system, medium and electronic terminal
KR101657585B1 (en) System for Suggesting Product Dealing based on Hash Tag using Mobile Application and Method therefor
JP2016130915A (en) Drive-through system
US11295348B2 (en) On-line advertisement method using advertisement website
KR101737424B1 (en) Method and server for providing advertisement based on purchase and participation possibility of user
WO2019025790A1 (en) Smart display system
KR101657583B1 (en) System for Extracting Hash Tag of Product Image using Mobile Application and Method therefor
KR20160005169A (en) Accounting system of advertising costs depending on the watch webpage checking
KR20210052237A (en) Product catalog automatic classification system based on artificial intelligence
Turov et al. Digital signage personalization through analysis of the visual information about viewers
KR101657589B1 (en) System for Providing Product Dealing Information based on Product Image using Mobile Application and Method therefor
KR101657587B1 (en) System for Providing Individual Tendency Information based on Product Dealing Information using Mobile Application and Method therefor
CN110597379A (en) Elevator advertisement putting system capable of automatically matching passengers
US12002071B2 (en) Method and system for gesture-based cross channel commerce and marketing
JP7259312B2 (en) Point giving system, point giving method, and program
JP7190620B2 (en) Information processing device, information delivery method, and information delivery program
EP4123550A1 (en) Method for managing advertisement

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18759674

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18759674

Country of ref document: EP

Kind code of ref document: A1