FR3114419A1 - Augmented reality smart drawer system and method - Google Patents

Abstract

Augmented reality smart drawer system and method The invention is a mobile device for monitoring the status of cosmetic items in a drawer or on a shelf. The apparatus comprises a camera, a display device and processing circuits. The processing circuitry is configured to capture an image of the drawer and its contents using the camera, to decode the captured image to identify the cosmetic items contained in the drawer and the respective location of the identified cosmetic items, and to display on the display device a notification of the cosmetic items identified as being expired. An augmented reality display device highlights a group of cosmetic items determined to be compatible and within a degree of freshness. The processing circuitry is configured to display on the mobile display a notification of a cosmetic item to be used, highlight the cosmetic item on the augmented reality display device, and record in a database the information of use of the cosmetic article. Figure for abstract: 2

Description

Augmented reality smart drawer system and method

The present invention relates to an augmented reality smart drawer system and method. The system and method monitors expired cosmetics and the compatibility of cosmetics in a drawer and guides the application of cosmetics.

Description of context

The description of "background" provided herein is intended to generally set forth the background of the invention. The work of the presently named inventors, to the extent described in this background section, and aspects of the description which might not otherwise qualify as prior art at the time of filing, are neither expressly nor impliedly admitted. as prior art against the present invention.

Furniture intended for use when using cosmetics, including cosmetics cabinets and other furniture containing drawers to place cosmetics can allow cosmetics to be organized to some degree. Cosmetic products can be organized by placing them in drawers or on shelves by product types or application areas; eg, lips, eye treatment, face, nails. In many cases, a piece of furniture can be limited by the number of drawers and/or shelves and, accordingly, all cosmetics can be placed in one or two drawers in a mixed way. In other cases, a piece of furniture can have several drawers, so that cosmetics are distributed in the drawers and can even be mixed with non-cosmetic products, e.g. scissors, hairbrushes, cotton balls/cotton swabs; office supplies, eg, notepads, pens, as well as products intended to remove cosmetic products, eg, nail polish remover, make-up removers.

Different cosmetics can be purchased over time for different reasons. For the purposes of the present invention, the terms cosmetic product and cosmetic article are used interchangeably and each refers to a range of products which are used for the care of the face and body or to improve or modify the appearance of the face or body. Replacement cosmetics can be purchased when a particular product runs out or is near empty. New cosmetics may be purchased due to interest in a new color, a new type of product, or a new part of the body to apply the cosmetic to. Therefore, new cosmetics can be purchased even when a similar product has already been purchased.

Some cosmetics that may not have been used for a long time or are rarely used, may be expired or may be stored in a drawer with other unrelated products. Such disorganization can lead to difficulties in locating desired cosmetics, infrequently used products being mixed with more frequently used products and failing to realize that empty or nearly empty cosmetics need to be restocked. The time spent searching and the time spent applying cosmetics inconsistently can be excessive, in some cases, to the point where some cosmetics are not applied while others are applied in a hurry.

Summary

One aspect is a mobile device intended to monitor the status of cosmetic items in a drawer or on a shelf. The apparatus comprises a camera, a display device and processing circuits. The processing circuitry is configured to capture an image of the drawer and its contents using the camera; to decode the captured image to identify the cosmetic items contained in the drawer and the respective location of the identified cosmetic items and to display on the display device a notification of the cosmetic items identified as being expired.

One aspect is an augmented reality system to indicate the status of cosmetic items in a drawer or on a shelf. The system includes a mobile device having a mobile display, a camera, processing circuitry, and an augmented reality display device. The processing circuitry is configured to capture an image of a drawer and its contents, recognize the cosmetic items contained in the drawer, analyze the recognized cosmetic items for compatibility and degree of freshness, and the display device augmented reality showcasing a group of cosmetic items determined to be compatible and exhibiting a certain degree of freshness.

One aspect is an augmented reality system to guide the application of cosmetic items that are in a drawer or on a shelf. The system comprising a mobile device having a mobile display, a camera and processing circuitry and an augmented reality display device. The processing circuitry is configured to display on the mobile display a notification of a cosmetic item to be used, highlight the cosmetic item on the augmented reality display device, and record in a database the information of use of the cosmetic article.

The above general description of illustrative embodiments and the following detailed description thereof are merely exemplary aspects of the teachings of the present invention and are not intended to be limiting.

A fuller appreciation of the present invention and the number of attendant advantages thereof will be readily obtained as these are better understood by reference to the following detailed description when considered in connection with the accompanying drawings in which :

Figure 1 comprises three Figures 1A, 1B and 1C, each of these figures representing a block diagram of an intelligent cosmetic drawer apparatus in accordance with an exemplary aspect of the invention;

Figure 2 is a block diagram of an augmented reality system for cosmetics in accordance with an exemplary aspect of the invention;

Figure 3 is a block diagram of a mobile device;

Figure 4 is a flowchart for a cosmetic product with an exemplary aspect of the invention;

Figure 5 is a display screen of a mobile device in accordance with an exemplary aspect of the invention;

Figure 6 is a flowchart of a method of guiding the application of cosmetics in accordance with an exemplary aspect of the invention and

Figure 7 is a flowchart of a method of bringing out compatible cosmetic products in accordance with an exemplary aspect of the invention.

detailed description

In the drawings, like reference numerals designate the same or corresponding parts throughout the various views. Further, as used herein, the words "one", "one" and similar words have, unless otherwise specified, generally a meaning of "one or more". The diagrams are, unless otherwise indicated, generally drawn to scale or illustrate schematic structures or flowcharts.

Furthermore, the terms “approximately”, “approximately” or “about” and similar terms generally refer to ranges that include the identified value with a margin of 20%, 10%, or preferably 5%, and all values between them.

Aspects of the present invention relate to an augmented reality system and method for tracking cosmetics in a smart drawer mobile application. One aspect is a method of tracking cosmetic product inventory in a smart drawer mobile application. One aspect is a method giving advice on the use of cosmetics using a smart drawer mobile app. One aspect is a smart drawer mobile application configured to identify compatible cosmetic products.

Figures 1A, 1B, 1C are block diagrams of cosmetic furniture according to an exemplary aspect of the invention; Conventional furniture for storing cosmetic products may contain one or more drawers and/or shelves, at least one mirror and, in some cases, integrated lighting. A high-end cosmetic cabinet may have a UV curing compartment. However, a classic cosmetic cabinet leaves the user free to decide where to place cosmetic items. Cosmetic items can be placed in an organized fashion or randomly. Even an organized arrangement of cosmetics is not without problems. Some cosmetic items may be depleted or low, but a subject user may not realize there is little of the cosmetic item left until it is needed. A subject user may not realize that certain cosmetic items are expired or close to their expiration date. A subject user may waste time by randomly or inconsistently using cosmetic items.

Referring to Figure 1A, a cosmetics cabinet 101 may include a drawer 111 configured to track product 113 position and inventory. The cosmetics cabinet 101 may include a proximity sensor 105 which can detect when a user is close to the cosmetics cabinet 101. The cosmetics cabinet 101 may include an integrated light 107 and a mirror 103. The cosmetics cabinet cosmetics 101 may include an optional charging station 109 to provide power to rechargeable electronic devices.

Referring to Figure 1B, a cosmetics cabinet 121 may include multiple drawers 127, with at least one of the drawers including position tracking and inventory management. Cosmetics cabinet 121 may include integrated lighting 123 and mirror 125. Cosmetics cabinet 121 may include a proximity sensor 129 that can detect when a subject user is near cabinet 101.

Referring to Figure 1C, a cosmetics cabinet 131 may be a dressing table. Cosmetics cabinet 131 may include integrated video screen 133, mirror 135 and docking station 139. Drawer 137 may be configured to track cosmetics position and inventory. Another drawer can be a 141 UV drying compartment to store items that need to be sterilized such as combs, brushes, scissors and other items that are used to apply cosmetics. The built-in 133 video screen can provide guidance on the steps of applying cosmetics. Although not pictured, cosmetics cabinet 131 may include integrated lighting.

Figure 2 is a block diagram of an augmented reality system for interacting with cosmetic products in accordance with an exemplary aspect of the invention. Cosmetic items contained in a physical drawer 111, 127, 137 may be displayed as an augmented camera image on a display screen 201. Display screen 201 may be an integrated video screen 133 or may be a separate display device, such as a tablet or other portable display device. The camera image of a physical drawer can be augmented with backlight type highlighting to indicate special status information about a cosmetic item. Highlighting a cosmetic item, especially item 1 (203), using a certain color, such as green 205, can be used to indicate that item 1 has been synced with a remote device 221 , in particular selected with a command received from the mobile device 215 in order to remotely display the status content 225 of the cosmetic article 1 selected. Highlighting an item, such as item 4 (207), using a certain color, such as red 209, can be used to indicate an alert that item 4 is out of date or is on the point of being.

The augmented display for display screen 201 can be achieved using display circuits 211 which include a video controller 213, a communication module 215 and a power supply or battery 217. The communication module 215 can be configured to establish and perform communications with mobile device 221. Communication module 215 can be configured to perform communications using Bluetooth, WiFi, or cellular protocol, to name a few.

A mobile device 221 may include a mobile application, also called an App, which communicates with display circuitry 211, provides notifications and guidance to a subject user, and performs processing operations on images obtained using a camera of the device 221. In some embodiments, the mobile device 221 may use the services of a cloud service 231 to perform some of the processing operations.

In one implementation, the processing functions and operations of mobile device 221 may be performed by one or more of respective processing circuits 326. A processing circuit includes a programmed processor because a processor includes circuitry. A processing circuit can also include devices such as an application-specific integrated circuit (ASIC) and conventional circuit components arranged to accomplish the listed functions. It should be noted that circuits refer to a circuit or a system of circuits. Circuits herein may be within a computer system (as shown in Figure 3) or may be distributed throughout a network of computer systems. Therefore, the circuitry in a server or example computer system may be on a single server or be distributed across different servers/computers.

Next, a hardware description of processing circuitry 326 based on exemplary embodiments is described with reference to Figure 3. In Figure 3, processing circuitry 326 includes a Mobile Processing Unit (MPU) 300 that performs the processing operations described herein. Data and processing instructions may be stored in memory 302. Such data and processing instructions may also be stored on a portable storage medium or may be stored remotely. Processing circuitry 326 may have a replaceable Subscriber Identity Module (SIM) 301 which contains information that is unique to the network service of mobile device 221.

Further, the disclosed embodiments are not limited by the form of the computer-readable media on which the instructions are stored. For example, instructions may be stored in FLASH, Synchronous Dynamic Random Access Memory (SDRAM), Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read Only Memory (PROM), Erasable Programmable Read Only Memory (EPROM), on an Electrically Erasable Programmable Read Only Memory (EEPROM), on a solid-state hard disk or on any other information processing device with which the processing circuit 326 communicates, in particular a server or a computer.

Further, the disclosed embodiments may be provided as a utility application, background daemon, component of an operating system, or a combination thereof, running in conjunction with the MPU 300 and a mobile operating system such as Android, Microsoft® Windows® 10 Mobile, Apple iOS® and other systems known to those skilled in the art.

In order to obtain the processing circuit 326, the hardware elements can be produced by various circuit elements known to those skilled in the art. For example, the MPU 300 may be a Qualcomm mobile processor, an Nvidia mobile processor, an Atom® mobile processor from Intel Corporation of America, a Samsung mobile processor or an Apple A7 mobile processor or perhaps other types of processors which will be recognized by a person skilled in the art. Alternatively, the MPU 300 can be implemented on a Field-Programmable Gate Array (FPGA), an Application Specific Integrated Circuit (ASIC), a Programmable Logic Device (PLD) or using discrete logic circuits, which a person skilled in the art would recognize. Further, the MPU 300 may be implemented as multiple processors working cooperatively in parallel to perform the instructions of the inventive processes described above.

Processing circuitry 326 in Figure 3 also includes a network controller 306, such as an Intel Ethernet PRO network interface card from Intel Corporation of America, for interfacing with a network 324. In the disclosed embodiments , network 324 includes access to a cloud service 231. As can be appreciated, network 324 may be a public network, including the Internet, or a private network such as a local area network (LAN) or a wide area network (WAN) or any combination thereof and may also include PSTN or ISDN subnets. The network 324 can also be wired, in particular an Ethernet network. The processing circuit may include various types of communication processors for wireless communications, including 3G, 4G and 5G wireless modems, WiFi®, Bluetooth®, GPS or other known forms of wireless communication.

The processing circuit 326 includes a Universal Serial Bus (USB) controller 325 which can be managed by the MPU 300.

The processing circuit 326 further includes a display controller 308, such as an NVIDIA® GeForce® GTX or Quadro® graphics adapter from NVIDIA Corporation of America to interface with the display 310. Inputs/Outputs 312 interfaces with buttons 314, in particular for volume control. Further, in addition to I/O interface 312 and display 310, processing circuitry 326 may further include a microphone 341 and one or more cameras 331. Microphone 341 may have associated circuitry 340 for processing sound into digital signals. Similarly, camera 331 may include a camera controller 330 for controlling the image capture operation of camera 331. In an exemplary aspect, camera 331 may include a Charge Coupled Device (CCD). Processing circuitry 326 may include audio circuitry 342 for generating sound output signals and may include an optional sound output port.

The Power Management and Touchscreen Controller 320 manages the power used by the 326 processing circuitry and touch control. The communication bus 322, which may be from Industry Standard Architecture (ISA), Extended Industry Standard Architecture (EISA), Video Electronics Standards Association (VESA), Peripheral Component Interface (PCI) or similar, is used to interconnect all the components of the processing circuit 326. A description of the general features and functions of display 310, buttons 314, as well as display controller 308, power management controller 320, network controller 306, and I/O interface 312 is omitted herein for brevity because these characteristics are known.

Figure 4 is a flowchart for the inventory of cosmetic products, consistent with an exemplary aspect of the invention. New cosmetics can be purchased, S401. The new cosmetic products can be applied to a subject user, in S403 and/or placed in S405 or S407, in a drawer 111, 127, 137.

In some embodiments, at step S409, a subject user may activate the mobile device's application (App) when new cosmetics are purchased, when cosmetic items need to be moved, or simply to check the status of stored cosmetic items in a cabinet for cosmetic products. Upon activation of the App, the App may begin by displaying a reminder to the subject user that if any changes have been made to the contents of the drawers, a photograph should be taken of the drawers and/or shelves to ensure that the App has the latest cosmetic product status information As an alternative to a reminder, the App may present a display that asks the subject user if any changes have been made to one any of the cosmetic items, including moving certain items, placing new items, or deleting items. The App may present a message asking the Subject User to take a photograph of one or more drawers and/or shelves containing cosmetics to confirm if any changes have been made since the last (s) photograph(s) has (have) been taken.

Upon placement in a drawer 111, 127, 137, in S411, a camera 331 of mobile device 221 may be used to capture an image of the top view of the open drawer. In some embodiments, multiple images of the contents of the drawer may be taken from different angles in an effort to obtain product information for each cosmetic item in the drawer. In some embodiments, video of the open drawer may be captured using a video feature of mobile device 221. Mobile device 221 may be configured with a calibration process, in which camera controller 330 senses the four camera angles. 'a drawer. An angle 253 can serve as a reference which is used by the display screen 201 for alignment with an angle of the screen. An angle 251 can be used as an error check code. An angle 255 can serve as a marker for the length dimension of the drawer. An angle 257 can be used as a dimension of the width of the drawer. The camera controller 330 can adjust the zoom of the camera 331 to obtain a full screen image of the drawer. Display circuitry 211 aligns the camera image with display screen 201 using the code associated with the angles. The angles can also be used as reference coordinates to determine the coordinates of the position of the cosmetic item.

In some embodiments, an image of the new cosmetic product, or at least its label, can be taken using camera 331 before placing the cosmetic product in the drawer. The mobile device 221 can identify the cosmetic product using the product image or the product label or the mobile device 221 can send the product image to the cloud service 231 to identify the cosmetic product. The product image can be subjected to image processing to extract features such as product shape and product color. The image of the product label can be subjected to optical character recognition (OCR) with the possibility of extracting the resulting text information.

In S413, the drawer top view image can be decoded to identify cosmetic items inside the drawer, detect the location of cosmetic items, and determine whether a cosmetic item has been opened. Cosmetic items can be recognized using image processing circuits by extracting certain features from a given image and comparing the selected features with the products inside a database. In some embodiments, a database of cosmetic product images, image features, and associated product information are maintained in the cloud service 231.

The characteristics of a cosmetic product stored in a database may include the three-dimensional shape of the container of the cosmetic product, the dimensions of the three-dimensional shape of the container, the location of a product label on the container, whether the container is transparent or not, the type of container (e.g., jar, stick/tube, paste dispenser, bottle, sprayer, bottle-applicator combination, metal (tinplate) can, box, pad, pencil) and the color.

Product information that may be stored in the database may include product identification number, product name, product category (including broad cosmetic category: skin care, personal hygiene , perfume or alternatively an area of the body – lips, face, eyelashes, eye area, nails or a specific category: lipstick, mascara, eye shadow, foundation, blush, highlighter, bronzer. The database may also contain information on the applicators of products, in particular brushes and pads.

In S415, in addition to information for identifying cosmetic products, a database can be maintained containing a status for each cosmetic product. Status information can include the quantity of a cosmetic product. If there are multiple cosmetics, each cosmetic item can include an initial product placement date, (date a product is first placed in a drawer), an expiration date, and contact information for the location in a drawer.

In some embodiments, a drawer 111, 127, 137 may be configured with a motion sensor to detect the movement of items inside the drawer, the placement or removal of a cosmetic item from the drawer, or just opening a drawer. In the event that an event is detected using the motion sensor, a message may be displayed on a screen of mobile device 221. The message may be a notification message that an inventory change may have occurred. The message may further recommend that one or more images be taken of the contents of the drawer.

In the event that a motion detector detects the placement of a cosmetic item in a drawer, the mobile device 221 may be configured to determine, in S413, using the captured images, in S411, whether a product has been purchased or renewed (S417). In the event that a motion detector detects the movement of cosmetic items in a drawer, the mobile device 221 may be configured to determine, at S413, using the captured images, at S411, whether the cosmetics inside the drawer have been reorganized (S419).

Furthermore, the mobile device 221 can detect that a product is expired and or is about to expire (S421). In the event that the mobile device 221 determines that the cosmetic product is expired or is about to expire, at S423 the mobile device 221 may display an expired cosmetic product notification message.

Figure 5 shows a screen of a mobile device according to an exemplary aspect of the invention. Mobile device 221 may be configured to display notification messages 225 as part of smart drawer mobile application 501.

Figure 6 is a flowchart of a method of guiding application of cosmetics in accordance with an exemplary aspect of the invention. Subject to known placement of cosmetic items in drawer 111, 137, or multiple drawers 127, screen 201 may advise on placement of cosmetic items in a sequence that is appropriate for a particular cosmetic application. A particular cosmetic application may include a specific part of the body and specify whether the application is for care and cleansing, replenishment and protection, or to enhance appearance.

The mobile device 221 may display notification messages in a specific sequence for the application of the cosmetic items. The display device 201 can display the location of each cosmetic item. Referring to Figure 6, at S601, a notification message for a cosmetic item to be used next is displayed by the mobile device 221.

At S603, a display of a drawer 111, 127, 137 and its contents is displayed, augmented with a highlight of the border of a cosmetic item that is listed in the mobile device screen 221 such that the item can be located in the drawer. An augmented display containing highlighting can be produced within mobile device 221 by performing edge detection and object recognition. Once the objects in an image are recognized, the border of the selected objects can be made transparent for a predetermined pixel width. The image with the transparent borders around certain objects can be overlaid with a backlit image of a certain color or a solid white image.

In some embodiments, video captured by mobile device 221 may be subjected to augmented reality processing to produce a backlight effect at the edges of selected objects. Augmented reality software may be implemented in the mobile device 221 or may be implemented in a cloud service 231. In one embodiment, the augmented reality software uses the ARCore software development kit available from Google. Images of different cosmetic products are stored in a cosmetic product database. It is preferred that the cosmetics database be kept in the cloud service 231 so that it can be updated for all users. The augmented reality software determines the coordinates for the concrete drawer or shelf that is captured in the video. The augmented reality software then performs feature detection and can recognize objects based on the detection of images stored in the cosmetics database. Recognized objects in the scene can be assigned markers to track with the objects.

In some embodiments, a subject user may select particular recognized objects for focus.

Augmented reality software may also recognize text in the video using optical character recognition (OCR) and may use the recognized text for identification of recognized objects as well as to provide other information about an object, including a product name.

The augmented reality software will provide object location information and, along with recognized object identification information, can extract product information from a local cosmetics database for cosmetic items at the interior of a cosmetic cabinet, including date of initial placement, number of times it has been used, other compatible products.

In S605, the cosmetic article can be applied according to the instructions of the notification message. In S607, an archive of the application of the cosmetic article can be stored in the database. In some embodiments, a cosmetic item may be characterized by an expected number of uses until it is exhausted. Mobile device 221 can keep track of the number of uses of a cosmetic item so that mobile device 221 can predict when the cosmetic item may need to be replaced or discarded. In S609, the mobile device 221 can determine if there is a next cosmetic item to apply in accordance with the cosmetic application.

In some embodiments, the particular cosmetic applications may include cosmetic items that may be stored with reference to a personal profile. Further, cosmetic items may be selected from those known in the database to be located in a drawer 111, 127, 137. If a cosmetic item that is referenced in a personal profile is missing or nearly sold out, a notification message may be displayed on the mobile device 221 to indicate that the cosmetic item must be purchased before the cosmetic application can be performed. In some embodiments, a particular cosmetic application may be for a cosmetic application template. For example, there might be a cosmetic app template for an actor profile.

Figure 7 is a flowchart of a method of guiding application of cosmetics in accordance with an exemplary aspect of the invention and, in certain embodiments, it may be useful to identify items cosmetics for cosmetic application which are compatible with each other. Compatible cosmetic articles in the context of a cosmetic application can be defined as cosmetic articles which are of compatible colors or which are composed of compatible chemicals. In some embodiments, compatible cosmetic items may be those which are compatible with cosmetic items used for other parts of the body.

For example, cosmetic articles such as those used for lips may need to be compatible with cosmetic articles used for eyes.

Referring to Figure 7, at S701 the mobile device 221 can be used to capture an image or video of an open drawer or shelf containing cosmetic items, and at S703 circuitry in the mobile device 221 or cloud service 231 can perform recognition of one or more cosmetic items inside the drawer or on the shelf. At S705, an analysis may be performed to determine the compatibility and freshness of recognized cosmetic items. In some embodiments, the compatibility of cosmetic items can be determined using a look-up table containing a pre-determined matrix that establishes a match between compatible cosmetic items. The freshness of cosmetic items can be determined using dates stored in the database, where dates that are within a predetermined duration are considered to have similar freshness. In S707, an increased display of the drawer or shelf can be displayed on the display screen 211 which highlights compatible products with the same colors.

Many modifications and variations of the present invention are possible in light of the above teachings. It is, therefore, understood that within the scope of the appended claims, the invention may be practiced other than as specifically described herein.

Claims (10)

A mobile device for monitoring the status of cosmetic items inside a drawer or on a shelf, the device comprising:
a camera,
a display device, and
processing circuits,
wherein the processing circuits are configured to
capture an image of the drawer and its contents using the camera,
decode the captured image to identify the cosmetic items contained in the drawer and the respective location of the identified cosmetic items, and
displaying on the display device a notification of the identified cosmetic items that are expired. The mobile device of claim 1, wherein the circuitry decodes the captured image by performing image analysis to extract a shape from the cosmetic items and
wherein the circuitry matches the shape of the cosmetic items with the shape stored in memory and with related identification information to identify the cosmetic items. The mobile device of claim 1, wherein the circuitry performs an augmented reality image processing which augments the captured image of the drawer and its contents with the highlighting around the border of a cosmetic item in which the check mark indicates that the cosmetic item is expired, and
wherein the circuitry displays the augmented captured image on the display device as a notification. An augmented reality system for indicating the status of cosmetic items inside a drawer or on a shelf, the system comprising:
a mobile device having a mobile display, camera and processing circuitry, and
an augmented reality display device,
the processing circuits being configured to
capture an image of a drawer and its contents,
recognize the cosmetic items contained in the drawer and
analyze recognized cosmetic items for compatibility and degree of freshness, and
the augmented reality display device highlighting a group of cosmetic items determined to be compatible and within the degree of freshness. The augmented reality system of claim 4, wherein the processing circuitry is further configured to
recognize cosmetic items by performing image analysis on the captured image to extract the shape of the cosmetic items and
to match the shape of the cosmetic articles with a database that stores the shapes and related identifying information to identify the cosmetic articles. The augmented reality system of claim 5, wherein the processing circuitry is further configured to
perform an augmented reality image process that augments the captured image of the drawer and its contents by highlighting the border of the selected cosmetic item and the border of cosmetic items that are compatible and
displaying on the display device the augmented captured image including the highlighted selected cosmetic item as well as the highlighted compatible cosmetic items. The augmented reality system of claim 6, wherein the processing circuitry is further configured to
displaying highlighting around the border of the selected cosmetic item in one color and highlighting around the border of compatible cosmetic items with the same color. An augmented reality system for guiding the application of cosmetic items within a drawer or on a shelf, the system comprising:
a mobile device having a mobile display, a camera and processing circuitry and
an augmented reality display device,
the processing circuits being configured to
displaying on the mobile display a notification of a cosmetic item to be used,
highlight the cosmetic item in the augmented reality display device and
save in a database the information of use of the cosmetic article. The augmented reality system of claim 8, wherein the processing circuitry is further configured to display in the augmented reality display device a drawer and its contents augmented with highlighting around a border of a cosmetic item to use as listed in the screen on the mobile display. The augmented reality system of claim 8, wherein the processing circuitry is further configured to
capture an image of the drawer and its contents using the camera,
decode the captured image to identify the cosmetic items contained in the drawer and the respective location of the identified cosmetic items and
displaying on the augmented reality display device the highlight around a border of the cosmetic item based on the respective location of the cosmetic item.