WO2021102417A1 - Système de miroir intelligent pour la collecte et la visualisation d'informations d'apparence physique - Google Patents
Système de miroir intelligent pour la collecte et la visualisation d'informations d'apparence physique Download PDFInfo
- Publication number
- WO2021102417A1 WO2021102417A1 PCT/US2020/061822 US2020061822W WO2021102417A1 WO 2021102417 A1 WO2021102417 A1 WO 2021102417A1 US 2020061822 W US2020061822 W US 2020061822W WO 2021102417 A1 WO2021102417 A1 WO 2021102417A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- user
- mirror
- collecting
- visualizing
- technology
- Prior art date
Links
Classifications
-
- A—HUMAN NECESSITIES
- A47—FURNITURE; DOMESTIC ARTICLES OR APPLIANCES; COFFEE MILLS; SPICE MILLS; SUCTION CLEANERS IN GENERAL
- A47G—HOUSEHOLD OR TABLE EQUIPMENT
- A47G1/00—Mirrors; Picture frames or the like, e.g. provided with heating, lighting or ventilating means
- A47G1/02—Mirrors used as equipment
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/57—Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
Definitions
- the present invention relates generally to an imaging and display system. More specifically, the present invention relates to a smart mirror system and/or device for collecting and visualizing a subject’s physical appearance information using sensors, cameras, and artificial intelligence technologies.
- the conventional mirror i.e., reflective surface
- the conventional mirror is the most common and reliable tool for individuals to use in reviewing their appearance in real-time, with many people looking in mirrors daily to check their appearance.
- the data that users obtain from traditional mirrors immediately disappear after walking away from the mirror.
- the present invention aims to solve the aforementioned problems and/or improve on conventional devices by integrating sensors and other related devices into a set of mirrors that form a substantially improved image collecting and display device.
- the present invention offers a smart mirror system that effectively and efficiently collects and visualizes appearance information and data of a user with artificial intelligence (AI) technologies, including, but not limited to, facial recognition, machine learning, deep learning.
- AI artificial intelligence
- the smart mirror system acquires the user’s appearance data using various sensors and multiple cameras while the user looks in a mirror of the smart mirror system.
- the cameras can capture three-dimensional (3D) image/video data of the user in a 360-degree range.
- the sensors can measure distances of all visible features of the user’s appearance including outfit, pose, etc., measure body temperature, color, etc. of the user. Additionally, the sensors can measure environmental temperature, relative humidity, light and color, etc.
- the smart mirror system allows the user to access the collected appearance information even when away from the mirror.
- the user’s appearance data that are effortlessly collected enables predictive analyses of future conditions based on the acquired appearance data and detects changes over time.
- the smart mirror system allows the user to obtain many categories of appearance information, including, but not limited to, current appearance, daily outfit, changes in body shape/condition, and medical condition, per the desired use of the acquired data. Additionally, the mirror system acquires video data together with sensor data to present videos including, but not limited to, dressing, clothing fittings, shaving, hair-dressing, face makeup, poses, exercises, etc., to the user.
- the mirror system can provide simulated images and/or videos in the form of a three-dimensional (3D) avatar to the user using AI technologies, including, but not limited to, deep learning, machine learning, facial recognition, etc.
- the 3D avatar may include, but is not limited to, a recommended outfit, pose, user’s body measurements and/or simulations, etc.
- the mirror system may present videos including, but not limited to, an instructor’s exercise video, a standard/recommended pose, etc., to the user along with the current video of the user.
- the mirror system may provide multiple mirrors to catch a 3D image/video of the user.
- the smart mirror system can be made in one piece, or a plurality of mirrors connected in a form of a foldable and movable system that further provides physical flexibility to the user.
- a computing device of the smart mirror system may include a memory and allow the user appearance data to be saved on the system.
- the computing device provides secure data transfer to any extraneous storage medium, including, but not limited to, a portable disk drive, a flash drive, etc.
- the computing device allows the mirror system to communicate to an extraneous device, including, but not limited to, a server, a computer, a cloud device, a mobile device, a personal computing (PC) device, etc., through a wireless network, including, but not limited to, Internet.
- the present invention offers the user a versatile, flexible, user-friendly, and effective smart mirror system.
- FIG. 1 is a front view of a smart mirror system of the present invention.
- FIG. 2 is a perspective rear view of one embodiment of the present invention, wherein a computing device attached to the back face of the smart mirror system.
- FIG. 3 is a perspective rear view of the smart mirror system of the present invention in a folded state.
- FIG. 4 is a top view of the smart mirror system of the present invention.
- FIG. 5 is an illustration showing one embodiment of the smart mirror system of the present invention, wherein a display is attached to the back face of the mirror.
- FIG. 6 is an illustration showing another embodiment of the smart mirror system of the present invention, wherein the display is attached to the mirror surface of the mirror.
- FIG. 7 is a system diagram of a computing device of the smart mirror system of the present invention.
- FIG. 8 is a flowchart diagram of the computing device of the present invention, wherein the computing device communicates with a server and a personal computing (PC) device through an extraneous network.
- PC personal computing
- FIG. 9 is an illustration showing one embodiment of the smart mirror system of the present invention in use, wherein acquired sensor data and a three-dimensional (3D) avatar of the user is displayed on the display.
- FIG. 10 is an illustration showing another embodiment of the smart mirror system of the present invention in use, wherein acquired sensor data, outfit recommendations, and a simulated three-dimensional (3D) avatar of the user are displayed on the display.
- FIG. 11 is an illustration showing another embodiment of the smart mirror system of the present invention in use, wherein acquired sensor data, user’s pose, and an instructor’s pose are displayed on the display.
- the present invention comprises a smart mirror system that effectively and efficiently acquires and visualizes appearance information of a user with artificial intelligence (AI) technologies through the use of multiple cameras and various sensors.
- the smart mirror system of the present invention comprises a mirror assembly 10, a display 50, and a computing device 70. More specifically, the mirror assembly 10 comprises a mirror 11, a plurality of cameras 12, and a plurality of distance detectors 13.
- the mirror 11 comprises a mirror surface 32 and a back face 33. Both the plurality of cameras 12 and the plurality of distance detectors 13 are distributed on the mirror surface 32 of the mirror 11. Additionally, both the plurality of cameras 12 and the plurality of distance detectors 13 are electronically connected to the computing device 70 through a wired or wireless connection.
- the display 50 is attached to the mirror 11 of the mirror assembly 10 and is electronically connected to the computer device 70 as well.
- the computing device 70 is mounted to the back face 33 of the mirror 11 of the mirror assembly 10. Additionally, the computing device 70 is adapted to acquire a plurality of appearance data of the user through the plurality of cameras 12 and the plurality of distance detectors 13. Further, the computing device 70 is adapted to display appearance information on the display 50 using the AI technologies, including, but not limited to, machine learning, deep learning, facial recognition, etc.
- the mirror 11 of the smart mirror assembly 10 may be a foldable mirror.
- a foldable mirror may be one continuous piece of mirror, or comprises a plurality of mirrors 31.
- the mirror 11 can include a special mirror that allows light through it to display information/data directly on the mirror 11 and the rest of the mirror surface 32 can reflect an object as does a normal mirror.
- the mirror assembly 10 comprises the plurality of cameras 12 and the plurality of distance detectors 13, a plurality of sensors 14, a light 15, a speaker 16, a plurality of handles 17, and a plurality of wheels 18.
- the plurality of cameras 12 may include, but is not limited to, a video camera, a Charge Coupled Device (CCD) camera, a camera and/or camera set-up enabling the capturing of 3D-images, an Intel ® Realsense D400 series camera, a stereoscopic camera, a camera integrated with infrared (IR) sensor, a camera comprising image sensors, including, but not limited to, CCD, Complementary Metal Oxide Semiconductor (CMOS), X3 ® PRO 10MTM CMOS image sensor from Foveon ® , etc.
- CCD Charge Coupled Device
- a camera and/or camera set-up enabling the capturing of 3D-images
- an Intel ® Realsense D400 series camera a stereoscopic camera
- IR infrared
- a camera comprising image sensors including, but not limited to, CCD, Complementary Metal Oxide Semiconductor (CMOS), X3 ® PRO 10MTM CMOS image sensor from Foveon
- the plurality of distance detectors 13 may include, but is not limited to, an IR sensor, a light detection and ranging (LiDAR) sensor, a short frequency infrared sensor, touchless LiDAR sensor, a LiDAR camera, an Intel ® Realsense L515, etc.
- the plurality of sensors 14 is distributed across the mirror surface 32 of the mirror 11 and electronically connected to the computing device 70.
- the plurality of sensors 14 may include, but is not limited to, a temperature sensor that measures room temperature and/or the user’s body temperature, a relative humidity sensor that measures room relative humidity, a proximity sensor, a touchless proximity sensor, a motion sensor, a color sensor, a touch sensor, a light sensor, a brightness sensor, a light dependent resistors (LDR) sensor, a microphone/sound sensor, IR sensor, etc.
- the IR sensor identifies the user’s height, shape, and distance. Additionally, the IR sensor allows the user’s image to be distinguished from the background.
- the color sensor is used to identify color, providing advanced pixel data for the captured image and helping to identify the user from the background.
- the light sensor can measure the illuminance and automatically adjust the brightness so that the subject can be properly illuminated with appropriate brightness.
- the plurality of sensors 14 provides the sensor data to the computing device 70 for real-time analyses of the user’s appearance, body conditions, environmental conditions, etc., and to display actual measured data and analysis results to the mirror system of the present invention.
- the light 15 is mounted to the mirror surface 32 of the mirror 11 and electronically connected to the computing device 70.
- the light 15 may include, but is not limited to, a light-emitting diode (LED), an organic light-emitting diode (OLED) light, etc., which can be attached to any place on the mirror 11, including the edges.
- the lighting can be of different colors to improve the existing lighting.
- the lighting may set the mood by changing the level and color, and a spotlight can allow users to see themselves brightly reflected in the mirror 11.
- the speaker 16 is mounted to the mirror surface 32 of the mirror 11 and electronically connected to the computing device 70 as well.
- a plurality of handles 17 is mounted to the back face 33 of the mirror 11 to provide flexible handling of the smart mirror system in situations, including, but not limited to, folding, extending, rotating, etc.
- the plurality of wheels is mounted to the bottom of the mirror system 10 and offers easy moving, rotating, etc., of the smart mirror system.
- the smart mirror system of the present invention comprises the display 50.
- the display 50 may comprise a plurality of displays.
- the computing device 70 may be adapted to project a specific display of the plurality of displays to each of the plurality of mirrors 31 of the mirror 11.
- the display 50 comprises a display panel 51, which is attached to the back face 33 of the mirror 11 to display the user appearance information and data on the mirror surface 32.
- the display 50 comprises a transparent display 52, which is positioned to the mirror surface 32 of the mirror 11.
- the display 50 can show current time, date, temperature, humidity, etc.
- the user may choose to check the user’s appearance data via the display 50, as can be seen in FIG. 9 to FIG. 11.
- the smart mirror system can be configured to use machine leaming/deep learning algorithms to provide suggestions based on predictive analyses of the user’s future conditions and detected changes over time.
- the user can access the information/data on a personal computing (PC) device, including, but not limited to, a mobile phone, a smart phone, a tablet, a smart watch, etc., or a website.
- the user can use natural languages to operate different types of applications powered by the computing device 70.
- the computing device 70 of the smart mirror system comprises a central processing unit (CPU) 71, an AI module 73, a graphics processing unit (GPU) 72, a deep learning accelerator 74, and a vision accelerator 75, a memory unit 76.
- the AI module 73 is electronically connected to the CPU 71, which is adapted to process the plurality of appearance data through the AI module 73 before displaying the appearance information of the user on the display 50.
- the deep learning accelerator 74 is electronically connected to the AI module 73.
- the GPU 72 is electronically connected to the CPU 71, while the vision accelerator 75 is electronically connected to the GPU 72.
- the CPU 71 is adapted to acquire a plurality of 3D data from the plurality of cameras 12 of the mirror assembly 10 through the GPU 72 and the vision accelerator 75, wherein each of the plurality of cameras 12 is a 3D camera. With the 3D data of the user’s appearance, the CPU 71 is adapted to process the plurality of 3D data through the AI module 73 and the deep learning accelerator 74. Subsequently, the CPU 71 creates a 3D avatar of the user and displays the 3D avatar of the user to the display 50. As can be seen in FIG. 9, in an alternative embodiment, the smart mirror system may use 3D video data acquired by the CPU 71 of the computing device 70 through the plurality of cameras 12 of the mirror assembly 10.
- the resulting 3D avatar is a 3D avatar video presentation of the user.
- the 3D avatar and/or 3D avatar video of the user may be displayed in a 360-degree rotating view on the display 50.
- the CPU 71 may be adapted to provide a plurality of appearance analysis results using the plurality of 3D data and the plurality appearance data of the user through the AI module 73, wherein the plurality of appearance analysis results includes a change of body condition of the user. The CPU 71 then displays the plurality of appearance analysis results to the display 50. As can be seen in FIG.
- the plurality of appearance analysis results includes an outfit video of the user
- the plurality of appearance analysis results includes an outfit video of the 3D avatar with an outfit recommended by the AI module 73 of the computing device 70.
- the AI technologies including, but not limited to, deep learning, machine learning, facial recognition, etc., are implemented through the AI module 73 of the computing device 70 to conduct the learning and analysis of the appearance input data and to create the outfit video for the user. As can be seen in FIG.
- the AI module 73 may conduct learning and analysis of a body pose of the user and provides pose image/video of the user as the plurality of appearance analysis results, which is displayed on the display 50. Further, the CPU 71 may simultaneously display a body pose of any instructor to the user to guide the user in any form of task, including, but not limited to, exercises, fitness lessons, physical training sessions, yoga lessons, or any other suitable teaching and learning. In another embodiment of the present invention, the AI module 73 may conduct learning and analysis of exercise videos of the user, thus the CPU 71 can display the actual video of the user with a simultaneous showing of an instructor’s video to provide efficient and effective learning to the user. As can be seen in FIG.
- the computing device 70 of the smart mirror system comprises the memory unit 76, which is electronically connected to the CPU 71 of the computing device 70.
- the CPU 71 is adapted to securely save the plurality of appearance data of the user to the memory unit 76.
- the memory unit 76 may comprise a storage device, including, but not limited to hard disk drive (HDD), solid-state drive (SSD), or any other storage device.
- the storage device may include an extraneous storage device, which is electronically connected to the memory unit 76.
- the connection may be made through a wired connection, including, but not limited to, a universal serial bus (USB), or a protected/secure wireless connection, including, but not limited to, Bluetooth ® .
- the CPU 71 is adapted to securely save the plurality of appearance data of the user to the extraneous electronic storage device through the memory unit 76.
- the storage device is used not only to enhance the storage capacity of the memory unit, but also to provide data security, especially when collecting the user’s body information and data.
- the storage device and the memory unit 76 can safely and securely keep the personal image/video and other data within the smart mirror system and utilize the data for machine leaming/deep learning of the AI module 73.
- the smart mirror system of the present invention comprises a server 77, a database 78, an extraneous communication network 79, and a personal computing (PC) device 90.
- the database 78 is stored on the server 77.
- the server 77 may include, but is not limited to, a server computer, a laptop computer, a desktop computer, a cloud computing device, and any other suitable computing device.
- the server 77 is managed by an administrator, including, but not limited to, a system administrator, a computer professional, a software engineer, a software specialist, an information technology (IT) engineer, and any other suitable person.
- the PC 90 device is electronically connected to the server 77 through the extraneous communication network 79, and the server 77 is electronically connected to the computing device 70 through the extraneous communication network 79 as well.
- the extraneous communication network 79 may include a wireless network, including, but not limited to, Internet, the fifth generation (5G) network, etc.
- the server 77 may be adapted to communicate with the computing device 70 to securely transfer all information, data, and analysis results and make safe storage to an external medium per the user’s request and/or by the administrator.
- the database 78 is used to store all the data captured by the smart mirror system and the computing device 70 can be configured to access the database78 to transfer any and all necessary data through the extraneous communication network 79.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Mirrors, Picture Frames, Photograph Stands, And Related Fastening Devices (AREA)
Abstract
Un système de miroir intelligent collecte et visualise des données d'apparence d'un utilisateur avec des technologies d'intelligence artificielle (IA). Le système utilise divers capteurs et de multiples caméras pour acquérir des données d'apparence en temps réel pendant que l'utilisateur se regarde dans un miroir du système. Grâce à l'utilisation d'IA, le système analyse les données et crée un avatar tridimensionnel (3D) de l'utilisateur, puis projette l'avatar 3D conjointement avec des données de mesure sur une unité d'affichage sur la surface de miroir d'un miroir pliable. De plus, le système fournit des images/vidéos de poses d'exercice de l'utilisateur avec un affichage simultané de la pose d'un instructeur pour guider l'apprentissage de l'utilisateur. En outre, le système peut fournir des suggestions de tenue par analyse de la tenue actuelle de l'utilisateur et l'affichage d'avatars 3D simulés avec des tenues suggérés pour la comparaison. Toutes les données et tous les résultats sont sauvegardés de manière sécurisée dans le système. Ainsi, le système propose un miroir intelligent polyvalent et innovant.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201962939055P | 2019-11-22 | 2019-11-22 | |
US62/939,055 | 2019-11-22 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2021102417A1 true WO2021102417A1 (fr) | 2021-05-27 |
Family
ID=75980028
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/US2020/061822 WO2021102417A1 (fr) | 2019-11-22 | 2020-11-23 | Système de miroir intelligent pour la collecte et la visualisation d'informations d'apparence physique |
Country Status (1)
Country | Link |
---|---|
WO (1) | WO2021102417A1 (fr) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN117575636A (zh) * | 2023-12-19 | 2024-02-20 | 东莞莱姆森科技建材有限公司 | 一种基于视频处理的智能镜控制方法及系统 |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20040078301A1 (en) * | 2002-06-10 | 2004-04-22 | Martin Illsley | Interactive trying-on cubicle |
US20130229482A1 (en) * | 2005-03-01 | 2013-09-05 | Nissi Vilcovsky | Devices, systems and methods of capturing and displaying appearances |
US20170287060A1 (en) * | 2016-03-31 | 2017-10-05 | Samsung Electronics Co., Ltd. | Method for composing image and electronic device thereof |
US20180247356A1 (en) * | 2015-02-26 | 2018-08-30 | Finity Technology Limited | Computer implemented platform for the creation of a virtual product |
-
2020
- 2020-11-23 WO PCT/US2020/061822 patent/WO2021102417A1/fr active Application Filing
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20040078301A1 (en) * | 2002-06-10 | 2004-04-22 | Martin Illsley | Interactive trying-on cubicle |
US20130229482A1 (en) * | 2005-03-01 | 2013-09-05 | Nissi Vilcovsky | Devices, systems and methods of capturing and displaying appearances |
US20180247356A1 (en) * | 2015-02-26 | 2018-08-30 | Finity Technology Limited | Computer implemented platform for the creation of a virtual product |
US20170287060A1 (en) * | 2016-03-31 | 2017-10-05 | Samsung Electronics Co., Ltd. | Method for composing image and electronic device thereof |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN117575636A (zh) * | 2023-12-19 | 2024-02-20 | 东莞莱姆森科技建材有限公司 | 一种基于视频处理的智能镜控制方法及系统 |
CN117575636B (zh) * | 2023-12-19 | 2024-05-24 | 东莞莱姆森科技建材有限公司 | 一种基于视频处理的智能镜控制方法及系统 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11341711B2 (en) | System and method for rendering dynamic three-dimensional appearing imagery on a two-dimensional user interface | |
Cai et al. | RGB-D datasets using microsoft kinect or similar sensors: a survey | |
US10789699B2 (en) | Capturing color information from a physical environment | |
US11736756B2 (en) | Producing realistic body movement using body images | |
US9495008B2 (en) | Detecting a primary user of a device | |
WO2020010979A1 (fr) | Procédé et appareil d'apprentissage de modèle permettant la reconnaissance de points clés d'une main et procédé et appareil de reconnaissance de points clés d'une main | |
US9245180B1 (en) | Body modeling and garment fitting using an electronic device | |
CN110363867B (zh) | 虚拟装扮系统、方法、设备及介质 | |
JP2019510297A (ja) | ユーザの真実の人体モデルへの仮想的な試着 | |
US20210319585A1 (en) | Method and system for gaze estimation | |
EP3394709B1 (fr) | Miroir augmenté | |
EP3394712A1 (fr) | Miroir augmenté | |
CN110377149A (zh) | 场景分类 | |
US20190045270A1 (en) | Intelligent Chatting on Digital Communication Network | |
WO2017108703A1 (fr) | Miroir augmenté | |
US20230065288A1 (en) | Electronic Devices with Body Composition Analysis Circuitry | |
US20170148225A1 (en) | Virtual dressing system and virtual dressing method | |
WO2021102417A1 (fr) | Système de miroir intelligent pour la collecte et la visualisation d'informations d'apparence physique | |
US11599739B2 (en) | Image suggestion apparatus, image suggestion method, and image suggestion program | |
WO2017108702A1 (fr) | Miroir augmenté | |
JP7428874B2 (ja) | 体型を判別するための演算デバイス、プロセッサー実施方法、および体型判別システム | |
Bhowmik | Interactive and immersive devices with perceptual computing technologies | |
Huang et al. | Implementation Simple Fitting System Using Image Recognition for Portable Device | |
TWI541732B (zh) | 活動基礎成本制的支援方法與系統 | |
Barapatre et al. | Sixth Sense Technology using gesture control |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 20890368 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 20890368 Country of ref document: EP Kind code of ref document: A1 |