KR20220055924A - Smart mirror device using ultra short throw laser projector - Google Patents

Abstract

The present invention relates to a smart mirror device using an ultrashort focal length laser projector and, more specifically, to a smart mirror device using an ultrashort focal length laser projector, which uses an ultrashort focal length laser projector to implement a large smart mirror of 100 inches or more and implement 1:1 scale with a subject, thereby implementing an interactive training environment. In addition, guidelines are provided on a screen during exercise in a fitness center and the like to correct the user's incorrect posture and related information is provided as feedback on the basis of augmented reality to increase exercise efficiency and effectiveness. The present patent (invention) has been performed based on the research fund support of 2020 Post-R&D Global Commercialization Support Project of the Jeonnam Information & Culture Industry Promotion Agency. The smart mirror device comprises a main body, a camera module, a storage module, an analysis and processing module, an ultrashort focal length laser projector, a control module, and a communication module.

Description

Smart mirror device using ultra short throw laser projector

The present invention relates to a smart mirror device using an ultra-short throw laser projector, and more particularly, a large smart mirror of 100 inches or more can be implemented using an ultra-short throw laser projector, and interactive by implementing a 1:1 scale with the subject. It relates to a smart mirror device using an ultra-short throw laser projector that can implement a training environment.

A mirror is a very friendly object that mankind has used for a long time, and is mainly used as a means to check oneself. In other words, people look at themselves through the mirror several times a day, and this can only be checked in real time and cannot be saved.

Recently, due to the development of the camera function, it is possible to not only check the appearance of oneself in real time by taking a self-portrait, but also to record it and check it later.

However, it is cumbersome to take a self-portrait using the camera you carry every time with the will to record your appearance.

Accordingly, there is a method in which a camera is mounted on a mirror and a user takes a picture when the user reflects his or her appearance in the mirror, but a method using a simple photographed picture is still insufficient.

Recently, by grafting information and communication (IT) technology to these mirrors, smart mirrors that can be used as mirrors in everyday life and provide various information to users have been developed.

However, in the conventional smart mirror, it is very difficult to implement a large screen due to the limitation of the panel size, and it is very difficult to implement a 1:1 scale with a subject when implementing the screen.

Even if a large screen is implemented using a large panel, commercialization is difficult because the cost is very high, and there is a problem that the subject is distorted during shooting due to the camera located on the top.

Republic of Korea Patent Publication No. 10-2017-0034086 Republic of Korea Patent Publication No. 10-2004-0055762

The present invention is to solve the conventional problems as described above, and it is possible to implement a large smart mirror of 100 inches or more using an ultra-short throw laser projector, and to implement an interactive environment by realizing a 1:1 scale with the subject. An object of the present invention is to provide a smart mirror device using an ultra-short throw laser projector.

In addition, the present invention provides a guideline on the screen during exercise in a fitness center, etc. using an ultra-short-throw laser projector that can correct a user's incorrect posture and feed back related information based on augmented reality to increase exercise efficiency and effect. An object of the present invention is to provide a smart mirror device.

In addition, the present invention provides a personalized service by linking exercise information through a dedicated app, and provides a smart mirror device using an ultra-short-throw laser projector that enables sharing of exercise history and performance through SNS. .

In addition, the present invention provides an API for registering various exercise contents that can be provided through smart mirror devices, such as PT, yoga, and Pilates, to create a profit model such as an app store. Smart using an ultra-short-throw laser projector An object of the present invention is to provide a mirror device.

In addition, an object of the present invention is to provide a smart mirror device using an ultra-short throw laser projector that can provide a variety of services suitable for users by photographing an image of a user reflected in the mirror, databaseizing and analyzing the captured image. .

A smart mirror device using an ultra-short throw laser projector according to the present invention for achieving the above object includes a half-mirror, a screen disposed to be spaced apart from the rear of the half-mirror by a predetermined distance, and wraps around the half-mirror and the edges of the screen a body including a bezel formed so as to a camera module provided on the central side of the screen to photograph a user; a storage module for storing the image captured by the camera module; An analysis processing module that extracts the user's current body shape information using the image captured by the camera module, and analyzes the user's body shape change information by comparing the currently extracted body shape information with the user's previous body shape information stored in advance in the storage module class; an ultra-short throw laser projector provided between the half mirror and the screen and projecting an image toward the screen; The ultra-short-throw laser projector is controlled so that the current body shape line according to the user's current body shape information analyzed in the analysis processing module and the previous body shape line according to the previous body shape information are respectively displayed on the display, and the user according to the user's body shape change information a control module for extracting at least one of the training images stored in the storage module to induce a change in the body shape of the body and controlling it to be displayed on the screen through the ultra-short-throw laser projector; and a communication module that supports communication with the user's smart terminal according to the control of the control module.

The control module controls the ultra-short-throw laser projector so that a current body line according to the current body shape information of the user analyzed by the analysis processing module and a previous body line line according to the previous body shape information are displayed on the screen, respectively, It characterized in that the ultra-short throw laser projector is controlled to display on the screen a shooting guideline guiding that at least one or more specific parts are located in at least one or more preset specific areas on the screen.

The smart mirror device using the ultra-short throw laser projector according to the present invention can implement a large smart mirror of 100 inches or more using the ultra-short throw laser projector, and implement an interactive training environment by implementing a 1:1 scale with the subject. .

In addition, the present invention can correct a user's wrong posture by providing guidelines on the screen during exercise in a fitness center, etc., and feed back related information based on augmented reality to increase exercise efficiency and effectiveness.

In addition, the present invention can provide a personalized service by linking training information through a dedicated app, and share exercise history and performance through SNS.

In addition, the present invention provides an API for registering various exercise contents that can be provided through smart mirror devices, such as PT, yoga, and Pilates, to create a revenue model such as an app store.

In addition, the present invention captures the user's image reflected in the mirror, databaseizes and analyzes the captured image, compares the currently captured image with the previously captured image, and projects the comparison image to allow the user to change his or her body shape. It is easy to recognize and provides various services suitable for users by providing exercise information and images for inducing changes in body type.

1 is a view showing a smart mirror device using an ultra-short throw laser projector according to the present invention.
2 is a view showing information displayed on a user's smart terminal.
3 is a block diagram showing the configuration of a smart mirror device using an ultra-short throw laser projector according to the present invention.
Figure 4 is a front view showing the operation of the smart mirror device using the ultra-short throw laser projector according to the present invention.

Hereinafter, a smart mirror device using an ultra-short throw laser projector according to a preferred embodiment of the present invention will be described in detail with reference to the accompanying drawings.

1 to 4 show a smart mirror device using an ultra-short throw laser projector according to the present invention. 1 to 4, the smart mirror device 1 using the ultra-short throw laser projector according to the present invention includes a body 110, a camera module 120, a storage module 130, and an analysis processing module ( 140), an ultra-short throw laser projector 150, a control module 160, a communication module 170, and an operation module.

In the smart mirror device 1 according to the present invention, the ultra-short-throw laser projector 150 is disposed on the upper rear side of the half-mirror so that it usually looks like a mirror, but only when the image is output from the ultra-short-throw laser projector 150 to the screen side, the image is half What was implemented so that the user 5 could check the image through the mirror was applied.

The main body 110 has a large flat-type half mirror 111, a screen spaced apart from the rear of the half mirror 111 at a predetermined interval and onto which an image is projected, and maintaining the interval between the half mirror 111 and the screen. It is configured to include a mirror 111 and a bezel 112 for finishing and protecting the edge of the screen. Here, the ultra-short throw laser projector 150 to be described later is disposed between the upper half mirror 111 of the main body 110 and the screen and projects an image toward the screen.

The main body 110 can reflect and display the user 5 located in the front position, and at the same time, the image being projected onto the screen by the ultra-short throw laser projector 150 located inside is transmitted through the half mirror 111 . By being displayed, the user 5 can check his/her appearance and the image at the same time.

The camera module 120 is installed in the center of the screen to take an image toward the half-mirror, and the operation is controlled by the control module 160 to be described later.

In this embodiment, one camera module 120 is provided in the center of the screen, but unlike this, it is of course possible to shoot a 3D image by installing the camera module 120 at the upper and lower left corners and upper and lower right corners of the screen, respectively. In this case, the analysis processing module 140 to be described later processes and synthesizes the images captured by the plurality of camera modules 120 to generate the image of the user 5 in 3D form.

The storage module 130 stores the image captured by the camera module 120 under the control of the control module 160 . In addition, the storage module 130 includes an operation control for controlling the operation of the ultra-short throw laser projector 150 according to the control of the control module 160 to be described later among the operations of the user 5 photographed by the camera module 120 . Information and training images for guiding to manage the body type of the user 5 are stored in advance.

The analysis processing module 140 identifies the user 5 using the image captured by the camera module 120, extracts the current body shape information of the currently photographed user 5, and stores the currently extracted body shape information The body shape change information of the user 5 is analyzed by comparing the previous body shape information of the user 5 pre-stored in the module 130 .

In addition, the analysis processing module 140 continuously receives the image of the user 5 obtained through the camera module 120 before extracting the body type information, and a specific body part of the user 5 is subjected to the ultra-short focus laser. It is determined whether the projector 150 is located in a specific area.

As an example, the analysis processing module 140, as shown in Figure 3, the user 5's head (B1), both shoulders (B2, B3), both hands (B4, B5), both feet (B6, B7) The specific first area A1, second area A2, and third area A3, fourth area A4, fifth area A5, and sixth area on the ultra-short throw laser projector 150, respectively. A6) and the seventh area A7 are determined respectively.

The ultra-short throw laser projector 150 is provided between the half mirror and the screen, and projects the image toward the screen. It is preferable to apply the ultra-short throw laser projector 150 capable of realizing UHD, 4K, and 8K image quality.

If the result determined by the analysis processing module 140 is that any one part is not located in the corresponding area, the control module 160 may provide a guide message or signal can be sent. This is to prevent the captured image of the user 5 from being changed according to the posture of the user 5. When comparing the previous image and the current image of the analysis processing module 140 by shooting the user 5 in a certain posture This is to minimize errors.

In addition, the analysis processing module 140 includes an image processing unit for identification of the user 5 , a calculation unit, and an identification unit.

The image processing unit includes an image of the upper limb corresponding to the arm of the user 5 among the entire image of the user 5 photographed by the camera module 120 and the entire image of the user 5 photographed by the camera module 120, From among the entire images of the user 5 captured by the camera module 120 , images of the lower extremities corresponding to the legs of the user 5 are respectively extracted.

The image processing unit converts RGB type color information for each pixel of the image of the user 5 captured by the camera module 120, that is, each pixel of the original image, into YCbCr type color information, and a color difference from the converted YCbCr type color information The information is removed and converted into an image having only a simple luminance component, and an edge is detected from the converted image to generate an edge image.

The above-described RGB type represents color information for a pixel with a red component, a green component, and a blue component, and the YCbCr type represents color information for a pixel with a luminance component and a color difference component, specifically, the difference between the luminance component and the blue component, luminance It is expressed as the difference between the component and the red component. In the conversion process of the image processing unit, the original image may be converted into an image having binary color information expressed in black and white by removing not only the color difference information but also the luminance information of pixels having a luminance value less than or equal to a set luminance value.

And, the edge image generated by the image processing unit may be detected from the original image through an edge detection algorithm that detects a boundary between two regions having different contrasts. In this embodiment, the algorithm for edge detection of the image processing unit includes a step of smoothing a large input image using a Gaussian filter, and gradients using a Sobel mask. Canny Edge Detection algorithm is applied, which includes a step of detection (detecting a portion in which the contrast value of a pixel changes rapidly), a step of double thresholding, and a step of edge tracking. do. In the double thresholding stage, a weak edge and a strong edge are judged based on two thresholds, and the broken edge is connected in the edge tracking stage.

The image processing unit generates a first corrected image through a dilation operation that expands the bright part and weakens the dark part in the edge image, that is, expands the brightly displayed edge area to its periphery (dilation of canny image), , fills a locally located black region in the first corrected image with white to create a second corrected image (Filling). This is to clearly distinguish the dark area in the edge image.

Then, the image processing unit generates a third corrected image by convolutional integration of the second corrected image with respect to a region of interest (ROI MASK) for selectively filtering only a desired region, that is, only a region of interest among the second corrected images. and labeling each pixel corresponding to the outer edge of the third corrected image (labeling). This is to set the outline in the third corrected image.

In addition, the image processing unit generates a fourth corrected image by performing an erosion operation to strengthen dark areas while weakening bright areas in the third corrected image, and detects the boundary of the generated fourth corrected image to obtain a fifth corrected image. A corrected image is generated, and the generated fifth corrected image is eroded again to generate a sixth corrected image.

Then, the image processing unit generates a seventh corrected image by performing an ROI threshold operation with respect to the sixth corrected image. In addition, the image processing unit generates an upper or lower extremity mask (Cabbage Mask) using the sixth correction image, and fills, erosion, and labeling processes for the created upper extremity or lower extremity mask (Cabbage Mask) for the entire user (5). Create a correction image.

Meanwhile, the image processing unit may perform a process for extracting the upper extremity image or the lower extremity image after the extraction of the entire corrected image is completed or during the process of extracting the entire corrected image. This creates an eighth corrected image corresponding to the upper or lower extremities from the seventh corrected image generated in the entire corrected image extraction process as described above, and a region matching the eighth corrected image from the original image. By extraction, an image of an upper limb or an image of the lower limb is generated.

The calculation unit includes length ratio information of the entire image of the user 5 and the upper extremity image extracted from the image processing unit, length ratio information of the entire image and the lower extremity image, length ratio information of the upper extremity image and the lower extremity image, and the total image and area ratio information of the lower limb image or the upper limb image is calculated, respectively. Then, the calculated ratio information is transmitted to the identification unit. The ratio information calculated by the calculator is separately stored in the storage module 130 .

The identification unit analyzes the degree of similarity between the ratio information calculated by the calculator and the sample information for each user stored in the storage module 130 and identifies the user 5 according to the analysis result.

As described above, the identification unit includes sample information pre-stored in the storage unit, that is, the total image length for each user (5): height, arm length, leg length, height-to-arm length ratio, height-to-leg length ratio, arm Leg length to length ratio information, total image area, upper limb area, lower limb area, upper limb area ratio to total image area, lower limb area ratio to total image area, and lower limb area ratio information to upper limb area Loaded and analyzed length information, ratio information, area information, and area ratio information for each of the loaded sample information and the similarity, and when the sample information with the greatest similarity appears, the user (5) is assigned to the sample information with the greatest similarity. It is determined as the user 5 corresponding to . At this time, the control module 160 to be described later loads the previously captured image and body shape information of the determined user 5 and controls to be displayed on the ultra-short-throw laser projector 150 .

The ultra-short-throw laser projector 150 is provided in the main body 110 and outputs the current body shape information, previous body shape information, and body shape change information of the user 5 analyzed by the camera module 120 and the analysis processing module 140 . do.

The control module 160 controls the operation of the camera module 120 and the ultra-short throw laser projector 150 , respectively.

The control module 160 uses the ultra-short focus laser projector 150 to display the current body shape line according to the current body shape information of the user 5 analyzed by the analysis processing module 140 and the previous body shape line according to the previous body shape information on the screen. ) to control

In addition, the control module 160 extracts at least one of the training images stored in the storage module 130 to induce a change in the body shape of the user 5 according to the body shape change information of the user 5 and displays it on the screen. Controls the ultra-short throw laser projector 150 .

In addition, the control module 160 displays on the screen a shooting guideline for guiding that at least one or more specific parts of the body of the user 5 are located in at least one or more preset specific areas on the ultra-short-throw laser projector 150 . Controls the ultra-short throw laser projector 150 to be so. In addition, when the control module 160 is analyzed that a specific part of the user 5 is located in a specific area on the screen through the analysis processing module 140 , the user 5 is photographed and photographed through the camera module 120 . The image is controlled to be stored in the storage module 130 and the user 5 is identified through the control and analysis processing module 140 .

In addition, when a motion of the user 5 corresponding to the motion control information is captured by the camera module 120 , the control module 160 controls the ultra-short throw laser projector 150 according to the motion control information.

For example, when the user 5 moves his hand from right to left, the control module 160 controls an image prior to the currently played image to be played back, and when the user 5 moves his hand from left to right , control so that the image following the currently played image is played, when the user 5 moves his hand from top to bottom, the currently playing image is controlled to stop, and the user 5 moves his hand from bottom to top When you do this, you can control the currently stopped image to be played again.

As described above, the control module 160 recognizes and detects the user's 5 motion to perform control related to reproduction and stop of the image, but differently projected and reproduced on the screen through the ultra-short throw laser projector 150 . Of course, you can perform various controls depending on the content being used.

In addition, the control module 160 is a training image suitable for changing the body shape of the user 5 according to the body shape change information of the user 5 analyzed by the analysis processing module 140 among the training images stored in the storage module 130 . By extracting and displaying on the screen, the user 5 can follow the training image. In this case, the analysis processing module 140 captures the training operation of the user 5 photographed by the camera module 120 and then determines whether the operation of the user 5 is the same as the operation displayed in the training image, and controls If the determination result is the same, the module 160 skips the training image displayed on the screen to the next image, and if not the same, controls the image to continuously appear.

The communication module 170 supports communication with the smart terminal 200 of the user 5 according to the control of the control module 160 . The communication module 170 analyzes the image captured by the camera module 120 through Bluetooth or Wi-Fi according to the control of the control module 160 and the image content stored in the storage module 130 and analyzed by the analysis processing module 140 . The information is wirelessly transmitted to the smart terminal 200 .

In the smart terminal 200 of the user 5 connected to the smart mirror device 1 through the communication module 170, an application for monitoring and managing body shape related information of the user 5 is installed, and through this application, the user (5) can check information related to one's body type anytime, anywhere.

The smart terminal 200 processes the user 5 body shape management service in conjunction with the smart mirror device 1 . That is, in conjunction with the smart mirror device 1, the user 5 body shape management and related service information are shared, and the branch or product information that manages the body shape is displayed on the ultra-short throw laser projector 150 or the smart mirror device Process as indicated in (1).

The smart terminal 200 includes a communication unit communicating with the smart mirror device 1 , a display unit displaying information related to the user 5 provided from the smart mirror device 1 , and a user 5 provided from the smart mirror device 1 . A control unit for processing a related information service is included, and the control unit works with the smart mirror device 1 to share service information.

As described above, the smart mirror device 1 according to the present invention identifies the user 5 by analyzing the current image captured by the camera module 120, and stores the previous image according to the identified user 5 information. Extracted from module 130 .

Then, the image is analyzed to extract a feature point according to a specific part of the user 5 , and the user 5 is identified through a function of identifying the user 5 using the feature point. In other words, by analyzing the real-time captured image, the user 5 is identified, and as described above to correspond to the user 5, previous images that are provided in itself or are databased in a storage medium provided in an external device are used. extract

Then, a screen according to the current image captured by the camera module 120 and the previous image of the user 5 extracted from the storage module 130 is generated and displayed on the screen.

The smart mirror device using the ultra-short throw laser projector according to the present invention can derive and display the comparison result by comparing the current image with the previous image, for example, it can display the comparison result so as to overlap the current image.

In addition, the smart mirror device 1 using the ultra-short throw laser projector according to the present invention communicates and interworks with the smart scale 300 to receive the user's weight and body fat information measured by the smart scale 300 and display the information. It can also be displayed on the screen.

The smart mirror device using the ultra-short-throw laser projector according to the present invention as described above can implement a large smart mirror of 100 inches or more using the ultra-short-throw laser projector, and an interactive training environment by implementing a 1:1 scale with the subject can be implemented.

In addition, the present invention can correct a user's wrong posture by providing guidelines on the screen during exercise in a fitness center, etc., and feed back related information based on augmented reality to increase exercise efficiency and effectiveness.

In addition, the present invention can provide a personalized service by linking training information through a dedicated app, and share exercise history and performance through SNS.

In addition, the present invention provides an API for registering various exercise contents that can be provided through smart mirror devices, such as PT, yoga, and Pilates, to create a revenue model such as an app store.

The smart mirror device using the ultra-short throw laser projector according to the present invention described above has been described with reference to the accompanying drawings, but this is only an example, and those of ordinary skill in the art can perform various modifications and equivalents therefrom. It will be appreciated that other embodiments are possible.

Accordingly, the scope of true technical protection of the present invention should be determined only by the technical spirit of the appended claims.

1: Smart Mirror Device
110: body
120: camera module
130: storage module
140: analysis processing module
150: ultra-short throw laser projector
160: control module
170: communication module
200: smart terminal

Claims (2)

a main body including a half mirror, a screen disposed to be spaced apart from the rear of the half mirror by a predetermined distance, and a bezel formed to surround the half mirror and edges of the screen;
a camera module provided on the central side of the screen to photograph a user;
a storage module for storing the image captured by the camera module;
An analysis processing module that extracts the user's current body shape information using the image captured by the camera module, and analyzes the user's body shape change information by comparing the currently extracted body shape information with the user's previous body shape information stored in advance in the storage module class;
an ultra-short throw laser projector provided between the half mirror and the screen and projecting an image toward the screen;
The ultra-short-throw laser projector is controlled so that the current body shape line according to the user's current body shape information analyzed in the analysis processing module and the previous body shape line according to the previous body shape information are respectively displayed on the display, and the user according to the user's body shape change information a control module for extracting at least one of the training images stored in the storage module to induce a change in the body shape of the body and controlling it to be displayed on the screen through the ultra-short-throw laser projector;
A smart mirror device using an ultra-short throw laser projector comprising a; a communication module that supports communication with a user's smart terminal according to the control of the control module.
According to claim 1,
The control module controls the ultra-short-throw laser projector to display a current body line according to the current body type information of the user analyzed by the analysis processing module and a previous body line line according to the previous body type information on the screen, respectively, Smart using an ultra-short-throw laser projector, characterized in that it controls the ultra-short-throw laser projector to display on the screen a shooting guideline guiding that at least one or more specific areas are located in at least one or more preset specific areas on the screen mirror device.

Images