KR102077515B1 - Smart mirror and customized personal training apparatus - Google Patents

Abstract

The smart mirror is a mirror, a display panel disposed at the rear of the mirror to provide a screen to the front of the mirror, an IoT management unit receiving sensing data from at least one sensor, and operated in an always active mode and detecting a first user voice. A user voice command processor activating the display panel based on whether the first user voice corresponds to a predefined initial operation command, and receiving and providing a second user voice to a voice service server when the display panel is activated. And a controller configured to receive a processing result of the second user voice from the voice service server and to overlay the sensing data on a screen visualizing the processing result and to generate the screen of the display panel.

Description

SMART MIRROR AND CUSTOMIZED PERSONAL TRAINING APPARATUS}

The present invention relates to a smart mirror technology, and more particularly, can be controlled through a user voice and connected to a temperature sensor, a humidity sensor, a fine dust sensor, and an exercise device to the IoT to provide a variety of services including personalized personal training. The present invention relates to a smart mirror and a personal training device using the same.

A mirror is an object that reflects the shape of an object by using light reflection, and may be used to see a user's appearance. Since the mirror is used to trim or make up the appearance, it can be used every day in everyday life, and in particular, can be used very frequently by the user.

Mirrors are usually made by applying amalgams to the back of the glass, which are used almost daily, but are not used to illuminate the user.

The following prior arts suggest directions for improving such mirrors.

Korean Patent Laid-Open No. 10-2015-0087793 is a smart mirror device, which is configured to include a camera unit for outputting a customer picture of a customer standing at a predetermined position and a camera unit, and receives a customer picture from the camera unit and displays the display. Or side-by-side on one side of the smart mirror and the smart mirror is formed so as to see the customer in a predetermined position to receive the customer information about the customer from the customer to check the authentication, and the customer picture from the camera unit It includes a touch authentication screen unit that is provided to display or provide a photo selected by the customer of the plurality of customer photos to the smart mirror.

Korean Patent No. 10-1664940 relates to a hair smart mirror system using virtual reality, comprising a mirror display installed on the wall of the beauty salon by configuring the camera and the angle adjusting means; A chair rotatably installed in front of the mirror display; Smart devices that are mirrored with the mirror display; to enable the mirror display through the operation of the smart device by the user seated on the chair, the smart device is suitable for each gender, age group using the app or server When the hairstyle is provided and displayed on the mirror display by mirroring, or the celebrity's hairstyle is provided through the search and displayed on the mirror display by mirroring, and when one of the provided hairstyles is selected using the mirror display or a smart device, The present invention discloses a technology in which a corresponding hairstyle is applied to an image of a user displayed on a mirror display through a camera, and thus is displayed in three dimensions according to the user's movement.

Korean Patent Laid-Open No. 10-2017-0064092 relates to a method of displaying content in a navigation map through mirroring of a smart device. Method of presenting contents in the navigation map through mirroring of smart devices so that the user can easily and accurately manipulate navigation map information and smart device-specific information by simultaneously distributing individual information of devices on the navigation map without damaging the information. It is about.

Korean Patent Publication No. 10-2015-0087793 (2015.07.30) Korea Patent Registration No. 10-1664940 (2016.10.05) Korean Patent Publication No. 10-2017-0064092 (2017.06.09)

An embodiment of the present invention is to provide a smart mirror and a personalized personal training device using the same, which can provide a variety of services, including personalized personal training connected to the temperature sensor, humidity sensor, fine dust sensor and exercise equipment and the Internet of Things do.

According to an embodiment of the present invention, when the display panel is deactivated, a smart mirror capable of maximizing the utilization of the smart mirror by activating the display panel first by detecting the presence of a person within a specific recognition range through a surrounding cognitive sensor To provide a customized personal training device used.

An embodiment of the present invention is to provide a smart mirror and a personal training device using the same that can register the user voice code in the user voice database in advance in the voice service server before the activation of the display panel through the first user voice.

An embodiment of the present invention is to provide a smart mirror and a personal training device using the same to display the exercise data receiving indicator on the activated display panel when the exercise data received from the user exercise equipment through the voice service server.

Among the embodiments, the smart mirror is a mirror, a display panel disposed at the rear of the mirror to provide a screen in front of the mirror, a specific recognition range when receiving sensing data from at least one sensor and the display panel is deactivated Internet of Things management unit for detecting the presence of a person within the active mode to activate the display panel in the operation mode, if the detected first user voice and the detected first user voice corresponds to a predefined initial operation command to the operation mode And a user voice command processor for activating the display panel in a standby mode when the detected first user voice corresponds to a predefined end operation command, and receiving a second user voice when the display panel is activated. To server Receives the processing result of the second user voice and the information on the mood of the user detected through the second user voice from the voice service server to visualize by reflecting the user's mood in color to the processing result And a controller configured to overlay the sensing data on one screen to generate the screen of the display panel.

In an embodiment, the IoT manager may display whether the received sensing data is out of a threshold range and display a threshold range deviation indicator on the activated display.

delete

In an embodiment, the controller may register a user voice code in a user voice database in advance in the voice service server before activation of the display panel through the first user voice.

In an embodiment, the controller may generate the user voice code by comparing the user test voice by the corresponding user with each of a plurality of reference voice characteristic codes in the registration process of the user voice.

In an embodiment, the controller may display an exercise data receiving indicator on the activated display panel when exercise data received from a user exercise device is provided through the voice service server.

In an embodiment, when the makeup guide voice is received, temperature sensing data and humidity sensing data are provided to the voice service server to receive the user's type and makeup order as a result of processing according to the makeup guide voice. can do.

delete

In one or more embodiments, a personalized training device may receive sensing data from a mirror, a display panel disposed at a rear of the mirror to provide a screen in front of the mirror, an exercise device, and at least one sensor, and the display panel may be deactivated. In the case of the IoT management unit for detecting the presence of a person within a certain cognitive range to activate the display panel in the operation mode, if the detected first user voice corresponds to a predefined initial operation command, the display A user voice command processor for activating the panel in the operation mode and deactivating the display panel in the standby mode when the detected first user voice corresponds to a predefined end operation command; and a second user voice if the display panel is activated. By listening It is provided to the service server and receives the processing result of the second user voice and the information on the mood of the user detected through the second user voice from the voice service server to reflect the user's mood in the processing result in color And a control unit for visualizing and overlaying the sensing data on the visualized screen to generate a screen of the display panel or overlaying exercise data received from the exercise device as a screen of the display panel.

The disclosed technique can have the following effects. However, since a specific embodiment does not mean to include all of the following effects or only the following effects, it should not be understood that the scope of the disclosed technology is limited by this.

According to an embodiment of the present invention, when the display panel is deactivated, the smart mirror and the personalized personal training device using the same include a smart mirror that detects the presence of a person within a specific recognition range through a surrounding cognitive sensor and activates the display panel first. Can provide.

A smart mirror and a personal training device using the same according to an embodiment of the present invention may provide a smart mirror that registers a user voice code in a user voice database in advance to a voice service server before activation of a display panel through a first user voice. have.

The smart mirror and the personalized personal training device using the same according to an embodiment of the present invention may display an exercise data reception indicator on the activated display panel when exercise data received from a user exercise device is provided through a voice service server.

1 is a view showing a smart mirror system of the present invention.
2 is a view for explaining the configuration of the smart mirror in FIG.
3 is a view for explaining the configuration of the voice service server shown in FIG.
4 is a flowchart illustrating the use of a smart mirror according to an embodiment of the present invention.
5 is a diagram illustrating control of a smart mirror through an application to a user terminal according to an embodiment of the present invention.

Description of the present invention is only an embodiment for structural or functional description, the scope of the present invention should not be construed as limited by the embodiments described in the text. That is, since the embodiments may be variously modified and may have various forms, the scope of the present invention should be understood to include equivalents for realizing the technical idea. In addition, the objects or effects presented in the present invention does not mean that a specific embodiment should include all or only such effects, the scope of the present invention should not be understood as being limited thereby.

On the other hand, the meaning of the terms described in the present application should be understood as follows.

Terms such as "first" and "second" are intended to distinguish one component from another component, and the scope of rights should not be limited by these terms. For example, the first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component.

When a component is referred to as being "connected" to another component, it should be understood that there may be other components in between, although it may be directly connected to the other component. On the other hand, when a component is referred to as being "directly connected" to another component, it should be understood that there is no other component in between. On the other hand, other expressions describing the relationship between the components, such as "between" and "immediately between" or "neighboring to" and "directly neighboring", should be interpreted as well.

Singular expressions should be understood to include plural expressions unless the context clearly indicates otherwise, and terms such as "comprise" or "have" refer to a feature, number, step, operation, component, part, or part thereof that is implemented. It is to be understood that the combination is intended to be present and does not exclude in advance the possibility of the presence or addition of one or more other features or numbers, steps, actions, components, parts or combinations thereof.

In each step, an identification code (e.g., a, b, c, etc.) is used for convenience of description, and the identification code does not describe the order of the steps, and each step is clearly contextual. Unless stated otherwise, they may occur out of the order noted. That is, each step may occur in the same order as specified, may be performed substantially simultaneously, or may be performed in the reverse order.

The present invention can be embodied as computer readable code on a computer readable recording medium, and the computer readable recording medium includes all kinds of recording devices in which data can be read by a computer system. . Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, and the like.

All terms used herein have the same meaning as commonly understood by one of ordinary skill in the art unless otherwise defined. The terms defined in the commonly used dictionary should be interpreted to coincide with the meanings in the context of the related art, and should not be interpreted as having ideal or excessively formal meanings unless clearly defined in the present application.

1 is a view for explaining a voice recognition smart mirror system according to an embodiment of the present invention.

Referring to FIG. 1, the voice recognition smart mirror system 100 includes at least one sensor 112, a smart mirror 110, a voice service server 120, a user voice database 130, a user terminal 140, and an exercise. The instrument 150, which can be connected via a network.

The voice recognition smart mirror system 100 may correspond to a personal training device customized through the smart mirror 110.

The at least one sensor 112 is a device that detects, distinguishes, and measures a physical quantity of heat, light, temperature, pressure, sound, or the like, and informs a predetermined signal, and provides sensing data. For example, the temperature sensor 112a may measure temperature to provide temperature sensing data. For another example, the surrounding cognition sensor 112b may detect the presence of a person and provide the surrounding cognition sensing data. For another example, the fine dust sensor 113c may measure fine dust concentration and provide fine dust sensing data.

The smart mirror 110 is a computing device combined with the mirror, and may perform a process related to a user voice and a connection with a smartphone. For example, the smart mirror 110 may perform the function of a mirror, and may be used when reflecting the shape of an object using light reflection. For another example, the smart mirror 110 may work with the voice service server 120 to process user voice. As another example, the smart mirror 110 may be connected to the user terminal 140 to receive a user input or control.

The smart mirror 110 may install an application. For example, the smart mirror 110 may receive a corresponding program from the voice service server 120 or may receive the corresponding program from the user terminal 140.

The voice service server 120 may provide a processing result to the smart mirror 110 in cooperation with the smart mirror 110, the user voice database 130, the user terminal 140, and the exercise device 150. In one embodiment, the voice service server 120 may obtain information according to a user request from an external server (not shown) through the Internet. For example, the voice service server 120 may receive a request for the current stock status through the user's voice, and interoperate with an external stock status providing server (not shown) to update the current stock status with the smart mirror 110. ) Can be provided.

The user voice database 130 may store processing results of the voice service server 120 or transmit stored data. More specifically, the user voice database 130 may store the processing result for each voice characteristic code registered by the voice service server 120. For example, the user voice database may receive a request from the voice service server 120 and transmit a voice characteristic code record.

The user terminal 140 is a mobile computing device such as a smartphone, and can install an application. For example, the program may be received from the voice service server 120 or the program may be received from the smart mirror 110. Here, the application may provide an input interface for controlling the smart mirror 110. For example, the input interface can be designed for the reception of user voices.

In one embodiment, in FIG. 5, when the user connects the smart mirror 110 and the user terminal 140 to the same network, the user may access the smart mirror 110 connected to the same network through an application installed in the user terminal 140. ) Can be controlled. For example, the user may be provided with a service capable of controlling the smart mirror 110 by using an application installed in the user terminal 140 even when the user is separated from the smart mirror 110 by a predetermined distance or more.

The exercise device 150 is a device for exercise of a user, such as a treadmill, and stores exercise data of the user and transmits exercise data to the voice service server 120 when a specific period or a specific event occurs. For example, the specific period may be determined by the user's setting, and the specific event may correspond to the time point when the exercise is completed.

FIG. 2 is a block diagram illustrating the smart mirror of FIG. 1.

Referring to FIG. 2, the smart mirror 110 includes a mirror 210, a display panel 220, an internet of things manager 230, a user voice command processor 240, and a controller 250.

The mirror 210 is a device that reflects the shape of an object by using light reflection, and may be used to see the user's appearance.

The display panel 220 is a screen determined according to a display format, a display content, and an input condition. The display panel 220 may be disposed at the rear of the mirror to provide a screen to the front of the mirror 210. The display panel 220 may visualize the processing result of the user voice and the sensing data. For example, the display panel 220 may visualize the processing result of the sensing data in real time, and may overlay the sensing data on a screen visualizing the processing result screen of the user's voice.

The IoT management unit 230 may receive sensing data from at least one sensor 112. For example, the IoT management unit 230 may receive the infrared sensing data and the temperature sensing data to visualize the temperature management recommendation screen on the display panel 220.

In one embodiment, the IoT management unit 230 may display whether the threshold range deviation indicator is displayed on the activated display panel 220 by examining whether the sensing data is out of the threshold range from the at least one sensor 112. Here, the threshold range may correspond to a section in which the upper and lower criteria of the normal category are set. For example, when the normal temperature range corresponds to 15 degrees to 28 degrees, the threshold range may correspond to 15 degrees as a lower limit and 30 degrees as an upper limit. The IoT manager 230 displays the deviation of the threshold range indicator. The display panel may display at a specific position of the panel 220 and change a specific position according to a user's setting.

When the display panel 220 is deactivated, the IoT management unit 230 may first activate the display panel 220 by detecting the presence of a person within a specific recognition range through the surrounding recognition sensor 112b. That is, the IoT management unit 230 may not display the threshold range deviation indicator when the display panel 220 operates in the standby mode, but activates the display panel 220 when the presence of a person is detected. The next threshold out of range indicator can be displayed.

The user voice command processor 240 determines whether the user voice corresponds to a predefined initial operation command when the user voice is detected in the standby mode (that is, in an inactive state) of the display panel 220, and if so, the display panel 220. ) Can be activated. In contrast, the user voice command processor 240 determines whether the user voice corresponds to a predefined end operation command when the user voice is detected in the operation mode (that is, the activated state) of the display panel 220. 220 may be deactivated.

When the display panel 220 is activated, the controller 250 receives a user voice and provides it to the voice service server 110. For example, the controller 250 may wait for the user voice in the operation mode of the display panel 220, and may provide the user voice to the voice service server 120 when the user voice is received. In an embodiment, the controller 250 may transmit sensing data together when the user voice is provided to the voice service server 120. This is because the voice service server 120 may utilize the sensing data in the process of processing the user voice.

In one embodiment, the controller 250 may additionally receive information about the mood of the user from the voice service server 120, and may be reflected in the visualization of the processing result of the user voice. For example, if the information on the mood of the user corresponds to depression, the controller 250 may visualize the processing result of the user's voice in bright colors.

3 is a block diagram illustrating a voice service server of FIG. 1.

Referring to FIG. 3, the voice service server 120 includes a data transceiver 310, a user manager 320, a voice processor 330, and a controller 340.

The data transceiver 310 may receive user voice data regarding the user voice received from the smart mirror 110, exercise data received from the exercise device 150, and sensing data received from the at least one sensor 112. In addition, the processing result of the integrated processing of the data may be provided to the smart mirror 110. In one embodiment, the data transmission and reception unit 310 may provide external data in the server in cooperation with an external server (for example, a content server).

The user manager 320 may register a user voice for accessing the service by the smart mirror 110. More specifically, the user manager 320 may store a voice characteristic code corresponding to the user voice in the user voice database 130 during the registration of the user voice. Here, the voice characteristic code may calculate a frequency difference by comparing the user test voice by the user with each of the plurality of reference voice characteristic codes, and generate a user voice code based on the frequency difference. For example, the user manager 320 may determine a male reference frequency, an intermediate reference frequency, and a female reference frequency, and determine a user voice code by comparing frequencies with respect to the user test voice.

The voice processor 330 analyzes user voice data regarding the user voice received by the data transceiver 310 and transmits the processing result regarding the user voice to the smart mirror 110 through the data transceiver 310. The analysis on the user voice data estimates the intention of the user and may be determined through learning through neural networks. The voice processor 330 may analyze the exercise data received from the data transceiver 310 and provide it to the smart mirror 110 when a specific period or a specific event occurs. The analysis of the exercise data may include biometric information (eg, calorie consumption) of the user that changes due to the user's exercise. The specific period may be determined according to a user's setting, and the specific event may include generation of exercise data. The voice processor 330 may analyze the sensing data received from the data transceiver 310 and provide it to the smart mirror 110. The analysis of the sensing data may include discrimination from the accumulated sensing data or discovery of a specific pattern.

In one embodiment, if the user voice data corresponds to the exercise data request, the voice processor 330 may process the exercise data in the user voice database unit 130 and provide it to the smart mirror 110, and the smart mirror 110. ) May receive the processed workout data and display the workout data by date to implement a personalized personal training device.

In another embodiment, if the user voice data corresponds to the makeup guide request, the voice processing unit 330 may reflect the temperature sensing data and the humidity sensing data received from the data transmitting and receiving unit 310 to determine the type of cosmetic according to temperature and humidity, and The makeup sequence may be transmitted as a result of user voice analysis. This reflects the case where the mirror 210 is used in the makeup of the user.

In another embodiment, if the user voice data corresponds to a specific serial content request, the voice processing unit 330 connects to an external content server (not shown) and provides the smart mirror 110 with the unread serial contents. can do. On the other hand, such unviewed sequence content may be recommended through CBF (Content based filtering).

For another example, when the user voice data corresponds to a specific data request and the plurality of specific data are plural, the voice processor 330 may determine the priority based on at least one of the number of views, the number of recommendations, and the search accuracy of the specific data. It may be provided to the mirror 110.

The controller 340 may control the overall operation of the voice service server 120 and may control data flow between the day transceiver 310, the user manager 320, and the voice processor 330.

FIG. 4 is a flowchart illustrating an operation of the smart mirror system of FIG. 1.

In FIG. 4, the smart mirror 110 may receive a user voice and transmit it to the voice service server 120 (step S410).

The voice service server 120 generates a voice characteristic code based on user voice data related to the user voice (step S420). In one embodiment, the voice service server 120 may generate a voice characteristic code through frequency analysis on the user voice data. The voice service server 120 compares the generated voice feature code with the voice feature code stored in the user voice database unit 130 to recognize the user (step S430).

When the user recognition is successfully performed, the voice service server 120 provides the smart mirror 110 with a customized GUI (Graphical User Interface) set by the corresponding user (step S440). A custom graphical user interface (GUI) may include the layout, colors, and fonts of display objects, such as temperature.

If the user recognition is not successfully performed, the voice service server 120 performs a determination regarding access approval (step S450). The voice service server 120 disallows user access to the smart mirror 110 if the access is not granted. The voice service server 120 provides a general GUI to the smart mirror 110 when the access is granted (step S460). Here, the generic GUI may correspond to the default GUI. In one embodiment, the access authorization may be set by the user of the smart mirror 110, and may be determined by a voice password input or an anonymous access permission scheme preset by the user.

The smart mirror 110 may receive the processing result corresponding to the user voice in cooperation with the voice service server 120 (step S470).

Although described above with reference to a preferred embodiment of the present invention, those skilled in the art various modifications and variations of the present invention without departing from the spirit and scope of the invention described in the claims below I can understand that you can.

100: smart mirror system
110: smart mirror 112: at least one sensor
112a: temperature sensor 112b: ambient perception sensor
112c: fine dust sensor 120: voice service server
130: user voice database 140: user terminal
150: fitness equipment 210: mirror
220: display panel 230: IoT management unit
240: user voice command processing unit 250: control unit
310: data transmission and reception unit 320: user management unit
330: voice processing unit 340: control unit

Claims (10)

mirror;
A display panel disposed at a rear side of the mirror to provide a screen to the front of the mirror;
An IoT management unit which receives sensing data from at least one sensor and detects the presence of a person within a specific recognition range when the display panel is deactivated to activate the display panel in an operation mode;
Detects a user voice and activates the display panel in an operation mode when the detected first user voice corresponds to a predefined initial operation command; and when the detected first user voice corresponds to a predefined end operation command. A user voice command processor configured to deactivate the display panel in a standby mode; And
When the display panel is activated, a second user's voice is received and provided to a voice service server, and the processing result of the second user's voice from the voice service server and information about the mood of the user detected through the second user's voice. And a control unit configured to receive and visualize by reflecting the user's mood as a color on the processing result and overlaying the sensing data on the visualized screen to generate a screen of the display panel.
delete delete The method of claim 1, wherein the control unit
And displaying an exercise data receiving indicator on the activated display panel when the exercise data received from the user exercise equipment is provided through the voice service server.
The method of claim 1, wherein the control unit
When the makeup guidance voice is received, the smart mirror, characterized in that to provide the temperature sensing data and humidity sensing data to the voice service server to receive the type and makeup order of the cosmetics to the corresponding user as a processing result according to the makeup guidance voice.
According to claim 1, wherein the IoT management unit
And checking whether the received sensing data is out of a threshold range and displaying a threshold out of range indicator on the activated display.
delete The method of claim 1, wherein the control unit
And a user voice code is registered in the user voice database in advance in the voice service server before activation of the display panel through the first user voice.
The method of claim 8, wherein the control unit
And generating a user voice code by comparing the user test voice by the corresponding user with each of a plurality of reference voice feature codes in the registration process of the user voice.
mirror;
A display panel disposed at a rear side of the mirror to provide a screen to the front of the mirror;
An IoT management unit configured to receive sensing data from an exercise device and at least one sensor and to detect the presence of a person within a specific recognition range when the display panel is deactivated to activate the display panel in an operation mode;
Detects a user voice and activates the display panel in an operation mode when the detected first user voice corresponds to a predefined initial operation command; and when the detected first user voice corresponds to a predefined end operation command. A user voice command processor configured to deactivate the display panel in a standby mode; And
When the display panel is activated, a second user's voice is received and provided to a voice service server, and the processing result of the second user's voice from the voice service server and information about the mood of the user detected through the second user's voice. Receives and visualizes by reflecting the user's mood to the processing result in color and overlaying the sensing data on the visualized screen to generate a screen of the display panel or overlaying the workout data received from the exercise equipment to display the display. Personalized training device comprising a control unit for generating a screen of the panel.

Images