KR20230139921A - Multi-joint robot apparatus for facial cleansing and massage, and method for controlling therefor - Google Patents

Abstract

A functional multi-joint robot device for face washing and massage and a control method thereof are disclosed. The articulated robot device includes an articulated robot arm provided with an end effector for washing or massaging the user's face; an image capture unit that captures the user's face; And detecting a face washing or massage area on the user's face using the user's face image and a pre-learned artificial intelligence model, establishing a movement path plan in the detected face washing or massage area, and establishing the established movement route. and a control unit that controls the articulated robot arm so that the articulated robot arm moves at a preset constant pressure on the user's face according to a plan.

Description

Face washing and massage functional multi-joint robot device and control method {MULTI-JOINT ROBOT APPARATUS FOR FACIAL CLEANSING AND MASSAGE, AND METHOD FOR CONTROLLING THEREFOR}

The present disclosure relates to an articulated robot device, and more specifically, to an articulated robot device with facial washing and massage functions and a control method thereof.

In general, cosmetics can be broadly divided into basic cosmetics that clean the skin, help supply and maintain moisture, and supply the oil and nutrients necessary for the skin, and color cosmetics that express the face, eyes, eyebrows, and lips in various colors. .

Color cosmetics have the advantage of beautifying the appearance of the skin by beautifully changing the appearance, but if the makeup is not removed properly, the pigment may settle, causing freckles, or toxins may penetrate the skin, causing skin problems.

Since color cosmetics contain a large amount of ingredients that cannot be completely removed by simply washing your face with water, various types of facial cleansing cosmetics such as cleansing tissues, cleansing foam, and cleansing cream are used to remove cosmetics applied to the skin and impurities on the skin. Remove.

The technical problem of the present disclosure is to provide a functional multi-joint robot device for face washing and massage and a control method thereof.

The technical problems to be achieved by this disclosure are not limited to the technical problems mentioned above, and other technical problems not mentioned can be clearly understood by those skilled in the art from the description below. You will be able to.

According to an embodiment of the present disclosure, a functional multi-joint robot device for face washing and massage and a face washing and massage method using the same are disclosed. The articulated robot device includes an articulated robot arm provided with an end effector for washing or massaging the user's face; an image capture unit that captures the user's face; And detecting a face washing or massage area on the user's face using the user's face image and a pre-learned artificial intelligence model, establishing a movement path plan in the detected face washing or massage area, and establishing the established movement route. and a control unit that controls the articulated robot arm so that the articulated robot arm moves at a preset constant pressure on the user's face according to a plan.

At this time, the image capture unit includes an RGB camera and a depth camera, and can capture a color image and a depth image of the user's face using the RGB camera and the depth camera.

At this time, the control unit may control the articulated robot arm according to the face washing mode and massage mode selected by the user and replace the end effector with an end effector corresponding to the selected mode.

At this time, the control unit may analyze the user's face image in real time, calculate the user's face movement distance in real time when the user's face moves, and reflect the face movement distance in the movement path plan.

At this time, the control unit may exclude the eye area, hairline area, and lip area from the detected face washing or massage area and establish the movement path plan in the remaining areas.

The method for controlling an articulated robot device includes: capturing an image of a user's face; Detecting a face washing or massage area on the user's face using the user's face image and a pre-trained artificial intelligence model; Establishing a movement route plan in the detected face washing or massage area; And a step of moving an articulated robot arm with an end effector to the user's face at a preset constant pressure according to the established movement path plan to perform facial cleansing or massage on the user's face.

The features briefly summarized above with respect to the present disclosure are merely exemplary aspects of the detailed description of the present disclosure described below, and do not limit the scope of the present disclosure.

According to the present disclosure, a functional multi-joint robot device for face washing and massage and a control method thereof can be provided.

The effects that can be obtained from the present disclosure are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art from the description below. will be.

Figure 1 is a diagram showing the configuration of a functional multi-joint robot device for face washing and massage according to an embodiment of the present disclosure.
Figure 2 is a diagram showing an example for explaining the operation of the articulated robot device of the present disclosure.
Figure 3 is a diagram showing an example of a user interface.
Figure 4 is a flowchart of a method for controlling a functional multi-joint robot device for face washing and massage according to an embodiment of the present disclosure.
Figure 5 is a diagram showing the configuration of a device to which a functional multi-joint robot device for face washing and massage is applied according to another embodiment of the present disclosure.

Hereinafter, with reference to the attached drawings, embodiments of the present disclosure will be described in detail so that those skilled in the art can easily practice them. However, the present disclosure may be implemented in many different forms and is not limited to the embodiments described herein.

In describing embodiments of the present disclosure, if it is determined that detailed descriptions of known configurations or functions may obscure the gist of the present disclosure, detailed descriptions thereof will be omitted. In addition, in the drawings, parts that are not related to the description of the present disclosure are omitted, and similar parts are given similar reference numerals.

In the present disclosure, when a component is said to be “connected,” “coupled,” or “connected” to another component, this is not only a direct connection relationship, but also an indirect connection relationship in which another component exists in between. It may also be included. In addition, when a component is said to "include" or "have" another component, this does not mean excluding the other component, but may further include another component, unless specifically stated to the contrary. .

In the present disclosure, terms such as first and second are used only for the purpose of distinguishing one component from other components, and do not limit the order or importance of the components unless specifically mentioned. Therefore, within the scope of the present disclosure, a first component in one embodiment may be referred to as a second component in another embodiment, and similarly, the second component in one embodiment may be referred to as a first component in another embodiment. It may also be called.

In the present disclosure, distinct components are only for clearly explaining each feature, and do not necessarily mean that the components are separated. That is, a plurality of components may be integrated to form one hardware or software unit, or one component may be distributed to form a plurality of hardware or software units. Accordingly, even if not specifically mentioned, such integrated or distributed embodiments are also included in the scope of the present disclosure.

In the present disclosure, components described in various embodiments do not necessarily mean essential components, and some may be optional components. Accordingly, embodiments consisting of a subset of the elements described in one embodiment are also included in the scope of the present disclosure. Additionally, embodiments that include other components in addition to the components described in the various embodiments are also included in the scope of the present disclosure.

In this disclosure, expressions of positional relationships used in this specification, such as top, bottom, left, right, etc., are described for convenience of explanation, and when the drawings shown in this specification are viewed in reverse, the positions described in the specification The relationship can also be interpreted the other way around.

With the purpose of replacing the effort in our daily lives, technologies using robotic arms are emerging one after another and technological development continues. The average modern person washes their face twice a day. However, on tired days, there are times when I don't want to move my body even a little, and there are times when I can't put water on my hands because I hurt them. In the case of women, a large percentage of women around the world apply skin makeup using sunscreen and powder every day, and depending on the situation, they may fall asleep without being able to remove their makeup.

The gist of embodiments of the present disclosure is to perform face washing and massage of the user's face using an articulated robot arm and an artificial intelligence model.

Figure 1 is a diagram showing the configuration of a functional multi-joint robot device for face washing and massage according to an embodiment of the present disclosure.

Referring to FIG. 1 , the functional multi-joint robot device 100 for face washing and massage according to an embodiment of the present disclosure includes an articulated robotic arm 130, an image capture unit 110, and a control unit 120.

The articulated robot arm 130 is operated under the control of the control unit 120 and may be provided with an end effector for washing or massaging the user's face.

At this time, the articulated robot arm 130 may be fixed at a specific location or may be attached to a movable device.

The articulated robot arm 130 may change the end effector according to the face washing mode, the massage mode, and the detailed settings of each mode under the control of the control unit 120.

In other words, the end effector of the articulated robot arm 130 can be replaced depending on the mode during the process of washing and massaging. For example, in face washing mode, it can be replaced with a brush-shaped rotating end effector, and in massage mode, it can be replaced with an end effector such as a stone or pressure ball. At this time, the end effector can be replaced by rotating the end of the robot arm under the control of the control unit 120, or it can be replaced by moving the robot arm. Of course, users can also replace the end effector themselves.

Furthermore, the articulated robot arm 130 may be equipped with a pressure sensor, sense contact pressure with the user's face through an end effector using the pressure sensor, and provide the sensed contact pressure to the control unit 120.

The image capture unit 110 is a means of capturing the face of a user lying in a specific place in real time, and can capture a color image or depth image of the user's face using at least one camera of an RGB camera and a depth camera. And the image captured in this way can be transmitted to the control unit 120.

At this time, the image capture unit 110 may be attached to a specific part of the articulated robot arm 130, or may be attached to a means to which the articulated robot arm is attached, to capture an image of the user's face in real time. You can.

The control unit 120 analyzes the image captured by the image capture unit 110, recognizes the user's face, and inputs at least one of the user's face image, for example, an RGB image and a depth image. Using the learned artificial intelligence model, the face washing or massage area is detected on the user's face, a movement path plan in the detected face washing or massage area is established, and the multi-joint robot arm 130 is moved according to the established movement path plan. By controlling the articulated robot arm 130 to move at a preset constant pressure on the user's face, the user's face can be washed or massaged.

At this time, the artificial intelligence model collects face images or face photos by race, gender, and age, divides the collected data into learning data, validation data, and test data, and then washes or washes the face in the face photos included in the learning data. By labeling the part corresponding to the massage area, an artificial intelligence model can be learned using the labeled learning data. Here, the artificial intelligence model may be a model that detects a face washing or massage area by using at least one of a face image and a depth image for the face image as input. Of course, the artificial intelligence model learned in this way can be verified using verification data, and model performance can be measured and improved using test data.

Furthermore, the artificial intelligence model can not only detect the face washing or massage area, but also detect the eye area, hairline area, and lip area that the robot arm should not pass through in the detected face washing or massage area. Of course, the artificial intelligence model can be trained using learning data to detect the eye area, hairline area, and lip area in a face image, for example, a depth image of the face. For example, the articulated robot device 100 according to an embodiment of the present disclosure receives an RGB image and a depth image for the face as input, and selects a face washing or massage area, an eye area where the robot arm should not pass, and hair in the face area. An artificial intelligence model can be trained to output the line area and lip area.

Accordingly, the control unit 120 may exclude the eye area, hairline area, and lip area from the detected face washing or massage area and establish a movement path plan to perform face washing or massage in the remaining areas. At this time, the control unit 120 processes the images captured through the RGB camera and the depth camera, and uses an artificial intelligence model that inputs the user's face model and the depth image (or depth value) to determine areas where the robot arm should not pass. It is possible to detect the area where the robot arm should move, excluding the eye area and hairline area.

At this time, the control unit 120 may establish a movement route plan based on the user interface setting values and the user's face recognition result. Depending on the embodiment, the control unit 120 combines the mode, pressure, and time selected by the user with the area to which the robot arm should move derived through face recognition to pass the corresponding facial area in the desired mode and at the desired pressure and time. A travel route plan can be calculated and established. For example, the control unit 120 may establish a movement path plan so that the robot arm can be moved in the following order: forehead, nose, cheeks, chin, temples, etc. At this time, the movement sequence can mimic the various sequences of facial skin massage and facial cleansing that are usually performed by human hands.

Furthermore, the control unit 120 can control the multi-joint robot arm 130 according to the face washing mode and massage mode selected by the user to replace the end effector with an end effector corresponding to the selected mode, and can replace the end effector with the end effector corresponding to the selected mode. By analyzing the video in real time, if the user's face moves, for example, if the location moves, the user's face movement distance is calculated in real time, and the face movement distance is reflected in the movement path plan, so that the movement path reflects the face movement distance. According to the plan, the articulated robot arm 130 can be controlled to move the articulated robot arm 130 at a preset constant pressure on the user's face.

In addition, the control unit 120 receives contact pressure with the user's face from the articulated robot arm 130 so that the articulated robot arm 130 moves along the movement path of the movement path plan while maintaining a preset constant input. The operation of the articulated robot arm 130 can be controlled.

The control unit 120 can select a face wash mode or a massage mode based on information set through the user interface, replace the end effector according to the selected mode, and change the end effector according to the face wash type, massage type, performance time, and pressure. The multi-joint robot arm 130 can be controlled.

Modes for controlling the operation of the articulated robot device 100 and setting values for each mode can be set through a user interface. As shown in FIG. 3, the user interface can set various values for the face washing mode 310 and massage mode 320. For example, the face washing mode (face washing function) 310 may include a face washing range setting item, a face washing type setting item, a face washing pressure setting item, a face washing time setting item, and a lotion application setting item, and a massage mode (massage Function) 320 may include massage type setting items, massage intensity and speed setting items, etc.

At this time, the face wash range setting item allows you to set the face washing range of facial skin, eyes, lips, hairline, etc., the face wash type setting item allows you to select makeup face wash, dust face wash and massage face wash, etc., and the face wash pressure setting item is pressure. You can set the range (0~100%) or pressure value, and the face wash time setting item is for setting the face washing time. You can select items such as quick face washing and precise face washing, and the lotion application setting item is for skin application and skin application. May include lotion application items. In addition, the massage type setting item can be selected from acupressure, roller and swelling removal items, and the massage intensity and speed setting item can set the intensity and speed as a range (0 to 100%) or a numerical value. Of course, the user interface is not limited or restricted to the items shown in FIG. 3 and may further include various items necessary to perform face washing and massage functions. For example, because the user interface may differ in the way of washing and massaging depending on the user, it is possible to directly set the movement path plan in face wash mode and the movement path plan in massage mode or select from various movement path plans. It may contain an item or, depending on the mode, an item that sets the end effector to be used.

The articulated robot device according to an embodiment of the present disclosure will be described with reference to FIG. 2 as follows.

Figure 2 is a diagram showing an example for explaining the operation of the articulated robot device of the present disclosure, and is a diagram showing an example for explaining the operation when the face washing mode is selected.

When explaining the operation of the articulated robot device 100, as shown in FIG. 2, the articulated robot device configures settings through a user interface, for example, information such as face wash mode, face wash time, face wash area, face wash pressure, etc. After is set, when an operation command by user input is received (a), an image including the face of the user lying in a specific location such as a bed is captured in real time using an RGB camera and a depth camera, and image processing of the captured image is performed. By recognizing the face through (b), an RGB image and a depth image of the user's face are generated. Here, the operation command may be received through a related app installed on a user terminal capable of communicating with the articulated robot device, such as a smartphone, or through an input means provided on the articulated robot device, such as a touch pad. may be received.

In addition, the articulated robot device 100 detects a face washing area in a recognized face image, for example, a depth image using a depth image and an artificial intelligence model, and detects the hairline, eye area, and lip area in the detected face washing area. After excluding, establish or calculate a robot path plan (c) for the robot arm to move in the remaining area.

Once the robot path plan for the face wash mode is established, the control unit 120 of the articulated robot device controls the operation of the robot arm in the movement order of the robot path plan to perform the user's face wash with the end effector, and the end effector and the user's face wash. By receiving feedback from the pressure sensor on the pressure sensed by face contact, the movement and pressure of the articulated robot arm 130 are controlled (d) so that the sensed pressure is maintained at a constant pressure.

Through the above-described process, the user's face can be washed using the articulated robot device 100, and when the massage mode is selected, the user's face can be massaged similarly to the above-described process. In other words, only the robot path plan, etc. can be changed by reflecting the settings of the massage mode set through the user interface, such as massage time, massage detailed mode, massage pressure value, etc., and the motion command process, face recognition process, and robot driving process. may operate as described above.

In this way, the face washing and massage functional multi-joint robot device according to an embodiment of the present disclosure performs face washing and massage of the user's face using a multi-joint robot arm and an artificial intelligence model, thereby washing the face and massaging the user's face without moving from the bed, etc. You can get a massage.

In addition, the facial washing and massage functional multi-joint robot device according to an embodiment of the present disclosure can provide acupressure and roller massage to the user's face using an articulated robot arm when the face is swollen or tired.

FIG. 4 is a diagram illustrating a flowchart of a method for controlling a functional articulated robot device for face washing and massage according to an embodiment of the present disclosure, and is a diagram showing operations performed in the articulated robot device of FIGS. 1 to 3 .

Referring to FIG. 4, the method of controlling the facial washing and massage functional multi-joint robot device according to an embodiment of the present disclosure captures an image of the user's face in real time where the user is lying, processes the captured image, and produces a processed image. For example, the face washing area or massage area is detected from the user's face image through an artificial intelligence model that uses the RGB image and depth image of the user's face as input (S410, S420).

When a face washing area or massage area is detected on the user's face in step S420, a movement path plan (or robot path plan) of the robot arm for washing or massaging the user's face in the face washing area or massage area is established or calculated ( S430).

Here, step S430 detects the eye area, hairline area, and lip area that the robot arm should not pass through in the detected face washing or massage area, and excludes the eye area, hairline area, and lip area from the face washing or massage area, You can plan your travel route to perform face washing or massage in the remaining areas.

The movement path plan can be established based on the settings of the user interface and the user's face recognition result. Step S430 determines the area to which the robot arm should move derived through the user-selected mode, pressure, time, and face recognition. In combination, a movement path plan can be calculated and established to pass through the corresponding facial area at the desired pressure and time in the desired mode.

In step S430, when a movement path plan along which the robot arm will move is established, the multi-joint robot arm can be moved to wash the user's face or perform a massage while maintaining a preset constant pressure according to the movement path plan ( S440).

Here, step S440 analyzes the user's face image in real time, calculates the user's face movement distance in real time when the user's face moves, for example, when the location moves, and reflects the face movement distance in the movement route plan. , the multi-joint robot arm can be moved according to a movement path plan that reflects the facial movement distance to wash the user's face or perform a massage.

Even if the description is omitted in the method of FIG. 4, the method according to an embodiment of the present disclosure may include all the contents described in the device of FIGS. 1 to 3, which will be obvious to those skilled in the art. do.

Figure 5 is a diagram showing the configuration of a device to which a functional multi-joint robot device for face washing and massage is applied according to another embodiment of the present disclosure.

For example, the functional multi-joint robot device for face washing and massage according to the embodiment of the present disclosure shown in FIG. 1 may be the device 1600 shown in FIG. 5 . Referring to FIG. 5, the device 1600 may include a memory 1602, a processor 1603, a transceiver 1604, and a peripheral device 1601. Additionally, as an example, the device 1600 may further include other components and is not limited to the above-described embodiment. At this time, the device 1600 may be, for example, a movable user terminal (e.g., smart phone, laptop, wearable device, etc.) or a fixed management device (e.g., server, PC, etc.).

More specifically, the device 1600 of FIG. 5 may be an example hardware/software architecture such as a robotic device, an eye washing device, a massage device, a beauty device, etc. At this time, as an example, the memory 1602 may be a non-removable memory or a removable memory. Additionally, as an example, the peripheral device 1601 may include a display, GPS, or other peripheral devices, and is not limited to the above-described embodiment.

Additionally, as an example, the above-described device 1600 may include a communication circuit like the transceiver 1604, and may communicate with an external device based on this.

Additionally, as an example, the processor 1603 may include a general-purpose processor, a digital signal processor (DSP), a DSP core, a controller, a microcontroller, Application Specific Integrated Circuits (ASICs), Field Programmable Gate Array (FPGA) circuits, and any other It may be at least one of a tangible integrated circuit (IC) and one or more microprocessors associated with a state machine. In other words, it may be a hardware/software configuration that performs a control role to control the device 1600 described above. Additionally, the processor 1603 can modularize and perform the functions of the control unit 120 of FIG. 1 described above.

At this time, the processor 1603 may execute computer-executable instructions stored in the memory 1602 to perform various essential functions of the multi-joint robot device for face washing and massage functions. As an example, the processor 1603 may control at least one of signal coding, data processing, power control, input/output processing, and communication operations. Additionally, the processor 1603 can control the physical layer, MAC layer, and application layer. Additionally, as an example, the processor 1603 may perform authentication and security procedures at the access layer and/or application layer, and is not limited to the above-described embodiment.

As an example, the processor 1603 may communicate with other devices through the transceiver 1604. As an example, the processor 1603 may control the facial washing and massage functional multi-jointed robot device to communicate with other devices through a network through execution of computer-executable instructions. That is, communication performed in this disclosure can be controlled. As an example, the transceiver 1604 may transmit an RF signal through an antenna and may transmit signals based on various communication networks.

Additionally, as an example, MIMO technology, beamforming, etc. may be applied as antenna technology, and is not limited to the above-described embodiment. Additionally, signals transmitted and received through the transmitting and receiving unit 1604 may be modulated and demodulated and controlled by the processor 1603, and are not limited to the above-described embodiment.

Exemplary methods of the present disclosure are expressed as a series of operations for clarity of explanation, but this is not intended to limit the order in which the steps are performed, and each step may be performed simultaneously or in a different order, if necessary. In order to implement the method according to the present disclosure, other steps may be included in addition to the exemplified steps, some steps may be excluded and the remaining steps may be included, or some steps may be excluded and additional other steps may be included.

The various embodiments of the present disclosure do not list all possible combinations but are intended to explain representative aspects of the present disclosure, and matters described in the various embodiments may be applied independently or in combination of two or more.

Additionally, various embodiments of the present disclosure may be implemented by hardware, firmware, software, or a combination thereof. For hardware implementation, one or more ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), DSPDs (Digital Signal Processing Devices), PLDs (Programmable Logic Devices), FPGAs (Field Programmable Gate Arrays), general purpose It can be implemented by a processor (general processor), controller, microcontroller, microprocessor, etc.

The scope of the present disclosure is software or machine-executable instructions (e.g., operating system, application, firmware, program, etc.) that cause operations according to the methods of various embodiments to be executed on a device or computer, and such software or It includes non-transitory computer-readable medium in which instructions, etc. are stored and can be executed on a device or computer.

100 articulated robot device
110 Video recording department
120 control unit
130 Jointed robot arm

Claims (1)

A multi-joint robot arm with an end effector for washing or massaging the user's face;
an image capture unit that captures the user's face; and
Detect a face washing or massage area on the user's face using the user's face image and a pre-trained artificial intelligence model, establish a movement path plan in the detected face washing or massage area, and plan the established movement route. A control unit that controls the articulated robot arm so that the articulated robot arm moves at a preset constant pressure on the user's face.
Including, an articulated robot device.

Images