KR20160050592A - Transparent and stretchable interactive human-machine interface system - Google Patents

Abstract

The present invention relates to a transparent and stretchable interactive human machine interface system. More particularly, the present invention relates to a transparent and stretchable motion sensor attached to a skin of a subject; A data processing unit for receiving a signal regarding an operation of an object obtained from the motion sensor and transmitting the signal to a machine to be executed; And a transparent, stretchable electrical stimulator that receives a signal relating to the information detected by the machine and applies a stimulus to the subject and is attached to the skin of the subject.

Description

Transparent and Stretchable Interactive Human Machine Interface System [

The present invention relates to a transparent and stretchable interactive human machine interface system. More particularly, the present invention relates to a transparent and stretchable motion sensor attached to a skin of a subject; A data processing unit for receiving a signal regarding an operation of an object obtained from the motion sensor and transmitting the signal to a machine to be executed; And a transparent, stretchable electrical stimulator that receives a signal relating to the information detected by the machine and applies a stimulus to the subject and is attached to the skin of the subject.

An interactive human machine interface (iHMI) is a bi-directional electronic system that effectively communicates human intent to a machine, collects feedback information from the machine, and connects humans and machines.

The wearable iHMI system is particularly suitable for smart glasses (Feng, S., et al . Immunochromatographic diagnostic test using using glass, ACS Nano 8, 3069-3079 (2014)) and a timepiece (Wile, DJ, Ranawaya, R. , Kiss, ZHT Smart watch accelerometry for analysis and diagnosis of tremor, J. Neurosci. Methods 230, 1-4 (2014)) Has recently been attracting attention with the recent development of wearable electronic devices.

Wearable devices incorporating robust sensors and actuators offer high performance and practicality, but inconveniences arise from mechanical incompatibility between the human body and bulky rigid devices (Kim, D.-H., et al . Epidermal Electronics. Science 333 , 838-843 (2011) (Son, D., et al . Multifunctional wearable devices for diagnosis and therapy of movement disorders. Nat . Nanotech . 9 , 397-404 (2014) (Jeong, J.-W., et al . Materials and optimized designs for human-machine interfaces via epidermal electronics. Adv. Mater . 25 , 68396846 (2013)); Due to the design irreversibility of rugged electronics (Reuss, RH, et al . Macroelectronics: perspectives on technology and applications, Proc. IEEE , 93 , 1239-1256 (2005)), An unnatural appearance of the wearer due to design misalignment; And signal artefacts due to the nonconfomal attachment of the robust sensor to the human body (Kim, D.-H., et al . Epidermal Electronics. Science 333 , 838-843 (2011) (Son, D., et al . Multifunctional wearable devices for diagnosis and therapy of movement disorders. Nat . Nanotech . 9 , 397-404 (2014) (Jeong, J.-W., et al . Materials and optimized designs for human-machine interfaces via epidermal electronics. Adv . Mater . 25 , 68396846 (2013)).

Therefore, there is a need for a new device that can be confomally laminated to human skin to have a natural appearance and a high signal-to-noise ratio (SNR).

The adoption of a flexible and stretchable design and subsequent reduction in thickness and weight of the device is an important goal with respect to wearable electronic device design.

Recently, a flexible inorganic electronic device (Kim, D.-H., et al . Epidermal Electronics. Science 333 , 838-843 (2011) (Son, D., et al . Multifunctional wearable devices for diagnosis and therapy of movement disorders. Nat . Nanotech . 9 , 397-404 (2014) (Webb, RC, et al . Ultrathin conformal devices for precise and continuous thermal characterization of human skin. Nat . Mater . 12 , 938-944 (2013)), ultra-light and lightweight organic sensors (Someya, T., et al . Conformable, flexible, large-area networks of pressure and thermal sensors with organic transistor active matrixes. Proc . Natl . Acad . Sci . USA 102, 12321-12325 (2005) ( Sekitani, T., Zchieschang, U., Klauk, H., Someya, T. Flexible organic transistors and circuits with extreme bending stability. Nat. Mater. 9, 1015-1022 (2010 ), flexible electronics skin (Takei, K., et al. Nanowire active-matrix circuitry for low-voltage artificial skin macroscale. Nat. Mater. 9, 821-826 (2010) (Wang, C., et al . User-interactive electronic skin for instantaneous pressure visualization. Nat . Mater . 12, 899-904 (2013), and highly sensitive and flexible mechanical sensor (Mannsfeld, SCB, et al. Highly sensitive flexible pressure sensors with microstructured rubber dielectric layers. Nat. Mater. 9, 859-864 (2010)) (Lipomi, DJ, meat al . Skin-like pressure and strain sensors based on transparent elastic films of carbon nanotubes. Nat . Nanotech . 6 , 788-792 (2011)) (Jung, S., et al . Reverse-Micelle-Induced Porous Pressure-Sensitive Rubber for Wearable HumanMachine Interfaces. Adv Mater. Early View (2014)) (Gong, S, et al . A wearable and highly sensitive pressure sensor with ultrathin gold nanowires. Nat. Comm, 5 , 3132 (2014)).

The ultra-thin, deformable design allows accurate data collection from the human body with minimal signal noise. However, these previously reported devices are made of opaque semiconductors and metals and appear different from human skin. In addition, most of these sensors consume large amounts of power and thus require large power supplies (Kim, D.-H., et al., Epidermal Electronics Science 333 , 838-843 (2011) D., et al . Multifunctional wearable devices for diagnosis and therapy of movement disorders. Nat . Nanotech . 9 , 397-404 (2014) (Jeong, J.-W., et al . Materials and optimized designs for human-machine interfaces via epidermal electronics. Adv. Mater . 25 , 68396846 (2013)).

Transparent electronic materials can make the wearable device invisible, creating a natural look and enhanced aesthetics.

For example, carbon-based nanomaterials (Grafin (Kim, KS , et al . Large-scale pattern growth of graphene films for stretchable transparent electrodes. Nature 457 , 706-710 (2009)) and carbon nanotubes (Wu, Z. , et al . Transparent, conductive carbon nanotube films. Science , Vol. 87 , 1273-1276 (2004)) and metal nanowires (NW) (Hu, L., Kim, HS, Lee, J.-Y., Peumans, P., Cui, Y., meat al . Scalable coatings and properties of transparent, flexible, silver nanowire electrodes. ACS Nano 4 , 2955-2963 (2010)) and gold nanowires (Moraq, A., Ezersky, V., Froumin, N., Moqiliansky, D., Jelinek, R., Transparent, conductive gold nanowire networks assembled from soluble Au thiocyanate. Chem. Comm ., 49 , 8552-8554 (2013))) have been intensively studied.

Such transparent nanomaterials can be used in flexible electronic devices because they dramatically reduce the bending strength of nanoscale structures (Son, D., et al . Multifunctional wearable devices for diagnosis and therapy of movement disorders. Nat . Nanotech . 9 , 397-404 (2014)) (Rogers, J., Lagally, MG, Nuzzo, RG, Synthesis, Assembly and Applications of Semiconductor Nanomembranes, Nature 477 , 45-53 (2011)).

On the other hand, self-powered mechanical sensors based on piezoelectric materials can reduce the power dissipation of electronic systems (Xu, S., et al., Self-powered nanowire devices, 5 , 366-373 2010).

Poly (vinylidene chloride) (Lee, M. , et al . A hybrid piezoelectric structure for wearable nanogenerators. Adv . Mater . 24, 1759-1764 (2012)) ( Persano, L., et al. High performance piezoelectric devices based on aligned arrays of nanofibers of poly (vinylidenefluoride-co-trifluoroethylene). Nat. Comm. 4, 1633 (2013)) and Polylactic acid (PLA) (Yoshida, T. , et al . In this paper, we propose a new type of chiral polymer film. Jpn . J. Appl . Phys . 50 , 09ND13-1-09ND13-5 (2011)), and zinc oxide (Wang, ZL, Song, J., Piezoelectric nanogenerators based on zinc oxide nanowire arrays. Science 312 , 242-246 (2006) And lead zirconate titanate (Qi, Y. , et al . Enhanced piezoelectricity and stretchability in energy harvesting devices fabricated from buckled PZT ribbons. Nano Lett . 11 , 1331-1336 (2011)), a wide variety of piezoelectric materials have been studied.

The present inventors have found that by observing human motion, converting recorded data into a signal for machine control, delivering a triggering signal to the machine, and transmitting feedback information from the machine to an operator via an electrotactile device We developed a transparent and elastic iHMI system consisting of stimulator and wearable sensor that can transmit.

An object of the present invention is to provide a transparent and stretchable motion sensor attached to a skin of an object; A data processing unit for receiving a signal regarding an operation of an object obtained from the motion sensor and transmitting the signal to a machine to be executed; And a transparent, stretchable electrical stimulator that receives a signal relating to the information detected by the machine and applies a stimulus to the subject and is attached to the skin of the subject.

An object of the present invention described above is to provide a transparent and stretchable motion sensor attached to a skin of a subject; A data processing unit for receiving a signal regarding an operation of an object obtained from the motion sensor and transmitting the signal to a machine to be executed; And a transparent, stretchable electrical stimulator attached to the skin of the subject to receive a signal relating to the information detected by the machine, to stimulate the subject, and to provide an interactive human-machine interface system .

In the interactive human-machine interface system of the present invention, the transparent and stretchable motion sensor comprises: a first protective layer; A first electrode layer formed adjacent to the first passivation layer; A piezoelectric layer formed adjacent to the first electrode layer; A second electrode layer formed adjacent to the piezoelectric layer; And a second protective layer formed adjacent to the second electrode layer.

In the transparent and stretchable motion sensor, the first protective layer may be polymethylmethacrylate (PMMA) or polylactic acid (PLA). The thickness of the first passivation layer may be 50 nm to 100 탆.

In the motion sensor of the interactive human-machine interface system of the present invention, the first electrode layer may be graphene, carbon nanotube (SWNT), or silver nanowire (AgNW). Moreover, the graphene may be graphene doped with gold. In addition, the thickness of the first electrode layer may be 1 nm to 100 nm.

In the motion sensor of the interactive human-machine interface system of the present invention, the piezoelectric layer may be a polylactic acid / single walled carbon nanotube (PLA / SWNT) composite film. In addition, the size of the single-walled carbon nanotube may be 50 nm to 1 mm. The thickness of the piezoelectric layer may be 100 nm to 500 mu m.

In the motion sensor of the interactive human-machine interface system of the present invention, the second electrode layer may be graphene, carbon nanotube (SWNT), or silver nanowire (AgNW). Moreover, the graphene may be graphene doped with gold. The thickness of the second electrode layer may be 1 nm to 100 nm.

In the motion sensor of the interactive human-machine interface system of the present invention, the second protective layer may be polymethylmethacrylate (PMMA) or polylactic acid (PLA). In addition, the thickness of the second protective layer may be 50 nm to 100 탆.

In the motion sensor of the interactive human-machine interface system of the present invention, the first protective layer, the first electrode layer, the piezoelectric layer, the second electrode layer, and the second protective layer may be patterned in a serpentine pattern . When patterned in this way, it is well adhered to the skin and well behaves like bending or stretching of the body.

In an interactive human-machine interface system of the present invention, the transparent, stretchable electrical stimulator comprises a patterned first protective layer; A patterned electrode layer formed adjacent to the patterned first passivation layer; A patterned second protective layer formed adjacent to the patterned electrode layer; And an adhesion layer formed adjacent to the patterned second protective layer.

In the electric stimulator of the interactive human-machine interface system of the present invention, the patterned first protective layer may be epoxy resin (SU8, Microchem) or benzocyclobutene (BCB). In addition, the thickness of the patterned first protective layer may be 100 nm to 100 탆.

In the electric stimulator of the interactive human-machine interface system of the present invention, the patterned electrode layer may be graphene, carbon nanotube (SWNT), silver nanowire (AgNW). Preferably, the patterned electrode layer may be a graphene / silver nanowire / graphene multilayer. The silver nanowire serves to increase the conductivity of graphene. Further, the thickness of the patterned conductive layer may be 1 nm to 100 nm.

In the electric stimulator of the interactive human-machine interface system of the present invention, the patterned second protective layer may be epoxy resin (SU8, Microchem) or benzocyclobutene (BCB). Also, the thickness of the patterned second protective layer may be 100 nm to 100 탆.

In the electrical stimulator of the interactive human-machine interface system of the present invention, the adhesion layer may be polydimethylsiloxane. Further, the thickness of the adhesion layer may be from 1 μm to 100 μm.

The motion sensor of the iHMI system according to the present invention is capable of collecting signals with minimal noise because it can be integrated with human skin in a flexible and conformal manner due to its ultra-thin serpentine design. In addition, the high conductivity of the graphene / AgNW multilayers of electrical stimulators of the iHMI system according to the present invention enables effective feedback on electrical stimulation, which is the result of the closed loop of an iHMI with skin- (close-loop).

The iHMI system of the present invention can be used to control a robot and can be used to convey a feedback signal in addition to causing the robot to perform various kinds of movements such as bending, catching and lifting objects.

Figure 1 is a photograph of a transparent piezoelectric motion sensor (top) and an electric stimulator (top, inset) of the present invention conformally contacting the upper part of a human wrist, ) Is partly peeled off.
2 is a control photograph of a motion sensor and an electric stimulator manufactured by replacing a graphen (GP) electrode with a gold thin film.
3 is a developed view of a patterned graphene (GP) heterostructure used for the motion sensor (left) and electric stimulator (right) of the present invention.
4 is a scanning electron microscope (SEM) photograph (PMMA / graphene: red, PLA: green) photographing a cross section of a patterned graphene (GP) heterostructure used in the motion sensor of the present invention.
5 is a scanning electron microscope (SEM) photograph (epoxy: blue, GP / AgNW / GP: pink) of a cross section of a patterned graphene (GP) heterostructure used in the electrical stimulator of the present invention.
6 is a schematic diagram of a closed loop system of an iHMI in accordance with the present invention.
Fig. 7 is a representative photograph showing human motion controlling the robot arm.
8 is a representative photograph showing a corresponding position of a robot arm controlled by a human motion.
Fig. 9 shows the transmittance in the visible light region of the motion sensor (PLA (polylactic acid): black, PLA / SWNT (polylactic acid / single wall carbon nanotube: red) and electric stimulator (blue)) of the present invention.
Figure 10 shows the sheet resistance of the initial (red), gold-doped (green), and silver nanowire-coated (blue) GP heterostructures. The inset of FIG. 10 is a SEM photograph of the GP coated with the silver nanowire.
Fig. 11 shows human motion (left), its enlarged view (middle) with its corresponding bending radius, and bending motion (right) at each bending radius.
Figure 12 shows the change in resistance per unit length of the initial (red), gold-doped (green), and silver nanowire-coated (blue) GP heterostructures, and the resistance varies inversely with the bending radius.
13 is a SEM photograph of the surface of the ITO film on the PET layer after bending the structure.
Figure 14 shows a finite element analysis showing the strain distribution of the GP heterostructure when the bending radius R is infinity (Figure 14a), 1.3 cm (Figure 14b), 0.52 cm (Figure 14c) and 0.38 cm (Figure 14d) (FEA) results and corresponding photographs of the bent heterostructure.
15 shows a process of fabricating a piezoelectric graphene heterostructure for a motion sensor of the present invention.
Fig. 16 shows the output current (Fig. 16A) and output voltage (Fig. 16B) of a motion sensor made of PLA (blue) and SWNT-encapsulated PLA (red) as a function of time.
17A is an X-ray diffraction spectrum of a PLA / SWNT composite film, FIG. 17B is a Raman spectrum of a PLA / SWNT composite, and FIG. 17C is a description of each Raman peak.
18A and 18B show the piezoelectric output current and piezoelectric output voltage of the PLA / SWNT composite film, which is half the SWNT concentration of the composite film of FIG. 16, and FIG. 18C shows the cycle reliability test (150 or more bending operations 18D is a schematic view (right side) of a step of applying shear stress to the PLA / SWNT composite film and the piezoelectric output voltage (left side) of the PLA / SWNT composite film when the shear stress is distinct, 18f are photographs of a forward connection and a reverse connection setup for external data acquisition (external DAQ) for electrical characteristic measurement, respectively.
Figures 19a and 19b show the output voltage of the PLA / SWNT composite as a function of time in forward and reverse connections, respectively.
Figure 20 shows the charge separation in a PLA / SWNT composite film under compression and tensile stress.
Figure 21A shows stress distributions of horizontal (left) and vertical (right) aligned SWNTs determined by FEA. 21B shows the output voltage (left side) and the corresponding output voltage of PLA / SWNT composite film over time as a function of the inverse of the bending radius, in the reciprocal of the other bending radius, Is a photograph showing the bending of the film of Fig. Figure 21c shows the output voltage (left) and the corresponding output voltage of the PLA / SWNT composite film over time at different pressures as a function of pressure (right), and the insertion angle is plotted in the vertical direction (red arrow) It is a photograph showing the film.
22 shows a schematic diagram of a punching process for the motion sensor of the present invention and an image (right side) of the motion sensor.
23A is a photograph of a piezoelectric motion sensor stretched by a stretching machine (left side) and a hand (right side), and FIG. 23B is a photograph showing the output voltage switching of a motion sensor connected to the forward (left) and reverse .
24 is a plot of the output voltage of the motion sensor as a function of time in four modes (unstrained, pressure-loaded, compressed and tensioned), and the inset of each graph is a photograph of the motion sensors of the four modes.
25 is a strain distribution corresponding to the four modes (unconstrained, pressure-loaded, compression and tensile) obtained through FEA.
26 is a schematic view showing a method of manufacturing a transparent electric stimulator of the present invention.
Figure 27 is a photograph of a transparent electrical stimulator of the present invention attached to pig skin.
28 is an illustration of an experimental apparatus for performing electrical stimulation and recording using the electric stimulator of the present invention.
Fig. 29 shows the measured voltages from the recorders 2, 3 and 4 shown in Fig.
Figure 30 shows the recorded voltage as a function of the number of skin layers for different stimulation currents.
Figure 31 plots the minimum stimulation voltage needed to be sensed as a function of frequency.
32 is FEA (Finite Element Analysis) results of three-dimensional potential distribution of the skin in the case of direct stimulation using the electric stimulator of the present invention.
Figure 33 is a photograph of the pristine (left) and elongated (right) states of the transparent, stretchable electrical stimulator of the present invention.
Figure 34 shows the resistance of the serpentine GP electrode as a function of applied strain.
Figure 35 is an FEA result showing strain distribution on an electric stimulator / recorder under 30% stretching.
36A and 36B are flow charts for the iHMI demonstration of the present invention and corresponding experiment settings, respectively.
37A shows a transparent motion sensor and electric stimulator angled on the upper surface of the wrist and on the lateral side of the forearm (the robot arm is stationary if no motion is detected), Figs. 37B- Apply pressure and show the robot arm bending, grasping, and lifting as it opens.
Figure 38 shows the detection signal obtained by the transparent pressure sensor (first column) and by the gripper (second column), and the stimulus signal delivered by the transparent stimulator (third column). An enlarged picture of the stimulus signal is shown in the inset of the graph in the third column. The programmed closing-motion data (blue line) and feedback-controlled closed-operation (red line) of the clamp are shown in the fourth column. The maximum motion of the forceps changes in the range of 149 degrees (completely closed) to 190 degrees (completely open, lower left insertion degree of the graph in the fourth column).
FIG. 39 is a snapshot showing representative operations of the forceps.

Hereinafter, the present invention will be described in more detail with reference to the following examples or drawings. It is to be understood, however, that the following description of the embodiments or drawings is intended to illustrate specific embodiments of the invention and is not intended to be exhaustive or to limit the scope of the invention to the precise forms disclosed.

Example  1. Silver Of nanowires (AgNW)  synthesis

5 ml of ethylene glycol (EG, JUNSEI, Japan) was poured into a 50 ml vial and heated in an oil bath (153 ° C) with stirring at 260 rpm to synthesize the nanowires. After heating for 30 minutes, 40 μL of 4 mM copper chloride (CuCl ₂ .2H ₂ O, 99%, DAEJUNG, Korea) solution dissolved in EG was added with heating for 15 minutes. 1.5 ml of 0.094 M silver nitrate (AgNO ₃ 99% +, Strem Chemicals, Inc.) dissolved in 1.5 mL of 0.147 M poly (vinylpyrrolidone) (PVP, avg. MW 55000, Aldrich, USA) and EG. , USA). After the injection, the reaction was allowed to proceed for 1 hour. Next, the silver nanowire solution was centrifuged and redispersed in acetone three times. Finally, the silver nanowires were dried and dispersed in ethanol to 1 wt%.

Example  2. Grapina  Doping and synthesis

Graphene was synthesized by chemical vapor deposition (CDV) on 25 μm copper foil (Alfa Aesar, USA). The copper foil was annealed at 1000 占 폚 for 1 hour with a constant hydrogen flow (8 sccm) and then methane gas (20 sccm) was inserted for 30 minutes. Subsequently, under a hydrogen atmosphere, the temperature of the chamber was rapidly cooled to room temperature. After the PMMA (A4, 495, Microchem, USA) was spin-coated, the graphene layer synthesized on the copper foil was immersed in a copper etching solution to etch copper, and the graphene / PMMA layer was floated. The separated graphene layer was washed with deionized water to remove the residual etchant. The backside of the graphene was doped by exposure to AuCl ₃ (Sigma-Aldrich, USA) solution (20 mM in DI) for 10 minutes.

Example  3. Transparent Wearable  Manufacture of motion / pressure sensor

PLA (Sigma Aldrich, USA) was dispersed in chloroform (98.5%, Samchun, Korea) at a concentration of 3 wt% using a magnetic stirrer. SWNT (Hanhwa, Korea) was also dispersed in chloroform under ultrasonic treatment (1.6 × 10 ^-6 wt%). The two dispersions were then drop cast into a slide glass to form a PLA / SWNT complex. After drying the dispersion for 24 hours at room temperature, the resulting PLA / SWNT composite film (about 70 μm thick) was separated from the slide glass. Next, PMMA was spin-coated on the synthesized graphene sheet, and the back side of the graphene sheet was exposed to the gold doping solution. The gold-doped graphene / PMMA film was transferred to a PLA / SWNT composite film. Finally, another gold doped graphene / PMMA film was transferred to the other side of the PLA / SWNT composite film.

Example  4. Transparent Wearable  Manufacture of electric stimulator

The method comprises the steps of: Starting from the wafer plating (Cr / Ni, 7 nm / 70 nm thickness), the substrate was then spin-coated with a negative epoxy (SU8, Microchem, USA). An epoxy pattern was formed by photolithography. Then, the graphene layer was transferred to the substrate. Next, the silver nanowire dispersion was spin-coated on the graphene layer at 3000 rpm for 30 seconds and baked at 200 DEG C for 5 minutes. Finally, a second graphene layer was transferred to the graphene layer containing the silver nanowires and separated using reactive ion etching (O ₂ plasma, 150 sccm, chamber pressure of 150 mTorr, high frequency power 100 W, etch time 20 seconds). The upper epoxy layer was patterned through photolithography. PMMA layer was spin-coated on top; This layer served as a supporting layer. A nickel etchant was used to remove the nickel film serving as a sacrificial layer. After the etching treatment was completed, the stimulator was transferred onto a PDMS / PVA film. Finally, the PMMA layer of the device was removed with an acetone solution.

Example  5. Connection between motion sensor and electric stimulator ( wiring connection ) Way

Using a silver paste, a flexible copper wire (150 μm in diameter, enamel coated) was connected to a gold pad on the motion sensor that transmits signals to the external DAQ board. After the silver paste was fully cured, epoxy (Araldite, Ciba-Geigy, Switzerland) was applied to the cured silver paste to ensure that the connection stays stable during operation. For the electrical stimulator, the stimulator was connected to a power source using a flexible cable (i.e., an anisotropic conductive film (Elform, USA)). The film was exposed to a sufficiently high temperature (160 占 폚 to 180 占 폚) and pressure (2 to 3 MPa) to adhere and firmly connect the film.

Example  6. Transparent and stretchy iHMI  system

Figure 1 shows a photograph of a transparent and stretchable iHMI, consisting of a piezoelectric motion sensor (top view) and an electric stimulator (inset; bottom). Partially separated photographs are also shown for the motion sensor (lower left) and the electrical stimulator (lower right). The device's ultra-thin, lightweight and stretch allows confomal intergration with the human skin and is considerably comfortable. Sprayed elastomeric films further improve adhesion. Also, due to the high transparency, the device is still difficult to notice after partial delamination. To easily test the design of the device, a control sample made of a gold electrode instead of a graphene electrode was prepared with the same design (Fig. 2). The invisible, skin-conforming device using a transparent material has a natural, aesthetically pleasing appearance and assures privacy of the individual.

The structural design of the device and the properties of the material used are shown in an enlarged view of the patterned graphene (GP) heterostructure (FIG. 3) used in the motion sensor (left) and electric stimulator (right). The motion sensor (FIG. 4) is composed of a piezoelectric polymer thin film (PLA, approximately 70 μm) embedded in a piezoelectric graphene heterostructure, ie SWNT, between the graphene electrodes and the insulating layers. The graphene used was grown via chemical vapor deposition using copper as a catalyst and doped with gold salt. A polymethacrylate (PMMA) layer insulates the device and improves the piezoelectric power-generation ability of the SWNTs ( see FIG. 3) . The electrical stimulator consists of a network of silver nanowires sandwiched between conductive graphene heterostructures (Figure 5), i.e., doped graphene layers, which leads to better conductivity and charge-injection into the skin . The silver nanowires were synthesized in ethylene glycol using a copper seed and a poly (vinyl pyrrolidone) ligand. Figures 4 and 5 show scanning electron microscope (SEM) photographs of cross sections of piezoelectric graphene heterostructures, respectively, used in motion sensors and stimulators. The PMMA / graphene and PLA layers of the motion sensor are shown in red and green, respectively, and the epoxy and graphene / AgNW / graphene layers of the stimulator are shown in blue and pink, respectively.

A circuit diagram of the closed loop system of the iHMI is shown in Fig . The motion sensor monitors human motion and converts the collected data into signals for controlling the robot. By the successful execution of the command by the robot, a feedback signal is generated which is transmitted to the wearer of the apparatus via the stimulator. For example, the process in which the robot picks up a target object is monitored by the piezoelectric pressure sensor integrated in the gripper. When this operation is successfully completed, the electrical stimulator located on the skin of the user is activated. The conformal integration of a flexible piezoelectric sensor to human skin minimizes signal noise and increases the accuracy of machine control. Figures 7 and 8 show representative photographs of the synchronized movement of the robot controlled by the human wearer (i. E., The bending of the robotic arm due to the controlling human wrist movement). A detailed demonstration of the iHMI is shown in Fig .

Example  7. Optical, electrical and mechanical properties

The transmittance of a stimulator made of GP / AgNW / GP and of a motion sensor made of PLA and PLA / SWNT was measured in the visible light region (FIG. 9, 380 to 780 nm); Both devices exhibit high transparency. Indium tin oxide (ITO) electrodes are widely used as transparent electrodes. However, in the pristine state, graphene has sheet resistance higher than that of ITO, and therefore graphene must be deformed in order to improve the conductivity. Thus, conductive graphene heterostructures comprising silver nanowires, comprised of multi-layer graphene doped with gold chloride, were used. Figure 10 shows the sheet resistance measured for an initial state (696.5? /?), Gold doping (354.5? /?) And silver nanowire coating (98.8? /?) For a graphene electrode; The measured values indicate that the conductivity of graphene can be successfully increased by doping. The inset shows a SEM photograph of silver nanowire-coated graphene.

One of the advantages of the graphene heterostructure for ITO is the extremely high mechanical deformability. The graphene heterostructure and the ITO film are bent at different bending radii using a bending step. The bending radius is measured based on the position of the bent human wrist (FIG. 11). In the bending test, it was confirmed that the graphene heterostructure maintains high electrical conductivity even in an extremely bent state, whereas ITO film shows a sharp increase in sheet resistance (FIG. 12). As shown in the SEM image of the ITO film (Fig. 13), the increase in resistance of the ITO is due to cracks formed when bent. On the other hand, cracks are not observed in the graphene heterostructure.

An analysis based on theoretical mechanics confirms this observation. Strain distributions for different bending radii of the graphene heterostructure and ITO films were obtained by FEA; Photographs of the experiment are shown in Figs. 14A to 14D. The red dotted box corresponds to the analysis area of FEA. The FEA results show that the local strain distributions of the graphene heterostructure and ITO film at the same bending radius are not much different. However, critical strains that cause mechanical cracks in the ITO film (<1%) and graphene heterostructures (> 5%) are different. The addition of silver nanowires further improves the mechanical strength of the graphene heterostructure. Thus, the conductive graphene heterostructure maintains its inherent electrical properties even during the dynamic operation of the human body.

Example  8. Piezoelectric GP Heterostructure

The steps involved in integrating the piezoelectric graphene heterostructure and the motion sensor are shown in Fig. Due to the low amplitude of the generated piezoelectric voltage and current, it is difficult to obtain a high SNR in the initial PLA layer (blue plot in FIGS. 16A and 16B). In order to amplify the generated piezoelectric signal, a SWNT is embedded in the PLA layer. The PLA / SWNT composite film exhibits a crystallinity of 39% as measured by X-ray diffraction analysis (Fig. 17A). The coexistence of PLA and SWNT in the PLA / SWNT complex was confirmed by Raman spectroscopy (FIGS. 17B and 17C). Under the same applied strain, the PLA / SWNT complex produced a current of about 4 nA and a voltage of about 210 mV (red plot in Figures 16a and 16b); These values are 8-fold and 5-fold higher than in the case of PLA in the initial state (blue plot in FIGS. 16A and 16B). The increase in voltage and current is related to the concentration of SWNT in the complex. Figures 18A and 18B show that the voltage and current produced by the PLA composite film, containing SWNTs that are half the original concentration, are between the voltage and current corresponding to the blue and red plots in Figures 16A and 16B. Improvement of the piezoelectric properties of the composite is also found after a number of bending cycles (Fig. 18C). The piezoelectric voltage in the PLA / SWNT composite can also be generated by applying shear stress (FIG. 18D).

To verify that the measured signal is caused by piezoelectric charge generation, the polarity of the first peak voltage in a forward connection (Fig. 19a, Fig. 18e) is plotted in a reverse connection (Fig. 19b, Fig. 18f) . The difference in polarity between the two cases confirms that the signal is essentially piezoelectric. Further, even when the bending direction is changed, such a change in polarity is observed (Figs. 20 and 4B ). When the PLA / SWNT complex is subjected to compression or tensile strain, negative or positive charges accumulate adjacent to the upper and lower electrodes. This accumulation of charge is due to charge separation by strain in the PLA. On the other hand, the high Young's modulus of the SWNT improves the local strain resistance of the PLA and thus maximizes charge generation. This is evidenced by the fact that the strain is concentrated near the SWNT; It is also confirmed by FEA (applying 1% strain to the PLA / SWNT complex). Aligning the SWNT horizontally or vertically within the PLA; Two different cases are modeled (left and right photographs in Fig. 21A, respectively). In both cases, the stress tends to be concentrated near the SWNTs (Park, K.-I., et al. Piezoelectric BaTiO ₃ thin film nano generator on plastic substrates. Nano Lett . 10 , 4939-4973 (2010)). In order to study the output voltage as a function of the applied strain, the voltage generated in the composite film was measured at various strains obtained by bending the film with different bending radii. 21B shows a linear calibration curve (right) with a corresponding voltage output (left) and a slope of about 0.12 mV / cm &lt;& quot ^{; 1 &} gt ;. Figure 21c also shows the piezoelectric voltage output produced by applying different pressures in the composite film (left); The linear correlation curve showing the relationship between the two has a slope of about 0.8 mV / kPa (right). If the relationship between the piezoelectricity produced in the graphene heterostructure and the mechanical stimulus applied externally is linear, then the heterostructure is suitable for use as a sensing element in an iHMI.

Example  9. For motion detection Patterned  Piezoelectric GP Heterostructure

The PLA / SWNT composite film is patterned using a mechanical punching process to ensure a completely twisted and flexible design (Figure 22). The custom-made punching mask is placed on a press machine disposed on the piezoelectric graphene heterostructure film. The machine is then pressurized and depressurized to a suitable pressure. The fully serpentine configuration of the motion sensor enables reversible stretching (Fig. 23A). The sensor can also be placed conformally to the human skin and thus remains in contact while the body is moving (Figure 24). The operation sensor generates a voltage during deformation such as pressing (right upper end), elongation (left lower end), or compression (right lower end) without generating a potential when it is not deformed (left upper end). The amplitude and polarity of the generated potential are dependent on the nature of the deformation (pressing versus elongation) and the direction of deformation (elongation versus compression). The generated potential is essentially piezoelectric; This is confirmed by a change in voltage polarity when different connections are used (Fig. 23B). The strain distribution in the motion sensor causing the output signal shown in Fig. 4B is predicted through FEA (Fig. 25); The FEA result describes the difference in amplitude and polarity of the potential. Using four different operation-induced signals, four command signals for controlling the machine can be generated (Fig. 6) .

Example  10. Electricity Stimulator  conductivity GP Heterostructure

After the robot executes the assigned command, the feedback information can be transmitted to the human operator through the electric stimulator forming the closed-loop composed of the iHMI and the sensor. In addition, the electrical stimulator must be transparent and stretchable, thereby aesthetically pleasing, and is also conformally integrated into the human skin for effective charge injection. 26 shows a schematic view of the stimulator manufacturing method. The above process starts with deposition of a sacrificial layer (Cr / Ni, 7 nm / 70 nm) patterned with a transparent epoxy (SU8, Microchem, USA). The GP / AgNW / GP multilayer is then transferred onto the epoxy layer, patterned and covered in an additional epoxy. Next, the whole device is transferred-printed onto a PDMS / polyvinyl alcohol (PVA) substrate. Finally, the electrical stimulator is delivered to the skin by dissolving the PVA layer in water.

Pig skin is used to know the charge injection characteristics of the electric stimulator. Electric stimulator and recording electrodes are arranged at the upper and lower sides of the pig skin piece, respectively, at an equal angle (Fig. 27). Figure 28 shows the experimental setting and the distance between the electrical stimulator and the recording electrode. The distance between the electrodes increases in the following order: (1) - (2) <(1) - (3) <(1) - (4). The amplitude of the recorded voltage between the two electrodes decreases with decreasing distance (Figure 29). Further, as the skin thickness increases, the write voltage decreases (FIG. 30). A stimulus voltage greater than the threshold that can be perceived must be applied; The voltage depends on the thickness of the skin. Figure 31 shows that the sensing voltage is a plot as a function of the stimulus frequency and the minimum sensing voltage is inversely proportional to the frequency. The spatial distribution of the mechanoreceptors that can be successfully stimulated can be estimated using FEA (Fig. 32). Even after the stimulator is elongated to greater than about 30% (Figure 33), stable charge injection is shown (Figure 34). 35 shows the strain distribution on the stimulator when it is stretched.

Example  11. Closed loop iHMI  Demonstration of the system

The iHMI system of the present invention is used to mutually control the robot arm (AX-18A, ROBOTIS, Korea). Figures 36a and 36b show a flow chart for the iHMI demonstration, respectively, and a corresponding experimental apparatus. Movements such as relaxation, bending and pressing of the human arm are detected by the motion sensor. The generated electrical signals are transmitted to a computer through a DAQ board (DAQ board) (NI USB-6289, National Instruments, USA). The DAQ processes data using a specially written software program (LabVIEW, National Instruments, USA). The program analyzes the signal obtained from the sensor, identifies the type of operation, and delivers appropriate commands to the robotic arm. When the pressure sensor mounted on the forceps senses that the robot arm picks up an object, the stimulator stacked on the user's cuff gives the user a feedback electric stimulus as a warning. Differential operational amplifiers with unity gains can be used to reduce output impedance to prevent crosstalk (Fig. 36B, top left inset); It is assumed that the output impedance of the motion sensor is theoretically infinite. The transparent motion sensor (FIG. 37A, blue dotted line box) and electric stimulator (FIG. 37A, green dotted line box) were attached to the upper end of the wrist and the side surface of the wrist, respectively. The bending, pressing and relaxing motion of the wrist allows the motion sensor to generate a specific type of electrical signal (Figs. 37A to 37D, upper right inset); These signals are transmitted to the computer (Figs. 37B-37D, blue arrows). Correspondingly, the pre-designated commands in the software cause the robotic arm to bend, pick and lift (Figs. 37B to 37D, bottom).

Figure 38 shows representative data collected from the motion sensor (first row), pressure sensor (second row), stimulator (third row), and forceps (fourth row). The position of the robot arm is recorded at the same time as the demonstration. 38, first line, (G)), the intensity of the generated peak is higher than the peak corresponding to the bending operation (Fig. 38, first line, (B)). If the intensity is above a certain threshold, the program sends an instruction to the gripper to perform a picking operation. That is, the robot arm picks up the target object. (Fig. 38, second line, (G)), the electric stimulus (Fig. 38, third line, (G)) is applied to the user and the tongue stops moving , Thus preventing the object from being overpressurized (Figure 38, line 4, (G)). The enlarged picture of the stimulus signal (FIG. 38, third line, upper right inset) shows that the stimulus signal is a high frequency square. FIG. 39 shows sequential captured snapshots in the recorded video for the iHMI system in use. The color acronyms in Figs. 37 to 38 are the same as those in Fig. The successful demonstration of the iHMI system demonstrates that the fabricated transparent motion sensor and stimulator can be used for interactive robot control with high reproducibility and accuracy.

Claims (26)

A transparent and stretchable motion sensor attached to the skin of the subject;
A data processing unit for receiving a signal regarding an operation of an object obtained from the motion sensor and transmitting the signal to a machine to be executed; And
And a transparent, stretchable electrical stimulator attached to the skin of the subject to receive a signal relating to the information detected by the machine,
Interactive human - machine interface system. The method of claim 1, wherein the transparent and stretchable motion sensor comprises: a first protective layer; A first electrode layer formed adjacent to the first passivation layer; A piezoelectric layer formed adjacent to the first electrode layer; A second electrode layer formed adjacent to the piezoelectric layer; And a second protective layer formed adjacent to the second electrode layer. 3. The interactive human-machine interface system of claim 2, wherein the first passivation layer is polymethyl methacrylate (PMMA). 3. The interactive human-machine interface system of claim 2, wherein the thickness of the first passivation layer is 50 nm to 100 占 퐉. 3. The interactive human-machine interface system of claim 2, wherein the first electrode layer is graphene. 6. The interactive human-machine interface system of claim 5, wherein the graphene is graphene doped with gold. 3. The interactive human-machine interface system according to claim 2, wherein the thickness of the first electrode layer is 1 nm to 100 nm. 3. The interactive human-machine interface system according to claim 2, wherein the piezoelectric layer is a polylactic acid / single-walled carbon nanotube composite film. 9. The interactive human-machine interface system of claim 8, wherein the single-walled carbon nanotubes have a length of 50 nm to 1 mm. 3. The interactive human-machine interface system according to claim 2, wherein the thickness of the piezoelectric layer is 100 nm to 500 m. 3. The interactive human-machine interface system of claim 2, wherein the second electrode layer is graphene. 12. The interactive human-machine interface system of claim 11, wherein the graphene is graphene doped with gold. 3. The interactive human-machine interface system according to claim 2, wherein the thickness of the second electrode layer is 1 nm to 100 nm. 3. The interactive human-machine interface system of claim 2, wherein the second passivation layer is polymethyl methacrylate (PMMA). The interactive human-machine interface system of claim 2, wherein the thickness of the second passivation layer is 50 nm to 100 μm. The interactive human-machine interface system according to claim 2, wherein the first protective layer, the first electrode layer, the piezoelectric layer, the second electrode layer, and the second protective layer are patterned in a serpentine pattern. 2. The device of claim 1, wherein the transparent, stretchable electrical stimulator is a patterned first protective layer; A patterned electrode layer formed adjacent to the patterned first passivation layer; A patterned second protective layer formed adjacent to the patterned electrode layer; And an adhesion layer formed adjacent to the patterned second protective layer. &Lt; Desc / Clms Page number 21 &gt; 18. The interactive human-machine interface system of claim 17, wherein the patterned first passivation layer is an epoxy resin. 18. The interactive human-machine interface system of claim 17, wherein the thickness of the patterned first passivation layer is between 1 [mu] m and 100 [mu] m. 18. The interactive human-machine interface system of claim 17, wherein the patterned electrode layer is graphene. 18. The interactive human-machine interface system of claim 17, wherein the patterned electrode layer is a graphene / silver nanowire / graphene multilayer. 18. The interactive human-machine interface system of claim 17, wherein the thickness of the patterned conductive layer is between 1 nm and 100 nm. 18. The interactive human-machine interface system of claim 17, wherein the patterned second protective layer is an epoxy resin. The interactive human-machine interface system of claim 17, wherein the thickness of the patterned second passivation layer is 100 nm to 100 μm. 18. The interactive human-machine interface system of claim 17, wherein the adhesive layer is a polydimethylsiloxane. 18. The interactive human-machine interface system of claim 17, wherein the thickness of the adhesive layer is from 1 占 퐉 to 100 占 퐉.

Images