CN109960410B

CN109960410B - Wall surface gesture interaction system and method based on electroluminescent material

Info

Publication number: CN109960410B
Application number: CN201910205641.3A
Authority: CN
Inventors: 孙晅; 崔鹏; 杜国铭
Original assignee: Harbin Tuobo Technology Co ltd
Current assignee: Harbin Tuobo Technology Co ltd
Priority date: 2019-03-19
Filing date: 2019-03-19
Publication date: 2020-05-22
Anticipated expiration: 2039-03-19
Also published as: CN109960410A

Abstract

The invention provides a wall surface gesture interaction system and method based on electroluminescent materials. In hardware, the system is built based on an embedded platform, the electromagnetic induction sensor is used as a gesture capture device, the display module adopts a light-emitting device based on an electroluminescent material as display equipment, and the wall hardware installation environment and the selected functions are adapted through customizing attributes such as the shape, color, size, brightness and the like of the electroluminescent material device, and the system can be completely integrated into a wall. The wall surface gesture interaction system based on the electroluminescent material can be widely applied to indoor application scenes such as homes, markets, museums and the like, and provides natural and convenient interaction experience for users.

Description

Wall surface gesture interaction system and method based on electroluminescent material

Technical Field

The invention belongs to the technical field of gesture interaction, and particularly relates to a wall surface gesture interaction system and method based on electroluminescent materials.

Background

Gesture control is one of the important research directions in the field of human-computer interaction. With the continuous development of the technology in recent years, the design gravity center of the gesture interaction system gradually shifts to the user experience from the technical application, and natural and convenient interaction experience is provided for the user. In a plurality of application scenes of gesture interaction, the wall gesture interaction can adapt to various building layouts, the building space is fully utilized, a good foundation is provided for user operation, and a large development space is still left in application.

Electroluminescent (EL) materials, also commonly referred to as luminescent or EL materials, emit soft light when current is passed through or in a strong electric field, and frequent turning on and off of the power does not affect their useful life. Common electroluminescent materials include copper and silver doped zinc sulfide (ZnS), gallium arsenide (GaAs), and the like, which are widely used in display devices. In recent years, the interest of electroluminescent material-based coatings has been increased, and the production and application of electroluminescent materials have been further promoted.

In the existing wall surface gesture interaction system, a part of the system imitates a traditional switch and only has simple visual feedback; the system is partially dependent on equipment such as a projector, a display and the like, and has certain requirements on indoor space and layout. In contrast, the electroluminescent material can present rich visual information under the condition of basically keeping indoor layout and complete wall body, and provides new possibility for wall surface gesture interaction. And the background and the luminous color of the electroluminescent material are various and adjustable, and the electroluminescent material is easy to match with indoor decoration.

In the prior art, the traditional wall switch does not have visual interaction, and each button has a single function. The gesture control equipment based on the electromagnetic induction is similar to a traditional wall switch in functions and appearance, only has basic visual feedback, and does not effectively improve the interaction experience of users. Some multifunctional wall consoles adopt buttons or touch panels for interaction, and usually protrude out of the wall body during installation, and the volume and the area are increased along with the increase of functions, so that the appearance style is limited. The gesture interaction equipment based on infrared induction needs to keep a smooth light path between the infrared transmitter and the receiver, and the surface of the gesture interaction equipment needs to be made of light-transmitting materials, so that the appearance style is also limited to a certain extent, and direct sunlight is avoided during use. Gesture interaction schemes based on display devices such as projectors and displays have certain requirements on indoor space and layout, and the cost may be high. In addition, some technical schemes such as luminous wall surface and ground are provided, so that the multifunctional wall surface and ground are mainly used for indoor decoration and do not have a gesture interaction function.

Disclosure of Invention

The invention aims to solve the technical problems in the prior art and provides a wall surface gesture interaction system and method based on electroluminescent materials.

The invention is realized by the following technical scheme, and provides a wall surface gesture interaction system based on an electroluminescent material, which comprises a main control module, a gesture capturing module, a display module and a communication module; the main control module is respectively connected with the gesture capturing module, the display module and the communication module;

the gesture capturing module adopts a sensor array formed by one or more sensors based on electromagnetic induction as a capturing device, the sensor is provided with a transmitting electrode and a plurality of receiving electrodes, an electric field is generated between each electrode after the sensor is electrified, when the hand of a user enters an induction range, the original electric field distribution is changed, the change is reflected as the signal intensity change of the total electrodes of the sensor and the signal intensity difference between each receiving electrode, and the hand state of the user can be obtained by analyzing the signal characteristics of the electrodes;

the main control module is a control and processing core of the system, controls the operation of other modules and monitors the working state of the modules in real time, processes and identifies signal data output by the gesture capture module when the system operates to obtain a user gesture command, and interrupts the operation in time and sends out an error prompt when the module operates abnormally;

the display module is used for realizing system display output and providing visual interaction for a user, adopts a light-emitting device based on an electroluminescent material as display equipment, adapts to the installation environment and selected functions of wall hardware by customizing the shape, color, size and brightness of the light-emitting device of the electroluminescent material, and can be completely integrated into a wall body;

the communication module is used for carrying out data transmission with external equipment and sending a control instruction corresponding to a user gesture command.

Further, the display device comprises a display unit or a display array of a plurality of display units, the display units having two states of "bright" and "dark".

Furthermore, the whole display space of the display device is uniformly distributed with electroluminescent materials, a liquid crystal panel covers the electroluminescent materials, and a transparent protective coating is attached to the liquid crystal panel.

Further, a system coordinate system is established before the gesture of the user is captured, o is a coordinate origin, coordinate axes ox and oy are in a plane of the wall surface, the oy is parallel to a plumb line of the wall surface, the direction pointing to the ground is a positive direction, ox is perpendicular to the oy, the right is the positive direction, oz is perpendicular to the wall surface, and the outward direction is the positive direction.

Further, the main control module processes and identifies signal data output by the gesture capture module to obtain a user gesture command, specifically:

step a, reading the signal intensity of each receiving electrode in each sensor;

step b, when only one sensor is available, executing step c, otherwise executing step e;

c, measuring and calculating a z coordinate of the hand according to the signal intensity of the total electrode of the sensor;

d, measuring and calculating x and y coordinates of the hand according to the signal intensity difference of each electrode of the sensor, and executing the step i;

e, dividing the sensor array into a plurality of blocks, so that each sensor contained in each block has similar overall electrode signal intensity, and each block corresponds to a hand target;

f, measuring and calculating the hand size according to the block size;

step g, measuring and calculating a z coordinate of the hand according to the average total electrode signal intensity of the blocks;

h, measuring and calculating x and y coordinates of the hand according to the signal intensity difference of the total electrodes of the sensors in the block;

step i, measuring and calculating the time sequence change of the electrode signal intensity according to the preorder captured data;

j, performing fusion analysis on each measurement and calculation result obtained in the steps a to i to obtain a user gesture command;

step k, outputting a user gesture command;

and step l, ending.

Further, when the display element is one, the display element includes location sign and function sign, the location sign adopts fluorescent material to make, the function sign is drawn by electroluminescent material and forms, the function sign is the same with fluorescent material colour under the on-state, and is the same with wall coating colour under the off-state.

Further, when the display device is a display array formed by a plurality of display units, each display unit can be independently controlled, the system comprises a plurality of function modes, a state mark drawn by electroluminescent materials is arranged below the display array and used for prompting the current function mode, and the color of the electroluminescent materials is the same as that of wall paint in a power-off state and is integrated with a wall.

Further, each sensor corresponds to 9 display units.

The invention also provides a wall surface gesture interaction method based on the electroluminescent material, which specifically comprises the following steps:

step 1, a gesture capturing module captures a user gesture;

step 2, the main control module processes and analyzes the captured gesture data to obtain a user gesture command;

step 3, the display module provides visual feedback for the user according to the user gesture command;

and 4, the communication module sends a corresponding control instruction according to the user gesture command.

Compared with the prior art, the invention has the following advantages:

1. the system has the gesture recognition function, can recognize the gesture action of the user and execute the corresponding function, and is convenient and fast to use;

2. the system is provided with a display module, so that rich visual interaction experience can be provided for a user;

3. the light-emitting device based on the electroluminescent material is adopted as the display equipment, and the shape, the size, the color, the light-emitting brightness and other attributes can be customized;

4. the system can be completely hidden in the wall body, and the appearance is concise and attractive;

5. the system has good adaptability to indoor space and layout and occupies less space;

6. the whole power consumption is low, and the energy is saved and the environment is protected;

7. the system has good expansibility and high customizable degree.

Drawings

FIG. 1 is a block diagram of an electroluminescent material based wall gesture interaction system according to the present invention;

FIG. 2 is a schematic diagram of a system coordinate system;

FIG. 3 is a flow chart of user gesture recognition;

FIG. 4 is a flowchart of a wall gesture interaction method based on electroluminescent material according to the present invention;

FIG. 5 is a schematic diagram of the hardware installation of example 1;

FIG. 6 is a schematic diagram of a patterned layer on the surface of a switch;

FIG. 7 is a diagram showing the operation of the switch;

FIG. 8 is a schematic diagram of a hardware installation of example 2;

FIG. 9 is a schematic diagram of a display array and status markers;

FIG. 10 is a diagram of the operating state of the interactive terminal;

FIG. 11 is a schematic view of a time display mode;

FIG. 12 is a schematic view of a lamp control mode;

FIG. 13 is a schematic view of an air conditioning control mode;

FIG. 14 is a schematic view of a display device based on a liquid crystal panel;

FIG. 15 is a schematic diagram of example 4 hardware installation;

FIG. 16 is a schematic diagram of an arrangement of sensors and display arrays;

FIG. 17 is a schematic view of video animation playback;

FIG. 18 is a schematic diagram of gesture interaction.

Detailed Description

The technical solutions in the embodiments of the present invention will be described clearly and completely with reference to the accompanying drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

With reference to fig. 1, the present invention provides a wall surface gesture interaction system based on electroluminescent material, which includes a main control module, a gesture capture module, a display module and a communication module; the main control module is respectively connected with the gesture capturing module, the display module and the communication module;

the gesture capturing module is used for capturing gesture actions such as approaching and waving of the hand of the user and the like and transmitting the gesture actions to the main control module for processing and analysis. The method comprises the steps that a sensor array consisting of one sensor or a plurality of sensors based on electromagnetic induction is adopted as a capturing device on hardware, the sensor is provided with a transmitting electrode and a plurality of receiving electrodes, an electric field is generated between the electrodes after the sensor is electrified, when the hand of a user enters an induction range, the original electric field distribution is changed, the change of the total electrode signal intensity of the sensor and the difference of the signal intensity between the receiving electrodes are reflected, and the hand state of the user can be obtained by analyzing the electrode signal characteristics; according to application requirements, the size of the sensor and the arrangement mode of the electrodes can be customized and adjusted. And the emission voltage and frequency of the sensor can be adjusted in real time in the using process, so that the capturing precision of the gesture action is further improved.

the communication module is used for carrying out data transmission with external equipment and sending a control instruction corresponding to a user gesture command. The communication module can be connected to an external device in a wired or wireless mode, and transmits data through a selected communication protocol. The communication protocol can adopt an industry general protocol and can be defined according to requirements.

The display equipment comprises a display array formed by one display unit or a plurality of display units, wherein the display unit has a bright state and a dark state, and the shape, the size, the color, the light-emitting brightness and other attributes of each display unit can be customized; or, the whole display space of the display device is uniformly distributed with electroluminescent materials, and meanwhile, a liquid crystal panel is covered above the electroluminescent materials, and a transparent protective coating is added on the liquid crystal panel. When the display element is one, the display element is including location sign and function sign, the location sign adopts fluorescent material to make, the function sign is formed by electroluminescent material drawing, the function sign is the same with fluorescent material colour under the on state, and is the same with wall coating colour under the off-state. When the display equipment is the display array that a plurality of display element are constituteed, every display element independent control, the system includes a plurality of functional mode, display array below has the state sign of drawing by electroluminescent material for the suggestion current functional mode, electroluminescent material colour is the same with wall coating under the outage state, fuses as an organic wholely with the wall.

Before capturing the gesture of the user, a system coordinate system is established, as shown in fig. 2, o is a coordinate origin, coordinate axes ox and oy are in a plane of the wall surface, the oy is parallel to a plumb line of the wall surface, the direction pointing to the ground is a forward direction, ox is perpendicular to the oy, the right is a forward direction, oz is perpendicular to the wall surface, and the outward direction is a forward direction.

With reference to fig. 3, the main control module processes and identifies the signal data output by the gesture capturing module to obtain a user gesture command, which specifically includes:

f, measuring and calculating the hand size according to the block size;

step k, outputting a user gesture command;

and step l, ending.

With reference to fig. 4, the present invention further provides a wall surface gesture interaction method based on an electroluminescent material, which specifically includes the following steps:

step 1, a gesture capturing module captures a user gesture;

Example 1 wall surface gesture control switch

In this embodiment, the wall gesture control switch is used for daily houses and can control the on-off and brightness adjustment of indoor lamps. The hardware installation mode is as shown in fig. 5, wherein the switch is wholly embedded in the wall body, the control circuit and the electromagnetic induction sensor are located at the innermost side, and the switch surface pattern layer covers the electromagnetic induction sensor and is covered by the transparent protective coating. The outer surface of the protective coating is flush with the outer surface of the wall coating.

The bottom color of the switch surface pattern layer is the same as the color of the wall paint, the positioning mark is made of fluorescent materials, and the functional mark is formed by drawing electroluminescent materials. The function mark has the same color as the fluorescent material in the power-on state (as shown in fig. 6 (a)), and has the same color as the ground color in the power-off state (as shown in fig. 6 (b)).

The switch operating state diagram is shown in fig. 7. During the use, under lamps and lanterns closed condition, the user is at first with the palm near switch location sign and wave, then lamps and lanterns open, and each function sign circular telegram is lighted. At this time, when the user brings the palm close to a certain function identifier, the corresponding function is executed. When the user does not operate for a long time, each function mark is powered off and is not responded to the gesture action of the user any more. At this time, the respective function indicators may be reactivated when the user brings the palm close to the positioning indicator again. When the lamp is in an open state, the lamp can be closed by continuously waving the palm near the positioning mark.

Example 2 wall gesture interaction terminal

In this embodiment, the wall gesture interaction terminal hardware installation manner is as shown in fig. 8, and the whole terminal is completely embedded in the wall. The display module employs a display array composed of a plurality of smaller sized electroluminescent material display cells, each of which is independently controllable as shown in fig. 9, so that more complex patterns can be displayed or animation, etc., can be played. The state mark drawn by electroluminescent material is arranged below the array and used for prompting the current function mode. The color of the electroluminescent material is the same as that of the wall coating in the power-off state, and the electroluminescent material is integrated with the wall.

Functionally, the wall gesture interaction terminal has three functional modes, namely a time display mode, a lamp control mode and an air conditioner control mode, wherein the lamp control mode comprises the functions of switching on and off of a lamp, adjusting the brightness and the like, and the air conditioner control mode comprises the functions of switching on and off of an air conditioner, adjusting the temperature and the like. The operation state of the interactive terminal is shown in fig. 10.

The default functional mode of the interactive terminal is a time display mode, as shown in fig. 11, at this time, the interactive terminal will display a clock with hour and minute hands. This mode is always in when the interactive terminal is idle. In the time display mode, when the hand of the user approaches, the interactive terminal is switched to the lamp control mode.

The lamp control mode display interface is shown in fig. 12, in which fig. 12(a) is an off state and fig. 12(b) is an on state. In the off state, when the user hovers the hand over the on-mark, the on-mark is switched to the on state, and the lamp is turned on. In the starting state, when the user hovers the hand over the adjusting mark, the brightness of the lamp can be adjusted; when the user hovers the hand over the closing mark, the hand is switched to the closing state, and the lamp is closed. Under the lamp control mode, when the user waves his hand to the left by a large margin, the control mode is switched to the air-conditioning control mode.

The air-conditioning control mode display interface is shown in fig. 13, where fig. 13(a) is an off state and fig. 13(b) is an on state, and in the on state, the currently set air-conditioning temperature will be displayed in real time, and the area displaying the temperature will also be used as an off identifier. In the closed state, when the user hovers the hand over the opening mark, the hand is switched to the open state, and the air conditioner is started. In the starting state, when a user hovers a hand over the adjusting mark, the preset temperature of the air conditioner can be adjusted; when the user hovers the hand over the closing mark, the hand is switched to the closing state, and the air conditioner is closed. Under the air-conditioning control mode, when the user waves the hand to the right by a large margin, the control mode is switched to the lamp control mode.

When the lamp is in the lamp control mode or the air conditioner control mode, if the user does not operate for a long time, the time display mode is automatically switched back.

Example 3 wall surface gesture interaction terminal based on liquid crystal panel

This embodiment is an extension of example 2, except that a combination of a liquid crystal panel and an electroluminescent material is used as a display device. As shown in fig. 14, the electroluminescent material is not arranged in an array, but is uniformly distributed in the whole display space, and simultaneously, the liquid crystal panel is covered above the electroluminescent material, and a transparent protective coating is further added on the liquid crystal panel. Except for this, other hardware installation methods and interactive terminal functions are the same as those of example 2.

Example 4 somatosensory interactive display wall

The present embodiment uses a plurality of electromagnetic induction sensors to form a sensor array and a plurality of electroluminescent material display cells to form a display array. The hardware installation mode is shown in fig. 15, and each sensor corresponds to 9 display units in spatial position, as shown in fig. 16. Wherein the dashed grid represents the display unit and the solid grid represents the sensor.

Functionally, the sensor array can not only detect hand targets, but also human proximity. According to the sensor array partition method, when a plurality of sensors detect a target at the same time and the signal intensity is small, the human body can be judged; when the single sensor detects the target and the signal strength is high, the hand can be judged. Therefore, when the approach of a human body is detected, a fixed video animation can be played, as shown in fig. 17; when a hand target is detected, interaction may occur, as shown in FIG. 18.

The wall surface gesture interaction system and method based on the electroluminescent material provided by the invention are described in detail, a specific example is applied in the text to explain the principle and the implementation mode of the invention, and the description of the embodiment is only used for helping to understand the method and the core idea of the invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.

Claims

1. The utility model provides a wall gesture interactive system based on electroluminescent material which characterized in that: the system comprises a main control module, a gesture capturing module, a display module and a communication module; the main control module is respectively connected with the gesture capturing module, the display module and the communication module;

the communication module is used for carrying out data transmission with external equipment and sending a control instruction corresponding to a user gesture command;

the whole display space of the display device is uniformly distributed with electroluminescent materials, a liquid crystal panel covers the electroluminescent materials, and a transparent protective coating is attached to the liquid crystal panel.

2. The system of claim 1, wherein the display device comprises a display unit or a display array of multiple display units, the display units having both "bright" and "dark" states.

3. The system of claim 1, wherein a system coordinate system is established prior to capturing the user gesture, o is an origin of coordinates, coordinate axes ox and oy are in a plane of the wall, oy is parallel to a vertical line of the wall and points in a positive direction to the ground, ox is perpendicular to oy and to the right is a positive direction, oz is perpendicular to the wall and outwards is a positive direction.

4. The system according to claim 3, wherein the main control module processes and recognizes the signal data output by the gesture capturing module to obtain a user gesture command, specifically:

f, measuring and calculating the hand size according to the block size;

step k, outputting a user gesture command;

and step l, ending.

5. The system according to claim 2, wherein when one display unit is provided, the display unit comprises a positioning mark and a function mark, the positioning mark is made of fluorescent material, the function mark is drawn by electroluminescent material, the color of the function mark is the same as that of the fluorescent material in the power-on state, and the color of the function mark is the same as that of the wall paint in the power-off state.

6. The system of claim 2, wherein each display unit is independently controllable when the display device is a display array of a plurality of display units, the system comprising a plurality of functional modes, and the display array having a status indicator drawn from the electroluminescent material below the display array for indicating the current functional mode, wherein the electroluminescent material has a color similar to that of the wall paint and blends with the wall paint in the power-off state.

7. The system of claim 2, wherein each sensor corresponds to 9 display units.

8. The interaction method of the wall surface gesture interaction system based on the electroluminescent material as claimed in any one of claims 1 to 7, characterized by comprising the following steps:

step 1, a gesture capturing module captures a user gesture;