CN109920347B - Motion or expression simulation device and method based on magnetic liquid - Google Patents

Motion or expression simulation device and method based on magnetic liquid Download PDF

Info

Publication number
CN109920347B
CN109920347B CN201910162342.6A CN201910162342A CN109920347B CN 109920347 B CN109920347 B CN 109920347B CN 201910162342 A CN201910162342 A CN 201910162342A CN 109920347 B CN109920347 B CN 109920347B
Authority
CN
China
Prior art keywords
magnetic liquid
demonstration
expression
demonstration area
upper computer
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
CN201910162342.6A
Other languages
Chinese (zh)
Other versions
CN109920347A (en
Inventor
谢磊
张永浩
邓嘉阳
罗萱
李兵
王聿浩
杨涛
李佩
廖昌荣
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Chongqing University
Original Assignee
Chongqing University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Chongqing University filed Critical Chongqing University
Priority to CN201910162342.6A priority Critical patent/CN109920347B/en
Publication of CN109920347A publication Critical patent/CN109920347A/en
Application granted granted Critical
Publication of CN109920347B publication Critical patent/CN109920347B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Landscapes

  • Magnetic Treatment Devices (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a motion or expression simulation device and method based on magnetic liquid, wherein the simulation device comprises a camera for shooting human motion or expression; the magnetic liquid demonstration instrument comprises a main demonstration area and a plurality of outer demonstration areas which are uniformly distributed around the main demonstration area along the circumferential direction; the upper computer is in signal connection with the camera and the controller of the magnetic liquid demonstration instrument; the simulation method comprises the following steps: s1. shooting human body motion or expression images through the camera and transmitting the images to the upper computer; s2, the upper computer identifies the shot action or expression image, codes the identified action or expression and transmits the codes to a controller of the magnetic liquid demonstration instrument; s3. the controller of the demonstration instrument generates corresponding control signals according to the codes, so that the magnetic liquid in the main demonstration area and the external demonstration area generates fluctuation changes representing the actions or expressions corresponding to the codes.

Description

Motion or expression simulation device and method based on magnetic liquid
Technical Field
The invention relates to the field of demonstration instruments, in particular to a device and a method for simulating actions or expressions based on magnetic liquid.
Background
Artificial intelligence is a branch of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a similar manner to human intelligence, a field of research that includes robotics, language recognition, image recognition, natural language processing, and expert systems, among others. With the development of intelligent machines and artificial intelligence, computer emotion analysis has become more and more important. The human face expression recognition is used as a perception means of emotional information communication, and has wide application potential in the fields of teaching cognitive state analysis and the like. At present, expression and action recognition methods based on deep learning are mature day by day, and a high recognition rate can be realized.
Magnetic liquid is a new type of functional material, which is usually a colloidal solution consisting of nanoscale magnetic particles, additives and carrier liquid. When a magnetic field is applied, the magnetic liquid is quickly magnetized, a phase change from liquid to solid occurs, and after the magnetic field is removed, the magnetic liquid is restored to the original state (namely, liquid). The magnetic liquid forms a 'magnetic hillock' (also called a 'mountain-shaped structure') of a solid-like conical structure on a macroscopic scale. Because of its magnetic phase transition property, magnetic liquid has wide application in practice and theoretically has high academic value. The magnetic fluid produced by the nano metal and alloy powder has excellent performance, and can be widely applied to the fields of magnetic fluid sealing, shock absorption, medical instruments, sound regulation, optical display, magnetic fluid mineral separation and the like under various severe conditions.
The method identifies the action and the expression through an artificial intelligence algorithm, and expresses the identified action or expression through fluctuation of the magnetic liquid, so that the method is suitable for popular science demonstration and high-end entertainment devices. The method is based on the image recognition algorithm and the intelligent material technology of deep learning, realizes real-time interaction between the popular science device and the user, is beneficial to the user to intuitively feel and understand the two advanced technologies, has good popular science demonstration effect, and is beneficial to improving the scientific literacy of teenagers and common people in the field. In addition, the fluctuation rhythm of the magnetic liquid under the control of the magnetic field has unique shape, has both ornamental and entertainment properties, and can be used for high-end entertainment.
Disclosure of Invention
In view of the above, the present invention is directed to an interactive demonstration of a magnetic liquid demonstration apparatus by using a facial recognition technology, and different demonstration effects are performed by recognizing different human body actions or expressions corresponding to the magnetic liquid demonstration apparatus.
The invention relates to a magnetic liquid-based action or expression simulation device, which comprises:
the camera is used for shooting human body actions or expressions;
the magnetic liquid demonstration instrument comprises a main demonstration area and a plurality of outer demonstration areas which are uniformly distributed around the main demonstration area along the circumferential direction;
and the upper computer is in signal connection with the camera and the controller of the magnetic liquid demonstration instrument.
Further, the fluctuation height of the magnetic liquid in the main demonstration area is controllable.
The invention also discloses a method for simulating actions or expressions by using the device, which comprises the following steps:
s1. shooting human body motion or expression images through the camera and transmitting the images to the upper computer;
s2, the upper computer identifies the shot action or expression image, codes the identified action or expression and transmits the codes to a controller of the magnetic liquid demonstration instrument;
s3. the controller of the demonstration instrument generates corresponding control signals according to the codes, so that the magnetic liquid in the main demonstration area and the external demonstration area generates fluctuation changes representing the actions or expressions corresponding to the codes.
Further, when the motion identified by the upper computer is head shaking, the magnetic liquid in the main demonstration area fluctuates, and meanwhile, the magnetic liquid in each external demonstration area fluctuates in a reciprocating manner around the main demonstration area;
when the action identified by the upper computer is nodding, the magnetic liquid in the main demonstration area fluctuates, and meanwhile, the magnetic liquid in each external demonstration area fluctuates synchronously;
when the motion identified by the upper computer is blinking, the magnetic liquid in the main demonstration area and the magnetic liquid in each external demonstration area synchronously fluctuate.
Further, when the expression identified by the upper computer is happy, the magnetic liquid in the main demonstration area rapidly fluctuates, and the magnetic liquid in each external demonstration area randomly fluctuates;
when the expression identified by the upper computer is angry, the magnetic liquid in the main demonstration area and the magnetic liquid in each external demonstration area synchronously and quickly rise, and synchronously and quickly fall down after a set time period is maintained;
when the expression identified by the upper computer is grief, the magnetic liquid in the main demonstration area and the magnetic liquid in each external demonstration area synchronously and slowly fluctuate;
when the expression that the host computer discerned is fear, the magnetic fluid in main demonstration district rises to setting for the high back quick shake of height, and the magnetic fluid of each outside demonstration district is whole to be risen and keep simultaneously.
Further, when the motion identified by the upper computer is blinking, the frequency of the synchronous fluctuation of the magnetic liquid in the main demonstration area and the external demonstration area is positively correlated with the frequency of the blinking motion.
Further, the fluctuation height of the magnetic liquid in the main demonstration area is controlled by the PWM current waveform duty ratio of the electromagnetic coil passing through the main demonstration area.
Further, one or more human body actions or expressions can be preset; and when the upper computer identifies the set action or expression, the code corresponding to the action or expression is transmitted to the controller of the magnetic liquid demonstration instrument.
And further, the upper computer performs recognition once every set time interval, the expression or action with the largest occurrence frequency in the recognition results is taken as a final recognition result, and the final result code is transmitted to the controller of the magnetic liquid demonstration instrument.
The invention has the beneficial effects that: the action or expression simulating device based on the magnetic liquid is characterized in that a small fluctuation area is added around a demonstration platform on the basis of the existing magnetic liquid demonstration instrument, the original demonstration area with a larger area is a main demonstration area and is positioned in the center of the demonstration platform, a plurality of small magnetic liquid fluctuation areas which are uniformly distributed are arranged around the main demonstration area and are called as external demonstration areas, and different actions or expressions of a human body can be vividly simulated by combining the actions of the magnetic liquid in different main demonstration areas and external demonstration areas, so that the interestingness of the magnetic liquid demonstration instrument is greatly improved.
Drawings
The invention is further described below with reference to the following figures and examples:
FIG. 1 is a schematic view of a presentation area of a presentation instrument of the present invention;
FIG. 2 is a schematic diagram of the fluctuation of the magnetic liquid according to the "head shaking motion" in the present invention;
FIG. 3 is a schematic view showing the fluctuation of the magnetic liquid according to the "nodding action" in the present invention;
FIG. 4 is a schematic diagram of the fluctuation of the magnetic liquid according to the blinking motion of the present invention;
FIG. 5 is a schematic diagram of the fluctuation of the magnetic liquid according to the present invention;
FIG. 6 is a diagram showing the fluctuation of the magnetic liquid according to the "angry expression" in the present invention;
FIG. 7 is a schematic diagram of the fluctuation of the magnetic liquid corresponding to the expression of sadness in the present invention;
fig. 8 is a schematic view of the fluctuation of the magnetic liquid according to the "fear expression" in the present invention.
Detailed Description
The first embodiment is as follows:
the simulation apparatus of human body motion or expression of the present embodiment includes:
the camera is used for shooting the expression and the action of a user;
magnetic fluid demonstrates appearance, including main demonstration district and a plurality of centers on main demonstration district is along the outside demonstration district of hoop evenly distributed, and wherein the area in main demonstration district is great, is located demonstration platform central point, and outside demonstration district sets up the several and is central symmetry distribution around main demonstration district. The main demonstration area can control the fluctuation height of the magnetic liquid, and the outer demonstration area can only realize the control of two states of falling and rising. The detailed construction of the magnetic liquid demonstration instrument is the prior art and is not described in detail herein.
And the upper computer processes the images shot by the camera by using an image processing algorithm based on deep learning, identifies expressions and actions and transmits the codes of the expressions and the actions to the controller of the demonstration instrument.
Example two:
the method for simulating the human body action or expression comprises the following steps:
s1. shooting the action and expression of the person by the camera and transmitting the shot image to the upper computer;
and S2, the upper computer processes the image by using an image recognition algorithm based on deep learning, the demonstration instrument has two modes of motion simulation or expression simulation, a user can independently select an expression recognition mode or a motion recognition mode, the expression recognition mode can recognize four expressions of 'happiness, anger, grief and fear', and the motion recognition mode can recognize three actions of 'shaking head, nodding head and blinking'. The image recognition algorithm processes the shot image in real time, the recognition result can be output once every 0.25 second, the four results are taken as a group, the expression or action with the largest occurrence frequency in the group is taken as the final recognition result, the final result is coded and transmitted to the control part of the magnetic liquid demonstration instrument, namely the upper computer outputs the code to the controller of the demonstration instrument once every second.
s3. the controller of the demonstration instrument of magnetic liquid generates corresponding control signal after detecting the motion or expression code. A schematic view of the presentation area of the presentation instrument is shown in fig. 1. The main demonstration area is controlled by a larger electromagnetic coil, the magnetic field intensity of the electromagnetic coil is controlled through the PWM current waveform duty ratio, and the magnetic field intensity controls the rising height of the magnetic liquid. The larger the PWM wave duty ratio is, the larger the magnetic field intensity of the corresponding electromagnetic coil is, the higher the rising height of the corresponding magnetic liquid is, the smaller the PWM wave duty ratio is, the smaller the magnetic field intensity of the corresponding electromagnetic coil is, and the lower the rising height of the corresponding magnetic liquid is. The number of the external demonstration areas is 8, the external demonstration areas are controlled by 8 small electromagnetic coils which are arranged below the liquid carrying disc in a one-to-one correspondence mode, when direct current voltage is applied to the electromagnetic coils, the magnetic liquid rises, and when no voltage is applied, the magnetic liquid falls.
s4. the demonstration table part of the demonstration instrument for magnetic liquid generates corresponding fluctuation of magnetic liquid synchronously after receiving the control signal.
Taking the "head shaking motion" as an example, the schematic diagram of the fluctuation form of the magnetic liquid corresponding to the "head shaking motion" is shown in fig. 2, the magnetic liquid in the main demonstration area fluctuates up and down, and the outer demonstration area fluctuates in a reciprocating manner in the circumferential direction.
The second embodiment:
in this embodiment, the method for simulating human body actions or expressions by using magnetic liquid includes the following steps:
s1. shooting the action and expression of the person by the camera and transmitting the shot image to the upper computer;
and S2, processing the image by the upper computer by using an image recognition algorithm based on deep learning. The user needs to select the expression or action to be recognized in advance, including four expressions of 'happy, angry, saddley, fear', and three actions of 'shaking head, nodding head, blinking', after selecting a certain expression or action, if the expression or action is detected, the output is yes, and if the expression or action is not detected, the output is no. The image recognition algorithm processes the shot image in real time, the recognition result can be output once every 0.25 second, the four results are taken as a group, the result with the largest occurrence frequency in the group is taken as the final recognition result, the final result is coded and transmitted to the control part of the magnetic liquid demonstration instrument, namely the upper computer outputs the code to the control part of the demonstration instrument once every second.
s3. the control part of the demonstration instrument generates corresponding control signals after detecting the motion or expression codes. A schematic view of the presentation area of the presentation instrument is shown in fig. 1. The main demonstration area is controlled by a larger electromagnetic coil, the magnetic field intensity of the electromagnetic coil is controlled through the PWM current waveform duty ratio, and the magnetic field intensity controls the rising height of the magnetic liquid. The larger the PWM wave duty ratio is, the larger the magnetic field intensity of the corresponding electromagnetic coil is, the higher the rising height of the corresponding magnetic liquid is, the smaller the PWM wave duty ratio is, the smaller the magnetic field intensity of the corresponding electromagnetic coil is, and the lower the rising height of the corresponding magnetic liquid is. The external demonstration area is controlled by eight small permanent magnets below the liquid carrying disc, when the magnets are driven by the motor to be close to the liquid carrying disc, the magnetic liquid rises, and when the magnets are far away from the liquid carrying disc, the magnetic liquid falls.
s4. the demonstration table part of the demonstration instrument for magnetic liquid generates corresponding fluctuation of magnetic liquid synchronously after receiving the control signal.
Taking the "nodding action" as an example, a schematic diagram of the fluctuation form of the magnetic liquid corresponding to the "nodding action" is shown in fig. 3, the magnetic liquid in the main demonstration area fluctuates up and down, and all the external demonstration areas fluctuate synchronously.
Finally, the above embodiments are only for illustrating the technical solutions of the present invention and not for limiting, although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications or equivalent substitutions may be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all of them should be covered in the claims of the present invention.

Claims (6)

1. A method for simulating by using a magnetic liquid-based motion or expression simulator, the method comprising: the magnetic liquid-based motion or expression simulation device comprises:
the camera is used for shooting human body actions or expressions;
the magnetic liquid demonstration instrument comprises a main demonstration area and a plurality of outer demonstration areas which are uniformly distributed around the main demonstration area along the circumferential direction;
the upper computer is in signal connection with the camera and the controller of the magnetic liquid demonstration instrument;
the main demonstration area is controlled by an electromagnetic coil, the magnetic field intensity of the electromagnetic coil is controlled through the PWM current waveform duty ratio, and the magnetic field intensity controls the rising height of the magnetic liquid; the number of the external demonstration areas is 8, and the external demonstration areas are controlled by 8 electromagnetic coils which are positioned below the liquid carrying discs in a one-to-one correspondence manner;
the method comprises the following steps:
s1. shooting human body motion or expression images through the camera and transmitting the images to the upper computer;
s2, the upper computer identifies the shot action or expression image, codes the identified action or expression and transmits the codes to a controller of the magnetic liquid demonstration instrument;
s3. the controller of the demonstration instrument generates corresponding control signals according to the codes, so that the magnetic liquid in the main demonstration area and the external demonstration area generates fluctuation changes representing the actions or expressions corresponding to the codes;
when the motion identified by the upper computer is head shaking, the magnetic liquid in the main demonstration area fluctuates, and meanwhile, the magnetic liquid in each external demonstration area fluctuates in a reciprocating manner around the main demonstration area;
when the action identified by the upper computer is nodding, the magnetic liquid in the main demonstration area fluctuates, and meanwhile, the magnetic liquid in each external demonstration area fluctuates synchronously;
when the motion identified by the upper computer is blinking, the magnetic liquid in the main demonstration area and the magnetic liquid in each external demonstration area synchronously fluctuate.
2. The method of claim 1, wherein: when the expression identified by the upper computer is pleased, the magnetic liquid in the main demonstration area rapidly fluctuates, and the magnetic liquid in each external demonstration area randomly fluctuates;
when the expression identified by the upper computer is angry, the magnetic liquid in the main demonstration area and the magnetic liquid in each external demonstration area synchronously and quickly rise, and synchronously and quickly fall down after a set time period is maintained;
when the expression identified by the upper computer is grief, the magnetic liquid in the main demonstration area and the magnetic liquid in each external demonstration area synchronously and slowly fluctuate;
when the expression that the host computer discerned is fear, the magnetic fluid in main demonstration district rises to setting for the high back quick shake of height, and the magnetic fluid of each outside demonstration district is whole to be risen and keep simultaneously.
3. The method of claim 2, wherein: when the motion identified by the upper computer is blinking, the frequency of the synchronous fluctuation of the magnetic liquid in the main demonstration area and the external demonstration area is positively correlated with the frequency of the blinking motion.
4. The method of claim 3, wherein: the fluctuation height of the magnetic liquid of the main demonstration area is controlled by the duty ratio of a PWM current waveform passing through the electromagnetic coil above the main demonstration area.
5. The method of claim 4, wherein: one or more human body actions or expressions can be preset; and when the upper computer identifies the set action or expression, the code corresponding to the action or expression is transmitted to the controller of the magnetic liquid demonstration instrument.
6. The method of claim 5, wherein: the upper computer performs recognition once every set time interval, the expression or the action with the largest occurrence frequency in the recognition results is taken as a final recognition result, and the final result code is transmitted to the controller of the magnetic liquid demonstration instrument.
CN201910162342.6A 2019-03-05 2019-03-05 Motion or expression simulation device and method based on magnetic liquid Expired - Fee Related CN109920347B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910162342.6A CN109920347B (en) 2019-03-05 2019-03-05 Motion or expression simulation device and method based on magnetic liquid

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910162342.6A CN109920347B (en) 2019-03-05 2019-03-05 Motion or expression simulation device and method based on magnetic liquid

Publications (2)

Publication Number Publication Date
CN109920347A CN109920347A (en) 2019-06-21
CN109920347B true CN109920347B (en) 2020-12-04

Family

ID=66963207

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910162342.6A Expired - Fee Related CN109920347B (en) 2019-03-05 2019-03-05 Motion or expression simulation device and method based on magnetic liquid

Country Status (1)

Country Link
CN (1) CN109920347B (en)

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN201638526U (en) * 2010-03-05 2010-11-17 上海未来伙伴机器人有限公司 Device for simulating human expressions and robot comprising the same
CN103339926A (en) * 2011-11-24 2013-10-02 株式会社Ntt都科摩 Expression output device and expression output method
CN106737745A (en) * 2017-01-05 2017-05-31 珠海市横琴麒翔科技有限公司 Intelligent robot
CN106919899A (en) * 2017-01-18 2017-07-04 北京光年无限科技有限公司 The method and system for imitating human face expression output based on intelligent robot
CN107316546A (en) * 2017-08-03 2017-11-03 重庆大学 Magnetic liquid demonstrator and its control method
CN108326878A (en) * 2017-01-18 2018-07-27 王怀亮 A kind of limb action electronic switching equipment, instruction identification method and recording/playback method
CN208358753U (en) * 2018-04-26 2019-01-11 昆山塔米机器人有限公司 A kind of apery expression display device and robot
CN109345634A (en) * 2018-10-26 2019-02-15 重庆大学 Magnetic field demenstration method and device based on magnetic liquid and mixed reality

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN201638526U (en) * 2010-03-05 2010-11-17 上海未来伙伴机器人有限公司 Device for simulating human expressions and robot comprising the same
CN103339926A (en) * 2011-11-24 2013-10-02 株式会社Ntt都科摩 Expression output device and expression output method
CN106737745A (en) * 2017-01-05 2017-05-31 珠海市横琴麒翔科技有限公司 Intelligent robot
CN106919899A (en) * 2017-01-18 2017-07-04 北京光年无限科技有限公司 The method and system for imitating human face expression output based on intelligent robot
CN108326878A (en) * 2017-01-18 2018-07-27 王怀亮 A kind of limb action electronic switching equipment, instruction identification method and recording/playback method
CN107316546A (en) * 2017-08-03 2017-11-03 重庆大学 Magnetic liquid demonstrator and its control method
CN208358753U (en) * 2018-04-26 2019-01-11 昆山塔米机器人有限公司 A kind of apery expression display device and robot
CN109345634A (en) * 2018-10-26 2019-02-15 重庆大学 Magnetic field demenstration method and device based on magnetic liquid and mixed reality

Also Published As

Publication number Publication date
CN109920347A (en) 2019-06-21

Similar Documents

Publication Publication Date Title
CN104541306B (en) Neurobehavioral animation system
Camurri et al. An architecture for emotional agents
JP2022524944A (en) Interaction methods, devices, electronic devices and storage media
US10846520B2 (en) Simulated sandtray system
CN107030691A (en) A kind of data processing method and device for nursing robot
KR101799980B1 (en) Apparatus, system and method for controlling virtual reality image and simulator
CN107544675A (en) Brain control formula virtual reality method
Takács et al. The virtual human interface: A photorealistic digital human
CN109920347B (en) Motion or expression simulation device and method based on magnetic liquid
Jost et al. PRIM Project: Playing and Recording with Interactivity and Multisensoriality
US20220387885A1 (en) Dynamic modification of multiple haptic effects
Ciciliani Towards an Aesthetic of Electronic-Music Performance Practice
CN105303909B (en) English learning method, device and system based on vibration
CN111984161A (en) Control method and device of intelligent robot
Kaul et al. Design and evaluation of on-the-head spatial tactile patterns
Matthias et al. Adaptive generation of emotional impact using enhanced virtual environments
Casasanto et al. Virtual reality
Herrero et al. A human based perception model for cooperative intelligent virtual agents
KR102267221B1 (en) Ear-mounted vibrational wearable apparatus and method for controlling motion thereof
CN108710325B (en) Micro expression generation device and method
KR20190059353A (en) System for generating virtual and augmented reality images for experience using diorama and game method using the system
Ye Immersive VR IS Based on Virtual Reality Technology
Chi et al. A collaborative ensemble system with picture story for aesthetic sensibility learning
CN118664618A (en) Robot simulated eyeball control method, device, equipment and storage medium
Kuo et al. An emotional robot with facial expressions

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
CF01 Termination of patent right due to non-payment of annual fee
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20201204