CN108919950A - Autism children based on Kinect interact device for image and method - Google Patents
Autism children based on Kinect interact device for image and method Download PDFInfo
- Publication number
- CN108919950A CN108919950A CN201810667807.9A CN201810667807A CN108919950A CN 108919950 A CN108919950 A CN 108919950A CN 201810667807 A CN201810667807 A CN 201810667807A CN 108919950 A CN108919950 A CN 108919950A
- Authority
- CN
- China
- Prior art keywords
- children
- kinect
- infant
- training
- autism
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/20—Software design
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H15/00—ICT specially adapted for medical reports, e.g. generation or transmission thereof
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H20/00—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance
- G16H20/70—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance relating to mental therapies, e.g. psychological therapy or autogenous training
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- Epidemiology (AREA)
- Public Health (AREA)
- Software Systems (AREA)
- Medical Informatics (AREA)
- Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Primary Health Care (AREA)
- Child & Adolescent Psychology (AREA)
- Developmental Disabilities (AREA)
- Hospice & Palliative Care (AREA)
- Psychiatry (AREA)
- Psychology (AREA)
- Social Psychology (AREA)
- Human Computer Interaction (AREA)
- Electrically Operated Instructional Devices (AREA)
Abstract
The present invention relates to a kind of, and the autism children based on Kinect interact device for image and method, Kinect somatosensory video camera is placed on bracket, and it is connect by data line with computer, computer controls Kinect somatosensory video camera and carries out the acquisition of depth data and the transmission of voice signal to infant, and the depth data received is sent to game engine by the bone coordinate data that infant is calculated in the bone capture function of Kinect somatosensory video camera, the movement for forming bone coordinate data is converted to the event that game engine defines, auxiliary game software can be included in the interactive training of children；Computer is connected with display, and training system and auxiliary game software are shown over the display, make infant station in front of the display, can the entire interactive training of complete independently by gesture and limb action；The speaker that computer is placed with two or so is connected, and the carpet of suitable children's activity is placed with before display, is placed with insulated column around carpet.
Description
Technical field
The present invention relates to a kind of interaction device for image, especially a kind of interaction device for image for autism children.
Background technique
Autism (autism) is also known as self-closing disease.Americanism disease society of america publication in 2013《Mental disease diagnosis and system
Count handbook》5th edition (the diagnostic and statistical manual of mental disorders, DSM-
V autism-spectrum obstacle (autistic spectrum disorders, ASD) concept) is formally proposed.The master of autism children
Exception in terms of wanting obstacle to show with people's communication, interest and behavior.Autistic symptoms, which can continue into annual meeting, makes one raw
Work can not take care of oneself, and 2-7 years old is the optimal therapy rehabilitation time, and early treatment can effectively prevent the formation of autistic symptoms.It is lonely
The etiology unknown of only disease, curative effect of medication is unobvious, therefore treatment method more commonly used at present has:Applied behavior analysis training,
Picture exchange AC system, the Intervention Therapy based on physiology and social supplemental training etc..Intervention Therapy based on physiology
It needs to cooperate diet and equipment, social supplemental training needs to spend a large amount of man power and material, is all not easy to promote on a large scale.Due to
Autism children do not feel good to interhuman exchange and answer, so they prefer some computer softwares or system, especially
It is preferable to vision imaging reflection.And the core of applied behavior analysis therapy is task resolving technique, is very suitable to computer
Auxiliary is implemented；The core of picture exchange AC system is to aid in autism children and carries out effectively in such a way that picture is linked up
Interaction, and can be reflected with lively image.Want applied behavior analysis training and picture exchange AC system two
Kind therapeutic modality reflects in conjunction with computer and with image system, is to solve catches to the movement of autism children first
It catches.A body-sensing video camera based on depth image that Kinect is issued as Microsoft can be carried out human body motion capture, realize bone
Bone tracking, has the advantages that cheap, perfect in shape and function, easy to use.There are three cameras altogether by Kinect, and forward direction is from left to right
It is RGB color camera, RF transmitter and infrared C MOS camera respectively.Kinect device bottom is microphone array
Column, can detect the voice of surrounding.There is 1 electric motor between pedestal and inductor, pitch angle can be adjusted by program.
These inductors are respectively intended to obtain RGB, depth data and audio data.Colour imagery shot maximum supports 1280 × 960 pixels
Imaging, infrared camera maximum support the imaging of 640 × 480 pixels.Kinect local Coordinate System be using infrared camera as origin,
It is Z axis positive direction outside its optical axis direction, when user faces Kinect, and horizontal direction is X-axis to the right on its optical axis vertical plane
Positive direction is Y-axis positive direction upwards.This integration of equipments function of motion capture and voice control can pass through USB interface and meter
The connected transmission for carrying out data and controlling signal of calculation machine, then data pass through Computer display on a display screen by treated, and
Accuracy of identification meets the requirement of children training.
Patent publication No. CN103794110A,《Intelligent autism children cognition Communication Behavior early exercise system》It is open
A kind of behavior game training method based on handheld terminal and Kinect sensor, specific implementation method are：Terminal
Carry out data transmission with handheld terminal, terminal is also connected with outlet terminal, and Kinect inductor and terminal connect
It connects, Behavioral training game and information relevant to software for treatment are all carried out in terminal, and terminal 1 is by game picture
It is projected in above large screen, trained autism children give computer using hand-held terminal device and its software operated above
Terminal sends order, realizes the interaction of hand-held terminal device and terminal, completes the interactive process with three dimensional character；And
Kinect inductor can detecte the location information of Deaf children, and be sent to the computer software of terminal, so that
Computer software may determine that the game process of next step by this location information, just complete the process of entire play therapy.
The invention patent facilitates the rehabilitation training and treatment of autism children cognition communication capability, but handheld terminal interactive experience compared with
Difference will disperse attention children.And it is not scientific enough and efficient with interacting for three dimensional character, though simple game can attract solitarily
The interest but therapeutic effect of disease children is bad.Applied behavior analysis training imitates principle using behavior shaping, is with positive reinforcement
Main promotion autism children's items ability development.Core is task resolving technique, and typical mission decomposition technique has 4 steps
Suddenly：Trainer issue instruction, children reaction, to children reaction response, pause, the method to autism children curative effect very
It is good.
Patent publication No. CN108098797A,《Autism children's complementary education robot》Disclose a kind of autism children
Complementary education robot system, specific implementation method is：It is allowed to audible instruction simultaneously by remote control operation by care provider
Movement is made, autism children is allowed to imitate, evaluation then is made to the performance of children.The body structure of anthropomorphic robot and people
It is similar, facilitate autism children carry out action imitation.This game for carrying out action imitation by care provider's remote control operation can have
Effect promotes the interest of autism children, improves the interaction capabilities of itself and people.The invention builds anthropomorphic robot using module part,
Module programming designs education content, is a kind of simple possible, open autism children's complementary education robot.And with application
Based on behavioural analysis training, the application of autism children's complementary education has been carried out in six institute's brain health child development center of Beijing University
Test experiments have received good effect.But the invention needs guardian's remote control operation, needs to expend a large amount of human resources.
And safety is not high, harder mechanical structure causes some potential safety problems to Autism children.
Patent publication No. CN102354349A,《Improve the human-computer interaction multi-modal morning of autism children social interaction ability
Phase interfering system》A kind of multi-modal early intervention system of human-computer interaction for improving autism children social interaction ability is disclosed,
Specific implementation method is：The system includes a multi-point touch panel, a computer and three be separately mounted to the touch screen left side,
The camera on the right and middle top, camera band microphone are connected by USB interface with computer, and system is equipped with visual signal
Processing, Speech processing, physics interactive interface, multi-modal fusion, intelligent console and real scene simulate six basic moulds
Block, fusion calculation machine vision, speech recognition, Activity recognition, intelligent body and virtual reality technology come support improve autism youngster
Virgin social interaction's ability.The invention has been related to " Interactive interface designing " in the treatment of autism-spectrum disorder with children, meets
Autism children track development and change of several children in this academic environment by six months to the hobby of image documentation equipment,
Wherein social interaction's ability of most children is significantly improved, and in addition the interaction capability of children, which also has, definitely improves.But this
The device is complicated for invention, and integrated level is not high, and each basic module independently realizes the function of oneself, and higher cost, is not suitable for big rule
Mould is promoted.
With patent《Intelligent autism children cognition Communication Behavior early exercise system》To represent, existing skill
Art needs Autism children using hand-held terminal device and its software operated above, sends and orders to terminal,
The interaction of hand-held terminal device and terminal is realized, to complete the interactive process with three dimensional character.This technology interactive body
It tests poor, will disperse attention children.In view of the shortcomings of the prior art, it needs in applied behavior analysis training and picture exchange
Design assistant Games Software on the basis of AC system therapy.Applied behavior analysis training imitates principle with using behavior shaping,
Promote autism children items ability development based on positive reinforcement.Core is task resolving technique, and typical mission decomposes
Technology has 4 steps：Trainer issues instruction, the reaction of children, the response to children's reaction, pause, and the method is to autism
Children's curative effect is fine.Widely used picture exchange AC system therapy is a kind of very effective training method, passes through training
Even if some children finally speech can not be fully utilized link up, can also be carried out with other people in such a way that picture is linked up
Effective interaction.
With patent《Intelligent autism children cognition Communication Behavior early exercise system》To represent, the prior art and three-dimensional
The interaction of personage is not scientific enough and efficient, though simple game can attract the interest therapeutic effect of autism children bad.
With patent《Autism children's complementary education robot》To represent, the prior art is alleviated using Robot assisted education
Autistic symptoms, this technology need guardian's remote control operation, and harder mechanical structure Autism children is caused it is certain
Security risk.In view of the shortcomings of the prior art, need to design a kind of autism children interaction device for image.With insulated column and it is suitble to
The carpet of children's activity ensures the activity safety of infant, and the speaker and infant placed with left and right carry out the guidance of sound, with display
Screen carries out the interaction of image with infant, and computer can create the training system of a safety, repeatable, predictable control
System is more conducive to autism children's study, assists without special messenger, saves very big human resources.
With patent《Improve the multi-modal early intervention system of human-computer interaction of autism children social interaction ability》To represent,
The prior art establishes the social energy that the interactive intelligent learning system of multi-modal fusion carrys out effective prevention, improves autism children
Power.Using the multiple devices including touch screen, computer, camera, microphone, and design and handled including visual signal,
Multiple modules including Speech processing, physics interactive interface, multi-modal fusion, intelligent console, real scene simulation, if
Standby sufficiently complex, integrated level is not high, and each basic module independently realizes the function of oneself, and higher cost, is not suitable for extensive
It promotes.In view of the shortcomings of the prior art, it needs to merge using Kinect somatosensory video camera and realizes multiple functions.Kinect cost compared with
It is low, have the function of dynamic capture, image identification, microphone input, speech recognition immediately.It can replace touch screen, computer, take the photograph
As head, microphone and precision can satisfy the requirement of infant training.It is noted that the Software Development Kit of Kinect
Kinect for Windows SDK includes driver, original sensing data flow formula development interface abundant, natural user
Interface, installation file and refer to routine.Development difficulty is small and powerful, operates quite convenient for developer.
Summary of the invention
The present invention is to provide for a kind of autism children interaction device for image and method based on Kinect, the invention patent
The movement of Autism children is captured using Kinect somatosensory video camera, Kinect core technology is that the depth of target can be obtained
Degree evidence has bone following function；The present invention is on the trained basis with picture exchange AC system therapy of applied behavior analysis
Upper Design assistant Games Software.Applied behavior analysis training imitates principle with using behavior shaping, is promoted based on positive reinforcement
Autism children's items ability development；The present invention designs a kind of autism children interaction device for image.With insulated column and suitable youngster
Virgin movable carpet ensures the activity safety of infant, and the speaker and infant placed with left and right carry out the guidance of sound, uses display screen
The interaction of image is carried out with infant, and computer can create the training system of a safety, repeatable, predictable control
It is more conducive to autism children's study, is assisted without special messenger, very big human resources can be saved；The present invention uses Kinect body
Multiple functions can be achieved in sense video camera fusion.
To achieve the above object, the technical scheme is that：A kind of autism children interaction image based on Kinect
Device is made of carpet, insulated column, computer and auxiliary game software, display, Kinect somatosensory video camera, speaker, described
Kinect somatosensory video camera is placed on bracket, and is connect by data line with computer, and computer controls Kinect somatosensory photography
Machine carries out the acquisition of depth data and the transmission of voice signal to infant, and the depth data received is passed through Kinect somatosensory
The bone coordinate data that infant is calculated in the bone capture function of video camera is sent to game engine, makes bone coordinate data shape
At movement be converted to the event that game engine defines, auxiliary game software can be included in the interactive training of children；It is described
Computer is connected with display, and training system and auxiliary game software are shown over the display, make infant station in display
Before, it can the entire interactive training of complete independently by gesture and limb action；The speaker phase that the computer is placed with two or so
Connection carries out language to it by Kinect- computer-speaker Trinity using when auxiliary game software for autism children
Sound guidance, and the interactive environment of an immersion is provided for infant by speaker, facilitate infant cognition communication capability
It improves；It is placed with the carpet of suitable children's activity before the display, is placed with insulated column around carpet.
The Kinect somatosensory video camera include three cameras, pedestal, inductor, electric motors, three cameras, just
To being from left to right RGB color camera, RF transmitter and infrared C MOS camera respectively, bottom has microphone array
Column are equipped with an electric motor, can adjust pitch angle by program for the voice around detecting between pedestal and inductor
Degree, the inductor are respectively intended to obtain RGB, depth data and audio data.
The size of the display is 22 cun, and resolution ratio is 1920 × 1080, screen proportion 16:9 .
A kind of design method for the auxiliary game software interacting device for image for the autism children based on Kinect, packet
It includes：
（1）Basic engineering auxiliary game software is trained for applied behavior analysis
Applied behavior analysis training imitates principle using behavior shaping, based on applied behavior analysis training, designs " with me
Do " action imitation auxiliary game software；
After the action imitation auxiliary game software for starting " doing with me ", virtual animated character carries out game introduction first；By infant
The difficulty level of action imitation is selected by Kinect voice control；Virtual animated character makes movement, is imitated by children；
Children make echomotism after, Kinect record the skeleton point of children and the registration of virtual image in real time, assessment is imitated correct
Whether；It does if children and is awarded praise to if by voice control, if children do wrong to give encouragement by voice control does one again
Time；Children may be selected other difficulty levels according to the experience sense of oneself or be exited the program.
（2）It is basic Design assistant Games Software with picture exchange AC system
Based on picture exchange AC system therapy, the cognitive training auxiliary game software of a " choosing choosing is seen " is designed；It should
The various objects that the cognitive training auxiliary game software of " choosing choosing is seen " is used to be trained autism children in identification daily life
Product, doctor or parent select some stimulators according to the hobby of children, as the progress of training depth can be in training
Stimulant is further expanded in system；The cognitive training auxiliary game software application that " choosing should be selected to see " can be divided into three phases：
There is virtual animated character and scene, virtual animated character starts to introduce present feelings after starting game in first stage
Scape, and provide selectable stimulant such as film or the screenshot of animation, the small screen animation play, and waits infant to utilize gesture
Identification carries out the selection of picture, once correctly picture is selected, the virtual animated character on screen make corresponding movement and
Expression encourages infant, is also equipped in training system corresponding music and encourages, by releasing outside speaker；
Second stage concentrates on simple verbal communication, allows infant to start to learn simple sentence composition, trains infant by word
The selection of " I wants " and picture combines, and to realize the requirement of expression " what I want ", children and virtual animated character start with simple
Language and picture combine and exchanged, while being accompanied by voice guidance；
Phase III, voice prompting is issued in conjunction with the selection of infant in a manner of question and answer, child says oneself according to voice prompting
Demand, such as specifically want what picture selected.
According to the therapeutic modality of autism, a variety of Games Softwares for autism children therapy and intervention are designed, such as：
It virtually drives a car, virtual book, virtual drawing board；It may also be combined with the body-sensing trivial games developed and intelligence carried out to autism children
The software of power exploitation, such as：The fruit person of bearing, Tetris.
A kind of application interacts device for image based on the autism children of Kinect and stores to the information that autism children are added
Appraisal procedure is carried out with interaction effect, including：
（1）Infant information is deleted in typing, modification；
（2）Infant stimulant information is deleted in typing, modification, and the preferences or detest of multiple stimulant classifications are directed to including infant
?；
（3）Typing and the assessment information for automatically generating multiple entries for multiple assessment categories；It is raw according to the assessment information
At the assessment report of graphic software platform；And according to the preferences or detest item and the assessment information, typing and automatic life
At next stage training objective.
The beneficial effects of the invention are as follows：
1, the present invention uses motion capture equipment of the Kinect as autism children, is integrated with dynamic capture immediately, image is distinguished
Knowledge, microphone input, the function of speech recognition can replace touch screen, computer, camera, microphone apparatus, and cost is relatively low.
And the training system that computer is write can control Kinect to carry out corresponding operation, is not necessarily to special messenger's remote control or auxiliary, is not necessarily to
Infant hand-held terminal device saves a large amount of human resources.
2, the present invention is that basic Design assistant game is soft with applied behavior analysis training and picture exchange AC system therapy
Part.It is more scientific and efficient.
3, the present invention has caught Autism children to like some computer softwares or system, especially to vision imaging reflect compared with
Good feature devises and interacts device for image with infant.This device is by carpet, insulated column, computer, display, Kinect
Body-sensing video camera, speaker composition.The Kinect being placed on bracket is connected with computer, computer respectively with display and two sounds
Case is connected, and face is placed with the carpet of suitable children's activity in front of the display, and carpet surrounding is placed with insulated column.Hardware design section
It learns, is interacted on the basis of ensureing safety from vision, the sense of hearing, tactile, movement various aspects with infant, facilitate autism youngster
The training and treatment of child's cognition communication capability.
4, the present invention is portable relatively strong and can carry out personalized customization.It can be mutual according to the individual of different autism children
Kinetoplast is tested, and current physical training condition and the corresponding auxiliary Games Software of Demand Design replacement are supported to include different child development paths.
A kind of autism children based on Kinect that the present invention designs interact device for image, are photographed using Kinect somatosensory
Machine, computer interact device for image with speaker composition, are aided with applied behavior analysis training and picture exchange AC system therapy
Based on Games Software, be fully able to meet exploitation autism children's intelligence and adjuvant treatment or improve autism infantile psychology
The society need of state, and be not required to special messenger's auxiliary, cost is relatively low, can the heart and domestic applications in the treatment simultaneously.
Detailed description of the invention
Fig. 1 is that the autism children of the invention based on Kinect interact device for image schematic diagram.
Specific embodiment
The invention will be further described with embodiment with reference to the accompanying drawing.
As shown in Figure 1, autism children of the invention based on Kinect interact device for image, by carpet 5, insulated column 6,
Computer 2, display 3, Kinect somatosensory video camera 1, speaker 4 form.It is placed in the Kinect somatosensory video camera 1 on bracket 8
Data line is connect with computer 2, and the speaker 4 that computer 2 is placed with display 3 and two or so respectively is connect, before display 4
Face is placed with the carpet 5 of suitable children's activity, is placed with insulated column 6 around carpet 5.Kinect somatosensory video camera 1, computer 2,
Display 3 and speaker 4 patch battery plate 7 by power supply line and take electricity.
It is as follows specifically to introduce core component：
A body-sensing video camera based on depth image that Kinect somatosensory video camera is issued as Microsoft, can be carried out human motion
It captures, realizes bone tracking, have the advantages that cheap, perfect in shape and function, easy to use.Camera there are three Kinect is total,
Forward direction is from left to right RGB color camera, RF transmitter and infrared C MOS camera respectively.Kinect device bottom
For microphone array, the voice of surrounding can be detected.There is 1 electric motor between pedestal and inductor, can be adjusted by program
Whole pitch angle.These inductors are respectively intended to obtain RGB, depth data and audio data.Colour imagery shot maximum is supported
The imaging of 1280 × 960 pixels, infrared camera maximum support the imaging of 640 × 480 pixels.Kinect local Coordinate System is with infrared
Camera is origin, is Z axis positive direction outside optical axis direction, vertical with its optical axis when user faces Kinect somatosensory video camera
Horizontal direction is X-axis positive direction to the right in plane, is upwards Y-axis positive direction.This integration of equipments motion capture and voice control
Function, can be connected by USB interface with computer and carry out data and control the transmission of signal, then will treated that data pass through
Over the display, and accuracy of identification meets the requirement of children training to Computer display.
Computer is the controller of the present apparatus.Computer is connected with Kinect somatosensory video camera, the training system control write
Kinect somatosensory video camera processed carries out the acquisition of depth data and the transmission of voice signal to infant.The depth data that will be received
The bone coordinate data that infant is calculated in function is captured by the bone of Kinect somatosensory video camera.Game engine is connected, it will
Bone coordinate data is sent to game engine, and the movement that bone coordinate data is formed is converted to the event that game engine defines, just
Auxiliary game software can be included in the interactive training of children；Computer is connected with display, by training system and auxiliary game
Software shows that over the display infant station in front of the display, can the entire interactive training of complete independently by gesture and limb action；
Computer is connected with speaker, passes through Kinect- computer-speaker Trinity when autism children use auxiliary game software
Voice guidance is carried out to it.And the interactive environment of an immersion is provided for infant by speaker, infant is more conducive to and recognizes
Know the raising of communication capability.
The size of display is 22 cun, and resolution ratio is 1920 × 1080, screen proportion 16:9 .Other are such as：Carpet, every
From column, the safety of device has been ensured, autism children are not required to wear any equipment, and zone of action range is fixed, without any touching
Electricity, dangerous occur such as damage.Thus this interaction device for image can be used for training on autism counseling center class, can also place and be in
In be used for peacetime training, make autism children that can also learn and play in the case where unattended.
The design side of the auxiliary game software that device for image is interacted for the autism children based on Kinect of the invention
Method, including：
（1）The auxiliary game software of basic engineering is trained for applied behavior analysis
Applied behavior analysis training imitates principle using behavior shaping, promotes autism children items ability based on positive reinforcement
Development.Core is task resolving technique, and typical mission decomposition technique has 4 steps：Trainer issue instruction, children it is anti-
It answers, the response to children's reaction, pause, the method is fine to autism children's curative effect.Based on applied behavior analysis training,
The invention patent devises the action imitation auxiliary game software of a entitled " doing with me ".
After starting game, virtual animated character carries out game introduction first；It is selected by infant by Kinect voice control
The difficulty level (simple, medium, difficult) of action imitation；Virtual animated character makes movement, is imitated by children；Children do
Out after echomotism, Kinect records the skeleton point of children and the registration of virtual image in real time, and correctness is imitated in assessment；If
Children do to then being awarded praise by voice control, are given encouragement and are done again one time by voice control if children do wrong；Children
Other difficulty levels may be selected according to the experience sense of oneself or exited the program.
（2）The auxiliary game software designed based on picture exchange AC system
Widely used picture exchange AC system therapy is a kind of very effective training method, it is arrogated to oneself using autism children
Long Visual Communication carries out cognition exchange to teach autism children using picture, and the cognitive ability of children is inspired with this,
The communication desire for exciting children cultivates it and recognizes communication exchange ability, and gradually links up their exchange way from picture and turn
Change to speech communication.Even if linking up by training some children that speech can not finally be fully utilized, pass through what picture was linked up
Mode also can effectively be interacted with other people.Based on picture exchange AC system therapy, the invention patent is devised
The cognitive training auxiliary game software of a entitled " choosing choosing is seen ".
The auxiliary game software is mainly the various articles being trained in identification daily life to autism children, doctor
Or parent according to the hobby of children select some stimulators, with training depth progress can in training system into
One step extends stimulant.The software can be divided into three phases, be described as follows：
There is virtual animated character and scene, virtual animated character starts to introduce present feelings after starting game in first stage
Scape, and selectable stimulant is provided, such as film or the screenshot of animation, the small screen animation play etc..And infant is waited to utilize hand
Gesture identification carries out the selection of picture, once correctly picture is selected, the virtual animated character on screen makes corresponding movement
Infant is encouraged with expression, corresponding music is also equipped in training system and is encouraged, by being released outside speaker.
Second stage focuses primarily upon simple verbal communication, allows infant to start to learn simple sentence composition, training is suffered from
Youngster combines word " I wants " and picture selection, and to realize the requirement of expression " what I want ", children and virtual animated character are opened
Begin to be combined with simple language with picture and be exchanged, while being accompanied by voice guidance.
Phase III can issue voice prompting in conjunction with the selection of infant in a manner of question and answer, and child is according to voice prompting
The demand of oneself is said, for example specifically wants what picture etc. selected.
In addition, can also be carried out for the auxiliary game software that the autism children based on Kinect interact device for image a
Propertyization customization, in addition to the auxiliary designed based on applied behavior analysis is trained with picture exchange AC system therapy in the present invention
Except Games Software, there are many therapeutic modality of autism, need to increase with autism children, also can be to auxiliary game software
It is adjusted and increases.It can be designed that a variety of game for autism children therapy and intervention, such as：It virtually drives a car, is empty
Quasi- books, virtual drawing board etc..It may also be combined with the body-sensing trivial games developed and intellectual development carried out to autism children, such as：
The fruit person of bearing, Tetris etc..Combine alternatively, it is also possible to the method for demonstration will be recorded a video with somatosensory device to autism children into
Trained or instruction after class on row class, such as：One section of simple dance movement is played, autism children finish watching oneself energy after this section of movement
It follows screen to show that accompanied by music makes identical movement, adapts to the demand of autism children.
Autism children based on Kinect of the invention interact device for image and can also be added in addition to function as described above
Information storage and the interaction effect evaluation function of autism children.Specifically comprise the following steps：Infant letter is deleted in typing, modification
Breath；Infant stimulant information is deleted in typing, modification, for the preferences of multiple stimulant classifications or detests item including infant；
Typing and the assessment information for automatically generating multiple entries for multiple assessment categories；It is generated according to the assessment information graphical
The assessment report of display；And according to the preferences or detest item and the assessment information, typing and automatically generate lower single order
Section training objective.
Claims (6)
1. a kind of autism children based on Kinect interact device for image, soft by carpet, insulated column, computer and auxiliary game
Part, display, Kinect somatosensory video camera, speaker composition, it is characterised in that：The Kinect somatosensory video camera is placed in bracket
On, and connect by data line with computer, computer controls Kinect somatosensory video camera and adopts to infant progress depth data
The transmission of collection and voice signal, and the depth data received is captured into function by the bone of Kinect somatosensory video camera and is calculated
The bone coordinate data for obtaining infant is sent to game engine, and it is fixed that the movement for forming bone coordinate data is converted to game engine
Auxiliary game software, can be included in the interactive training of children by the event of justice；The computer is connected with display, will instruct
Practice system and auxiliary game software is shown over the display, makes infant station in front of the display, it can be only by gesture and limb action
It is vertical to complete entire interactive training；The speaker that the computer is placed with two or so is connected, and uses for autism children auxiliary
It helps logical Kinect- computer-speaker Trinity when Games Software to carry out voice guidance to it, and is mentioned by speaker for infant
The interactive environment for having supplied an immersion facilitates the raising of infant cognition communication capability；It is placed with before the display suitable
The carpet of children's activity is closed, is placed with insulated column around carpet.
2. the autism children according to claim 1 based on Kinect interact device for image, it is characterised in that：It is described
Kinect somatosensory video camera includes three cameras, pedestal, inductor, electric motors, and three cameras are positive from left to right to divide
It is not RGB color camera, RF transmitter and infrared C MOS camera, bottom has microphone array, for detecting
The voice of surrounding is equipped with an electric motor, can adjust pitch angle, the induction by program between pedestal and inductor
Device is respectively intended to obtain RGB, depth data and audio data.
3. the autism children according to claim 1 based on Kinect interact device for image, it is characterised in that：It is described aobvious
The size for showing device is 22 cun, and resolution ratio is 1920 × 1080, screen proportion 16:9 .
4. a kind of auxiliary trip for interacting device for image for any autism children based on Kinect of claim 1-3
The design method of play software, it is characterised in that：Including：
（1）Basic engineering auxiliary game software is trained for applied behavior analysis
Applied behavior analysis training imitates principle using behavior shaping, based on applied behavior analysis training, designs " with me
Do " action imitation auxiliary game software；
After the action imitation auxiliary game software for starting " doing with me ", virtual animated character carries out game introduction first；By infant
The difficulty level of action imitation is selected by Kinect voice control；Virtual animated character makes movement, is imitated by children；
Children make echomotism after, Kinect record the skeleton point of children and the registration of virtual image in real time, assessment is imitated correct
Whether；It does if children and is awarded praise to if by voice control, if children do wrong to give encouragement by voice control does one again
Time；Children may be selected other difficulty levels according to the experience sense of oneself or be exited the program；
（2）It is basic Design assistant Games Software with picture exchange AC system
Based on picture exchange AC system therapy, the cognitive training auxiliary game software of a " choosing choosing is seen " is designed；It should
The various objects that the cognitive training auxiliary game software of " choosing choosing is seen " is used to be trained autism children in identification daily life
Product, doctor or parent select some stimulators according to the hobby of children, as the progress of training depth can be in training
Stimulant is further expanded in system；The cognitive training auxiliary game software application that " choosing should be selected to see " can be divided into three phases：
There is virtual animated character and scene, virtual animated character starts to introduce present feelings after starting game in first stage
Scape, and provide selectable stimulant such as film or the screenshot of animation, the small screen animation play, and waits infant to utilize gesture
Identification carries out the selection of picture, once correctly picture is selected, the virtual animated character on screen make corresponding movement and
Expression encourages infant, is also equipped in training system corresponding music and encourages, by releasing outside speaker；
Second stage concentrates on simple verbal communication, allows infant to start to learn simple sentence composition, trains infant by word
The selection of " I wants " and picture combines, and to realize the requirement of expression " what I want ", children and virtual animated character start with simple
Language and picture combine and exchanged, while being accompanied by voice guidance；
Phase III, voice prompting is issued in conjunction with the selection of infant in a manner of question and answer, child says oneself according to voice prompting
Demand, such as specifically want what picture selected.
5. design method according to claim 4, it is characterised in that：According to the therapeutic modality of autism, can also design more
Kind is directed to the Games Software of autism children therapy and intervention, such as：It virtually drives a car, virtual book, virtual drawing board；It may also be combined with
The body-sensing trivial games developed carry out the software of intellectual development to autism children, such as：The fruit person of bearing, Tetris.
6. a kind of interact device for image to addition orphan using any autism children based on Kinect of claim 1-3
Information storage and the interaction effect of only disease children carries out appraisal procedure, including：
（1）Infant information is deleted in typing, modification；
（2）Infant stimulant information is deleted in typing, modification, and the preferences or detest of multiple stimulant classifications are directed to including infant
?；
（3）Typing and the assessment information for automatically generating multiple entries for multiple assessment categories；It is raw according to the assessment information
At the assessment report of graphic software platform；And according to the preferences or detest item and the assessment information, typing and automatic life
At next stage training objective.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810667807.9A CN108919950A (en) | 2018-06-26 | 2018-06-26 | Autism children based on Kinect interact device for image and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810667807.9A CN108919950A (en) | 2018-06-26 | 2018-06-26 | Autism children based on Kinect interact device for image and method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108919950A true CN108919950A (en) | 2018-11-30 |
Family
ID=64421841
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810667807.9A Pending CN108919950A (en) | 2018-06-26 | 2018-06-26 | Autism children based on Kinect interact device for image and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108919950A (en) |
Cited By (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109333544A (en) * | 2018-09-11 | 2019-02-15 | 厦门大学 | A kind of image exchange method for the marionette performance that spectators participate in |
| CN109545293A (en) * | 2018-12-04 | 2019-03-29 | 北京大学 | A kind of autism high-risk infants screening system based on APP |
| CN109717878A (en) * | 2018-12-28 | 2019-05-07 | 上海交通大学 | A kind of detection system and application method paying attention to diagnosing normal form jointly for autism |
| CN109876264A (en) * | 2019-01-30 | 2019-06-14 | 上海市精神卫生中心（上海市心理咨询培训中心） | A kind of neurodevelopmental disorder patient sense system supplemental training method and system |
| CN109903629A (en) * | 2019-04-15 | 2019-06-18 | 黑龙江省小儿脑性瘫痪防治疗育中心 | A kind of interactive training device of autism children |
| CN109939325A (en) * | 2019-01-30 | 2019-06-28 | 上海市精神卫生中心（上海市心理咨询培训中心） | A kind of neurodevelopmental disorder music training system based on virtual reality technology |
| CN110404148A (en) * | 2019-08-07 | 2019-11-05 | 上海市精神卫生中心（上海市心理咨询培训中心） | A kind of self-closing disease mood based on virtual reality technology pacifies method and system |
| CN110610754A (en) * | 2019-08-16 | 2019-12-24 | 天津职业技术师范大学(中国职业培训指导教师进修中心) | Immersive wearable diagnosis and treatment device |
| CN110992745A (en) * | 2019-12-23 | 2020-04-10 | 英奇源(北京)教育科技有限公司 | Interaction method and system for assisting infant to know four seasons based on motion sensing device |
| WO2020119355A1 (en) * | 2018-12-14 | 2020-06-18 | 深圳先进技术研究院 | Method for evaluating multi-modal emotional understanding capability of patient with autism spectrum disorder |
| CN112230777A (en) * | 2020-10-29 | 2021-01-15 | 浙江工业大学 | Cognitive training system based on non-contact interaction |
| CN113040745A (en) * | 2021-03-15 | 2021-06-29 | 华中科技大学同济医学院附属协和医院 | Active guiding method and system for painless examination and treatment of children |
| CN113096805A (en) * | 2021-04-12 | 2021-07-09 | 华中师范大学 | Autism emotion cognition and intervention system |
| CN113506624A (en) * | 2021-08-16 | 2021-10-15 | 北京阿叟阿巴科技有限公司 | Autism child cognitive ability assessment intervention system based on layer-by-layer generalization push logic |
| CN113741689A (en) * | 2021-08-20 | 2021-12-03 | 苏州创捷传媒展览股份有限公司 | Display system based on Kinect human body contour recognition interaction |
| CN113856213A (en) * | 2021-09-16 | 2021-12-31 | 浙江理工大学 | Music interaction toy assisting self-closing children in exercise rehabilitation |
| CN114968054A (en) * | 2022-05-11 | 2022-08-30 | 山东大学 | Cognitive training interaction system and method based on mixed reality |
| CN116312081A (en) * | 2022-09-07 | 2023-06-23 | 中山大学 | Child autism treatment device based on ball game |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN202257985U (en) * | 2011-10-27 | 2012-05-30 | 温萍萍 | Image interaction device suitable for autism of children |
| CN103164995A (en) * | 2013-04-03 | 2013-06-19 | 湖南第一师范学院 | Children somatic sense interactive learning system and method |
| CN103220543A (en) * | 2013-04-25 | 2013-07-24 | 同济大学 | Real time three dimensional (3D) video communication system and implement method thereof based on Kinect |
| CN103914129A (en) * | 2013-01-04 | 2014-07-09 | 云联（北京）信息技术有限公司 | Man-machine interactive system and method |
| CN104524742A (en) * | 2015-01-05 | 2015-04-22 | 河海大学常州校区 | Cerebral palsy child rehabilitation training method based on Kinect sensor |
| CN106502390A (en) * | 2016-10-08 | 2017-03-15 | 华南理工大学 | A kind of visual human's interactive system and method based on dynamic 3D Handwritten Digit Recognitions |
| CN107301611A (en) * | 2017-07-04 | 2017-10-27 | 北京师范大学 | A kind of autism child intelligence Teaching Evaluation System |
-
2018
- 2018-06-26 CN CN201810667807.9A patent/CN108919950A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN202257985U (en) * | 2011-10-27 | 2012-05-30 | 温萍萍 | Image interaction device suitable for autism of children |
| CN103914129A (en) * | 2013-01-04 | 2014-07-09 | 云联（北京）信息技术有限公司 | Man-machine interactive system and method |
| CN103164995A (en) * | 2013-04-03 | 2013-06-19 | 湖南第一师范学院 | Children somatic sense interactive learning system and method |
| CN103220543A (en) * | 2013-04-25 | 2013-07-24 | 同济大学 | Real time three dimensional (3D) video communication system and implement method thereof based on Kinect |
| CN104524742A (en) * | 2015-01-05 | 2015-04-22 | 河海大学常州校区 | Cerebral palsy child rehabilitation training method based on Kinect sensor |
| CN106502390A (en) * | 2016-10-08 | 2017-03-15 | 华南理工大学 | A kind of visual human's interactive system and method based on dynamic 3D Handwritten Digit Recognitions |
| CN107301611A (en) * | 2017-07-04 | 2017-10-27 | 北京师范大学 | A kind of autism child intelligence Teaching Evaluation System |
Cited By (24)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109333544A (en) * | 2018-09-11 | 2019-02-15 | 厦门大学 | A kind of image exchange method for the marionette performance that spectators participate in |
| CN109333544B (en) * | 2018-09-11 | 2020-07-17 | 厦门大学 | Doll interaction method for marionette performance participated by audience |
| CN109545293A (en) * | 2018-12-04 | 2019-03-29 | 北京大学 | A kind of autism high-risk infants screening system based on APP |
| WO2020119355A1 (en) * | 2018-12-14 | 2020-06-18 | 深圳先进技术研究院 | Method for evaluating multi-modal emotional understanding capability of patient with autism spectrum disorder |
| CN109717878A (en) * | 2018-12-28 | 2019-05-07 | 上海交通大学 | A kind of detection system and application method paying attention to diagnosing normal form jointly for autism |
| CN109876264A (en) * | 2019-01-30 | 2019-06-14 | 上海市精神卫生中心（上海市心理咨询培训中心） | A kind of neurodevelopmental disorder patient sense system supplemental training method and system |
| CN109939325A (en) * | 2019-01-30 | 2019-06-28 | 上海市精神卫生中心（上海市心理咨询培训中心） | A kind of neurodevelopmental disorder music training system based on virtual reality technology |
| CN109903629A (en) * | 2019-04-15 | 2019-06-18 | 黑龙江省小儿脑性瘫痪防治疗育中心 | A kind of interactive training device of autism children |
| CN110404148A (en) * | 2019-08-07 | 2019-11-05 | 上海市精神卫生中心（上海市心理咨询培训中心） | A kind of self-closing disease mood based on virtual reality technology pacifies method and system |
| CN110610754A (en) * | 2019-08-16 | 2019-12-24 | 天津职业技术师范大学(中国职业培训指导教师进修中心) | Immersive wearable diagnosis and treatment device |
| CN110992745A (en) * | 2019-12-23 | 2020-04-10 | 英奇源(北京)教育科技有限公司 | Interaction method and system for assisting infant to know four seasons based on motion sensing device |
| CN112230777A (en) * | 2020-10-29 | 2021-01-15 | 浙江工业大学 | Cognitive training system based on non-contact interaction |
| CN113040745A (en) * | 2021-03-15 | 2021-06-29 | 华中科技大学同济医学院附属协和医院 | Active guiding method and system for painless examination and treatment of children |
| CN113096805A (en) * | 2021-04-12 | 2021-07-09 | 华中师范大学 | Autism emotion cognition and intervention system |
| CN113096805B (en) * | 2021-04-12 | 2024-02-13 | 华中师范大学 | Autism emotion cognition and intervention system |
| CN113506624A (en) * | 2021-08-16 | 2021-10-15 | 北京阿叟阿巴科技有限公司 | Autism child cognitive ability assessment intervention system based on layer-by-layer generalization push logic |
| CN113506624B (en) * | 2021-08-16 | 2023-08-08 | 北京阿叟阿巴科技有限公司 | Autism children cognitive ability evaluation intervention system based on hierarchical generalization push logic |
| CN113741689A (en) * | 2021-08-20 | 2021-12-03 | 苏州创捷传媒展览股份有限公司 | Display system based on Kinect human body contour recognition interaction |
| CN113741689B (en) * | 2021-08-20 | 2024-01-19 | 苏州创捷传媒展览股份有限公司 | Kinect-based human body contour recognition interaction display system |
| CN113856213A (en) * | 2021-09-16 | 2021-12-31 | 浙江理工大学 | Music interaction toy assisting self-closing children in exercise rehabilitation |
| CN114968054A (en) * | 2022-05-11 | 2022-08-30 | 山东大学 | Cognitive training interaction system and method based on mixed reality |
| CN114968054B (en) * | 2022-05-11 | 2023-09-15 | 山东大学 | Interaction system and method for cognitive training based on mixed reality |
| CN116312081A (en) * | 2022-09-07 | 2023-06-23 | 中山大学 | Child autism treatment device based on ball game |
| CN116312081B (en) * | 2022-09-07 | 2024-05-07 | 中山大学 | Child autism treatment device based on ball game |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108919950A (en) | Autism children based on Kinect interact device for image and method | |
| Fasola et al. | A socially assistive robot exercise coach for the elderly | |
| Boucenna et al. | Interactive technologies for autistic children: A review | |
| Erden | Emotional postures for the humanoid-robot nao | |
| Fasola et al. | Robot exercise instructor: A socially assistive robot system to monitor and encourage physical exercise for the elderly | |
| Billard | Robota: Clever toy and educational tool | |
| Nacher et al. | Interactive technologies for preschool game-based instruction: Experiences and future challenges | |
| CN110931111A (en) | Autism auxiliary intervention system and method based on virtual reality and multi-mode information | |
| CN107077229B (en) | Human-machine interface device and system | |
| Tacgin | Virtual and augmented reality: an educational handbook | |
| Roglić et al. | Serious gaming based on Kinect technology for autistic children in Serbia | |
| Johansson et al. | Epi: An open humanoid platform for developmental robotics | |
| Peña et al. | Circus in Motion: a multimodal exergame supporting vestibular therapy for children with autism | |
| Liu et al. | Cognitive computing for intelligent robots in assisting preschool children | |
| Louie et al. | Playing a memory game with a socially assistive robot: A case study at a long-term care facility | |
| Ma et al. | Development of the Interactive Rehabilitation Game System for Children with Autism Based on Game Psychology | |
| Palestra et al. | Assistive robot, RGB-D sensor and graphical user interface to encourage communication skills in ASD population | |
| Huang et al. | A study on the development of a mixed reality system applied to the practice of socially interactive behaviors of children with autism Spectrum disorder | |
| Faria et al. | Towards ai-based interactive game intervention to monitor concentration levels in children with attention deficit | |
| Al-Qbilat | Accessibility requirements for human-robot interaction for socially assistive robots | |
| Wainer | Facilitating collaboration among children with autism through robot-assisted play | |
| Matsumaru et al. | Image-projecting desktop arm trainer for hand-eye coordination training | |
| Mensah et al. | Preliminary observations from interactions among Ghanaian autistic children and Rosye, a humanoid robotic assistive technology | |
| Zedda et al. | A Cooking Game for Cognitive Training of Older Adults Interacting with a Humanoid Robot. | |
| Feil-Seifer | Data-driven interaction methods for socially assistive Robotics: validation with children with Autism spectrum disorders |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20181130 |
|
| RJ01 | Rejection of invention patent application after publication |