AU2018102139A4

AU2018102139A4 - Advanced Learning System

Info

Publication number: AU2018102139A4
Application number: AU2018102139A
Authority: AU
Inventors: Johannes Andries DE BEER
Original assignee: Learning Institute For Science And Technology Pty Ltd; Learning Institute For Science And Tech Pty Ltd; List Premier Education Pty Ltd
Current assignee: Learning Institute For Science And Technology Pty Ltd; List Premier Education Pty Ltd
Priority date: 2016-02-10
Filing date: 2018-08-01
Publication date: 2019-10-03
Anticipated expiration: 2027-01-23

Abstract

The present invention relates to an advanced learning system. The system includes a touchscreen device including a display which a learner can touch to interact with educational content. The system further includes a dynamic teaching avatar to teach the learner about the educational content with which the learner is interacting. Advantageously, the use of a dynamic teaching avatar may enhance the complexity of content which can be taught using the touchscreen device, thereby increasing utilization in the classroom compared with simply using a conventional touch screen device in isolation. Furthermore, the avatar teaches the learner about the educational content and may maintain the student's interest.

Description

2018102139 01 Aug 2018

ADVANCED LEARNING SYSTEM

TECHNICAL FIELD [0001] The present invention relates to advance learning systems with touch screen devices such as touch boards. The present invention has particular, although not exclusive application to the classroom.

BACKGROUND [0002] The reference to any prior art in this specification is not, and should not be taken as an acknowledgement or any form of suggestion that the prior art forms part of the common general knowledge.

[0003] A known touch screen device includes a display which a learner can touch to interact with educational content. Such a device is particularly effective in teaching children and autistic children.

[0004] However, the Applicant has perceived that such touch screen devices are underutilized as teaching aids and are often only used in a peripheral teaching capacity at schools. The educational content presented using such touch screen devices also tends to be relatively rudimental when compared with formal lessons and does not maintain the student’s interest for long.

[0005] The preferred embodiment provides a system which increases utilization of touch screen devices in the classroom and maintains the student’s interest in learning educational content.

SUMMARY OF THE INVENTION [0006] According to one aspect of the present invention, there is provided an advanced learning system including:

a touchscreen device including a display which a learner can touch to interact with educational content; and

2018102139 01 Aug 2018 a dynamic teaching avatar to teach the learner about the educational content with which the learner is interacting, wherein the avatar is configured to dynamically embody a plurality of expressions or gestures.

[0007] Advantageously, the use of a teaching avatar may enhance the complexity of content which can be taught using the touchscreen device, thereby increasing utilization in the classroom compared with simply using a conventional touch screen device in isolation. Furthermore, the avatar teaches the learner about the educational content and may maintain the student’s interest.

[0008] The system includes an editor for editing the educational content and the avatar. The editor may include a presentation chart displaying parameter representations (e.g. icons) relating to the educational content and the avatar, the parameter representations displayed relative to a timeline. The editor may include a selector for selecting the parameter representations to be introduced onto the chart and locating the representations along the timeline for execution at a corresponding time. The representations may be editable or fixed. The timeline may automatically vary depending upon the number of parameter representations.

[0009] The parameter representations in the chart may be located within corresponding categories to facilitate planning of the educational content and the avatar. The parameter representations may be prohibited from being located within other categories. The parameter representations may include any one or more of content representations relating to lesson content, audio (e.g. music) representations relating to audio, avatar speech representations relating to avatar speech characteristics, avatar expression representations relating to avatar expressions and avatar gesture representations relating to avatar gestures.

[00010] The editor may further include actuators for actuating functions relating to the presentation chart. The actuators may include any one or more of: a new presentation actuator for creating a new presentation chart, a open presentation actuator for opening a saved presentation chart, save actuator for saving a presentation chart and a preview actuator for previewing the execution of the presentation chart.

2018102139 01 Aug 2018 [00011] The editor may further include an avatar selection tool for selecting the avatar from a number of possible avatars.

[00012] The system may further include a remote control for controlling the avatar. The editor may include one or more text to speech boxes in which text is received that is spoken by the avatar.

[00013] The display may include a single screen on which the avatar is displayed.

The display may include a double screen, being either a single display partitioned in two or two displays adjacent each other, with the educational content appearing on one screen and the avatar appearing on the other screen.

[00014] The system may further include another display separate from the touch screen device for displaying the avatar. The other display may be positioned anywhere in a classroom, but preferably in the front, whereby the avatar assumes the role of an instructor. The system further includes a projector for projecting the avatar on the other display.

[00015] The system may include a holographic display for displaying the avatar as a holographic image. The system may include a human profile shaped cut-out with the avatar displayed on the cut-out. The avatar may resemble a robot or animatronic device to facilitate transfer of technical knowledge to the learner.

[00016] The system may include a virtual reality headset with the avatar being viewed through the virtual reality headset. The headset may provide immersive learning whereby, through virtual reality, the student can be immersed into a simulated real world with simulated circumstances. The avatar may provide guidance and assume the form of many different characters within a virtual world.

[00017] The touch screen device may include a touch board, TV screen, computer screen, laptop screen, tablet screen, mobile phone screen, projected screen, or other like platform used for teaching.

[00018] The system may include a controller for controlling the avatar. The avatar may be controlled through voice commands, voice prompts, physical hand or body gestures detected by a motion recognizing camera, or simply selecting options

2018102139 01 Aug 2018 displayed on the touchscreen device or other user interface (e.g. keyboard). The system may further include a computer, with which a human teacher can select avatar programs in a database that relate to the educational content. A voice command system may be provided for allowing voice recognition commands given to the avatar to enhance the learning experience and ease of lesson delivery. Through voice commands the Avatar can find answers to common questions using the world wide web and existing search and browser engines (such as google voice command) to showcase possible answers and results.

[00019] The avatar may assume the role of a teacher. The avatar may follow a predetermined teaching program (e.g. course, curriculum, lesson, etc.) that is selected by the human teacher. In accordance with the program, the avatar may perform any one or more of: present theory, ask questions, introduce activities, provide periodical reminders or prompts to the human teacher to conduct assessment and evaluation, and answer predetermined questions.

[00020] The avatar may assume the role of a teacher's assistant. The avatar may provide a teacher's notes. The avatar may provide additional information to the human teacher upon request. The avatar may provide general guidance to the teacher as well as explaining how to conduct activities. The avatar may narrate at least part of a lesson to save the human teacher time and energy.

[00021] The avatar may communicate and socialize with the learner through an artificial intelligence system or software. The leaner may engage with the avatar in either basic conversation, in a two way discussion, or open ended role play. The role play may be enabled in a virtual world.

[00022] According to another aspect of the present invention, there is provided an advanced learning method including:

enabling a learner to touch and interact with educational content on a display of a touchscreen device; and providing a dynamic teaching avatar to teach the learner about the educational content with which the learner is interacting, wherein the avatar is configured to dynamically embody a plurality of expressions or gestures.

2018102139 01 Aug 2018 [00023] According to another aspect of the present invention, there is provided an advanced avatar system including:

a touchscreen device including a display which a learner can touch to interact with content; and a dynamic avatar used in conjunction with the content with which the learner is interacting, wherein the avatar is configured to dynamically embody a plurality of expressions or gestures.

[00024] Any of the features described herein can be combined in any combination with any one or more of the other features described herein within the scope of the invention.

BRIEF DESCRIPTION OF THE DRAWINGS [00025] Preferred features, embodiments and variations of the invention may be discerned from the following Detailed Description which provides sufficient information for those skilled in the art to perform the invention. The Detailed Description is not to be regarded as limiting the scope of the preceding Summary of the Invention in anyway. The Detailed Description will make reference to a number of drawings as follows:

[00026] Figure 1 is schematic view of a advanced learning system in accordance with an embodiment of the present invention;

[00027] Figure 2 is a schematic view of an alternative touchscreen device for use with the advanced learning system of Figure 1;

[00028] Figure 3 is a schematic diagram of an editor of the system of Figure 1;

[00029] Figure 4 is a schematic diagram showing saving of a presentation using the editor of Figure 3;

[00030] Figure 5 is a schematic diagram of a remote control of the system of Figure 1; and [00031] Figure 6 is a schematic diagram of the remote control of Figure 5 with an artificial intelligence (Al) button.

2018102139 01 Aug 2018

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS [00032] According to an embodiment of the present invention, there is provided an advanced learning system 100 for a classroom as shown in Figure 1. The system 100 includes a touchscreen device 102 including a display 104 which a student 106 (i.e. learner) can touch to interact with educational content. A teaching avatar 108 is provided to teach the student 106 about the educational content with which the student 106 is interacting.

[00033] Advantageously, the use of the teaching avatar 108 enhances the complexity of content which can be taught using the touchscreen device 102, thereby increasing utilization in the classroom compared with simply using a conventional touch screen device in isolation. Furthermore, the avatar 108 teaches the learner about the educational content and actively maintains the student’s interest.

[00034] The system 100 further includes another display 110, separate from the touch screen device 102, for displaying the avatar 108. The mobile avatar display 110 can be positioned anywhere in a classroom, but preferably in the front, whereby the avatar 108 assumes the role of a teacher (i.e. instructor). The system 100 further includes a projector 112 for projecting the avatar 110 onto the avatar display 110.

[00035] The system 100 further includes a controller 114 for controlling the avatar 108. The avatar 108 can be controlled through detected voice commands, voice prompts, physical gestures, or simply selecting options displayed on the touchscreen device 102 or other user interface such as a keyboard 116. The control commands for the avatar 108, in the form of programs, are stored in an avatar database 118. The system 100 further includes a teacher’s computer 120, with which a human teacher 122 can select avatar programs in the database 118 that relate to the educational content being taught to the student 106. The system hardware is typically interconnected by a network such as a local area network (LAN).

[00036] The avatar 108 can assume the primary role of a teacher. The avatar 108 can follow a predetermined teaching program (e.g. course, curriculum, lesson, etc.) that is selected by the human teacher 122 using the computer 120. In accordance with the

2018102139 01 Aug 2018 program, the avatar 108 performs any one or more of: presenting theory, asking questions, introducing activities, providing periodical reminders or prompts to the human teacher 122 o conduct assessment and evaluation, and answering predetermined questions. For example, the avatar 108 may ask a predetermined mathematical question, and then provide the correct answer once the student 106 has logged his answer with the touchscreen device 102. Through voice commands the avatar 108 can also find answers to common questions using the world wide web and existing search and browser engines (such as google voice command) to showcase possible answers and results.

[00037] The avatar 108 can also assume the role of a teacher's assistant. The avatar 108 can provide a teacher's notes, and provide additional information to the human teacher 122 upon request. The avatar 108 can provide general guidance to the teacher 122 as well as explain how to conduct teaching activities. Advantageously, the avatar 108 can narrate at least part of a lesson to save the human teacher 122 time and energy.

[00038] The avatar 108 can also directly communicate and socialize with the student 106 through an artificial intelligence system or software. The student 106 can engage with the avatar 108 in either basic conversation, in a two way discussion, or in open ended role play. The role play may be enabled in a virtual world.

[00039] The system 100 can further include a virtual reality headset 124. The avatar 108 can also be viewed by the student 106 wearing the virtual reality headset 124. The headset 124 provides immersive learning whereby, through virtual reality, the student 106 can be immersed into a simulated real world with simulated circumstances. The avatar 108 provides guidance and can assume the form of many different characters within the virtual world.

[00040] The system 100 further includes a remote control 126 for communicating with the controller 114 to control the avatar 108.

[00041] The foregoing avatar functionality is performed using a presentation view of the software.

2018102139 01 Aug 2018 [00042] As shown in Figure 3, the system 100 also includes an editor 300, accessible on teacher’s computer 120, for editing the educational content and the avatar 108. The editor 300 includes a presentation (or lesson) chart 302 displaying parameter icons 304 (i.e. representations) relating to the educational content and the avatar 108. The parameter icons 304 are displayed on the chart 302 relative to a horizontal timeline 306.

[00043] The parameter icons 304 in the chart 302 are located within corresponding category strips 308-316 to facilitate planning of the educational content and the avatar 108. A content strip 308 relates to lesson content, audio (e.g. music) strip 310 relates to audio, avatar speech strip 312 relates to avatar speech characteristics, avatar expression strip 314 relates to avatar expressions and avatar gesture strip 316 relates to avatar gestures.

[00044] The editor 300 includes an icon selector 318 for selecting the parameter icons 304 to be introduced onto the chart 302 and locating the icons 304 in the corresponding category strip 308-316 and along the timeline 306 for execution at a corresponding time. The icon selector 318 includes groups 320-326 of icons 304 generally corresponding to the category strips 308-316. Icons 304 can be dragged by the teacher 122 from the groups 320-326 and dropped into appropriate strips 308-316 at locations corresponding to required times for their execution. The parameter icons 304 are prohibited from being located within unrelated category strips (which are ghosted). As more icons 304 are added to the chart 302, the timeline 306 automatically extends and vice versa.

[00045] The editor 300 further includes actuators 328 for actuating functions relating to the presentation chart 302. The actuators 328 include any one or more of: a new presentation actuator 328a for creating a new presentation chart 302, a open presentation actuator 328b for opening a saved presentation chart 302, save actuator 328c,d for saving a presentation chart 302 and a preview actuator 328e for previewing the execution of the presentation chart 302 whilst remaining within the editor 300.

[00046] The editor 300 further includes an avatar selection tool 330 for selecting the avatar 108 from a number of possible avatars or models. Only a single avatar 108 is typically be used for a particular presentation associated with the chart 302, but in some embodiments the avatar 108 may change during the presentation. The selection tool

2018102139 01 Aug 2018

330 further enables the teacher 122 to build an avatar 108 by varying avatar characteristics such as skin colour, hair styles, hair colour, eye colour, eye shape, etc [00047] The icons 304 can be editable or fixed. For example, a text icon 304 includes one or more text to speech boxes in which customized text is received that can be spoken by the avatar 108 at the appropriate time on the timeline 306. The text can be displayed as closed-caption subtitles. The language can also be customized by the teacher 122, or a translation tool (e.g. Google translate or Bing translate) can provide language translations. Other icons 304, such as video and audio icons, can be associated with a required URL or stored filename for execution at the appropriate time on the timeline 306. A user such as the teacher 122 can record their own voice and associate the recording with an audio icon. A start time of the icon 304 is specified relative the timeline 306, and the length of the execution of the icon 304 is automatically determined.

[00048] The avatar expression and gesture icons are generally fixed, although the duration for the expression can be varied. The icons 304 can be given unique identifying labels.

[00049] As shown in Figure 4, the presentation chart 302 can be saved using the editor 300 including associated parameters stored in a dialog box 400 such as a title, age group of audience, topic, skill and any other teacher notes that can be accessed. The created content or lesson can also be exported as a video file (e.g. mp4).

[00050] Turning to Figure 5, the remote control 126 includes a teacher notes button 500 for accessing the teacher notes. A power button 502 is used to close the current presentation, a load button 504 is used to load a stored presentation, an editor button 506 is used to open the editor 300, and control buttons 508 (i.e. previous, next, pause, play and stop) control the presentation. Toggle switch 510 can be toggled between autoplay off mode where a saved presentation will not start after it has finished loading and autoplay on mode where the presentation will start as soon as it has loaded.

[00051] The help button 512 directs the user to a website’s frequently asked questions page. The horizontal left and right arrow buttons 514 can be used to scroll through various avatars when the presentation has not been loaded. The vertical zoom ίο

2018102139 01 Aug 2018 in and zoom out buttons 516 allow the user to zoom in on the face of the avatar 108 or zoom out to show the whole avatar 108. The zoom button is 516 can be used at any time during the presentation.

[00052] The display 104 displays the educational content whereas the avatar display 110 displays the dynamic avatar 108, when the avatar software is loaded using the load button 504 and played using the control buttons 508. The displayed content and avatar 108 change over time, as various icons 304 are executed on the timeline 306, in accordance with the loaded presentation formed using the editor 300. The teacher 122 or student 106 can create interactive content within the loaded lesson by touching and selecting from a variety of options during the lesson which will result in different results or output. For example in a lesson, the teacher 122 or student 106 can touch a flag to choose the language. Then the content will be presented in that language. A teacher 122 or student 106 can be asked to select a displayed image or button on the screen which will present different pre-determined results.

[00053] The teacher 122 or student 106 can select a repeat actuator to repeat the content or the lesson in a loop.

[00054] Turning to Figure 6, the remote control 126 includes an Artificial Intelligence (Al) button 600 for enabling Al API capability such as Google Assistant, Apples Suri, or Amazon’s Alexa. In this manner, the touchscreen device 102 may be in the form of a smartphone executing android or iOs Application software (an App). The device 102 is effectively a portable viewer for viewing lessons (i.e. content) created using the editor 300 using the personal computer 120.

[00055] The student 106 or teacher 122 can engage with the dynamic avatar 108, resembling a living being, via the App through either voice command or the click of a displayed button on the device 102. In particular, the student 106 or teacher 122 can ask many questions to the avatar 108 such as:

spell the word...

what does the word .... mean?

how far is ....from....?

2018102139 01 Aug 2018 what is .... +.....

what is the square root of.....?

and many other mathematical operations, etc.

who was.....?

where is.....? (example where is Cameroon?)

The possibilities are endless with asking questions about historical events, facts, science, maths, etc.

[00056] The lips of the avatar 108 move when the Al response is elicited. Eliciting a realistic response in this manner is very useful and practical, whereby the student 106 can use the portable device 102 to conveniently do some research by asking appropriate questions. As the portable device 102 is more intimate than the public forum, the student 106 interacting one-on-one with the avatar 108 will improve development of communication skills whether listening, speaking or both, [00057] A person skilled in the art will appreciate that many embodiments and variations can be made without departing from the ambit of the present invention.

[00058] In one embodiment, the display 102 can include a single screen on which the avatar 108 and educational content are both displayed. Turning to Figure 2, an alternative display 200 can include a double screen being a single display 200 partitioned in two by a vertical line 202. The educational content appears on the left screen 204 and the avatar 108 appears on the right screen 206. The double screen can also be formed by connecting two adjacent displays 102 so as to form an extra wide display.

[00059] The touch screen device can include a touch board, TV screen, computer screen, laptop screen, tablet screen, mobile phone screen, projected screen, or other like platform used for teaching.

2018102139 01 Aug 2018 [00060] The system 100 can include a holographic display for displaying the avatar 108 as a holographic image. Further, the system 100 can include a human profileshaped cut-out with the avatar 108 resembling a teacher and displayed on the cut-out. The avatar 108 can resemble a robot or animatronic device to facilitate transfer of technical knowledge to the student 106.

[00061] In one embodiment, physical hand or body gestures of the student 106 or teacher 122 are detected by a motion recognizing camera. An advanced system can be provided for allowing voice recognition commands given to the avatar to enhance the learning experience and easy of lesson delivery.

[00062] In compliance with the statute, the invention has been described in language more or less specific to structural or methodical features. It is to be understood that the invention is not limited to specific features shown or described since the means herein described comprises preferred forms of putting the invention into effect.

[00063] Reference throughout this specification to One embodiment’ or ‘an embodiment’ means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Thus, the appearance of the phrases ‘in one embodiment’ or ‘in an embodiment’ in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more combinations.

Claims

2018102139 01 Aug 2018

The claims defining the invention are as follows:

1. An advanced learning system including:

a touchscreen device including a display which a learner can touch to interact with educational content; and a dynamic teaching avatar to teach the learner about the educational content with which the learner is interacting, wherein the avatar is configured to dynamically embody a plurality of expressions or gestures.
2. An advanced learning system as claimed in claim 1, further including an editor for editing the educational content and the avatar.
3. An advanced learning system as claimed in claim 2, wherein the editor includes a presentation chart displaying parameter representations relating to the educational content and the avatar, the parameter representations displayed relative to a timeline.
4. An advanced learning system as claimed in claim 2 or claim 3, wherein the editor further includes a selector for selecting the parameter representations to be introduced onto the chart and locating the representations along the timeline for execution at a corresponding time.
5. An advanced learning system as claimed in claim 4, wherein the timeline automatically varies depending upon the number of parameter representations on the chart.
6. An advanced learning system as claimed in any one of claims 3 to 5, wherein the parameter representations in the chart are located within corresponding categories to facilitate planning of the educational content and the avatar.
7. An advanced learning system as claimed in claim 6, wherein the parameter representations are prohibited from being located within incongruous categories.
8. An advanced learning system as claimed in any one of claims claim 3 to 7, wherein the parameter representations include any one or more of: content representations relating to lesson content, audio representations relating to audio, avatar speech representations relating to avatar speech characteristics, avatar

2018102139 01 Aug 2018 expression representations relating to avatar expressions and avatar gesture representations relating to avatar gestures.
9. An advanced learning system as claimed in any one of claims 2 to 8, wherein the editor further includes actuators for actuating functions relating to a presentation chart, the actuators including any one or more of: a new presentation actuator for creating a new presentation chart, a open presentation actuator for opening a saved presentation chart, save actuator for saving a presentation chart and a preview actuator for previewing the execution of the presentation chart.
10. An advanced learning system as claimed in any one of claims 2 to 9, wherein the editor further includes an avatar selection tool for selecting the avatar from a number of possible avatars, or for building an avatar by varying avatar characteristics.
11. An advanced learning system as claimed in any one of claims 2 to 10, wherein the editor includes one or more text to speech boxes in which text is received that is spoken by the avatar, the editor optionally able to be configured to provide language translations or subtitles.
12. An advanced learning system as claimed in any one of claims 1 to 11, further including a remote control for controlling the avatar.
13. An advanced learning system as claimed in any one of claims 1 to 12, wherein the display includes either: (1) a single screen on which the avatar is displayed; or (2) a double screen, being either a single display partitioned in two or two displays adjacent each other, with the educational content appearing on one screen and the avatar appearing on the other screen.
14. An advanced learning system as claimed in any one of claims 1 to 12, further including another display separate from the touch screen device for displaying the avatar, the other display able to be positioned anywhere in a classroom, but preferably in the front, whereby the avatar assumes the role of an instructor.
15. An advanced learning system as claimed in any one of claims 1 to 14, wherein the system includes a virtual reality headset with the avatar being viewed through the virtual reality headset, the headset providing immersive learning whereby, through

2018102139 01 Aug 2018 virtual reality, the student can be immersed into a simulated real world with simulated circumstances.
16. An advanced learning system as claimed in anyone of claims 1 to 15, including a controller for controlling the avatar, the avatar able to be controlled through voice commands, voice prompts, physical hand or body gestures detected by a motion recognizing camera, or simply selecting options displayed on the touchscreen device or other user interface.
17. An advanced learning system as claimed in any one of claims 1 to 16, further including a computer, with which a human teacher can select avatar programs in a database that relate to the educational content.
18. An advanced learning system as claimed in claim 17, wherein the avatar can assume the role of a teacher and follow a predetermined teaching program that is selected by the human teacher, the avatar performing any one or more of: present theory, ask questions, introduce activities, provide periodical reminders or prompts to the human teacher to conduct assessment and evaluation, answer predetermined questions and find answers to common questions using the world wide web and existing search and browser engines to showcase possible answers and results.
19. An advanced learning system as claimed in claim 17, wherein the avatar can assume the role of a teacher's assistant, the avatar providing any one or more of: a teacher's notes, additional information to the human teacher upon request, general guidance to the teacher as well as explaining how to conduct activities, and narration of at least part of a lesson to save the human teacher time and energy.
20. An advanced learning system as claimed in any one of claims 1 to 19, wherein the avatar can communicate and socialize with the learner, the leaner able to engage with the avatar in either basic conversation, in a two way discussion, or open ended role play in a virtual world.
21. An advanced learning system as claimed in any one of claims 1 to 20, wherein the touchscreen device is portable and includes an Al App for providing responses from the avatar to queries posed by the learner.

2018102139 01 Aug 2018
22. An advanced learning system as claimed in claim 21, wherein the avatars lips move when the Al responses are provided.
23. An advanced learning system as claimed in any one of claims 1 to 22, wherein the educational content can be repeated in a loop.
24. An advanced learning method including:

enabling a learner to touch and interact with educational content on a display of a touchscreen device; and providing a dynamic teaching avatar to teach the learner about the educational content with which the learner is interacting, wherein the avatar is configured to dynamically embody a plurality of expressions or gestures.
25. An advanced avatar system including:

a touchscreen device including a display which a learner can touch to interact with content; and a dynamic avatar used in conjunction with the content with which the learner is interacting, wherein the avatar is configured to dynamically embody a plurality of expressions or gestures.