WO2022263720A1

WO2022263720A1 - Smart learning system, method and computer program product

Info

Publication number: WO2022263720A1
Application number: PCT/FI2022/050412
Authority: WO
Inventors: Roope Rainisto; Janne Jormalainen
Original assignee: New Nordic School Oy
Priority date: 2021-06-14
Filing date: 2022-06-14
Publication date: 2022-12-22

Abstract

Receiving data from applications executed in connection with offline lessons and online lessons. Based on the received data, learningprogress, a learning progress speed, skill level, a learning methodpreference, attentiveness, emotional state, language knowledge,interaction with other students, learning material performance andschedule performance is tracked. Determining a lesson for teachingat least a part of the one or more subjects to at least a part of thestudents. Determining based on a lesson for teaching a subject tostudents and the tracked data, at least one combination of learningmethods and learning materials for each student. Determiningbased on the determined at least one combination of learningmethods and learning materials for each student, a grouping of saidat least a part of the students for the lesson. Providing, to each groupdetermined based on the grouping, based on receiving a user inputindicating acceptance of the determined grouping of said at least apart of the students for the lesson, a group-specific lesson plan.

Description

SMART LEARNING SYSTEM, METHOD AND COMPUTER PROGRAM PRODUCT

[0001] The present invention relates to a smart learning system and particularly to a smart learning system for online and offline learning.

Background

[0002] Distance learning became a topic during a global pandemic forcing children to participate school from their homes. However, despite the pandemic, home school and distance learning have always been a target of interest to educate school-age children. Sometimes, distance learning would be desired for short periods of time, for example when a child is too sick to participate classroom learning, but well enough to follow lessons remotely. For others, remote learning can save time and therefore increase learning efficiency. For example, when a student lives on a distance from the school, the student doesn’t have to spend a long time travelling each day. Instead, the time spent in traveling can be used for learning. Each student has individual preferences in relation to their learning. Some of the students may learn a particular subject matter better if they can study the subject at their own time and pace, using an intelligent learning system that provides materials and exercises that best help them to learn.

[0003] In classroom learning, the teacher can conduct individual lessons according to a lesson schedule and control the teaching pace of each subject such that the lessons can be conducted according to the lesson schedule. Additionally, the teacher can adapt the lesson contents such that the students could achieve a target skill level at the end of the lesson.

[0004] Total time to teach a particular subject to students can have individual differences between students. If both distance learning and classroom learning are both used to teach a subject to students, direct interaction, i.e. face-to-face communications, between a teacher his/her students is decreased and replaced at least to some extent by the students interacting with educational applications which decrease the interaction with fellow students and the teacher. Due to the differences of the teaching times to teach the subjects to the students and without necessarily having direct interaction between students and the teacher, students who are progressing fast can be idling since the teacher is unaware of a need of the students for new tasks and students who are progressing slows can have difficulties in keeping up with the pace of the lessons without the teacher being aware of this.

Summary

[0005] The scope of protection sought for various embodiments of the invention is set out by the independent claims. The embodiments, examples and features, if any, described in this specification that do not fall under the scope of the independent claims are to be interpreted as examples useful for understanding various embodiments of the invention.

[0006] According to a first aspect there is provided smart learning system according to claim 1 .

[0007] According to a second aspect there is provided a method according to claim 11 .

[0008] According to a third aspect there is provided a computer program comprising instructions which, when the program is executed by a processor of, to perform at least the method of claim 11 .

[0009] According to a fourth aspect there is provided a non-transitory computer readable medium comprising program instructions for causing an apparatus, arrangement or a smart learning system, to perform at least the method of claim 11 .

[0010] Some further aspects are defined in the dependent claims. The embodiments that do not fall under the scope of the claims are to be interpreted as examples useful for understanding the disclosure.

Brid description of the Drawings

[0011] In the following, various embodiments will be described in more detail with reference to the appended drawings, in which Figure 1 illustrates an example of a smart learning system according to an embodiment;

Figure 2 illustrates examples of devices participating to the overall system operation according to an embodiment;

Figure 3 illustrates an example of a device according to an embodiment; Figure 4 shows an example of a hybrid learning environment according to an embodiment; and

Figures 5 to 10 illustrates examples of methods according to embodiments. Description of Example Embodiments

[0012] The following description and drawings are illustrative and are not to be construed as unnecessarily limiting. The specific details are provided for a thorough understanding of the disclosure. Flowever, in certain instances, well- known or conventional details are not described in order to avoid obscuring the description. References to one or an embodiment in the present disclosure can be, but not necessarily are, reference to the same embodiment and such references mean at least one of the embodiments.

[0013] Reference in this specification to “one embodiment” or “an embodiment” means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the disclosure.

[0014] Use of ordinal terms such as “first,” “second,” “third,” etc., in the claims and description to modify a described feature does not by itself connote any priority, precedence, or order of one described feature over another or the temporal order in which acts of a method are performed, but are used merely as labels to distinguish one described feature having a certain name from another described feature having a same name (but for use of the ordinal term) to distinguish the described feature.

[0015] The verbs “to comprise” and “to include” are used in this document as open limitations that neither exclude nor require the existence of un-recited features. The features recited in dependent claims are mutually freely combinable unless otherwise explicitly stated. Furthermore, it is to be understood that the use of “a” or “an”, i.e. a singular form, throughout this document does not exclude a plurality.

[0016] A lesson plan is a teacher's guide for facilitating a lesson. It typically includes the goal that defines what students need to learn, how the goal will be achieved, e.g. the method of delivery and procedure, and a way to measure how well the goal was reached, e.g. usually via homework assignments or testing. Group-specific lesson plan is a lesson plan based on a combination of one or more learning methods and one or more learning materials determined for the group. In an example, students in the group have the same combination of one or more learning methods and one or more learning materials for the lesson, whereby the activities, e.g. schedules of learning methods within a lesson may be synchronized for the students in the same group.

[0017] A schedule for a lesson comprises a start time and end time of the lesson and a date of the lesson. Group-specific schedule for a lesson enables scheduling the lesson for the group at a time that facilitates learning the subject of the lesson b the student in the group.

[0018] A learning method may be an online learning method or an offline learning method. Students may have preferences for learning methods, e.g. learning by reading, hearing, seeing videos, doing exercises, doing hands-on, trying simulations, talking with others.

[0019] An offline learning method comprises a learning method for a student to learn a subject by directly interacting with the teacher and/or performing an activity that does not require a data transfer connection, e.g. an internet connection. The direct interaction between the student and the teacher may comprise that the teacher and the student interact face-to-face for example in a classroom environment. Therefore, communications between the student and the teacher may be performed without necessarily having any devices for encoding and decoding their communications. Examples of offline learning methods comprise, reading a book, writing notes to a notebook, listening to offline audio, watching offline video, the students following educational content presented by the teacher on a blackboard or a display device, live discussions, live presentations by the students, live presentation by the teacher, doing craftworks.

[0020] Offline learning material comprises an offline video, an offline audio and/or textual content such as literature that may be consumed without a data transfer connection, e.g. an internet connection. Additionally, the offline learning material may be material for craftworks, e.g. wood, thread or fabric. The offline learning material may be also learning material generated during a lesson, e.g. by a student or teacher. The offline learning material may be generated in a classroom environment and used for learning a subject without a data transfer connection, e.g. an internet connection. In an example, the offline learning material generated during the lesson may comprise a drawing drawn on a blackboard during the lesson or a formula or text written on the blackboard, or notes written down during a lesson.

[0021] An online learning method comprises, a student using an educational software on a student user device, participating in a discussion on an online platform (e.g. chat), participating in a group work on an online platform. An example of an educational software is a language learning application or a chat application for collaboration and discussions.

[0022] Online learning material comprises an educational application, video, audio and/or textual content such as literature that is retrieved from an online service/platform that may be connected over a data transfer connection. Examples of the educational application comprise a chat application, a language learning application and an electronic book.

[0023] A learning activity is a combination of a learning material and a learning method, e.g. a combination of an online/offline learning method and an online/offline learning material.

[0024] An online lesson comprises a lesson utilizing one or more online learning methods and one or more online materials for teaching one or more students at least one subject. [0025] An offline lesson comprises a lesson utilizing one or more offline learning methods and one or more offline materials for teaching one or more students at least one subject.

[0026] A hybrid lesson may utilize one or more online learning methods, one or more online materials, one or more offline learning methods and one or more offline materials for teaching one or more students at least one subject. In an example, the hybrid lesson comprises at least one offline lesson portion and at least one online lesson portion. The offline lesson portion may be similar to an online lesson and the offline lesson portion may be similar to an offline lesson with the difference that the online lesson portion and offline lesson portion are a part of a lesson plan of the hybrid lesson and their scheduling is dependent on the scheduling of the hybrid lesson.

[0027] The present embodiments are related to a smart learning system, also referred to as smart learning platform, an example of which is illustrated in Figure 1 . The smart learning system is a digital tool for assisting learning in hybrid learning environment comprising face-to-face learning, i.e. classroom learning environment, and digital learning occurring online or offline. The smart learning system uses data from several sources to optimize the learning between these two learning environments.

[0028] The smart learning system 100 generally comprises a first component 110 for classroom learning and a second component 120 for distance learning. The second component 120 for distance learning can be utilized in both online and offline learning mode. Both the first and the second components comprise various computer modules to carry out functionalities of the smart learning system. The modules and their respective functionalities are discussed in more detailed manner later. In the following the first component 110 may be referred to an offline learning component or classroom learning component, since the offline learning component provides communications with applications executed in connection with one or more offline lessons for teaching one or more subjects to students based on one or more offline learning methods. In the following the second component 120 may be referred to an online learning component or a remote learning component, since the online learning component provides communications with applications executed in connection with one or more online lessons for teaching said one or more subjects to students based on one or more online learning methods and one or more online learning materials. The smart learning system 100 further comprises a lesson planner component 132 for scheduling one or more hybrid lessons for teaching subjects to students. The smart learning system 100 comprises a machine learning component 130, or a machine learning model, operatively connected to the offline learning component and online learning component for communication with the applications, e.g. at least for receiving data from the applications for tracking the offline learning and/or online learning, e.g. for tracking one or more of a learning progress, a learning a progress speed, a skill level, a learning method preference, attentiveness, emotional state, language knowledge, interaction with other students, learning material performance, and schedule performance. The lesson planner component may be operatively connected to the machine learning component 130 to receive the tracked data for determining learning methods, learning materials, grouping of students, group-specific lesson plans and/or group- specific schedules for lessons, and for providing one or more of the determined grouping of students, group-specific lesson plans and group-specific schedules for lessons to students.

[0029] The smart learning system 100 may communicate with various client applications 99 through an interface 105. The client applications may be independent third-party applications that are used as learning and/or teaching platforms by students and teachers. One or more of such applications 99, or at least their respective user interfaces, may be located on client devices, e.g. student user devices and/or teacher user devices. An application may be any educational application or a platform that may be used for receiving schoolwork and educational material from a teacher user, and completed homework and other school-related data from a student user. Said applications communicate with the smart learning system 100, whereupon the smart learning system 100 may analyze the learning curve of the student and control and adjust the educational material of the teacher.

[0030] An offline learning method may comprise direct person to person interaction for learning by way of a teacher providing e.g. video, audio, presentation material and/or questionnaires to one or more students in a classroom or equivalent physical surroundings for teaching the one or more students a subject, and the students answering questions of the teacher and/or the students asking help assistance from the teacher for learning the subject. [0031] An application executed in connection with an offline lesson may provide educational content related during the offline lesson. The educational content may comprise video, audio, presentation material and/or questionnaires, for example. Additionally or alternatively, the application executed in connection with an online lesson may be configured to track and store data relating to one or more student’s actions during the offline lesson. In an example, the tracked and stored data may comprise video and audio recordings of a student and user interface activity of a student device used to execute the application.

[0032] An online learning method may comprise a student interacting with an application configured to provide educational content for learning a subject. The direct interaction with the student is performed by the application. The teacher may interact with the student via the same application that provides the educational content or a different application may be provided for the communications between the teacher and the student.

[0033] An application executed in connection with an online lesson, i.e. online learning application, may be configured to provide educational content related to the online lesson. The online learning application may be executed in accordance with an online lesson that is conducted in accordance with a lesson plan. The educational content may comprise video, audio, presentation material and/or questionnaires. Additionally or alternatively, the application executed in connection with an online lesson may be configured to track and store data relating to user’s actions during the online lesson. In an example, the tracked and stored data may comprise video and audio recordings of a student and user interface activity of a student device used to execute the application.

[0034] Examples of applications that are configured to provide educational content comprise at least language learning applications that may be executed on mobile device or desktop computer platforms. Examples of applications for communications between the teacher and the student comprise at least Teams provided by Microsoft and Zoom provided by Zoom video Communications. [0035] The data that is received from the applications may be processed by the machine learning component 130. The machine learning component 130 operates with the various modules of the smart learning system to generate intelligent decisions based on the received data according to the needs of said modules. The machine learning component may implement a machine learning algorithm, which may be based on deep learning. For example, a (deep) neural network - as an example of a machine learning algorithm - comprises an input layer and an output layer, and a plurality of hidden layers therein between. Each of the layers comprises units that are configured to implements an activation function based on an input it receives. The output of the activation function is forwarded to the units of the next layer, if the output exceeds a certain threshold.

[0036] The machine learning component 130 can be referred to as “global machine learning service”. In addition to the global machine learning service, there can be a ML algorithm located on a student user device, and a ML algorithm located on teacher’s device. These ML algorithms are referred to as “local machine learning algorithms”. The operation of these algorithms is discussed later in this description.

[0037] The operation of any of the machine learning algorithms is based on a training, which is performed by using a training dataset comprising data that is representative of the actual data being received by the machine learning algorithm. The purpose of the training is to find parameters (also referred to as “weights”) by means of which the loss of the algorithm will be minimized (optimization). The training can occur in a supervised manner or unsupervised manner. In the context of smart learning, the data having been gathered through years can be used as a training data set. Such data comprises digital educational material; exams; assignments; historical data on students’ input to such exams and assignments; students’ progress on the educational material; students’ feedback on the lessons; etc. In addition to the historical data, also data that is gathered during use of the smart learning system for example during lessons conducted using the smart learning system, is continuously fed to the machine learning algorithm to enable continuous learning of the algorithm. The training dataset may comprise video and audio recordings of a student (online learning or offline learning, e.g. near a computer or at the classroom), student user device activity during class and corresponding labels (i.e., was the student focused at a particular time frame), learning materials of lessons, assessment of student skills before and after one or more lessons, learning methods of lessons, heart rate, physical activity data. Dataset may be anonymized in order to protect privacy of the students. The dataset may be for example in Extensible Markup Language (XML) format.

[0038] Figure 2 illustrates devices participating to the overall system operation. The smart learning platform with the first and second components can be arranged into a server 200. The smart learning platform is connectable by a teacher user device 210 and a plurality of student user devices 220. The server 200 may comprise, or be in connection also with, a machine learning model 230 and a database 240. The teacher user device 210 and the plurality of student user devices 220 may communicate with the server 200 through first and second components (as discussed with reference to Figure 1). According to an embodiment, the teacher user device 210 uses the first component for the classroom learning for carrying out the teaching in the classroom. According to an embodiment, the teacher user device 210 uses the second component for distance learning for providing teaching over network. The student user device 220 uses the second component when the student is participating the lesson online or offline. According to an embodiment, the student user device 220 has a first component to be utilized when participating the lesson in the classroom.

[0039] A student user device 220 can be a smart phone, a tablet device, a laptop computer a personal computer or any other similar device. A teacher user device 210 may be a smart phone, a tablet device, a laptop computer, a personal computer or any similar device. In addition to the student and teacher user devices, the system may comprise tracking devices, such as video and/or still cameras, and/or microphones, and/or audio recording means. The student user device 220 is usable by a student while studying remotely, but sometimes also when studying at the classroom. The student user device 220 may comprise a local ML algorithm by means of which data from devices (watches, video camera, microphone, navigation tracker, etc.) is synchronized. The local ML algorithm is configured to locally determine how focused the respective student is at the moment, and to send statistics to the global ML service. [0040] The teacher user device 210 is also usable at the classroom environment, but the teacher is also able to use the teacher user device 210 outside the classroom. The teacher user device 210 may comprise a local ML algorithm to track students by using cameras/microphones installed in classroom, and to send statistic to the global ML service.

[0041] The tracking devices may be integrated to the student user device and/or teacher user device, but the system may contain independent tracking devices being installed in classrooms or student’s room at home.

[0042] Figure 3 illustrates an example of a user device 300. The user device can be a teacher user device or a student user device for the purposes of a system shown in Figure 2. The device 300 comprises a processor, a memory, a communication interface and a user interface. The memory comprises a computer program code for causing the device to carry out various functionalities. The memory may also contain other data concerning one or more of the following: teacher’s profile (age, gender, educational background, experience, Professional Degree history, work time management/follow-up); students’ profile (age, gender, educational history, special education needs, other medical history); school’s profile (country, location, size, curriculum, co ed/segregated, learning environment specifications, specialized teachers); class’ profile (number of students, students, student-teacher ratio); planned learning activities (project topics, standards, subject-matter, level, duration, learning material, teachers, students); project based learning modules (Sustainable Development Goals, standards, subjects, core competences, content areas, desired end product, assessment, differentiation, activities to reach the objectives, materials); learning activity (time, duration, action of the teacher/student); observation/assessment (time, duration, assessment result), hardware used for learning (how much, when, where); software used for learning (how much, when, where); self-evaluation data or everyone in the organization.

[0043] The communication interface enables wired or wireless short or long range communication with other devices. Examples of the communication networks comprises any data transfer technology, such WLAN (wireless local area network), wireless mobile networks of different generations (3G - 5G and forward); LAN (local area network); etc. Data transfer networks that are utilized by the smart learning system comprises the available or the future technologies, and therefore they are not discussed further in this specification. [0044] The user interface comprises means for a user (e.g. a teacher or a student) to provide data to the system and to view data from the system. The user interface may be a graphical user interface, e.g. a dashboard view, being tailored for certain user roles. The user interface is able to show data retrieved from a database, and receive inputs from users. The user interface comprises graphical elements to be used for example in connection of accomplishing assignments or of requesting assistance. In addition, the user interface may display or play recordings made in the classroom.

[0045] The main aspect of the hybrid learning in which the teacher user device and the student user device and the smart learning platform operate, is the independence of time and of a physical place. In general, when time and place are taken into consideration, the learning may be 1 ) “classic learning” occurring at the same time, at same physical location;

2) “remote learning” occurring at the same time, at different physical locations;

3) “asynchronous learning” occurring at different times; at different physical locations;

4) a learning occurring at different times, at same physical location.

[0046] With respect to an idea of hybrid learning, options 2 and 3 come up. Therefore term “hybrid learning environment” refers to following learning situations:

1 ) a situation, where some of the students are located remotely from the class environment, and where some students are located in the class. Thus, the hybrid class environment combines both classroom learning and distance learning (“remote learning”) simultaneously.

2) a situation, where all students are self-learning at different locations and at different times (“asynchronous learning”).

[0047] These aspects create a few challenges relating to student monitoring, material selection, student co-operation, student preferences, learning method selection, learning material selection and lesson planning.

[0048] Student monitoring is possible in a conventional, i.e., classic, class learning environment. In such situation, a teacher is able to monitor students to see how they participate and follow the lesson. The teacher is also able to react to problems, for example, when a student is being stuck. For the remote participants, i.e. students participating to distance learning online, the student monitoring is much harder, especially if the distance learning occurs at the same time with the classroom learning. The harder the monitoring becomes, when the student is in the distance learning environment, but in offline mode. In such a situation, the teacher has basically no means to evaluate student’s engagement to the self-learning.

[0049] With respect to the material selection, there are different methods, materials and assessments to be used for remote learning and classroom learning. At the classroom learning, the particular learning objective can be achieved by using a material A and learning method B. However, if the student is participating remotely, a different set of material X and self-learning method Y may be a better combination. Therefore, the challenge occurring at the hybrid learning environment, is to select these materials. Student co-operation is also an issue raising from a remote learning. In all learning environments, people learn also from each other, not just from the teacher. This is possible at the classroom environment. However, in a hybrid environment, the students should also be able to learn from each other by communicating, by participating and by sharing views.

[0050] With respect to the learning method selection and learning material selection, the teacher may plan a lesson for teaching a subject using any of the learning situations 1 ) or 2) or a combination of the learning situations. On the other hand the students may have their own preferences for learning methods. Selection of the learning method, e.g. online learning (the distance learning) or offline learning (the classroom learning or remote learning offline) should take advantage of the various learning situations provided by the hybrid learning environment, while still enabling the teacher to control the selected learning method for progressing the learning of the subject to the students at a suitable learning pace within a time window reserved for the learning, e.g. within a lesson duration. Accordingly, while the various learning situations of the hybrid learning environment are available, their application to groups of students and individual students should be controlled in order to provide a satisfactory learning performance for the students within a time window reserved for the learning.

[0051] Thus, the present solution is targeted to at least the following issues occurring in a hybrid learning environment:

1 ) how to teach students simultaneously, when some of the students are located in a classroom and some of the students participate remotely;

2) how to correct gaps in learning resulting from a temporal distance learning when a student returns to classroom learning environment; 3) how to combine distance learning and classroom learning environment for a single student so that both learning situations support each other as well as possible;

4) how to provide instructions to a student at the distance learning environment, when the student is offline and/or not performing the studies at the same time with the students at the classroom learning environment;

5) how to support learning performance of students who attend hybrid lessons; and

6) how to support learning conducting hybrid lessons for teaching a subject within time window reserved for the learning the subject.

[0052] In order to tackle the matter, the machine learning model according to present embodiments is configured to track one or more of a learning progress, a learning progress speed, a skill level, a learning method preference, attentiveness, emotional state, language knowledge, interaction with other students, learning material performance, and schedule performance. The tracking may be based on the machine learning model receiving data generated by applications executed in connection with one or more offline lessons for teaching one or more subjects to students based on one or more offline learning methods and one or more offline learning materials and data generated by applications executed in connection with one or more online lessons for teaching said one or more subjects to students based on one or more online learning methods and one or more online learning materials. The data may be received by the machine learning model with the help of the online learning component and the offline learning component. It should be noted that both the remote students and students in the classroom, i.e. classroom students, may be tracked, whereby the learning progress, learning progress speed, skill level, a learning method preference, attentiveness, emotional state, language knowledge, interaction with other students, learning material performance, and schedule performance of the remote students may be compared between the remote students, between the classroom students and remote students, and between the classroom students. Based on the comparison, the machine learning model may determine performance metrics of teaching at least a part of the one or more subjects to at least a part of the students, whereby the determined performance metrics may indicate whether one or more individual students or groups of students that need support, whether the students are on schedule and/or a grouping of the students. The grouping of the students may at least comprise that any remote students may be grouped into one or more groups, i.e. remote student groups, and any students in the classroom may be grouped into one or more other groups, i.e. classroom student groups. The grouping of the remote students provides that the groups may be assigned group-specific schedules for lessons and/or group-specific lesson plans for teaching the groups at least a part of the one or more subjects within a time window reserved for the learning.

[0053] In an example, the grouping of the students may be determined based on the performance metrics, whereby students that have similar performance metrics may be assigned to the same group. In an example each group may be defined based on value ranges of the performance metrics of the students. The performance metrics of the students may be evaluated after the comparing of the tracked data between students. The evaluation of the performance metrics may provide determining the grouping of the students based on their performance metrics into a given number of groups. Accordingly, each group may be defined by a number of students assigned to a group and a value range of performance metrics of the students assigned to the group. In this way each group may be assigned a sufficient number of students while controlling the students that can be assigned to the group based on whether the students’ performance metrics are within a value range of the group. In this way the grouping may provide similarity between students. For example, the students may be grouped based on skill levels of the students ranging from 1 to 5 in integer values, whereby 1 first group may be for students that have skill levels 1 and 2, a second group may be for students that have skill levels 3 and 4 and a third group may be for students that have a skill level. As a further requirement, each group may be assigned a minimum and a maximum number of students for controlling the number of students assigned to each group. The minimum and maximum number of students may be the same for all groups or the minimum and the maximum number of students may be different for each group.

[0054] In an example attentiveness may be tracked by the machine learning model based on observing the actions and behavior of at least the remote students through the software on their computer. The attentiveness may indicate on how the remote students are focusing on the assignments or is their focus targeted somewhere else (e.g. some social media application). In addition, the local machine learning algorithm at the student’s device is configured to analyze the emotional state of the remote students by using a webcam capturing as means to obtain data on facial expressions of the remote student and/or by using other sensors that are capable of measuring physical and psychical parameters from a student. The emotional state of the student may be quantified into e.g. focused, anxious, happy and sad.

[0055] According to an example, the machine learning model also observes the actions and behavior of the students through their computers and/or through an additional monitoring system comprising various data recording devices. The observation is targeted to students’ focus and attentiveness at the classroom environment.

[0056] The system according to present embodiments may be configured to continuously give an overview of students’ progress, at least when being remote, and pays more attention on students that are stuck in an assignment, proceeding too slowly, not focusing or doing some other things etc. Such observations may be corrected automatically by the system, and/or indicated to a teacher or to the student, and/or stored to a memory for further analysis. [0057] Figure 4 illustrates an example of a hybrid learning environment according to an embodiment. As shown in Figure 4, the smart learning system 450 is applicable in a classroom learning environment 400 and a distance learning environment 405. The classroom learning environment 400 refers to a physical environment, where teacher and students operate. The classroom learning environment 400 comprises at least a teacher user device 401 , and in some embodiments, also a plurality of student user devices. In addition, the classroom learning environment comprises a camera 402 with a microphone for recording video/audio/image material on the classroom learning. The distance learning environment 405 comprises a student user device 406 having a camera 407 and one or more applications 408, 409 executed on the student user device 406.

[0058] The smart learning system 450 may comprise an online learning component 453, an offline learning component 456, a lesson planner component 455, and an interface 457 for communications of data with one or more applications executed on a teacher user device 401 and/or a student user device 406. It should be noted that although the interface is illustrated as a separate entity, the interface may also be included to the online learning component 453, offline learning component 456 and the lesson planner component 455, whereby a separate interface may be omitted. The online learning component 453 is configured to receive a first data set generated by one or more applications 408, 409 in connection with execution of said applications 408, 410 for teaching one or more subjects to students based on one or more online learning methods. In an example, at least one of said applications is an educational application for teaching one or more subjects and at least one other of said applications may be a monitoring application. The monitoring application is configured to track and store data relating to user’s actions on a student user device 406. The online learning component 453 is configured, based on the received first data set to track for example one or more of the following: a learning progress, a learning a progress speed, a skill level, a learning method preference, attentiveness, emotional state, language knowledge, interaction with other students, learning material performance, and schedule performance of the student.

[0059] In addition, the offline learning component 453 may be configured to receive a second data set generated by one or more applications 410 in connection with execution of said applications 410 for teaching one or more subjects to students based on one or more offline learning methods. At least one of said applications may be an educational application for teaching one or more subjects and at least one other of said applications is a monitoring application. The monitoring application is configured to track and store data relating to user’s actions in the classroom learning environment 400. The offline learning component 453 is configured, based on the received second data set to track for example one or more of the following: a learning progress, a learning a progress speed, a skill level, a learning method preference, attentiveness, emotional state, language knowledge, interaction with other students, learning material performance, and schedule performance of the student.

[0060] Accordingly, the second data set may comprise data recorded from the classroom learning environment 400, in particular from the one or more applications 410 being executed on the teacher user device 401 and/or a camera 402. On the other hand, the first data set may comprise data recorded from the distance learning environment 405, in particular from the one or more applications 408, 409 being executed on the student user device 406. The first data set may comprise data recorded from a plurality of student devices. [0061] The lesson planner 455 component may be connected to both the offline learning component and the online learning component for implementing one or more examples described herein. Therefore, the first component tracks and analyzes the learning progress occurring at the classroom learning environment, and the second component tracks and analyzes the learning progress occurring at the distance learning environment. Both the first and the second component may utilize the machine learning model by providing corresponding input data to the machine learning model. [0062] Figure 5 illustrates an example of a method. The method may be performed by a smart learning system described in Fig. 1 , e.g. by a machine learning component of the smart learning system. The smart learning system comprises an offline learning component for communications with applications executed in connection with one or more offline lessons for teaching one or more subjects to students based on one or more offline learning methods, an online learning component for communications with applications executed in connection with one or more online lessons for teaching said one or more subjects to students based on one or more online learning methods and one or more online learning materials, a lesson planner component for scheduling one or more hybrid lessons for teaching subjects to students, wherein a hybrid lesson comprises at least one offline lesson portion and at least one online lesson portion, and a machine learning component operatively connected to the offline learning component, the online learning component and the lesson planner component.

[0063] Phase 502 comprises receiving data generated by applications executed in connection with one or more offline lessons for teaching one or more subjects to students based on one or more offline learning methods and one or more offline learning materials and data generated by applications executed in connection with one or more online lessons for teaching said one or more subjects to students based on one or more online learning methods and one or more online learning materials.

[0064] Phase 504 comprises tracking based on the received data from the offline learning component and the online learning component, one or more of a learning progress, skill level, a learning method preference, attentiveness, emotional state, language knowledge, interaction with other students, learning material performance, and schedule performance.

[0065] Phase 506 comprises determining a lesson for teaching at least a part of the one or more subjects to at least a part of the students.

[0066] Phase 508 comprises determining based on the determined lesson and the tracked one or more of a learning progress, skill level, a learning method preference, attentiveness, emotional state, language knowledge, interaction with other students, learning material performance, and schedule performance, at least one combination of learning methods and learning materials for each student of said at least a part of the students for learning said at least a part of the one or more subjects, wherein the at least one combination of learning methods and learning materials comprises one or more of offline learning methods, one or more offline learning materials, one or more online learning methods and one or more online learning materials. [0067] Phase 510 comprises determining based on the determined at least one combination of learning methods and learning materials for each student, a grouping of said at least a part of the students for the lesson.

[0068] Phase 512 comprises providing, to each group determined based on the grouping, based on a user input received by the lesson planner component indicating acceptance of the determined grouping of said at least a part of the students for the lesson, a group-specific lesson plan.

[0069] For carrying out the phases 502 and 504, online learning component and offline learning component may be connected by a data transfer network with student user devices, a teacher user device and monitoring devices in order to receive and track data regarding online learning and offline learning. The user devices, a teacher user device and monitoring devices may be deployed in different learning environments, for example a distance learning environment and a classroom learning environment. The online learning component and offline learning component may control various functionalities and applications at the student user devices and the monitoring devices to enable them to provide data concerning the activity of the students. In addition, the online learning component and offline learning component may receive data on students’ pointer movements on displays of the student user devices. The online learning component functions with the student user device in both simultaneous and non-simultaneous learning, wherein simultaneous refers to learning occurring at the same time with the classroom learning and non- simultaneous refers to learning occurring outside normal lesson hours. For example, the online learning component and offline learning component may be configured to receive data from an educational software used by a student on a student user device. The received data may comprise data indicating which page the student is at the moment, how long the student stays on certain pages, which assignments have been completed, which assignments have been skipped, how long the student stays on a certain assignment, which actions does he/she take, etc. In addition, the online learning component and offline learning component may receive data from other applications or web sites being opened at the student user device, i.e. which application is currently active, where the student is inputting data, etc. In addition, the online learning component and offline learning component may receive data from student user device’s camera equipment to analyze facial expressions, eye movement, overall pose, etc. In addition, the online learning component and offline learning component may receive data from the student user device’s audio capturing means to analyze sounds and words made by the student. In addition, the online learning component and offline learning component may be able to receive data on student’s wearable device, either directly or through corresponding application at the student user device. Such wearable devices may comprise a smartwatch, an activity wrist device, a ring, or other health tracker, etc. Image and video detection neural network may be utilized to extract bounding boxes on student’s faces, to identify student’s emotions, which in combination with a set of features relating to the persons closeness to the screen, to the followed page on the education material, to the input devices the student is using at the moment, will reveal whether the student is focused on a lesson. By this, the machine learning algorithm may determine the level of concentration and tension when communicating with the teacher, and also to determine the level of involvement in the learning process.

[0070] In an example, when carrying out the phases 502 and 504, the online learning component and/or the offline learning component may also be configured to receive data on students in a classroom learning environment. Such tracked data comprises observed actions and behavior of students in the classroom, which have been obtained by capturing images and/or video on students. The interest is on where the students sit, what and where are they looking at, and what are they doing. [0071] The data that is gathered from the classroom learning environment may be used as basis when determining the phase of learning, or a learning progress for the remote students, i.e. defining a threshold value for the phase of learning. Since the teacher is acting in the classroom learning environment physically, s/he is able to see the progress of the students, and adjust the teaching accordingly. Therefore, it is important to indicate whether there are remote students that are not following anymore.

[0072] In an example, phases 506, 508 and 510 may be carried out by the lesson planner component. The lesson planner component provides scheduling one or more lessons, e.g. hybrid lessons, for teaching at least a part of one or more subjects to students. In phase 506 the lesson planner component may select the lesson that is scheduled by the smart learning system to the students. In an example, the lesson may be a lesson, e.g. hybrid lesson, for a given subject, e.g. mathematics, physics, history, biology, a language, etc. The lesson may be scheduled by the lesson planner as a hybrid lesson, whereby the lesson comprises an online lesson and an offline lesson. Accordingly, the hybrid lesson comprises features/portions that are implemented as an online lesson or an offline lesson. Examples of the features comprise learning methods and learning materials. The lesson planner component may provide determining the grouping of the students, group- specific lesson plans and/or group-specific schedules for the lesson. The scheduling of one or more hybrid lessons for teaching the one or more subjects to students may be based on analysis of the tracked data. In the analysis, the system, in particular the machine learning model, may match/compare at least a part of the tracked data comprising one or more of a learning progress, a learning a progress speed, a skill level, a learning method preference, attentiveness, emotional state, language knowledge, interaction with other students of different students or different student groups (either in the same room or partially present and partially remote) that have participated in lessons for the same subjects. Due to the matching, the system notices similarities between the students and may determine one or more combinations of learning methods and learning materials for the same students or similar students for teaching one or more further lessons concerning at least a part of the one or more subjects. Thanks to the matching, the system may determine at least one combination of learning methods and learning materials for a student for learning one or more subjects, or at least one subject, and use the determined combination of learning methods to determine a grouping of the students. Accordingly, the system can effectively assign students to groups that each have a combination of learning methods and learning materials. The determined combination of learning methods and learning materials forms a basis of a group-specific lesson plan and a group-specific schedule of each group. Any matching or comparison can also be made against historical data. The system stores historical data concerning the previous students and groups working with the same material, and their speed and level of progress, and can use this data to compare the current students by means of the machine learning model. Examples of combinations of learning methods and learning materials for a student are presented in Table 1.

Table 1 Examples of combinations learning methods and learning materials for students.

[0073] Following the example of combinations described in Table 1, the students phase 508 may comprise that each student may be determined at least one of the combinations of Ύ, ‘2’, or ‘3’ in the Table 1. Then in phase 510, the students may be assigned to groups based on the determined combinations, whereby groups corresponding to the combinations may be formed. In example, the lesson determined in phase 506 is a language learning lesson, e.g. a French lesson. Then, in phase 508, based on the tracked data, i.e. learning progress, a learning progress speed, skill level, a learning method preference, attentiveness, emotional state, language knowledge, interaction with other students, learning material performance, and schedule performance, very few or even no students are assigned the combination ‘3’ in Table 1 for learning the language. This may happen, if the tracked data indicates a low likelihood of achieving the goal of the language lesson based on the learning methods and/or materials using the combination ‘3’. One reason for the tracked data to indicate a low likelihood of achieving the goal of the language lesson may be that the learning method includes doing craftworks from wood, which does not teach the students the language, e.g. French. On the other hand, for some students, in phase 508 the tracked data may indicate a need to assist the students emotionally for achieving the goals for the language lesson and that for these students their emotional state has historically improved when doing craftworks. Therefore, these students may be determined to be assigned the combination ‘3’ for the French lesson. It should be noted that the combination for the language lesson including craftworks may in practice be implemented as more than one lesson due to practical considerations. Therefore, also combinations of learning material and learning methods that do not directly teach the subject, in this example French, could be determined for students, based on the tracked data.

[0074] In an example of determining a group-specific schedule for the lesson, the phase 510 may comprise determining a time for the lesson, where attentiveness of the students of the group is favorable for learning the subject. The best time for learning the subject may be different for different subjects and for students and the tracked data may be used to determine a time for the lesson, where a likelihood of the students reaching the goal of the lesson may be high as well as the free for the lesson for all the students.

[0075] In an example phase 510 comprises determining a group-specific lesson plan for each of the groups. Determining a group-specific lesson plan may comprise determining schedules for learning activities performed during the lesson. The learning activities may comprise for example combinations of learning methods and learning materials.

[0076] In an example phase 510 comprises that the lesson planner receives user input from a teacher regarding the acceptance of the determined grouping of said at least a part of the students for the lesson. After the acceptance has been received, in phase 512, the lesson planner may provide the determined grouping of said at least a part of the students for the lesson, group-specific lesson plans and/or group-specific schedule for the lesson to the the student user devices. The determined grouping of said at least a part of the students for the lesson, group-specific lesson plans and/or group-specific schedule for the lesson may be made accessible at the smart learning system to the student user devices, whereby the student user device may be delivered the determined grouping of said at least a part of the students for the lesson, group-specific lesson plans and/or group-specific schedule for the lesson, when the student user devices are connected to the smart learning system. [0077] In an example, phase 512 comprises that the smart learning system, or the lesson planner component, is configured to present a dashboard view to a teacher and/or a dashboard view to a student. A dashboard view of the teacher, or teacher dashboard view, may be displayed on the teacher user device and comprise the determined grouping of students, group-specific lesson plans and/or group-specific schedules for the lesson. A dashboard view of the student may be displayed on the student user device and comprise the determined grouping of students, group-specific lesson plans and/or group- specific schedules for the lesson. The teacher dashboard view may be capable of receiving user input from the teacher, whereby the teacher may accept or reject at least one of the determined grouping of students, group-specific lesson plans and/or group-specific schedules for the lesson by entering his/her acceptance or rejection to the dashboard view. If the user input indicates acceptance of the determined grouping of students, group-specific lesson plans and/or group-specific schedules for the lesson, may be provided to the groups, i.e. to the dashboard view of the students, or student dashboard views, that may be displayed on the student user devices. In this way the group- specific lesson plans and/or group-specific schedules for the lesson become available to the students via their devices in accordance with their grouping, after the teacher has accepted them.

[0078] In an example, in phase 508, the combination of learning methods and learning materials comprise a combination of learning methods and learning materials for one or more lessons, e.g. the lesson determined in phase 506. The lesson may be a hybrid lesson, an online lesson or an offline lesson. [0079] In an example, phase 508 comprises determining based on the determined at least one combination of learning methods and learning materials for each student, group-specific lesson plans and group specific schedules for the lesson.

[0080] The present embodiments are clarified by means of a use case. The first use case relates to a hybrid learning situation combining both classroom learning and distance learning occurring simultaneously. Student X participates the lesson online from a distance learning environment. Students Y, Z participates the lesson at the classroom. The educational material that is provided to the student X is synchronous to the educational material that is presented by the teacher to students Y, Z. The smart learning system tracks student’s X progress on the material, and especially assignments a - c, and notices that the student X is not able to finish assignment b. At the same time, the smart learning system tracks the learning of the students Y, Z, and teaching process in the classroom environment. From the classroom learning environment, the smart learning system gathers recordings on teacher’s and students speech, from which the smart learning system is able to recognize the topic and the phase, and possible problems the students have. From the educational material the teacher is using, the smart learning system notices that the students Y, Z are proceeding to assignment d already, while student X is still struggling with the assignment b. As a corrective action, the smart learning system tries to help student X by playing, on the student’s device, the recording of the teacher’s speech concerning the topic of the assignment b or special advices directly targeted to the assignment b. If the smart learning system notices that the given assistance does not help the student X, the smart learning system is configured to indicate the situation to the teacher.

[0081] In a second example use case, which extends the first use case, the students Y and Z in the classroom also have personal digital devices, which they use for accomplishing assignments provided by the teacher. In such a case the learning progress can be digitally monitored and tracked, whereupon comparison between the learning progress of remote students and learning progress of classroom students can be more easily determined.

[0082] The first and second uses case refers to tracking students in the classroom. There are two embodiments for implementing this. As a first embodiment, the local students (i.e. students at the classroom) may have the same digital equipment and the same educational software as the remote students by means of which the lessons are viewed and assignments are accomplished. As a second embodiment, the local students are tacked by an additional tracking mechanisms and devices. A classroom may be installed with dedicated cameras for capturing image data (video / still) on the classroom. In addition to the cameras, other recording sensors may be used as well. The data may be provided to a computer program having algorithms to identify and track users. Instead, the cameras may be equipped with such algorithms to perform student identification and tracking. The first and the second embodiments may be separate embodiments, or they can be combined. When combined, the smart learning system implements a data fusion which combines the tracking data obtained from the computer and additional sensors, such as cameras. The tracking data obtained from the classroom is utilized when evaluating the progress and possible difficulties of the students at the classroom environment. In addition, the tracking data, and the evaluation results in particular, may be utilized when evaluating the progress and possibilities of the remote students. If tracking data is unavailable from the classroom environment, then the remote students and their progress is being evaluated by utilizing history data of earlier remote students.

[0083] In a third example use case a part of the learning happens in online and part in offline environment. In such arrangement, the smart learning system will track the online assignments and notify the teacher on the gaps in accomplishing the assignments as well as recommend the study material for the next online and offline classes. If there are gaps in online or offline learning results compared to lesson plans, the system will recommend additional lessons for the student, the additional lessons focusing on the gaps in the classroom setting. Since the machine learning model continuously gathers data on student’s progress and possible problems, the system, with the help of machine learning model can identify individual learning styles. For example, if student repeatedly has problems with a certain type of online or offline learning methods, the system can recommend different learning methods. [0084] In an example, phase 506 comprises that the at least combination of learning methods and learning materials.

[0085] Figure 6 illustrates an example of a method. The method may be performed by a smart learning system described in Fig. 1, for example in connection phase 502 of Fig. 5.

[0086] Phase 602 comprises determining, based on the received data from the offline learning component and the online learning component, performance metrics of teaching said one or more subjects to students.

[0087] In an example, the performance metrics in phase 602 comprise one or more of offline learning method performance metrics 606, online learning material performance metrics 604 and online learning method performance metrics 608.

[0088] In an example, the performance metrics in phase 602 are determined based on said on one or more offline learning methods 614, said one or more offline learning materials 624, said one or more online learning methods 634 and/or said one or more online learning materials 644.

[0089] In an example, phase 602 comprises that the performance metrics may be determined based on the tracked past data of each student.

[0090] In an example the performance metrics comprise student-specific performance metrics, learning material performance metrics and learning activity performance metrics. The student-specific performance metrics indicate for a single student a learning progress, a learning a progress speed, a skill level, a learning method preference, attentiveness, emotional state, language knowledge, interaction with other students. The learning material performance metrics indicate performance of the learning material. The schedule performance metrics indicates a performance of a learning activity. [0091] The learning progress indicates how far the student has progressed in learning a subject and/or has the student missed lessons or topics the missed lessons or topics may indicate that the student should learn the missed lessons or topics before progressing further in the subject.

[0092] The skill level indicates the student’s skill level in the subject. The skill level enables the student to be compared with other students and determine whether the student needs a challenge or more support.

[0093] The learning method preference gives insights over time after the student has used different learning methods and the student’s learning has been assessed. The learning method preference may indicate one or more combinations of learning methods and learning materials that are preferred for the student for learning a subject. The learning method preferences may be specific to a subject, whereby the student may have different learning method preferences for different subjects. The learning method preference may be established by tracking learning results of the student and the learning methods used to achieve the learning results. The learning results of the student may be assessed based on learning results of other students and determining differences between the learning results. The tracking of the learning method preference may be started by initially delivering different students different learning material to get a meaningful set of results for each student.

[0094] The attentiveness and emotional state enable scheduling lessons at times that are most attractive regarding the likelihood of the student to learn the subject and achieve the goals of the lesson. In an example attentiveness may be tracked by the machine learning model based on observing the actions and behavior of at least the remote students through the software on their computer. The attentiveness may indicate how they are focusing on the assignments or is their focus targeted somewhere else (e.g. some social media application). In addition, the local machine learning algorithm at the student’s device is configured to analyze the emotional state of the remote students by using a webcam capturing as means to obtain data on facial expressions of the remote student and/or by using other sensors that are capable of measuring physical and psychical parameters from a student. The emotional state of the student may be quantified into e.g. focused, anxious, happy and sad.

[0095] The language knowledge may be tracked by the smart learning system based on the smart learning system tracking the student’s skill and understanding of the teaching language. The tracking may be based on analyzing the language used by the student in his/her assignments e The system can automatically track - through analyzing the language each student uses in their assignments - their skill and understanding of the language used for teaching.

[0096] The student’s interaction with other students can be recorded for determining which students prefer to collaborate in learning a subject [0097] The learning material performance indicates effectiveness of a learning material to teach a subject to students. The learning material performance can be automatically measured across all of its use on the platform. The measurement of the learning material performance may comprise comparing assessment results of all students having seen consumed the learning material while studying the subject in comparison to students who did not consume the material while studying the same subject.

[0098] The schedule performance indicates a performance of a learning activity. The performance of the learning activity may comprise a duration of the learning activity and results achieved by the learning activity. The results may comprise assessment results of the students for example. A schedule performance may be a ratio of the results and a time spent to achieve the results. The time spent to achieve the results may be determined based on the duration of the learning activity that was performed for achieving the results. [0099] In an example, the performance metrics in phase 602 are determined based on at least one of: o time for teaching said one or more subjects based on offline learning methods and offline learning materials, o time for teaching said one or more subjects based on online learning methods and online learning materials, o skill levels of the students before online lessons, skill level of the students after online lessons, and o skill levels of the students before online lessons, skill level of the students after online lessons.

[0100] In an example, the time for teaching said one or more subjects in phase 602 may be determined based on tracking the schedule performance for a learning activity. The learning activity may be defined by a combination of a learning material and learning method, online or offline.

[0101] In an example, the skill levels of the students before and after online lessons and/or online lessons may be determined based on assessments of the students. In this way, skill level progress of the students may be tracked. The assessment may be performed by a teacher by entering an assessment to the mart learning system via a teacher user device. In an example the teacher user device comprises a dashboard comprising an input element for assessment of each student. On the other hand the assessment may be performed based on an exam that may be on paper, i.e. offline, or an online exam, executed on a student user device.

[0102] Figure 7 illustrates an example of a method. The method may be performed by a smart learning system described in Fig. 1, for example in connection phase 510 of Figure 5.

[0103] Phase 702 comprises determining the grouping of said at least a part of the students for the lesson based on the determined performance metrics and the determined at least one combination of learning methods and learning materials for each student of said at least a part of the students for learning said at least a part of the one or more subjects.

[0104] In an example phase 702 comprises that the grouping is determined based on performance metrics of each student. The performance metrics may be determined in accordance with Figure 6 based on one or more offline learning methods 614, said one or more offline learning materials 624, said one or more online learning methods 634 and/or said one or more online learning materials 644. The determined performance metrics may be offline learning method performance metrics, offline learning material performance metrics, online learning material performance metrics and/or online learning method performance metrics.

[0105] In an example, phase 702 comprises that the grouping of the students may be determined based on the performance metrics, whereby students (Studentl , Student2, ...,StudentN) that have similar performance metrics may be assigned to the same group. In an example each group may be defined based on value ranges of the performance metrics of the students. The performance metrics of the students may be evaluated after the comparing of the tracked data between students. The evaluation of the performance metrics may provide determining the grouping of the students based on their performance metrics into a given number of groups. Accordingly, each group may be defined by a number of students assigned to a group and a value range of performance metrics of the students assigned to the group. In this way each group may be assigned a sufficient number of students while controlling the students that can be assigned to the group based on whether the students’ performance metrics are within a value range of the group. In this way the grouping may provide similarity between students. For example, the students may be grouped based on skill levels of the students ranging from 1 to 5 in integer values, whereby 1 first group may be for students that have skill levels 1 and 2, a second group may be for students that have skill levels 3 and 4 and a third group may be for students that have a skill level 5. As a further requirement, each group may be assigned a minimum and a maximum number of students for controlling the number of students assigned to each group. The minimum and maximum number of students may be the same for all groups or the minimum and the maximum number of students may be different for each group. Following the example of Table 1 of the combinations, the performance metrics may be used to assign students that have the combination Ύ to more than one group based on the performance metrics. In this way there would be at least two groups having the combination Ύ, one of which has students with skill levels 1 and 2 and one of which has students with skill levels 3 and 4. [0106] Figure 8 illustrates an example of a method. The method may be performed by a smart learning system described in Fig. 1 , for example in connection phase 602 of Figure 6.

[0107] Phase 802 comprises determining at least one change of skill levels of the students based on the skill levels of the students before the lesson and after the lesson.

[0108] Phase 804 comprises applying at least one weight, based on the determined at least one change, to at least part of the determined performance metrics. In this way the determined change may be used for adjusting the performance metrics of teaching said one or more subjects to students, whereby the grouping of students for the lesson may be determined based on the performance metrics that have been adjusted by the weight.

[0109] In an example, phase 802 comprises determining a change of a skill level of individual students and/or changes of skill levels for a group of students. The change of a skill level of individual students provides that the performance metrics may be weighted for each student based on their individual performance. The changes of skill levels for a group of students provides that the performance metrics may be weighted based on the performance of the group of students in which case positive or negative performance on a level of the group may be used for weighting the performance metrics.

[0110] In an example, the at least one weight in phase 804 comprises a weight for teaching a subject to a student using a specific combination of learning methods and learning materials.

[0111] In an example, phase 804 comprises determining the at least one weight based on the determined at least one change. For example, if the skill levels of the students are lower before the lesson than after the lesson, the at least weight may be a positive weight. On the other hand if the skill levels of the students are higher before the lesson than after the lesson, the at least weight may be a negative weight. A difference between the skill levels before and after the lesson may be compared with a threshold and if the threshold has been met, or exceeded, the at least one weight may be determined to be a positive weight. On the other hand, if the threshold has not been met, the at least one weight may be determined to be a positive weight. In an example, the skill levels of the students may be determined based on exams/tests/assessments of the students. Accordingly, the threshold may be a grade value in a given value range, e.g. an integer value within a value range from 0 to 5, whereby the skill levels may be floating point values from 0.0 to 5.0. The grade value may be determined based on an exam/test/assessment Then, for a group of students the change of the skill levels may be an average of the grades of the students before and after the lesson. On the other hand for an individual student, a change of the skill level may be a difference between a grade for an exam/test/assessment conducted after the lesson to an average grade of past exams/tests/assessments or an individual previous exam, e.g. the last previous exam, performed by the student or a combination of the average grade and the individual previous exam performed by the student.

[0112] In an example, phase 804 comprises determining a plurality of weights for each student. The determined weights may be presented in a vector form, where the length of the vector is defined by the number of weights. Accordingly, each element of the vector may be a weight for teaching a subject to a student using a specific combination of learning methods and learning materials.

[0113] In an example, phase 804 comprises applying the weight to a schedule performance or learning material performance.

[0114] Fig. 9 illustrates an example of a method. The method may be performed by a smart learning system described in Fig. 1 , for example in connection phase 508 of Figure 5. The method enables providing students group-specific lesson plans while taking into account learning method preferences of the students. [0115] Phase 902 comprises determining a level of correspondence between learning method preferences of said at least a part of the students and the group-specific lesson plans of the students.

[0116] Phase 904 comprises determining if the determined level meets a first threshold. If the first threshold has been met, the method may proceed to phase 906 comprising applying the determined grouping. After the grouping has been applied, the method may end 908. In an example the grouping may be applied in accordance with phase 512. After the grouping has been applied the method may end 908. If the first threshold has not been met, the method may proceed to end 908 without applying the grouping.

[0117] In an example, phase 902 comprises comparing the group-specific lesson plan assigned to a group to a learning preference of a student that has been assigned to the group. The comparison may provide a similarity score, e.g. in percentages, that indicate a level of correspondence between the learning preference of the student and the group-specific lesson plan of the group. If the similarity score meets a threshold, or exceeds the threshold, the grouping may be considered to sufficiently similar to the student’s learning preference. In an example, the group-specific lesson plan may comprise online learning, offline learning, online learning materials and/or offline learning materials. A combination of online learning, offline learning, online learning materials and/or offline learning materials may be defined e.g. based on time durations indicating shares of a lesson duration allocated between online learning and offline learning and corresponding materials. Accordingly, the combination may be 70% of online learning, 30% of offline learning, 90% online learning materials and/or 10% offline learning materials.

[0118] In an example phase 902 comprises determining whether a student has been assigned to a group that has a group specific lesson plan that is according to the learning method preference of the student, partially according to the learning method preference of the student or not at all according to the learning method preference of the student. In an example, referring to Table 1 , if the learning method preference of the student would be in accordance with the combination Ύ, the level of correspondence would be 100%, which should mee the threshold in phase 904 and the method would proceed to phase 906. On the other hand if the if the learning method preference of the student would not include ‘reading’ and ‘reading literature’ in the Combination Ύ but the learning method preference would include ‘language learning application’ and ‘student uses language learning application’, the level of correspondence would be 50% . If the threshold in phase 904 is at most 50%, the method would proceed to phase 906, but if the threshold in phase 904 is over 50%, the method would proceed to end 908 without applying the determined grouping. On the other hand if the learning method preference of the student would not include ‘reading’, ‘reading literature’, ‘language learning application’ and ‘student uses language learning application’ in the Combination Ύ, the level of correspondence would be 0% . If the threshold in phase 904 is over 0%, the method would proceed to end 908 without applying the determined grouping. [0119] Figure 10 illustrates an example of a method. The method may be performed by a smart learning system described in Fig. 1 , for example in connection phase 508 of Figure 5. The method provides providing students group-specific lesson plans based while taking into account learning method preferences of the students.

[0120] Phase 1002 comprises estimating performance metrics of the group- specific lesson plans of the students.

[0121] Phase 1004 comprises determining if the estimated performance metrics meet at least one second threshold. If the at least one second threshold has been met, the method may proceed to phase 1006 comprising applying the determined grouping. In an example the grouping may be applied in accordance with phase 512. After the grouping has been applied the method may end 1010. If the at least one second threshold has not been met, the method may proceed to end 1010 without applying the grouping.

[0122] In an example, phase 1002 comprises that the performance metrics are estimated for individual students and/or for one or more groups of the students. Accordingly, the estimated performance metrics may indicate an estimate of a performance metric of an individual student and/or an estimate of a performance metric of one or more groups. The estimated performance metrics of a group-specific lesson plan provides an estimated change of a skill level of a student of a group and/or the group. The estimate may be generated based on past data on performance of the students that have conducted the lesson.

[0123] In an example, the estimate determined in phase 1002 may be displayed on a student user device or a teacher user device. Alternative groupings, group-specific lesson plans and group-specific schedule may be determined for example based on grouping the students based on a different sets of performance metrics in accordance with phase 702. For example, one alternative grouping may be performed based on skill level of the students and another alternative grouping may be performed based on a learning progress of the students. The performance metrics of the group-specific lesson plans may be determined for group-specific lesson plans of all grouping alternatives. Then, the determined estimates may be displayed on the student user device or a teacher user device. The teacher user device and/or student user device may display information indicating whether the estimated performance metrics meet the threshold in phase 1004. The grouping may be selected by the teacher based on entering user input to the teacher user device, provide the threshold was met in phase 1004. The alternative groupings provide that different groupings of students may be applied, whereby different groupings of students may be tried.

[0124] In an example, a machine learning component for a smart learning system may be trained in phases. In a first phase the machine learning component may be trained based on training datasets comprising lesson plans. The lesson plans comprise one or more offline lessons for teaching one or more subjects to students based on one or more offline learning methods and one or more offline learning materials and one or more online lessons for teaching said one or more subjects to students based on one or more online learning methods and one or more online learning materials. Preferably the training data sets represent different scenarios at least in terms of learning materials, teaching methods and grouping of students. Since each lesson plan, or a group-specific lesson plan, is associated with a group of students, the machine learning component can be trained to identify which lesson plans work for which group e.g. based on manual data labeling. In a second phase, once the machine learning component is trained based on the training datasets, the machine learning component may be used to determine group- specific lesson plans for the groups based on performances of the students, for example a learning progress, a learning progress speed, skill level, a learning method preference, attentiveness, emotional state, language knowledge, interaction with other students, learning material performance, and schedule performance of each student. In the second phase, the students may be grouped manually as needed based on their performance. The manually grouping can be used to train the machine learning component further. In time, the need for manual grouping may become unnecessary as the training of the machine learning component progresses.

[0125] In an example, a machine learning component for a smart learning system may be initially trained based on training data gathered from lessons. The training data can be anonymized and stored in servers. After the initial training the machine learning component can be used during real lessons in an inference mode. In the inference mode, the machine learning component is trained based on data gathered from particular students. The data may comprise for example a learning progress, a learning progress speed, skill level, a learning method preference, attentiveness, emotional state, language knowledge, interaction with other students, learning material performance, and schedule performance of a student. Gathering of the data may comprise gathering data from a user device, teacher device and/or tracking device(s). The gathered data may be obtained from an educational application and/or monitoring application. Web application may provide data from a camera and also support other tracking e.g. based on tracking devices such as wearable sensors. It should be noted that lessons in some schools can significantly differ from the training data. In that case, the machine learning component may be fine-tuned, i.e. trained further, so that it makes accurate predictions for new students. In an example of the fine-tuning, a particular lesson can be recorded in a given school and the machine learning component can be used to make predictions, i.e. group-specific lesson plan(s), based on the recorded lesson. The teachers and/ or students may review the predictions and correct/verify the predictions. Based on this data, the machine learning component will be automatically trained, verified. The process can be repeated for different lessons in the school if performance of the machine learning component should be improved. Finally, the school will have a machine learning model specifically trained for the school. An example of data gathering for lessons may utilize at least one of Apple HealthKit and FitBit API. Apple HealthKit and FitBit API provide data about data about sleep, heart rate or physical activity which can be used for determining condition of a student during classes and for analyzing how his/her behavior out of class affects in-class performance. Additionally, the student’s phone usage may be tracked during class. The phone tracing may comprise information about used apps. The gathered data may be anonymized based on generating pseudonyms for each user, generalizing data by scaling, as well as removing certain fields from the dataset. Random noise can be added to the data for improving training results of the machine learning component, which gets us higher training results and serves also anonymization of the data. Data may be stored in NOSQL database.

[0126] It should be noted that the data gathering may be performed during real lessons with different teachers and for two groups of students - one in class, and another on the remote. In this way training of the machine learning component for hybrid lessons may be supported. During the lessons data may gathered and the students may be tracked, e.g. from cameras, tracked student activity on a learning platform and their data from fitness devices, as well as their phone activity. Then manual data labeling may be used to label the students’ emotions using camera, as well as mark student's engagement score for each time frame during the lesson. Data regarding body status may be used as-is, given that a name of a device providing the body status is present in the dataset. The machine learning model may be initially trained based on an initial training dataset formed based on the gathered data. If more accurate results from the machine learning model, the data gathering maybe used for fine- tuning of the machine learning model.

[0127] The present embodiments can be applied in various learning environment combinations. For example, the learning environment may comprise classroom environment and distance learning environment, which are occurring at the same time or at different times. In addition, the distance learning can occur online or offline. Regardless the type of the distance learning, the purpose is to provide a view to student’s progress in distance learning, and to solve possible problems being occurred therein, in a classroom learning environment.

[0128] According to an embodiment, there is provided an apparatus or system comprising means for receiving data generated by applications executed in connection with one or more offline lessons for teaching one or more subjects to students based on one or more offline learning methods and one or more offline learning materials and data generated by applications executed in connection with one or more online lessons for teaching said one or more subjects to students based on one or more online learning methods and one or more online learning materials; means for tracking based on the received data from the offline learning component and the online learning component, one or more of a learning progress, a learning progress speed, skill level, a learning method preference, attentiveness, emotional state, language knowledge, interaction with other students, learning material performance, and schedule performance; means for determining a lesson for teaching at least a part of the one or more subjects to at least a part of the students; means for determining based on the determined lesson and the tracked one or more of a learning progress, a learning progress speed, skill level, a learning method preference, attentiveness, emotional state, language knowledge, interaction with other students, learning material performance, and schedule performance, at least one combination of learning methods and learning materials for each student of said at least a part of the students for learning said at least a part of the one or more subjects, wherein the at least one combination of learning methods and learning materials comprises one or more of offline learning methods, one or more offline learning materials, one or more online learning methods and one or more online learning materials; means for determining based on the determined at least one combination of learning methods and learning materials for each student, a grouping of said at least a part of the students for the lesson; means for providing, to each group determined based on the grouping, based on receiving a user input indicating acceptance of the determined grouping of said at least a part of the students for the lesson, a group-specific lesson plan.

[0129] The system may be a smart learning system. The system may comprise a memory stored with computer program code thereon, wherein the at least one memory and the computer program code are configured, with at least one processor of the smart learning system, to cause the smart learning system at least to perform on a method or at least part of functionalities of a method. The memory may be a non-transitory computer readable medium. [0130] It is to be understood that the embodiments disclosed are not limited to the particular structures, process steps, or materials disclosed herein, but are extended to equivalents thereof as would be recognized by those ordinarily skilled in the relevant arts. It should also be understood that terminology employed herein is used for the purpose of describing particular embodiments only and is not intended to be limiting.

[0131] Reference throughout this specification to one embodiment or an embodiment means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. Thus, appearances of the phrases “in/according to one embodiment” or “in/according to an embodiment” in various places throughout this specification are not necessarily all referring to the same embodiment. Where reference is made to a numerical value using a term such as, for example, about or substantially, the exact numerical value is also disclosed. [0132] As used herein, a plurality of items, structural elements, compositional elements, and/or materials may be presented in a common list for convenience. However, these lists should be construed as though each member of the list is individually identified as a separate and unique member. Thus, no individual member of such list should be construed as a de facto equivalent of any other member of the same list solely based on their presentation in a common group without indications to the contrary. In addition, various embodiments and examples may be referred to herein along with alternatives for the various components thereof. It is understood that such embodiments, examples, and alternatives are not to be construed as de facto equivalents of one another, but are to be considered as separate and autonomous representations.

[0133] The foregoing description has provided by way of exemplary and non-limiting examples a full and informative description of the exemplary embodiment of this invention. However, various modifications and adaptations may become apparent to those skilled in the relevant arts in view of the foregoing description, when read in conjunction with the accompanying drawings and the appended examples. However, all such and similar modifications of the teachings of this invention will still fall within the scope of this invention.

Claims

1. A smart learning system comprising: o an offline learning component for communications with applications executed in connection with one or more offline lessons for teaching one or more subjects to students based on one or more offline learning methods; o an online learning component for communications with applications executed in connection with one or more online lessons for teaching said one or more subjects to students based on one or more online learning methods and one or more online learning materials, o a lesson planner component for scheduling one or more hybrid lessons for teaching subjects to students, wherein a hybrid lesson comprises at least one offline lesson portion and at least one online lesson portion, and o a machine learning component operatively connected to the offline learning component, the online learning component and the lesson planner component wherein the smart learning system is configured to: o receive data generated by applications executed in connection with one or more offline lessons for teaching one or more subjects to students based on one or more offline learning methods and one or more offline learning materials and data generated by applications executed in connection with one or more online lessons for teaching said one or more subjects to students based on one or more online learning methods and one or more online learning materials; o track based on the received data from the offline learning component and the online learning component, one or more of a learning progress, a learning a progress speed, a skill level, a learning method preference, attentiveness, emotional state, language knowledge, interaction with other students, learning material performance, and schedule performance; o determine a lesson for teaching at least a part of the one or more subjects to at least a part of the students; o determine based on the determined lesson and the tracked one or more of a learning progress, a learning progress speed, skill level, a learning method preference, attentiveness, emotional state, language knowledge, interaction with other students, learning material performance, and schedule performance, at least one combination of learning methods and learning materials for each student of said at least a part of the students for learning said at least a part of the one or more subjects, wherein the at least one combination of learning methods and learning materials comprises one or more of offline learning methods, one or more offline learning materials, one or more online learning methods and one or more online learning materials; o determine based on the determined at least one combination of learning methods and learning materials for each student, a grouping of said at least a part of the students for the lesson; o provide, to each group determined based on the grouping, based on a user input received by the lesson planner component indicating acceptance of the determined grouping of said at least a part of the students for the lesson, a group-specific lesson plan.

2. The smart learning system according to claim 1 , configured to: o determine based on the determined at least one combination of learning methods and learning materials for each student, group-specific lesson plans and group specific schedules for the lesson.

3. The smart learning system according to claim 1 or 2, configured to: o determine, based on the received data from the offline learning component and the online learning component, performance metrics of teaching said one or more subjects to students.

4. The smart learning system according to claim 3, wherein the performance metrics comprise one or more of offline learning method performance metrics, offline learning material performance metrics, online learning material performance metrics and online learning method performance metrics.

5. The smart learning system according to claim 3 or 4, wherein the performance metrics are determined based on said on one or more offline learning methods, said one or more offline learning materials, said one or more online learning methods and/or said one or more online learning materials.

6. The smart learning system according to any of claims 3 to 5, configured to: o determine the grouping of said at least a part of the students for the lesson based on the determined performance metrics and the determined at least one combination of learning methods and learning materials for each student of said at least a part of the students for learning said at least a part of the one or more subjects.

7. The smart learning system according to any of claims 3 to 6, wherein the performance metrics are determined based on at least one of: o time for teaching said one or more subjects based on offline learning methods and offline learning materials, o time for teaching said one or more subjects based on online learning methods and online learning materials, o skill levels of the students before online lessons, skill level of the students after online lessons, and o skill levels of the students before online lessons, skill level of the students after online lessons.

8. The smart learning system according to any of claims 3 to 7, configured to o determining at least one change of skill levels of the students based on the skill levels of the students before the lesson and after the lesson; o applying at least one weight, based on the determined at least one change, to at least part of the determined performance metrics.

9. The smart learning system according to any of claims 1 to 8, configured to o determine a level of correspondence between learning method preferences of said at least a part of the students and the group- specific lesson plans of the students; o apply the determined grouping, if the level meets a first threshold.

10. The smart learning system according to any of claims 1 to 9, configured to: o estimating performance metrics of the group-specific lesson plans of the students; and o applying the determined grouping, if the estimated performance metrics meet at least a second threshold.

11 .A method comprising: o receiving data generated by applications executed in connection with one or more offline lessons for teaching one or more subjects to students based on one or more offline learning methods and one or more offline learning materials and data generated by applications executed in connection with one or more online lessons for teaching said one or more subjects to students based on one or more online learning methods and one or more online learning materials; o tracking based on the received data from the offline learning component and the online learning component, one or more of a learning progress, a learning progress speed, skill level, a learning method preference, attentiveness, emotional state, language knowledge, interaction with other students, learning material performance, and schedule performance; o determining a lesson for teaching at least a part of the one or more subjects to at least a part of the students; o determining based on the determined lesson and the tracked one or more of a learning progress, a learning progress speed, skill level, a learning method preference, attentiveness, emotional state, language knowledge, interaction with other students, learning material performance, and schedule performance, at least one combination of learning methods and learning materials for each student of said at least a part of the students for learning said at least a part of the one or more subjects, wherein the at least one combination of learning methods and learning materials comprises one or more of offline learning methods, one or more offline learning materials, one or more online learning methods and one or more online learning materials; o determining based on the determined at least one combination of learning methods and learning materials for each student, a grouping of said at least a part of the students for the lesson; o providing, to each group determined based on the grouping, based on receiving a user input indicating acceptance of the determined grouping of said at least a part of the students for the lesson, a group-specific lesson plan.

12. A computer program comprising instructions which, when the program is executed by a processor of, cause to carry out the method of claim 11.