KR20230043034A - Method and apparatus for providing non-face-to-face online education service for checking the student's lecture concentration in real time - Google Patents

Abstract

According to various embodiments, an online education server for providing an online education service including a real-time online class and a recorded online class may comprise: a service providing unit for registering a lecture requested from a lecturer terminal, obtaining a lecture application input for the lecture through a user terminal, and providing learning content included in the lecture to the user terminal according to the obtained lecture application input; a lecture supplement section determining unit for determining a lecture supplement section of the lecture based on lecture information of each of first users who take the real-time class included in the lecture online; a lecture preference calculating unit for determining lecture regions of each of the users including the first users and second users who take a recorded class included in the lecture online, and calculating a lecture preference of each of the determined lecture regions; an image analyzing unit for determining a lecture concentration of a user corresponding to the user terminal based on a user moving image obtained from the user terminal; a lecture completion calculating unit for calculating lecture completion of the user based on the lecture concentration; and a learning schedule determining unit for determining a learning schedule for the lecture according to the lecture completion. According to various embodiments, it is possible to improve the quality of lectures.

Description

Method and apparatus for providing non-face-to-face online education service to check student's lecture concentration in real time

The present invention relates to a method and apparatus for providing a non-face-to-face online education service for checking a student's lecture concentration in real time. It relates to a method and apparatus for determining a user's course completion degree.

Today, online lectures are emerging as a useful means to provide various knowledge and learning experiences from remote locations through a network-based online learning environment. Online lectures allow students to overcome the physical limitations of time and space, enabling them to learn at the time and place they want. However, despite the above advantages, the current online lecture system needs to supplement the shortcomings in various aspects. This includes the absence of a way to check whether or not the student has faithfully taken the online lecture content. For example, if a lecturer attends a lecture by outputting another previously stored video instead of outputting an actual video of himself filming himself in an online lecture, the lecturer and the lecture system check whether the lecturer has actually taken the lecture faithfully. I'm having trouble judging.

In addition, after the education and classes are conducted, feedback on the lectures is received and the contents of the lectures are supplemented. Feedback on the lectures is usually done through surveys. However, there is a problem in that it is not quick to receive feedback on lectures through surveys, and the feedback cannot be reflected in a short time. This can be a negative factor that reduces trust and expectations for online lectures themselves and prevents further expansion of online lecture services.

An object of the present invention to solve the above problems is to provide a method and apparatus for determining a lecture complement section for an online lecture and providing the lecture supplement section to a lecturer terminal.

Another object of the present invention to solve the above problems is to provide a method and apparatus for calculating a course completion degree of a user taking an online lecture.

In various embodiments, an online education server providing an online education service including real-time online classes and recorded online classes registers a lecture requested from a lecturer terminal, obtains an input for registering for the lecture through a user terminal, a service providing unit that provides learning contents included in the lecture to the user terminal according to the acquired course registration input; a lecture complement section determiner for determining a lecture supplement section of the lecture based on class information of each of the first users taking the real-time class included in the lecture online; A lecture preference calculation unit for determining course areas of each of the first users and users including second users who are taking a recorded class included in the lecture online, and calculating a lecture preference of each of the determined taking areas. ; An image analyzer for determining a degree of concentration of a user corresponding to the user terminal based on a user video obtained from the user terminal; a course completion degree calculation unit that calculates a course completion degree of the user for the lecture based on the degree of concentration of the lecture; and a learning schedule determining unit configured to determine a learning schedule for the lecture according to the degree of completion of the course. The service providing unit may provide the user video to the lecturer terminal, and provide the lecturer video in which the lecturer corresponding to the lecturer terminal is photographed to the user terminal.

In various embodiments, the lecture supplementary section determining unit calculates a real-time lecture viewing rate of the lecture based on the number of first users taking the real-time class compared to all users who have applied for the real-time class, and the calculated An audience rating function and a viewer rating graph are generated based on the viewer rating of the real-time lecture, and the supplementary section of the lecture is determined based on at least one of an average viewer rating based on the viewer rating of the real-time lecture, the viewer rating function, and the viewer rating graph.

In various embodiments, the lecture preference calculation unit may calculate the lecture preference based on the number of users included in each of the course enrollment areas and the average viewing rate for the lecture in each of the course enrollment areas.

In various embodiments, the video analyzer detects the instructor's face region and the instructor's hand region in the lecturer's video displayed through the user terminal, and the left eye region of the user in the user's video obtained through the user terminal. and the right eye area are detected, the positions of the pupils are determined in each of the detected left eye region and the right eye region, and based on the distance between the user and the user terminal and the position of the pupils, the focus of the user is the lecturer It is determined whether the video is located in the lecturer's face region or the lecturer's hand region, and the degree of concentration of the user in the lecture can be determined according to the user's focus position.

In various embodiments, the video analysis unit determines whether the user video obtained from the user terminal is authentic, and determines pixel values of first video frames constituting the user video and second video constituting the video stored in the image DB. A first similarity is calculated based on a pixel value of an image frame, a second similarity is calculated based on a face detected in each of the user video and a video stored in the video DB, and the first similarity and the second similarity are calculated. Based on this, an image similarity between the user video and the video stored in the video DB is calculated, and the user video may be determined as a fake video when the image similarity is greater than or equal to a threshold similarity.

In various embodiments, a method for providing an online education service including a real-time online class and a recorded online class includes registering a lecture requested from a lecturer terminal; Obtaining a course registration input for the lecture through a user terminal; providing learning content included in the lecture to the user terminal according to the acquired course registration input; determining a lecture supplementary section of the lecture based on class information of each of the first users who take the real-time class included in the class online; determining course attendance areas of each of the users including the first users and second users who are taking the recorded class included in the lecture online; Calculating a preference for each of the lecture areas; determining a degree of concentration of a lecture of a user corresponding to the user terminal based on a user video obtained from the user terminal; calculating a course completion degree of the user for the lecture based on the degree of concentration of the lecture; and determining a learning schedule for the lecture according to the degree of completion of the course.

In various embodiments, the online education server may include a non-transitory recording medium in which a program for executing the method of the online education server is recorded and read by a computer.

In various embodiments, the online education server may include a computer program recorded on a non-transitory recording medium to execute the method.

In various embodiments, a system for providing online education services including real-time online classes and recorded online classes includes: a lecturer terminal providing learning contents to an online education server; a user terminal acquiring the learning content from the online education server; and the online education server interworking with the user terminal and the lecturer terminal and providing an online education service to a user corresponding to the user terminal. The online education server registers a lecture requested from the lecturer terminal, obtains a course registration input for the lecture through the user terminal, and displays learning contents included in the lecture according to the acquired course registration input. a service providing unit provided to a user terminal; a lecture complement section determiner for determining a lecture supplement section of the lecture based on class information of each of the first users taking the real-time class included in the lecture online; a lecture preference calculator configured to determine lecture areas of each of the first users and users including second users who are taking a recorded class included in the lecture online, and to calculate a preference for each of the lecture areas; an image analyzer configured to analyze a user video obtained from the user terminal and determine a degree of concentration of the user on the basis of the analyzed user video; a course completion degree calculation unit that calculates a course completion degree of the user for the lecture based on the degree of concentration of the lecture; and a learning schedule determining unit configured to determine a learning schedule for the lecture according to the degree of completion of the course.

In various embodiments, the supplementary section determining unit of the lecture calculates the average viewership rating through the following equation,

In the above equation, A may be the average viewing rate, f(x) may be a viewing rate function for the real-time lecture, and k may be the total lecture time of the lecture. The lecture supplementary section determining unit determines a section in which the viewer rating is lower than the average viewer rating in the entire lecture time as an audience rating low section, and determines a section in which the viewer rating decreases in the determined low viewer rating section as a lecture supplementary section, and the lecture A supplementary section and lecture contents corresponding to the supplementary section of the lecture may be transmitted to the lecturer terminal.

In various embodiments, the lecture preference calculation unit calculates the lecture preference of the lecture area to which the user belongs through the following equation,

In the above equation, S _n is the lecture preference, A is the average viewing rate of users belonging to the attending area and taking real-time lectures online, and h is the highest viewing rate of users belonging to the attending area and taking real-time lectures online. , l is the lowest viewing rate of users belonging to the attending area and taking real-time lectures online, _Np is the number of users belonging to the attending area and taking real-time lectures online, D _- is the taking area from the lecturer's terminal It can mean the distance between The river preference calculation unit may determine, as the river preference area, the area where the lecture preference is greater than or equal to a preset threshold.

In various embodiments, the course completion degree calculation unit calculates the course completion degree through the following equation,

In the above equation, C _d is the course completion degree, T _r is the reproduction time of the fake video, T _c is the lecture time of the lecture, and T- _s is the lecturer's face region or the lecturer's hand region. This may mean a time during which the user's focus is maintained.

In various embodiments, the service providing unit determines a discount price and a course enrollment extension period for the user based on the user's preference for the course in the course enrollment area, and determines the course enrollment extension period through the following equation, ,

In the above equation, P _e may mean the course enrollment expansion period, M _s may mean the standard course enrollment period, and S _n may mean the course preference.

The service provider determines the discount price of the lecture through the following equation,

In the above equation, R _d may mean the discount price of the lecture, R _r may mean the standard price of the lecture, and S _n may mean the preference of the lecture.

In various embodiments, the image analysis unit, among the first image frames constituting the user's video, targets two image frames that are temporally adjacent to each other and differentiates between pixel values whose positions correspond to each other to obtain a first difference image. Frames are generated, each of the first difference image frames is divided into a plurality of regions, first average values of pixels for each of the divided regions are calculated, and the first average values of the calculated pixels are calculated. First feature vectors are generated based on the first feature vectors, and among the second image frames constituting the video stored in the image DB, pixel values corresponding to positions are differentiated between two image frames that are temporally adjacent to each other, and the second image frames are determined. Secondary image frames are generated, each of the second difference image frames is divided into a plurality of regions, second average values of pixels for each of the plurality of divided regions are calculated, and second average values of the calculated pixels are calculated. second feature vectors are generated based on average values, and the first similarity between the user video and the video stored in the video DB is calculated based on the first feature vectors and the second feature vectors; Third image frames are extracted at predetermined time intervals from each of the user's video and the video stored in the image DB, and based on the area of the face detected in each of the extracted third image frames, the detected facial expression change, etc. Calculate a second similarity, and calculate the image similarity between the user video and the video stored in the video DB through the following equation,

In the above equation, S is the image similarity, S ₁ is the first similarity, S ₂ is the second similarity, w ₁ is a first weight for the first similarity, and w ₂ is a second similarity for the second similarity. This means a weight, and if the image similarity is greater than or equal to a threshold similarity, the user's video may be determined as a fake video.

According to various embodiments disclosed in this document, the quality of a lecture can be improved by informing a lecturer of a section requiring supplementation for a lecture.

Also, according to various embodiments, it may be determined whether a user uses a fake video to get out of the lecturer's line of sight.

Also, according to various embodiments, it is possible to monitor whether the online lecture is normally completed without disturbing the user's learning.

In addition to this, various effects identified directly or indirectly through this document may be provided.

1 is a diagram illustrating an online education providing system according to an embodiment.
2 is a diagram illustrating users taking a real-time lecture online and users taking a real-time lecture offline, according to an embodiment.
Figure 3 is a diagram showing the main components of the online education server.
4 is a diagram relating to determining a supplementary section of a lecture based on class information of each of users taking a real-time lecture online, and transmitting the determined supplementary section to a lecturer terminal.
5 is a diagram related to determining a preferred region of a lecture based on class attendance information of each of users taking real-time lectures online.
6 is a diagram for calculating a user's course completion degree based on a user video obtained from a user terminal.
7 is a diagram related to determining a user's learning schedule.
8 is a diagram showing the hardware configuration of the online education server according to FIG. 1 .
9 is a diagram illustrating a wireless communication system that can be applied in a communication process according to an embodiment of the present invention.
10 is a diagram illustrating a base station in the wireless communication system according to FIG. 9 .
11 is a diagram illustrating a terminal in the wireless communication system according to FIG. 9 .
12 is a diagram illustrating a communication interface in the wireless communication system according to FIG. 9 .

Since the present invention can make various changes and have various embodiments, specific embodiments will be illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, and should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention. Like reference numerals have been used for like elements throughout the description of each figure.

Terms such as first, second, A, and B may be used to describe various components, but the components should not be limited by the terms. These terms are only used for the purpose of distinguishing one component from another. For example, a first element may be termed a second element, and similarly, a second element may be termed a first element, without departing from the scope of the present invention. The terms and/or include any combination of a plurality of related recited items or any of a plurality of related recited items.

It is understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, but other elements may exist in the middle. It should be. On the other hand, when an element is referred to as “directly connected” or “directly connected” to another element, it should be understood that no other element exists in the middle.

Terms used in this application are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, the terms "include" or "have" are intended to designate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, but one or more other features It should be understood that the presence or addition of numbers, steps, operations, components, parts, or combinations thereof is not precluded.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related art, and unless explicitly defined in the present application, they should not be interpreted in an ideal or excessively formal meaning. don't

Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram illustrating an online education service providing system 10 according to an embodiment. 2 is a diagram illustrating users taking a real-time lecture online and users taking a real-time lecture offline, according to an embodiment.

Referring to FIG. 1 , an online education service providing system 10 may include an online education server 100 , a user terminal 200 and a lecturer terminal 300 . The online education service provided by the online education service providing system 10 may be provided through a web page and/or application controlled by the online education server 100 .

The online education server 100 may receive a lecture registration request from a lecturer through the lecturer terminal 300 . The online education server 100 may register a lecture requested from the lecturer terminal 300 . The online education server 100 may receive a request for a lecture of a registered lecture from the user terminal 200 . The online education server 100 may provide lecture data to the user terminal 200 based on the received request. In other words, the online education server 100 provides the lecture data uploaded by the lecturer providing the lecture to the user terminal 200 . The lecturer may upload lecture data to the online education server 100 through the lecturer terminal 300 . For example, the lecture data includes learning content, and may include a video of a lecturer, an image for a lecture, audio data for a lecture, and text data for a lecture.

The online education server 100 may selectively provide the lecture data stored in the online education server 100 to the user terminal 200 according to the selection of users who have applied for the lecture. In other words, the user terminal 200 accesses the online providing server 100, can search for lectures provided by the lecturer through the lecturer terminal 300, selects at least one lecture, and provides the selected lecture online. It may be provided from the server 100 .

The online education server 100 may determine a lecture complement section of the lecture and provide the determined lecture supplement section to the lecturer terminal 300 . The online education server 100 may calculate the user's lecture concentration and/or course completion degree, and determine the user's learning schedule based on the lecture concentration and/or course completion degree.

The lecturer terminal 300 means a terminal of a lecturer who wants to provide a lecture, and the user terminal 200 can mean a terminal of a user who wants to take a lecture provided by the lecturer terminal 300 . The user terminal 200 and the lecturer terminal 300 are communicable desktop computers, laptop computers, notebooks, smart phones, tablet PCs, mobile phones ( mobile phone), smart watch, smart glass, e-book reader, PMP (portable multimedia player), portable game device, navigation device, digital camera, DMB (digital multimedia) broadcasting) players, digital audio recorders, digital audio players, digital video recorders, digital video players, and personal digital assistants (PDAs). there is.

Referring to FIG. 2 , users taking lectures offline and users taking lectures online are shown.

The online education server 100 may acquire a lecturer video being filmed through a camera, and display the obtained lecturer video on a user terminal of each user taking a lecture online. Users who take the lecture online may include first users who take the real-time class included in the lecture online and second users who take the recorded class included in the lecture online.

The online education server 100 may display a virtual reality space including a lecturer on the user terminal 200 as well as simply displaying a lecturer video on the user terminal 200 .

3 is a diagram showing the main components of the online education server 100.

Referring to FIG. 3 , the online education server 100 includes a service providing unit 101, a lecture complement section determining unit 102, a lecture preference calculating unit 103, an image analyzing unit 104, and a course completion degree calculating unit ( 105) and a learning schedule determining unit 106.

The service providing unit 101 may provide the user terminal 200 with learning contents for the lecture obtained from the lecturer terminal 300 . The service providing unit 101 may provide various services such as lecture progress, lecture reservation, lecture cost payment, and lecture period setting.

The lecture complement section determining unit 102 may determine a lecture supplement section based on the class information of each of the first users taking the real-time lecture online. The lecture complement section determining unit 102 may calculate a real-time lecture rating based on the class enrollment information, and determine a lecture supplement section of the lecture based on the calculated real-time lecture rating.

The lecture preference calculation unit 103 may determine a class enrollment area to which the user belongs, and may determine a class preference for the class in the class enrollment area. The lecture preference calculation unit 103 determines a class taking area based on the location information of the user terminal 200 or based on at least one of the user's residence area, work area, and main activity area obtained from the user terminal 200 You can decide where to study. Alternatively, the lecture area may be determined according to a user's input obtained through the lecturer terminal 300 .

The video analyzer 104 may compare the user video displayed through the lecturer terminal 300 with the video stored in the video DB, and determine whether the user video displayed through the lecturer terminal 300 is a fake video through the comparison. there is. In other words, in the trend of expansion of online non-face-to-face education, there are times when attendance is checked by outputting a fake video, but the lecture is not actually listened to. In order to prevent this case, the video analysis unit 104 may determine whether a user video is a fake video. For example, the video DB is a lecturer video obtained by photographing a lecturer through the lecturer terminal 300 obtained while giving a lecture in the online education server 100, and a user obtained by photographing a user through the user terminal 200. Videos and videos acquired through an external online server can be saved. The user video may be a video of a user, and the lecturer video may be a video of a lecturer.

The video analyzer 104 may determine whether the user is concentrating on the lecture based on the camera installed in the user terminal 200 and the video of the lecturer displayed through the user terminal 200 . The image analyzer 104 may check the position of the user's eyes through the camera installed in the user terminal 200 and determine the position of the user's focus based on the checked position of the eyes. The image analyzer 104 may calculate the degree of concentration of the user in the lecture according to whether the focus position of the user is located in a specific region of the lecturer's video.

The course completion degree calculation unit 105 may calculate the user's course completion degree according to the degree of concentration of the lecture calculated through the image analyzer 104 and whether or not the fake video is used. A detailed description of this may be additionally explained through FIG. 6 below.

FIG. 4 is a diagram for determining a supplementary section for a lecture based on class information of each of first users taking a real-time lecture online, and transmitting the determined supplementary section for the lecture to a lecturer terminal.

The lecture complement section determining unit 102 may determine a lecture supplement section based on the class information of each of the first users taking the real-time lecture online. The lecture complement section determining unit 102 may analyze the lecture contents through the lecturer's voice output in the determined lecture supplement section. The lecture supplementary section determining unit 102 may provide help for the lecturer to reference in creating a lecture of better quality by providing the determined supplementary section of the lecture and the analyzed lecture contents to the lecturer terminal 300 .

The lecture supplementary section determining unit 102 may calculate real-time lecture ratings based on the class attendance information of each of the first users, and provide a viewership rating function and viewership graph for determining a lecture supplementary section based on the calculated real-time lecture viewership. can create The lecture complement section determining unit 102 may calculate a real-time lecture viewing rate based on the number of first users taking the real-time lecture compared to all users who have applied for the real-time lecture. For example, if 100 people have applied for the first lecture and 70 people have taken the first lecture at a specific time, the real-time lecture viewing rate at the specific time may be 70%.

The supplementary section determining unit 102 of the lecture may calculate an average audience rating based on the real-time lecture audience rating. The supplementary section determining unit 102 of the lecture may calculate the average viewer rating through Equation 1 below.

In Equation 1, A may be an average viewing rate, f(x) may be a viewing rate function for a real-time lecture, and k may be the total lecture time of the corresponding lecture.

The lecture supplementary section determining unit 102 may determine a section with an audience rating lower than the average viewer rating as an audience rating low section in the entire lecture time of the corresponding lecture, and transmit the determined rating low viewer rating section to the lecturer terminal 300. The supplementary section determining unit 102 of the lecture may determine a section in which the viewership rating decreases as a supplementary section of the lecture, among the sections where the audience rating is insufficient.

The lecture complement section determining unit 102 may transmit the lecture supplement section and the lecture content corresponding to the lecture supplement section to the lecturer terminal 300 . The lecture supplementary section determining unit 102 provides the lecture supplementary section and the lecture content corresponding to the lecture supplementary section to the lecturer through the lecturer terminal 300, thereby providing help to the lecturer in which part to supplement.

5 is a diagram related to determining a preferred region of a lecture based on class attendance information of each of first users who take a real-time lecture online.

The lecture preference calculation unit 103 may determine a class area of each user taking a class online, and may determine a class preference for the class in the class area.

The lecture preference calculation unit 103 may obtain location information of each user taking the lecture online. The lecture preference calculator 103 may determine a plurality of lecture areas based on location information of each user. The location information may include information about the location of the user terminal 200, information about the user's residence or address, information about the user's place of work, and the like. Location information may be input by a user through the user terminal 200 .

The lecture preference calculation unit 103 is based on the number of users included in each of the plurality of course participation regions where users are located, the average viewership rating of each of the course attendance regions, the distance between each of the course attendance regions and the lecturer's terminal, etc. It is possible to calculate a preference for each of the lectures in the lecture areas. For example, the longer the distance between the lecture area and the lecturer's terminal is, the more people want to listen to the lecture even online. high can be determined. For example, as the number of users included in the course enrollment area increases, the preference for taking classes in the enrollment area increases. Therefore, the lecture preference calculation unit 103 may determine the preference for lectures in the enrollment area as the number of users included in the course enrollment area increases. there is.

The river preference calculation unit 103 may calculate the river preference of each of the course enrollment areas through Equation 2 below.

In Equation 2, S _n is the lecture preference, A is the average viewing rate of the first users in each of the course taking areas, h is the highest viewership rating of the first users in each of the taking areas, l is the N _p , the lowest viewership rating of second users, may mean the number of first users in each of the course enrollment areas, and D _- may mean a distance from the lecturer terminal to each of the course enrollment areas.

The river preference calculation unit 103 may determine, as the river preference areas, the lecture areas in which the preference for rivers is equal to or greater than a preset threshold.

The service provider 101 may provide lecture benefits to users belonging to the river preference area determined by the river preference calculation unit 103 . Course benefits may include an increase in the duration of the course, a decrease in the price of the course, and the like. The service providing unit 101 may determine a discount price for a lecture or extend a course enrollment period based on the preference for each lecture in preferred regions. When users belonging to a preferred region of a river apply for a lecture, the service provider 101 may determine a higher discount price for the lecture or extend the course duration as the preference for taking a lecture in the preferred region to which the user belongs increases.

The service provider 101 may determine a price obtained by subtracting the discount price of the lecture from the standard lecture price as the price to be paid by the user. The service provider 101 may determine a period obtained by adding a standard class enrollment period to an extended class enrollment period as a period during which the user can attend lectures.

The service provider 101 may determine the course enrollment expansion period through Equation 3 below.

In Equation 4, P _e may mean a course enrollment expansion period, M _s a standard course enrollment period, and _Sn may mean a course preference.

The service provider 101 may determine the discount price of the lecture through Equation 4 below.

In Equation 4, R _d may mean the discount price of the steel, R _r may mean the standard price of the steel, and S _n may mean the preference of the steel.

The service provider 101 provides information about the lecture (e.g., next class) to the user in the preferred area for the lecture through the mobile number of the user stored in the online education server 100 and a specific application registered with the mobile number. Uploading lectures), as well as information on other lectures (eg, newly registered lectures) may be provided to the user terminal 200 through e-mail, application push message, or SNS message.

The service provider 101 may perform an advertisement for the lecture in order to increase the demand for the lecture in the non-preferred area for the lecture, since the demand for the corresponding lecture is low in the non-preferred area for the lecture. there is.

6 is a diagram for calculating a user's course completion degree based on a user video obtained from a user terminal.

The video analyzer 104 analyzes the user's video obtained from the user terminal 200, and the course completion degree calculation unit 105 may calculate the class completion degree based on the analyzed user video.

The image analyzer 104 may calculate the degree of concentration of the user on the lecture.

The video analyzer 104 may detect the lecturer's face and the lecturer's hand from the lecturer's video displayed through the user terminal 200 . The image analyzer 104 may determine a face region corresponding to the lecturer's face and a hand region corresponding to the lecturer's hand in the lecturer's video.

The image analyzer 104 may detect a user's face from a user video obtained through the user terminal 200 . The image analyzer 104 may detect a left eye region and a right eye region of the user from the detected face, and may determine positions of pupils in each of the detected left eye region and the right eye region. The image analyzer 104 determines whether the focus of the user is located on the lecturer's face region and/or the lecturer's hand region of the lecturer's video, based on the distance between the user and the user terminal and the positions of the pupils. can judge The image analyzer 104 may calculate the degree of concentration of the user for the lecture according to the focus position of the user.

The image analyzer 104 may calculate a time for which the focus of the user is maintained on the face region and/or the hand region. The image analyzer 104 may calculate the degree of concentration of the lecture based on the time the user's focus is maintained on the face region and/or the hand region and the number of times the focus enters or exits the face region and/or the hand region.

The video analyzer 104 may determine whether the user video obtained from the user terminal 200 is a video of a real user. In other words, the video analyzer 104 compares the user video obtained from the user terminal 200 and the video stored in the video DB, and calculates the video similarity between the user video and the video stored in the video DB through the comparison. can The image analyzer 104 may determine whether the user's video is a fake video based on the calculated image similarity.

The image analyzer 104 may calculate the first similarity based on pixel values of image frames included in the user's video obtained from the user terminal 200 and pixel values of the image frames included in the video stored in the image DB. .

The image analyzer 104 generates first difference image frames by differentiating pixel values of corresponding positions of two image frames that are temporally adjacent to each other among image frames constituting a user's video. can do. The image analyzer 104 may divide each of the first difference image frames into a plurality of regions and calculate an average value of pixels for each of the plurality of divided regions. The image analyzer 104 may generate first feature vectors based on the calculated average values of pixels.

The image analyzer 104 differentiates pixel values corresponding to each other in two image frames temporally adjacent to each other among image frames constituting a video stored in the image DB, and then obtains a second difference image. frames can be created. The image analyzer 104 may divide each of the second difference image frames into a plurality of regions and calculate an average value of pixels for each of the plurality of divided regions. The image analyzer 104 may generate second feature vectors based on the calculated average values of pixels.

The image analyzer 104 may calculate a first similarity between the user video and the video stored in the video DB based on the first feature vectors and the second feature vectors.

The image analysis unit 104 extracts image frames at predetermined time intervals from the user's video and the video stored in the video DB, and determines the area of the face detected in each of the extracted image frames, the facial expression change of the detected face, and the like. 2 similarity can be calculated.

The image analyzer 104 may calculate the image similarity based on the calculated first similarity and the calculated second similarity. The image analyzer 104 may calculate the image similarity through Equation 5 below.

In Equation 5, S is the image similarity, S ₁ is the first similarity, S ₂ is the second similarity, w ₁ is the first weight for the first similarity, and w ₂ is the second weight for the second similarity. can

The video analyzer 104 may determine the user video as a fake video when the video similarity between the user video obtained from the user terminal 200 and the video stored in the video DB is greater than or equal to a threshold similarity.

The course completion degree calculation unit 105 may determine the course completion degree based on a ratio of playback time of the fake video to total time of the corresponding lecture. The course completion degree calculation unit 105 may calculate the course completion degree through Equation 6 below.

In Equation 6, C _d is the course completion degree, T _r is the playback time of the fake video, T _c is the lecture time of the corresponding lecture, T- _s is the user's focus on at least one of the lecturer's face region or the lecturer's hand region This may mean the holding time.

7 is a diagram related to determining a user's learning schedule.

The learning schedule determining unit 106 may determine the user's learning schedule based on the class preference and the class completion degree calculated through the class completion degree calculation unit 105 . The learning schedule determination unit 106 may provide the user terminal 200 with the determined learning schedule of the user.

The learning schedule determining unit 106 may allow a user who does not meet a predetermined criterion for the lecture to review the lessons included in the lecture. The learning schedule determining unit 106 may determine a possible review period based on the degree of course completion. For example, the learning schedule determining unit 106 may determine a longer period available for review as the degree of completion of classes increases, and determine a shorter period available for review as the degree of completion of classes decreases. The learning schedule determining unit 106 may allow users who exceed a preset standard to attend the next class of the lecture.

FIG. 8 is a diagram showing the hardware configuration of the online education server 100 according to FIG. 1 .

Referring to FIG. 8, the online education server 100 stores at least one processor 110 and instructions instructing the at least one processor 110 to perform at least one operation. May contain memory.

The at least one operation may include at least some of the operations or functions of the online education server 100 and may be implemented in the form of instructions and performed by the processor 110 .

Here, the at least one processor 110 may mean a central processing unit (CPU), a graphics processing unit (GPU), or a dedicated processor for performing methods according to embodiments of the present invention. can Each of the memory 120 and the storage device 160 may include at least one of a volatile storage medium and a non-volatile storage medium. For example, the memory 120 may be one of a read only memory (ROM) and a random access memory (RAM), and the storage device 160 may be a flash-memory. , a hard disk drive (HDD), a solid state drive (SSD), or various memory cards (eg, a micro SD card).

In addition, the server 100 may include a transceiver 130 that performs communication through a wireless network. In addition, the online education server 100 may further include an input interface device 140, an output interface device 150, a storage device 160, and the like. Each component included in the server 100 may be connected by a bus 170 to communicate with each other.

9 is a diagram illustrating a wireless communication system that can be applied in a communication process according to an embodiment of the present invention. 10 is a diagram illustrating a base station in the wireless communication system according to FIG. 9 . 11 is a diagram illustrating a terminal in the wireless communication system according to FIG. 9 . 12 is a diagram illustrating a communication interface in the wireless communication system according to FIG. 9 .

Hereinafter, an example of a wireless communication network system supporting communication between a server and terminals according to the present invention will be described in detail as an example. In the following description, a first node (device) may be an anchor/donor node or a centralized unit (CU) of an anchor/donor node, and a second node (device) may be an anchor/donor node or a distributed unit (DU) of a relay node can be

A base station (BS), terminal, server, etc. may be included as a part of a node using a radio channel in a wireless communication system.

A base station is a terminal and a network infrastructure that provides wireless access to terminals. A base station has a coverage defined as a predetermined geographic area according to a distance over which a signal can be transmitted.

A base station, like a "base station," is referred to as "access point (AP)", "enodeb (eNB)", "5th generation (5G) node", "wireless point", " It may be referred to as a transmission/reception point (TRP).

The base station, the terminal, and the terminal may transmit and receive wireless signals in a millimeter wave (mmWave) band (eg, 28 GHz, 30 GHz, 38 GHz, and 60 GHz). At this time, the base station, the terminal, and the terminal may perform beamforming to improve the channel gain. Beamforming may include transmit beamforming and receive beamforming. That is, the base station, the terminal, and the terminal can give directivity to the transmitted signal and the received signal. To this end, the base station, the terminal, and the terminal may select a serving beam through a beam search procedure or a beam management procedure. After that, communication may be performed using a resource carrying a serving beam and a resource having a quasi co-located relationship.

A first antenna port and a second antenna port are considered quasi-colocated if the large-scale properties of the channel through which the symbol of the first antenna port carries can be inferred from the channel through which the symbol of the second antenna port carries. The large-scale properties may include one or more of delay spread, Doppler spread, Doppler shift, average gain, average delay, and spatial Rx parameters.

Hereinafter, a base station in the wireless communication system described above is exemplified. The terms "-module", "-unit" or "-er" used below may mean a unit that processes at least one function or operation, and may include hardware, software, or both hardware and software. It can be implemented as a combination of

The base station may include a wireless communication interface, a backhaul communication interface, a storage unit and a controller.

The wireless communication interface performs a function of transmitting and receiving signals through a wireless channel. For example, the wireless communication interface may perform a conversion function between a baseband signal and a bit stream according to a physical layer standard of the system. For example, in data transmission, a radio communication interface encodes and modulates a transmitted bit stream to generate composite symbols. Also, upon receiving data, the wireless communication interface demodulates and decodes the baseband signal to reconstruct the received bit stream.

The wireless communication interface performs a function of transmitting and receiving signals through a wireless channel. For example, the wireless communication interface may perform a conversion function between a baseband signal and a bit stream according to a system physical layer standard. For example, in data transmission, a radio communication interface encodes and modulates a transmitted bit stream to generate composite symbols. Also, upon receiving data, the wireless communication interface demodulates and decodes the baseband signal to reconstruct the received bit stream.

In addition, the wireless communication interface up-converts a baseband signal into a radio frequency (RF) band signal, transmits the converted signal through an antenna, and down-converts the RF band signal received through the antenna into a baseband signal. To this end, the wireless communication interface includes a transmission filter, a reception filter, an amplifier, a mixer, an oscillator, a digital-to-analog converter (DAC), An analog-to-digital converter (ADC) and the like may be included. Also, the wireless communication interface may include a plurality of transmit/receive paths. Additionally, the wireless communication interface may include at least one antenna array comprising a plurality of antenna elements.

In terms of hardware, the wireless communication interface may include a digital unit and an analog unit, and the analog unit may include a plurality of sub-units according to operating power, operating frequency, and the like. A digital unit may be implemented with at least one processor (eg, a digital signal processor (DSP)).

The wireless communication interface transmits and receives signals as described above. Accordingly, a wireless communication interface may be referred to as a “transmitter”, “receiver” or “transceiver”. In addition, in the following description, transmission and reception performed through a wireless channel may be used as a meaning including processing performed in a wireless communication interface as described above.

The backhaul communication interface provides an interface for communicating with other nodes in the network. That is, the backhaul communication interface converts the bit stream transmitted to other nodes, for example, other access nodes, other base stations, upper nodes or core networks from base stations into physical signals, and the physical signals received from other nodes into bits. convert to stream

The storage unit stores data such as basic programs, applications, and setting information for operation of the base station. The storage unit may include volatile memory, non-volatile memory, or a combination of volatile and non-volatile memory.

The controller controls the overall operation of the base station. For example, the controller transmits and receives signals through a wireless communication interface or a backhaul communication interface. Also, the controller writes data to the storage unit and reads the recorded data. The controller can perform protocol stack functions required by communication standards. According to another implementation, a protocol stack may be included in a wireless communication interface. To this end, the controller may include at least one processor.

According to an embodiment, the controller may control the base station to perform an operation according to an embodiment of the present invention.

According to various embodiments, a donor node of a wireless communication system includes at least one processor, includes a transceiver operably coupled to the at least one processor, and includes a plurality of radio bearers for a terminal accessing the relay node. configured to transmit to a relay node a first message including first information related to the donor node about; receive a second message including second information related to the relay node regarding a plurality of radio bearers for the terminal from the relay node; Data for the terminal may be transmitted to the relay node. Data may be transmitted to the terminal through a plurality of radio bearers based on the first information and the second information.

According to various embodiments, a radio bearer among a plurality of radio bearers may integrate a plurality of radio bearers. the at least one processor is also configured to determine a radio bearer for a terminal accessing the relay node and multiple radio bearers aggregated by the radio bearer; Alternatively, a radio bearer for a UE accessing a relay node may be determined.

According to various embodiments, the first message may include one or more of: identification of a terminal accessing the relay node; display information indicating the type of terminal accessing the relay node; information about a radio bearer of a terminal accessing a relay node; information about a radio bearer transmitted by a terminal accessing a relay node; information about a tunnel established for a radio bearer between the donor node and the relay node; information about integrated multiple radio bearers; radio bearer mapping information; information about the address of the side of the donor node; information about the address of the relay node side; indication information corresponding to a radio bearer of a terminal accessing the relay node; indication information indicating the relay node to allocate a new address to a radio bearer for a terminal accessing the relay node; a list of address information that cannot be used by a relay node that transmits radio bearer data of a terminal accessing the relay node; and information related to security configuration.

According to various embodiments, the second message may include one or more of: identification of a terminal accessing the relay node; information about the radio bearer granted by the relay node; information about radio bearers not acknowledged by the relay node; information about radio bearers partially granted by the relay node; radio bearer mapping information; Configuration information of a terminal accessing the relay node created by the relay node; information about the address of the relay node side; and information related to security configuration.

According to various embodiments, the second message may further include information on integrated multiple radio bearers.

According to various embodiments, the donor node may include a central unit of the donor node, and the relay node may include a distribution unit of the donor node.

According to various embodiments, a relay node of a wireless communication system includes at least one processor, includes a transceiver operably coupled to the at least one processor, and provides a plurality of information from a donor node to a terminal accessing the relay node. configured to receive a first message including first information related to a donor node for a radio bearer of; transmit to the donor node a second message including second information related to the relay node for the plurality of radio bearers for the terminal; Data on the terminal may be received from the donor node. Data may be transmitted to the terminal through a plurality of radio bearers based on the first information and the second information.

According to various embodiments, a radio bearer among a plurality of radio bearers may integrate a plurality of radio bearers. the at least one processor is also configured to determine a radio bearer for a terminal accessing the relay node and multiple radio bearers aggregated by the radio bearer; Alternatively, multiple radio bearers integrated by radio bearer may be determined.

According to various embodiments, the first message may include one or more of: identification of a terminal accessing the relay node; display information indicating the type of terminal accessing the relay node; information about a radio bearer of a terminal accessing a relay node; information about a radio bearer transmitted by a terminal accessing a relay node; information about a tunnel established for a radio bearer between the donor node and the relay node; information about integrated multiple radio bearers; radio bearer mapping information; information about the address of the side of the donor node; information about the address of the relay node side; indication information corresponding to a radio bearer of a terminal accessing the relay node; indication information indicating the relay node to allocate a new address to a radio bearer for a terminal accessing the relay node; a list of address information that cannot be used by a relay node that transmits radio bearer data of a terminal accessing the relay node; and information related to security configuration.

According to various embodiments, the second message may include one or more of: identification of a terminal accessing the relay node; information about the radio bearer granted by the relay node; information about radio bearers not acknowledged by the relay node; information about radio bearers partially granted by the relay node; radio bearer mapping information; Configuration information of a terminal accessing the relay node created by the relay node; information about the address of the relay node side; and information related to security configuration.

According to various embodiments, the second message may further include information on integrated multiple radio bearers.

According to various embodiments, the donor node may include a central unit of the donor node, and the relay node may include a distribution unit of the donor node.

Hereinafter, components of a terminal in the wireless communication system described above are illustrated. Components of a terminal to be described below are components of a general-purpose terminal supported by a wireless communication system, and may be merged or integrated with components of a terminal according to the foregoing contents, and may overlap or conflict with the above drawings. It can be interpreted that the content described with reference takes precedence. The terms "-module", "-unit" or "-er" used below may mean a unit that processes at least one function.

The terminal includes a communication interface, a storage unit and a controller.

The communication interface performs a function of transmitting and receiving signals through a wireless channel. For example, the communication interface performs a conversion function between a baseband signal and a bit stream according to the physical layer standard of the system. For example, in data transmission, a communication interface encodes and modulates a transmission bit stream to generate composite symbols. Also, when receiving data, the communication interface demodulates and decodes the baseband signal to reconstruct the received bit stream. Further, the communication interface up-converts the baseband signal to an RF-band signal, transmits the converted signal through an antenna, and down-converts the RF-band signal received through the antenna into a baseband signal. For example, the communication interface includes a transmission filter, a reception filter, an amplifier, a mixer, an oscillator, and a digital-to-analog converter (DAC). , an analog-to-digital converter (ADC), and the like.

Also, the communication interface may include a plurality of transmit/receive paths. Additionally, the communication interface may include at least one antenna array comprising a plurality of antenna elements. On the hardware side, the wireless communication interface may include a digital circuit and an analog circuit (eg, a radio frequency integrated circuit (RFIC)). A digital circuit may be implemented with at least one processor (eg, DSP). A communication interface may include multiple RF chains. The communication interface may perform beamforming.

The communication interface transmits and receives signals as described above. Accordingly, a communication interface may be referred to as a “transmitter”, “receiver” or “transceiver”. In addition, in the following description, transmission and reception performed through a radio channel may be used as a meaning including processing performed in a communication interface as described above.

The storage unit stores data such as basic programs for operation of the terminal, applications, and setting information. The storage unit may include volatile memory, non-volatile memory, or a combination of volatile and non-volatile memory. In addition, the storage unit provides stored data according to the request of the controller.

The controller controls the overall operation of the terminal. For example, the controller sends and receives signals through a communication interface. Also, the controller writes data to the storage unit and reads the recorded data. The controller can perform protocol stack functions required by communication standards. According to another implementation, a protocol stack may be included in the communication interface. To this end, the controller may include at least one processor or microprocessor or reproduce parts of a processor. Also, a part of the communication interface or controller may be referred to as a communication processor (CP).

According to an embodiment of the present invention, a controller may control a terminal to perform an operation according to an embodiment of the present invention.

Hereinafter, a communication interface in a wireless communication system is illustrated.

The communication interface includes encoding and modulation circuitry, digital beamforming circuitry, a plurality of transmission paths, and analog beamforming circuitry.

Encoding and modulation circuitry performs channel encoding. At least one of a low-density parity check (LDPC) code, a convolution code, and a polar code may be used for channel encoding. An encoding and modulation circuit generates modulation symbols by performing constellation mapping.

A digital beamforming circuit performs beamforming on a digital signal (eg, a modulation symbol). To this end, a digital beamforming circuit multiplexes modulation symbols by beamforming weights. Beamforming weights can be used to change the size and phrase of a signal, and can be referred to as a "precoding matrix" or "precoder". The digital beamforming circuit outputs digital beamformed modulation symbols to a plurality of transmission paths. In this case, according to a multiple input multiple output (MIMO) transmission method, modulation symbols may be multiplexed or the same modulation symbol may be provided to a plurality of transmission paths.

The plurality of transmission paths convert digital beamformed digital signals into analog signals. To this end, each of the plurality of transmission paths may include an inverse fast fourier transform (IFFT) computation unit, a cyclic prefix (CP) insertion unit, a DAC, and an up conversion unit. The CP insertion unit is for an orthogonal frequency division multiplexing (OFDM) method and may be omitted when another physical layer method (eg, a filter bank multi-carrier: FBMC) is applied. That is, the plurality of transmission paths provide independent signal processing processes for a plurality of streams generated through digital beamforming. However, depending on the implementation, some elements of the plurality of transmission paths may be commonly used.

An analog beamforming circuit performs beamforming on an analog signal. To this end, the digital beamforming circuit multiplexes analog signals by beamforming weighting values. The beamformed weights are used to change the amplitude and phrase of the signal. More specifically, the analog beamforming circuit may be configured in various ways according to a connection structure between a plurality of transmission paths and an antenna. For example, each of a plurality of transmission paths may be connected to one antenna array. In another example, multiple transmission paths may be coupled to one antenna array. In another example, multiple transmission paths may be adaptively coupled to one antenna array or may be coupled to two or more antenna arrays.

The methods according to the present invention may be implemented in the form of program instructions that can be executed by various computer means and recorded on a computer readable medium. Computer readable media may include program instructions, data files, data structures, etc. alone or in combination. Program instructions recorded on a computer readable medium may be specially designed and configured for the present invention or may be known and usable to those skilled in computer software.

Examples of computer readable media may include hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions may include not only machine language codes generated by a compiler but also high-level language codes that can be executed by a computer using an interpreter and the like. The hardware device described above may be configured to operate with at least one software module to perform the operations of the present invention, and vice versa.

In addition, the above-described method or device may be implemented by combining all or some of its components or functions, or may be implemented separately.

Although the above has been described with reference to preferred embodiments of the present invention, those skilled in the art will variously modify and change the present invention within the scope not departing from the spirit and scope of the present invention described in the claims below. You will understand that it can be done.

100: online education server 200: user terminal
300: lecturer terminal

Claims (9)

An online education server that provides online education services including real-time online classes and recorded online classes,
A service providing unit that registers a lecture requested from a lecturer terminal, obtains a course registration input for the lecture through a user terminal, and provides learning contents included in the lecture to the user terminal according to the obtained course registration input. ;
a lecture complement section determiner for determining a lecture supplement section of the lecture based on class information of each of the first users taking the real-time class included in the lecture online;
A lecture preference calculation unit for determining course areas of each of the first users and users including second users who are taking a recorded class included in the lecture online, and calculating a lecture preference of each of the determined taking areas. ;
An image analyzer for determining a degree of concentration of a user corresponding to the user terminal based on a user video obtained from the user terminal;
a course completion degree calculation unit that calculates a course completion degree of the user for the lecture based on the degree of concentration of the lecture; and
A learning schedule determining unit determining a learning schedule for the lecture according to the degree of completion of the course;
The service provider,
An online education server that provides the user video to the lecturer terminal and provides the lecturer video in which the lecturer corresponding to the lecturer terminal is photographed to the user terminal. In claim 1,
The supplementary section determining unit of the river,
Based on the number of the first users taking the real-time class compared to all users who have applied for the real-time class, calculating the real-time lecture viewership of the lecture,
Generating an audience rating function and a viewer rating graph based on the calculated viewer rating of the real-time lecture;
The online education server for determining the supplementary section of the lecture based on at least one of an average viewer rating based on the real-time lecture viewer rating, the viewer rating function, and the viewer rating graph. In claim 2,
The river preference calculator,
The online education server for calculating the preference for the lecture based on the number of users included in each of the attending areas and the average viewing rate for the lecture in each of the attending areas. The method of claim 1,
The video analysis unit,
Detecting the lecturer's face region and the lecturer's hand region in the lecturer's video displayed through the user terminal;
detecting a left eye region and a right eye region of the user in a user video obtained through the user terminal, and determining positions of pupils in each of the detected left eye region and the right eye region;
Based on the distance between the user and the user terminal and the positions of the pupils, it is determined whether the focus of the user is located in the lecturer's face region or the lecturer's hand region of the lecturer's video,
The online education server for determining the degree of concentration of the user in the lecture according to the focus position of the user. In claim 4,
The video analysis unit,
Determining authenticity of the user video obtained from the user terminal;
calculating a first similarity based on pixel values of first image frames constituting the user's video and pixel values of second image frames constituting the video stored in an image DB;
Calculating a second similarity based on a face detected in each of the user video and the video stored in the video DB;
calculating an image similarity between the user video and a video stored in the video DB based on the first similarity and the second similarity;
If the image similarity is greater than or equal to the threshold similarity, determining the user video as a fake video, the online education server. As a method for providing online education services including real-time online classes and recorded online classes,
Registering a lecture requested from a lecturer terminal;
Obtaining a course registration input for the lecture through a user terminal;
providing learning content included in the lecture to the user terminal according to the acquired course registration input;
determining a lecture supplementary section of the lecture based on class information of each of the first users who take the real-time class included in the class online;
determining course attendance areas of each of the users including the first users and second users who are taking the recorded class included in the lecture online;
Calculating a preference for each of the lecture areas;
determining a degree of concentration of a lecture of a user corresponding to the user terminal based on a user video obtained from the user terminal;
calculating a course completion degree of the user for the lecture based on the degree of concentration of the lecture; and
And determining a learning schedule for the lecture according to the degree of completion of the course. A non-transitory recording medium on which a program for executing the method according to claim 6 is recorded and can be read by a computer. A computer program recorded on a non-transitory recording medium to execute the method according to claim 6 in an online education server. A system that provides online education services including real-time online classes and recorded online classes,
Lecturer terminal providing learning content to the online education server;
a user terminal acquiring the learning content from the online education server; and
The online education server interworking with the user terminal and the lecturer terminal and providing an online education service to a user corresponding to the user terminal,
The online education server,
A service for registering a lecture requested by the lecturer terminal, acquiring a course registration input for the lecture through the user terminal, and providing learning contents included in the lecture to the user terminal according to the obtained course registration input. provision department;
a lecture supplementary section determiner for determining a supplementary section of the lecture based on class information of each of the first users who take the real-time class included in the lecture online;
a lecture preference calculator configured to determine lecture areas of each of the first users and users including second users who are taking a recorded class included in the lecture online, and to calculate a preference for each of the lecture areas;
an image analyzer for analyzing a user video obtained from the user terminal and determining a degree of concentration of the user on the basis of the analyzed user video;
a course completion degree calculation unit that calculates a course completion degree of the user for the lecture based on the degree of concentration of the lecture; and
A system for providing online education services comprising a learning schedule determining unit determining a learning schedule for the lecture according to the degree of completion of the course.

Images