CN115393615A

CN115393615A - Remote intelligent monitoring method based on picture similarity comparison

Info

Publication number: CN115393615A
Application number: CN202210771693.9A
Authority: CN
Inventors: 蒋现新
Original assignee: Capital University of Physical Education and Sports
Current assignee: Capital University of Physical Education and Sports
Priority date: 2022-06-30
Filing date: 2022-06-30
Publication date: 2022-11-25

Abstract

The invention discloses a remote intelligent monitoring method based on picture similarity comparison, which comprises the steps of acquiring a remote computer or mobile phone desktop through a screen capture method, then compressing pictures, uploading to a monitoring server, and timely discovering students watching irrelevant contents during a class through an intelligent picture similarity comparison algorithm so that teachers and parents can know the class state of the students in real time. Due to epidemic situations, the number of times of home class or home work becomes large, and a method for remotely monitoring the online class is urgently needed, so that students are prevented from playing games or doing other irrelevant things during class hours, and the quality of the online class is improved. The invention realizes remote real-time intelligent monitoring, occupies very small network bandwidth, does not influence the experience of students on the internet class, runs at the background, has no perception to students and has important and profound significance for improving the national teaching technology level.

Description

Remote intelligent monitoring method based on picture similarity comparison

Technical Field

Remote education, remote monitoring

Background

Because of epidemic situation and other reasons, the number of times that the student surfs the internet lesson becomes many, and online education is a distance education mode, nevertheless because the student's of surfing the internet lesson quantity is many and not in the front, and the mr of attending a lesson can't in time know student's state of attending a lesson and what is all doing in the lesson. A common method is to install a camera near each student and remotely observe the students in a video mode. However, the monitoring mode of the camera needs to occupy a larger bandwidth, so that a blocking phenomenon is easily caused to normal students during class, and the students can be aware of avoiding the camera and cannot complete real monitoring. In addition, video monitoring requires manual visual observation, and monitoring tens of students at the same time is a very time-consuming and labor-consuming matter. The invention aims at solving the problems and provides a novel technical solution.

Disclosure of Invention

The invention comprises three parts, namely a student client, a server and a teacher client. The student client side mainly obtains what the student is doing through an API mode of desktop snapshot, compresses desktop snapshot pictures and transmits the pictures to the server through a network. The server mainly receives the picture information sent by the student client, performs picture similarity algorithm processing with standard teacher content, completes automatic anomaly detection, and sends the abnormal student desktop pictures to the teacher client. The teacher client is mainly used for manually confirming the abnormal pictures, and can remind students or inform parents of the abnormal pictures in time if the students are confirmed not to take lessons seriously.

The API of the desktop screenshot is provided on the Windows platform, the IOS platform and the Android, and corresponding student client sides can be developed on the platforms, so that monitoring on various types of equipment is achieved. By setting the sampling time of the desktop screenshot and adopting the picture compression technology, the network bandwidth of picture transmission can be greatly reduced, and the influence on normal class attendance is avoided. Meanwhile, the picture acquisition is automatically carried out in the background, no perception is provided for students, and the students cannot avoid monitoring. Through the image similarity comparison algorithm, the intellectualization and the automation of abnormal condition discovery are realized, and the workload of teachers is greatly reduced.

Principle of operation of the system

1. Before a student is in class, a monitoring client needs to be installed in equipment used for the class, and the student logs in a server through an account.

2. When a teacher starts to attend a class, the content of the class is displayed on the desktop of the student in a video mode through the network and is explained through audio. Meanwhile, the server needs to record the video of the teacher in class.

3. A timer is started at a student client, the contents of the student desktop which are being displayed are acquired periodically through a desktop screenshot API (application program interface), so that what contents are watched by the student can be known, and the current timestamp information can be recorded.

4. And (4) compressing the screenshot at the student client, wherein the screenshot can be compressed by adopting a compression algorithm such as JPEG (joint photographic experts group) or other image compression algorithms. And then transmitting the compressed picture, the timestamp information of the screenshot, the student information and the like to a server through a network.

5. The server receives the pictures sent by the student client, acquires compression algorithm information according to the compression format, and decompresses the pictures by using the corresponding decompression algorithm. And then acquiring pictures in videos for recording the teaching of the teachers by the corresponding servers according to the timestamp information of the screenshots.

6. And comparing the desktop screenshot pictures of the students with the teacher teaching pictures recorded by the server by using a picture similarity comparison intelligent algorithm, and if the similarity is lower than a specified value, sending the related information of the students and the desktop screenshot of the students to a teacher client.

7. And the teacher confirms whether the students watch the teaching or do other things manually according to the received information. Then remind the student or inform the parents of the student in time.

Technical requirements

1. The system time of the student client needs to be corrected through a time synchronization protocol NTP with the time of the server, and the time difference between the student client and the server is avoided.

2. Desktop screenshots of the student client are not frequent, and generally 1 second to 5 seconds are required for screenshot.

3. The compression algorithm and the comparison algorithm of the picture are suggested to be realized by OpenCV open source software, and can also be realized by other third-party software.

4. In the server, a real-time video for recording teaching of the teacher needs to be provided for picture comparison of the invention, and can also be used for playback of future teaching.

Intelligent algorithm for picture similarity

Considering that the desktop resolution of the student client is different from the video resolution recorded by the server, an intelligent picture similarity algorithm is required to be adopted to compare the similarity of the two pictures. The method mainly comprises the following steps:

1) Inputting a desktop screenshot of the student, which is called picture 1; the picture of the lecture recorded by the server is input and is called picture 2.

2) And acquiring the resolutions of the picture 1 and the picture 2, selecting the lowest resolution as a target, and adjusting the size of the high-resolution picture to the size with the lower resolution. Two pictures with the same resolution are obtained.

3) The length and the width of two pictures with the same resolution are reduced to 1/4 size so as to blur slight difference and ensure that the similarity has better fault tolerance.

4) And converting the two reduced pictures into gray pictures.

5) And counting the total number of pixels of one gray picture.

6) And comparing the two gray level pictures point by point, and recording the point of which the difference between the two pixel values of the same coordinate is larger than a specified range as a difference point. The specified range is adjusted according to actual needs, and the larger the range is, the better the fault tolerance is, but the problem finding capability is reduced. Typically, for pixel values of 0 to 255, the difference selection 15 is referenced.

7) And counting the number of the difference points.

8) The number of difference points is divided by the total number of pixels, and this ratio is output as the similarity.

Drawings

Fig. 1 is a functional diagram of a remote monitoring whole system based on picture comparison, and describes key components, main principles and main methods of the system.

FIG. 2 is a functional diagram of an intelligent comparison algorithm for picture similarity.

The technical scheme has the advantages

The remote monitoring method based on the picture similarity comparison provides a technical means for implementing real-time remote monitoring on students accessing a lesson, so that a teacher can know the class state of the students in time, and finds out the students not paying attention in class through an intelligent picture similarity algorithm, thereby greatly reducing the burden of the teachers accessing the lesson. The technical scheme of the invention has the advantages of small occupied resource, slight occupation of bandwidth and CPU computing resource, no perception of data acquisition to students, no influence on normal class of students and important and profound significance on development and popularization of remote education.

Detailed Description

1) The software of the student client and the software of the student on the internet lesson are implemented more conveniently, and the monitoring software is started at the same time when the student starts the software of the internet lesson, so that the student cannot sense the monitoring.

2) Preferably, the teacher has two devices to log on to the class, one device giving the student a normal class, and the student can see all the teacher's operations on that device. The other device is dedicated to receiving the monitoring messages, and the students cannot see the messages of the teacher's monitoring client. Therefore, students do not know that the students are remotely monitored, and the school attendance performance is more real. If the teacher runs the lesson-taking software and the monitoring software on one device, the monitoring information is likely to be seen by students when the operation is not proper, and the emotion of the students can be influenced.

3) The server may be implemented by a single process or multiple processes, such as a single thread or multiple threads, depending on the performance of the server. Related to the number of students attending class at the same time and related to the time frequency of acquisition of the screenshots on the desktop. The proposal adopts a single-process multi-thread mode, receives the message sent by the student client at a fixed port, and adopts a UDP communication protocol.

Claims

1. A remote intelligent monitoring method based on picture similarity comparison is characterized in that the content being displayed on a student terminal is obtained through a desktop snapshot method, snapshot pictures are compressed, timestamp information is uploaded to a monitoring server, the monitoring server obtains corresponding teacher teaching content pictures according to timestamps, then whether the student desktop snapshots contain content irrelevant to the lessons or not is detected through a picture similarity comparison intelligent algorithm, if the content irrelevant to the lessons is found, the teacher is informed in time, and the teacher is reminded of students or parents of the students in time.

2. The remote monitoring method according to claim 1, wherein the content being displayed at the remote end is obtained in a desktop snapshot manner.

3. The remote monitoring method according to claim 1, wherein whether the remote end displays content irrelevant to the lecture is detected by a picture similarity comparison method, which is referred to as abnormal detection for short.

4. The remote monitoring method according to claim 1, wherein a teacher giving lessons is notified in time after a remote abnormality is detected by abnormality detection.