Intelligent experiment investigation system based on multi-scene video analysis
Technical Field
The invention belongs to the technical field of practice teaching, and particularly relates to an intelligent experiment investigation system based on multi-scene video analysis.
Background
Besides theoretical teaching, practice teaching is an important link for talent culture, and is a main mode for deepening students 'understanding of theories learned and improving students' practice ability. However, compared with theoretical teaching, practical teaching is the weakest link in the teaching process. The existing experimental teaching management measures have the following problems: in practice, the lack of supervision of the student's performance by the instructor results in a serious shortage of information. In the practical classroom, generally, a teacher is guided to conduct coaching, answering and accepting tests and handling emergencies on students, and the teacher often has no time to take each student due to a large number of students, so that severe conditions such as modern making and copying cannot be avoided. The current practice monitoring is completely undertaken by the manual labor of the teacher, the obtained information amount is small, the related range is narrow, the accurate judgment of the teacher on the learning condition is influenced, and the learning attitude and the cultivation of the autonomous learning ability of the students are influenced.
Since the 21 st century, the development of internet technology and artificial intelligence technology has been a breakthrough and is becoming mature. "internet + education" and "AI + education" are gradually becoming research hotspots. The technology of website development, face recognition, gesture detection and the like is applied to the management of practice teaching, so that the problem of insufficient supervision in teaching can be solved, and perfect and efficient practice teaching practice can be provided through data feedback. At present, researchers at home and abroad carry out considerable research on detection of cheating behaviors automatically detected by using computer vision, for example, in a document [1] (Meiyi, Sunwei, Emulation of examination cheating behaviors based on intelligent vision [ J ] computer simulation, 2014,31(04):219-222.), an examination cheating behavior identification method based on an immune multi-Agent algorithm is adopted, and features related to the examination cheating behaviors are extracted by using an immune multi-Fang distance minimum value method to complete identification of the examination cheating behaviors; or in the document [2] (Yohannes, V.Ayumi and M.I.Fanany, "Multimodal compliant modules by superpixel segmentation and point-in-time chemistry detection,"2016International Conference on Advanced Computer Science and Information Systems (ICACSIS), Malang,2016, pp.391-396.) by using a Multimodal resolvable Model (MODEC) with superpixel segmentation to detect body joint positions and to classify and detect certain classes of cheating activities using Conditional Random Fields (CRF).
However, these studies only discuss the detection of cheating in a written exam scenario. For the scene of the pen-test examination, because the scene is located in an examination room, only a small amount of articles such as stationery, test paper and the like are arranged on the desktop of an examinee, the seats of the examinee are also very dispersed, the visual field is wide, and abnormal behaviors of students are very easy to find. Therefore, the observation and the detection of cheating behaviors of all examinees in a classroom can be realized by using only a few cameras, and the hardware realization difficulty is relatively low. In the teaching scene in the experimental classroom, a large number of experimental devices are often arranged, the positions are more crowded, and the difficulty in detecting abnormal activities of students is higher. If a scheme similar to the detection of cheating behavior in an examination scenario is adopted, the activities of the student will be difficult to perceive. Therefore, the detection scheme of cheating behaviors in the written examination scene is not suitable for detecting similar illegal behaviors in an experimental classroom.
Meanwhile, the detection of cheating behaviors in the scene of the written examination only focuses on the identification and classification of user behaviors, does not consider the application of technologies such as article tracking identification, digital detection recording and the like, and does not develop the functions such as automatic recording and the like. For the detection of the violation in the written examination scene, the techniques may not be used necessarily because the violation mainly consists of obvious cheating actions such as peeping, paper slip transmission and the like. However, the illegal behavior in the experimental teaching not only exists in the abnormal behavior of the user, but also is easily realized through data modification, equipment exchange and other modes, so that the technology is indispensable to intelligent investigation in the experimental teaching.
Therefore, in the scene of classroom experiments, the research scope of the existing research is too narrow, and the applicability degree is very low.
Disclosure of Invention
Aiming at the defects or improvement requirements of the prior art, the invention provides an intelligent experiment investigation system based on multi-scene video analysis, and aims to solve the technical problems that experiment records are complicated and illegal behaviors are difficult to detect in the experiment teaching process.
In order to achieve the above object, the present invention provides an intelligent experimental investigation system based on multi-scene video analysis, comprising: the system comprises an interactive terminal, a multi-angle camera and an experimental host;
the interactive terminal is used for performing man-machine interaction with a user, and realizing identity verification, experiment authorization and experiment data submission;
the multi-angle camera is used for collecting user operation behaviors and experimental data displayed by an experimental instrument in a multi-angle manner;
the experiment host is used for analyzing, processing and result feedback of images collected by the multi-angle camera and alarming abnormal behaviors.
Furthermore, the multi-angle camera comprises a top shooting camera, a bottom shooting camera and a fixed camera;
the overhead camera is fixed right above the operation area of the experiment table and used for acquiring images in the experiment process of the operation area;
the overhead camera is fixed on a desktop beside an operation area of the experiment table and is used for acquiring images of behaviors of the user;
and the fixed camera is fixed at the edge of the experiment table and used for acquiring the image of the experimental data displayed by the experimental instrument.
Further, the image data specifically collected by the overhead camera includes: the number of experimental devices and the number of hands of the user.
Further, the interactive terminal performs man-machine interaction with the user to realize identity authentication and experimental authorization, and specifically includes:
numbering the experimental equipment, the experimental table and the user respectively, and matching the experimental equipment, the experimental table and the user in a one-to-one manner;
a user applies for entering an experiment through an interactive terminal on the experiment table with the matched serial number;
the camera head is used for collecting face information of the user and judging whether the face information is consistent with the face information matched with the background; if so, successfully verifying the identity and starting an experiment; if not, the identity authentication fails, and the face information acquisition is carried out again.
Further, the experiment host computer carries out analysis, processing and result feedback to the image that multi-angle camera gathered to report to the police unusual action, specifically include:
according to image information acquired by the downward shooting camera, whether the downward shooting camera is shielded or not, and abnormal alarms are given to experimental equipment and the hand of a user;
and according to the image information acquired by the upward shooting camera, performing abnormal alarm on whether the upward shooting camera is shielded or not and the identity and the behavior of the user.
Further, carrying out abnormity alarm on the experimental equipment specifically comprises:
analyzing the serial numbers of the experimental devices collected by the overhead camera, and triggering an alarm when the experimental devices with the specified serial numbers are not collected within the set time in the operation process; when the experiment equipment with the designated number is replaced in the operation process, an alarm is triggered; when a plurality of operating equipment appear in the operating process, an alarm is triggered.
Further, the alarming the abnormality of the user hand specifically comprises:
analyzing the number of the user hands acquired by the overhead camera, and triggering video analysis of the overhead camera when the user hands are not acquired in an operation area within a set time in the operation process; when the number of abnormal hands appears in the operation process, triggering the video analysis of the upward shooting camera; the abnormal hand number is a case where the hand number is greater than 2.
Further, performing abnormal alarm on the user identity and the behavior specifically includes:
after the hands of the user are abnormally alarmed, analyzing user information acquired by the upward shooting camera, and triggering alarm when the user leaves the experiment table or the user is replaced;
in the whole experiment process, the user images captured by the upward shooting camera at set time intervals are analyzed, and when user replacement occurs, an alarm is triggered.
In general, the above technical solutions contemplated by the present invention can achieve the following advantageous effects compared to the prior art.
The invention can automatically generate the result of experimental investigation by arranging a plurality of cameras, collecting the images of multiple scenes such as the operation area, the experimental data displayed by experimental instruments, the behavior and the action of operators and the like in the experimental process, and simultaneously detect and alarm abnormal behaviors, can solve the problem of difficult process monitoring in the practice link in the current experimental teaching mode, is generally applicable to modern experimental investigation scenes (such as experimental teaching scenes of information technology class, physical class and the like), and has higher value for the intellectualization and informatization of the current teaching mode.
Drawings
FIG. 1 is a working schematic diagram of an intelligent experimental investigation system based on multi-scene video analysis according to an embodiment of the present invention;
fig. 2 is a structural diagram of an intelligent experimental investigation system based on multi-scene video analysis according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In addition, the technical features involved in the embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.
The embodiment of the invention provides an intelligent experiment investigation system based on multi-scene video analysis, which comprises: the system comprises an interactive terminal, a multi-angle camera and an experimental host; the working principle is as shown in fig. 1, and the interactive terminal is used for performing man-machine interaction with a user, and realizing identity verification, experiment authorization and experiment data submission; the multi-angle camera is used for collecting user operation behaviors and experimental data displayed by an experimental instrument in a multi-angle mode; and the experiment host is used for analyzing, processing and result feedback of the images acquired by the multi-angle camera and alarming abnormal behaviors. The multi-angle camera comprises a downward shooting camera, an upward shooting camera and a fixed camera; as shown in fig. 2, the overhead camera is fixed right above the operation area of the experiment table, and is used for acquiring images of the experiment process of the operation area, and the specifically acquired image data includes: the number of the experimental equipment and the number of the hands of the user; the overhead camera is fixed on a desktop beside an operation area of the experiment table and is used for acquiring images of behaviors of the user; and the fixed camera is fixed at the edge of the experiment table and used for acquiring the image of the experimental data displayed by the experimental instrument.
The users of the present invention include, but are not limited to, students, teachers, or other laboratory teaching managers. The embodiment of the invention takes students as an example, and the system experiment process and the result recording process are as follows: in the student experiment process, the overhead camera can shoot the experiment process to perform video backup; the fixed camera can shoot data of the display area of the experimental instrument at regular time and reserve the data as a screenshot, and the final experimental data is recorded when the experiment is completed. According to various data, a digital abstract and an experimental report can be automatically formed in a local experimental host. After the student experiment is finished, the application for completing the experiment is submitted at the interactive terminal. After the system receives the application, the video camera records the current student operation condition, and the fixed camera polls and records various data displayed by the experimental instrument. And (5) after receiving the application and the submitted data, the teacher verifies, and the experiment is ended. In the process, the system can monitor the behavior and the experimental data of the students in real time, and mainly detects whether the students independently and independently complete the experimental operation, whether abnormal behaviors exist in the operation process and whether the experimental data are real.
Further, the interactive terminal performs man-machine interaction with the user to realize identity verification and experiment authorization, and specifically comprises:
numbering the experimental equipment, the experimental table and the user respectively, and matching the experimental equipment, the experimental table and the user in a one-to-one manner; specifically, before the experiment starts, a teacher can perform one-to-one matching on the student number, the experiment table number, the experiment equipment number and the like, and inform the student of the matching result; wherein, the related experimental equipment is reasonably arranged on an experimental bench and is marked with numbers;
a user applies for entering an experiment through an interactive terminal on the experiment table with the matched serial number; specifically, the interactive terminal can be operated by the student through an input and output device, such as a keyboard, a mouse, or a touch screen.
The camera head is used for collecting face information of the user and judging whether the face information is consistent with the face information matched with the background; if so, successfully verifying the identity and starting an experiment; if not, the identity authentication fails, and the face information acquisition is carried out again. Specifically, a teacher or a teaching assistant has entered relevant student basic information and photo information into a background system before a class is started.
Further, the experiment host computer carries out analysis, processing and result feedback to the image that multi-angle camera gathered to report to the police unusual action, specifically include:
according to image information acquired by the downward shooting camera, whether the downward shooting camera is shielded or not, and abnormal alarms are given to experimental equipment and the hand of a user; and according to the image information acquired by the upward shooting camera, performing abnormal alarm on whether the upward shooting camera is shielded or not and the identity and the behavior of the user.
Wherein, report to the police unusually to experimental device, specifically include: judging whether experimental equipment with a specified number is acquired in the operation process according to the experimental equipment number acquired by the overhead camera, and triggering an alarm if the experimental equipment with the specified number is not acquired within a set time; judging whether the experiment equipment with the specified number is replaced in the operation process; if yes, triggering an alarm; judging whether a plurality of operating equipment appear in the operating process; if yes, an alarm is triggered.
Carry out unusual warning to user's hand, specifically include: judging whether the user hand appears in an operation area in the operation process according to the number of the user hand collected by the overhead camera, and triggering video analysis of the overhead camera if the user hand is not collected in the set time; judging whether the number of abnormal hands appears in the operation process, if so, triggering video analysis of an upward shooting camera; the abnormal hand number is a case where the hand number is greater than 2.
Carrying out abnormal alarm on the user identity and the behavior, specifically comprising the following steps: after the hands of the user are abnormally alarmed, judging whether the user leaves the experiment table or whether user replacement occurs according to user information collected by the upward shooting camera; if yes, triggering an alarm; in the whole experiment process, whether user replacement occurs or not is judged according to the user images captured by the upward camera at set time intervals, and if yes, an alarm is triggered. According to the embodiment of the invention, the upward shooting cameras can shoot at intervals of 10-15min randomly, and whether students are replaced in the experiment process is checked.
In order to explain the above system in detail, this embodiment provides an experimental investigation system for electronic circuit experiments. The equipment required for electronic circuit experiments comprises: a bread board for students to operate, a power supply generator for supplying power and a waveform display. The student operates and sets up the circuit on the bread board, the center of gravity of the whole experiment is the center of gravity, and the waveform display is used for displaying the waveform of the signal generated by the set up circuit. Corresponding serial numbers are engraved on the bread board, a downward shooting camera is arranged right above the experiment table, and an upward shooting camera is arranged on the experiment table.
The specific use steps are as follows:
s1: the teacher matches the students with the experiment table and the bread board and informs the students;
s2: the students take the corresponding bread boards and find own experiment tables, and start experiments through application of the interactive terminals;
s3: the student clicks the student end to start verification authorization operation, looks up the camera to snapshot the face and matches the face information, if normal, enters the experimental environment, and if failure, takes a snapshot again;
s4: the prone shooting camera and the upward shooting camera monitor the operation of students in real time, and if abnormity is found, abnormal events are recorded and a teacher is informed. The alarm content is as follows: 1) when the specified bread board is not detected, displaying: the designated bread board is not placed in the operation area. 2) When the appointed equipment discovers change or has a plurality of bread boards, displaying: bread boards belonging to your own are not used. 3) When the hand is not detected and the face cannot be detected by upward shooting, displaying: leave the laboratory bench for a long time. 4) When the hands are not detected and the student is confirmed to not leave the experiment table by upward shooting, the following are displayed: no operation was performed in the bench top operating area. 5) When the number of hands is abnormal and face shooting is performed to confirm that face information of other classmates appears, the display is as follows: and (4) operation by multiple persons. 6) Face information of other classmates appears during random overhead shooting, and the face information is displayed: and (5) carrying out operation by a person instead. 7) When the beat is sheltered from, show: shelter from the camera of bowing. 8) When the upward shooting is blocked, displaying: the camera head is shot without shielding.
S5: when the current experiment stage is completed, the students confirm the experiment data through the interactive terminal;
s6: after the confirmation is finished, the student initiates an application for finishing the experiment at the interactive terminal;
s7: recording the current student operation condition by a top shooting camera, polling and recording various data displayed by an experimental instrument by a fixed camera;
s8: and (4) after receiving the application and the submitted data, the teacher verifies, if the verification is successful, enters the next stage of the experiment or selects the end of the experiment.
For the above embodiment, the investigation system provided by the present invention effectively avoids the following situations, as shown in table 1:
TABLE 1
It will be understood by those skilled in the art that the foregoing is only a preferred embodiment of the present invention, and is not intended to limit the invention, and that any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the scope of the present invention.