CN115205727A - Experiment intelligent scoring method and system based on unsupervised learning - Google Patents
Experiment intelligent scoring method and system based on unsupervised learning Download PDFInfo
- Publication number
- CN115205727A CN115205727A CN202210608779.XA CN202210608779A CN115205727A CN 115205727 A CN115205727 A CN 115205727A CN 202210608779 A CN202210608779 A CN 202210608779A CN 115205727 A CN115205727 A CN 115205727A
- Authority
- CN
- China
- Prior art keywords
- data
- experiment
- experimental
- target detection
- pictures
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/36—Creation of semantic tools, e.g. ontology or thesauri
- G06F16/367—Ontology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/088—Non-supervised learning, e.g. competitive learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Software Systems (AREA)
- Databases & Information Systems (AREA)
- Computing Systems (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- General Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Medical Informatics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Mathematical Physics (AREA)
- Animal Behavior & Ethology (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses an experiment intelligent scoring method based on unsupervised learning, which comprises the following steps of firstly, collecting experiment operation pictures and video data of experiment operation of students on an experiment operation table, wherein the experiment operation pictures comprise experiment operation pictures with marked data and experiment operation pictures without marked data; then, performing feature extraction and image segmentation on the video data to obtain an experimental equipment picture containing three-dimensional information; secondly, inputting the experiment operation picture and the experiment equipment picture into a visual convolutional neural network which is constructed in advance for training based on a contrast learning method to obtain an optimal target detection model; and finally, inputting the picture to be scored into the optimal target detection model for target detection, and obtaining the experimental score of the picture to be scored based on the target detection result. According to the scheme, a manual marking link is removed through an unsupervised learning method, new knowledge points or score points can be automatically marked, and the adaptability of the model is improved.
Description
Technical Field
The invention relates to the technical field of artificial intelligence, in particular to an experiment intelligent scoring method and system based on unsupervised learning, computing equipment and a storage medium.
Background
At present, in the experiment teaching or examination of physics, chemistry, biology and the like of middle school, the targeted and comprehensive teaching guidance and supervision of each student can not be carried out, so that the efficiency of the experiment teaching or examination is low; the traditional mode of supervising and scoring a plurality of students on the spot by teachers is adopted for scoring in the experimental examinations, so that the scoring efficiency is low, and unobtrusive scoring factors exist. As an important driving force of a new technological revolution and an industrial revolution, artificial intelligence is deeply changing the way people live, work and learn education. In recent years, artificial intelligence technology is increasingly applied to each link of teaching management.
In order to realize the intellectualization of the evaluation of the experimental examination, the experimental operation data needs to be intelligently assigned. The existing intelligent assigning algorithm is based on supervised learning, not only needs a large amount of data for support, but also needs a large amount of manpower and financial resources in the data labeling and processing links; and the relevant evaluation points, operation points and demand points which are not covered by data and not subjected to supervised learning cannot be judged and applied.
Therefore, it is required to provide an intelligent scoring system and method based on unsupervised learning, which can be applied to middle school experiment teaching and examination scenes to solve the problems of high manual data annotation cost, incomplete scoring result, low scoring efficiency and inconsistent scoring standards.
Disclosure of Invention
In view of the above, the present invention proposes an experimental intelligent scoring method, system, computing device and storage medium based on unsupervised learning that overcomes or at least partially solves the above mentioned problems.
According to one aspect of the invention, an experiment intelligent scoring method based on unsupervised learning is provided, in the method, firstly, experiment operation pictures and video data of experiment operation of students on an experiment operation platform are collected, wherein the experiment operation pictures comprise experiment operation pictures with marked data and experiment operation pictures without marked data; then, extracting characteristic points and segmenting images of the video data to obtain an experimental equipment picture containing three-dimensional information; secondly, inputting experiment operation pictures and experiment equipment pictures into a pre-constructed visual convolutional neural network for training based on a contrast learning method to obtain an optimal target detection model; and finally, inputting the picture to be scored into the optimal target detection model for target detection, and obtaining the experimental score of the picture to be scored based on the target detection result.
The method can fully utilize the characteristics of the education closed loop, can remove the tedious, time-consuming and labor-consuming manual marking link through the unsupervised learning method, and can automatically mark and identify new knowledge points or operation points and new scoring points, thereby accelerating the application of the intelligent scoring algorithm and improving the adaptability and the individuation of the model.
Optionally, in the method according to the present invention, the annotation data may include experimental operation point or knowledge point annotation data, experimental equipment label data, experimental operation evaluation data, experimental result scoring data, and the like.
Optionally, in the method according to the present invention, the collected experiment operation pictures may be further screened and processed, so as to remove abnormal data, correct the label of the experiment operation picture with labeled data, and generate a corresponding label for the experiment operation picture without labeled data.
Therefore, the reliability of subsequent model training data can be further improved, and the accuracy of model detection is improved.
Optionally, in the method according to the present invention, the collected experimental operation pictures may be subjected to abnormal data elimination based on a clustering algorithm; carrying out data expansion on the collected experimental operation pictures, wherein the data expansion method comprises mirroring, random cutting, turning, color conversion and the like; and generating corresponding labels for the collected experimental operation pictures without the labeled data or correcting the labels in the experimental operation pictures with the labeled data based on the knowledge graph network.
Optionally, in the method according to the present invention, the step of model training may comprise: carrying out different data enhancement processing on the experiment operation picture and the experiment equipment picture to obtain a first enhanced image and a second enhanced image, wherein the image enhancement processing comprises overturning, rotating, randomly cutting, random color distortion and random Gaussian blur processing; performing feature extraction on the first enhanced image and the second enhanced image to obtain a first feature vector and a second feature vector; inputting the first feature vector and the second feature vector into a visual convolutional neural network for training, and calculating a contrast loss function based on the Euclidean distance between the first feature vector and the second feature vector, wherein the visual convolutional neural network is a weak supervision target detection network based on CAM; and when the loss value of the contrast loss function is smaller than a preset threshold value, obtaining an optimal target detection model.
The self-supervision contrast learning method can directly use data as supervision information, trains and optimizes the model by maximizing the similarity of various images after transformation and minimizing the consistency among different image transformations, and can improve the performance of the model in subsequent target detection tasks.
Optionally, in the method according to the present invention, after obtaining the optimized target detection model, the to-be-scored picture may be input into the optimal target detection model for target detection, so as to obtain the position information of the student action feature points and the experimental device feature points; and then, comparing the obtained student action characteristic points and the positions of the experimental equipment characteristic points with the positions of the student action characteristic points and the experimental equipment characteristic points in the standard operation step, and obtaining the score, the error analysis and the correct guidance of the student experimental operation according to the comparison result. The intelligent scoring method can obviously reduce the labor cost.
According to another aspect of the invention, an experiment intelligent scoring system based on unsupervised learning is provided, and the system comprises a plurality of student terminals, a plurality of teacher terminals, an intelligent scoring module and a database, wherein the intelligent scoring module comprises a data collection unit, a data processing unit, a model training unit, a model updating and deploying unit and an intelligent scoring unit.
The student terminal comprises a camera, an experiment operating platform and a client, the client is suitable for acquiring identity information of students and input experiment report data, and the camera is suitable for collecting video data of experiment operations of the students on the experiment operating platform; the teacher terminal is suitable for receiving teaching resource data, homework or examination data and reading data uploaded by a teacher, and is also suitable for performing information interaction with the student terminals; the database is suitable for storing the data uploaded by the student terminal and the teacher terminal; the data collection unit is suitable for collecting experiment operation pictures and video data of experiment operations of students on the experiment operation table from the database, wherein the experiment operation pictures comprise the experiment operation pictures with marked data and the experiment operation pictures without marked data; the data processing unit is suitable for extracting characteristic points and segmenting images of the video data to obtain an experimental equipment picture containing three-dimensional information, and screening and processing the collected experimental operation picture; the model training unit is suitable for inputting experiment operation pictures and experiment equipment pictures into a pre-constructed visual convolutional neural network for training on the basis of a contrast learning method to obtain an optimal target detection model; the model updating and deploying unit is suitable for updating and deploying the optimal target detection model obtained through training in a hot switching mode; the intelligent scoring unit is suitable for inputting the picture to be scored into the optimal target detection model for target detection, and obtaining the experimental score of the picture to be scored based on the target detection result.
Optionally, in the system, a reading module, a job module, a knowledge graph module, a statistical analysis module and a personalized recommendation module may be further included. The reading and amending module is suitable for providing a reading and amending interface for the teacher terminal or the intelligent scoring module; the operation module is suitable for providing an operation exercise interface for the student terminal; the knowledge map module is suitable for classifying and associating experimental knowledge points or operation points according to the teaching outline map and the label resource data uploaded by the teacher terminal; the statistical analysis module is suitable for performing statistical analysis on the experimental knowledge points, the difficulty, the association degree and the mastery degree of the knowledge points by students according to the preset dimensions and storing the obtained statistical analysis data in a database; the personalized recommendation module is suitable for providing targeted experiment guidance for the student terminal according to the statistical analysis data obtained by the statistical analysis module and the grading data obtained by the intelligent grading module.
The system can meet the requirements of daily homework and examination experiment operation evaluation under a teaching scene and intelligent assignment under an examination scene, and can also realize timely feedback of the learning achievements of students, statistics of global student knowledge mastering conditions and targeted analysis of individual student learning conditions while lightening the correction burden of teachers, so that personalized teaching is further realized.
According to yet another aspect of the invention, there is provided a computing device comprising: at least one processor; and a memory storing program instructions, wherein the program instructions are configured to be executed by the at least one processor, the program instructions comprising instructions for performing the above-described method.
According to yet another aspect of the present invention, there is provided a readable storage medium storing program instructions that, when read and executed by a computing device, cause the computing device to perform the above-described method.
According to the scheme of the invention, through the model training and application scheme based on unsupervised learning, the tedious, time-consuming and labor-consuming manual marking link can be eliminated, and meanwhile, the automatic marking and identification can be performed on new knowledge points or experimental operation points in time, so that the adaptability and individuation of the model are improved. The intelligent scoring system provided by the scheme can realize the functions of intelligent correction, knowledge map updating, personalized recommendation, teaching result statistics, student learning condition analysis and the like in daily teaching practice or examination scenes by means of an unsupervised learning method and a knowledge map network, and further achieves personalized teaching.
The foregoing description is only an overview of the technical solutions of the present invention, and the embodiments of the present invention are described below in order to make the technical means of the present invention more clearly understood and to make the above and other objects, features, and advantages of the present invention more clearly understandable.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:
FIG. 1 shows a schematic structural diagram of an experiment intelligent scoring system 100 based on unsupervised learning according to an embodiment of the invention;
FIG. 2 illustrates a block diagram of a computing device 200, according to one embodiment of the invention;
fig. 3 shows a flowchart of an experimental intelligence scoring method 300 based on unsupervised learning according to an embodiment of the invention.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
Under the teaching closed-loop scene, namely, a teacher and students form a complete closed-loop system through teaching, learning, practicing, testing and evaluating, the intellectualization and pertinence of grading in experiment teaching, practicing or examination are realized, and the teaching efficiency of the experiment teaching can be enhanced. The existing model is used for predicting the label-free data, the prediction result is used for assisting the model training, and the self-enhanced learning mode can obtain a good prediction effect on the closed-loop data. Aiming at the defects of the existing intelligent scoring system based on the supervised learning algorithm, the experimental intelligent scoring method and system based on unsupervised learning are provided, the situation that manually marked data consumes a large amount of manpower, the judgment of unmarked related evaluation points and operation points cannot be made, and the generalization capability of a model can be limited by a large amount of marked data is considered, so that the experimental operation data can be accurately predicted and evaluated under the condition of lacking of marked data, useful expression can be learned from the experimental operation pictures of the unmarked data, and the target identification capability, the generalization capability and the robustness of the model can be improved. And the autonomous updating and deployment of the optimal model can be realized under the teaching closed-loop scene.
Fig. 1 shows a schematic structural diagram of an experimental intelligent scoring system 100 based on unsupervised learning according to an embodiment of the invention. As shown in fig. 1, the system comprises 1-n student terminals, 1-m teacher terminals, a database 110 and an intelligent scoring module 120, wherein the student terminals comprise clients providing data input interfaces, experiment operation tables and cameras arranged at a plurality of angles or orientations. The student can input identity information such as name, school number, class and the like through the client and input experimental report data. The camera can set up the place ahead and the left and right sides at student's experiment operation panel, and the accessible is adjusted the angle of making a video recording and is gathered the video data that the student carries out the experiment operation at the experiment operation panel all-roundly. A plurality of experimental devices such as test tubes, wires, experimental solutions and the like are placed on the experiment operating platform. The teacher terminal can receive teaching resource data uploaded by a teacher, formulated teaching outline or teaching plan, homework or examination tasks, homework or examination paper reading and amending data and the like, can perform information interaction with the student terminals, for example, the teacher terminal issues experiment examinations or examination tasks, sends out experiment starting or ending instructions and the like, and the teacher terminal can also read and amend experiment report data uploaded by the student terminals. The database 110 may store data uploaded by student terminals and teacher terminals.
In one embodiment of the present invention, the intelligent scoring module 120 may include a data collection unit 121, a data processing unit 122, a model training unit 123, a model update deployment unit 124, and an intelligent scoring unit 125. The data collection unit 121 may collect, from the database 110, experiment operation pictures and video data of experiment operations performed by students on an experiment console, where the experiment operation pictures include experiment operation pictures with labeled data and experiment operation pictures without labeled data. The labeling data can include experiment operation point or knowledge point labeling data, experiment equipment label data, experiment operation evaluation data, experiment result grading data and the like. The data processing unit 122 may screen and process the experiment operation pictures collected by the data collecting unit 121, and perform feature point extraction and image segmentation on the video data collected by the camera to obtain an experiment equipment picture containing three-dimensional information. For example, the collected experimental operation pictures can be subjected to abnormal data elimination processing based on a clustering algorithm; carrying out data expansion on the collected experimental operation pictures, wherein the data expansion method comprises mirroring, random cutting, turning, color conversion and the like; and generating corresponding labels for the collected experimental operation pictures without the labeled data or correcting the labels in the experimental operation pictures with the labeled data based on the knowledge graph network. Performing image segmentation on an image in the video data, and adding the segmented experimental equipment picture into a training image; and extracting three-dimensional characteristic points and segmenting images of the video data to obtain an experimental equipment picture containing three-dimensional information of the experimental equipment. The data processing methods can improve the efficiency of subsequent model training.
The model training unit 123 may input the experimental operation picture processed by the data processing unit 122 and the extracted three-dimensional coordinate information into a pre-constructed visual convolutional neural network for training based on a contrast learning method, so as to obtain an optimal target detection model. The existing unsupervised learning method can be divided into a generation formula and a discriminant formula, wherein the generation method usually depends on self-coding or antagonistic learning of the image and directly operates in a pixel space, and pixel-level details required by image generation may not be necessary for learning high-level representation. Unlike the generation method, contrast learning avoids the time-consuming generation step by zooming in on the representations of different views (i.e., positive and negative pairs) of the same image and separating the representations of the views of different images (i.e., negative pairs). By adopting the contrast learning method, the aim of maximizing the consistency among different transformed views of the same image and minimizing the consistency among the transformed views of different images is realized to learn the universal representation. In an embodiment of the present invention, a weakly supervised object detection network based on CAM (class thermodynamic diagram) may be used for unsupervised learning training, and the step of model training may include: carrying out different data enhancement processing on the experiment operation picture and the experiment equipment picture to obtain a first enhanced image and a second enhanced image, wherein the image enhancement processing comprises overturning, rotating, random cutting, random color distortion, random Gaussian blur processing and the like; then, extracting features of the first enhanced image and the second enhanced image to obtain a first feature vector and a second feature vector; and then inputting the first feature vector and the second feature vector into a visual convolutional neural network for training, and calculating a contrast loss function based on Euclidean distance between the first feature vector and the second feature vector, wherein the visual convolutional neural network is a weak supervision target detection network based on CAM (class thermodynamic diagram). And when the loss value of the contrast loss function is smaller than a preset threshold value, obtaining an optimal target detection model. The method for detecting the targets does not need to label a bounding box on each target in the picture, training can be started only by giving one label to the whole picture, the global pooling layer is used for identifying the hot spot area in the picture, the hot spot area is multiplied by the weight and then superposed to obtain the thermodynamic diagram of the picture for the label, and the visual information can be used for guiding the network to learn better. Meanwhile, the constraint of three-dimensional information is added in the model training, so that constraint judgment can be simultaneously carried out from the global characteristic and the local characteristic, and the accuracy of model identification is further improved.
The model updating and deploying unit 124 may update and deploy the optimal target detection model obtained by training of the model training unit 123 in a hot-swap manner, so as to achieve iterative updating and deployment of the model driven by data. The intelligent scoring unit 125 may input the picture to be scored into the optimal target detection model for target detection, and obtain an experimental score of the picture to be scored based on a target detection result, thereby improving the scoring efficiency and the consistency of the scoring standard in an experimental examination or exercise.
In an embodiment of the present invention, the experiment intelligent scoring system 100 further includes an approval module 130, a job module 140, a knowledge graph module 150, a statistical analysis module 160, and a personalized recommendation module 170. The reading and amending module 130 can be suitable for providing a reading and amending interface for the teacher terminal or the intelligent scoring module 120, so that the manual reading and amending and intelligent reading functions are realized; the assignment module 140 may provide assignment exercise interfaces to student terminals, and may provide autonomous exercise and personalized exercise interfaces to students. The knowledge map module 150 may classify and associate the experimental knowledge points or operation points according to the pre-constructed teaching outline map and the label resource data uploaded by the teacher terminal. The knowledge graph network is a data structure consisting of entities, relations and attributes, if the knowledge graph network is an existing knowledge point, classification is automatically carried out, if the knowledge graph network is a new knowledge point, new branches are added in the knowledge graph, continuous updating and optimization of the knowledge graph network are achieved, and data support can be provided for a follow-up personalized recommendation module and a statistical analysis module. The statistical analysis module 160 may perform statistical analysis on the experimental knowledge points, difficulty, association degree, and mastery degree of the knowledge points by the students according to the preset dimensions, and store the obtained statistical analysis data in the database 110. For example, the statistical analysis module can perform statistical analysis on related knowledge points, experiment difficulty degrees, knowledge point association degrees and knowledge point mastering conditions of students from different dimensions of school districts, classes, teachers, students and the like, and can provide data support for a subsequent personalized recommendation module. The personalized recommendation module 170 may provide a targeted experimental guidance to the student terminal according to the statistical analysis data obtained by the statistical analysis module 160 and the scoring data obtained by the intelligent scoring module 120.
FIG. 2 shows a block diagram of a computing device 200, according to one embodiment of the invention. As shown in FIG. 2, in a basic configuration 202, a computing device 200 typically includes a system memory 206 and one or more processors 204. A memory bus 208 may be used for communication between the processor 204 and the system memory 206.
Depending on the desired configuration, the processor 204 may be any type of processing, including but not limited to: a microprocessor (μ P), a microcontroller (μ C), a digital information processor (DSP), or any combination thereof. The processor 204 may include one or more levels of cache, such as a level one cache 210 and a level two cache 212, a processor core 214, and registers 216. Example processor core 214 may include an Arithmetic Logic Unit (ALU), a Floating Point Unit (FPU), a digital signal processing core (DSP core), or any combination thereof. The example memory controller 218 may be used with the processor 204, or in some implementations the memory controller 218 may be an internal part of the processor 204.
Depending on the desired configuration, system memory 206 may be any type of memory, including but not limited to: volatile memory (such as RAM), non-volatile memory (such as ROM, flash memory, etc.), or any combination thereof. The physical memory in the computing device is usually referred to as a volatile memory RAM, and data in the disk needs to be loaded into the physical memory to be read by the processor 204. System memory 206 may include an operating system 220, one or more applications 222, and program data 224. The application 222 is actually a plurality of program instructions that direct the processor 204 to perform corresponding operations. In some embodiments, the application 222 may be arranged to execute instructions on an operating system with the program data 224 by one or more processors 204 in some embodiments. Operating system 220 may be, for example, linux, windows, etc., which includes program instructions for handling basic system services and performing hardware dependent tasks. The application 222 includes program instructions for implementing various user-desired functions, and the application 222 may be, for example, but not limited to, a browser, instant messenger, a software development tool (e.g., an integrated development environment IDE, compiler, etc.), and the like. When the application 222 is installed into the computing device 200, a driver module may be added to the operating system 220.
When the computing device 200 is started, the processor 204 reads program instructions of the operating system 220 from the memory 206 and executes them. Applications 222 run on top of operating system 220, utilizing the interface provided by operating system 220 and the underlying hardware to implement various user-desired functions. When the user starts the application 222, the application 222 is loaded into the memory 206, and the processor 204 reads the program instructions of the application 222 from the memory 206 and executes the program instructions.
Computing device 200 also includes storage device 232, storage device 232 including removable storage 236 and non-removable storage 238, each of removable storage 236 and non-removable storage 238 being connected to storage interface bus 234.
Computing device 200 may also include an interface bus 240 that facilitates communication from various interface devices (e.g., output devices 242, peripheral interfaces 244, and communication devices 246) to the basic configuration 202 via the bus/interface controller 230. The example output device 242 includes a graphics processing unit 248 and an audio processing unit 250. They may be configured to facilitate communication with various external devices, such as a display or speakers, via one or more a/V ports 252. Example peripheral interfaces 244 can include a serial interface controller 254 and a parallel interface controller 256, which can be configured to facilitate communications with external devices such as input devices (e.g., keyboard, mouse, pen, voice input device, touch input device) or other peripherals (e.g., printer, scanner, etc.) via one or more I/O ports 258. An example communication device 246 may include a network controller 260, which may be arranged to facilitate communications with one or more other computing devices 262 over a network communication link via one or more communication ports 264.
A network communication link may be one example of a communication medium. Communication media may typically be embodied by computer readable instructions, data structures, program modules, in a modulated data signal, such as a carrier wave or other transport mechanism, and may include any information delivery media. A "modulated data signal" may be a signal that has one or more of its data set or its changes made in such a manner as to encode information in the signal. By way of non-limiting example, communication media may include wired media such as a wired network or private-wired network, and various wireless media such as acoustic, radio Frequency (RF), microwave, infrared (IR), or other wireless media. The term computer readable media as used herein may include both storage media and communication media.
Computing device 200 also includes a storage interface bus 234 coupled to bus/interface controller 230. The storage interface bus 234 is coupled to the storage device 232, and the storage device 232 is adapted to store data. The example storage devices 232 may include removable storage 236 (e.g., CD, DVD, usb disk, removable hard disk, etc.) and non-removable storage 238 (e.g., hard disk drive HDD, etc.). In the computing device 200 according to the invention, the application 222 comprises a plurality of program instructions that perform the method 300.
Fig. 3 shows a flowchart of an experimental intelligence scoring method 300 based on unsupervised learning according to an embodiment of the invention. As shown in fig. 3, the method 300 starts with step S310, and collects experimental operation pictures and video data of experimental operations performed by students on an experimental operation desk, where the experimental operation pictures include experimental operation pictures with labeled data and experimental operation pictures without labeled data. The experiment operation picture can be an experiment operation picture in a teaching material or a test question, and can comprise name labels of experiment equipment and name labels of experiment equipment read by teachers. The labeling data can comprise experiment operation point or knowledge point labeling data, experiment equipment label data, experiment operation evaluation data, experiment result grading data and the like. The video data of the experiment operation of the student on the experiment operation table can be acquired through video acquisition equipment such as a camera arranged at a student terminal. In the embodiment of the invention, the collected experimental operation pictures can be screened and processed to be used as the training images of the subsequent visual convolutional neural network. For example, the collected experimental operation pictures can be subjected to abnormal data elimination processing based on a clustering algorithm; carrying out data expansion on the collected experimental operation pictures, wherein the data expansion method comprises mirroring, random cutting, turning and color conversion; and generating corresponding labels for the collected experimental operation pictures without the labeled data based on the knowledge graph network, or correcting the labels in the experimental operation pictures with labeled data and the like.
Then, step S320 is executed to perform feature point extraction and image segmentation on the video data to obtain an experimental device picture containing three-dimensional information. Because the video data acquired by the cameras with the plurality of azimuth angles comprises the three-dimensional information of the experimental equipment and the student actions, the pictures of the experimental equipment can be extracted according to an image segmentation algorithm, and the feature point extraction is carried out on the student actions and the experimental equipment in the video images to obtain the three-dimensional information or point cloud data of the experimental equipment and the student actions, so that the global supervision and the local supervision can be simultaneously trained based on the three-dimensional information constraint in the subsequent model training process.
And step S330 is executed, and the experiment operation picture and the experiment equipment picture are input into a visual convolutional neural network which is constructed in advance for training on the basis of a contrast learning method, so that an optimal target detection model is obtained. In the embodiment of the invention, an automatic supervision target detection network without a labeling box is used as a backbone network of an intelligent scoring algorithm. For example, a CAM-based visual convolutional neural network can be used as a weak supervision target detection network, firstly, a feature vector is generated on the convolutional layer of the last layer, then the feature vector is input into a classifier with fully connected layers, a prediction score of a picture is generated, finally, an activation map of each class is segmented through a threshold technology, and a candidate box of each class is generated by using the CAM. Specifically, the model training process is as follows: firstly, respectively carrying out different data enhancement processing twice on an experiment operation picture and an experiment equipment picture to obtain a first enhanced image and a second enhanced image, wherein the image enhancement processing comprises overturning, rotating, random cutting, random color distortion and random Gaussian blur processing. Each pair of enhanced images constitutes a set of opposite images. Then, feature extraction is carried out on the first enhanced image and the second enhanced image to obtain a first feature vector and a second feature vector; and then, inputting the first feature vector and the second feature vector into a visual convolutional neural network for training, adding constraint of three-dimensional information into model training, and performing constraint judgment from the global feature and the local feature at the same time to improve the accuracy of model prediction. And finally, calculating a contrast loss function based on the Euclidean distance between the first characteristic vector and the second characteristic vector, and obtaining an optimal target detection model when the loss value of the contrast loss function is smaller than a preset threshold value.
And finally, executing the step S340, inputting the picture to be scored into the optimal target detection model for target detection, and obtaining the experimental score of the picture to be scored based on the target detection result. Specifically, the picture to be scored can be input into the optimal target detection model for target detection, so as to obtain the position information of the student action characteristic points and the experimental device characteristic points; and then, comparing the obtained student action characteristic points and the positions of the experimental equipment characteristic points with the positions of the student action characteristic points and the experimental equipment characteristic points in the standard operation step, and obtaining the score, the error analysis and the correct guidance of the student experimental operation according to the comparison result. For example, when the relative position between the student action feature point and the experimental equipment is basically consistent with the relative position between the student action feature point and the experimental equipment feature point in the standard operation step, the student operation action standard can be judged, the corresponding experiment score is higher, and when the difference between the relative positions is larger, the student experiment operation action is judged to be wrong or not, the student operation action can be scored according to the scoring standard, and corresponding error analysis and correct guidance are given.
Through the scheme, compared with an intelligent scoring scheme for supervised learning, the intelligent scoring method for the supervised learning can remove complicated, time-consuming and labor-consuming manual labeling links, can automatically label and identify new knowledge points or experiment operation points in time, and improves the adaptability and the individuation of the model. The intelligent scoring system provided by the scheme can realize the functions of intelligent correction, knowledge map updating, personalized recommendation, teaching result statistics, student learning condition analysis and the like in daily teaching practice or examination scenes by means of an unsupervised learning method and a knowledge map network, and further achieves personalized teaching.
In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.
Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. However, the disclosed method should not be construed to reflect the intent: rather, the invention as claimed requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this invention.
Those skilled in the art will appreciate that the modules or units or components of the devices in the examples disclosed herein may be arranged in a device as described in this embodiment or alternatively may be located in one or more devices different from the devices in this example. The modules in the foregoing examples may be combined into one module or may additionally be divided into multiple sub-modules.
Those skilled in the art will appreciate that the modules in the device in an embodiment may be adaptively changed and disposed in one or more devices different from the embodiment. The modules or units or components of the embodiments may be combined into one module or unit or component, and furthermore they may be divided into a plurality of sub-modules or sub-units or sub-components. All of the features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or elements of any method or apparatus so disclosed, may be combined in any combination, except combinations where at least some of such features and/or processes or elements are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.
Furthermore, those skilled in the art will appreciate that while some embodiments described herein include some features included in other embodiments, rather than other features, combinations of features of different embodiments are meant to be within the scope of the invention and form different embodiments. For example, in the following claims, any of the claimed embodiments may be used in any combination.
Furthermore, some of the described embodiments are described herein as a method or combination of method elements that can be performed by a processor of a computer system or by other means of performing the described functions. A processor with the necessary instructions for carrying out the method or the method elements thus forms a device for carrying out the method or the method elements. Further, the elements of the apparatus embodiments described herein are examples of the following apparatus: the means for performing the functions performed by the elements for the purpose of carrying out the invention.
As used herein, unless otherwise specified the use of the ordinal adjectives "first", "second", "third", etc., to describe a common object, merely indicate that different instances of like objects are being referred to, and are not intended to imply that the objects so described must be in a given sequence, either temporally, spatially, in ranking, or in any other manner.
While the invention has been described with respect to a limited number of embodiments, those skilled in the art, having benefit of this description, will appreciate that other embodiments can be devised which do not depart from the scope of the invention as described herein. Moreover, it should be noted that the language used in the specification has been principally selected for readability and instructional purposes, and may not have been selected to delineate or circumscribe the inventive subject matter. Accordingly, many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the appended claims. The present invention has been disclosed in an illustrative rather than a restrictive sense with respect to the scope of the invention, as defined in the appended claims.
Claims (10)
1. An experiment intelligent scoring method based on unsupervised learning, which is suitable for being executed in a computing device, and is characterized by comprising the following steps of:
collecting experiment operation pictures and video data of experiment operation of students on an experiment operation table, wherein the experiment operation pictures comprise experiment operation pictures with marked data and experiment operation pictures without marked data;
extracting characteristic points and segmenting images of the video data to obtain an experimental equipment picture containing three-dimensional information;
inputting the experiment operation pictures and the experiment equipment pictures into a visual convolutional neural network which is constructed in advance for training on the basis of a contrast learning method to obtain an optimal target detection model;
and inputting the picture to be scored into the optimal target detection model for target detection, and obtaining the experimental score of the picture to be scored based on the target detection result.
2. The method of claim 1, wherein the annotation data comprises experimental operating point or knowledge point annotation data, experimental device label data, experimental operating evaluation data, and experimental result scoring data.
3. The method of claim 1, further comprising:
and screening and processing the collected experimental operation pictures so as to eliminate abnormal data, correct the labels of the experimental operation pictures with labeled data and generate corresponding labels for the experimental operation pictures without labeled data.
4. The method of claim 3, wherein the step of screening and processing the collected experimental procedure pictures comprises:
based on a clustering algorithm, carrying out abnormal data elimination processing on the collected experimental operation pictures;
carrying out data expansion on the collected experimental operation pictures, wherein the data expansion method comprises mirroring, random cutting, turning and color conversion;
and generating corresponding labels for the collected experimental operation pictures without the labeled data or correcting the labels in the experimental operation pictures with the labeled data based on the knowledge graph network.
5. The method according to claim 1, wherein the step of inputting the experiment operation pictures and the experiment equipment pictures into a pre-constructed visual convolutional neural network for training based on a contrast learning method to obtain an optimal target detection model comprises:
carrying out different data enhancement processing on the experiment operation picture and the experiment equipment picture to obtain a first enhanced image and a second enhanced image, wherein the image enhancement processing comprises overturning, rotating, randomly cutting, randomly distorting colors and randomly carrying out Gaussian blur processing;
performing feature extraction on the first enhanced image and the second enhanced image to obtain a first feature vector and a second feature vector;
inputting the first feature vector and the second feature vector into a visual convolutional neural network for training, and calculating a contrast loss function based on Euclidean distance between the first feature vector and the second feature vector, wherein the visual convolutional neural network is a weak supervision target detection network based on CAM;
and when the loss value of the contrast loss function is smaller than a preset threshold value, obtaining an optimal target detection model.
6. The method according to claim 1, wherein the step of inputting the picture to be scored into the optimal target detection model for target detection to obtain the experimental score of the picture to be scored comprises:
inputting the picture to be evaluated into the optimal target detection model to carry out target detection, and obtaining position information of the student action characteristic points and the experimental equipment characteristic points;
and comparing the obtained student action characteristic points and the experimental equipment characteristic point positions with the student action characteristic points and the experimental equipment characteristic point positions in the standard operation step, and obtaining student experiment operation scores, error analysis and correct guidance according to the comparison result.
7. An experiment intelligent scoring system based on unsupervised learning is characterized by comprising a plurality of student terminals, a plurality of teacher terminals, an intelligent scoring module and a database, wherein the intelligent scoring module comprises a data collection unit, a data processing unit, a model training unit, a model updating and deploying unit and an intelligent scoring unit,
the student terminal comprises a camera, an experiment operating platform and a client, wherein the client is suitable for acquiring identity information of a student and input experiment report data, and the camera is suitable for collecting video data of experiment operation of the student on the experiment operating platform;
the teacher terminal is suitable for receiving teaching resource data, homework or examination data and reading data uploaded by a teacher, and is also suitable for performing information interaction with the student terminals;
the database is suitable for storing the data uploaded by the student terminal and the teacher terminal;
the data collection unit is suitable for collecting experiment operation pictures and video data of experiment operations of students on an experiment operation table from a database, wherein the experiment operation pictures comprise experiment operation pictures with marked data and experiment operation pictures without marked data;
the data processing unit is suitable for extracting characteristic points and segmenting images of the video data to obtain an experimental equipment picture containing three-dimensional information, and screening and processing the collected experimental operation pictures;
the model training unit is suitable for inputting the experiment operation pictures and the experiment equipment pictures into a visual convolutional neural network which is constructed in advance for training on the basis of a contrast learning method to obtain an optimal target detection model;
the model updating and deploying unit is suitable for updating and deploying the optimal target detection model obtained through training in a hot switching mode;
the intelligent scoring unit is suitable for inputting the picture to be scored into the optimal target detection model for target detection, and obtaining the experimental score of the picture to be scored based on the target detection result.
8. The intelligent scoring system according to claim 7, wherein the system further comprises an endorsement module, a job module, a knowledge-graph module, a statistical analysis module, and a personalized recommendation module,
the reading and amending module is suitable for providing a reading and amending interface for the teacher terminal or the intelligent scoring module;
the homework module is suitable for providing a homework exercise interface for the student terminal;
the knowledge map module is suitable for classifying and associating experimental knowledge points or operation points according to the teaching outline map and the label resource data uploaded by the teacher terminal;
the statistical analysis module is suitable for performing statistical analysis on the experimental knowledge points, the difficulty, the association degree and the mastery degree of the knowledge points by students according to preset dimensions and storing the obtained statistical analysis data in a database;
the personalized recommendation module is suitable for providing targeted experiment guidance for the student terminal according to the statistical analysis data obtained by the statistical analysis module and the grading data obtained by the intelligent grading module.
9. A computing device, comprising:
at least one processor; and a memory storing program instructions, wherein the program instructions are configured to be executed by the at least one processor, the program instructions comprising instructions for performing the method of any of claims 1-6.
10. A readable storage medium storing program instructions that, when read and executed by a computing device, cause the computing device to perform the method of any of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210608779.XA CN115205727A (en) | 2022-05-31 | 2022-05-31 | Experiment intelligent scoring method and system based on unsupervised learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210608779.XA CN115205727A (en) | 2022-05-31 | 2022-05-31 | Experiment intelligent scoring method and system based on unsupervised learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115205727A true CN115205727A (en) | 2022-10-18 |
Family
ID=83575892
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210608779.XA Pending CN115205727A (en) | 2022-05-31 | 2022-05-31 | Experiment intelligent scoring method and system based on unsupervised learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115205727A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115471216A (en) * | 2022-11-03 | 2022-12-13 | 深圳市顺源科技有限公司 | Data management method of intelligent laboratory management platform |
| CN117539367A (en) * | 2023-11-20 | 2024-02-09 | 广东海洋大学 | Image recognition tracking method based on interactive intelligent experiment teaching system |
| CN117789078A (en) * | 2023-12-18 | 2024-03-29 | 广东广视通智慧教育科技有限公司 | Experiment operation evaluation method and system based on AI visual recognition |
-
2022
- 2022-05-31 CN CN202210608779.XA patent/CN115205727A/en active Pending
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115471216A (en) * | 2022-11-03 | 2022-12-13 | 深圳市顺源科技有限公司 | Data management method of intelligent laboratory management platform |
| CN117539367A (en) * | 2023-11-20 | 2024-02-09 | 广东海洋大学 | Image recognition tracking method based on interactive intelligent experiment teaching system |
| CN117539367B (en) * | 2023-11-20 | 2024-04-12 | 广东海洋大学 | Image recognition tracking method based on interactive intelligent experiment teaching system |
| CN117789078A (en) * | 2023-12-18 | 2024-03-29 | 广东广视通智慧教育科技有限公司 | Experiment operation evaluation method and system based on AI visual recognition |
| CN117789078B (en) * | 2023-12-18 | 2024-05-31 | 广东广视通智慧教育科技有限公司 | Experiment operation evaluation method and system based on AI visual recognition |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Tabernik et al. | Deep learning for large-scale traffic-sign detection and recognition | |
| CN115205727A (en) | Experiment intelligent scoring method and system based on unsupervised learning | |
| US20210020059A1 (en) | Answer evaluation method, answer evaluation system, electronic device, and medium | |
| CN107240047B (en) | Score evaluation method and device for teaching video | |
| Xu et al. | Learning-based shadow recognition and removal from monochromatic natural images | |
| CN111476284A (en) | Image recognition model training method, image recognition model training device, image recognition method, image recognition device and electronic equipment | |
| CN107609541A (en) | A kind of estimation method of human posture based on deformable convolutional neural networks | |
| CN110110780B (en) | Image classification method based on antagonistic neural network and massive noise data | |
| CN112528862B (en) | Remote sensing image target detection method based on improved cross entropy loss function | |
| CN110781648A (en) | Test paper automatic transcription system and method based on deep learning | |
| CN116091858A (en) | Semi-supervised learning power equipment target detection model training method, detection method and device | |
| CN110689000B (en) | Vehicle license plate recognition method based on license plate sample generated in complex environment | |
| CN116935447B (en) | Self-adaptive teacher-student structure-based unsupervised domain pedestrian re-recognition method and system | |
| CN112257665A (en) | Image content recognition method, image recognition model training method, and medium | |
| CN103824090A (en) | Adaptive face low-level feature selection method and face attribute recognition method | |
| CN112766170B (en) | Self-adaptive segmentation detection method and device based on cluster unmanned aerial vehicle image | |
| WO2021174820A1 (en) | Discovery method and apparatus for difficult sample, and computer device | |
| CN110866564A (en) | Season classification method, system, electronic device and medium for multiple semi-supervised images | |
| CN112001219A (en) | Multi-angle multi-face recognition attendance checking method and system | |
| CN116258937A (en) | Small sample segmentation method, device, terminal and medium based on attention mechanism | |
| CN114882204A (en) | Automatic ship name recognition method | |
| CN113628113B (en) | Image stitching method and related equipment thereof | |
| CN113569940B (en) | Knowledge migration and probability correction-based few-sample target detection method | |
| CN110705535A (en) | Method for automatically detecting test paper layout character line | |
| CN115496960A (en) | Sample generation method, target detection model training method, target detection method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |