CN111768668A

CN111768668A - Experimental operation scoring method, device, equipment and storage medium

Info

Publication number: CN111768668A
Application number: CN202010244830.4A
Authority: CN
Inventors: 邓亦梁
Original assignee: Hangzhou Hikvision Digital Technology Co Ltd
Current assignee: Hangzhou Hikvision Digital Technology Co Ltd
Priority date: 2020-03-31
Filing date: 2020-03-31
Publication date: 2020-10-13
Anticipated expiration: 2040-03-31
Also published as: CN111768668B

Abstract

The embodiment of the invention provides an experimental operation scoring method, an experimental operation scoring device, experimental operation scoring equipment and a storage medium, wherein the method comprises the following steps: and acquiring an operation video of the heating experiment, and then automatically determining the scores of the heating operation score point and the heat-extinguishing operation score point by the electronic equipment based on the operation video. Therefore, the scheme does not need manual observation and experiment operation and scoring, and reduces the consumption of manpower.

Description

Experimental operation scoring method, device, equipment and storage medium

Technical Field

The invention relates to the technical field of experimental operation, in particular to an experimental operation scoring method, device, equipment and storage medium.

Background

The experimental examination can detect the mastery degree of the knowledge of students. For example, a heating experiment is a common experiment, and in an examination of the heating experiment, whether the experiment operation of a student is in compliance needs to be evaluated.

At present, the experimental operation of students is generally observed by a proctor, and the operation of students is scored based on the observation result. However, many examinees in the experimental examination are difficult for the invigilator to observe the experimental operation details of a plurality of examinees simultaneously. In order to improve the scoring accuracy, the number of invigilators in an examination room is increased, which consumes more manpower.

Disclosure of Invention

The embodiment of the invention aims to provide an experimental operation scoring method, device, equipment and storage medium so as to reduce labor consumption.

In order to achieve the above object, an embodiment of the present invention provides an experimental operation scoring method, including:

acquiring an operation video of a heating experiment;

identifying a heat source, a heated instrument, and a heat-extinguished instrument in the operational video;

determining the grade of the heating operation scoring point according to whether the position relation between the heat source and the heated instrument accords with a first target position state corresponding to the heating operation scoring point;

and determining the grade of the heat extinguishing operation scoring point according to whether the position relation between the heat extinguishing instrument and the heat source accords with the second target position state corresponding to the heat extinguishing operation scoring point.

Optionally, the identifying a heat source, a heated instrument and a heat-extinguishing instrument in the operation video includes:

identifying a heat source, a heated instrument and a heat-extinguishing instrument from the operation video to obtain an identification frame of the heat source, an identification frame of the heated instrument, an identification frame of the heat-extinguishing instrument and a key point of the heated instrument; the key points include: a target heating location point on the heated instrument.

Optionally, the determining the score of the heating operation score point according to whether the position relationship between the heat source and the heated instrument conforms to the first target position state corresponding to the heating operation score point includes:

judging whether the key point of the heated instrument is positioned in the identification frame of the heat source;

calculating the distance between the key point of the heated instrument and a designated point in the identification frame of the heat source;

and if the key point is positioned in the identification frame of the heat source and the distance is matched with the target distance corresponding to the heating operation scoring point, determining the score of the heating operation scoring point.

Optionally, if the distance is matched with a target distance corresponding to the heating operation scoring point, determining the score of the heating operation scoring point includes:

for each frame of image in the operation video, if the key point of the heated instrument in the frame of image is located in the identification frame of the heat source and the distance between the key point and the designated point in the identification frame of the heat source is matched with the target distance corresponding to the heating operation scoring point, adding first marking information for the frame of image;

sequentially sliding and taking continuous N through sliding window₁Judging whether the quantity of the first mark information which is slid and taken meets a first preset condition or not according to the first mark information of the frame image, and if so, determining the score of the heating operation scoring point; said N is₁Is a positive integer greater than 1.

Optionally, the identifying a heat source, a heated instrument and a heat-extinguishing instrument from the operation video to obtain an identification frame of the heat source, an identification frame of the heated instrument, an identification frame of the heat-extinguishing instrument and a key point of the heated instrument includes:

identifying a heat source, a heated instrument, a heat-extinguishing instrument and a heat source instrument from the operation video to obtain an identification frame of the heat source, an identification frame of the heated instrument, an identification frame of the heat-extinguishing instrument, an identification frame of the heat source instrument and a key point of the heated instrument; the heat source instrument is a carrier of the heat source;

the determining the grade of the heat extinguishing operation scoring point according to whether the position relation between the heat extinguishing instrument and the heat source accords with the second target position state corresponding to the heat extinguishing operation scoring point comprises the following steps:

and if the identification frame of the heat extinguishing instrument is intersected with the identification frame of the heat source instrument and the identification frame of the heat extinguishing instrument is intersected with the identification frame of the heat source, determining the score of the heat extinguishing operation scoring point.

Optionally, if there is an intersection between the identification frame of the heat-extinguishing apparatus and the identification frame of the heat-source apparatus, and there is an intersection between the identification frame of the heat-extinguishing apparatus and the identification frame of the heat source, determining the score of the heat-extinguishing operation scoring point includes:

for each frame of image in the operation video, if the intersection exists between the identification frame of the heat extinguishing instrument and the identification frame of the heat source instrument in the frame of image and the intersection exists between the identification frame of the heat extinguishing instrument and the identification frame of the heat source, adding third mark information to the frame of image;

sequentially sliding and taking continuous N through sliding window₃Judging whether the number of the third mark information which is slid and taken meets a third preset condition or not according to third mark information of the frame image, and if so, determining the score of the heat-extinguishing operation scoring point; said N is₃Is a positive integer greater than 1.

Optionally, the target heating position points are: a bottom location point of the heated instrument; the key points further include: a mouth location point of the heated instrument; the method further comprises the following steps:

calculating a horizontal included angle and a vertical included angle of the heated instrument based on the mouth position point and the bottom position point; the horizontal included angle is an included angle between the heated instrument and a horizontal plane, and the vertical included angle is an included angle between the heated instrument and a vertical plane corresponding to an operator;

and if the bottom position point is located in the heat source target frame, the horizontal included angle is located in a first preset interval, and the vertical included angle is located in a second preset interval, determining the score of the attitude scoring point.

Optionally, if the bottom position point is located in the heat source target frame, the horizontal included angle is located in a first preset interval, and the vertical included angle is located in a second preset interval, determining a score of the posture scoring point, including:

for each frame of image in the operation video, if the bottom position point of the heated instrument in the frame of image is located in the heat source target frame, the horizontal included angle of the heated instrument is located in a first preset interval, and the vertical included angle of the heated instrument is located in a second preset interval, adding second marking information to the frame of image;

sequentially sliding and taking continuous N through sliding window₂Second marking information of the frame image, judging whether the quantity of the second marking information which is slid and taken meets a second preset condition, and if so, determining the score of the attitude scoring point(ii) a Said N is₂Is a positive integer greater than 1.

Optionally, the identifying a heat source, a heated instrument, a heat-extinguishing instrument and a heat-source instrument from the operation video to obtain an identification frame of the heat source, an identification frame of the heated instrument, an identification frame of the heat-extinguishing instrument, an identification frame of the heat-source instrument and a key point of the heated instrument includes:

for each frame of image in the operation video, performing down-sampling on the frame of image to obtain a down-sampled image; converting the down-sampled image from a YUV format to an RGB format to obtain an RGB format image;

inputting each frame of RGB format image into a neural network model to obtain an identification frame of the heat source, an identification frame of the heated instrument, an identification frame of the heat extinguishing instrument, an identification frame of the heat source instrument and key points of the heated instrument which are output by the neural network model.

Optionally, the acquiring an operation video of the heating experiment includes:

acquiring a first video acquired by a first camera and a second video acquired by a second camera, wherein the shooting angle of the first camera is flat shooting, the shooting angle of the second camera is depression shooting, and the first video and the second video are operation videos aiming at the same heating experiment;

calculating a horizontal included angle and a vertical included angle of the heated instrument based on the mouth position point and the bottom position point, including:

calculating a horizontal included angle of the heated instrument based on the mouth position point and the bottom position point in the first video;

in the second video, calculating a vertical included angle of the heated instrument based on the mouth position point and the bottom position point;

the determining the grade of the heating operation scoring point according to whether the position relation between the heat source and the heated instrument conforms to a first target position state corresponding to the heating operation scoring point comprises the following steps:

determining a positional relationship of the heat source to the heated instrument in the first video;

determining a positional relationship of the heat-extinguishing instrument and the heat source in the first video;

In order to achieve the above object, an embodiment of the present invention further provides an experimental operation scoring apparatus, including:

the acquisition module is used for acquiring an operation video of the heating experiment;

the identification module is used for identifying a heat source, a heated instrument and a heat extinguishing instrument in the operation video;

the first scoring module is used for determining the scoring of the heating operation scoring points according to whether the position relation between the heat source and the heated instrument accords with a first target position state corresponding to the heating operation scoring points;

and the second scoring module is used for determining the score of the heat extinguishing operation scoring point according to whether the position relation between the heat extinguishing instrument and the heat source accords with the second target position state corresponding to the heat extinguishing operation scoring point.

Optionally, the identification module is specifically configured to:

Optionally, the first scoring module includes:

the judgment submodule is used for judging whether the key point of the heated instrument is positioned in the identification frame of the heat source or not;

the calculation submodule is used for calculating the distance between the key point of the heated instrument and the designated point in the identification frame of the heat source;

a determination submodule configured to determine a heating operation score point when the key point is located within the recognition frame of the heat source and the distance matches a target distance corresponding to the heating operation score point.

Optionally, the determining sub-module is specifically configured to:

Optionally, the identification module is specifically configured to: identifying a heat source, a heated instrument, a heat-extinguishing instrument and a heat source instrument from the operation video to obtain an identification frame of the heat source, an identification frame of the heated instrument, an identification frame of the heat-extinguishing instrument, an identification frame of the heat source instrument and a key point of the heated instrument; the heat source instrument is a carrier of the heat source;

the second scoring module is specifically configured to: and if the identification frame of the heat extinguishing instrument is intersected with the identification frame of the heat source instrument and the identification frame of the heat extinguishing instrument is intersected with the identification frame of the heat source, determining the score of the heat extinguishing operation scoring point.

Optionally, the second scoring module is specifically configured to:

Optionally, the target heating position points are: a bottom location point of the heated instrument; the key points further include: a mouth location point of the heated instrument; the device further comprises:

the calculation module is used for calculating a horizontal included angle and a vertical included angle of the heated instrument based on the mouth part position point and the bottom part position point; the horizontal included angle is an included angle between the heated instrument and a horizontal plane, and the vertical included angle is an included angle between the heated instrument and a vertical plane corresponding to an operator;

and the determining module is used for determining the score of the attitude scoring point under the condition that the bottom position point is positioned in the heat source target frame, the horizontal included angle is positioned in a first preset interval, and the vertical included angle is positioned in a second preset interval.

Optionally, the determining module is specifically configured to:

sequentially sliding and taking continuous N through sliding window₂Second marking information of the frame image, judging whether the quantity of the second marking information which is slid and taken meets a second preset condition, if so, determining a posture scoring pointScoring; said N is₂Is a positive integer greater than 1.

Optionally, the identification module is specifically configured to:

Optionally, the obtaining module is specifically configured to:

the calculation module is specifically configured to: calculating a horizontal included angle of the heated instrument based on the mouth position point and the bottom position point in the first video; in the second video, calculating a vertical included angle of the heated instrument based on the mouth position point and the bottom position point;

the first scoring module is specifically configured to: determining a positional relationship of the heat source to the heated instrument in the first video; determining the grade of the heating operation scoring point according to whether the position relation between the heat source and the heated instrument accords with a first target position state corresponding to the heating operation scoring point;

the second scoring module is specifically configured to: determining a positional relationship of the heat-extinguishing instrument and the heat source in the first video; and determining the grade of the heat extinguishing operation scoring point according to whether the position relation between the heat extinguishing instrument and the heat source accords with the second target position state corresponding to the heat extinguishing operation scoring point.

In order to achieve the above object, an embodiment of the present invention further provides an electronic device, including a processor and a memory;

a memory for storing a computer program;

and the processor is used for realizing any one of the above experimental operation scoring methods when executing the program stored in the memory.

In order to achieve the above object, an embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the computer program implements any one of the above experimental operation scoring methods.

By applying the embodiment of the invention, the operation video of the heating experiment is obtained, and then the scores of the heating operation score point and the heat-extinguishing operation score point are automatically determined by the electronic equipment based on the operation video. Therefore, the scheme does not need manual observation and experiment operation and scoring, and reduces the consumption of manpower.

Of course, not all of the advantages described above need to be achieved at the same time in the practice of any one product or method of the invention.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.

FIG. 1 is a first schematic flow chart of a scoring method for experimental operations according to an embodiment of the present invention;

fig. 2 is a scene schematic diagram of a heating experiment according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of the identification box and keypoints identified in FIG. 2;

FIG. 4 is a schematic diagram of a heat-quenching operation provided by an embodiment of the present invention;

FIG. 5 is a schematic illustration of the identification box labeled in FIG. 4;

FIG. 6 is a schematic view of a heating experiment in a forward and top view according to an embodiment of the present invention;

FIG. 7 is a second flowchart of an experimental operation scoring method according to an embodiment of the present invention;

fig. 8 is a schematic structural diagram of an experimental operation scoring device according to an embodiment of the present invention;

fig. 9 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

In order to achieve the above object, embodiments of the present invention provide a method, an apparatus, a device and a storage medium for scoring experimental operations, where the method and the apparatus may be applied to various electronic devices, and are not limited specifically. The method of scoring the experimental procedure will first be described in detail below.

Fig. 1 is a first schematic flow chart of an experimental operation scoring method according to an embodiment of the present invention, including:

s101: an operational video of the heating experiment was acquired.

For example, in an experimental examination scene, one or more cameras can be arranged for each examinee's laboratory bench to collect operation videos of students.

The heating experiment in S101 may be an experiment including a heating operation, such as a solid identification experiment, an experiment for preparing oxygen from potassium chlorate, and the like, and is not particularly limited. For example, in the case of a solid-state assay, the assay process may include: the water in the measuring cylinder is poured into a test tube filled with red solid, dilute sulfuric acid solution is added into the test tube, then the test tube is heated, and the reaction result is observed to judge whether the solid is copper oxide or carbon powder.

S102: identifying a heat source, a heated instrument, and a heat-extinguished instrument in the operating video.

Referring to the heating experiment shown in fig. 2, the test tube was clamped by the test tube clamp and heated by the alcohol lamp. The heat source may be a flame of an alcohol lamp, the heated instrument may be a test tube, and the heat-quenching instrument may be a cap of an alcohol lamp (not shown in fig. 2). In different heating experiments, different experimental instruments are used, and a heat source, a heated instrument and a heat extinguishing instrument are not completely the same and are not listed.

For example, the scoring points for the heating experiment may include any one or more of: a heating operation scoring point for evaluating whether the heating operation is in compliance, for example, the heating operation in compliance may be: heating with alcohol burner. The posture scoring point is used for evaluating whether the posture of the instrument is in compliance, for example, the posture of the instrument in compliance may be: when heating, the test tube mouth inclines upwards for about 45 degrees and is not facing people; and thirdly, a heat-extinguishing operation scoring point for evaluating whether the heat-extinguishing operation is in compliance, for example, the heat-extinguishing operation in compliance can be as follows: the alcohol lamp cap extinguishes the alcohol lamp.

In one embodiment, S102 may include: identifying a heat source, a heated instrument and a heat-extinguishing instrument from the operation video to obtain an identification frame of the heat source, an identification frame of the heated instrument, an identification frame of the heat-extinguishing instrument and a key point of the heated instrument; the key points include: a target heating location point on the heated instrument.

For example, each frame of image in the operation video may be identified by using an image identification algorithm, and an identification frame of each identification object is obtained: identification of heat sources, heated instruments, extinguished instruments, etc. For example, each frame of image in the operation video may be input into the neural network model, and the recognition frame of each recognition object output by the neural network model is obtained. The Neural Network model may be a CNN (Convolutional Neural Network), such as MASK-RCNN.

In one case, in addition to the recognition frame of each recognition object, the key point of the heated instrument can be obtained. The key point may comprise a target heating location point on the heated instrument. Assuming that the heated instrument is a test tube, the target heating location point may be a bottom location point of the test tube.

S103: and determining the grade of the heating operation scoring point according to whether the position relation between the heat source and the heated instrument conforms to the first target position state corresponding to the heating operation scoring point.

As described above, the heating operation evaluation point is used to evaluate whether the heating operation is in compliance, for example, the heating operation in compliance may be: heating with alcohol burner. In one embodiment, an identification frame of key points and heat sources of the heated instrument is obtained in S102; thus, S103 may include: judging whether the key point of the heated instrument is positioned in the identification frame of the heat source; calculating the distance between the key point of the heated instrument and a designated point in the identification frame of the heat source; and if the key point is positioned in the identification frame of the heat source and the distance is matched with the target distance corresponding to the heating operation scoring point, determining the score of the heating operation scoring point.

In one case, referring to fig. 3, the key point of the heated instrument may be a bottom position point a of the test tube, the designated point in the recognition frame of the heat source may be an upper left corner point B of the recognition frame, and the distance between the point a and the point B is calculated. In this case, the target distance corresponding to the heating operation scoring point may be 0 to 20 pixel points; and if the distance between the A and the B is within 20 pixel points and the A is positioned in the identification frame of the heat source, determining the heating operation scoring point score.

Alternatively, in other cases, the designated point in the identification frame of the heat source may be another point, such as the upper right corner point or the middle point of the upper frame, and the like, which is not limited specifically.

In one embodiment, for each frame of image in the operation video, if a key point of the heated instrument in the frame of image is located within the recognition frame of the heat source and the distance between the key point and a designated point in the recognition frame of the heat source matches with the target distance corresponding to the heating operation scoring point, adding first marking information to the frame of image; sequentially slide and take out the connecting piece through the sliding windowContinuous N₁Judging whether the quantity of the first mark information which is slid and taken meets a first preset condition or not according to the first mark information of the frame image, and if so, determining the score of the heating operation scoring point; said N is₁Is a positive integer greater than 1.

Still referring to fig. 3 for example, for each frame of image in the operation video, if the frame of image identifies the identification frame of the heat source (flame), it is determined whether the tube bottom position point a is located within the heat source identification frame, and a difference y _ diff between the y coordinate of the point a and the y coordinate of the upper left corner point B of the heat source identification frame is calculated; if the point a is located within the heat source identification box and y _ diff is within the range of [ -20,20] pixels, first mark information is added to the frame image, for example, the flag position of the heating operation score point corresponding to the frame image may be set to 1. If the point a is outside the heat source identification box or y _ diff is not in the range of [ -20,20] pixels, the flag position of the heating operation score point corresponding to the frame image may be set to 0.

In this way, the flag bit of the heating operation score point corresponding to each frame of image in the operation video is 0 or 1, and then continuous N are slid through the sliding window every time₁The flag bit of the heating operation score point in the frame is counted by N₁And (4) summing the zone bits of the heating operation scoring points in the frame to obtain sum _1, scoring the heating operation scoring points if sum _1 meets a first preset condition, and not scoring if sum _1 does not meet the first preset condition. For example, N₁May be 50, 60, etc., and the first preset condition may be greater than 35, or greater than 40, etc., without limitation.

S104: and determining the grade of the heat extinguishing operation scoring point according to whether the position relation between the heat extinguishing instrument and the heat source accords with the second target position state corresponding to the heat extinguishing operation scoring point.

As described above, the extinction operation score point is used to evaluate whether the extinction operation is compliant, for example, a compliant extinction operation may be as shown in fig. 4: the alcohol lamp cap extinguishes the alcohol lamp.

As described above, each frame of image in the operation video may be identified by using an image identification algorithm to obtain an identification frame of each identification object. If the heat source is the flame of an alcohol burner, the heat source instrument may be an alcohol burner. In such an embodiment, S104 may include: and if the identification frame of the heat extinguishing instrument is intersected with the identification frame of the heat source instrument and the identification frame of the heat extinguishing instrument is intersected with the identification frame of the heat source, determining the score of the heat extinguishing operation scoring point.

Referring to fig. 5, the heat-extinguishing apparatus is an alcohol lamp cap, the heat-source apparatus is an alcohol lamp, the heat source is a flame, the intersection iou1 of the recognition frame of the alcohol lamp cap and the recognition frame of the alcohol lamp, and the intersection iou2 of the recognition frame of the alcohol lamp cap and the recognition frame of the flame are calculated, and if iou1 is greater than 0 and iou2 is greater than 0, the score of the heat-extinguishing operation score point is determined.

In one embodiment, for each frame of image in the operation video, if an intersection exists between the identification frame of the heat-extinguishing apparatus and the identification frame of the heat-source apparatus in the frame of image, and an intersection exists between the identification frame of the heat-extinguishing apparatus and the identification frame of the heat source, adding third label information to the frame of image; sequentially sliding and taking continuous N through sliding window₃Judging whether the number of the third mark information which is slid and taken meets a third preset condition or not according to third mark information of the frame image, and if so, determining the score of the heat-extinguishing operation scoring point; said N is₃Is a positive integer greater than 1.

Still taking fig. 5 as an example, for each frame of image in the operation video, if the intersection iou1 of the recognition frame of the alcohol lamp cap and the recognition frame of the alcohol lamp and the intersection iou2 of the recognition frame of the alcohol lamp cap and the recognition frame of the flame in the frame of image are both greater than 0, third marker information is added to the frame of image, for example, the marker position of the heat-off operation score point corresponding to the frame of image may be set to 1. If iou1 and iou2 are not both greater than 0, the flag position of the corresponding heat-off operation score point of the frame image can be set to 0.

In this way, the flag bit of the heat-off operation score point corresponding to each frame of image in the operation video is 0 or 1, and then continuous N are obtained by sliding through the sliding window every time₃The flag bit of the heating operation score point in the frame is counted by N₃And (3) the sum of the zone bits of the heating operation scoring points in the frame is sum _3, if sum _3 meets a third preset condition, the heating operation scoring points are scored, and otherwise, the scoring is not scored. For example, N₃May be 50, 60, etc., and the third preset condition may be greater than 5, or greater than 6, etc., and the third preset condition is not limited.

In one embodiment, the scoring points of the heating experiment may include a posture scoring point for evaluating whether the posture of the heated instrument is compliant. As described above, in addition to identifying the identification frames of the respective identification objects, a key point of the heated instrument may be obtained, which may include a target heating position point on the heated instrument; in this embodiment, the target heating position point may be a bottom position point of the heated instrument; the key points may also include the mouth location points of the heated instrument. In other words, the key points identified in S102 may include: the mouth position point of the heating instrument and the bottom position point of the heated instrument.

Evaluating whether the pose of the heated instrument is compliant may include: calculating a horizontal included angle and a vertical included angle of the heated instrument based on the mouth position point and the bottom position point; the horizontal included angle is an included angle between the heated instrument and a horizontal plane, and the vertical included angle is an included angle between the heated instrument and a vertical plane corresponding to an operator; and if the bottom position point is located in the heat source target frame, the horizontal included angle is located in a first preset interval, and the vertical included angle is located in a second preset interval, determining the score of the attitude scoring point.

Still referring to the alcohol lamp heated tube experiment shown in FIG. 2, the compliant tube attitude may be: the test tube mouth inclines upwards about 45 degrees and the test tube mouth is not to people when heating. The test tube opening is not inclined towards the person, i.e. away from the operator.

Referring to fig. 6, (b) is a schematic diagram showing a top view of (a), and a vertical plane corresponding to the operator can be understood as a vertical plane perpendicular to the ground formed when the operator stands, wherein a key point a represents a bottom position point of the test tube, and a key point C represents a mouth position point of the test tube. The horizontal angle of the test tube shown in (a) and the vertical angle of the test tube shown in (b) can be calculated based on the key point a and the key point C. If the key point A is located in the flame recognition frame, the horizontal included angle is located in a first preset interval, and the vertical included angle is located in a second preset interval, indicating test tube posture compliance, and determining a posture scoring point score.

In one embodiment, for each frame of image in the operation video, if a bottom position point of the heated instrument in the frame of image is located in the heat source target frame, a horizontal included angle of the heated instrument is located in a first preset interval, and a vertical included angle of the heated instrument is located in a second preset interval, adding second mark information to the frame of image; sequentially sliding and taking continuous N through sliding window₂Judging whether the quantity of the second mark information which is slid and taken by the second mark information of the frame image meets a second preset condition, and if so, determining the score of the attitude scoring point; said N is₂Is a positive integer greater than 1.

Still taking fig. 6 as an example, for each frame of image in the operation video, at a forward shooting viewing angle, as shown in (a), a horizontal included angle alpha of the tube is calculated based on the key point a and the key point C, if the key point a is within the identification frame of the flame and the alpha is located in a first preset interval, such as [40,50] degree range, then the lean _ flag of the frame of image is set to 1, otherwise, the lean _ flag is set to 0, and the lean _ flag may indicate that the flag position is tilted upward by about 45 °. Under a top-down view, as shown in (b), an included angle beta between the test tube and the vertical direction vector is calculated based on the key point A and the key point C, if the beta is located in a second preset interval, for example, in a range of [0,30] degrees, the towards _ flag of the frame image is set to be 1, otherwise, the towards _ flag is set to be 0, and the towards _ flag can indicate that the test tube port does not flag the human body. If the lean _ flag is 1 and the towards _ flag is 1, adding second flag information to the frame image, for example, setting a flag position of a pose score corresponding to the frame image to 1. If the lean _ flag and the towards _ flag are not both 1, the flag position of the attitude score point corresponding to the frame image may be set to 0.

In this way, the flag bit of the attitude score point corresponding to each frame of image in the operation video is 0 or 1, and then the operation video is connected with the corresponding attitude score point through sliding window every timeContinuous N₂The flag bit of the attitude score point in the frame is counted by N₂And (4) the sum of the zone bits of the posture score points in the frame is sum _2, if sum _2 meets a second preset condition, the posture score points are scored, and otherwise, the posture score points are not scored. For example, N₂May be 50, 60, etc., and is not particularly limited, and the second preset condition may be greater than 35, or greater than 40, etc., and is not particularly limited.

As described above, each frame image in the operation video may be input into the neural network model. In one embodiment, the obtained operation video may be in YUV (Y represents brightness: Luma or Chroma; U and V represent Chroma: Chroma or Chroma) format, and each frame of image in the operation video may be down-sampled to obtain a down-sampled image; converting the down-sampling image from a YUV format to an RGB (Red Green Blue) format to obtain an RGB format image; and inputting the RGB format image into a neural network model to obtain an identification frame of the heat source, an identification frame of the heated instrument, an identification frame of the heat extinguishing instrument, an identification frame of the heat source instrument and key points of the heated instrument which are output by the neural network model.

The YUV format occupies a small storage space, and the occupation of the storage space is reduced by acquiring the image in the YUV format. The image in the YUV format is subjected to down sampling and format conversion, so that the neural network model can be well adapted, and the identification accuracy of the neural network model is improved.

As described above, in the experimental examination scene, one or more cameras may be provided for each examinee's laboratory bench to capture the operation video of the student. In one embodiment, a first camera and a second camera can be set for each examinee's laboratory bench, the shooting angle of the first camera is flat shooting, and the shooting angle of the second camera is flat shooting. Thus, S101 may include: the method comprises the steps of acquiring a first video acquired by a first camera and a second video acquired by a second camera, wherein the shooting angle of the first camera is flat shooting, the shooting angle of the second camera is depression shooting, and the first video and the second video are operation videos aiming at the same heating experiment.

In this embodiment, the heating operation scoring point and the heat-extinguishing operation scoring point may be scored based on the first video, the posture scoring point may be scored based on the first video and the second video, and as shown in fig. 6, the horizontal included angle of the heated instrument may be determined based on the first video, and the vertical included angle of the heated instrument may be determined based on the second video.

In this embodiment, a horizontal included angle of the heated instrument may be calculated based on the mouth position point and the bottom position point in the first video; in the second video, calculating a vertical included angle of the heated instrument based on the mouth position point and the bottom position point; determining a positional relationship of the heat source to the heated instrument in the first video; determining the grade of the heating operation scoring point according to whether the position relation between the heat source and the heated instrument accords with a first target position state corresponding to the heating operation scoring point; determining a positional relationship of the heat-extinguishing instrument and the heat source in the first video; and determining the grade of the heat extinguishing operation scoring point according to whether the position relation between the heat extinguishing instrument and the heat source accords with the second target position state corresponding to the heat extinguishing operation scoring point. Therefore, the scoring method scores the experimental operation based on videos at different viewing angles, and the scoring accuracy is higher.

With the embodiment of the invention, in the first aspect, the operation video of the heating experiment is acquired, and then the scores of the heating operation score point and the heat-off operation score point are automatically determined by the electronic device based on the operation video. Therefore, the scheme does not need manual observation and experiment operation and scoring, and reduces the consumption of manpower. In a second aspect, in some embodiments, the neural network model is used for positioning, so that the recognition accuracy is improved, and the scoring accuracy is further improved.

A specific embodiment will be described below by taking the heating experiment shown in FIGS. 2 and 3 as an example. In fig. 2 and 3, the tube is clamped by the tube clamp and heated by the alcohol lamp.

The scoring points for this experiment may include: the heating operation evaluation point is used for evaluating whether the heating operation is in compliance, and the heating operation in compliance can be as follows: heating with alcohol burner. The posture evaluation point is used for evaluating whether the posture of the instrument is in compliance, and the attitude in compliance can be as follows: when heating, the test tube mouth inclines upwards for about 45 degrees and is not facing people; and thirdly, a heat extinguishing operation score point for evaluating whether the heat extinguishing operation is in compliance, wherein the compliance heat extinguishing operation can be as follows: the alcohol lamp cap extinguishes the alcohol lamp.

As shown in fig. 7, the experimental procedure scoring method may include:

s701: acquiring an operation video of a heating experiment, wherein the operation video comprises: a first video captured by the first camera and a second video captured by the second camera.

For example, in an experimental examination scene, a first camera and a second camera may be set for each examinee's laboratory bench, the shooting angle of the first camera is flat shooting, the shooting angle of the second camera is flat shooting, and the first video and the second video are operation videos for the same heating experiment.

S702: inputting each frame of image in the first video into a neural network model to obtain a flame recognition frame, a test tube recognition frame, an alcohol lamp cap recognition frame, an alcohol lamp recognition frame, and a bottom position point and an oral position point of the test tube, which are output by the neural network model;

and inputting each frame of image in the second video into the neural network model to obtain the bottom position point and the mouth position point of the test tube output by the neural network model.

In one embodiment, the operation video acquired in S701 may be in a YUV format, and each frame of image in the operation video may be down-sampled to obtain a down-sampled image; converting the down-sampled image from a YUV format to an RGB format to obtain an RGB format image; inputting the RGB format image into a neural network model.

S703: the following operations are respectively executed for each frame of image in the first video:

S703A: and calculating the difference y _ diff between the y coordinate of the bottom position point of the test tube in the frame image and the y coordinate of the upper left corner point of the flame identification frame, and adding first mark information to the frame image if the bottom position point of the test tube is in the flame identification frame and the y _ diff is in the range of [ -20,20] pixels.

S703B: and calculating a horizontal included angle alpha of the test tube based on the bottom position point and the mouth position point of the test tube in the frame image, and if the bottom position point of the test tube is in the identification frame of the flame and the alpha is in the range of [40,50] degrees, setting the lean _ flag of the frame image to be 1, otherwise, setting the lean _ flag of the frame image to be 0.

S703C: and calculating the intersection iou1 of the recognition frame of the alcohol lamp cap and the recognition frame of the alcohol lamp, and the intersection iou2 of the recognition frame of the alcohol lamp cap and the recognition frame of the flame in the frame image, and adding third mark information to the frame image if the iou1 is greater than 0 and the iou2 is greater than 0.

S704: and for each frame of image in the second video, calculating a vertical included angle beta of the test tube based on the bottom position point and the mouth position point of the test tube in the frame of image, and if the beta is in a [0,30] degree range, setting the towards _ flag of the frame of image to be 1, otherwise, setting the towards _ flag of the frame of image to be 0.

S705: for a picture in a first video and a picture in a second video with the same timestamp, if a bean _ flag of the picture in the first video is 1 and a towards _ flag of the picture in the second video is 1, adding second flag information to the picture in the first video.

S706: sequentially sliding first mark information, second mark information and third mark information of continuous N frames of images in the first video through a sliding window; and counting the number of the first mark information obtained by each sliding and recording the number as sum _1, counting the number of the second mark information obtained by each sliding and recording the number as sum _2, and counting the number of the third mark information obtained by each sliding and recording the number as sum _ 3.

S707: judging whether sum _1 meeting a first preset condition exists or not, and if yes, determining a heating operation score point; judging whether sum _2 meeting a second preset condition exists or not, and if yes, determining the score of the attitude scoring point; and judging whether sum _3 meeting a third preset condition exists or not, and if so, determining the score of the heat-extinguishing operation scoring point.

For example, N may be 50, the first preset condition may be greater than 35, the second preset condition may be greater than 35, and the third preset condition may be greater than 5.

Corresponding to the above method embodiment, an embodiment of the present invention further provides an experimental operation scoring apparatus, as shown in fig. 8, including:

an obtaining module 801, configured to obtain an operation video of a heating experiment;

an identification module 802 for identifying a heat source, a heated instrument, and a heat-extinguished instrument in the operation video;

a first scoring module 803, configured to determine a score of the heating operation scoring point according to whether the position relationship between the heat source and the heated instrument conforms to a first target position state corresponding to the heating operation scoring point;

and the second scoring module 804 is used for determining the score of the heat extinguishing operation scoring point according to whether the position relationship between the heat extinguishing instrument and the heat source accords with the second target position state corresponding to the heat extinguishing operation scoring point.

In one embodiment, the identifying module 802 is specifically configured to:

In one embodiment, the first scoring module 803 includes: a decision sub-module, a calculation sub-module and a determination sub-module (not shown), wherein,

In one embodiment, the determining submodule is specifically configured to:

In one embodiment, the identifying module 802 is specifically configured to: identifying a heat source, a heated instrument, a heat-extinguishing instrument and a heat source instrument from the operation video to obtain an identification frame of the heat source, an identification frame of the heated instrument, an identification frame of the heat-extinguishing instrument, an identification frame of the heat source instrument and a key point of the heated instrument; the heat source instrument is a carrier of the heat source;

the second scoring module 804 is specifically configured to: and if the identification frame of the heat extinguishing instrument is intersected with the identification frame of the heat source instrument and the identification frame of the heat extinguishing instrument is intersected with the identification frame of the heat source, determining the score of the heat extinguishing operation scoring point.

In one embodiment, the second scoring module 804 is specifically configured to:

In one embodiment, the target heating location points are: a bottom location point of the heated instrument; the key points further include: a mouth location point of the heated instrument; the device further comprises: a calculation module and a determination module (not shown in the figures), wherein,

In an embodiment, the determining module is specifically configured to:

sequentially sliding and taking continuous N through sliding window₂Judging whether the quantity of the second mark information which is slid and taken by the second mark information of the frame image meets a second preset condition, and if so, determining the score of the attitude scoring point; said N is₂Is a positive integer greater than 1.

In one embodiment, the identifying module 802 is specifically configured to:

In one embodiment, the obtaining module 801 is specifically configured to:

the first scoring module 803 is specifically configured to: determining a positional relationship of the heat source to the heated instrument in the first video; determining the grade of the heating operation scoring point according to whether the position relation between the heat source and the heated instrument accords with a first target position state corresponding to the heating operation scoring point;

the second scoring module 804 is specifically configured to: determining a positional relationship of the heat-extinguishing instrument and the heat source in the first video; and determining the grade of the heat extinguishing operation scoring point according to whether the position relation between the heat extinguishing instrument and the heat source accords with the second target position state corresponding to the heat extinguishing operation scoring point.

An embodiment of the present invention further provides an electronic device, as shown in fig. 9, including a processor 901 and a memory 902,

a memory 902 for storing a computer program;

the processor 901 is configured to implement any one of the above-described experimental operation scoring methods when executing the program stored in the memory 902.

The Memory mentioned in the above electronic device may include a Random Access Memory (RAM) or a Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.

The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, discrete hardware component.

In another embodiment of the present invention, a computer-readable storage medium is further provided, in which a computer program is stored, and the computer program, when executed by a processor, implements any one of the above-mentioned experimental operation scoring methods.

In yet another embodiment, a computer program product containing instructions is provided, which when run on a computer, causes the computer to perform any one of the above described experimental operation scoring methods.

In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the invention to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website site, computer, server, or data center to another website site, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.

It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.

All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, apparatus embodiments, electronic device embodiments, computer-readable storage medium embodiments, and computer program product embodiments are described with relative simplicity as they are substantially similar to method embodiments, where relevant only as described in portions of the method embodiments.

The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.

Claims

1. A method for scoring an experimental procedure, comprising:

acquiring an operation video of a heating experiment;

2. The method of claim 1, wherein the identifying heat sources, heated instruments, and extinguished heat instruments in the operational video comprises:

3. The method of claim 2, wherein determining the score of the heating operation score point according to whether the positional relationship of the heat source and the heated instrument conforms to the first target position state of the heating operation score point comprises:

4. The method of claim 3, wherein determining the heating operation score point score if the distance matches a target distance corresponding to a heating operation score point comprises:

sequentially sliding and taking continuous N through sliding window₁The method comprises the steps of obtaining first mark information of a frame image, judging whether the quantity of the first mark information which is slid and obtained meets a first preset condition, and if so, determining the heating operation scorePoint scoring; said N is₁Is a positive integer greater than 1.

5. The method of claim 2, wherein said identifying a heat source, a heated instrument, and a heat-extinguished instrument from the operation video, resulting in an identification frame of the heat source, an identification frame of the heated instrument, an identification frame of the heat-extinguished instrument, and a keypoint of the heated instrument, comprises:

6. The method of claim 5, wherein determining the flameout operation score point score if the identification frame of the flameout instrument intersects the identification frame of the heat source instrument and the identification frame of the flameout instrument intersects the identification frame of the heat source comprises:

sequentially sliding and taking continuous N through sliding window₃The third marking information of the frame image judges whether the quantity of the third marking information which is slid and fetched meets a third preset bar or notIf yes, determining the score of the heat-extinguishing operation scoring point; said N is₃Is a positive integer greater than 1.

7. The method of claim 2, wherein the target heating location points are: a bottom location point of the heated instrument; the key points further include: a mouth location point of the heated instrument; the method further comprises the following steps:

8. The method of claim 7, wherein determining a pose score point score if the bottom position point is within the heat source target frame, the horizontal included angle is within a first predetermined range, and the vertical included angle is within a second predetermined range comprises:

9. The method of claim 7, wherein the obtaining the operational video of the heating experiment comprises:

10. An experimental operation scoring device, comprising:

11. An electronic device comprising a processor and a memory;

a memory for storing a computer program;

a processor for implementing the method steps of any of claims 1-9 when executing a program stored in the memory.

12. A computer-readable storage medium, characterized in that a computer program is stored in the computer-readable storage medium, which computer program, when being executed by a processor, carries out the method steps of any one of the claims 1-9.