CN108268854B - Teaching assistance big data intelligent analysis method based on feature recognition - Google Patents

Teaching assistance big data intelligent analysis method based on feature recognition Download PDF

Info

Publication number
CN108268854B
CN108268854B CN201810106613.1A CN201810106613A CN108268854B CN 108268854 B CN108268854 B CN 108268854B CN 201810106613 A CN201810106613 A CN 201810106613A CN 108268854 B CN108268854 B CN 108268854B
Authority
CN
China
Prior art keywords
image
gradient
feature
action
width
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201810106613.1A
Other languages
Chinese (zh)
Other versions
CN108268854A (en
Inventor
谈加杰
徐金玉
康志恒
宋娜
李柠
李恒涛
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Uwoo Technology Co ltd
Original Assignee
Shanghai Uwoo Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Uwoo Technology Co ltd filed Critical Shanghai Uwoo Technology Co ltd
Priority to CN201810106613.1A priority Critical patent/CN108268854B/en
Publication of CN108268854A publication Critical patent/CN108268854A/en
Application granted granted Critical
Publication of CN108268854B publication Critical patent/CN108268854B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/172—Classification, e.g. identification

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Human Computer Interaction (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Health & Medical Sciences (AREA)
  • Data Mining & Analysis (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Health & Medical Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a teaching assistance big data intelligent analysis method based on feature recognition, which comprises the following steps: s1, obtaining static images of students; s2, extracting the directional gradient feature of the image; s3, machine learning; s4, sliding a window; s5, selecting the action with the highest probability under action classification, and identifying the action; and S6, performing face recognition on the action sender in S5. The invention can extract the behavior characteristics of the students, thereby identifying the behaviors and actions of the students, facilitating the management of the students and being beneficial to the improvement of the teaching quality.

Description

Teaching assistance big data intelligent analysis method based on feature recognition
Technical Field
The invention relates to a big data intelligent analysis method, in particular to a teaching assistance big data intelligent analysis method based on feature recognition.
Background
With the development of big data analysis technology, people are beginning to apply big data technology to the field of education, such as Chinese patent applications with publication numbers CN107316261A, CN106023013A, CN104573071A, and the like.
In a classroom, at least dozens of students generally cannot observe the action of each student all the time when teachers teach, so that the interaction between the students and the teachers can be reduced, or the teachers cannot find the students who are not in class seriously.
Therefore, the method is very important for analyzing the behaviors and actions of students, but no technology for specially analyzing the behaviors and actions of the students exists in China at present. Therefore, the applicant provides a teaching assistance big data intelligent analysis method based on feature recognition, which can extract behavior features of students so as to recognize actions of the students, perform teaching supervision, improve teaching quality and the like.
Disclosure of Invention
In view of the above defects in the prior art, the technical problem to be solved by the present invention is to provide a teaching assistance big data intelligent analysis method based on feature recognition.
In order to achieve the purpose, the invention provides a teaching assistance big data intelligent analysis method based on feature recognition, which comprises the following steps:
s1, obtaining static images of the students;
s2, extracting the directional gradient characteristics of the image; judging whether the model trained by the machine learning of S3 belongs to a certain category or not by calculation;
s3, machine learning;
s4, sliding a window;
s5, selecting the action with the highest probability under action classification, and identifying the action;
and S6, performing face recognition on the action sender in S5.
Preferably, S2, extracting the directional gradient feature of the image, includes the following steps:
S21, converting the color image into a corresponding gray image, and regarding the gray image as a three-dimensional image, where each pixel point on the image is represented by (x, y, gray), where x represents an x coordinate of the pixel point in the image, y represents a y coordinate of the pixel point in the image, and gray represents a gray value of the pixel point, and the conversion formula is as follows:
Gray＝0.3*R+0.59*G+0.11*B；
s22, adjusting the color contrast of the image, reducing the influence of shadow caused by uneven local illumination on gradient characteristics, and adopting Gamma correction method, wherein the specific calculation formula is as follows:
I(x，y)＝I(x，y)gamma
if gamma is more than 1, the contrast ratio of the part with darker gray scale can be improved;
s23, calculating the gradient feature of each pixel point on the image, wherein the gradient relation between the pixels can be extracted, and the influence of illumination on the image can be further inhibited, and the specific calculation formula is as follows:
Figure GDA0001615897910000021
Figure GDA0001615897910000031
the table represents the gray values of the pixels corresponding to the gray image obtained in S22; since the directions of the grayscale image have an x direction and a y direction, the gradient direction needs to calculate the gradients in the two directions respectively and find the gradient magnitude according to the amplitude formula:
Figure GDA0001615897910000032
Figure GDA0001615897910000033
the gradient magnitude and gradient direction of point (x, y) are found according to the magnitude formula:
Figure GDA0001615897910000034
Figure GDA0001615897910000035
；
S24, dividing the image into a plurality of cell intervals according to pixels, and counting the gray directions of all pixel points in each cell interval;
s25, normalizing the gradient features extracted from the cell units, splicing the gradient features extracted from a plurality of adjacent cell units into a gradient feature of a larger area, and then normalizing the gradient features by using a two-norm normalization method, wherein the method comprises the following specific steps:
suppose v is a feature vector before normalization, | v | | luminance2Representing the two-norm of the feature vector v, and solving the formula as follows:
Figure GDA0001615897910000036
the normalized feature vector is:
Figure GDA0001615897910000037
where ε is a small number, which may be 0.01, to prevent the denominator in the above formula from appearing to be approximately 0
S26, after the gradient feature vectors in all the intervals composed of the 'cell intervals' are standardized, the gradient feature vectors in all the intervals need to be spliced to form the gradient feature vector corresponding to the whole image, and the gradient feature vector dimension calculation formula of the image is as follows:
Figure GDA0001615897910000041
wherein size is the dimension of the gradient feature of the finally obtained image, blocksize.width and blocksize.height are the width and height corresponding to the interval, cellsize.width and cellsize.height are the width and height corresponding to the 'cell interval', winsize.width and winsize.height are the width and height corresponding to the detected image, blocksize.width and blocksize.height are the width and height corresponding to the interval Step, nbin represents the number of regional channels used in the gradient angle statistics in Step 4;
Preferably, S3, the machine learning, includes the following steps: after the gradient feature of each sample image is obtained in the S2, training the gradient feature of a class of behavior action of the student by adopting an SVM, and corresponding to a classification model; the classification model can be a gradient characteristic model for lifting hands, reading, writing and the like;
s31, the user marks some relevant retrieval results as positive samples and some negative samples irrelevant to retrieval, and then the image retrieval system updates the retrieval results based on the feedback samples; the two steps are alternately carried out, so that the computer can gradually learn the requirements of the user and the retrieval function is enhanced;
s311, dividing the image gradient feature space into a plurality of areas, wherein the positive sample farthest from the specific classification plane is used as the most relevant sample, and the sample closest to the classification plane is used as the sample with the most information content;
s312, an associated feedback method of the SVM-based image retrieval system comprises the following steps: SVM is an effective two-classification, multi-classification algorithm, and the following takes two classification problems as an example (
Figure GDA0001615897910000042
And y isi{ +1, -1}), a multi-class problem can be analogized;
wherein x isiIs an n-dimensional feature vector, yiIs the class information to which the feature vector belongs,
The SVM divides two types of sample points through a hyperplane, namely:
wTx+b＝0
where w is an input vector, x is an adaptive weight vector, b is an offset, and the SVM maximizes the geometric boundary
Figure GDA0001615897910000051
Obtaining parameters w and b of the optimal hyperplane, and satisfying:
yi(wTxi+b)≥+1
by introducing lagrangian coefficients, solving the dual problem can be obtained:
Figure GDA0001615897910000052
and simultaneously satisfies:
αi≥0
Figure GDA0001615897910000053
in the dual mode, data points only appear in the form of inner products, and in order to obtain a better representation of the data, these data points can be mapped into the hubert inner product space by a replacement operation:
xi·xj→φ(xi)·φ(xj)＝K(xi，xj)
wherein K (·) is a sum function;
a sum functional representation of the dual problem can be obtained:
Figure GDA0001615897910000054
for any given sum function, the SVM classifier can be expressed as:
F(x)＝sgn(f(x))
where f (x) is the decision function of the SVM classification hyperplane:
Figure GDA0001615897910000055
for a given image sample, the higher | f (x) l, the higher the corresponding confidence in the prediction, and conversely, the lower | f (x) l, the lower the corresponding confidence in the prediction.
Preferably, the S4 sliding window includes the following steps:
s41, selecting an image area, and selecting the image as an area of interest;
s42, scaling the selected region of interest to the size of the feature detection window;
S43, solving HOG characteristics of the images with the same size;
s44, comparing the HOG with the model trained by the SVM, and classifying the action of the region to be detected;
s45, moving the selected image area one step to the right until the sliding window moves to the rightmost end of the image line;
s46, moving the selected image area down by one step until the sliding window moves to the last line of the image;
s47, enlarging the size of the selected image area;
s48, repeating S41 to S47, the region of the current picture with all sizes can be traversed.
Preferably, S5, selecting the action with the highest probability under the action classification, and identifying the action includes the following steps:
s51, traversing all detection images of the sliding window and establishing an image pyramid;
s52, collecting all current target windows containing current action classification, and counting to obtain the highest corresponding window W;
s53, eliminating all windows obviously overlapping with the window W;
s54, moving to the next window with the highest response certainty, and repeating S51-S53 at the current scale:
s55, after the process is completed, the next dimension of the image pyramid is moved, and the process of S51-S55 is repeated.
Preferably, the S6, performing face recognition on the action issuer in S5, includes the following steps:
S61, adopting an LBP feature extraction mode of an equivalent mode, and enabling an LBP coding mode to be equivalent to a hopping problem between 0 and 1;
s62, in the design process of the system, LBP codes with jump times less than 2 in binary codes are uniformly classified into an equivalent mode class, and through the improved mode, the class of the binary codes is largeGreatly reduced, no information loss, and the binary code number is 2pThe number of the sampling pixel points in the neighborhood is reduced to p (p-1) +2, wherein p represents the number of the sampling pixel points in the neighborhood.
The beneficial effects of the invention are: the invention can identify the behaviors and actions of students, is convenient for the management of the students and is beneficial to the improvement of teaching quality.
Drawings
FIG. 1 is a schematic diagram of the compartmentalization of cells according to the invention.
Fig. 2 is a schematic diagram of LBP conversion.
Detailed Description
The invention is further illustrated with reference to the following figures and examples:
referring to fig. 1-2, a teaching assistance big data intelligent analysis method based on feature recognition includes the following steps:
s1, acquiring static images of the students, and acquiring the static images of the students by adopting image acquisition equipment such as a camera and the like;
s2, extracting the directional gradient feature of the image;
S3, machine learning;
s4, sliding a window;
s5, selecting the action with the highest probability under action classification, and identifying the action;
and S6, performing face recognition on the action sender in S5.
Further, S2, extracting the directional gradient feature of the image, includes the following steps:
s21, because the contribution degree of the color to the gradient feature is not large, for the efficiency of the subsequent calculation, converting three channels (RGB) of the color into a corresponding gray image, and regarding the gray image as a three-dimensional image, each pixel point on the image can be represented by (x, y, gray), where x represents the x coordinate of the pixel point in the image, y represents the y coordinate of the pixel point in the image, and gray represents the gray value (0 to 255) of the pixel point, and the conversion formula is as follows:
Gray＝0.3*R+0.59*G+0.11*B；
s22, adjusting the color contrast of the image, reducing the influence of shadow caused by uneven local illumination on the gradient feature, and simultaneously inhibiting the influence of high-frequency noise on the gradient feature of the image, wherein the method mainly used is a Gamma correction method, and the specific calculation formula is as follows:
I(x，y)＝I(x，y)Gamma
if gamma is more than 1, the contrast ratio of the part with darker gray scale can be improved;
s23, calculating the gradient characteristics (including the gradient size and the gradient direction) of each pixel point on the image, extracting the gradient relation among the pixels in the step, and further inhibiting the influence of illumination on the image, wherein the specific calculation formula is as follows:
H(x-1，y-1) H(x，y-1) H(x+1，y-1)
H(x-1，y) H(x，y) H(x+1，y)
H(x-1，y+1) H(x，y+1) H(x+1，y+1)
The table represents the gray values of the corresponding pixels of the gray image obtained in S22, such as: h (x, y) represents that the gradation value at (x, y) of the gradation map is H (x, y); since there are two directions (x direction and y direction) of the grayscale image, the gradient direction needs to calculate the gradient in the two directions respectively and find the gradient size according to the amplitude formula:
Figure GDA0001615897910000091
Figure GDA0001615897910000092
the gradient magnitude and gradient direction of point (x, y) are found according to the magnitude formula:
Figure GDA0001615897910000093
Figure GDA0001615897910000094
；
and S24, dividing the image into a plurality of cell intervals according to pixels, and counting the gray directions of all pixel points in each cell interval.
In the invention, the size of each cell interval is 8 pixels by 8 pixels, 20 degrees in the gradient direction are used as interval division, and the gradient size is used as a weight value in statistics;
assuming that the gradient size G (x, y) of a certain pixel (x, y) in a "cell unit" is 2 and α (x, y) is 137.6 °, the area z6 where the pixel is located should be increased by 2 statistics when counting;
s25, the gradient features extracted from the cell units are normalized, because the illumination intensity in different cell units of the image is possibly affected by the illumination intensity differently, and the gradient changes are sometimes very large, in the invention, the gradient features extracted from a plurality of adjacent cell units are spliced into the gradient features of a larger area, and then the gradient features are normalized by a two-norm normalization method, and the specific principle is as follows:
Suppose v is a feature vector before normalization, | v | | luminance2Stands for specially adaptedThe two-norm of the eigenvector v is solved by the formula:
Figure GDA0001615897910000101
the normalized feature vector is:
Figure GDA0001615897910000102
wherein ε is a small number to prevent the denominator in the above formula from being close to 0;
s26, after the gradient feature vectors in all the intervals composed of the 'cell intervals' are standardized, the gradient feature vectors in all the intervals need to be spliced to form the gradient feature vector corresponding to the whole image, and the gradient feature vector dimension calculation formula of the image is as follows:
Figure GDA0001615897910000103
wherein size is the dimension of the gradient feature of the finally obtained image, blocksize.width and blocksize.height are the width and height corresponding to the interval, cellsize.width and cellsize.height are the width and height corresponding to the 'cell interval', winsize.width and winsize.height are the width and height corresponding to the detected image, blocksize.width and blocksize.height are the width and height corresponding to the interval Step, nbin represents the number of regional channels used in the gradient angle statistics in Step 4;
in the invention, CellSize.width ═ CellSize.height ═ 8 is taken,
BlockSize.width＝BlockSize.height＝16，
BlockStride.width＝BlockStride.height＝8，
WinSize.width＝64，BlockStride.height＝128，
nbin＝9
s3, machine learning, wherein after the gradient features of each sample image are obtained, the gradient features of a class of behaviors of students need to be trained by using a machine learning method, and the classification models correspond to the gradient features;
The training method used by the invention is SVM (Support Vector Machine), which is a supervised statistical learning algorithm and can minimize the experience error and maximize the geometric margin.
The invention relates to a content-based image detection system, which mainly comprises the following steps:
the user marks some related retrieval results as positive samples and some negative samples irrelevant to retrieval, and then the image retrieval system updates the retrieval results based on the feedback samples; the two steps are alternately carried out, so that the computer can gradually learn the requirements of the user, and the retrieval function is enhanced; the method comprises the following specific steps:
dividing the image gradient feature space into a plurality of areas, wherein the positive sample farthest from a specific classification plane is used as the most relevant sample, and the sample closest to the classification plane is used as the sample with the most information content;
the relevance feedback method of the SVM-based image retrieval system comprises the following steps: SVM is an effective two-classification, multi-classification algorithm, and the following takes two classification problems as an example (
Figure GDA0001615897910000111
And y isi{ +1, -1}), a multi-class problem can be analogized;
wherein x isiIs an n-dimensional feature vector, yiIs the class information to which the feature vector belongs,
the SVM divides two types of sample points by a hyperplane, namely:
wTx+b＝0
Where w is an input vector, x is an adaptive weight vector, and b is an offset. SVM by maximizing geometric boundary
Figure GDA0001615897910000112
Obtaining parameters w and b of the optimal hyperplane, and satisfying:
yi(wTxi+b)≥+1
by introducing lagrangian coefficients, solving the dual problem can be obtained:
Figure GDA0001615897910000113
and simultaneously satisfies:
αi≥0
Figure GDA0001615897910000121
in the dual mode, data points only appear in the form of inner products, and in order to obtain a better representation of the data, these data points can be mapped into the hubert inner product space by a replacement operation:
xi·xj→φ(xi)·φ(xj)＝K(xi，xj)
wherein K (·) is a sum function;
a sum functional representation of the dual problem can be obtained:
Figure GDA0001615897910000122
for any given sum function, the SVM classifier can be expressed as:
F(x)＝sgn(f(x))
where f (x) is the decision function of the SVM classification hyperplane:
Figure GDA0001615897910000123
for a given image sample, the higher | f (x) l, the higher the corresponding confidence in the prediction, and conversely, the lower | f (x) l, the lower the corresponding confidence in the prediction.
The invention can detect the behavior of students in a fixed area and can also realize the following functions: a plurality of objects of the same motion in the image are detected, and the positions of the objects of the motion in the current image are determined. To do both, a sliding window method is required. Sliding windows is a technique used for computer vision, which involves the examination of parts to be moved in an image (sliding windows) and the detection of parts using an image pyramid. This is to detect objects at multiple scales.
The sliding window solves the positioning text image by scanning a smaller area of a larger image, and then the scanning is repeated under different scales of the same image. This technique entails decomposing each image into portions, then dropping those portions that are unlikely to contain objects, and classifying the remaining portions.
The core idea of the sliding window is mainly that the size of an H feature detection window is scaled to one region of interest at a time, and then corresponding feature extraction and classification are executed;
s4, sliding the window, including the following steps:
s41, selecting an image area, and selecting the image as an area of interest;
s42, scaling the selected region of interest to the size of the feature detection window;
s43, obtaining HOG characteristics of the image according with the size;
s44, comparing the HOG with the model trained by the SVM, and classifying the action of the region to be detected;
s45, moving the selected image area one step to the right until the sliding window moves to the rightmost end of the image line;
s46, moving the selected image area down by one step until the sliding window moves to the last line of the image;
and S47, enlarging the size of the selected image area.
S48, repeating S41 to S47, the region of the current picture with all sizes can be traversed.
The method of sliding windows may cause a problem of region overlap, which refers to a phenomenon that sliding windows adjacent to each other in the same motion overlap each other when performing motion classification determination on an image. Each window drops several pixels each time, which means that a sliding window can be matched with different positions of the same action of the same student, at this time, a non-maximum inhibition algorithm is needed to be used for screening the overlapped images, and the non-maximum inhibition algorithm can screen a given group of overlapped regions by using the maximum score or the maximum region to screen the region which can represent the action position of the student most.
Non-maximum suppression (NMS) non-maximum suppression as the name implies suppresses elements that are not maxima and searches for local maxima. The local representation is a neighborhood which has two variable parameters, namely the dimension of the neighborhood and the size of the neighborhood. The generic NMS algorithm is not discussed here but is used to extract the highest scoring window in object detection. For example, in pedestrian detection, a sliding window is subjected to feature extraction, and after classification and identification by a classifier, each window is subjected to a score. But sliding windows can result in many windows containing or mostly crossing other windows. The NMS is then used to select the window with the highest score in the neighborhood (which is the highest probability under the action category) and suppress those with low scores.
S5, selecting the action with the highest probability under the action classification, and the specific steps are as follows:
s51, traversing all detection images of the sliding window and establishing an image pyramid;
s52, collecting all current target windows containing current action classification (the certainty factor exceeds a certain threshold value), and counting to obtain the highest corresponding window W;
s53, eliminating all windows which are obviously overlapped with the window W (the area percentage of the overlapped area exceeds a certain threshold);
s54, move to the next window with the highest confidence level of response, and repeat the above process at the current scale.
S55, after the process is completed, the next dimension of the image pyramid is moved, and the process of S51-S55 is repeated.
Preferably, S6, the face recognition, comprises the steps of:
after the actions of the students are judged, the students need to judge the actions by which the actions are made by a face recognition method, a feature extraction and machine learning method is needed by the face detection and recognition technology, and the face features used in the invention are LBP (Local Binary Pattern, Chinese is a Local Binary Pattern) features and an Adaboost training method.
The original LBP characteristic description method and the calculation method comprise the following steps:
For any pixel point (x, y) selected on the image, the gray value of the pixel point is marked as H (x, y), H (x, y) is used as a threshold value to be compared with the gray values of other 8 pixel points in the 3 x 3 area adjacent to the pixel point, and if the gray value of the adjacent pixel is greater than H (x, y), the gray value is marked as 1; if the gray value of the neighboring pixel is less than H (x, y), then it is marked as 0, and finally the binary value of the neighboring pixel is coded as a decimal number, the formula is as follows:
Figure GDA0001615897910000151
wherein, LBP (x)c，yc) Representing a pixel point (x)c，yc) LBP characteristic value of (1)cRepresenting a pixel (x)c，yc) Gray value of (i)pRepresenting a pixel (x)p，yp) Gray value of (x), pixel pointp，yp) At a pixel point (x)c，yc) Is a comparison function, as follows;
Figure GDA0001615897910000152
thus, LBP (x)c，yc) Can reflect the pixel point (x)c，yc) The LBP feature of (a).
Unifonn Pattern LBP features (also called equivalent patterns or uniform patterns), the time and storage space for the computation of raw LBP feature extraction versus computation increases exponentially with the size of the LBP computation region, as: 3 x 3 region twoThe range of the binary code is 0-28The binary coding range of the region 4 x 4 is 0-215The binary coding range of the 32 × 32 region is 0-2 1023. Such a huge binary encoding method is not favorable for feature extraction, storage and subsequent machine learning.
In the invention, an LBP feature extraction mode of an equivalent mode is used, so that the original LBP feature is subjected to dimension reduction, and the LBP feature of the image can not be optimally restored under the condition of reducing the data volume. In order to solve the problem of more binary coding modes, the invention uses an LBP feature extraction mode of an equivalent mode, and comprises the following steps:
the LBP coding scheme is "equivalent" to the hopping problem between 0 and 1, such as: 00000000(0 jump), 11111111(0 jump), 10010111(4 jumps).
In the design process of the system, LBP codes with the hopping times less than 2 in the binary codes are uniformly classified into an equivalent mode class, the types of the binary codes can be greatly reduced through the improved mode, information cannot be lost, and the number of the binary codes is also 2 from the original numberpThe number of the sampling pixel points in the neighborhood is reduced to p (p-1) +2, wherein p represents the number of the sampling pixel points in the neighborhood.
Such as: the uniform pattern bin for 3 x 3 regions is 0-58, the bin for 4 x 4 regions is 0-212, and the bin for 32 x 32 regions is 0-932.
The invention is not described in detail, but is well known to those skilled in the art.
The foregoing detailed description of the preferred embodiments of the invention has been presented. It should be understood that numerous modifications and variations could be devised by those skilled in the art in light of the present teachings without departing from the inventive concepts. Therefore, the technical solutions available to those skilled in the art through logic analysis, reasoning and limited experiments based on the prior art according to the concept of the present invention should be within the scope of protection defined by the claims.

Claims (7)

1. A teaching assistance big data intelligent analysis method based on feature recognition is characterized by comprising the following steps:
s1, obtaining static images of students;
s2, extracting the directional gradient feature of the image;
s3, machine learning;
s4, sliding a window;
s5, selecting the action with the highest probability under action classification, and identifying the action;
s6, carrying out face recognition on the action sender in S5;
wherein, S3, machine learning, including the following step: after the gradient feature of each sample image is obtained in the S2, training the gradient feature of a class-I behavior action of the student by adopting an SVM algorithm, and corresponding to a classification model;
S31, the user marks some relevant retrieval results as positive samples and some negative samples irrelevant to retrieval, and then the image retrieval system updates the retrieval results based on the feedback samples; the two steps are alternately carried out, so that the computer can gradually learn the requirements of the user and the retrieval function is enhanced;
s311, dividing the image gradient feature space into a plurality of regions, wherein a positive sample which is farthest from the classification plane is used as a most relevant sample, and a sample which is closest to the classification plane is used as a sample with the most information content;
s312, an associated feedback method of the SVM-based image retrieval system comprises the following steps: SVM is an effective two-classification, multi-classification algorithm, and the following takes two classification problems as an example (
Figure FDA0003439332310000011
And y isi{ +1, -1}), a multi-class problem can be analogized;
wherein x isiIs an n-dimensional feature vector, yiIs the class information to which the feature vector belongs,
the SVM divides two types of sample points by a hyperplane, namely:
wTx+b＝0
where w is an input vector and where,x is an adaptive weight vector, b is an offset, and the SVM operates by maximizing the geometric boundary
Figure FDA0003439332310000021
Obtaining parameters w and b of the optimal hyperplane, and satisfying:
yi(wTxi+b)≥+1
by introducing lagrangian coefficients, solving the dual problem can be obtained:
Figure FDA0003439332310000022
And simultaneously satisfies:
αi≥0
Figure FDA0003439332310000023
in the dual mode, the data points only appear in the form of inner products, and in order to obtain a better representation of the data, they can be mapped into the hubert inner product space by a replacement operation:
xi·xj→φ(xi)·φ(xj)＝K(xi,xj)
wherein K (-) is a sum function;
a sum functional representation of the dual problem can be obtained:
Figure FDA0003439332310000024
for any given sum function, the SVM classifier can be expressed as:
F(x)＝sgn(f(x))
where f (x) is the decision function of the SVM classification hyperplane:
Figure FDA0003439332310000025
for a given image sample, the higher | f (x) l, the higher the corresponding confidence in the prediction, and conversely, the lower | f (x) l, the lower the corresponding confidence in the prediction.
2. The intelligent analysis method for teaching assistance big data based on feature recognition as claimed in claim 1, wherein the step of extracting the directional gradient feature of the image S2 comprises the following steps:
s21, converting the color image into a corresponding gray image, and regarding the gray image as a three-dimensional image, where each pixel point on the image is represented by (x, y, gray), where x represents an x coordinate of the pixel point in the image, y represents a y coordinate of the pixel point in the image, and gray represents a gray value of the pixel point, and the conversion formula is as follows:
Gray＝0.3*R+0.59*G+0.11*B；
s22, adjusting the color contrast of the image, reducing the influence of shadow caused by uneven local illumination on the gradient characteristics, and adopting a Gamma correction method, wherein the specific calculation formula is as follows:
I(x,y)＝I(x,y)gamma
If gamma is more than 1, the contrast ratio of the part with darker gray scale can be improved;
s23, calculating the gradient feature of each pixel point on the image, wherein the gradient relation between the pixels can be extracted, and the influence of illumination on the image can be further inhibited, wherein: h (x-1, y-1), H (x +1, y-1), H (x-1, y), H (x +1, y), H (x-1, y +1), H (x, y +1) and H (x +1, y +1) respectively represent the gray value of the corresponding pixel of the gray image obtained in S22; since the directions of the grayscale image have an x direction and a y direction, the gradient direction needs to calculate the gradients in the two directions respectively and find the gradient magnitude according to the amplitude formula:
Figure FDA0003439332310000031
Figure FDA0003439332310000041
the gradient magnitude and gradient direction of point (x, y) are found according to the magnitude formula:
Figure FDA0003439332310000042
Figure FDA0003439332310000043
；
s24, dividing the image into a plurality of cell intervals according to pixels, and counting the gray directions of all pixel points in each cell interval;
s25, normalizing the gradient features extracted from the cell units, splicing the gradient features extracted from a plurality of adjacent cell units into a gradient feature of a larger area, and then normalizing the gradient features by using a two-norm normalization method, wherein the method comprises the following specific steps:
Assuming v is the feature vector before normalization, | v | | calvities2Representing the two-norm of the feature vector v, and solving the formula as follows:
Figure FDA0003439332310000044
the normalized feature vectors are:
Figure FDA0003439332310000045
wherein ε is a small number to prevent the denominator in the above formula from being close to 0;
s26, after the gradient feature vectors in all the intervals composed of the 'cell intervals' are standardized, the gradient feature vectors in all the intervals need to be spliced to form the gradient feature vector corresponding to the whole image, and the gradient feature vector dimension calculation formula of the image is as follows:
Figure FDA0003439332310000046
where size is the dimension of the gradient feature of the finally obtained image, block size.width and block size.height are the width and height corresponding to the interval, cell size.width and cell size.height are the width and height corresponding to the "cell interval", winsize.width and winsize.height are the width and height corresponding to the detected image, block size.width and block size.height are the width and height corresponding to the interval step, nbin represents the number of regional channels used in the gradient angle statistics in S24.
3. The intelligent analysis method for teaching assistance big data based on feature recognition as claimed in claim 2, wherein in S24, cell size.width ═ cell size.height ═ 8 is taken;
BlockSize.width＝BlockSize.height＝16；
BlockStride.width＝BlockStride.height＝8
WinSize.width＝64，BlockStride.height＝128，nbin＝9。
4. The intelligent analysis method for teaching assistance big data based on feature recognition according to claim 2, wherein in S26, the size of each "cell interval" is 8 pixels by 8 pixels, each 20 degrees is used as interval division for gradient direction, and the gradient size is used as a weight value in statistics;
assuming that the gradient size G (x, y) of a certain pixel (x, y) in a "cell unit" is 2 and α (x, y) is 137.6 °, the area z6 where the pixel is should be increased by 2 statistics when counting.
5. The intelligent analysis method for teaching assistance big data based on feature recognition as claimed in claim 1, wherein the S4 sliding window comprises the following steps:
s41, selecting an image area, and selecting the image as an area of interest;
s42, scaling the selected region of interest to the size of the feature detection window;
s43, solving HOG characteristics of the images with the same size;
s44, comparing the HOG with the model trained by the SVM, and classifying the action of the region to be detected;
s45, moving the selected image area one step to the right until the sliding window moves to the rightmost end of the image line;
s46, moving the selected image area down by one step until the sliding window moves to the last line of the image;
S47, enlarging the size of the selected image area;
s48, repeating S41 to S47, thereby traversing all the size areas of the current picture.
6. The intelligent analysis method for teaching assistance big data based on feature recognition as claimed in claim 1, wherein S5, the action with the highest probability under the action classification is selected, and the action is recognized, comprising the following steps:
s51, traversing all detection images of the sliding window and establishing an image pyramid;
s52, collecting all current target windows containing current action classification, and counting to obtain a window W with the highest response;
s53, eliminating all windows obviously overlapping with the window W;
s54, moving to the next window with the highest response certainty factor, and repeating S51-S53 at the current scale;
s55, after the process is completed, the next dimension of the image pyramid is moved, and the process of S51-S55 is repeated.
7. The intelligent analysis method for teaching assistance big data based on feature recognition as claimed in claim 1, wherein the step of performing face recognition on the action issuer in the step S5 at S6 comprises the steps of:
s61, adopting an LBP feature extraction mode of an equivalent mode, and enabling an LBP coding mode to be equivalent to a hopping problem between 0 and 1;
S62, in the design process of the system, LBP codes with jump times less than 2 in binary codes are classified into an equivalent mode class, through the improved mode, the class of the binary codes can be greatly reduced, information can not be lost, and the number of the binary codes is also from the original 2pThe number of the sampling pixel points in the neighborhood is reduced to p (p-1) +2, wherein p represents the number of the sampling pixel points in the neighborhood.
CN201810106613.1A 2018-02-02 2018-02-02 Teaching assistance big data intelligent analysis method based on feature recognition Active CN108268854B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810106613.1A CN108268854B (en) 2018-02-02 2018-02-02 Teaching assistance big data intelligent analysis method based on feature recognition

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810106613.1A CN108268854B (en) 2018-02-02 2018-02-02 Teaching assistance big data intelligent analysis method based on feature recognition

Publications (2)

Publication Number Publication Date
CN108268854A CN108268854A (en) 2018-07-10
CN108268854B true CN108268854B (en) 2022-06-10

Family

ID=62773516

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810106613.1A Active CN108268854B (en) 2018-02-02 2018-02-02 Teaching assistance big data intelligent analysis method based on feature recognition

Country Status (1)

Country Link
CN (1) CN108268854B (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110031461B (en) * 2019-02-14 2022-03-18 江苏恒力化纤股份有限公司 Polyester filament yarn dyeing uniformity test method
CN112613342A (en) * 2020-11-27 2021-04-06 深圳市捷视飞通科技股份有限公司 Behavior analysis method and apparatus, computer device, and storage medium
CN116993643B (en) * 2023-09-27 2023-12-12 山东建筑大学 Unmanned aerial vehicle photogrammetry image correction method based on artificial intelligence

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102243687A (en) * 2011-04-22 2011-11-16 安徽寰智信息科技股份有限公司 Physical education teaching auxiliary system based on motion identification technology and implementation method of physical education teaching auxiliary system
US9230159B1 (en) * 2013-12-09 2016-01-05 Google Inc. Action recognition and detection on videos
CN105913025A (en) * 2016-04-12 2016-08-31 湖北工业大学 Deep learning face identification method based on multiple-characteristic fusion
CN107085721A (en) * 2017-06-26 2017-08-22 厦门劢联科技有限公司 A kind of intelligence based on Identification of Images patrols class management system
CN107609517A (en) * 2017-09-15 2018-01-19 华中科技大学 A kind of classroom behavior detecting system based on computer vision

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106462747B (en) * 2014-06-17 2020-03-17 河谷控股Ip有限责任公司 Activity recognition system and method

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102243687A (en) * 2011-04-22 2011-11-16 安徽寰智信息科技股份有限公司 Physical education teaching auxiliary system based on motion identification technology and implementation method of physical education teaching auxiliary system
US9230159B1 (en) * 2013-12-09 2016-01-05 Google Inc. Action recognition and detection on videos
CN105913025A (en) * 2016-04-12 2016-08-31 湖北工业大学 Deep learning face identification method based on multiple-characteristic fusion
CN107085721A (en) * 2017-06-26 2017-08-22 厦门劢联科技有限公司 A kind of intelligence based on Identification of Images patrols class management system
CN107609517A (en) * 2017-09-15 2018-01-19 华中科技大学 A kind of classroom behavior detecting system based on computer vision

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
Pose-based Human Action Recognition with Extreme Gradient Boosting;Vina Ayumi 等;《 2016 IEEE Student Conference on Research and Development (SCOReD)》;20170109;1-5 *
基于视频分析的课堂教学效果自动评价系统;文楷 等;《技术在线》;20160430(第08期);50-52 *
基于视频的人体运动行为分析;王克耀;《万方》;20150925;第1-75页，正文第1.5章，第5.3章，图5-1，第5.3.2章 *

Also Published As

Publication number Publication date
CN108268854A (en) 2018-07-10

Similar Documents

Publication Publication Date Title
CN111401372B (en) Method for extracting and identifying image-text information of scanned document
Masci et al. Steel defect classification with max-pooling convolutional neural networks
Hafemann et al. Forest species recognition using deep convolutional neural networks
Zhang et al. Real-time object classification in video surveillance based on appearance learning
CN103984959B (en) A kind of image classification method based on data and task-driven
CN104680127A (en) Gesture identification method and gesture identification system
CN108268854B (en) Teaching assistance big data intelligent analysis method based on feature recognition
CN112307919B (en) Improved YOLOv 3-based digital information area identification method in document image
CN104182772A (en) Gesture recognition method based on deep learning
CN103049763A (en) Context-constraint-based target identification method
CN105469047A (en) Chinese detection method based on unsupervised learning and deep learning network and system thereof
CN103824090B (en) Adaptive face low-level feature selection method and face attribute recognition method
CN103886342A (en) Hyperspectral image classification method based on spectrums and neighbourhood information dictionary learning
CN105279519A (en) Remote sensing image water body extraction method and system based on cooperative training semi-supervised learning
CN110414616B (en) Remote sensing image dictionary learning and classifying method utilizing spatial relationship
Yadav et al. A robust approach for offline English character recognition
CN111783885A (en) Millimeter wave image quality classification model construction method based on local enhancement
CN111539330A (en) Transformer substation digital display instrument identification method based on double-SVM multi-classifier
Arora et al. A novel approach for handwritten devnagari character recognition
CN117437691A (en) Real-time multi-person abnormal behavior identification method and system based on lightweight network
CN116612478A (en) Off-line handwritten Chinese character scoring method, device and storage medium
CN115880566A (en) Intelligent marking system based on visual analysis
CN112418085B (en) Facial expression recognition method under partial shielding working condition
Ibrahim et al. Braille Image Recognition for Beginners
CN108197663A (en) Based on the calligraphy work image classification method to pairing set Multi-label learning

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant