CN109829898B - Measurement detection system and method based on neural network for Internet detection - Google Patents
Measurement detection system and method based on neural network for Internet detection Download PDFInfo
- Publication number
- CN109829898B CN109829898B CN201910043311.9A CN201910043311A CN109829898B CN 109829898 B CN109829898 B CN 109829898B CN 201910043311 A CN201910043311 A CN 201910043311A CN 109829898 B CN109829898 B CN 109829898B
- Authority
- CN
- China
- Prior art keywords
- data
- model
- layer
- application scene
- test paper
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 34
- 238000000034 method Methods 0.000 title claims abstract description 28
- 238000013528 artificial neural network Methods 0.000 title claims abstract description 22
- 238000005259 measurement Methods 0.000 title claims abstract description 12
- 238000012360 testing method Methods 0.000 claims abstract description 59
- 238000012549 training Methods 0.000 claims abstract description 32
- 238000010276 construction Methods 0.000 claims abstract description 16
- 238000013523 data management Methods 0.000 claims abstract description 14
- 238000012795 verification Methods 0.000 claims abstract description 12
- 238000011176 pooling Methods 0.000 claims description 23
- 230000006870 function Effects 0.000 claims description 20
- 238000002372 labelling Methods 0.000 claims description 19
- 238000011478 gradient descent method Methods 0.000 claims description 10
- 239000011159 matrix material Substances 0.000 claims description 10
- 238000007726 management method Methods 0.000 claims description 9
- 238000000605 extraction Methods 0.000 claims description 8
- 238000005286 illumination Methods 0.000 claims description 8
- 230000008569 process Effects 0.000 claims description 7
- 230000004913 activation Effects 0.000 claims description 5
- 238000004364 calculation method Methods 0.000 claims description 5
- 238000012937 correction Methods 0.000 claims description 4
- 238000007781 pre-processing Methods 0.000 claims description 4
- 238000005070 sampling Methods 0.000 claims description 4
- 230000006835 compression Effects 0.000 claims description 3
- 238000007906 compression Methods 0.000 claims description 3
- 238000000691 measurement method Methods 0.000 claims description 2
- 230000001788 irregular Effects 0.000 abstract description 3
- 238000010801 machine learning Methods 0.000 description 4
- 238000013135 deep learning Methods 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000010923 batch production Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks
Abstract
The invention relates to a measurement detection system and a measurement detection method based on a neural network in Internet detection, wherein a data reading and initializing module reads baseline data, a conventional application scene data set A1 and an unconventional application scene data set A2 from a data management server; the method comprises the steps that a marking unit additionally marks an irregular application scene data set A2, data which is considered to be invalid or unrecognizable by test paper is marked as invalid independently, the rest data are marked as valid by default, and all data in the regular application scene data set A1 are marked as valid uniformly; mixing the data sets A1 and A2 together to form an A3 data set, randomly extracting part of data from the S3 as a training set T3, and taking the rest of data as a verification set V3; the model construction module is used for constructing a prediction model, outputting a result predicted value, judging whether the test paper is effective or not, and conforming to the condition that the test paper does not conform to the conventional application scene. And a plurality of output results are obtained simultaneously, and the results are accurate and quick to measure.
Description
Technical Field
The invention relates to the field of Internet detection, in particular to a measuring system and method based on deep learning in the field of Internet detection.
Background
Along with the rapid development of machine learning technology in the artificial intelligence field, the field of internet detection has also developed rapidly, and a method for obtaining a model by using a traditional machine learning method is mentioned in a measurement system and a method based on machine learning applied to internet detection.
The weakness of this approach is that, in general, a model can only have one output, and if multiple outputs are desired, a model must be built for each output, which is time-consuming and labor-intensive. In internet detection, a situation that multiple results need to be output at the same time is often encountered, for example, the test paper should be judged to be valid or invalid first, and if valid, the detection result of the test paper should be judged again; or the same test paper should output a plurality of different results. These situations are overly cumbersome with traditional machine learning methods.
Disclosure of Invention
The invention provides a measurement detection system based on a neural network, which can be used in internet detection, and comprises a model management server end and a data management server, wherein the model management server end comprises a data reading and initializing module, a model constructing module, a model training module and a model checking module;
the data reading and initializing module reads data from the data management server, and takes the value of the baseline data after the characteristic acquisition algorithm as the true value of a sample of training data, namely a label value; taking data obtained after each piece of data shot in a conventional application scene passes through a characteristic acquisition algorithm as a conventional application scene data set A1, and taking data obtained after data acquired by non-conventional scene application passes through the characteristic acquisition algorithm as a non-conventional application scene data set A2; the system further comprises a labeling unit, a labeling unit and a labeling unit, wherein the labeling unit is used for additionally labeling the non-conventional application scene data set A2, independently labeling the data which are considered to be invalid or unrecognizable by the test paper as invalid, labeling the rest data as valid by default, and uniformly labeling all the data in the conventional application scene data set A1 as valid; mixing the data sets A1 and A2 together to form an A3 data set, randomly extracting part of data from the S3 as a training set T3, and taking the rest of data as a verification set V3;
the model construction module is used for constructing a prediction model according to the baseline data, the conventional application scene data set A1 and the unconventional application scene data set A2; the model can output a result predicted value, test paper effective and ineffective judgment and accords with the condition that the model does not accord with the conventional application scene;
the model training module carries out training iteration by utilizing training set data T3 of the A3 data set until the result predicted value on the verification set V3, test paper effective and ineffective judgment and judgment accuracy conforming to the condition of not conforming to the conventional application are not improved;
and the model checking module compares the predicted value of the model with the standard value of the batch of test paper to obtain a final result.
Further, the baseline data refers to: in the data acquisition, external illumination, mobile phone camera exposure compensation, white balance state, characteristic value of a test paper glazing correction unit, mobile phone posture and the like are carried out in the process of fixed acquisition, test paper picture shooting is carried out, data are acquired for multiple times by each test paper, and characteristic extraction is carried out on the acquired data to calculate a corresponding value as baseline data.
Further, the conventional application scene data set A1 is obtained according to the following method: when the data is acquired, various control condition factors are not required to be fixed, and the shooting can be carried out only by conforming to the definition of the conventional scene application, namely, in the case, the picture can be shot at will, and the acquired data after the acquired data passes through the characteristic acquisition algorithm is the conventional application scene data set A1.
Further, the irregular application scene data set A2 is obtained according to the following method: various control condition factors are fixed during acquisition, so that the control condition factors do not meet the application conditions of the conventional scene, wherein the control condition factors comprise shooting angles, illumination color temperature, brightness, a camera white balance mode and a camera exposure compensation mode, and the obtained data are obtained after the obtained data are subjected to a characteristic acquisition algorithm, namely the data set A2 of the non-conventional application scene.
Further, the model construction module constructs a prediction model by the following algorithm:
first initialize a model
The model uses a square loss objective function J:
wherein θ is a parameter of the neural network; h is a θ Is a model of the entire neural network; x is x (i) Is the ith sample data; y is (i) A tag value for the i-th sample; m is a natural number;
secondly, updating model parameters, wherein the method comprises the following steps:
(1) Firstly, deriving parameters to be updated by adopting a gradient descent method:
wherein j is the number of current iterations; θ j Parameters for the j-th iteration;
is the sample set in the jth iteration;
(2) The parameters are then updated:
wherein ,θ′j Is the updated parameter; lambda is the learning rate;
finally, the model is carried out in a stacking mode:
original data input layer, convolution layer, pooling layer and output layer
Assuming that the first layer is a convolution layer, the first layer+1 is a pooling layer, and the residual error of the pooling layer is:
the residual of the convolution layer is: />
The relational expression of the two is:
the symbol +.here represents the dot product operation of the matrix, i.e. correspondingThe product of the elements is used to determine,
the derivative of the activation function at the j-th node of the layer i is shown. The function unsamply () is an up-sampling process;
assuming that the first layer is a pooling layer, and has N channels, the first layer+1 is a convolution layer, and has M features, the residual calculation formula is as follows:
the symbol ∈ represents the convolution operation of the matrix, kij represents the convolution kernel;
the residual error is sequentially transmitted to the output of the upper layer by combining the formula with a back propagation algorithm, the parameter gradient obtained by deriving the parameter by using the function of the upper layer is optimized by combining a gradient descent method, and the accuracy of the output is further improved.
Further, the output comprises a predicted value corresponding to the test paper picture data; judging whether the test paper picture data are valid or not; judging whether the test paper comes from a conventional application scene; the input layer performs data preprocessing, the convolution layer performs feature extraction, the pooling layer performs feature compression, and the output layer classifies or regresses according to the extracted features.
Further, the model construction can adopt any layer number and layer connection structure.
The invention also provides a measurement and detection method based on the neural network in the internet detection, which comprises the following steps:
initializing a model management server end, and confirming that a data management server is started;
reading and initializing data from a data management server by using a construction data reading and initializing module; taking the value of the baseline data after the characteristic acquisition algorithm as the true value of a sample of training data, namely a label value; taking data obtained after each piece of data shot in the conventional application scene passes through a characteristic acquisition algorithm as a conventional application scene data set A1; the data acquired after the characteristic acquisition algorithm is carried out on the data acquired by the non-conventional scene application is used as a non-conventional application scene data set A2;
the marking unit is used for additionally marking the non-conventional application scene data set A2, independently marking the data which is considered to be invalid or unrecognizable by the test paper as invalid, marking the rest data as valid by default, and uniformly marking all the data in the conventional application scene data set A1 as valid; mixing the data sets A1 and A2 together to form an A3 data set, randomly extracting part of data from the S3 as a training set T3, and taking the rest of data as a verification set V3;
a model construction module is utilized to construct a prediction model based on the baseline data, the conventional application scene data set A1 and the unconventional application scene data set A2, so that the prediction model can output a result predicted value, test paper valid and invalid judgment and accords with the condition that the conventional application scene is not met;
training iteration is carried out by utilizing the training set data T3 of the A3 data set by utilizing the model training module until the result predicted value on the verification set V3, test paper effective and ineffective judgment and judgment accuracy conforming to the condition of being not conforming to the conventional application scene are not improved;
and comparing the predicted value of the prediction model with the standard value of the batch of test paper by using a model checking module to obtain a final result.
Further, the model construction module constructs a prediction model by the following algorithm:
first initialize a model
The model uses a square loss objective function J:
wherein θ is a parameter of the neural network; h is a θ Is a model of the entire neural network;x (i) is the ith sample data; y is (i) A tag value for the i-th sample; m is a natural number;
secondly, updating model parameters, wherein the method comprises the following steps:
(1) Firstly, deriving parameters to be updated by adopting a gradient descent method:
wherein j is the number of current iterations; θ j Parameters for the j-th iteration;
is the sample set in the jth iteration;
(2) The parameters are then updated:
wherein ,θ′j Is the updated parameter; lambda is the learning rate;
finally, the model is carried out in a stacking mode:
original data input layer, convolution layer, pooling layer and output layer
Assuming that the first layer is a convolution layer, the first layer+1 is a pooling layer, and the residual error of the pooling layer is:
the residual of the convolution layer is: />
The relational expression of the two is:
the symbol +.here represents the dot product operation of the matrix, i.e. the pairThe product of the elements should be taken,
the derivative of the activation function at the j-th node of the layer i is shown. The function unsamply () is an up-sampling process;
assuming that the first layer is a pooling layer, and has N channels, the first layer+1 is a convolution layer, and has M features, the residual calculation formula is as follows:
the symbol ∈ represents the convolution operation of the matrix, kij represents the convolution kernel;
the residual error is sequentially transmitted to the output of the upper layer by combining the formula with a back propagation algorithm, the parameter gradient obtained by deriving the parameter by using the function of the upper layer is optimized by combining a gradient descent method, and the accuracy of the output is further improved.
The invention predicts the reaction characteristics of the test paper by using a deep learning mode, and simultaneously obtains a plurality of output results, and the results are accurate and quick to measure.
Drawings
The construction, principles, operational features and advantages of the present invention may be better understood with reference to the following description when considered in connection with the accompanying drawings, which are set forth herein to provide a further explanation of the invention, and the accompanying drawings are only for the purpose of illustrating the invention and are not to be construed as unduly limiting the invention.
FIG. 1 is a schematic diagram of the detection system of the present invention;
FIG. 2 is a flow chart of the detection method of the present invention.
Detailed Description
The invention discloses a measurement detection system based on a neural network in internet detection, which comprises a model management server and a data management server, wherein the model management server comprises a data reading and initializing module, a model constructing module, a model training module and a model checking module;
the data reading and initializing module is mainly used for reading and initializing data from the data management system and comprises conventional data processing operations such as preprocessing, normalization and the like on the data. Model construction module: the method is mainly used for constructing the parameters of the model and configuring the model. Model training module: the method is mainly used for training the model and performing basic verification. Model checking module: the method is mainly used for testing the prediction capability of the model.
Initializing a model management server side, and confirming that the data management server side is started. The data reading and initializing module reads data from the data management server, takes the value of the baseline data after the characteristic acquisition algorithm as the true value, namely the label value, of the sample of the training data, takes each piece of data of the conventional application scene as the characteristic of the sample, and performs the same operation on each piece of data of the non-conventional application scene to obtain a conventional application scene data set A1 and a non-conventional application scene data set A2.
The baseline data refers to: in the data acquisition, external illumination, mobile phone camera exposure compensation, white balance state, characteristic values of a test paper glazing correction unit, mobile phone gestures and the like are carried out in the process of fixed acquisition, each test paper acquires, for example, 5-10 times of data, a plurality of test papers are acquired, characteristic extraction is carried out on the acquired data to calculate a corresponding value as baseline data, and at the moment, the average value of the acquired data can be taken as the baseline data. The features may be RGB, HARR, HOG, SIFT, LBP, etc. values of the picture. The feature extraction method can be performed by using the prior art. The value of the baseline data is the true value of the sample of training data, i.e., the label value. I.e. each test paper, eventually corresponds to a label value.
Each piece of data shot in the conventional application scene is taken as a characteristic of a sample. Each piece of data shot in the conventional application scene is obtained according to the following method: for the same test paper, various control condition factors are not required to be fixed during data acquisition, and shooting can be carried out only by enabling all factors to meet the definition of conventional scene application, namely in the case, pictures can be shot randomly, each test paper is acquired for a plurality of times, obtained data are subjected to a characteristic acquisition algorithm, for example, data finally obtained by an average value acquisition algorithm are carried out and serve as shot data in a conventional application scene, and finally each test paper corresponds to the obtained conventional application scene data. And collecting a plurality of test papers to obtain a conventional application scene data set A1.
And (3) acquiring non-conventional scene application data, wherein various control condition factors are fixed during acquisition so as not to meet conventional scene application conditions, and the control condition factors comprise shooting angles, illumination color temperatures, brightness, a camera white balance mode, a camera exposure compensation mode and the like. And (3) acquiring the acquired data after the acquired data passes through a characteristic acquisition algorithm, wherein each test paper is acquired for a plurality of times, and acquiring the acquired data after the acquired data passes through the characteristic acquisition algorithm, for example, taking the data finally acquired by an average algorithm as the photographed data in an unconventional application scene. And collecting a plurality of test papers to obtain an unconventional application scene data set A2.
S3, the system further comprises a labeling unit for additionally labeling the data set A2 of the irregular application scene, wherein the labeling unit can label the data which can be considered invalid or unrecognizable by the test paper as invalid by manual operation, and the rest data are labeled as valid by default. All data in the conventional application scene data set A1 are uniformly marked as valid.
S4, mixing the A1 data set and the A2 data set together to form an A3 data set. 75% of the data are randomly extracted from S3 as training set T3, and the remaining 25% of the data are used as verification set V3.
The model construction module adopts the following steps to construct the model.
S5, initializing a model, wherein the model adopts a square loss objective function J:
wherein θ is a parameter of the neural network; h is a θ Is a model of the entire neural network; x is x (i) Is the ith sample data; y is (i) A tag value for the i-th sample; m is a natural number.
The method comprises the steps of adopting a gradient descent method for model parameter updating, and firstly deriving parameters to be updated:
wherein j is the number of current iterations; θ j Parameters for the j-th iteration;
is the sample set in the jth iteration;
the parameters are then updated:
wherein ,θ′j Is the updated parameter; lambda is the learning rate.
The model is carried out in a stacking mode:
original data input layer- & gt convolution layer- & gt pooling layer- & gt output layer (output 1: predicted value corresponding to test paper picture data, output 2: test paper picture data valid and invalid, and output 3: whether test paper comes from a conventional application scene or not). The input layer performs data preprocessing, the convolution layer performs feature extraction, the pooling layer performs feature compression, and the output layer classifies or regresses according to the extracted features.
Assuming that the first layer is a convolution layer, the first layer+1 is a pooling layer, and the residual error of the pooling layer is:
the residual of the convolution layer is: />
The relational expression of the two is:
here the symbol +.is denoted the dot product operation of the matrix, i.e. the product of the corresponding elements,
the derivative of the activation function at the j-th node of the layer i is shown. The function unscample () therein is an upsampling process.
Assuming that the first layer is a pooling layer, and has N channels, the first layer+1 is a convolution layer, and has M features, the residual calculation formula is as follows:
the symbol ∈ represents the convolution operation of the matrix, kij represents the convolution kernel.
The residual error is sequentially transmitted to the output of the upper layer by combining the formula with a back propagation algorithm, the parameter gradient obtained by deriving the parameter by using the function of the upper layer is optimized by combining a gradient descent method, and the accuracy of the output is further improved.
The model training module carries out training iteration by utilizing the A3 training set data T3 until the result predicted value on the verification set V3, the test paper is effective and invalid to judge, and the judgment accuracy which accords with the condition that the test paper does not accord with the conventional application scene is not improved.
It is essential that the model described in the present invention can take any number of layers and layer connection structure.
For any picture or data for detection, three types of results including predicted values of the results, effective and invalid test paper and accordance with non-conventional application scenes can be simultaneously output by using the model.
For any pair of batch production test paper, firstly judging whether the test paper is effective or not, and if so, reading the collected standard value from the data management server.
And the model checking module compares the predicted value of the model with the standard value of the batch of test paper to obtain a final result.
The judgment that the conventional application scene is not met can be returned as prompt information. The results obtained in conventional application scenarios are generally more accurate.
Although the present invention has been described in detail with reference to the embodiments, it should be understood by those skilled in the art that the present invention is not limited to the particular embodiments, but various modifications, changes and substitutions can be made without departing from the spirit and spirit of the present application, and all changes and substitutions fall within the scope of the present application.
Claims (6)
1. The measurement detection system based on the neural network for the Internet detection comprises a model management server and a data management server, wherein the model management server comprises a data reading and initializing module, a model building module, a model training module and a model checking module;
baseline data refers to: in the data acquisition, external illumination, mobile phone camera exposure compensation, white balance state, characteristic value of a test paper glazing correction unit and mobile phone posture are carried out test paper picture shooting during fixed acquisition, each test paper acquires multiple times of data, and characteristic extraction is carried out on the acquired data to calculate a corresponding value as baseline data;
the conventional application scenario data set A1 is obtained according to the following method: when the data is acquired, various control condition factors are not required to be fixed, and shooting can be carried out only by enabling the factors to meet the definition of the conventional scene application, namely, in the case, pictures can be shot at will, and the acquired data after the acquired data are subjected to the characteristic acquisition algorithm is the conventional application scene data set A1;
the non-conventional application scenario data set A2 is obtained according to the following method: fixing various control condition factors during acquisition to ensure that the control condition factors do not meet the application conditions of the conventional scene, wherein the control condition factors comprise shooting angles, illumination color temperatures, brightness, a camera white balance mode and a camera exposure compensation mode, and the acquired data after the acquired data are subjected to a characteristic acquisition algorithm is the data set A2 of the non-conventional application scene;
the data reading and initializing module reads data from the data management server, and takes the value of the baseline data after the characteristic acquisition algorithm as the true value of a sample of training data, namely a label value; taking data obtained after each piece of data shot in a conventional application scene passes through a characteristic acquisition algorithm as a conventional application scene data set A1, and taking data obtained after data acquired by non-conventional scene application passes through the characteristic acquisition algorithm as a non-conventional application scene data set A2; the system further comprises a labeling unit, a labeling unit and a labeling unit, wherein the labeling unit is used for additionally labeling the non-conventional application scene data set A2, independently labeling the data which are considered to be invalid or unrecognizable by the test paper as invalid, labeling the rest data as valid by default, and uniformly labeling all the data in the conventional application scene data set A1 as valid; mixing the data sets A1 and A2 together to form an A3 data set, randomly extracting part of data from the S3 as a training set T3, and taking the rest of data as a verification set V3;
the model construction module is used for constructing a prediction model according to the baseline data, the conventional application scene data set A1 and the unconventional application scene data set A2; the model can output a result predicted value, test paper effective and ineffective judgment and accords with the condition that the model does not accord with the conventional application scene;
the model training module carries out training iteration by utilizing training set data T3 of the A3 data set until the result predicted value on the verification set V3, test paper effective and ineffective judgment and judgment accuracy conforming to the condition of not conforming to the conventional application are not improved;
and the model checking module compares the predicted value of the model with the standard value of the batch of test paper to obtain a final result.
2. The neural network-based measurement and detection system for use in internet detection according to claim 1, wherein the model construction module constructs the predictive model by the following algorithm:
first initialize a model
The model uses a square loss objective function J:
wherein θ is a parameter of the neural network; h is a θ Is a model of the entire neural network; x is x (i) Is the ith sample data; y is (i) A tag value for the i-th sample; m is a natural number;
secondly, updating model parameters, wherein the method comprises the following steps:
(1) Firstly, deriving parameters to be updated by adopting a gradient descent method:
wherein j is the number of current iterations; θ j Parameters for the j-th iteration;
is the sample set in the jth iteration;
(2) The parameters are then updated:
wherein ,θ′j Is the updated parameter; lambda is the learning rate;
finally, the model is carried out in a stacking mode:
original data input layer, convolution layer, pooling layer and output layer
Assuming that the first layer is a convolution layer, the first +1 layer is a pooling layer, and the residual error of the pooling layer is:
the residual of the convolution layer is: />
The relational expression of the two is:
here the symbol +.is denoted the dot product operation of the matrix, i.e. the product of the corresponding elements,
the derivative of the activation function at the j-th node of the first layer is shown, wherein the function unlock () is an up-sampling process;
assuming that the first layer is a pooling layer, and has N channels, the first layer+1 is a convolution layer, and has M features, the residual calculation formula is as follows:
the symbol ∈ represents the convolution operation of the matrix, kij represents the convolution kernel;
the residual error is sequentially transmitted to the output of the upper layer by combining the formula with a back propagation algorithm, the parameter gradient obtained by deriving the parameter by using the function of the upper layer is optimized by combining a gradient descent method, and the accuracy of the output is further improved.
3. The neural network-based measurement and detection system for internet detection according to claim 2, wherein the output includes a predicted value corresponding to test paper picture data; judging whether the test paper picture data are valid or not; judging whether the test paper comes from a conventional application scene; the input layer performs data preprocessing, the convolution layer performs feature extraction, the pooling layer performs feature compression, and the output layer classifies or regresses according to the extracted features.
4. The neural network-based measurement and detection system for use in internet detection according to claim 2, wherein the model construction adopts an arbitrary number of layers and layer connection structure.
5. The measurement detection method based on the neural network for the Internet detection comprises the following steps:
initializing a model management server end, and confirming that a data management server is started;
baseline data refers to: in the data acquisition, external illumination, mobile phone camera exposure compensation, white balance state, characteristic value of a test paper glazing correction unit and mobile phone posture are carried out test paper picture shooting during fixed acquisition, each test paper acquires multiple times of data, and characteristic extraction is carried out on the acquired data to calculate a corresponding value as baseline data;
the conventional application scenario data set A1 is obtained according to the following method: when the data is acquired, various control condition factors are not required to be fixed, and shooting can be carried out only by enabling the factors to meet the definition of the conventional scene application, namely, in the case, pictures can be shot at will, and the acquired data after the acquired data are subjected to the characteristic acquisition algorithm is the conventional application scene data set A1;
the non-conventional application scenario data set A2 is obtained according to the following method: fixing various control condition factors during acquisition to ensure that the control condition factors do not meet the application conditions of the conventional scene, wherein the control condition factors comprise shooting angles, illumination color temperatures, brightness, a camera white balance mode and a camera exposure compensation mode, and the acquired data after the acquired data are subjected to a characteristic acquisition algorithm is the data set A2 of the non-conventional application scene;
reading and initializing data from a data management server by using a construction data reading and initializing module; taking the value of the baseline data after the characteristic acquisition algorithm as the true value of a sample of training data, namely a label value; taking data obtained after each piece of data shot in the conventional application scene passes through a characteristic acquisition algorithm as a conventional application scene data set A1; the data acquired after the characteristic acquisition algorithm is carried out on the data acquired by the non-conventional scene application is used as a non-conventional application scene data set A2;
the marking unit is used for additionally marking the non-conventional application scene data set A2, independently marking the data which is considered to be invalid or unrecognizable by the test paper as invalid, marking the rest data as valid by default, and uniformly marking all the data in the conventional application scene data set A1 as valid; mixing the data sets A1 and A2 together to form an A3 data set, randomly extracting part of data from the S3 as a training set T3, and taking the rest of data as a verification set V3;
a model construction module is utilized to construct a prediction model based on the baseline data, the conventional application scene data set A1 and the unconventional application scene data set A2, so that the prediction model can output a result predicted value, test paper valid and invalid judgment and accords with the condition that the conventional application scene is not met;
training iteration is carried out by utilizing the training set data T3 of the A3 data set by utilizing the model training module until the result predicted value on the verification set V3, test paper effective and ineffective judgment and judgment accuracy conforming to the condition of being not conforming to the conventional application scene are not improved;
and comparing the predicted value of the prediction model with the standard value of the batch of test paper by using a model checking module to obtain a final result.
6. The neural network-based measurement and detection method for use in internet detection according to claim 5, wherein the model construction module constructs the predictive model by the following algorithm:
first initialize a model
The model uses a square loss objective function J:
wherein θ is a parameter of the neural network; h is a θ Is a model of the entire neural network; x is x (i) Is the ith sample data; y is (i) A tag value for the i-th sample; m is a natural number;
secondly, updating model parameters, wherein the method comprises the following steps:
(1) Firstly, deriving parameters to be updated by adopting a gradient descent method:
wherein j is the number of current iterations; θ j Parameters for the j-th iteration;
is the sample set in the jth iteration;
(2) The parameters are then updated:
wherein ,
is the updated parameter; lambda is the learning rate;
finally, the model is carried out in a stacking mode:
original data input layer, convolution layer, pooling layer and output layer
Assuming that the first layer is a convolution layer, the first +1 layer is a pooling layer, and the residual error of the pooling layer is:
the residual of the convolution layer is: />
The relational expression of the two is:
here the symbol +.is denoted the dot product operation of the matrix, i.e. the product of the corresponding elements,
the derivative of the activation function at the j-th node of the first layer is shown, wherein the function unlock () is an up-sampling process;
assuming that the first layer is a pooling layer, and has N channels, the first layer+1 is a convolution layer, and has M features, the residual calculation formula is as follows:
the symbol ∈ represents the convolution operation of the matrix, kij represents the convolution kernel;
the residual error is sequentially transmitted to the output of the upper layer by combining the formula with a back propagation algorithm, the parameter gradient obtained by deriving the parameter by using the function of the upper layer is optimized by combining a gradient descent method, and the accuracy of the output is further improved.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910043311.9A CN109829898B (en) | 2019-01-17 | 2019-01-17 | Measurement detection system and method based on neural network for Internet detection |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910043311.9A CN109829898B (en) | 2019-01-17 | 2019-01-17 | Measurement detection system and method based on neural network for Internet detection |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109829898A CN109829898A (en) | 2019-05-31 |
| CN109829898B true CN109829898B (en) | 2023-05-16 |
Family
ID=66860418
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910043311.9A Active CN109829898B (en) | 2019-01-17 | 2019-01-17 | Measurement detection system and method based on neural network for Internet detection |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109829898B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104794468A (en) * | 2015-05-20 | 2015-07-22 | 成都通甲优博科技有限责任公司 | Human face detection and tracking method based on unmanned aerial vehicle mobile platform |
| CN104820998A (en) * | 2015-05-27 | 2015-08-05 | 成都通甲优博科技有限责任公司 | Human body detection and tracking method and device based on unmanned aerial vehicle mobile platform |
| CN105930277A (en) * | 2016-07-11 | 2016-09-07 | 南京大学 | Defect source code locating method based on defect report analysis |
| CN107729838A (en) * | 2017-10-12 | 2018-02-23 | 中科视拓（北京）科技有限公司 | A kind of head pose evaluation method based on deep learning |
| CN108009525A (en) * | 2017-12-25 | 2018-05-08 | 北京航空航天大学 | A kind of specific objective recognition methods over the ground of the unmanned plane based on convolutional neural networks |
| CN108470586A (en) * | 2018-03-15 | 2018-08-31 | 柳州康云互联科技有限公司 | A kind of health management system arranged and method with universality and high compliance |
| CN108597604A (en) * | 2018-05-11 | 2018-09-28 | 广西大学 | A kind of dyschromicum skin disease systematicalian system based on cloud database |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104293646A (en) * | 2009-08-22 | 2015-01-21 | 里兰斯坦福初级大学理事会 | Imaging and evaluating embryos, oocytes, and stem cells |
| WO2012116185A1 (en) * | 2011-02-23 | 2012-08-30 | The Board Of Trustees Of The Leland Stanford Junior University | Methods of detecting aneuploidy in human embryos |
-
2019
- 2019-01-17 CN CN201910043311.9A patent/CN109829898B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104794468A (en) * | 2015-05-20 | 2015-07-22 | 成都通甲优博科技有限责任公司 | Human face detection and tracking method based on unmanned aerial vehicle mobile platform |
| CN104820998A (en) * | 2015-05-27 | 2015-08-05 | 成都通甲优博科技有限责任公司 | Human body detection and tracking method and device based on unmanned aerial vehicle mobile platform |
| CN105930277A (en) * | 2016-07-11 | 2016-09-07 | 南京大学 | Defect source code locating method based on defect report analysis |
| CN107729838A (en) * | 2017-10-12 | 2018-02-23 | 中科视拓（北京）科技有限公司 | A kind of head pose evaluation method based on deep learning |
| CN108009525A (en) * | 2017-12-25 | 2018-05-08 | 北京航空航天大学 | A kind of specific objective recognition methods over the ground of the unmanned plane based on convolutional neural networks |
| CN108470586A (en) * | 2018-03-15 | 2018-08-31 | 柳州康云互联科技有限公司 | A kind of health management system arranged and method with universality and high compliance |
| CN108597604A (en) * | 2018-05-11 | 2018-09-28 | 广西大学 | A kind of dyschromicum skin disease systematicalian system based on cloud database |
Non-Patent Citations (4)
| Title |
|---|
| JPEG基线算法对BP神经网络图像压缩处理的影响;杨光临等;《工程图学学报》;20051208;第26卷(第6期);181-186 * |
| 基于图像处理的金免疫层析试条定量测试系统;夏成锋等;《仲恺农业工程学院学报》;20091231;第22卷(第3期);29-32 * |
| 基于机器视觉的稻田灌溉用水铜离子浓度检测;曹晓曼等;《广东农业科学》;20150225(第04期);153-158 * |
| 自适应增强卷积神经网络图像识别;刘万军等;《中国图象图形学报》;20171216(第12期);105-118 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109829898A (en) | 2019-05-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2021052159A1 (en) | Adversarial transfer learning-based face beauty prediction method and device | |
| CN108416326B (en) | Face recognition method and device | |
| JP2021532434A (en) | Face feature extraction model Training method, face feature extraction method, device, equipment and storage medium | |
| CN108734283A (en) | Nerve network system | |
| CN109033107A (en) | Image search method and device, computer equipment and storage medium | |
| CN111008643B (en) | Picture classification method and device based on semi-supervised learning and computer equipment | |
| CN110009614A (en) | Method and apparatus for output information | |
| CN111368672A (en) | Construction method and device for genetic disease facial recognition model | |
| CN111797672B (en) | Object recognition system and object recognition method | |
| CN110738102A (en) | face recognition method and system | |
| CN111832581B (en) | Lung feature recognition method and device, computer equipment and storage medium | |
| CN112818821B (en) | Human face acquisition source detection method and device based on visible light and infrared light | |
| CN110532746B (en) | Face checking method, device, server and readable storage medium | |
| CN110909657A (en) | Method for identifying apparent tunnel disease image | |
| CN113095370A (en) | Image recognition method and device, electronic equipment and storage medium | |
| CN109886923B (en) | Measurement detection system and method based on machine learning for Internet detection | |
| CN113221695B (en) | Method for training skin color recognition model, method for recognizing skin color and related device | |
| CN108549899B (en) | Image identification method and device | |
| CN112651333B (en) | Silence living body detection method, silence living body detection device, terminal equipment and storage medium | |
| CN111968087B (en) | Plant disease area detection method | |
| CN106960188B (en) | Weather image classification method and device | |
| CN109829898B (en) | Measurement detection system and method based on neural network for Internet detection | |
| WO2018137226A1 (en) | Fingerprint extraction method and device | |
| CN112614570A (en) | Sample set labeling method, pathological image classification method and classification model construction method and device | |
| CN112559791A (en) | Cloth classification retrieval method based on deep learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |