CN108229690A - A kind of method and apparatus of machine learning model effect assessment - Google Patents
A kind of method and apparatus of machine learning model effect assessment Download PDFInfo
- Publication number
- CN108229690A CN108229690A CN201810066061.6A CN201810066061A CN108229690A CN 108229690 A CN108229690 A CN 108229690A CN 201810066061 A CN201810066061 A CN 201810066061A CN 108229690 A CN108229690 A CN 108229690A
- Authority
- CN
- China
- Prior art keywords
- shock resistance
- target
- machine learning
- psychological
- monitoring data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Medical Informatics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Artificial Intelligence (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The present invention provides a kind of method and apparatus of machine learning model effect assessment, are related to the technical field of data mining, including：Psychological shock resistance monitoring data are obtained, obtain target data set；The psychological shock resistance monitoring data sample structure target study collection and target detection collection concentrated based on the target data；Target study collection and the target detection collection are separately input in a variety of machine learning classification models, to determine the test accuracy rate of each machine learning classification model according to result of calculation.The present invention, which is alleviated, assesses psychological impact resistance due to selecting the preferable machine learning model of effectiveness not from a variety of machine moulds in the prior art, leads to the technical issues of poor to psychological impact resistance prediction effect occur.
Description
Technical field
The present invention relates to the technical field of data mining, more particularly, to a kind of method of machine learning model effect assessment
And device.
Background technology
Currently, since detection object is inconsistent to critical incident impact resistance, object is detected due to psychological anti-impact in part
It is weaker and accident is caused to happen occasionally to hit ability, for example, student, employee are because pressure or setback are excessive and select to commit suicide or to learning
School society makes reprisals.Then the psychological impact resistance of crowd is analyzed, has found that it is likely that make abnormal behaviour in time
Detection object, excavate potential risk, and it is dredged in time just extremely necessary.
In existing technical solution, by quantifying psychological characteristics, model is established using machine learning algorithm, can be predicted
Different mental feature detects impact resistance of the object to a variety of critical incidents, so as to find potential risks.However, existing
In a variety of machine learning algorithms having, any model can preferably predict this kind of data it is preferably accurate to possess
Rate and Generalization Capability, there is presently no similar conclusion or research method and systems.
In view of the above-mentioned problems, effective solution is not proposed also.
Invention content
In view of this, the purpose of the present invention is to provide a kind of method and apparatus of machine learning model effect assessment, with
It alleviates anti-to psychology due to the preferable machine learning model of effectiveness being selected not from a variety of machine moulds in the prior art
Impact capacity is assessed, and leads to the technical issues of poor to psychological impact resistance prediction effect occur.
In a first aspect, an embodiment of the present invention provides a kind of method of machine learning model effect assessment, this method includes
Psychological shock resistance monitoring data are obtained, obtain target data set, wherein, the target data concentration includes multiple detection objects
Psychological shock resistance monitoring data sample, the psychological shock resistance monitoring data sample of each detection object include multiple psychology and examine
The test result of test question and the objective impact-level of multiple events, the objective impact-level of event are intrinsic for the event of reacting
To detect object psychological impact data；The psychological shock resistance monitoring data sample structure concentrated based on the target data
Target study collection and target detection collection；Target study collection and the target detection collection are separately input to a variety of machine learning
In disaggregated model, to determine the test accuracy rate of each machine learning classification model according to result of calculation.
Further, based on the target data concentrate psychological shock resistance monitoring data sample structure target study collection and
Target detection collection includes：It concentrates each psychological shock resistance monitoring data sample that processing is reconstructed to the target data, obtains
Reconstruction result, wherein, the reconstruction result of each psychology shock resistance monitoring data sample includes the first variable and the second variable, institute
It is to be determined based on the event impact-level in the psychological shock resistance monitoring data sample and psychological characteristics variable to state the first variable
, second variable is to be determined based on the shock resistance level to critical incident in the psychological shock resistance monitoring data sample
's；The target data set after reconstruct is cut according to preset ratio, obtains the target study collection and the mesh
Mark test set.
Further, the test result includes multiple sub- test results, and each psychology is concentrated to the target data
Processing is reconstructed in shock resistance monitoring data sample, obtains reconstruction result and includes：From each psychological shock resistance monitoring data
In test result corresponding to sample, first group of sub- test result of extraction is horizontal as the shock resistance to critical incident；It will be each
Other in test result corresponding to the psychology shock resistance monitoring data sample in addition to first group of sub- test result
Sub- test result is converted to multiple psychological characteristics；Institute is determined based on the objective impact-level of the event and the multiple psychological characteristics
It states the first variable and second variable is determined based on the shock resistance level to critical incident.
Further, a variety of machine learning classification models, and the target learning sample that target study is concentrated are called
It is input in each machine learning classification model and carries out learning training, wherein, described first in the target learning sample becomes
Measure the input for machine learning classification model, second variable in the target learning sample is machine learning classification model
Output；The target detection sample that the target detection is concentrated is input in each machine learning classification model, calculates each
The test accuracy rate of machine learning classification model.
Further, the method further includes：The test accuracy rate of each machine learning classification model is compared, will be surveyed
Try the highest machine learning classification model of accuracy rate machine learning classification model as a purpose.
Further, the target detection sample that the target detection is concentrated is input to each machine learning classification model
In, the test accuracy rate for calculating each machine learning classification model includes：For machine learning classification model each described, it is based on
The first variable that the target detection concentration includes calculates the psychology of each detection object that the target detection collection includes
The psychological shock resistance of shock resistance monitoring data sample is horizontal, and it is horizontal to obtain first group of psychology shock resistance；By the target detection collection
In the second variable for including it is horizontal as second group of psychology shock resistance；Based on first group of psychology shock resistance level and described
The accuracy rate of two groups of psychology shock resistance level calculation each machine learning classification models.
Second aspect, an embodiment of the present invention provides a kind of device of machine learning model effect assessment, which includes：
Acquisition device obtains psychological shock resistance monitoring data, obtains target data set, wherein, the target data concentration includes multiple
The psychological shock resistance monitoring data sample of object is detected, the psychological shock resistance monitoring data sample of each detection object includes
Multiple psychology detection test results of examination question and the objective impact-level of multiple events, the objective impact-level of event are for anti-
The data for the psychological impact to detecting object for answering event intrinsic；Construction device, the construction device are used for based on the target
Psychological shock resistance monitoring data sample structure target study collection and target detection collection in data set；Apparatus for evaluating, the assessment
Device is used to target study collection and the target detection collection being separately input in a variety of machine learning classification models, with root
The test accuracy rate of each machine learning classification model is determined according to result of calculation.
Further, the construction device is additionally operable to：Each psychological shock resistance monitoring data are concentrated to the target data
Processing is reconstructed in sample, obtains reconstruction result, wherein, the reconstruction result of each psychology shock resistance monitoring data sample includes
First variable and the second variable, first variable are based on the objective punching of event in the psychological shock resistance monitoring data sample
Strike waters what gentle psychological characteristics variable determined, second variable is based on pair in the psychological shock resistance monitoring data sample
What the shock resistance level of critical incident determined；The target data set after reconstruct is cut according to preset ratio, is obtained
To target study collection and the target detection collection.
Further, the construction device is additionally operable to：From corresponding to each psychological shock resistance monitoring data sample
In test result, first group of sub- test result of extraction is horizontal as the shock resistance to critical incident, described that critical incident is resisted
Impact-level is the data of the psychological shock resistance level for reaction detection object；It will each psychological shock resistance monitoring data
Other sub- test results in test result corresponding to sample in addition to first group of sub- test result are converted to multiple hearts
Manage feature；First variable is determined and based on institute based on the objective impact-level of the event and the multiple psychological characteristics
It states the shock resistance level to critical incident and determines second variable.
Further, the apparatus for evaluating is additionally operable to：A variety of machine learning classification models are called, and the target is learnt
The target learning sample of concentration is input in each machine learning classification model and carries out learning training, wherein, the target study
First variable in sample is the input of machine learning classification model, second variable in the target learning sample
Output for machine learning classification model；The target detection sample that the target detection is concentrated is input to each machine learning point
In class model, the test accuracy rate of each machine learning classification model is calculated.
In embodiments of the present invention, first, psychological shock resistance monitoring data are obtained, obtain target data set；Then, it is based on
The psychological shock resistance monitoring data sample structure target study collection and target detection collection that the target data is concentrated；Finally, by mesh
Mark study collection and the target detection collection are separately input in a variety of machine learning classification models, to be determined often according to result of calculation
The test accuracy rate of kind machine learning classification model.In embodiments of the present invention, by determining a variety of machine learning classification models
In the test accuracy rate of each machine learning classification model select the corresponding machine learning classification model to carry out psychological shock resistance
The assessment of ability, so as to alleviate due to selecting the preferable machine learning of effectiveness not from a variety of machine moulds in the prior art
Model assesses psychological impact resistance, leads to the technology poor to psychological impact resistance prediction effect occur to ask
Topic has reached the technology effect for the best machine learning model of effectiveness that psychological impact resistance is assessed in a variety of machine moulds of acquisition
Fruit.
Other features and advantages of the present invention will illustrate in the following description, also, partly become from specification
It obtains it is clear that being understood by implementing the present invention.The purpose of the present invention and other advantages are in specification, claims
And specifically noted structure is realized and is obtained in attached drawing.
For the above objects, features and advantages of the present invention is enable to be clearer and more comprehensible, preferred embodiment cited below particularly, and coordinate
Appended attached drawing, is described in detail below.
Description of the drawings
It, below will be to specific in order to illustrate more clearly of the specific embodiment of the invention or technical solution of the prior art
Embodiment or attached drawing needed to be used in the description of the prior art are briefly described, it should be apparent that, in being described below
Attached drawing is some embodiments of the present invention, for those of ordinary skill in the art, before not making the creative labor
It puts, can also be obtained according to these attached drawings other attached drawings.
Fig. 1 is a kind of flow chart of the method for machine learning model effect assessment provided in an embodiment of the present invention；
Fig. 2 is the flow chart of the method for another machine learning model effect assessment provided in an embodiment of the present invention；
Fig. 3 is a kind of detail flowchart of the method for machine learning model effect assessment provided in an embodiment of the present invention；
Fig. 4 is the flow chart of the method for another machine learning model effect assessment provided in an embodiment of the present invention；
Fig. 5 is the schematic diagram of the device of another machine learning model effect assessment provided in an embodiment of the present invention.
Specific embodiment
Purpose, technical scheme and advantage to make the embodiment of the present invention are clearer, below in conjunction with attached drawing to the present invention
Technical solution be clearly and completely described, it is clear that described embodiment be part of the embodiment of the present invention rather than
Whole embodiments.Based on the embodiments of the present invention, those of ordinary skill in the art are not making creative work premise
Lower all other embodiments obtained, shall fall within the protection scope of the present invention.
Embodiment one：
According to embodiments of the present invention, a kind of embodiment of the method for machine learning model effect assessment is provided, needs to illustrate
, step shown in the flowchart of the accompanying drawings can hold in the computer system of such as a group of computer-executable instructions
Row, although also, show logical order in flow charts, it in some cases, can be to be different from sequence herein
Perform shown or described step.
Fig. 1 is a kind of flow chart of the method for machine learning model effect assessment according to embodiments of the present invention, such as Fig. 1 institutes
Show, this method comprises the following steps：
Step S102 obtains psychological shock resistance monitoring data, obtains target data set, wherein, the target data is concentrated
Include the psychological shock resistance monitoring data sample of multiple detection objects, the psychological shock resistance monitoring data of each detection object
Sample includes multiple psychology and detects the test result of examination question and the objective impact-level of multiple events, the objective impact-level of event
To be used for the data of the intrinsic psychological impact to detecting object of the event of reacting.
Step S104, based on the target data concentrate psychological shock resistance monitoring data sample structure target study collection and
Target detection collection.
Target study collection and the target detection collection are separately input to a variety of machine learning classification moulds by step S108
In type, to determine the test accuracy rate of each machine learning classification model according to result of calculation.
In embodiments of the present invention, first, psychological shock resistance monitoring data are obtained, obtain target data set；Then, it is based on
The psychological shock resistance monitoring data sample structure target study collection and target detection collection that the target data is concentrated；Finally, by mesh
Mark study collection and the target detection collection are separately input in a variety of machine learning classification models, to be determined often according to result of calculation
The test accuracy rate of kind machine learning classification model.In embodiments of the present invention, by determining a variety of machine learning classification models
In the test accuracy rate of each machine learning classification model select the corresponding machine learning classification model to carry out psychological shock resistance
The assessment of ability, so as to alleviate due to selecting the preferable machine learning of effectiveness not from a variety of machine moulds in the prior art
Model assesses psychological impact resistance, leads to the technology poor to psychological impact resistance prediction effect occur to ask
Topic has reached the technology effect for the best machine learning model of effectiveness that psychological impact resistance is assessed in a variety of machine moulds of acquisition
Fruit.
It should be noted that the psychology shock resistance monitoring data can monitor system by psychological shock resistance, psychology resists
The forms such as Impact monitoring APP or questionnaire obtain；In addition, the test result of the multiple psychology detection examination question can be inspection
Survey the answer score obtained after object answers the problems in questionnaire.
In embodiments of the present invention, as shown in Fig. 2, the step S104 is further included：
Step S1041 concentrates the target data each psychological shock resistance monitoring data sample processing is reconstructed, obtains
To reconstruction result.
The target data set after reconstruct according to preset ratio is cut, obtains the target by step S1042
Study collection and the target detection collection.
It should be noted that the reconstruction result of each psychology shock resistance monitoring data sample includes the first variable and second
Variable, first variable are special based on the objective impact-level of event in the psychological shock resistance monitoring data sample and psychology
Sign variable determines that second variable is based on the anti-impact to critical incident in the psychological shock resistance monitoring data sample
It strikes waters and puts down what is determined.
Specifically, concentrate each psychological shock resistance monitoring data sample that processing is reconstructed by the target data；So
Afterwards, the target data set after reconstruct is cut according to preset ratio, obtains the target study collection and the mesh
Test set is marked, for example, the target data is concentrated comprising 1000 psychological shock resistance monitoring data samples, it will wherein 700 hearts
Shock resistance monitoring data sample composition target study collection is managed, by remaining 300 psychological shock resistance monitoring data sample composition mesh
Mark test set；The preset ratio is not specifically limited in embodiments of the present invention by user's sets itself.
In embodiments of the present invention, by the reconstruct and cutting to the target data set, the target detection collection is turned
Turn to study collection and test set that machine learning classification model can use.
Optionally, as shown in figure 3, the step S1041 is further included：
Step S11 from the test result corresponding to each psychological shock resistance monitoring data sample, extracts first group
Sub- test result is horizontal as the shock resistance to critical incident, and the shock resistance level to critical incident is for reaction detection
The data of the psychological shock resistance level of object.
Step S12 will remove described first in the test result corresponding to each psychological shock resistance monitoring data sample
Other sub- test results except the sub- test result of group are converted to multiple psychological characteristics.
Step S13 determines first variable based on the objective impact-level of the event and the multiple psychological characteristics, with
And second variable is determined based on the shock resistance level to critical incident.
Specifically, first, to the missing values in each psychological shock resistance monitoring data sample or the institute of abnormal typing
The test result for stating psychology detection examination question is cleaned, to by the missing in each psychological shock resistance monitoring data sample
Value or the test result of the psychology detection examination question of abnormal typing are from each psychological shock resistance monitoring data sample
Middle deletion.
Then, according to the design of entity in questionnaire, from each psychological shock resistance monitoring data sample after cleaning
The test result that the L psychology detection examination questions are extracted in this forms first group of sub- test result, pair as the data sample
The horizontal R of shock resistance of critical incident；L event visitor is obtained from each psychological shock resistance monitoring data sample after cleaning
See impact-level S；Described first group son will be removed in test result corresponding to each psychological shock resistance monitoring data sample
Other m sub- test results except test result are converted to multiple psychological characteristics.
Then, the above-mentioned sub- test result of multiple psychological characteristics is normalized.
Finally, described first is determined based on m psychological characteristics after the objective impact-level S of the L event and normalization
Variable X, wherein, the first variable X is m psychic trait variable；It is determined based on the L Rs horizontal to the shock resistance of critical incident
The second variable y, wherein, the second variable y is observation of the L event to individual impact-level.
In embodiments of the present invention, by the cleaning and conversion concentrated to the target data, by the target data set
In include psychology detection examination question test result and the objective impact-level of event be converted to machine learning model can identify and
The data of processing.
In embodiments of the present invention, as shown in Fig. 2, the step S106, further includes：
Step S1061 calls a variety of machine learning classification models, and the target learning sample that target study is concentrated
It is input in each machine learning classification model and carries out learning training.
The target detection sample that the target detection is concentrated is input to each machine learning classification model by step S1062
In, calculate the test accuracy rate of each machine learning classification model.
It should be noted that first variable in the target learning sample is the defeated of machine learning classification model
Enter, second variable in the target learning sample is the output of machine learning classification model.
In embodiments of the present invention, first, it is horizontal to obtain first group of psychology shock resistance；Then, the target is learnt to collect
In target learning sample be input in each machine learning classification model and carry out learning training；Finally, by the target detection
The target detection sample of concentration is input in each machine learning classification model, calculates the test of each machine learning classification model
Accuracy rate.
Specifically, the knot to the horizontal accuracy rate of psychological shock resistance of each machine learning classification model is obtained by calculation
Fruit has achieved the purpose that each machine learning classification model effect assessment, each described machine learning classification model accuracy rate
Effectiveness of each machine learning classification model in the psychological shock resistance level of analysis is reacted.
Optionally, as shown in figure 3, the step S1062 is further included：
Step S21 for machine learning classification model each described, becomes based on target detection concentration includes first
Amount calculates the psychological anti-impact of the psychological shock resistance monitoring data sample of each detection object that the target detection collection includes
It strikes waters flat, it is horizontal to obtain first group of psychology shock resistance.
Step S22, the second variable that target detection concentration is included are horizontal as second group of psychology shock resistance.
Step S23, it is every based on first group of psychology shock resistance level and second group of psychology shock resistance level calculation
The accuracy rate of kind machine learning classification model.
In embodiments of the present invention, first, target study collection is sequentially input into a variety of machine learning classification moulds
In type, and calculate the psychological anti-impact that the psychological shock resistance monitoring data sample of each detection object is concentrated in the target study
It strikes waters flat, it is horizontal to obtain first group of psychology shock resistance；
Then, it is horizontal to obtain second group of psychology shock resistance.
Finally, based on first group of psychology shock resistance level and described each machine of second group of psychology shock resistance level calculation
The accuracy rate of device learning classification model；The accuracy rate calculation of machine learning classification model is a variety of, specifically calculation
It needs a variety of different indexs observed to be needed to determine based on appraiser, in embodiments of the present invention, be not specifically limited.
The specific calculation for introducing two kinds of machine learning classification model accuracys rate is described below：
The first calculation of the machine learning classification model accuracy rate：
First, the horizontal difference between j-th of psychological shock resistance level of i-th of psychological shock resistance is calculated, is obtained multiple
Difference, described i-th psychological shock resistance level are struck waters for i-th of psychological anti-impact in first group of psychology shock resistance level
Flat, described j-th psychological shock resistance level is strikes waters with described i-th psychological anti-impact in second group of psychology shock resistance level
Flat corresponding psychological shock resistance is horizontal, and i takes 1 to N successively, and N manages shock resistance for first group of psychology shock resistance horizontal centre
Horizontal quantity.
Then, i-th in first group of psychology shock resistance level corresponding to target difference psychological anti-impact is counted to strike waters
Flat quantity, wherein, the target difference is to be less than or equal to the difference (preset difference value of default value in the multiple difference
Less than or equal to 5 or less than or equal to 10), the size of specific preset difference value can voluntarily be set according to actual conditions by appraiser
It puts, does not do specific setting in embodiments of the present invention.
Finally, horizontal total of the quantity and the psychological shock resistance that is included in first group of psychology shock resistance level is calculated
The ratio of quantity, using the ratio as the accuracy rate of machine learning classification model each described.For example, the quantity is 80,
The total quantity of psychological shock resistance level included in first group of psychology shock resistance level is 100, then the machine learning point
The accuracy rate of class model is 80/100*100%=80%.
Second of calculation of the machine learning classification model accuracy rate：
First, the covariance C of first group of psychology shock resistance level and second group of psychology shock resistance level is calculated.
Then, the variance V of second group of psychology shock resistance level is calculated.
Finally, (1-C/V) * 100% is calculated, using result of calculation as the accuracy rate of the machine learning classification model, example
Such as, the result of calculation of the covariance C is equal to 0.1, and the result of calculation of the variance V is equal to 0.5, then the machine learning classification
The accuracy rate of model is equal to (1-0.1/0.5) * 100%=80%.
In embodiments of the present invention, the accuracy rate of each machine learning classification model is obtained by calculation, passes through the standard
True rate has reacted effectiveness of each machine learning classification model when assessing detection object psychology impact resistance.
In embodiments of the present invention, as shown in figure 4, the appraisal procedure of the effectiveness of the machine learning model further includes：
Step S108 compares the test accuracy rate of each machine learning classification model, and test accuracy rate is highest
Machine learning classification model machine learning classification model as a purpose.
In embodiments of the present invention, it by comparing the test accuracy rate of each machine learning classification model, will test
The highest machine learning classification model of accuracy rate machine learning classification model as a purpose, and by the target machine learning classification
Machine sort learning model of the model as later assessment detection object psychology impact resistance.
Embodiment two：
The embodiment of the present invention additionally provides a kind of device of machine learning model effect assessment, the machine learning model effectiveness
Apparatus for evaluating for performing the appraisal procedure of machine learning model effectiveness that the above of the embodiment of the present invention is provided, below
Specific introduction is done to the apparatus for evaluating of machine learning model effectiveness provided in an embodiment of the present invention.
Fig. 5 is according to a kind of schematic device of machine learning model effect assessment of the embodiment of the present invention, such as Fig. 5 institutes
Show, the apparatus for evaluating of the machine learning model effectiveness mainly includes：Acquisition device 10, processing unit 20 and are commented construction device 30
Estimate device 40, wherein,
The acquisition device 10 obtains psychological shock resistance monitoring data, obtains target data set, wherein, the target data
Concentrate the psychological shock resistance monitoring data sample for including multiple detection objects, the psychological shock resistance monitoring of each detection object
Data sample includes multiple psychology and detects the test result of examination question and the objective impact-level of event, the objective impact-level of event
To be used for the data of the intrinsic psychological impact to detecting object of the event of reacting；
The psychological shock resistance monitoring data sample that the construction device 20 is used to concentrate based on the target data builds mesh
Mark study collection and target detection collection；
Target study collection and the target detection collection are separately input to a variety of machine learning by the apparatus for evaluating 30
In disaggregated model, to determine the test accuracy rate of each machine learning classification model according to result of calculation.
In embodiments of the present invention, first, psychological shock resistance monitoring data are obtained, obtain target data set；Then, it is based on
The psychological shock resistance monitoring data sample structure target study collection and target detection collection that the target data is concentrated；Finally, by mesh
Mark study collection and the target detection collection are separately input in a variety of machine learning classification models, to be determined often according to result of calculation
The test accuracy rate of kind machine learning classification model.In embodiments of the present invention, by determining a variety of machine learning classification models
In the test accuracy rate of each machine learning classification model select the corresponding machine learning classification model to carry out psychological shock resistance
The assessment of ability, so as to alleviate due to selecting the preferable machine learning of effectiveness not from a variety of machine moulds in the prior art
Model assesses psychological impact resistance, leads to the technology poor to psychological impact resistance prediction effect occur to ask
Topic has reached the technology effect for the best machine learning model of effectiveness that psychological impact resistance is assessed in a variety of machine moulds of acquisition
Fruit.
In addition, in the description of the embodiment of the present invention unless specifically defined or limited otherwise, term " installation ", " phase
Even ", " connection " should be interpreted broadly, for example, it may be being fixedly connected or being detachably connected or be integrally connected；It can
To be mechanical connection or be electrically connected；It can be directly connected, can also be indirectly connected by intermediary, Ke Yishi
Connection inside two elements.For the ordinary skill in the art, above-mentioned term can be understood at this with concrete condition
Concrete meaning in invention.
In the description of the present invention, it should be noted that term " " center ", " on ", " under ", "left", "right", " vertical ",
The orientation or position relationship of the instructions such as " level ", " interior ", " outer " be based on orientation shown in the drawings or position relationship, merely to
Convenient for the description present invention and simplify description rather than instruction or imply signified device or element must have specific orientation,
With specific azimuth configuration and operation, therefore it is not considered as limiting the invention.In addition, term " first ", " second ",
" third " is only used for description purpose, and it is not intended that instruction or hint relative importance.
It is apparent to those skilled in the art that for convenience and simplicity of description, the system of foregoing description,
The specific work process of device and unit can refer to the corresponding process in preceding method embodiment, and details are not described herein.
In several embodiments provided herein, it should be understood that disclosed systems, devices and methods, it can be with
It realizes by another way.The apparatus embodiments described above are merely exemplary, for example, the division of the unit,
Only a kind of division of logic function, can there is other dividing mode in actual implementation, in another example, multiple units or component can
To combine or be desirably integrated into another system or some features can be ignored or does not perform.Another point, it is shown or beg for
The mutual coupling, direct-coupling or communication connection of opinion can be by some communication interfaces, device or unit it is indirect
Coupling or communication connection can be electrical, machinery or other forms.
The unit illustrated as separating component may or may not be physically separate, be shown as unit
The component shown may or may not be physical unit, you can be located at a place or can also be distributed to multiple
In network element.Some or all of unit therein can be selected according to the actual needs to realize the mesh of this embodiment scheme
's.
In addition, each functional unit in each embodiment of the present invention can be integrated in a processing unit, it can also
That each unit is individually physically present, can also two or more units integrate in a unit.
If the function is realized in the form of SFU software functional unit and is independent product sale or in use, can be with
It is stored in the non-volatile computer read/write memory medium that a processor can perform.Based on such understanding, the present invention
The part that substantially contributes in other words to the prior art of technical solution or the part of the technical solution can be with software
The form of product embodies, which is stored in a storage medium, including some instructions use so that
One computer equipment (can be personal computer, server or the network equipment etc.) performs each embodiment institute of the present invention
State all or part of step of method.And aforementioned storage medium includes：USB flash disk, mobile hard disk, read-only memory (ROM, Read-
Only Memory), random access memory (RAM, Random Access Memory), magnetic disc or CD etc. are various can be with
Store the medium of program code.
Finally it should be noted that：Embodiment described above, only specific embodiment of the invention, to illustrate the present invention
Technical solution, rather than its limitations, protection scope of the present invention is not limited thereto, although with reference to the foregoing embodiments to this hair
It is bright to be described in detail, it will be understood by those of ordinary skill in the art that：Any one skilled in the art
In the technical scope disclosed by the present invention, it can still modify to the technical solution recorded in previous embodiment or can be light
It is readily conceivable that variation or equivalent replacement is carried out to which part technical characteristic；And these modifications, variation or replacement, do not make
The essence of corresponding technical solution is detached from the spirit and scope of technical solution of the embodiment of the present invention, should all cover the protection in the present invention
Within the scope of.Therefore, protection scope of the present invention described should be subject to the protection scope in claims.
Claims (10)
- A kind of 1. method of machine learning model effect assessment, which is characterized in that including：Psychological shock resistance monitoring data are obtained, obtain target data set, wherein, the target data concentration includes multiple detections pair The psychological shock resistance monitoring data sample of elephant, the psychological shock resistance monitoring data sample of each detection object include multiple hearts The reason detection test result of examination question and the objective impact-level of multiple events, the objective impact-level of event are for the event of reacting The data of the intrinsic psychological impact to detecting object；The psychological shock resistance monitoring data sample structure target study collection and target detection collection concentrated based on the target data；Target study collection and the target detection collection are separately input in a variety of machine learning classification models, by terms of Calculate the test accuracy rate that result determines each machine learning classification model.
- 2. the according to the method described in claim 1, it is characterized in that, psychological shock resistance monitoring concentrated based on the target data Data sample structure target study collection and target detection collection include：It concentrates each psychological shock resistance monitoring data sample that processing is reconstructed to the target data, obtains reconstruction result, In, the reconstruction result of each psychology shock resistance monitoring data sample includes the first variable and the second variable, first variable It is described for what is determined based on the objective impact-level of event in the psychological shock resistance monitoring data sample and psychological characteristics variable Second variable is to be determined based on the shock resistance level to critical incident in the psychological shock resistance monitoring data sample；The target data set after reconstruct is cut according to preset ratio, obtains the target study collection and the mesh Mark test set.
- It is 3. right according to the method described in claim 2, it is characterized in that, the test result includes multiple sub- test results The target data concentrates each psychological shock resistance monitoring data sample that processing is reconstructed, and obtains reconstruction result and includes：From the test result corresponding to each psychological shock resistance monitoring data sample, the sub- test result of first group of extraction is made Horizontal for the shock resistance to critical incident, the shock resistance level to critical incident is resists for the psychology of reaction detection object The data of impact-level；First group of sub- test result will be removed in test result corresponding to each psychological shock resistance monitoring data sample Except other sub- test results be converted to multiple psychological characteristics；First variable is determined and based on institute based on the objective impact-level of the multiple event and the multiple psychological characteristics It states the shock resistance level to critical incident and determines second variable.
- 4. according to the method described in claim 2, it is characterized in that, target study collection and the target detection collection are distinguished It is input in a variety of machine learning classification models, to determine that the test of each machine learning classification model is accurate according to result of calculation Rate includes：A variety of machine learning classification models are called, and the target learning sample that target study is concentrated is input to each machine Learning training is carried out in learning classification model, wherein, first variable in the target learning sample is machine learning point The input of class model, second variable in the target learning sample are the output of machine learning classification model；The target detection sample that the target detection is concentrated is input in each machine learning classification model, calculates each machine The test accuracy rate of learning classification model.
- 5. according to the method described in claim 4, it is characterized in that, the method further includes：The test accuracy rate of each machine learning classification model is compared, by the highest machine learning classification mould of test accuracy rate Type machine learning classification model as a purpose.
- 6. according to the method described in claim 4, it is characterized in that, the target detection sample that the target detection is concentrated inputs Into each machine learning classification model, the test accuracy rate for calculating each machine learning classification model includes：For machine learning classification model each described, based on the first variable that target detection concentration includes, described in calculating The psychological shock resistance of the psychological shock resistance monitoring data sample of each detection object that target detection collection includes is horizontal, obtains First group of psychology shock resistance is horizontal；The second variable that target detection concentration is included is horizontal as second group of psychology shock resistance；Based on first group of psychology shock resistance level and described each machine learning of second group of psychology shock resistance level calculation point The accuracy rate of class model.
- 7. a kind of device of machine learning model effect assessment, which is characterized in that described device includes：Acquisition device, the acquisition device are used to obtain psychological shock resistance monitoring data, obtain target data set, wherein, it is described Target data concentration includes the psychological shock resistance monitoring data sample of multiple detection objects, and the psychology of each detection object resists Impact monitoring data sample includes multiple psychology and detects the test result of examination question and the objective impact-level of multiple events, the event Objective impact-level is the data for the intrinsic psychological impact to detecting object of the event of reacting；Construction device, the construction device are used for the psychological shock resistance monitoring data sample structure concentrated based on the target data Target study collection and target detection collection；Apparatus for evaluating, the apparatus for evaluating are used to target study collection and the target detection collection being separately input to a variety of machines In device learning classification model, to determine the test accuracy rate of each machine learning classification model according to result of calculation.
- 8. device according to claim 7, which is characterized in that the construction device is additionally operable to：It concentrates each psychological shock resistance monitoring data sample that processing is reconstructed to the target data, obtains reconstruction result, In, the reconstruction result of each psychology shock resistance monitoring data sample includes the first variable and the second variable, first variable It is described for what is determined based on the objective impact-level of event in the psychological shock resistance monitoring data sample and psychological characteristics variable Second variable is to be determined based on the shock resistance level to critical incident in the psychological shock resistance monitoring data sample；The target data set after reconstruct is cut according to preset ratio, obtains the target study collection and the mesh Mark test set.
- 9. device according to claim 8, which is characterized in that the construction device is additionally operable to：From the test result corresponding to each psychological shock resistance monitoring data sample, the sub- test result of first group of extraction is made Horizontal for the shock resistance to critical incident, the shock resistance level to critical incident is resists for the psychology of reaction detection object The data of impact-level；First group of sub- test result will be removed in test result corresponding to each psychological shock resistance monitoring data sample Except other sub- test results be converted to multiple psychological characteristics；First variable is determined and based on described right based on the objective impact-level of the event and the multiple psychological characteristics The shock resistance level of critical incident determines second variable.
- 10. device according to claim 8, which is characterized in that the apparatus for evaluating is additionally operable to：A variety of machine learning classification models are called, and the target learning sample that target study is concentrated is input to each machine Learning training is carried out in learning classification model, wherein, first variable in the target learning sample is machine learning point The input of class model, second variable in the target learning sample are the output of machine learning classification model；The target detection sample that the target detection is concentrated is input in each machine learning classification model, calculates each machine The test accuracy rate of learning classification model.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810066061.6A CN108229690A (en) | 2018-01-22 | 2018-01-22 | A kind of method and apparatus of machine learning model effect assessment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810066061.6A CN108229690A (en) | 2018-01-22 | 2018-01-22 | A kind of method and apparatus of machine learning model effect assessment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108229690A true CN108229690A (en) | 2018-06-29 |
Family
ID=62668559
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810066061.6A Pending CN108229690A (en) | 2018-01-22 | 2018-01-22 | A kind of method and apparatus of machine learning model effect assessment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108229690A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113935031A (en) * | 2020-12-03 | 2022-01-14 | 网神信息技术（北京）股份有限公司 | Method and system for file feature extraction range configuration and static malicious software identification |
| WO2024061252A1 (en) * | 2022-09-20 | 2024-03-28 | 维沃移动通信有限公司 | Information acquisition method and apparatus, network side device and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105069304A (en) * | 2015-08-18 | 2015-11-18 | 广东顺德中山大学卡内基梅隆大学国际联合研究院 | Machine learning-based method for evaluating and predicting ASD |
| CN106407672A (en) * | 2016-09-08 | 2017-02-15 | 山东腾泰医疗科技有限公司 | Mental health evaluation system based on Internet |
| CN106599230A (en) * | 2016-12-19 | 2017-04-26 | 北京天元创新科技有限公司 | Method and system for evaluating distributed data mining model |
-
2018
- 2018-01-22 CN CN201810066061.6A patent/CN108229690A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105069304A (en) * | 2015-08-18 | 2015-11-18 | 广东顺德中山大学卡内基梅隆大学国际联合研究院 | Machine learning-based method for evaluating and predicting ASD |
| CN106407672A (en) * | 2016-09-08 | 2017-02-15 | 山东腾泰医疗科技有限公司 | Mental health evaluation system based on Internet |
| CN106599230A (en) * | 2016-12-19 | 2017-04-26 | 北京天元创新科技有限公司 | Method and system for evaluating distributed data mining model |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113935031A (en) * | 2020-12-03 | 2022-01-14 | 网神信息技术（北京）股份有限公司 | Method and system for file feature extraction range configuration and static malicious software identification |
| WO2024061252A1 (en) * | 2022-09-20 | 2024-03-28 | 维沃移动通信有限公司 | Information acquisition method and apparatus, network side device and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10896080B2 (en) | S.M.A.R.T. threshold optimization method used for disk failure detection | |
| CN105653444B (en) | Software defect fault recognition method and system based on internet daily record data | |
| CN105809035B (en) | The malware detection method and system of real-time behavior is applied based on Android | |
| CN107301118B (en) | A kind of fault indices automatic marking method and system based on log | |
| Znidersic et al. | Using visualization and machine learning methods to monitor low detectability species—The least bittern as a case study | |
| Sebald et al. | Human or natural? Landscape context improves the attribution of forest disturbances mapped from Landsat in Central Europe | |
| CN109034036A (en) | A kind of video analysis method, Method of Teaching Quality Evaluation and system, computer readable storage medium | |
| CN114066242A (en) | Enterprise risk early warning method and device | |
| CN109936582A (en) | Construct the method and device based on the PU malicious traffic stream detection model learnt | |
| CN108268893B (en) | Chemical industry park early warning method and device based on machine learning | |
| Nocera et al. | Let's agree to disagree: Comparing auto-acoustic identification programs for northeastern bats | |
| CN113435314B (en) | Rolling bearing acoustic signal early fault sensitivity characteristic screening method and system | |
| CN101738998B (en) | System and method for monitoring industrial process based on local discriminatory analysis | |
| CN110738255A (en) | device state monitoring method based on clustering algorithm | |
| Sethi et al. | Soundscapes predict species occurrence in tropical forests | |
| CN106792883A (en) | Sensor network abnormal deviation data examination method and system | |
| CN108268624A (en) | User data method for visualizing and system | |
| CN108229690A (en) | A kind of method and apparatus of machine learning model effect assessment | |
| CN109325888A (en) | A kind of students ' behavior prediction technique based on artificial neural network | |
| CN110490486B (en) | Enterprise big data management system | |
| CN116415931A (en) | Big data-based power equipment operation state monitoring method and system | |
| Prates et al. | Intervention analysis of hurricane effects on snail abundance in a tropical forest using long-term spatiotemporal data | |
| CN107480703A (en) | Transaction fault detection method and device | |
| CN116824311A (en) | Performance detection method, device, equipment and storage medium of crowd analysis algorithm | |
| Long et al. | Monitoring seagrasses in tropical ports and harbours |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180629 |
|
| RJ01 | Rejection of invention patent application after publication |