CN109800999A - Personnel Overall Qualities platform based on big data analysis - Google Patents
Personnel Overall Qualities platform based on big data analysis Download PDFInfo
- Publication number
- CN109800999A CN109800999A CN201910103488.3A CN201910103488A CN109800999A CN 109800999 A CN109800999 A CN 109800999A CN 201910103488 A CN201910103488 A CN 201910103488A CN 109800999 A CN109800999 A CN 109800999A
- Authority
- CN
- China
- Prior art keywords
- data
- student
- overall qualities
- layer
- big data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a kind of Personnel Overall Qualities platform based on big data analysis, comprising: talent's data of each data source are converged to HDFS storage by uniform data acquisition layer in a manner of being adapted to；Base data platform layer carries out big data analysis to talent's data from the uniform data acquisition layer, forms Evaluating Personnel Overall Qualities case history report, and provide publication and inquiry；Using presentation layer, the Evaluating Personnel Overall Qualities case history report issued or inquired by the Base data platform layer is shown.The present invention realizes Evaluating Personnel Overall Qualities and management, accuracy, convenience and safety with higher.
Description
Technical field
The present invention relates to big data technical fields, more particularly to utilize the Personnel Overall Qualities platform of big data analysis.
Background technique
According to relevant policies, the research work of students in middle and primary schools' Qualities Evaluation, and plan Students ' Comprehensive element are carried out in the whole nation
Matter evaluation of programme and policy text, implementing plan have middle and primary schools' overall qualities appraisement system of requirements of the times.Overall qualities are commented
Valence system pays close attention to the outer social practice's record of school, student examination track record, student's electronic identity card for students were applied
The recording data information of Cheng Jilu, recording of growing up volume, green index etc., objective record student is in school eduaction, extracurricular
The process of activity, social venue study and practice process carries out on the basis of information is collected in one side on Information function
The information of different levels different dimensions is presented, and social venue and school etc. educational institutions is on the other hand helped preferably to coordinate and manage
Science and engineering is made.
By big data analysis, forming objective, accurate Evaluating Personnel Overall Qualities report is particularly important.At this
In the process, data source is more, and needs mining data, forms the overall qualities evaluation data of the talent.Therefore foundation construction is needed
Data center constructs unified data storage and publication system, unified to carry out data modeling, establishes base for the value presentation of data
Plinth.
Summary of the invention
The purpose of the present invention is to provide a kind of the Personnel Overall Qualities platform based on big data analysis, realization talent's synthesis
Qualities Evaluation and management, accuracy, convenience and safety with higher.
Realizing the technical solution of above-mentioned purpose is:
A kind of Personnel Overall Qualities platform based on big data analysis, comprising:
Talent's data of each data source are converged to HDFS (Hadoop by uniform data acquisition layer in a manner of being adapted to
Distributed File System, distributed file system) storage；
Base data platform layer carries out big data analysis to talent's data from the uniform data acquisition layer, is formed
Evaluating Personnel Overall Qualities case history report, and publication and inquiry are provided；And
Using presentation layer, the Evaluating Personnel Overall Qualities record report that will be issued or inquired by the Base data platform layer
Announcement is shown.
Preferably, the adaptation mode that the uniform data acquisition layer uses includes: SQL (Structured Query
Language, structured query language) acquisition adaptation, file acquisition adaptation and HDFS acquisition adaptation.
Preferably, the uniform data acquisition layer sends Flume-NG (day by file interface mode for talent's data
Will collection system) cluster, Flume-NG cluster is by internal storage data transmission mode, by received data in real time by the side HDFS
Formula converges to the Base data platform layer.
Preferably, the uniform data acquisition layer passes through a kind of Kafka (distributed post subscription message system of high-throughput
System) message management layer as unified acquisition platform, and using HAProxy (HAProxy be one make to show a C language certainly
By and open-source software, provide high availability, load balancing, and the Application Agent based on TCP and HTTP)+
(keepalived is one and is similar to layer3, the software of 4&5 exchanging mechanism, i.e., the 3rd layer usually said, the 4th Keepalived
Layer and the 5th layer of exchange.Effect is the state of detection service device) mode of+Flume-NG realizes distributed data acquisition.
Preferably, the data source include: school's essential information, class's information, student's essential information, curriculum information,
Student result data, student health information, student's activities information, student's activities practice record, student's rewards and punishments record, student's research
Project record, student's self-assessment record, evaluation of teacher's record, student's training record and students ' reading record.
Preferably, the Base data platform layer includes:
Data-mining module is cleaned and is converted to talent's data using HIVE (Tool for Data Warehouse), forms feature
Wide table, and data mining model is constructed using Spark R (computing engines), talent's data are carried out by data mining model big
Data analysis；
Overall qualities evaluate big data publication and enquiry module, using HBASE (HBASE be one it is distributed, towards column
PostgreSQL database) technology and RDBMS (relational database management system) technology realize Evaluating Personnel Overall Qualities case history report
Publication and inquiry；
Data mining model constructs big data analysis engine with the trunking mode of Spark On Yarn.
Preferably, Evaluating Personnel Overall Qualities case history report includes statistical information and managing detailed catalogue.
Preferably, the overall qualities evaluation big data publication and enquiry module, first record Evaluating Personnel Overall Qualities
The result set of the statistical information and original detail reported in fact is stored in HBASE, and storage result collection is created in HBASE
HBASE table is inquired by open HBASE API after the file warehousing of generation；It will be bright after the original detail collating sort
Thin information is for statistical analysis, and stores to RDBMS, provides publication and inquiry.
Preferably, Evaluating Personnel Overall Qualities case history report is divided into student's report and school's report,
The special topic of self-assessment, student's course achievement, each research topic that student's report is actively made a report on including student
Record and practical activity record；
School report includes school's basic condition, situation of not giving a course, teacher is equipped with situation, practical activity is carried out in school
With the prominent student's list of activity.
It preferably, further include big data monitoring management system, which includes operation layer monitoring mould
Block, application layer monitoring module and system layer monitoring module；
The operation layer monitoring module is monitored Evaluating Personnel Overall Qualities data, and monitoring data includes: comprehensive element
Matter evaluation case history report generation quantity, case history report check that quantity, login failure rate and interface call success rate；
The application layer monitoring module monitors various middlewares and computing engines, and monitoring data includes in Java Virtual Machine heap
It deposits, CPU (central processing unit) utilization rate, Thread Count and handling capacity；
The system layer monitoring module monitors server contention states in real time, and monitor control index includes memory, disk, CPU, net
Network flow and system process.
The beneficial effects of the present invention are: establishing the overall qualities evaluation handbook electricity of the talent the present invention is based on big data analysis
Unified student individual reports, school's report are realized in sonization management, and establish the required external data docking of overall qualities report
And relation mechanism, including achievement, health, social venue record etc., it ensure that accuracy, the authority of Evaluating Personnel Overall Qualities
Property and safety.
Detailed description of the invention
Fig. 1 is the structure chart of the Personnel Overall Qualities platform of the invention based on big data analysis；
Fig. 2 is the structure chart of uniform data acquisition layer in the present invention；
Fig. 3 is the structure chart of Base data platform layer in the present invention；
Fig. 4 is the structure chart that presentation layer is applied in the present invention；
Fig. 5 is the structure chart of another embodiment of Personnel Overall Qualities platform of the invention.
Specific embodiment
The present invention will be further described with reference to the accompanying drawings.
Fig. 1-Fig. 4, the Personnel Overall Qualities platform of the invention based on big data analysis, including uniform data is please referred to adopt
Collect layer 11, Base data platform layer 21 and applies presentation layer 31.
Talent's data of each data source are converged to HDFS storage by uniform data acquisition layer 11 in a manner of being adapted to.Adaptation side
Formula includes: SQL acquisition adaptation, file acquisition adaptation and HDFS acquisition adaptation.Uniform data acquisition layer 11 is by Kafka as system
The message management layer of one acquisition platform flexibly docks, is adapted to various data sources acquisition (integrated Flume), provide flexibly, can match
The data acquisition ability set, and realize that high-performance High Availabitity is distributed by the way of HAProxy+Keepalived+Flume-NG
The acquisition of formula data.
Data source includes: school's essential information, class's information, student's essential information, curriculum information, student result data,
Student health information, student's activities information, student's activities practice record, student's rewards and punishments record, student's research topic record, student
Self-assessment record, evaluation of teacher's record, student's training record, students ' reading record.
Uniform data acquisition layer 11 sends Flume-NG cluster, Flume-NG by file interface mode for talent's data
Received data are converged to Base data platform layer in real time by HDFS mode by internal storage data transmission mode by cluster
21。
21 pairs of Base data platform layer talent's data from uniform data acquisition layer 11 carry out big data analysis, form people
Ability overall qualities evaluate case history report, and provide publication and inquiry.Specifically, Base data platform layer 21 is commented including overall qualities
The publication of valence big data and enquiry module 211 and data-mining module 212.
Data-mining module 212, as data cleansing engine, provides PB (Petabyte, petabyte or ten million using HIVE
Hundred million bytes or thousand T bytes) grade data prediction, processing, integrated service, it forms the wide table of feature and is adopted based on the data of the wide table of feature
With Spark R, cluster, classification scheduling algorithm are called, model development, the model evaluation, model application of data mining are carried out.Spark
R provides API (the Application Programming Interface, using journey of elasticity distribution formula data set in Spark
Sequence programming interface), user can pass through the operation task of Cmd Shell on cluster, and data mining model is with Spark On
The trunking mode of Yarn constructs big data analysis engine.Big data analysis is carried out to talent's data by data mining model.
Overall qualities evaluate big data publication and enquiry module 211, first by Evaluating Personnel Overall Qualities case history report
The result set of statistical information and original detail is stored in HBASE, and the HBASE table of storage result collection is created in HBASE, is generated
File warehousing after inquired by open HBASE API, using HBASE technology can provide mass data efficient publication and
Inquiry.The detailed data of magnanimity after the original detail collating sort is for statistical analysis, and store to RDBMS, it provides
The statistical data and Evaluating Personnel Overall Qualities case history report that Personnel Overall Qualities height summarizes, meet conventional statistical report form need
It asks, reduce platform uses threshold.
Evaluating Personnel Overall Qualities case history report includes statistical information and managing detailed catalogue.Evaluating Personnel Overall Qualities record report
It accuses and reports that two parts form by student's report and school.Student's report is recorded and is reported presentation with individual students angle, is wrapped
Include the self-assessment (overall merit, ideological and ethical standard etc.) that student actively makes a report on, student's course achievement (end of term including basic course
It examines, uniformly examine, expansion class, research class, reward etc.), the topic record of each research topic, practical activity record (can be associated with electronics
Student's identity card application record) etc..School's report is recorded and is reported in terms of learning activities in schools is carried out with the other angle of student
It presents, including school's basic condition, does not give a course situation (basic course, expansion class, research class etc.), teacher is equipped with the situation (head of a theatrical troupe
Appoint, teacher), practical activity, the prominent student's list of activity etc. are carried out in school；Similar content in student's report and school's report
It can correspond to each other, prove each other.
The Evaluating Personnel Overall Qualities record report that will be issued or inquired by Base data platform layer 21 using presentation layer 31
Announcement is shown.It is divided into overall qualities statistical information and overall qualities managing detailed catalogue, shows that Evaluating Personnel Overall Qualities are recorded respectively
The statistical data and detailed data reported in fact.
Further, to increase accuracy and safety, as shown in figure 5, the present invention increases big data monitoring management system 41,
The big data monitoring management system 41 includes operation layer monitoring module, application layer monitoring module and system layer monitoring module.
Operation layer monitoring is monitored to Evaluating Personnel Overall Qualities data, can find that program bug or business are patrolled in time
Design defect is collected, for example overall qualities evaluation case history report generation quantity, case history report check quantity, login failure rate, interface
Call success rate etc..Application layer monitoring is not limited to operation system, further includes various middlewares, computing engines, monitoring data is also
Include Java Virtual Machine heap memory, CPU usage, Thread Count, handling capacity etc.；Server work shape is grasped in system layer monitoring in real time
State, monitor control index include the system level performance metrics such as memory, disk, CPU, network flow, system process.
By the above-mentioned Personnel Overall Qualities platform using above-mentioned based on big data analysis, the overall qualities for establishing the talent are commented
Valence handbook electronic management function is realized unified student individual reports, school's report, and is established needed for overall qualities report
External data docking and relation mechanism, including achievement, health, social venue record etc., by big data analysis, ensure that people
Accuracy, authority and the safety of ability overall qualities evaluation.
Above embodiments are used for illustrative purposes only, rather than limitation of the present invention, the technology people in relation to technical field
Member, without departing from the spirit and scope of the present invention, can also make various transformation or modification, therefore all equivalent
Technical solution also should belong to scope of the invention, should be limited by each claim.
Claims (10)
1. a kind of Personnel Overall Qualities platform based on big data analysis characterized by comprising
Talent's data of each data source are converged to HDFS storage by uniform data acquisition layer in a manner of being adapted to；
Base data platform layer carries out big data analysis to talent's data from the uniform data acquisition layer, forms the talent
Overall qualities evaluate case history report, and provide publication and inquiry；And
Using presentation layer, the Evaluating Personnel Overall Qualities case history report exhibition that will be issued or inquired by the Base data platform layer
It shows and.
2. the Personnel Overall Qualities platform according to claim 1 based on big data analysis, which is characterized in that the unification
The adaptation mode that data collection layer uses includes: SQL acquisition adaptation, file acquisition adaptation and HDFS acquisition adaptation.
3. the Personnel Overall Qualities platform according to claim 1 based on big data analysis, which is characterized in that the unification
Data collection layer sends Flume-NG cluster by file interface mode for talent's data, and Flume-NG cluster passes through memory number
According to transmission mode, received data are converged into the Base data platform layer in real time by HDFS mode.
4. the Personnel Overall Qualities platform according to claim 2 based on big data analysis, which is characterized in that the unification
Message management layer of the data collection layer by Kafka as unified acquisition platform, and use HAProxy+Keepalived+
The mode of Flume-NG realizes distributed data acquisition.
5. the Personnel Overall Qualities platform according to claim 1 based on big data analysis, which is characterized in that the number
It include: school's essential information, class's information, student's essential information, curriculum information, student result data, student health letter according to source
Breath, student's activities information, student's activities practice record, student's rewards and punishments record, student's research topic record, student's self-assessment note
Record, evaluation of teacher's record, student's training record and students ' reading record.
6. the Personnel Overall Qualities platform according to claim 1 based on big data analysis, which is characterized in that the basis
Data platform layer includes:
Data-mining module is cleaned and is converted to talent's data using HIVE, forms the wide table of feature, and use Spark R
Data mining model is constructed, big data analysis is carried out to talent's data by data mining model；
Overall qualities evaluate big data publication and enquiry module, realize Personnel Overall Qualities using HBASE technology and RDBMS technology
Evaluate the publication and inquiry of case history report；
Data mining model constructs big data analysis engine with the trunking mode of Spark On Yarn.
7. the Personnel Overall Qualities platform according to claim 6 based on big data analysis, which is characterized in that the talent is comprehensive
Qualities Evaluation case history report includes statistical information and managing detailed catalogue.
8. the Personnel Overall Qualities platform according to claim 7 based on big data analysis, which is characterized in that the synthesis
The publication of Qualities Evaluation big data and enquiry module, first by the statistical information of Evaluating Personnel Overall Qualities case history report and original bright
Thin result set is stored in HBASE, and the HBASE table of storage result collection is created in HBASE, is passed through after the file warehousing of generation
HBASE API is opened to inquire；Managing detailed catalogue after the original detail collating sort is for statistical analysis, and store to
RDBMS provides publication and inquiry.
9. the Personnel Overall Qualities platform according to claim 7 based on big data analysis, which is characterized in that the talent is comprehensive
Qualities Evaluation case history report is divided into student's report and school's report,
The topic record of self-assessment, student's course achievement, each research topic that student's report is actively made a report on including student
It is recorded with practical activity；
School report includes school's basic condition, situation of not giving a course, teacher is equipped with situation, practical activity and work are carried out in school
Dynamic prominent student's list.
10. the Personnel Overall Qualities platform according to claim 1 based on big data analysis, which is characterized in that further include
Big data monitoring management system, the big data monitoring management system include operation layer monitoring module, application layer monitoring module and are
System layer monitoring module；
The operation layer monitoring module is monitored Evaluating Personnel Overall Qualities data, and monitoring data includes: overall qualities are commented
Valence case history report generation quantity, case history report check that quantity, login failure rate and interface call success rate；
The application layer monitoring module monitors various middlewares and computing engines, monitoring data include Java Virtual Machine heap memory,
CPU usage, Thread Count and handling capacity；
The system layer monitoring module monitors server contention states in real time, and monitor control index includes memory, disk, CPU, network flow
Amount and system process.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910103488.3A CN109800999A (en) | 2019-02-01 | 2019-02-01 | Personnel Overall Qualities platform based on big data analysis |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910103488.3A CN109800999A (en) | 2019-02-01 | 2019-02-01 | Personnel Overall Qualities platform based on big data analysis |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109800999A true CN109800999A (en) | 2019-05-24 |
Family
ID=66561954
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910103488.3A Pending CN109800999A (en) | 2019-02-01 | 2019-02-01 | Personnel Overall Qualities platform based on big data analysis |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109800999A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111126852A (en) * | 2019-12-25 | 2020-05-08 | 江苏三六五网络股份有限公司 | BI application system based on big data modeling |
| CN112348721A (en) * | 2020-11-10 | 2021-02-09 | 创而新(北京)教育科技有限公司 | Primary and secondary school student comprehensive quality evaluation platform and method based on big data acquisition |
| CN115796457A (en) * | 2023-02-03 | 2023-03-14 | 山东铁路投资控股集团有限公司 | Personnel and enterprise rating method and system based on multidimensional data |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104573071A (en) * | 2015-01-26 | 2015-04-29 | 湖南大学 | Intelligent school situation analysis system and method based on megadata technology |
| CN106919685A (en) * | 2017-03-02 | 2017-07-04 | 浪潮软件集团有限公司 | Mass data file processing method |
| CN107808356A (en) * | 2017-09-28 | 2018-03-16 | 浪潮软件集团有限公司 | Comprehensive quality evaluation system and method for evaluating comprehensive quality of students |
| CN108874982A (en) * | 2018-06-11 | 2018-11-23 | 华南理工大学 | A method of based on the offline real-time processing data of Spark big data frame |
-
2019
- 2019-02-01 CN CN201910103488.3A patent/CN109800999A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104573071A (en) * | 2015-01-26 | 2015-04-29 | 湖南大学 | Intelligent school situation analysis system and method based on megadata technology |
| CN106919685A (en) * | 2017-03-02 | 2017-07-04 | 浪潮软件集团有限公司 | Mass data file processing method |
| CN107808356A (en) * | 2017-09-28 | 2018-03-16 | 浪潮软件集团有限公司 | Comprehensive quality evaluation system and method for evaluating comprehensive quality of students |
| CN108874982A (en) * | 2018-06-11 | 2018-11-23 | 华南理工大学 | A method of based on the offline real-time processing data of Spark big data frame |
Non-Patent Citations (3)
| Title |
|---|
| 佚名: "揭开迷雾，点数成金-企业大数据之路", 《HTTPS://WWW.DOC88.COM/P-3292318298883.HTML》 * |
| 朱琦等: "《分布式应用系统运维理论与实践》", 31 October 2014 * |
| 杨飞等: "基于Ceph对象存储集群的高可用设计与实现", 《微电子学与计算机》 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111126852A (en) * | 2019-12-25 | 2020-05-08 | 江苏三六五网络股份有限公司 | BI application system based on big data modeling |
| CN112348721A (en) * | 2020-11-10 | 2021-02-09 | 创而新(北京)教育科技有限公司 | Primary and secondary school student comprehensive quality evaluation platform and method based on big data acquisition |
| CN115796457A (en) * | 2023-02-03 | 2023-03-14 | 山东铁路投资控股集团有限公司 | Personnel and enterprise rating method and system based on multidimensional data |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Mehmood et al. | UTiLearn: a personalised ubiquitous teaching and learning system for smart societies | |
| Daniel | B ig D ata and analytics in higher education: Opportunities and challenges | |
| Fulantelli et al. | A framework to support educational decision making in mobile learning | |
| Jindal et al. | A survey on educational data mining and research trends | |
| CN109800999A (en) | Personnel Overall Qualities platform based on big data analysis | |
| Ekong et al. | Harnessing big data technology to benefit effective delivery and performance maximization in pedagogy | |
| CN109670770A (en) | A kind of campus platform based on professional education informationization | |
| Johnson et al. | The status of econometrics in the economics major: A survey | |
| Minguillón et al. | Learning analytics in practice: Providing E-learning researchers and practitioners with activity data | |
| Sun | 10 Bigs: Big data and its ten big characteristics | |
| CN111768059A (en) | University student growth path evaluation system | |
| Wang | Big data era influence on college students' ideological and political education and innovation strategy | |
| Mukthar et al. | Big Data analytics for higher education in Saudi Arabia | |
| Zhao et al. | Learning quality evaluation of MOOC based on big data analysis | |
| Liao | Intelligent Algorithm Big Data Analysis for the Construction of Smart Campus | |
| Moreira et al. | Learning analytics as a core component for higher education disruption: governance stakeholder | |
| Liang et al. | Predicting students’ academic performance based on improved PSO-Xgboost: a campus behavior perspective | |
| Qiu | A Kind of Ideological and Political Education Management Platform Design Based on Web Technology | |
| Sui et al. | Research on the Application of Educational Big Data Analysis in Online Learning Behavior of Computer Basic Teaching | |
| Segooa et al. | Improve decision making towards universities performance through big data analytics | |
| Xiao et al. | Design of student behavior characteristics analysis platform based on K-means clustering algorithm in local universities | |
| Tian | Analysis of University Teaching Evaluation in the Era of BD | |
| Hu et al. | A Framework for Analysis Learning Pattern Toward Online Forum in Programming Course | |
| Travica | Think process, think in time: Advancing study of informing systems | |
| Buldaev et al. | Designing Feature Space for Educational Analytics Tasks Based on Data from Electronic Information Educational Environment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190524 |
|
| RJ01 | Rejection of invention patent application after publication |