RU2656699C1 - Icl-test - instrument for measuring information-communication competence in digital environment - Google Patents

Abstract

FIELD: calculating; counting.

SUBSTANCE: invention relates to an automatic evaluation system of information and communication competence of a student. System comprises a system for developing test tasks, comprising a server, interacting through a system for delivering the test to a test-taker, which is channels and flash drives, with a system for presenting a test case to a test-taker, comprising a computer interacting with a test result processing system, comprising a server, as well as a library of test tasks, test results, questionnaires and answers to questions of the questionnaire, which is a database associated with all said systems.

EFFECT: technical result is improved accuracy and reliability of the evaluation of the results of testing information and communication competence of students.

1 cl, 5 dwg, 5 tbl

Description

The invention relates to the field of tools (assessment systems) that determine the information and communication competencies of students, and more specifically, to a method for determining the readiness of students and graduates of educational institutions to use digital information and communication technologies in order to access information, determine it, evaluate, manage, integration, creation and transfer, observing ethical and legal standards.

Similar testing systems are known that are associated with the assessment of ownership of information and communication technologies, which are aimed at determining the formation of technological skills and algorithmic thinking. An ICL test allows you to determine the level of cognitive skills needed for a successful life in a post-industrial society.

The number of instruments measuring information and communication competence, information literacy and related constructs (media literacy, digital competency - [12], [14], [24], [31], [35]) exceeds 50 ([25], [34] , [37], the most recent review is [28]). The development of these tools was carried out by specialists in various disciplines - library science, information science, computer science, applied research and psychometrics. Most of these tools measure technical knowledge and skills about computer, networks, basic programs ([23]), some of them are information literacy outside the digital context ([13]), there are tools that measure both technical and cognitive skills ([ 6], [7], [20], [21], [28], [30], [36]), in most of which the authors deliberately combine cognitive and technical skills in one scale, in some of them ([28] ), contrary to development trends, the classical theory of tests is used.

Unlike the above tools, the ICL test allows you to measure the cognitive skills of working with information, it uses a systematic approach to the development of test tasks - a method based on a system of evidence ([32], [33]). The main advantage of this approach is its consistency, clarity and reversibility of logic - the developer of the test task always understands how concrete evidence is associated with the measured construct, and, therefore, as latent (not directly measured) variables (information and communication competence and its components) are associated with their observed indicators (evidence), which confirms the high design validity of the ICL test. In addition, a systematic approach to the development of test tasks allows the use of Bayesian networks for automatic processing of results and the test person receives their results immediately after passing the test on a computer screen. In the USA, the ETS (Educational Testing Service) company developed the iSkills assessment tool [[26], [27], which measures IR competency in the digital environment among students of American universities, while the measurement is free of technical competencies of students. In this system, an automatic assessment of IR competency the IRT method is used to process the test results ([22]), there is no automatic processing of the results with the presentation of test results immediately after the test, there is no consistency with Russian educational standards by mouths; the test is designed to test a different target audience, similar to the ICL test of automatic assessment of cognitive skills for working with information (information and communication competence and other similar constructs) among high school students (and other target groups) in the Russian system education not developed.

ICL test is a system for the automatic assessment of information and communication (IR) competence of 13-15-year-old students of a general educational organization ([1], [2] ([1]).

The system of automatic assessment of students' IR competency consists of (Fig. 1):

1. Test development systems.

2. Systems for delivering the test to the test person.

3. Systems for presenting a test variant to a testee.

4. Systems for processing test results.

5. Libraries of test tasks, test results, questionnaires and answers to questionnaire questions, etc.

Test Development System

Each task begins with a short text describing the general context of the situation and designed to bring the test task closer to real life. On the following screens, the task itself is presented (an example in Fig. 2): on the left side of the screen, the final goal of the task is formulated and all necessary instructions are given, on the right are the task material and tools, presented as an imitation of widespread programs, such as a web browser, text editor , spreadsheet, database, etc.

The system of automatic assessment of IR competence is focused on a certain age of the target audience (13-15 years), therefore, the developers were faced with the task of formulating tasks in such a way that students did not have problems with their perception and understanding.

In world practice, when evaluating the quality of a text, the concept of “readability” is used, which is interpreted as comprehensibility, readability or readability. Sometimes the graphic form of presentation of the material (fonts, format, style) is also included in this concept, but in the first place it means the verbal form, that is, the text itself, the words and the syntax.

Readability assessment methods are based on the number of syllables in a word, words in a sentence, sentences in a paragraph, etc. A quantitative assessment of these parameters allows you to determine the degree of comprehensibility of the text, compliance with age and educational level (class) for which the text is designed. To verify the tasks of the ICL test, the Flesch Reading Ease formula ([18], [19]) was used, which predicts the ease of reading, and the Flash-Kincaid formula ([29]) converts this grade into the level of education necessary to understand the grade text. Since the formulas were developed for texts in English, the coefficients for the Russian language were adjusted to take into account the longer average length of words and sentences ([3]).

During the study, all test tasks in Russian and English were checked. Table 1 shows the share of test tasks in accordance with the age and level of education that is necessary for their easy understanding. The analysis showed that the texts of at least 90% of the test items correspond to the target age group (graduates of the main secondary school) or are younger. Therefore, the texts of the test case scripts used to form the ICL test variant have a high level of accessibility for the stated target group with different levels of reader literacy.

Test delivery system

The system for delivering a test to a testee provides three possibilities for performing test tasks:

1. Testing in on-line mode. This mode is possible when the testing site has an Internet channel of at least 256 Kb / s. In this case, the student will immediately be able to see his results and receive recommendations for improving his IR competency immediately after passing the test. All test results are collected on a central server and can be used for further research and analysis.

2. Testing using a local server. This mode is used when the bandwidth of the Internet channel is insufficient, but there is a local area network in the room where the test is being conducted. In this case, the local server is deployed on which the ICL test is installed, and the test person receives the results and recommendations immediately after the test is completed.

3. Testing using flash drives. This mode is used only in cases where in the room where the testing takes place there are only individual computers that are not connected to the Internet and are not connected to the local network. After passing the test, the results are transmitted to the server, there they are processed and placed in a common database of results. In this case, the tested receive their results and recommendations only after processing the data on the server.

Different methods of testing do not affect the resulting assessment of the level of IR competence and recommendations, therefore, the results obtained in different ways can be considered completely equivalent.

System for presenting test variant to testee

The test assessing IR competence includes 16 scenario-type test tasks (13 simple, 2 medium and 1 difficult), and the test itself takes 2 academic hours. Such a structure, combining simple, medium and complex tasks, requires the use of cognitive strategies and the need to comply with the conditional independence of test tasks to be quantified.

Given that the purpose of the assessment is to check the formation of cognitive (rather than technical) skills, each test combines the necessary number of cognitive and technical elements. Naturally, more complex test items test more complex cognitive skills than simple tasks that evaluate one of the components of IR competency.

Test tasks involve the use of a wide range of digital technologies, including Internet applications, databases, modeling environments, multimedia, spreadsheets, etc. The task context may be educational or may be related to a real life situation. Serious learning tasks alternate in the test with tasks that allow you to evaluate the experience of those tested in the use of ICT, their knowledge of modern culture and the ability to solve practical problems.

The structure, configuration, and one of the possible test specification options are presented in Table 2.

Before testing, an oral instruction from the testing administrator follows, a screen with key testing rules is shown, and it is proposed to perform three simple tasks to familiarize yourself with the test interface. As a rule, upon completion of testing, but before presenting preliminary results, students are asked to fill out a questionnaire with various questions about studying and using computers at home and at school, which is necessary to study the factors of formation of IR competence. The questionnaire may also include various methods for studying the validity of the test (see below).

Skills and knowledge about specific computer programs are not required to complete the test task, except for the most basic ones, such as selecting from the drop-down list, dragging and dropping elements and others, but their application is described in the Help section. Thus, the system of automatic assessment of IR competency - the ICL test is also available to test takers, who are actually not familiar with the computer. During the task, depending on its duration, one or more simulations of software applications can be used, the “windows” of various applications can be used with the ability to switch between them.

The testee himself decides when to consider the task completed. The top panel displays the number of remaining tasks and the "Next" button, which allows you to interrupt the task and go to the next. If the test person has not even started to execute, an additional warning is displayed to avoid “clicking” the test.

Test Results Processing System

Each test task is aimed at measuring one or more components of IR competency. Actions and combinations of actions of the test person are recorded in the database. The actions of the testee are processed automatically, and upon completion of all tasks or after the testing time has passed, the testee is presented with a screen with its result and, if it is not the highest, is accompanied by recommendations on how to increase his level of IR competence. Recommendations on the individual components of IR competency are presented depending on the level of each component of IR competency.

The results processing system uses the capabilities of a systematic approach to developing the development of test tasks (a method based on the evidence system - [32], [33]). Despite the fact that the test presented to the test person consists of 16 test tasks, the assessment of his level of IR competency occurs using a significantly larger number of indicators (evidence). These indicators are called observable variables, their total number is 60-67, depending on which tasks are used in a particular test case. Each task includes from 3 to 7 observed variables that evaluate certain aspects of the task. Each observed variable has 3 levels characterizing low, medium or high compliance of the activity of the test person with the given criteria. The test is assessed according to 7 components of IR competence (a list of all components of IR competency is shown in Fig. 3), each of which also has 3 levels: unacceptable, acceptable and competent. Medium and complex test cases include evidence of several components of IR competency.

The most common model for evaluating test results, the modern theory of tests (IRT - [6], [17], [38]), for several reasons, is not suitable for solving this problem. Firstly, the latent construct is multidimensional, since the IR competency has seven interconnected components. Secondly, since the observed variables are grouped into test tasks, the problem of local dependence is embedded in the test design itself. As part of the IRT, there are models for working with multidimensional constructs (multidimensional models), there are also models that allow you to control local dependence (facet models). However, in the case when the data is both multidimensional and has groups of locally dependent test tasks, IRT does not allow for completely correct calculation of the results. Finally, all latent variables in this case are discrete, while IRT is aimed at obtaining a continuous latent variable.

To solve these problems, Bayesian networks are used to evaluate test results in the ICL test ([8], [9]. [40]). Bayesian network is a directed acyclic graph that reflects causal relationships between observable and latent variables (components of IR competence, an example in Fig. 4). It is convenient to represent Bayesian networks in the form of block diagrams in which latent variables are independent variables and observables are dependent. The relations between the variables are described by conditional probability tables, which can be set using mathematical functions. For example, a simple graph in which there is one latent variable, and the relations between it and each of the dichotomous observable variables can be described by a logistic function, which will be similar to a typical IRT model. In Bayesian networks, one can supplement these relationships with any others, for example, between observable variables, to reflect local dependence. The Bayesian nature of networks is expressed in the fact that initially each node of this network has an a priori value, which then, using the data obtained, is updated in accordance with the Bayes formula. An important advantage of Bayesian networks is that they are well combined with a systematic approach to creating tests, since they reflect causal relationships, which means it is easy to trace the chain from evidence (observed variable) to a specific level of the measured construct (IR competency and its components) .

The description of the relationships between variables in the ICL test is described by the Samejim-Dibello logistic function ([9], [32]). This approach differs from the classical IRT in that it uses the “effective theta” method to discrete the latent variable, that is, the distribution of the latent variable measured in the logits (component of IR competency) is divided by quartiles and after rounding corresponds to -1 for an unacceptable level, 0 for an acceptable and 1 for competent.

Since the components of IR competence are defined by three levels, two steps of correctness are required: the transition from low to medium, and from medium to high. Therefore, for each node of the observed variable, two difficulty parameters were used.

Pairs of latent and observed variables nodes make up a large part of the network, but in addition to them, we also use an additional node to take into account the common component of 7 components of the KIC, that is, the KIC itself. Since the results for 7 components are highly correlated, this node is needed in order to “absorb” their total dispersion. This node has three discrete states for convenience, and the conditional probability tables between it and the components of IR competence are filled out by experts.

Bayesian networks are integrated into the ICL test result processing system, causal relationships are explicit in the model, which ensures the conceptual integrity of the calculation system and the overall test framework.

One of the methods for analyzing the accuracy of network functioning is based on comparing “true” estimates with those predicted by the network, which is sometimes called a statistical experiment. To do this, the data was simulated that fully corresponded to the parameters laid down in the network, and at the next stage, only the values of the observed variables were introduced into the network and the proportion of correctly predicted values of the latent variables (components of the CQI) was calculated. This procedure was repeated thousands of times and as a result, the frequency of correct classification into one of the levels of each component of the KIC was calculated. In addition, the average probability of correct predictions was calculated. A comparison of true and predicted values allows you to judge how accurately the model works. As can be seen from table 3, the proportion of correctly predicted values is high and ranges from 0.87 to 0.90, the same can be said about the average probability of correct predictions. This indicates the high accuracy of the Bayesian network, used to calculate the test results, according to the assessment of IR competence.

Thus, the main technical result of the claimed system of automatic assessment is to increase the accuracy and reliability of the assessment of the results of testing the IR competence of students.

ICL test validity

Analysis of the test results was based on the concept of validity, which is defined in the Standards for Educational and Psychological Testing: “Validity is a characteristic of how much empirical data and theoretical constructions support an adequate interpretation of measurement results."

Several procedures were performed to study the validity of the ICL test. One of the sources of constructive validity is the systematic approach to the development of the test itself. Since the ICL test was developed on the basis of generalized standards, a study was conducted on its compliance with Russian and international educational standards.

Alignment with educational standards (alignment)

The test's compliance with educational standards was determined using a special procedure developed by Webb ([39]). Consistency is understood as the degree of coincidence of the predicted results of the development of the educational program and their estimates obtained with the help of test tools, as well as the degree of their correspondence to each other. The test, consistent with educational standards, allows you to use it productively in order to assess the effectiveness of both the entire education system and specific teaching methods. Consistency is essentially evidence of the substantive validity of the test.

The IR competency test was checked for consistency with three types of educational standards: the American Library Association's ICC Standards for Higher Education ([10]), partly based on the test framework, the ETS general standards for testing quality and fairness ([16]), and part of the new the Russian Federal State Educational Standard ([4]) basic general education related to ICT, as well as its refinement in the form of the Model basic educational program of an educational institution ([5]). On the test side, the theoretical framework, specification, bank of observable variables (evidence), and texts of test assignments participated in the comparison.

The procedure for determining Webb consistency is to compare each of the standards with the relevant parts of the test. For each part Webb developed a special criterion. The results represent the proportion of standards to which compliance was found in the test. Often, it is not possible to calculate a specific share, since educational standards can be narrative, which forces the use of “quality” criteria and is based on the opinions of experts and developers. In this case, the result is one of the possible meanings: “weak”, “acceptable” or “full” consistency.

Webb identified several groups of criteria for checking consistency: substantive, understanding of progress, equality and justice, pedagogical conclusions and applicability to the education system. Each of the criteria indicates a portion of the documentation that has been compared. For example, when using the criterion “conformity of the general measured concept”, the definitions of the measured construct were subjected to comparison.

The ICC standards for higher education of the American Library Association consist of five parts, according to which an ICT-competent student should be able to assess the sufficiency of information, receive it effectively, critically evaluate information and its sources, embed information in their own knowledge, etc. For most criteria, consistency is complete or acceptable. The weak coincidence of the knowledge range is explained by the fact that this criterion involves comparing a specific test case with all standards. Of course, one test option cannot meet all of the listed standards, due to the limited time and number of tasks. A complete task bank covers up to 75% of indicators in the standards.

In accordance with the “Sample main educational program of an educational institution” developed within the framework of the requirements of the Federal State Educational Standard, ICT competency includes not only the student’s ability to work with information, but also the technical aspect, which relates to the skills of working with certain equipment, software, etc. ., while in the framework of the ICL-test tool, technical competence is absent. This explains the incomplete consistency of standards and test. At the same time, the found levels of consistency are high and acceptable by all criteria.

Separately, it is worth mentioning the criterion of equality and justice, the consistency of which was based on other documents: “Quality and Justice” and the Russian Law “On Education” since other documents do not contain this information. Incomplete compliance with the ETS standard is explained by the fact that the test is not suitable for students with low vision. The rest is full compliance. One of the confirmations of this correspondence is the results of the DIF analysis.

Table 4 summarizes the results of the consistency study.

So, the test is to an acceptable degree consistent with both Russian and international educational standards, which suggests that it is ready to be used directly in the Russian education system and, after some adaptation, in the international educational market.

Validation studies using external variables

To check for convergent, divergent and current validity, 4 questionnaires are used, which are built into the social questionnaire, which the testers fill out after the test.

Using correlation analysis, the results of these questionnaires are compared with the test results. The general logic of this analysis is that if the test really measures the desired construct (IR competency), then its results should positively correlate with the self-assessment of the KIC, since it to some extent reflects the real level of the KKI, and with the grades at school, since the KKI is a meta-subject Competency should help in learning, and should not be related to technical literacy, since this is one of the key criteria for determining the KIC in contrast to similar constructs.

If the IR competency test really measures the indicated construct, its results should be moderately and positively related to the students themselves evaluating their IR competency. According to the results of other studies, the correlation coefficient between them should be from 0.20 to 0.40.

The average grade for subjects in the school is a traditional object of analysis in validation studies of tests related to education. Students who receive high ICL scores have cognitive abilities that also allow them to study well. This hypothesis is supported by the neuropsychological and genetic theoretical framework of general intelligence.

The questionnaire on the frequency of accessing IR competency skills consists of questions about how much time the respondent spends at the computer, doing various types of activities. It was expected that the frequency of work at the computer would not be connected with the KIC, which is primarily a measure of the ability to work with information that requires support from teachers, parents, and peer help. The expected correlation coefficient is from -0.20 to 0.20.

The technical literacy questionnaire consisted of questions about specific functions of software packages and other technical parameters. The system of automated assessment of IR competence was created in such a way as not to require any special technical knowledge from the respondents, therefore, here we also expect a low connection (correlation coefficient from -0.10 to 0.10).

Table 5 presents the correlations of the results of the ICL test with external variables and the characteristics of the scales. Reliability in this case was measured using the Cronbach alpha coefficient ([15]).

Satisfactory values for psychological tests lie in the range 0.60-0.95 (large values indicate greater internal consistency). From the data obtained it follows that within the framework of each of the questionnaires, the included questions measure the same construct.

The quality control of additional questionnaires was carried out by analyzing all scales by the IRT model (partial credit model), issues with unsatisfactory parameters were not considered when calculating the final score. The study showed that all additional scales have high agreement with the model, have high reliability, the answer categories of questions work correctly.

As can be seen from table 5, all correlation coefficients are within the standard errors of the expected values. All correlation coefficients are significant at the level of 0.05. A moderate correlation of the IR competency test score with the level of confidence in one's own IR competency speaks in favor of the convergent validity of the test results. A close to zero relationship with technical literacy and the frequency of referring to IR competency skills speaks in favor of divergent validity. The connection with the average grade for school subjects speaks in favor of the current validity of the results of the ICL test.

Thus, the technical result of this assessment system is confirmed - high accuracy and reliability of the measurement results.

Summary

Studies of the ICL test have confirmed a number of its advantages in relation to other tools for assessing ICT skills:

- assesses the ability to effectively work with information, solve practical problems using the capabilities of ICT, and not the level of technological skills in ICT;

- IC competency as a whole and its components are evaluated;

- test tasks are based on real life or educational situations;

- ICL test can be used on any computer, regardless of the operating system and the availability of Internet access;

- The ICL test can be used to conduct both comparative and longitudinal (repeated slice) studies;

- The ICL test is one of the few tools that allows you to solve the problem of evaluating meta-subject results taking into account the requirements of the new FSES and the only one in Russia that allows you to assess ICT competency ([4]).

The data obtained using the ICL test allow not only to fix the level of IR competence of graduates of a primary school with high accuracy and reliability, but also to identify factors that influence its formation and its dynamics when conducting longitudinal studies, and also provide grounds for recommendations to educational organizations who took part in testing and the educational system as a whole to increase the level of IR competence of students.

The ICL test can be used to conduct both national and regional monitoring studies on the effectiveness of implementing various aspects of educational development programs, especially in the field of introducing new GEF, informatization of education, assessing educational results, etc. It is also suitable for in-school monitoring related to the quality of the educational process, changes in the forms and methods of educational work, and the effectiveness of work in the educational information environment.

Literature:

1. Avdeeva S.M., Baryshnikova M.Yu., Kovalenko S.K., Melnikov A.E. On the experience of implementing a model for assessing ICT competence // Journal "Informatization of Education and Science". - 2009. - No. 2.

2. Avdeeva S.M., Khudenko L.A. The study of information and communication competence of students in the 9th grade of the Republic of Belarus: goal and main results // Journal "Education Herald". - 2014. - No. 1, pp. 3-10.

3. Oborneva I.V. Automated assessment of the complexity of educational texts based on statistical parameters: Dis. Cand. ped Sciences: 13.00.02 Moscow, 2006.

4. The Federal State Educational Standard of Basic General Education (approved by order of the Ministry of Education and Science of the Russian Federation of December 17, 2010 No. 1897), 50 pp., (Http://www.edu.ru/db/mo/Data/d_10 /prm1897-1.pdf).

5. Kondakov A.M., Kezina L.P. An approximate basic educational program of an educational institution. Basic school. Institute for Strategic Studies in Education of the Russian Academy of Education, Moscow, Education, 2011, 454 pp., (Http://window.edu.ru/resource/594/75594/files/Programma_5_9.pdf).

6. Aesaert, K., Van Nijlen, D., Vanderlinde, R., & van Braak, J. (2014) Direct measures of digital information processing and communication skills in primary education: using item response theory for the development and validation of an ICT competence scale // Computers & Education. No. 76. P. 168-181.

7. Ainley, J., Fraillon, J., & Freeman, C. (2007) National Assessment Program. ICT Literacy Years 6 & 10 Report, 2005. Ministerial Council on Education, Employment, Training and Youth Affairs (NJ1).

8. Almond, R.G. (2010) I can name that Bayesian network in two matrixes // International Journal of Approximate Reasoning. No 51. P. 167

9. Almond, R. G., DiBello, L., Jenkins, F., Mislevy, R. J., Senturk, D., Steinberg, L., et al. (2001) Models for conditional probability tables in educational assessment. Artificial intelligence and statistics 2001.P. 137-143. San Mateo, CA: Morgan Kaufmann.

10. American Library Association. (2000) Information literacy competency standards for higher education.

11. Avdeeva S., Zelman M. (2011) ICT Competence Assessment Instrument and Its Use. The International Association for Educational Assessment (IAEA), 36th annual conference. Assessment for the future generations, Bangkok, Thailand, report, 2011.

12. Brown, J. (1998) Media literacy perspectives // Journal of Communication. No 48. P. 44-57.

13. Cameron, L., Wise, S. L., & Lottridge, S. M. (2007) The development and validation of the information literacy test // College & Research Libraries. Vol. 68.No 3. P. 229-237.

14. Celot, P., Tornero, JMP (2009) A comprehensive view of the concept of media literacy and understanding of how media literacy levels in Europe should be assessed // Study on Assessment Criteria for Media Lioteracy Levels / For the European Commission Directorate General Information Society and Media; Media Literacy Unit.

15. Cronbach, L.J. (1971) Test validation / In R.L. Thorndike (Ed.), Educational measurement (2nd edition) Washington, DC: American Council on Education. P. 443-507.

16. Educational Testing Service (2002) ETS Standards for Equality and Fairness. ETS, Princeton.

17. Embretson, S.E., & Reise, S.P. (2013) Item response theory. Psychology Press.

18. Flesch, R. (1974) The art of readable writing. New York: Harper & Row.

19. Flesch, R. (1981) Why Johnny still can't read: A new look at the scandal of our schools. New York: Harper & Row.

20. Fraillon, J., & Ainley, J. (2010) The IEA international study of computer and information literacy (ICILS). Retrieved April, 20, 2010.

21. Fraillon, J., Schulz, W., & Ainley, J. (2013) International Computer and Information Literacy Study: Assessment framework. Amsterdam: IEA.

22. Hambleton, R.K., Swaminathan, H., Jane Rogers, H.J. (1991) Fundamentals of Item Response Theory, Measurement Methods for the Social Science, SAGE Publications.

23. Hohlfeld, T.N., Ritzhaupt, A.D., & Barron, A.E. (2010). Development and validation of the Student Tool for Technology Literacy (ST2L) // Journal of Research on Technology in Education. Vol. 42.No 4. P. 361-389.

24. Jenkins, H. (2009) Confronting the challenges of participatory culture: Media education for the 21st century. MIT Press.

25. Johnston, B., & Webber, S. (2003) Information Literacy in Higher Education: A review and case study // Studies in Higher Education. Vol. 28, Iss. 3. P. 335-352.

26. Katz, I.R. (2013) Testing information literacy in digital environments: ETS's iSkills assessment. Information technology and Libraries, 26 (3), 3-12.

27. Katz, I.R., & Macklin, A.S. (2007) Information and communication technology (ICT) literacy: Integration and assessment in higher education. // Systemics, Cybernetics and Informatics. Vol. 5. No. 4. P. 50-55. (http://www.iiisci.org/Journal/CV$/sci/pdfs/P890541.pdf).

28. Kim, H-S., Kil, H-J., Shin, A. (2014) An analysis of variables affecting the ICT literacy level of Korean elementary school students // Computers & Education. Vol. 77. P. 29-38.

29. Kincaid, J.P., Gamble, L.G. (1977) Ease of comprehension of standard and readable insurance policies as a function of reading ability // Journal of Reading Behavior. Vol. 9.No 1. P. 87-95.

30. Lennon, M., Kirsch, I., Von Davier, M., Wagner, M., & Yamamoto, K. (2003) Feasibility Study for the PISA ICT Literacy Assessment: Report to Network A. Educational Testing Service.

31. Livingstone, S., Thumim N. (2003) Assessing the media literacy of UK adults / A review of Academic Literature produced for Broadcasting Standards Commission Independent Television Commission NIACE (http://www.ofcom.org.uk/static/ archive / itc / uploads / Assessing_the_media_literacy_of_UK_adults1.pdf).

32. Mislevy, R.J., Almond, R., Dibello, L., Jenkins F., Steinberg, L. (2002) Modeling Conditional Probabilities in Complex Educational Assessments. CSE Technical Report.

33. Mislevy, R.J., Levy, R. (2007) Bayesian psychometric modeling from an evidence-centered design perspective / In: Rao, C.R., Sinharay, S. eds. (2007) Handbook of statistics. Elsevier, Amsterdam. P. 839-865.

34. Rader, H.B. (2002) Information literacy 1973-2002: A selected literature review // Library trends. Vol. 51.No 2. P. 242-259.

35. UNESCO Communication and Information Sector, UNESCO Institute for Statistics (2013) Global Media and Information Literacy Assessment Framework: Country Readiness and Competencies. United Nations Educational, Scientific and Cultural Organization. Paris, France.

36. Van Joolingen, W. (2004) The PISA framework for assessment of ICT literacy. (http://www.ictliteracy.info/rf.pdf/PISA%20framework.ppt.)

37. Virkus, S. (2003) Information literacy in Europe: a literature review // Information Research Vol. 8 No. 4.

38. Wang, W.C. & Wilson, M. (2005) The Rasch Testlet Model. Applied Psychological Measurement.

39. Webb, N.L. (1997) Criteria for Alignment of Expectations and Assessments in Mathematics and Science Education. Research monograph No. 8.

40. Wick, R., Thayer, D.T., Lewis, C. (1999) An Empirical Bayes Approach to Mantel-Haenszel DIF Analysis // Journal of Educational Measurement, Vol. 36. No LP. 1-28.

* Consistency with the general standards of testing "Quality and Fairness" [ETS.]

** Consistency with federal state standards of basic general education

* Correlation coefficients are significant at the 0.05 level .

Claims (1)

The system of automatic assessment of student’s information and communication competency, consisting of a test task development system containing a server, interacting via a test delivery system for the test person, consisting of channels and flash drives, with a system for presenting a test variant to the test subject, containing a computer interacting with the results processing system a test containing a server, as well as a library of test tasks, test results, questionnaires and answers to questionnaire questions, representing This is a database associated with all the mentioned systems, while the system for processing test results is capable of analyzing the validity and reliability of test results, and each test task includes from 3 to 7 observable variables, each of which has 3 levels characterizing low, medium or high compliance of the activity of the tested with the given criteria, the assessment of the systematic processing of the test results takes place according to 7 components of information and communication competence, each of which also has a 3 levels: unacceptable, acceptable and competent, while for calculating the test results the test results processing system uses Bayesian networks, which are a directed acyclic graph that reflects causal relationships between observed and latent variables that make up information and communication competence, and a description of the relationships between variables in the test is described by the Samejim-Dibello logistic function, which provides a discretization of the latent variable.

Images