KR101946551B1 - Server for generating diagnosis indicator to evaluate university administration service - Google Patents

Abstract

According to the present invention, a method for creating a diagnosis indicator to evaluate a university administration service can create a diagnosis indicator based on a social issues centered diagnosis indicator, a data-centered diagnosis indicator, and a future-oriented diagnosis indicator rather than a past-oriented diagnosis indicator or a theoretical-oriented diagnosis indicator and increase the reliability and validity of diagnostic indicators by configuring more realistic problem-driven centered survey questions reflecting big data-based keywords and social issues when organizing the survey questions related to the university administration service and creating a diagnosis indicator to evaluate the university administration service based on the survey questions.

Description

TECHNICAL FIELD [0001] The present invention relates to a server and a method for generating a diagnostic indicator for a university administration service evaluation,

The present invention relates to a questionnaire, and more particularly, in the generation of diagnostic indicators for the evaluation of university administration services, in the construction of questionnaires related to university administration services, a more realistic and problem- To create diagnostic indicators for the evaluation of university administration services based on the questionnaire items, so as to generate social issue-oriented, data-oriented, and future-oriented diagnostic indicators instead of past-oriented or theoretical-oriented diagnostic indicators. The present invention relates to a diagnostic indicator generation server and method for evaluation of administrative services.

In recent years, colleges are changing from a seller-centric market to a buyer-centric market with a significant change in the educational environment. Nowadays, the importance of administrative services that support and activate not only education service which is a core service provision area of university is higher than ever.

However, in the past, the development and verification studies of the evaluation index (questionnaire item), which is an important means to measure the degree of satisfaction of the administrative service of the university, are insufficient.

In other words, previous studies for measuring the satisfaction of existing university administration services suggest that the SERVQUAL model is modified by many parts of the study. In this case, the measurement part of the service expectation is omitted and Parasuraman et al. (1988 , 1990). However, the specificity of specific organizations such as universities is not considered.

In the case of universities, for example, the structural specificity of a university organization may be similar, but the way the university operates and the way of providing services for students who are core customers may differ. Therefore, It may not be accurate to determine the strategic direction from the results.

Therefore, a more reliable diagnosis index that allows the university's structural and functional characteristics to be taken into account to more accurately measure the satisfaction level of the university's administrative service, and to accurately diagnose the satisfaction of the customer's needs and the state of the administrative service in the current situation (Questionnaire item) has been demanded.

(Patent Literature)

Korean Patent Laid-open No. 10-2007-0064885 (Published date July 18, 2007)

Accordingly, in the generation of the diagnostic indicators for the evaluation of the university administration service in the present invention, in the generation of the diagnostic indicators for the evaluation of the university administration service, in the construction of the questionnaire items related to the university administration service, The results of this study are as follows. First, the questionnaire was composed of more realistic and problem - solving questionnaires to reflect on the questionnaires, And to provide a diagnostic indicator generation server and method for evaluating administrative services that can generate future-oriented diagnostic indicators.

The present invention provides a diagnostic indicator generation server for evaluation of university administration service, which provides a first questionnaire related to evaluation of university administration service to a first terminal used by a predefined evaluator, Wherein at least two first answers to the question items are inputted from the first terminal, a part of the plurality of questionnaire items is selected based on the distribution of the inputted first answer, And the second answer to the selected question is inputted from the second terminal, the evaluation of the administrative service is performed based on the inputted second answer. .

In addition, in the case where the diagnosis index generation server selects a part of the plurality of questionnaire items based on the distribution of the input first answer, the distributed value of the distribution of the first answer input is calculated for each of the plurality of questionnaire items And selecting the part of the plurality of questionnaire items based on whether the calculated variance value is lower than a preset reference.

Further, the present invention provides a diagnostic index generation server for evaluation of university administration service, which provides an initial questionnaire related to evaluation of university administration service to a first terminal used by a predefined evaluator, A diagnostic index generator for selecting at least two of the plurality of question items on the basis of the distribution of the input first answers when at least two first answers to the question items are inputted from the first terminal; A questionnaire constructing unit for constructing a final questionnaire with the selected questionnaire items among the questionnaire items of the first questionnaire by receiving information on questionnaire items selected from the generating unit, 2 terminal, and if a second answer to the final questionnaire is input from the second terminal, And an evaluator selection unit for selecting the preliminary evaluator to answer the initial questionnaire or the final questionnaire and the evaluator inside or outside the university.

The diagnostic index generator may be configured to select a part of the plurality of question items based on the distribution of the input first answer, and to calculate a variance value of the distribution of the first answer inputted for each of the plurality of question items And selecting the part of the plurality of questionnaire items based on whether the calculated variance value is lower than a preset reference.

According to another aspect of the present invention, there is provided a method of generating a diagnostic indicator for evaluating a university administration service, the method comprising: providing an initial questionnaire related to evaluation of a university administration service to a first terminal used by a predefined evaluator; Selecting at least two of the plurality of question items based on the distribution of the first answer when at least two first answers for each of the plurality of question items are input from the first terminal; A final questionnaire for evaluating the university administration service as an item; and providing the final questionnaire to a second terminal used by an internal or external evaluator of a university, and a second answer to the final questionnaire is provided for the second And performing an evaluation on the administrative service based on the input second answer, if the input is received from the terminal.

The selecting may further include: calculating a variance value of the distribution of the first answers received for each of the plurality of question items; determining whether the calculated variance value is lower than a predetermined criterion, And selecting the part of the questionnaire.

According to the present invention, in the generation of the diagnostic indicators for the evaluation of the university administration service, in the generation of the diagnostic indicators for the evaluation of the university administration service, the keywords and the social issues of the big data- The results of this study are as follows. First, the questionnaire was composed of more realistic and problem - solving questionnaires to reflect on the questionnaires, It is possible to generate a future-oriented diagnosis index, and the reliability and validity of the diagnosis index can be improved.

1 is a configuration diagram of a diagnostic indicator generation system for evaluation of administrative services according to an embodiment of the present invention;
2 is a detailed block diagram of a diagnostic indicator generation server according to an embodiment of the present invention;
3 is a flowchart of operation control for generating a diagnostic index according to an embodiment of the present invention.

Hereinafter, the operation principle of the present invention will be described in detail with reference to the accompanying drawings. In the following description of the present invention, detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear. The following terms are defined in consideration of the functions of the present invention, and these may be changed according to the intention of the user, the operator, or the like. Therefore, the definition should be based on the contents throughout this specification.

1 shows a configuration of a diagnostic indicator generation system for evaluation of administrative services according to an embodiment of the present invention.

Hereinafter, operation of each component of the diagnostic indicator generation system for evaluating satisfaction of the university administration service of the present invention will be described in detail with reference to FIG.

First, the evaluator terminal 100 is connected to the system through a communication network such as the Internet, participates in a university administration service survey conducted by the diagnostic index generation server 150, (Smartphone), a tablet PC, a notebook, a desktop PC, etc., which can be connected to a communication network, and the like. But is not limited thereto.

Further, in the embodiment of the present invention, the evaluator responding to the questionnaire responds to the questionnaire items included in the questionnaire through the evaluator terminal 100, but the present invention is not limited thereto. That is, the evaluator may receive the questionnaire as a printed matter and respond with a handwritten response, and such response may be input by an administrator or the like. Such an evaluator may be, for example, but is not limited to a faculty member, a professor, a lecturer, a student, an administrative assistant, a bachelor's assistant, a local resident,

The administrator terminal 130 is a device for accessing the diagnostic index generation server 150 through a communication network such as the Internet and registering the questionnaire items of the questionnaire related to the university administration service evaluation. The administrator terminal 130 selects an evaluator Or management of a university administration service evaluation information according to a questionnaire response, and the like. In addition, the administrator who operates the administrator terminal 130 may be one of the experts of the administrative administration group operating the university administration, but is not limited thereto.

The diagnostic index generation server 150 may generate a diagnostic test report for the first questionnaire registered when the first questionnaire, which is the first questionnaire without the pilot test or the pre-test for the university administration service evaluation, A pilot test is firstly performed through the evaluator of the first group selected from all the evaluators.

At this time, the first questionnaire can be a questionnaire including a plurality of questionnaire items for evaluating the university administration service. In addition, these questionnaires can be composed of a plurality of questionnaire items including keywords having relevance to university administration service higher than the standard value through keyword analysis of big data based on the questionnaire items related to evaluation of university administration service . Therefore, it is possible to construct a more realistic and problem-solving questionnaire that reflects big data-based keywords and social issues, and by generating diagnostic indicators for the evaluation of university administration services based on the questionnaire items, It is possible to generate social-issue-oriented, data-oriented, and future-oriented diagnostic indicators instead of theoretical-oriented diagnostic indicators.

Hereinafter, keyword extraction and survey item construction related to university administration service based on Big data will be described in more detail.

First, in the keyword extraction, the word related to the administration service of the university can be extracted from the web portal, the blog, etc. In the keyword extraction, for example, two methods can be used.

The first method is to use a machine learning technique to learn a machine learning device that extracts text related to university administration services (here, CNN can be used as a machine learning technique, CNN is a public technology A detailed description thereof will be omitted), and the articles, blogs, etc. of the web portal can be sorted using the learned machine learning apparatus. Then, the keyword that is the most frequent in the culled article or blog can be selected as the keyword. At this time, in the keyword selection, a Tf-Idf (Term Frequency-Inverse Document Frequency) method can be used.

The Tf-Idf is a weight value used in information retrieval and text mining. It represents a statistical value indicating how important a word is in a specific document, that is, a document group composed of several documents. It can be used to determine the ranking of search results in the engine, to obtain similarity between documents, and so on. Since the Tf-Idf technology is a known technology, a detailed description will be omitted.

In the second method, students can create questions and ques- tions on the homepage of the university to store questions and inquiries, and select the keywords most frequently appearing in the query. At this time, the Tf-Idf method can be similarly used for keyword selection.

Once the keywords are selected as described above, the questionnaire is composed of questionnaires with high relevance to the administrative service, using the selected keywords. That is, for example, set the priority of the culled keyword, assign a high score to a high priority and a low score to a low priority. Then, based on the keywords included in the questionnaire items, the score for each questionnaire is calculated, and the questionnaire can be composed of the questionnaire items having high relevance to the administrative service by filtering according to the order of the questionnaire items having high scores.

In addition, these question items are categorized into categories such as administrative service quality, administrative service delivery, and administrative service environment, and within each category, questions such as convenience, expertise, innovation, responsiveness, reliability, agility, , Accessibility, etc., and can be composed of a predetermined number of question items in each sub-item, but the present invention is not limited thereto.

In addition, the pilot test means that before conducting the questionnaire to the entire questionnaire, the plurality of questionnaire items included in the first questionnaire are evaluated in advance as appropriate questions to evaluate the university administration service, A test for selecting only the question items satisfying the preset first criterion can be said. In addition, the evaluator of the first group may refer to a part of the evaluators selected for the pilot test among the entire evaluators. The evaluator of the first group may be an evaluator selected from among the in-school evaluator selected by a professor, an employee, But is not limited thereto.

After the pilot test, the content validity of each questionnaire included in the first questionnaire is evaluated. Then, only the questionnaire items satisfying the predetermined first criterion are selected based on the content validity evaluation to construct the second questionnaire.

At this time, in the evaluation of the validity of the question of the pilot test, the diagnostic index generation server 150 transmits the initial questionnaire related to the evaluation of the university administration service, for example, to the first questionnaire used by the first group of evaluators, The method of claim 1, further comprising: providing at least two first answers to each of the plurality of question items included in the first questionnaire from the first terminal, And a second questionnaire can be constructed by selecting some of them. In this case, the first terminal may be a terminal used by the first group of evaluators selected for the pilot test among the evaluator terminals 100, and the first terminal may constitute a part of all the evaluator terminals 100. Also, the second terminal may be a terminal used by an internal or external evaluator of a university selected to perform an answer to a final questionnaire constructed by sorting survey items of the evaluator terminal 100, (100).

In this case, when the diagnostic index generating server 150 selects a part of the plurality of question items based on the distribution of the first answer, the variance value of the distribution of the first answer is calculated for each of the plurality of question items, A portion of the plurality of questionnaire items can be selected based on whether or not the value is lower than a preset reference.

The operation of selecting the questionnaire items using the above-described variance value will be described in more detail. The plurality of questionnaire items included in the initial questionnaire are transmitted to the internal evaluator and a selected one of the external evaluators, And the variance value of the distribution of the answers to the questionnaire items is calculated based on the results of the preliminary questionnaire and then the questionnaire items having the variance value lower than the predetermined first criterion can be selected to constitute the second questionnaire have.

For example, one question on the first questionnaire is "** University has a clear talent," and the evaluator's choice is "not at all", "not so much", "just so ? "," It is so, "and" very much ".

At this time, the distribution of the respondents' responses to the above questionnaire items may be concentrated in a specific response among the five responses, or may be uniformly distributed in five different responses. In this case, it can be judged that the question is relatively valid as compared with the case where the preliminary questionnaire by the plural evaluators about the questionnaire is focused on the specific answer, as compared with the case where the questionnaire is dispersed by different responses.

Accordingly, the evaluation index generation server 150 calculates a variance value according to the distribution of answers to each question item based on the results of the previous questionnaire, and then selects the questionnaire items whose variance value is lower than the predetermined first criterion, 2 questionnaires. At this time, the variance value according to the answer distribution may be low value when the answer distribution is concentrated on a specific response, and may appear to be high value when the answer distribution is uniformly distributed in a plurality of responses. Accordingly, the first variance value may be set to the specific variance value selected by the administrator among the pre-calculated variance values that can be determined that the answer distribution is concentrated on the specific response, but the present invention is not limited thereto.

Next, the diagnostic index generation server 150 performs a pre-test on the second questionnaire composed of the questionnaire items selected from the first questionnaire, secondarily through the second group of the evaluator, and then the plurality of questionnaires included in the second questionnaire After conducting in-depth analysis such as exploratory factor analysis, confirmatory factor analysis, and validity analysis of the items, questionnaire items satisfying the predetermined second criterion among the plurality of questionnaire items are selected based on the analyzed results The third questionnaire is completed. In this case, the pre-test means that before questioning the second questionnaire to the whole evaluator, the plurality of questionnaire items included in the second questionnaire are evaluated in advance as appropriate questionnaire items to evaluate the university administration service, A test for selecting a question item satisfying a predetermined second criterion. Also, the evaluator of the second group may refer to a part of the evaluators selected for the pre-test among the entire evaluators, and the evaluator of the second group may be an external evaluator selected from, for example, an internal evaluator selected by a professor, an employee, And the like, but the present invention is not limited thereto.

The diagnostic index creation server 150 provides the third completed questionnaire to the second terminal used by the evaluator inside or outside the university selected to evaluate the university administration service through the preliminary test as described above, The evaluation of the administrative service can be performed based on the second answer when the answer to the question item of the questionnaire is input from the second terminal. Then, the diagnostic index generation server 150 provides the evaluation execution result to the administrator terminal 130.

2 is a detailed block diagram of a diagnostic indicator generation server 150 according to an embodiment of the present invention. The diagnostic indicator generation server 150 includes a communication unit 200, an evaluator selection unit 202, a diagnostic indicator generation unit 210, a questionnaire configuration unit 204, a control unit 206, a memory unit 208, and the like.

Hereinafter, the operation of each component of the diagnostic index generation server 150 will be described in more detail with reference to FIG.

The communication unit 200 may be connected to the evaluator terminal 100 or the administrator terminal 130 through a communication network and may be connected to the evaluator terminal 100 or the administrator terminal 130 And the data transmitted and received between the server 150 and the server 150.

That is, for example, the communication unit 200 may request a project participant to the evaluator terminal 100 of the evaluator selected to participate in the project according to the performance of the questionnaire diagnosis index generation project, And receives the questionnaire response, which is the questionnaire response information in which the answer to the questionnaire item is input from the evaluator terminal 100. [ At this time, the questionnaire response means that each evaluator inputs individual evaluation information for a plurality of questionnaire items included in the questionnaire.

The evaluator selection unit 202 receives the questionnaire for the questionnaire prepared for the evaluation of the university administration service and selects the evaluator to input the answer in the questionnaire included in the questionnaire. At this time, the evaluator who performs the evaluation on the university administration service may be, for example, a faculty member of a university, a professor, a lecturer, a student, an administrative assistant, a bachelor assistant, a local resident, Among the evaluators, the evaluator who performs the pilot test and the pre-test on the questionnaire can be selected as the first group (pilot test) evaluator and the second group (pretest) evaluator according to the predetermined criteria.

The questionnaire configuration unit 204 stores the first questionnaire including the questionnaire items prepared for the evaluation of the university administration service. In constructing the above questionnaire, the questionnaire constructing unit 204 determines whether the association with the university administration service is higher than the reference value by the keyword analysis based on the big data among the questionnaire items related to the evaluation of the university administration service The first questionnaire can be written based on the questionnaire.

Accordingly, it is possible to construct a more realistic and problem-solving questionnaire item that reflects the big data-based keywords and social issues. Based on the questionnaire items, the evaluation index generating unit 210 can evaluate the university administration service , It is possible to generate social-issue-oriented, data-oriented, and future-oriented diagnostic indices rather than past-oriented or theoretical-oriented diagnostic indices.

In addition, the questionnaire formulating unit 204 selects, from the plurality of questionnaire items included in the first questionnaire based on the questionnaire item selection information provided from the diagnostic indicator generating unit, The final questionnaire consists of the questionnaire items. This final questionnaire may be provided to the evaluator terminal 100 of an unspecified number of evaluators selected to evaluate the university administration service through the communication unit 200 under the control of the control unit 206. [

The control unit 206 controls the overall operation of the diagnostic index generation server 150 according to an operation program stored in the memory unit 208. [

If a survey questionnaire diagnostic index generation project for the university administration service evaluation is generated according to the embodiment of the present invention, the evaluator selection unit 202 transmits the questionnaire to the selected evaluator through the communication unit 200 to participate in the project When a response to the questionnaire is received from the evaluator, the questionnaire to which the evaluation information about the questionnaire item is inputted is provided to the diagnostic indicator generator 210 as a questionnaire response of the evaluator, so that the diagnosis of the questionnaire items included in the questionnaire So that the indicator generating operation can be performed.

If the final questionnaire is constructed by the questionnaire formatter 204, the control unit 206 transmits the final questionnaire to the communication unit 210. [ To the evaluator terminal 100 selected to evaluate the university administration service through the Internet 200.

When the first questionnaire for the evaluation of the university administration service is registered by the administrator, the diagnostic indicator generator 210 firstly performs a pilot test on the registered first questionnaire through the evaluator of the first group selected among all the evaluators ) To evaluate the content validity of the multiple questionnaires included in the questionnaire, and then to formulate the second questionnaire only by selecting the questionnaire items satisfying the predetermined first criterion based on the content validity evaluation.

At this time, in the evaluation of the validity of the question of the pilot test, the diagnostic index generation server 150 transmits the initial questionnaire related to the evaluation of the university administration service, for example, to the first questionnaire used by the first group of evaluators, The method of claim 1, further comprising: providing at least two first answers to each of the plurality of question items included in the first questionnaire from the first terminal, And a second questionnaire can be constructed by selecting some of them.

Hereinafter, the process of preparing the second questionnaire through the content validity test of the first questionnaire will be described in more detail.

First, as shown in [Table 1] below, the evaluation area is classified into three categories (service quality / service delivery / service environment) for the first questionnaire constructed first, It is categorized into individual standards (convenience, expertise, innovation, responsiveness, reliability, agility, cooperation, convenience, accessibility). In addition, the evaluation criteria are classified into two categories (evaluation content / evaluation subject), and then classified again into detailed items and classified into six criteria (clarity, inclusiveness, interchangeability, degree of bonding, sensitivity and difficulty) , And conducts a content feasibility analysis on each question item based on the questionnaire response of the evaluator.

[Table 1] shows the results of the questionnaire responses of the first group to the first questionnaire. When referring to [Table 1], for the first questionnaire, Four questionnaires were presented, but two unsuitable questionnaires were deleted and one questionnaire item was revised in the simplicity and suitability of the questionnaires. In the field of expertise, two questionnaires were deleted because of the relevance of the questionnaire to the personality of the questionnaire. In the innovation area, as in the expertise area, the relevance was pointed out as a problem, so all three questionnaire items were deleted .

The first four questions were raised in the responsiveness area in the 'service delivery', but one question item that did not meet the criteria in the questionnaire 's fitness and inclusiveness was deleted. In the reliability area, the criterion The questionnaire was deleted, and one question item was revised. Also, in the affinity area, two questionnaires that did not meet the criterion of fitness and sensitivity were deleted, and one question item was revised.

In the 'service environment', the first four items were raised in the area of cooperation, but three question items that did not meet the criteria in the clarity, inclusiveness, and suitability of the question items were deleted, and one question item was modified Able to know. In addition, two questionnaires were deleted from the criteria of convenience and sensitivity in the area of convenience, and two questionnaires that did not meet the criteria in the clarity and inclusiveness were deleted in the accessibility area, and one questionnaire item was corrected .

That is, the diagnostic indicator generator 210 selects the questionnaire items satisfying the predetermined first criterion among the plurality of questionnaire items included in the first questionnaire through the content validity check as described above, and creates the second questionnaire.

Then, the diagnostic index generator 210 performs a pre-test on the second questionnaire consisting of the questionnaire items selected from the first questionnaire, secondarily through the second group of the evaluators, After conducting in-depth analysis such as exploratory factor analysis, confirmatory factor analysis, and validity analysis of multiple included questionnaire items, based on the analyzed results, the user satisfies the predetermined second criterion among the plurality of questionnaire items And the third questionnaire is selected. In this case, the pre-test means that a plurality of questionnaire items included in the second questionnaire are appropriately asked to evaluate the university administration service before the second questionnaire is conducted to the entire subject, A test for selecting only the question items satisfying the preset second criterion can be said. Also, the evaluator of the second group may refer to a part of the evaluators selected for the pre-test among the whole evaluators, and the evaluator of the second group may be, for example, an internal evaluator selected by a professor, an employee, or an external evaluator But is not limited thereto.

Hereinafter, based on the results of the pre-test performed by the second group of evaluators preset for the second questionnaire by the diagnostic indicator generating unit 210, an in-depth analysis such as exploratory factor analysis, confirmatory factor analysis, The analysis of the questionnaire items to be included in the third questionnaire will be described in more detail.

First, in the exploratory factor analysis, the diagnostic index generation unit 210 generates a diagnostic index based on the results of the Varimax square rotation method for each question item in order to extract a valid factor for measuring the variable in the embodiment of the present invention. Perform factor analysis and extract questionnaire items that are bound by factors.

Table 2 below shows the result of performing exploratory factor analysis on the 16 questionnaires included in the second questionnaire selected through the content validity examination in the embodiment of the present invention, for example.

As shown in Table 2, the standard fit was 0.976, and the sphericity test was performed on the null hypothesis "H_0: correlation = unit matrix". As a result, approximation Chi-square = 69397.956 (sig. 0.000), the null hypothesis can be rejected. In other words, the factor analysis is meaningful because the variables applied to the factor analysis are statistically significant.

The results of this study are as follows. First, the results of this study are as follows. First, the results of the analysis are as follows. First, ), And it can be seen that they are bound together.

At this time, the first '⑫ questionnaire item: employees are cooperating with each other in terms of work' was considered as cooperativeness, which is a sub-area of service environment, but was grouped into service delivery group in exploratory factor analysis. However, this is related to the infra for provision of high quality administrative services during the question development stage, so it can be placed in the service environment factor in the confirmatory factor analysis.

The eigenvalues of each factor were 11.724 for service delivery, 0.817 for service environment, and 0.502 for service quality, indicating that the overall cumulative explanatory variable had a very high explanatory power of 81.514%. In addition, significant factor loadings of 0.50 or more commonality were found for each factor, and explanatory power of each sub - measurement variable for each construct was very reasonable from 0.65 to 0.80. In the reliability analysis, each construct from the minimum 0.925 to the maximum 0.963 has a very high level of reliability, and the reliability of the overall evaluation index is also 0.975 Is secured.

Next, in the confirmatory factor analysis,

The diagnostic index generation unit 210 performs a confirmatory factor analysis by separating the research models into constructs in order to test the structural relevance of the sub-constituent factors of the perceived satisfaction with the university administration service extracted through exploratory factor analysis .

In general, there are two methods for evaluating the model: Chi-square test and fitness index. The Chi-square test can be applied to test the completeness of the model, but as in the embodiment of the present invention, when the sample size is large (N = 3,782), there may be a problem of rejecting the null hypothesis very sensitively according to the complexity of the model . Therefore, in the embodiment of the present invention, a model is evaluated using a descriptive fit index.

In many studies, Marsh, Bella and McDonald (1988) have chosen the fit index based on the studies in the past, but this is limited in the following points. It should be noted that the fit index should not change sensitively according to sample size, and it should be a fit index that takes account of the model's interrelation (Hu and Bentler, 1995). In particular, it can be concluded that the model's interrelation is related to the efficiency of the research through the model, since it is linked with how simple the model can lead to high explanatory power. Taking this into account, Hu and Bentler (1999) find that the TLI (Tucker-Lewis index) or NNFI (non-normed fit index) and RMSEA root mean square error of approximation. However, in the embodiment of the present invention, an incremental index of fir (IFI) is considered as a criterion of fitness evaluation instead of the RMSEA that considers the covariance matrix and the like because there is a model fitness test in a state where the construct is separated (Bollen, 1986 ). For the TLI and IFI, 0.9 or better is considered as a good fitness measure. In the embodiment of the present invention, CFI (comparative fit index) together with TLI and IFI are considered as a fitness index. CFI is less sensitive to the influence of the sample, We do not consider reputation. We also see 0.9 or better as a good fit.

Next, the diagnostic index generation unit 210 performs AVE and CR in the centralized validity judgment for each construct. In this case, AVE and CR can be set as 0.5 or more and 0.7 or more as a criterion with reasonable relevance, and AVE and CR can be calculated as shown in the following Equation (1).

At this time, as can be seen from [Table 4] below, it can be seen that AVE and CR exceed standards in all constructions.

In order to verify the validity of the discrimination, the verification of the two methods is considered as the core. First, it is a method to evaluate the discriminant validity by using the correlation coefficient and the standard error to determine whether the constructs are equal to each other (= 1.0). That is, the confidence interval (2 * SE) of the correlation coefficient showing the correlation between potential factors should not include 1. Therefore, when the confidence interval of the correlation coefficient between two potential factors does not include 1, it can be considered that it satisfies the discriminant validity. As shown in the table below, the correlation coefficient confidence interval 1, it can be concluded that all constructs have validity in the first validation.

Next, we select a pair with two constructs and then compare the model with the fixed correlation between the constants and the unconstrained model with free correlations. At this time, if the free model is superior to the constrained model, it can be considered that there is a discriminant validity. When the chi-square difference between the two models is significant (chi-square / df> 3.84) after analyzing the chi-square difference between the two constraint models with covariance fixed at 1, It can be seen that the discrimination validity is secured. In this case, since DELTA x ^ 2 / df, which is the evaluation criterion of the discriminant validity, exceeds the constraint model 3.84 (x_0.05 ^ 2 (1)) as compared with the non-constraint model, The results of this study are as follows.

Also, in the case of the rule validity, the sub-constituent factors of the university's administrative service satisfaction model handled in the embodiment of the present invention are theoretically and logically related to each construct statically correlated. As a result of the correlation analysis of each constructs, as shown in [Table 6], all the constructs showed a static correlation, so that it can be considered that the lawfulness is secured between the individual constructs in the embodiment of the present invention.

Next, in the validity analysis of the scale,

In the embodiment of the present invention, two kinds of analysis are performed in order to examine whether the measurement value can be accurately measured by the measurement variable by analyzing the validity of the scale.

The diagnostic index generation unit 210 designates each construct as a first-order construct and designates a covariance for the construct. As a result, the fitness was Chi-square = 1762.354 p = .000 df = 101 IFI = .953 TLI = .944 CFI = .953 as shown in Table 7 below. However, this is considered as a reference index because it is sensitive to the number of samples, and then TLI, IFI and CFI index are considered. All of the three indicators show satisfactory fitness as a whole when the reference point is 0.9 or more.

Next, the diagnostic index generation unit 210 checks whether the sub-constituent indexes of the three constructs are properly loaded as intended to verify the unity of the dimension. As a result, the high-order factor model was estimated using the first-order concept of the three sub-constituents of administrative service satisfaction and the second-order administrative service (AS) as the virtual concept. ].

Table 8 shows that the constructs of each first-order are significantly loaded in the second-order administrative service, but the fitness index is the same as the first-order model Chi-square = 1762.354 p = .000 df = 101 IFI = .953 TLI = .944 CFI = .953). The first-order factor model and the second-order factor model selection criterion are effective to focus on the degree of conformity with the initial research model. Since the research model regards the second-order construct as a model with intercon- nectivity and interactivity, rather than as a separate item from the first-order construct, the first-order factor model Can be effective.

FIG. 3 shows an operation control flow for selecting an evaluation index in the diagnostic index generation server 150 for evaluation of university administration service according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to FIGS. 1 to 3. FIG.

First, when the first questionnaire for university administration service evaluation is received by the administrator terminal 130, the diagnostic indicator generation server 150 registers the first questionnaire as a diagnostic indicator generation questionnaire (S300).

At this time, the first questionnaire can be a questionnaire including a plurality of questionnaire items for evaluating the university administration service. In addition, these questionnaires can be composed of a plurality of questionnaire items including keywords having relevance to university administration service higher than the standard value through keyword analysis of big data based on the questionnaire items related to evaluation of university administration service . Therefore, it is possible to construct a more realistic and problem-solving questionnaire that reflects big data-based keywords and social issues, and by generating diagnostic indicators for the evaluation of university administration services based on the questionnaire items, It is possible to generate social-issue-oriented, data-oriented, and future-oriented diagnostic indicators instead of theoretical-oriented diagnostic indicators.

In addition, these question items are categorized into categories such as administrative service quality, administrative service delivery, and administrative service environment, and within each category, questions such as convenience, expertise, innovation, responsiveness, reliability, agility, , Accessibility, etc., and can be composed of a predetermined number of question items in each sub-item, but the present invention is not limited thereto.

Next, the diagnostic index generation server 150 selects a first group of evaluators for performing the pilot test on the first questionnaire registered as described above (S302).

In this case, before conducting the questionnaire to the whole questionnaire, the pilot test is to judge whether a plurality of questionnaire items included in the first questionnaire are proper questions to evaluate the university administration service, A test for selecting only the question items satisfying the preset first criterion can be said. Also, the evaluator of the first group may refer to a part of the evaluators selected for the pilot test among the entire evaluators, and the evaluator of the first group may be, for example, an internal evaluator or an external evaluator selected by a professor, an employee, But is not limited thereto.

 In the case of selecting the first group of evaluators to perform the pilot test as described above, the diagnostic index generation server 150 transmits the first questionnaire to the first group of evaluators and requests a response to the question items (S304).

Accordingly, the evaluator of the first group that received the first questionnaire performs the evaluation of each question item of the first questionnaire. When the answer to all the questionnaire items is inputted, the first questionnaire in which the answer is inputted is diagnosed To the index creation server 150.

Then, the diagnostic index generation server 150 receives a response to the first questionnaire received from the evaluator of the first group (S306).

Then, the diagnostic index generation server 150 analyzes the content validity of each question included in the first questionnaire based on the response to the first questionnaire, and only the questionnaire item satisfying the first criterion (S308) and a second questionnaire is prepared (S310). At this time, the number of question items included in the second questionnaire may be smaller than or equal to the number of questionnaire items included in the first questionnaire.

Next, the diagnostic index generation server 150 selects a second group of evaluators for performing a pre-test on the second questionnaire created through the content validity determination as described above.

In this case, the pre-test means that a plurality of questionnaire items included in the second questionnaire are appropriately asked to evaluate the university administration service before the second questionnaire is conducted to the entire subject, A test for selecting only the question items satisfying the preset second criterion can be said. Also, the evaluator of the second group may refer to a part of the evaluators selected for the pre-test among the whole evaluators, and the evaluator of the second group may be, for example, an internal evaluator selected by a professor, an employee, or an external evaluator But is not limited thereto.

 When the evaluator of the second group to be subjected to the pre-test is selected as described above, the diagnostic indicator generation server 150 transmits a second questionnaire to the evaluator of the second group and requests a response to the question item (S312).

Accordingly, the evaluator of the second group who received the second questionnaire performs the evaluation for each question item of the second questionnaire. When the answer to all the questionnaire items is input, the second questionnaire to which the answer is inputted is diagnosed To the index creation server 150.

Then, the diagnostic index generation server 150 receives a response to the second questionnaire received from the evaluator of the second group (S314).

Next, the diagnostic index generation server 150 performs an in-depth analysis for performing an inquiry factor analysis, a confirmatory factor analysis, and a feasibility analysis on each question item included in the second questionnaire based on the response to the second questionnaire (S316). Only the questionnaire items satisfying the second criterion are selected (S318), and the third questionnaire is prepared (S320). In this case, the number of the questionnaire items included in the third questionnaire may be smaller than or equal to the number of the questionnaire items included in the second questionnaire.

At this time, the operation of performing the in-depth analysis by the diagnostic index generation server 150 is as described above in the operation description of the diagnostic index generation unit 210 of FIG.

Accordingly, the questionnaire of the first questionnaire, which was first registered by the administrator for the evaluation of the university administration service, is based on the information evaluated by the evaluator of the first group and the evaluator of the second group through the pilot test and the pre-test It may consist of the most appropriate questionnaires to evaluate university administration services.

Thus, the third questionnaire constructed can be distributed to an unspecified number of people who want to be evaluated for university administration service, and can be evaluated more accurately based on the evaluator's response to the third questionnaire do.

As described above, according to the present invention, in the generation of the diagnostic indicators for the evaluation of the university administrative service, in the generation of the diagnostic indicators for the evaluation of the university administration service, In this paper, we propose a methodology for the evaluation of university administrative services based on questionnaires. In this paper, we construct a questionnaire with more realistic and problem - Centered, data-driven, future-oriented diagnostic indicators can be generated, and the reliability and validity of diagnostic indicators can be increased.

Combinations of the steps of each flowchart attached to the present invention may be performed by computer program instructions. These computer program instructions may be loaded into a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus so that the instructions, which are executed via a processor of a computer or other programmable data processing apparatus, Lt; / RTI > These computer program instructions may also be stored in a computer usable or computer readable memory capable of directing a computer or other programmable data processing apparatus to implement the functionality in a particular manner so that the computer usable or computer readable memory It is also possible to produce manufacturing items that contain instruction means for performing the functions described in each step of the flowchart. Computer program instructions may also be stored on a computer or other programmable data processing equipment so that a series of operating steps may be performed on a computer or other programmable data processing equipment to create a computer- It is also possible for the instructions to perform the processing equipment to provide steps for executing the functions described in each step of the flowchart.

In addition, each step may represent a module, segment, or portion of code that includes one or more executable instructions for executing the specified logical function (s). It should also be noted that in some alternative embodiments, the functions mentioned in the steps may occur out of order. For example, the two steps shown in succession may in fact be performed substantially concurrently, or the steps may sometimes be performed in reverse order according to the corresponding function.

While the invention has been shown and described with reference to certain preferred embodiments thereof, it will be understood by those skilled in the art that various changes and modifications may be made without departing from the spirit and scope of the invention. Accordingly, the scope of the invention should not be limited by the described embodiments but should be defined by the appended claims.

100: Evaluator terminal 130:
150: diagnosis index generation server 200:
202: Evaluator selection section 204: Questionnaire configuration section
206: control unit 208: memory unit
210: diagnostic index generator

Claims (7)

A plurality of questionnaires including the selected keyword are selected by selecting a keyword having a relevance to the university administration service higher than a reference value through a keyword analysis of a big data base among the previously registered questionnaires, The first questionnaire was written as an item,
The first questionnaire is provided to a first terminal used by a predefined evaluator,
Selecting at least two of the plurality of question items on the basis of the distribution of the inputted first answer when at least two first answers for each of the plurality of question items included in the first questionnaire are inputted from the first terminal,
Providing the selected survey item to a second terminal used by an evaluator inside or outside the university,
And when the second answer to the selected question item is input from the second terminal, evaluates the administrative service based on the inputted second answer. The method according to claim 1,
Wherein the diagnosis index generation server selects a part of the plurality of questionnaire items based on the distribution of the input first answers,
Calculating a variance value of the first distribution of the received answers for each of the plurality of questionnaire items and selecting the part of the plurality of questionnaire items based on whether the calculated variance value is lower than a predetermined criterion
Diagnostic Indicator Generation Server for Evaluation of University Administration Service. A plurality of questionnaires including the selected keyword are selected by selecting a keyword having a relevance to the university administration service higher than a reference value through a keyword analysis of a big data base among the previously registered questionnaires, And the first questionnaire is provided to a first terminal used by a predefined evaluator, and at least two first answers to each of a plurality of questionnaire items included in the first questionnaire are provided to the first terminal, A diagnostic index generator for selecting a portion of the plurality of questionnaire items based on the input distribution of the first answer;
A questionnaire constructing unit configured to receive information on questionnaire items selected from the diagnostic indicator generating unit and construct a final questionnaire item from the questionnaire items of the first questionnaire,
Providing the final questionnaire to a second terminal used by an evaluator inside or outside the university, and if a second answer to the final questionnaire is input from the second terminal, A controller for performing an evaluation on the received signal,
A preliminary evaluator who will respond to the initial questionnaire or the final questionnaire, and an evaluator selection unit that selects an evaluator inside or outside the university,
A diagnostic indicator generation server for evaluation of university administration services, including. The method of claim 3,
Wherein the diagnostic index generator comprises:
The method of claim 1, wherein, in selecting a portion of the plurality of questionnaire items based on the distribution of the input first answer, a variance value of the distribution of the first answer is calculated for each of the plurality of questionnaire items, Wherein the server is configured to select a portion of the plurality of questionnaires based on whether the predetermined criterion is lower than a predetermined criterion. A diagnostic indicator generation method performed by a diagnostic indicator generation server,
A plurality of questionnaires including the selected keyword are selected by selecting a keyword having a relevance to the university administration service higher than a reference value through a keyword analysis of a big data base among the previously registered questionnaires, The first questionnaire is composed of questions,
Providing the initial questionnaire to a first terminal used by a predefined evaluator;
Selecting at least two of the plurality of question items on the basis of the distribution of the inputted first answers when at least two first answers for each of the plurality of question items included in the first questionnaire are input from the first terminal Wow,
Constructing a final questionnaire for the evaluation of the university administration service in the selected questionnaire;
Providing the final questionnaire to a second terminal used by an evaluator inside or outside the university, and if a second answer to the final questionnaire is input from the second terminal, Steps to Perform the Evaluation
A method of generating a diagnostic indicator for the evaluation of a university administration service. 6. The method of claim 5,
The step of selecting,
Calculating a variance value of the distribution of the input first answers for each of the plurality of questionnaire items;
Selecting the part of the plurality of questionnaire items based on whether the calculated variance value is lower than a predetermined criterion
A method of generating a diagnostic indicator for the evaluation of a university administration service. A computer-readable medium having recorded thereon a program for causing a processor to perform the method of claim 5 or claim 6.

Images