JP2011008554A - Programming ability diagnosis device, programming ability diagnosis method, and computer program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To easily and rapidly diagnose the ability of solving a programming task while preventing a program from being incorrectly evaluated due to misunderstanding of content of the programming task.SOLUTION: The programming ability diagnosis device (client terminal 2A) reads, after a test whether a desired program output can be obtained is performed in the client terminal 2B, a successful program and program configuration information for restoring a plurality of intermediate programs generated in the process that a subject forms the successful program, to diagnose the programming ability of the subject. The process of forming the successful program is regenerated, and a drawing for visualizing each processing procedure of one or a plurality of processing modules or a structure of one processing module or that between a plurality of processing modules is generated. The process of forming the successful program and the generated drawing are displayed on a display part 25.

Description

  The present invention relates to a programming ability diagnostic apparatus, a programming ability diagnostic method, and a computer program that give a subject a programming task, obtain an answer to the programming task from the subject, and diagnose the problem solving ability.

  In IT companies, various skill diagnosis is performed for the purpose of evaluation and training of programmers. For example, when a programmer is employed, there is a case where a small-scale programming task is given and the necessity of the employment is determined by skill of an answer (coding) created by the programmer. Also, in a promotion test for employees, some programming task related to work is given, and the approval or disapproval of the promotion may be determined depending on whether or not the answer to the programming task is possible. In these cases, the evaluator is required to appropriately evaluate the presence or absence of programming ability by looking at the program created by the programmer. However, it is generally not easy to properly evaluate programming ability.

  For example, when two subjects are given programming tasks and the two created programs are compared, even if both programs are close to the model answer, one subject codes in 10 minutes and the other subject When coding in 30 minutes, there is of course a difference in programming ability, and the former evaluation should be high. In addition, while one subject had many mistakes in the programming process and the number of debugging was frequent, the other subject naturally had a difference in programming ability even when there was almost no debugging, this time The latter evaluation should be high. However, the conventional evaluation method is often an evaluation method only for comparing created programs, and it has been difficult to appropriately evaluate the programming ability with such an evaluation method.

  Therefore, focusing on the process of creating a program, programming education support disclosed in, for example, Patent Document 1 is a technique for reproducing the programming process, measuring the time elapsed in the programming process, counting the number of compilations in the programming process, and the like. There is a system. Specifically, the answering (coding) process of a programming task is grasped in real time and made into a database, the answering process (programming process) is reproduced based on the recorded data, the elapsed time of programming is measured, the progress By visualizing the number of input characters with respect to time and a graph of compilation timing, it is possible to grasp the thinking process of each learner (subject) and the part of lack of understanding, and thus the degree of programming proficiency.

JP 2006-227218 A (paragraph [0007] etc.)

  In general, when creating a program of a large scale, a program with a simple and clear logical structure is preferable from the viewpoint of ease of bug correction, maintenance efficiency, and the like. For example, a program in which individual function modules are converted into subroutines in a basic structure such as a main function is more preferable as a design concept than a program in which individual function modules are simply arranged in order from the top. However, the ability to build a program structure such as whether the logical structure is simple and clear just by visualizing the number of input characters against the elapsed programming time and the graph of compilation timing as in the conventional programming education support system. It is difficult to evaluate.

  In this regard, it is considered that a certain degree of logical structure can be grasped by looking at the whole program created by the subject. However, it is difficult for those who are not familiar with programming, such as in-house personnel personnel, to grasp the logical structure of the program. Even if the evaluator is familiar with programming, for example, it takes time to grasp the logical structure of the entire program of several tens to several hundred lines. In particular, when considering recruitment tests for programmers and dispatching programmers, it is often necessary to evaluate the programming ability of tens or hundreds of programmers. The work of looking through is very cumbersome and time consuming.

  Furthermore, in the programming education support system described above, it is impossible to determine whether or not the program is created by misunderstanding the content of the programming task. For example, when a program is created that has a simple logical structure at a glance with a small number of lines, even if the grammatical error is eliminated by the subject's own compilation, it is created by misunderstanding the content of the programming task. If it is, it should be rated low. However, in the programming education support system described above, it is assumed that the subject correctly understands the content of the programming task. Therefore, despite the misunderstanding of the content of the programming task, a program with a simple logical structure. If is created, there is no denying the possibility of accidentally becoming highly rated.

  As described above, with the conventional evaluation method, it has been difficult to diagnose the programming ability of the subject, particularly the ability to construct the structure of the program. In addition, when the number of subjects increases, it is difficult to easily and quickly diagnose the program structure construction ability of all subjects unless a person who is familiar with programming takes a considerable amount of time. Furthermore, it has been difficult to prevent erroneous evaluation of programs due to misunderstanding of the contents of programming tasks.

  The present invention has been made in view of the above points. The purpose of the present invention is to easily and quickly diagnose the ability to solve a programming problem, particularly the ability to construct the structure of a program. It is an object of the present invention to provide a programming ability diagnostic device, a programming ability diagnostic method, and a computer program that prevent erroneous evaluation of a program created by misunderstanding.

  A programming ability diagnosis apparatus according to a first aspect of the present invention is a program creation device in which a subject who has given a programming task creates a program that solves the programming task. The program is created by the subject and is set by the questioner who has given the programming task. A test passing program that is configured from one processing module or a plurality of processing modules and that has obtained the desired program output and passed the test. A program configuration information that can restore a plurality of intermediate generation programs generated in the process of creating a test passing program by a test subject, and diagnosing the subject's programming ability. The test passing program and the program configuration information Storage unit A reproduction unit that reproduces the above-described test passing program creation process based on a plurality of intermediate generation programs restored by the program configuration information, and one processing module or each of the plurality of processing modules based on the test passing program. A visualization diagram generating unit that generates a diagram in which a processing procedure or a structure of one processing module or a plurality of processing modules is visualized, a process of creating the test passing program reproduced by the reproducing unit, and a visualization diagram generating unit A control unit that displays the generated diagram on the display unit.

  According to such a configuration, the program to be subjected to the programming ability diagnosis is a test passing program in which a desired programming output set by the questioner who has given the programming task is obtained. Further, based on the test passing program, a diagram in which the processing procedure of each processing module or a plurality of processing modules or the structure between one processing module or a plurality of processing modules is visualized is generated. Then, the visualized diagram and the process of creating the test passing program are displayed on the display unit. Therefore, since a program created by misunderstanding the content of the programming task is excluded from the object of the programming ability diagnosis, it is possible to prevent an erroneous evaluation of the program due to being created by misunderstanding the content of the programming task. In addition, a person who performs a programming ability diagnosis visually recognizes a processing procedure of one processing module or a plurality of processing modules or a diagram in which a structure between one processing module or a plurality of processing modules is visualized through the display unit. Even if you are not familiar with programming, you can easily diagnose the ability to build the structure of a program. In addition, even if there are a large number of subjects, by visually checking this figure, subjects who are clearly inferior in programming ability are excluded, and for the remaining subjects, a test passing program that is played back by the playback unit as necessary The ability to build the program structure can be diagnosed efficiently and quickly. The “processing module” in this specification refers to a collection of some work steps as a single process, and not only a function module that is a general component of a program, It may be of any type and name, such as a subroutine that summarizes the work whose contents are collected as one procedure, a procedure that combines a plurality of processes into one, and a subprogram that functions as a partial program of the program. Further, the “program configuration information” referred to in the present specification may be any information as long as it is information that can restore a plurality of intermediate generation programs generated in the process of creating a test passing program by the subject.

  In the programming capability diagnosis apparatus according to the second invention, a timing analysis for analyzing a timing at which one processing module or each of the plurality of processing modules is generated based on a plurality of intermediate generation programs restored by the program configuration information described above The control unit displays the analysis result of the timing analysis unit on the display unit.

  According to such a configuration, the person who performs the programming ability diagnosis knows the timing at which one processing module or each of the plurality of processing modules is generated, and builds the structure of the program in his head quickly. Ability to diagnose whether or not For example, it can be seen that a subject who has all the processing modules necessary to complete a program at an early timing has an excellent ability to see through the program structure. On the other hand, it can be seen that the test subject who has not been able to generate the necessary processing module is inferior in such power.

  In the programming ability diagnostic device according to the third invention, for each of the one processing module or the plurality of processing modules, whether or not there is a description of the processing module by the subject before the timing analyzed by the timing analysis unit, A search unit for searching using a plurality of intermediate generation programs restored by the program configuration information is provided, and the control unit displays the search result of the search unit on the display unit.

  According to such a configuration, it is possible to grasp the timing when the processing module is described by the subject. That is, it is possible to analyze the timing at which one processing module or each of a plurality of processing modules is generated by the timing analysis unit described above, but this is only the timing at which the analysis by the timing analysis unit is successful. For example, the timing immediately after the test subject describes the processing module in the source code has not yet been completed, so the analysis by the timing analysis unit described above is not successful and cannot be known only by the timing analysis unit. Therefore, by using a plurality of intermediate generation programs restored by the program configuration information, by searching for whether there is a description of the processing module by the subject in the program, it is possible to know until the timing immediately after the subject has described the processing module. As a result, it is possible to grasp the subject's thought process such as the time when the subject notices the necessity of the processing module.

  According to a fourth aspect of the present invention, there is provided a programming capability diagnostic apparatus including a code duplication rate analysis unit that analyzes a code duplication rate indicating a degree of code duplication within one processing module or between a plurality of processing modules. The analysis result of the rate analysis unit is displayed on the display unit.

  According to such a configuration, the person who performs the programming ability diagnosis can diagnose the ability such as whether the program structure can be efficiently constructed. For example, it can be seen that a subject who has created a program with a low code duplication rate has an excellent ability to construct an efficient program structure with little waste. On the contrary, the test subject who created the program with a high code duplication rate shows that such a power is inferior. In addition, a program with a high code duplication rate tends to have a lot of changed parts at the time of specification change due to code duplication. Therefore, a program with a low code duplication rate is excellent in maintainability when a specification change occurs.

  In the programming ability diagnostic device according to the fifth aspect of the present invention, the display unit includes a reception unit that receives a user operation, and the control unit is generated by the visualization diagram generation unit based on an operation signal from the reception unit; The analysis result of the timing analysis unit and the analysis result of the code duplication rate analysis unit are selectively displayed on the display unit.

  According to such a configuration, the person who performs the programming ability diagnosis operates, for example, a reception unit including a mouse and the like, the diagram generated by the visualization diagram generation unit, the analysis result of the timing analysis unit, and the code duplication rate Since the analysis result of the analysis unit can be selectively switched on the display unit, the ability to construct the program structure can be easily and quickly diagnosed from various viewpoints.

  In the programming ability diagnosing method according to the sixth aspect of the present invention, in a program creation device in which a subject who has given a programming task creates a program that solves the programming task, the program is created by the subject and set by the questioner who has given the programming task A test passing program that is configured from one processing module or a plurality of processing modules and that has obtained the desired program output and passed the test. A programming ability diagnosis method for diagnosing a subject's programming ability by reading program configuration information capable of restoring a plurality of intermediate generation programs generated in the course of creating a test passage program by a subject, Program structure A storage step for storing information, a reproduction step for reproducing the creation process of the test passing program based on a plurality of intermediate generation programs restored by the program configuration information, and one processing module based on the test passing program or Visualization diagram generation step for generating a diagram in which the processing procedure of each of the plurality of processing modules or the structure of one processing module or a plurality of processing modules is visualized, the process of creating a test passing program reproduced by the reproduction step, and visualization Displaying the diagram generated by the diagram generating step on the display unit.

  In the computer program according to the seventh aspect of the present invention, in a program creation device in which a subject who has given a programming task creates a program that solves the programming task, the program is created by the subject and set by the questioner who has given the programming task After the test of whether or not a desired program output is obtained, a test passing program composed of one processing module or a plurality of processing modules and obtaining a desired program output and passing the test, and the subject A computer for causing a computer to function as a programming ability diagnostic device that reads program configuration information capable of restoring a plurality of intermediate generation programs generated in the process of creating a test passing program and diagnoses a subject's programming ability. A storage unit for storing a test passing program and program configuration information, a playback unit for playing back a process of creating the test passing program based on a plurality of intermediate generation programs restored by the program configuration information, and a test Based on the passage program, a visualization diagram generation unit that generates a visualization of the processing procedure of each processing module or each processing module or the structure between one processing module or a plurality of processing modules, and playback by the playback unit The computer is caused to function as a programming ability diagnostic apparatus including a process for creating a test passing program to be performed and a control unit that causes the display unit to display a diagram generated by the visualization diagram generation unit.

  According to the programming ability diagnosis method and the computer program having such a configuration, the ability to construct the structure of the program can be diagnosed easily and quickly, similarly to the programming ability diagnostic apparatus according to the first invention described above. At the same time, it is possible to prevent erroneous evaluation of the program due to misunderstanding of the contents of the programming task.

  As described above, according to the present invention, the person who performs the programming ability diagnosis visualized the processing procedure of each processing module or a plurality of processing modules or the structure between the one processing module or the plurality of processing modules. By visually recognizing the diagram, it is possible to easily and quickly diagnose the ability to construct the structure of the program, even if it is not familiar with programming or there are many subjects. In addition, programs created by misunderstanding the contents of programming tasks can be excluded from the target of programming ability diagnosis, thus preventing erroneous evaluation due to the fact that programs were created by misunderstanding the contents of programming tasks. be able to.

It is a conceptual diagram which shows the system configuration | structure of the system by which the programming capability diagnostic apparatus (client terminal) which concerns on embodiment of this invention is used. It is a block diagram which shows the hardware structural example of the server shown in FIG. It is a block diagram which shows the hardware structural example of a client terminal. It is a flowchart which shows the process until it downloads the test subject's answer data with respect to a programming subject from a server to a client terminal. It is an example of a top menu screen. It is an example of an assignment registration screen. It is an example of a registration setting dialog screen. It is an example of an assignment URL issue screen. It is an example of an issue setting dialog screen. It is an example of the issue screen displayed on the display part of the client terminal which a subject questioner operates. It is an example of an issue screen. This shows that nothing has been entered in the assignment display / edit area. It is a figure which shows a mode that the content (problem sentence) of the task which a subject question person wants a test subject to receive is displayed. It is a figure which shows a mode that the warning display is made. It is a figure which shows a mode that the error display is made. It is an example of the answer confirmation screen for a subject questioner to confirm a test subject's answer. It is a functional block diagram which shows the software function of the client terminal after the reproduction | regeneration observation application is installed. It is a figure which shows the starting screen displayed on a display part when a reproduction | regeneration observation application is started in a client terminal. It is a figure which shows a mode that one data file used as analysis object was added. It is a figure which shows a mode that the creation process of the test passing program is reproduced | regenerated. It is a figure which shows a mode that the build is performed once by the test subject. It is a figure which shows a mode that the programming task is displayed on the reproduction | regeneration column. It is a figure which shows a mode when more time has passed compared with FIG. It is a figure which shows a mode that the intermediate production | generation program produced | generated last is displayed. It is a figure for demonstrating reproduction | regeneration on a real time basis. It is a figure which shows the mode of each parameter transition among the analysis modes which display the figure for analyzing a programming preparation process. It is a figure which shows a function life cycle among the analysis modes which display the figure for analyzing a programming creation process. It is a figure which shows a flowchart among analysis modes which display the figure for analyzing a programming preparation process. It is a figure which shows a flowchart among analysis modes which display the figure for analyzing a programming preparation process. It is a figure which shows a mode that each parameter transition of two test subjects is compared. It is a figure which shows a mode that not only a horizontal axis but the vertical axis | shaft is also normalized. It is a figure which shows a mode that each compares the function life cycle of two test subjects. It is a figure which shows a mode that each compares the flowchart of three test subjects. It is a figure which shows a mode that the code duplication of each of three test subjects is compared. FIG. 27 is a diagram showing a state where the click position of the radio button of the horizontal axis value in parameter setting is changed from the number of operations to time, unlike FIG. It is an example of a facial expression mode setting screen that changes the facial expression of a character when a time period (waiting) during which the subject is not operating anything occurs for a predetermined time. It is an illustration figure of the data structure memorize | stored in answer DB. It is an illustration figure which shows the specific example of the programming subject shown. It is an illustration figure of the program by the C language which the test subject created according to the presented programming subject. It is a flowchart which shows the procedure of the data collection process of CPU of a server. The flowchart of the process sequence of capability evaluation value calculation of CPU is shown. It is an illustration figure of transition data of the complexity calculated by the server. It is an illustration figure of the approximated curve calculated by the server. It is an illustration figure of the effective area extracted by the server. It is an illustration figure of the approximated curve created by the server when it is necessary to correct | amend an effective area. It is an illustration figure which shows the transition data of the complexity of four characteristic subjects A, B, C, and D. FIG. It is a figure which shows a mode that the positive direction is reversed unlike FIG. It is a figure which shows an example of a diagnostic report (1st sheet). It is a figure which shows an example of a diagnostic report (2nd sheet).

  Hereinafter, a programming capability diagnosis apparatus according to an embodiment of the present invention will be specifically described with reference to the drawings. First, an outline of a system in which the programming ability diagnostic device is used will be described.

[System Overview]
FIG. 1 is a conceptual diagram showing a system configuration of a system in which a programming capability diagnosis device (client terminal 2A in FIG. 1) according to an embodiment of the present invention is used. A usage example of this system will be described as a specific example.

  The first usage pattern is a usage pattern in which a system operating company possesses a client terminal 2A as a programming ability diagnostic device and a server 1. As the system operation company, for example, a company that operates a temporary staffing business can be considered. The server 1 is installed in a predetermined data center by a system operating company, and is connected to a client terminal 2A in the system operating company via a communication network 100 such as the Internet. The server 1 is also connected to client terminals 2B (for example, home PCs) owned by a plurality of programmers registered with the system operating company (for example, personnel dispatch registration).

  A personnel evaluation staff member of a system operating company accesses the server 1 by operating the client terminal 2A, registers a programming task to be received by a programmer as a test subject (for example, a programmer who has registered personnel dispatch), and The URL of the link destination that can input the answer to the programming task is acquired. Then, the URL is separately notified to the subject by e-mail or telephone. The URL indicates the location of a programming task (file) that exists in the server 1. The subject operates his / her client terminal 2B at his convenience or within a predetermined time limit, accesses the link destination (web page) indicated by the URL, and inputs an answer to the programming task. The input answer is sent to the server 1 and stored in a predetermined storage area in the server 1. Thereafter, the person in charge of personnel evaluation operates the client terminal 2A to access the server 1, confirms the content of the answer input by the subject, downloads the content to the client terminal 2A, and diagnoses the programming ability of the subject. . Details of the diagnostic contents of the programming ability will be described later in [Programming ability diagnosis].

  The second usage mode is a usage mode in which only the server 1 is owned by the system operating company. A client terminal 2A as a programming ability diagnostic device and a client terminal 2B operated by a subject are owned by a system use company that has concluded a system use contract with a system operation company. Examples of system operating companies include software as a service (SaaS) companies that provide a programming ability diagnosis service, and examples of system using companies include software companies that have many programmers. An information university that is not a company but has many programming classes can be a system user.

  The SaaS person in charge at the system operation company installs the server 1 in a predetermined data center and concludes a system use contract of the system use company. A person in charge of personnel in a system user company registers a programming task that a programmer (for example, a programmer who is considering hiring) as a test subject wants to receive. The rest is the same as the first usage pattern. That is, the URL is communicated to the programmer as the subject, and the subject operates the client terminal 2B to access the link destination in the server 1 indicated by the URL, and inputs an answer to the programming task. Thereafter, the personnel officer operates the client terminal 2A to access the server 1, confirms the content of the answer input by the subject, downloads the content to the client terminal 2A, and diagnoses the programming ability of the subject.

  As described above, in this embodiment, in both the first usage mode and the second usage mode, the content of the answer to the programming task input by the subject is transmitted from the server 1 to the client terminal via the communication network 100. Download to 2A. Thereby, the answer content of the subject can be acquired easily and quickly on the client terminal 2A, and the programming ability of the subject can be diagnosed easily and quickly. In particular, when the number of subjects increases, diagnosis can be performed more easily and more quickly.

  Note that the client terminal 2A as the programming capability diagnosis device does not necessarily have to be incorporated in the system shown in FIG. By moving various answer data of the test subject to the programming task to the client terminal 2A using a recording medium such as USB or DVD, the programming ability diagnosis on the client terminal 2A becomes possible.

  In the following, the system shown in FIG. 1 is used to perform the programming ability diagnosis of the subject in the client terminal 2A. First, the process up to downloading the answer data of the subject to the programming task from the server 1 to the client terminal 2A via the communication network 100 will be described in detail (see [Acquisition of Answer Data]). Thereafter, the programming ability diagnosis performed on the client terminal 2A will be described in detail using the answer data obtained by downloading (see [Programming ability diagnosis] described later).

[Get answer data]
FIG. 2 is a block diagram illustrating a hardware configuration example of the server 1 illustrated in FIG. In the system shown in FIG. 1, the number of servers 1 is one, but the number of servers 1 may be distributed. Further, when considering a multi-tenant (for example, a case where a plurality of companies become system use companies in the second usage mode described above), more client terminals 2A and 2B are connected to the server 1. .

  As shown in FIG. 2, the server 1 stores a CPU (Central Processing Unit) 11 that performs numerical calculation and information processing based on various programs and controls each part of the hardware, a startup program, an initialization program, and the like. A ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13 that functions as a work area when the CPU 11 executes various programs, a communication unit 16 that is communicably connected to the external communication network 100, And an HDD (Hard Disk Drive) 17 functioning as auxiliary storage means, and communication between them is performed via a bus such as a data bus or an address bus.

  The HDD 17 of the server 1 is installed with an OS (Operating System) that executes various applications and manages the entire server 1 in an integrated manner (not shown). In addition, as an application (application) to be executed on the OS, a task to be presented to the subject is selected and set, and a task setting / answer confirmation application 172 for confirming the subject's answer and the subject's skill are quantitatively determined. Information such as HTML (Hyper Text Markup Language) documents, XML (Extensible Markup Language) documents, images, etc. in response to requests from the productivity / quality evaluation application 174 and the web browser application 272 of the client terminals 2A and 2B Is installed to the client terminals 2A and 2B. By installing the web application 175, the server 1 can function as a web server. Each application is loaded (deployed) from the HDD 17 to the RAM 13 and executed by the CPU 11 based on each application program.

  As described above, the server 1 provides the user interface for displaying information such as an image transmitted from the server 1 on the display unit 25 by the operation of the reception unit 24 of the client terminals 2A and 2B by the user. A web application 175 is included.

  Further, the HDD 17 of the server 1 has an assignment database (DB, abbreviated hereinafter) 176 for registering assignments to be given to subjects, a question assignment DB 178 for registering assignment candidates having system use authority, and a client terminal 2A. , 2B, an issue DB 179 for storing HTML documents and images to be transmitted, and an answer DB 180 for storing answers obtained from subjects.

  More specifically, the assignment DB 176 registers the assignment name and contents of the programming assignment created by the assignment examiner so that past assignments (software assets) can be easily reused. This is a DB to be stored. The question-taker DB 178 is a DB in which the login ID and password of the question-taker questioner are registered in order to confirm whether or not there is an authority to use the system as the question-taker questioner. The issue DB 179 is a DB that stores an HTML document, an image, or the like that causes the subject's client terminal 2B to display a screen for the subject to input an answer to the programming task. Those HTML documents and images are transmitted to the client terminal 2B of the subject in response to a request from the web browser application 272 (FIG. 3) of the client terminal 2B described later. The answer DB 180 stores (stores) the answer obtained from the subject, that is, log data (source code created by the subject) sent from the client terminal 2B (for example, every time the subject inputs a source code). This DB Although details will be described later, the timing at which the server 1 receives the log data is also stored in the answer DB 180.

  The HDD 17 of the server 1 can be replaced with a flash memory, a USB (Universal Serial Bus) memory (and a memory controller), an optical disk drive, or the like. Each application program can be stored in the HDD 17 in advance, or can be stored in the HDD 17 from a recording medium such as a CD or DVD, or downloaded from an external computer via the communication network 100 and stored in the HDD 17. You can also Further, for convenience of explanation, the above-described applications are separately stored in one HDD 17, but can be distributed and stored in a plurality of HDDs 17, and a plurality of applications are integrated. It can also be stored in one HDD 17.

  FIG. 3 is a block diagram illustrating a hardware configuration example of the client terminal 2. Note that the hardware configuration of the client terminal 2A operated by the subject person and the hardware configuration of the client terminal 2B operated by the subject are the same, and therefore, the hardware configuration of the client terminal 2 will be described without particular distinction. To do. Note that details of the client terminal 2A functioning as a programming ability diagnostic device will be separately described later using a functional block diagram shown in FIG.

  As shown in FIG. 3, the client terminal 2 includes a CPU 21, a ROM 22, a RAM 23, a receiving unit 24 that receives a user operation using a mouse or a keyboard, a display unit 25 such as a monitor, and a communication unit such as a network card. 26 is connected via a bus. The functions provided by these units are basically the same as the corresponding units of the server 1. The client terminal 2 has an HDD 27, and the HDD 27 uses an input / output function by the receiving unit 24 and the display unit 25, a function for executing various applications, and a GUI in the display unit 25. For example, an OS for managing the entire client terminal 2 is installed (not shown).

  In addition, as an application (application) executed on the OS, an e-mail application 271 that transmits and receives e-mails and a web browser application 272 for browsing a programming problem on which questions are given by the challenger are installed. ing. By installing the web browser application 272, the client terminal 2 can function as a web client. For each of these applications, an application program is loaded (expanded) from the HDD 27 to the RAM 23 and is executed by the CPU 21 based on each application program. As with the HDD 17 of the server 1, the HDD 27 of the client terminal 2 can be replaced with a flash memory, a USB memory (and a memory controller), an optical disk drive, or the like.

  FIG. 4 is a flowchart showing a process for downloading the answer data of the subject to the programming task from the server 1 to the client terminal 2A via the communication network 100 using the system shown in FIG. The outline of each process will be explained. Step S1 for logging in to the server 1 in order to register the programming task in the server 1 in order to register the programming task in the server 1, and creating the programming task that the questioner wants the subject to receive Then, in step S2, which is registered in the server 1, the questioning person wants to give the subject a programming task (which may be registered in step S2, may be one that he has registered in the past, or another questioning person may Step S3 for selecting (which may have been registered in), Step S4 for setting (issuing) a URL indicating a website for inputting an answer to the selected programming task, and a subject who is informed of the URL by e-mail or the like Accesses the website indicated by the URL in step S5, subject's program Step S6 accepting input of answers to timing issues, questions who question the programming challenge, step S7 confirms answers of the subject and, to diagnose and programming capability, step S8 to perform the skill evaluation consists flow of.

  As shown in FIG. 4, first, the challenger is logged in (step S <b> 1). Specifically, the challenger starts up the web browser of his / her client terminal 2 </ b> A and accesses the server 1 (to a predetermined link destination) via the communication network 100. When the client terminal 2A requests the server 1 to transmit an HTML document or the like for displaying a login screen on the display unit 25, the CPU 11 transmits the HTML document or the like to the client terminal 2A.

  The challenger inputs his login ID and password by operating the reception unit 24 (mouse, keyboard, etc.) of the client terminal 2A. Then, after the login ID and password are sent to the server 1 and collation thereof is performed, the login of the subject questioner is completed. As described above, the login ID and password are collated by the CPU 11 accessing the questioner DB 178 in the HDD 17. As a result, it is confirmed that the task candidate has the authority to use the skill diagnosis system. If verification fails, login is rejected.

  When login of the challenger is completed, the top menu of the programming skill diagnosis system is displayed. Specifically, when the CPU 11 of the server 1 receives the operation signal from the reception unit 24 of the client terminal 2A and recognizes the login of the questioning person, the CPU 11 displays the top menu screen on the display unit 25 of the client terminal 2A.

  FIG. 5 is an example of a top menu screen. The top menu screen shown in FIG. 5 includes an assignment registration button 61 for registering a new assignment and editing assignment contents, an assignment URL issue button 62 for setting and issuing a URL for setting a programming assignment. An answer confirmation button 63 for confirming the answer of the subject is arranged in this order from the top. In this way, buttons for performing task registration, URL issuance, and task confirmation are arranged on one screen of the top menu screen, so that the task candidate creates a programming task that the subject wants to receive, and the server 1 (Step S2 in FIG. 4), selecting a programming task that the subject wants to receive (step S3 in FIG. 4), and setting a URL indicating a website for inputting an answer to the selected programming task A series of operations of (issuing) (step S4 in FIG. 4) and confirming the answer of the subject (step S7 in FIG. 4) can be performed easily and quickly.

  On the right side of the answer confirmation button 63, an answer newly arrived number display column 64 indicating the number of newly arrived answers is arranged. As a result, the challenger can check whether the answer from the subject has arrived at the server 1 at the same time as viewing the top menu screen.

  Next, the challenger registers the assignment from the top menu screen shown in FIG. 5 (step S2 in FIG. 4). Specifically, when the assignment registration button 61 on the top menu screen is clicked by the assignment examiner, the assignment registration screen is displayed on the display unit 25 of the client terminal 2A.

  FIG. 6 is an example of an assignment registration screen. In the upper part of the assignment registration screen shown in FIG. 6, a list of assignments already registered by the assignment examiner is displayed. As a result, the subject person can easily confirm at a glance the issues that can be given to the subject. The “registrant” in the assignment list is a person who created a programming assignment indicated by the assignment name in the past, that is, a person who previously registered the programming assignment indicated by the assignment name in the server 1. The “difficulty” is a comment by the registrant. Whether the difficulty level is high or low is displayed as a comment based on the personal opinion of the registrant. In addition, when there is no assignment registered by the assignment candidate, it is displayed blank.

  In the assignment registration screen shown in FIG. 6, a new registration button 617 for registering a new assignment and an open button 618 for opening the selected assignment for editing are arranged below the assignment list. Further below these buttons, a letter “TOP” 619 linked to the top menu screen is displayed.

  There are two types of procedure for registering assignments: clicking the above-mentioned new registration button 617 to register a new assignment, or clicking the above-mentioned open button 618 to edit a past assignment. Regardless of the registration procedure, the registration setting dialog screen is displayed, and the process proceeds to setting the registration conditions.

  FIG. 7 is an example of a registration setting dialog screen. The registration setting dialog screen shown in FIG. 7 is an image immediately after the dialog screen is displayed when the open button 618 is clicked. When the new registration button 617 is clicked, all the dialog screens are not set (blank display). )become. As described above, the open button 618 is arranged side by side under the new registration button 617, and the registration setting dialog screen in which past assignments are automatically read is displayed by clicking the open button 618. Thus, the issuer can easily reuse past issues (past assets).

  For example, when it is desired to create a derived task of the task “English word count”, the cursor is placed on “English word count” in the task list shown in FIG. 6, and this is opened by clicking the open button 618. Then, the content is arranged on the registration setting dialog screen shown in FIG. 7, another task name (such as “English word count 2”) is input in the task name text box 6181, and the registration button 6189 is clicked. As a result, the issuer can easily create a derivative issue of a past issue.

  Above the registration setting dialog screen shown in FIG. 7, an assignment name text box 6181 in which an assignment name can be newly entered, and a language list box (pull-down menu) 6182 in which a language such as C or JAVA (registered trademark) can be selected are provided. Yes. If an assignment name text box 6181 is provided, a derived assignment can be easily created as described above. In addition, if the language list box 6182 is provided, the question candidate can freely select C language or JAVA (registered trademark) language, and the number of issue support languages (such as Ruby) will increase in the future. Can be easily added to the case.

  Below the language list box 6182, an assignment content registration area 6183 for directly inputting the contents of the programming assignment is arranged. In the case of a text-only task, the task can be created in a text format, and in the case where it is desired to add a diagram to the task, the task can be created in a PDF format or a JPEG format. In this embodiment, the initial state of the task content registration area 6183 is a text box. Therefore, when a task candidate registers a task text, the task text may be directly written in the task content registration area 6183, or may be copied and pasted from another document.

  Below the assignment contents registration area 6183, a test code registration area 6185 for directly inputting and registering a test code for automatic unit testing is arranged. The automatic unit test is a test of whether or not a desired program output set by a questioner who has given a programming task can be obtained for a program created by the subject, and is executed at the client terminal 2B of the subject through the server 1. Is done. Originally, this automatic unit test is performed by the subject on the client terminal 2B, but by providing a test button 6187 at the bottom of the registration setting dialog screen shown in FIG. At the stage, you can try out the answers to your programming tasks yourself and run automated unit tests on the prototyped program. Details of the test button 6187 for executing the automatic unit test will be described later. If the clear content button 6186 is clicked, the content entered in the test code registration area 6185 is deleted.

  In FIG. 7, a registration button 6189 arranged at the bottom of the registration setting dialog screen is a button for registering a programming task in accordance with the setting content input in the task content registration area 6183 and the test code registration area 6185, and a cancel button. Reference numeral 6188 denotes a button for discarding the setting contents input in the assignment contents registration area 6183 and the test code registration area 6185 and closing the dialog. The former registration button 6189 will be specifically described. The CPU 11 registers the setting contents input in the assignment content registration area 6183 and the test code registration area 6185 in the assignment DB 176 of the HDD 17.

  Next, the challenger returns to the top menu screen shown in FIG. 5 and selects an assignment (step S3 in FIG. 4). Specifically, when the assignment URL issue button 62 on the top menu screen is clicked by the assignment examiner, the assignment URL issue screen is displayed on the display unit 25 of the client terminal 2A.

  FIG. 8 is an example of an assignment URL issue screen. Above the assignment URL issue screen shown in FIG. 8, a list of assignments already registered by the assignment examiner is displayed. The assignment list is the same as the assignment list shown in FIG. Accordingly, the task list is displayed in two places in FIG. 6 and FIG. 8, and comments (such as “difficulty level”) written by the task candidate on both screens must be reflected on each other.

  The issue history list is displayed below the assignment list, and the display items are arranged in the order of issue date 621, assignment name 622, expiration date 623, issuer 624, and purpose 625 from the left column. Yes. The items of issue date 621, assignment name 622, and expiration date 623 are items that are automatically displayed by the system, and the issuer 624 and purpose 625 items are comments that are arbitrarily entered by the issuer. In particular, the expiry date 623 will be described in detail. It indicates whether or not the expiry date of the URL (the link destination indicated by the URL) issued by the question candidate has passed (whether or not the subject can use the URL). If the expiration date has not passed, a black circle is attached. If a question with a black circle is given, answers may be sent not only from the subject this time, but also from subjects who have already given the task, and management becomes complicated ( For tasks that are not marked with a black circle, the answer will not be sent from anyone other than the subject because the expiration date for access to the answer screen has passed. Therefore, by providing the item of the expiration date 623, the convenience of issue issue management can be enhanced. The details of the expiration date of the URL (link destination indicated by) will be described later with reference to FIG. In this specification, a URL is used as an identifier indicating a location where data such as an HTML file or an image is stored on the communication network 100 such as the Internet. This is called a URI (Uniform Resource Identifier). You can also

  There are two procedures for issuing a task URL from the task URL issuing screen shown in FIG. One is a method of selecting a registered programming task and clicking a new issue button 627, and the other is a method of selecting a past issue history and clicking a correction / reissue button 629. It is. In any of these procedures, a later-described issue setting dialog screen (see FIG. 9) is displayed on the display unit 25. Each setting item when the issue setting dialog is opened is that when the new issue button 627 is clicked, only the assignment name is set and all others are displayed in an unset state (displayed blank), while being corrected / reissued When the button 629 is clicked, it is displayed in a state where the conditions when it was issued in the past are set. As a result, when reusing URLs issued in the past, it is possible to save time and effort for the task giver to input the conditions again.

  FIG. 9 is an example of the issue setting dialog screen. At the top of the issuance setting dialog screen shown in FIG. 9, a list box 701 capable of selecting a currently registered issue for each issue name is displayed. This makes it possible to save the trouble of setting the URL expiration date and times when the URL is issued by changing only the task to be presented to the subject while setting the same as the past issue history.

  Below the list box 701, a URL expiration date input field 702 composed of a text box or the like is displayed in order to set the expiration date of the URL to be issued (the link destination indicated by). In the URL expiry date input field 702, the expiry date of the URL (link destination indicated by) is specified under three types of conditions: designating the date and time, designating the period from issuance, or designating the period from the last access. Can be set. Thereby, even if the URL is leaked, unauthorized use of the system after the expiration date can be prevented. If the expiration date of the URL (link destination indicated by) can be set, the URL of the link destination that has expired can be grasped at a glance as described above with reference to FIG. The convenience of issue issue management can be improved.

  Below the URL expiration date entry field 702, a password entry field 703 for setting whether or not the subject is requested to enter a password when answering the task is displayed. Below the password entry field 703, a time limit entry field 704 is displayed. Has been. Note that counting the number of times the password is input and measuring the answer time can be realized by using a built-in timer or an external timer of the CPU 11.

  Below the time limit input field 704, an issue title input field 705, a message input area 706, and an answer input field setting field 707 are displayed. These are setting fields / setting areas for customizing the screen to be displayed on the display unit 25 of the client terminal 2B operated by the subject. In particular, the answer input field setting field 707 allows the number of characters that can be entered to be limited when the subject inputs his / her name and can be entered when the subject inputs his / her mobile mail address. This is a field that makes it possible to limit the number of characters.

  At the bottom of the issue setting dialog screen shown in FIG. 9, a test button 708, a cancel button 709, and an issue button 710 are arranged side by side. The test button 708 is a button for displaying the issuance screen shown in FIG. 10 according to the set content, and is a button for the subject questioner to confirm the content actually displayed to the subject. The test button 708 is a button for displaying an issuance screen for confirmation only, and a URL for actually inputting an answer to the programming task is not issued. That is, unlike the issue button 710 described later, the setting contents shown in FIG. 9 and the programming task are not associated with each other and stored in the issue DB 179. A cancel button 709 is a button for discarding the setting contents and closing the issue setting dialog.

  The issue button 710 is a button for associating the setting contents shown in FIG. 9 with the programming task and storing them in the issue DB 179. More specifically, when the issue button 710 is clicked by the issuer, the CPU 11 that has received the operation signal of the issue button 710 issues the subject that the subject wants to receive from the issue DB 176 (in FIG. Count “)” is copied onto the RAM 13 and read out. Further, the setting contents shown in FIG. 9 are associated with the problem and stored in the issue DB 179 in the HDD 17. As described above, unlike the test button 708, when the issue button 710 is clicked, the assignment and the setting contents are actually associated with each other and stored in the issue DB 179 in the HDD 17. The file stored in the issue DB 179 is stored as an HTML document, an XML document, or the like in order to display an issue screen shown in FIG.

  Further, the issue button 710 is also a button for displaying, for example, the issue screen shown in FIG. 10 on the display unit 25 of the client terminal 2A in accordance with the setting contents shown in FIG. More specifically, when the issue button 710 is clicked by the issuer, the issue screen shown in FIG. 10 is automatically generated according to the setting contents shown in FIG. 9, and the issue screen is displayed on the display unit of the client terminal 2A. 25.

  FIG. 10 is an example of an issuance screen displayed on the display unit 25 of the client terminal 2A that is operated by the challenger. FIG. 11 shows a state where the issue screen shown in FIG. 10 is displayed on the browser of the display unit 25 in the client terminal 2A of the question giver. The URL (http: //www.abc000...) Of the issuance screen of FIG. 10 automatically generated as described above, that is, the URL 1000 shown in FIG. It is associated with the expiration date input in the URL expiration date input field 702 shown in FIG. 9, the password input in the password input field 703 shown in FIG. 10 and 11, the contents input in the issue title input field 705 and the message input area 706 in FIG. 9 are displayed. The issue screen shown in FIG. 10 is a screen displayed on the display unit 25 of the client terminal 2B operated by the subject, as will be described later.

  Next, after the subject questioner copies and pastes the URL 1000 of the issuance screen shown in FIG. 11 and sends it to the subject by e-mail, the subject who has received the e-mail operates the client terminal 2B to set the URL. Access is made (step S5 shown in FIG. 4). Specifically, the e-mail application 271 in the HDD 27 is executed in the client terminal 2A of the challenger and the client terminal 2B of the subject, and the e-mail with the URL described above is transferred from the client terminal 2A to the client terminal 2B. Is sent.

  The test subject executes the web browser application 272 of the client terminal 2B, and accesses the URL attached to the e-mail in order to obtain a web page in which an answer to the task selected by the task candidate can be input (e-mail) You can also click on the hyperlinked URL above).

  In response to a request from the web browser application 272 of the client terminal 2B, the server 1 copies and reads out an HTML document or the like stored in the issuance DB 179 on the RAM 13, and sends it to the client terminal via the communication unit 16. Send to 2B. As a result, task information related to the programming task selected by the task giver is transmitted to the subject through the issue screen shown in FIG.

  Next, the subject browses the issue screen shown in FIG. 11 displayed on the display unit 25 of the client terminal 2 </ b> B, inputs necessary items, and then inputs an answer to the task. That is, when viewed from the server 1 side, an answer input for the task is accepted (step S6 in FIG. 4).

  In the issuance screen shown in FIG. 11, in order from the top, the URL 1000, the content 801 entered by the assignment candidate in the issue title input field 705 (FIG. 9), the password entry field 802, and the assignment person in the message input area 706 (FIG. 9) ) (Symbol omitted), answerer name input field 803, mobile mail address input field 804, mobile mail address re-input field 805, and start button 806 are displayed.

  The subject enters the password “test2009” that has been previously known by e-mail or the like in the password input field 802, and enters the name and mail address in the answerer name input field 803 and the mobile mail address input field 804, and then starts. Click the button 806. Then, a programming screen (FIG. 12) is displayed on the display unit 25 of the client terminal 2B. Specifically, in response to a request from the web browser application 272 of the client terminal 2B, the web application 175 of the server 1 stores an HTML document or the like for displaying a programming screen on the display unit 25 on the RAM 13 from the issuance DB 179. Copy and read, and send it to the client terminal 2B. This provides a programming environment for the subject. FIG. 12 shows a state in which nothing has been entered in the assignment display / edit area 901 yet.

  When the “task” tab in the task display / edit area 901 is clicked in the programming screen shown in FIG. 12, the content (task text) of the task that the task giver wants to receive is displayed as shown in FIG. (See the task content registration area 6183 shown in FIG. 7). As described above, when the task content is omitted (when the task content has already been communicated verbally or the like), even if the “task” tab is clicked, a blank is displayed.

  In FIG. 12 or 13, an assignment display / edit area 901 is an area for displaying assignment contents on the “issue” tab, and an area for creating and editing source code on the “edit” tab. You can switch between task display and source code editing at any time during programming. Thereby, the test subject can confirm the task content as appropriate even when the task is forgotten.

  An output area 902 displayed below the assignment display / edit area 901 is a standard output (STDOUT) area for the subject. The build results and program output results are displayed here. For example, a warning display as shown in FIG. 14 is made, or an error display as shown in FIG. 15 is made. Below the output area 902, a build button 903, an execution button 904, a test button 905, and a completion button 906 are arranged in order from the left.

  A build button 903 builds a program being created in the server 1. More specifically, the source code input by the subject into the assignment display / edit area 901 is sequentially sent to the server 1. For example, it is possible to use Ajax (Asynchronous JavaScript + XML) that performs asynchronous communication using JavaScript and exchanges XML data between the server 1 and the client terminal 2B. Thereby, the client terminal 2B can continue sending the test subject's programming process data (log data) to the server 1 asynchronously.

  Note that the log data may be sent every time the subject inputs one character of the source code, or may be sent every time a predetermined block (for example, an arbitrary function) is completed. In this embodiment, it is assumed that log data (source code written by the subject so far) is sent to the server 1 every time the subject inputs one character of the source code. As a result, even if a part of the log data transmitted due to a communication failure or the like is lost, the server 1 can recover the log data (replenish the missing part) by receiving the subsequent log data. As a result, the reliability of the log data to be built can be improved. In addition to Ajax, any technique may be adopted as long as the server 1 can receive log data. For example, the log data may be sent to the server 1 at a predetermined timing (for example, reload timing) using DHTML (Dynamic Hyper Text Markup Language) that performs synchronous communication with the server 1. In addition, Flash (registered trademark) having a higher degree of freedom may be used on the premise of installation of a plug-in.

  When the CPU 11 of the server 1 receives the log data transmitted from the client terminal 2 </ b> B via the communication unit 16, the CPU 11 sequentially stores the log data in the answer DB 180 in the HDD 17 in association with the answered subject. At this time, the timing at which the communication unit 16 receives the log data is stored in the answer DB 180 by a built-in timer or an external timer of the CPU 11 or a clock function. This timing is used for [programming ability diagnosis] or [quantitative skill evaluation] described later.

  As described above, when the test subject clicks the build button 903 in a state where the log data is sequentially stored in the answer DB 180, the CPU 21 of the client terminal 2B transmits a build execution command to the server 1 via the communication unit 26. When the CPU 11 of the server 1 receives the build execution command via the communication unit 16, the CPU 11 reads the log data currently stored (corresponding to the source code created by the subject so far) from the answer DB 180 and performs the build. The build is a process for automatically generating an executable file by linking compile and link. If there is a syntax error during the build process, a compiler message is displayed in the output area 902. In the present embodiment, the log data is read from the answer DB 180 at the time when the build is performed. However, if the log data is missing, the reliability of the build result is lowered. Therefore, for example, the server 1 can retrieve log data from the client terminal 2B again at the timing of receiving the build execution instruction. As a result, the build can be performed without any missing log data, so that the reliability of the build result can be improved.

  The execution button 904 is a button for executing the program being created. By clicking this button, an executable file automatically generated by the build is opened. Standard output from the program is displayed in an output area 902. If the execution button 904 is clicked when the source code being created has been changed, the build has not yet been performed for the changed source code. The OK button to allow is automatically made by a pop-up window, for example. Then, the subject directly clicks the build button 903, or after clicking the execute button 904, allows the build by clicking the OK button in the pop-up window, etc. It is possible to check whether there are any grammatical errors. In addition, when it is not built when the execution button 904 is clicked without displaying “Do you want to build?”, It is built automatically (forced), and then automatically the executable file May be opened.

  Here, even if there is no grammatical error or the like, there is a possibility that the output result intended by the task candidate cannot be obtained. Therefore, the subject uses the test button 905. The test button 905 is a button for executing an automatic unit test, and the automatic unit test is a test for automatically checking whether or not the source code answered to the task is logically correct. In the test code registration area 6185 shown in FIG. 7 described above, a main function and a structure (struct) of the name TestCase (this name is an example and may be another name) are described (partially extracted). ). When the test subject clicks the test button 905, the test code registered by the question candidate through the test code registration area 6185 shown in FIG. 7 is merged with the source code described in the assignment display / edit area 901 shown in FIG. Is done. Thereafter, the automatically generated executable file is opened, and the output result is displayed in the output area 902. In this way, the subject can confirm whether or not the output result intended by the task candidate is obtained before submitting the task.

  In other words, the HDD 17 of the server 1 is installed with a programming application (not shown in FIG. 2) that provides functions as a compiler and a linker. Build the merged source code and test code.

  The method for merging the source code and the test code may be any method. For example, when there is a description corresponding to the main function in the source code described in the assignment display / edit area 901, the part is deleted or the main function in the test code is a true main function. You can merge the source code and test code after editing as necessary, such as changing attributes). When the build is completed, the CPU 11 executes the obtained execution format file and displays the output result on the display unit 25 (the output area 902 of the programming screen shown in FIG. 15).

  Thus, the subject can execute the automatic unit test on his / her client terminal 2B by receiving the programming application function of the server 1. That is, in the present embodiment, the client terminal 2B operated by the subject is an example of a program creation device in which the subject who has been asked the programming task creates a program that solves the programming task. In the client terminal 2B, a program is created by the subject, and an automatic unit test is executed as a test of whether or not a desired program output set by a questioner who has given a programming task can be obtained. In other words, an automatic unit test is executed as a test for obtaining the output result of the test program generated by combining the test code registered in the server 1 and the source code created by the subject. .

  In addition, in addition to the test button 6187 described with reference to FIG. 7, as described above, this button tries to prototype the answer to the programming task at the stage where the task questioner registers the task content. It is a button for trying to execute an automatic unit test for the prototype program. In other words, when the test button 6187 is clicked by the task giver, the programming screen shown in FIG. 12 is displayed on the display unit 25 of the client terminal 2A of the issuer. Then, the task giver tries to describe the source code for answering the task as a test in the task display / edit area 901 on the programming screen. When the task is completed, the test subject clicks a test button 905 at the bottom of the programming screen in FIG. Then, the test code registered through the test code registration area 6185 shown in FIG. 7 is merged with the source code described in the task display / edit area 901 in the same manner as when the subject performs the automatic unit test, and the build is performed. . In this way, at the stage of registering the task content (see FIG. 7), the task taker can click on the test button 6187 shown in FIG. In other words, it is possible to test whether the intended output result can be obtained by the model source code and the model test code created by the user.

  Returning to FIG. 12, when the subject finally clicks the completion button 906, the test of the programming task is completed. However, if the automatic unit test by the test button 905 is not passed, a message “All test cases have not been passed. Do you want to finish?” Is displayed. In other words, the CPU 21 of the client terminal 2B that has received the operation signal from the accepting unit 24 transmits an examination completion signal that informs the server 1 of the fact, while the CPU 11 of the server 1 receives the examination completion signal, the answer DB 180 To check the pass / fail history of the automatic unit test by the test button 905. Note that all the test cases referred to here are test codes registered by the challenger through the test code registration area 6185 shown in FIG. 7 (for example, if a first value is assigned to a certain variable, the output result is the second Meaning all cases). If it is determined that even one test case of the automatic unit test has not passed, an HTML document for displaying the above-described message is transmitted to the client terminal 2B. As a result, the subject can be encouraged to submit the created program after passing the automatic unit test created by the subject.

  In this manner, as a result, the answer DB 180 of the server 1 obtains a desired program output set by the task giver and passes the automatic unit test (test passing program), and the test subject passes the test passing program. A plurality of programs (intermediate generation programs) generated during the creation process are stored. In the present embodiment, data transfer is performed from the client terminal 2 </ b> B to the server 1 every time the subject operates the reception unit 24 once using the technique such as Ajax described above. Note that one or more test passing programs may be used.

  Returning to FIG. 12, when the test of the programming task by the subject is completed by clicking the completion button 906, the answer newly arrived number display column 64 of the top menu screen shown in FIG. 5 is incremented by 1. That is, when the CPU 11 of the server 1 receives an examination completion signal from the client terminal 2 </ b> B, it adds 1 to the number displayed in the answer newly arrived number display column 64. Then, the subject person can check the answer of the subject on the answer confirmation screen by clicking the answer confirmation button 63 on the top menu screen shown in FIG. 5 (step S7 shown in FIG. 4).

  FIG. 16 is an example of an answer confirmation screen for the subject-questioner to confirm the answer of the subject. In the answer confirmation screen shown in FIG. 16, a new answer number display field 631 is displayed as in the new answer number display field 64 displayed on the top menu screen. In addition, below the answer newly arrived number display column 631, a list (result list) of tasks answered by the subject is displayed. In the result list, the reference 632, the assignment name 633, and the answer date 634 are items that are always displayed by the system. The column of the standard 632 is used for [quantitative skill evaluation] described later.

  Below the result list, a chart display button 637, a download button 638, an assignment information button 639, and an issue information button 640 are displayed. In the result list, newly arrived answer data that is not opened by any of these four buttons is shown in bold. As a result, the challenger can easily grasp new arrival data that has not yet been opened.

  The chart display button 637 is a button for displaying an evaluation result of [quantitative skill evaluation] described later in a productivity / quality chart. The download button 638 is a button for reading the subject's answer from the answer DB 180 and downloading it. Specifically, when the download button 638 is clicked on the display unit 25 of the client terminal 2A of the question giver, the CPU 11 of the server 1 answers the test subject's answer stored in the answer DB 180 (the test passing program and Intermediate generation program) is read out and transmitted to the client terminal 2A. As a result, the client terminal 2A of the question giver can download and read the test passing program and the intermediate generation program. Note that what is to be downloaded is current data currently selected by a cursor or the like on the answer confirmation screen shown in FIG. 16 (in FIG. 16, answer data created by answerer Q of No. 3). Although the intermediate generation program itself is downloaded here, program configuration information that can restore the intermediate generation program may be downloaded. For example, an intermediate generation program composed of source code in a state where “i” is input, an intermediate generation program composed of source code in a state where “in” is input, and a source where “int” is input An intermediate generation program consisting of codes can be downloaded, or source code with "i" input, source code with "n" input, source with "t" input Program configuration information consisting of codes can also be downloaded. If this program configuration information is present, both an intermediate generation program composed of source code in a state where “i” is input and an intermediate generation program composed of source code in a state where “in” is input are input as “int”. It is also possible to restore an intermediate generation program consisting of the source code in a state of being restored. For example, if the source code with "i" input is combined with the source code with "n" input, the intermediate generation program consisting of the source code with "in" input is restored. can do. In short, the program configuration information may be any information as long as the information can restore the intermediate generation program. In light of this, the intermediate generation program itself can also be program configuration information.

  As described above, in performing the diagnosis of the subject's programming ability at the client terminal 2A as the programming ability diagnosis device, the answer data of the subject with respect to the programming task is downloaded from the server 1 to the client terminal 2A via the communication network 100. Can do.

  Next, the programming ability diagnosis performed on the client terminal 2A using the downloaded answer data (test passing program and intermediate generation program) will be specifically described.

[Programming ability diagnosis]
First, the challenger clicks on the text 641 “Click here to download the playback observation application” hyperlinked at the bottom of the screen shown in FIG. 16, downloads the installer file, and installs it on his client terminal 2A. Specifically, the HDD 17 of the server 1 also stores a software program reproduction observation application (not shown), and the CPU 11 of the server 1 performs the reproduction observation application according to the request of the web browser application 272 of the client terminal 2A. Send the installer file. The CPU 21 of the client terminal 2A executes the received installer file and stores the file generated as a result in the HDD 27. Thereby, the reproduction observation application is installed in the client terminal 2A. In the present embodiment, the playback observation application is downloaded via the communication network 100, but it can also be installed on the client terminal 2A using a storage medium such as a USB or DVD.

  FIG. 17 is a functional block diagram showing software functions of the client terminal 2A after the reproduction observation application is installed. In the present embodiment, the client terminal 2A is an example of a programming capability diagnostic device that diagnoses the programming capability of the subject.

  As illustrated in FIG. 17, the client terminal 2A includes a control unit 20, a reception unit 24, a display unit 25, a communication unit 26, and a storage unit 27, a reproduction unit 31, and a visualization diagram generation unit 32. A timing analysis unit 33, a search unit 34, and a code duplication rate analysis unit 35. The receiving unit 24, the display unit 25, the communication unit 26, and the storage unit 27 are realized by software by the receiving unit 24, the display unit 25, the communication unit 26, and the HDD 27 shown in FIG.

  In particular, the storage unit 27 has a function of storing a test passing program composed of a plurality of function modules downloaded from the server 1 and program configuration information capable of restoring a plurality of intermediate generation programs. The reproduction unit 31 has a function of reproducing the test passing program creation process based on a plurality of intermediate generation programs restored by the program configuration information. The visualization diagram generation unit 32 has a function of generating a diagram in which a processing procedure of each function module or a plurality of function modules or a structure between one function module or a plurality of function modules is visualized based on the test passing program. Have. The control unit 20 has a function of causing the display unit 25 to display a process of creating a test passing program reproduced by the reproduction unit 31 and a diagram generated by the visualization diagram generation unit 32.

  Further, the timing analysis unit 33 has a function of analyzing the timing at which one function module or each of a plurality of function modules is generated based on a plurality of intermediate generation programs. The search unit 34 determines whether or not there is a description of the function module by the subject before the timing analyzed by the timing analysis unit 33 for each of the one function module or the plurality of function modules. It has a function to search using. The code duplication rate analysis unit 35 has a function of analyzing a code duplication rate indicating the degree of code duplication within one function module or between a plurality of function modules.

  The control unit 20 has a function of causing the display unit 25 to display the analysis result of the timing analysis unit 33, the search result of the search unit 34, and the analysis result of the code duplication rate analysis unit 35. In addition, the control unit 20 uses the analysis result of the timing analysis unit 33, the search result of the search unit 34, and the code duplication rate analysis unit 35 based on an operation signal from the reception unit 24 that receives a user operation on the display unit 25. A function of selectively displaying the analysis result on the display unit 25 (so-called selection switching display function) is provided.

  The functions of the playback unit 31, visualization diagram generation unit 32, timing analysis unit 33, search unit 34, code duplication rate analysis unit 35, and control unit 20 are the CPU 21, ROM 22, and RAM 23 shown in FIG. And functionally realized by software of a reproduction observation application stored in the storage unit 27. Hereinafter, based on the example of a screen shown in Drawing 18-Drawing 40, the details of the programming ability diagnostic method concerning this embodiment are explained.

  FIG. 18 is a diagram illustrating an example of an activation screen displayed on the display unit 25 when the reproduction observation application is activated on the client terminal 2A. The startup screen shown in FIG. 18 is roughly divided into three areas. The screen mode 1001, the target data file 1002, and the CSV conversion 1003 are displayed in the left column, and the playback column 1004 is displayed in the middle column. , An output column 1005 is displayed, and an elapsed time 1006 and a playback mode 1007 are displayed on the right column.

  The screen mode 1001 includes a “playback mode” for playing back a test passing program creation process and an “analysis mode” for displaying a diagram for analyzing the programming creation process, such as a diagram generated by the visualization diagram generation unit 32. This is for switching, and is composed of radio buttons in FIG. A person who performs a programming ability diagnosis can switch between the reproduction mode and the analysis mode only by setting the click position of the radio button to a desired position.

  The target data file 1002 is a file existing in the reference folder designated by the reference button 1002a, and in order to make it an analysis target, a file selection screen (not shown) is displayed by the add button 1002b, and a desired file is selected. It can be added (how it is added will be described later).

  The CSV conversion 1003 is a function for outputting data used in each parameter transition described later in a CSV format so that it can be analyzed by general-purpose software such as EXCEL (registered trademark). Which data of which file is output is designated by the reference button 1003a by displaying a data selection screen (not shown) by the setting button 1003b, setting desired data, and clicking the conversion button 1003c. Output CSV file converted to CSV to the reference folder. Thus, a person who performs a programming ability diagnosis can perform a desired analysis using software capable of handling a CSV file such as EXCEL (registered trademark).

  The reproduction column 1004 is a column for reproducing the process of creating the test passing program, and the output column 1005 is a column for displaying the result compiled by the subject and the result of executing the automatic unit test (reproduction / reproduction). The display state will be described later).

  The elapsed time 1006 is the time actually passed in the process of creating the test progress program by the subject, and the playback mode 1007 switches the click position of the radio button 1007a, thereby reproducing on the real time basis and on the operation basis. You can switch between playback. That is, the reproduction on the basis of real time that the subject reproduces according to the passage of time in the process of creating the test progress program, and the operation base that is reproduced according to the number of times the subject operates in the process of creating the test progress program. You can switch between playback.

  For example, in the case of reproduction on a real time basis, intermediate generation programs after 1 second have elapsed, 2 seconds have elapsed, and 3 seconds have elapsed are sequentially displayed in the reproduction column 1004. On the other hand, in the case of operation-based playback, after the first operation (eg, after pressing i on the keyboard), after the second operation (eg, after pressing n on the keyboard), after the third operation ( For example, after t is pressed on the keyboard, the intermediate generation program (ie, source code with "i" entered → source code with "in" entered → source with "int" entered Code) are sequentially displayed in the reproduction field 1004.

  By operating the speed adjustment bar 1007b, the speed reproduced in the reproduction column 1004 can be adjusted. That is, by moving the speed adjustment bar 1007b toward “fast” (upward), so-called fast-forward playback is possible, and by moving the speed adjustment bar 1007b toward “slow” (downward), so-called slow playback is possible. Is possible. When moved downward, constant speed reproduction is possible. In this manner, the test passing program creation process can be reproduced at a desired speed only by moving the speed adjustment bar 1007b.

  Next, the person who performs the programming ability diagnosis activates the playback observation application, and then clicks an add button 1002b shown in FIG. 18 to display a file selection screen (not shown), which is designated by the reference button 1002a. Add data files to be analyzed by selecting files that exist in the reference folder. In this way, the reproducing unit 31 reads the test pass program and the intermediate generation program stored in the storage unit 27 as an analysis target file.

  FIG. 19 is a diagram illustrating a state in which one data file to be analyzed is added. In FIG. 19, an object file (indicated by “English word count_A”) created by Mr. A is added to the “English word count” programming task. Note that the deletion button 1002d is activated by adding the target file created by Mr. A.

  Also, below the add button 1002b, a check box 1002c for determining whether or not to enable the “automatic add” function is provided. When the check box 1002c is checked, the playback observation application is activated. The file existing in the reference folder designated by the reference button 1002a is automatically read as the target file. Further, the number of files automatically added as target files can be limited to a desired number. In FIG. 19, as shown on the right side of the check box 1002c, the number is limited to one.

  Next, the person who performs the programming ability diagnosis clicks the playback button 1006a arranged under the elapsed time 1006 shown in the upper right of FIG. Then, the reproduction of the test passing program creation process is started.

  FIG. 20 is a diagram showing a state in which the process of creating a test passing program is reproduced. In FIG. 20, the pause button 1006b arranged on the right side of the play button 1006a is clicked. When the stop button 1006c arranged on the right side of the pause button 1006b is clicked, the reproduction column 1004 is cleared.

  As shown in FIG. 20, in the reproduction column 1004, 12 lines of source code are described. When the elapsed time 1006 is viewed, the source code is 1 minute 47 seconds after the start of the answer to the programming task. It turns out to be a code. That is, it can be seen that the subject described 12 lines of source code in 1 minute 47 seconds. Below the elapsed time 1006, the number of operations 1008a is 180 times, the number of characters 1008b is 206 characters (in consideration of copy and paste, etc., the number of characters may be larger than the number of operations), and the number of lines 16 lines (blank lines are also counted here), 18 seconds as the task confirmation time 1008d (the task confirmation time will be described later with reference to FIG. 22), 0 times as the number of builds 1008e, 0 times as the number of executions 1008f, 0 times is displayed as the number of automatic tests 1008 g.

  The build count 1008e indicates the number of clicks on the build button 903 shown in FIG. 12 in the process of creating the test pass program by the subject described above, and the execution count 1008f is the process of creating the test pass program in FIG. The automatic test count 1008g indicates the number of times the test subject clicked the test button 905 shown in FIG. 12 in the process of creating the test passing program.

  A thumbnail graph 1009 displayed under the automatic test count 1008g is a thumbnail of a graph (details will be described later) displayed when the analysis mode is selected in the screen mode 1001. In particular, a thumbnail reproduction position bar 1009a is displayed so as to overlap in the thumbnail graph 1009 having the number of operations or the reproduction time on the horizontal axis so that it can be seen to what extent the reproduction has been completed. It has become.

  In addition, a playback position bar 1010 is displayed at the lower part of FIG. 20, and to what extent playback has been completed is displayed. The person who performs the programming ability diagnosis clicks the reproduction position bar 1010 and moves it to the left and right to thereby generate an intermediate generation program (desired elapsed time in the case of reproduction on a real time basis). Source code).

  Here, the playback position bar 1010 and the thumbnail playback position bar 1009a are displayed in conjunction with each other. That is, when the playback position bar 1010 is moved to the left or right, the thumbnail playback position bar 1009a is also moved accordingly. In this way, when playing back the test passing program creation process, a playback position bar 1010 indicating the progress of the number of operations (when playing on an operation basis) or playing time (when playing on a real time basis) A thumbnail reproduction position bar 1009a which is arranged so as to overlap a thumbnail graph 1009 displaying a plurality of parameters (details will be described later) for analyzing the programming creation process, and indicates the number of operations or the elapsed time of the reproduction time, The control unit 20 causes the display unit 25 to perform interlocking display. As a result, the playback position bar 1010 can be moved while visually recognizing the parameters in the thumbnail graph 1009, and the desired intermediate generation program can be quickly displayed in the playback field 1004.

  For example, if there is a location in the thumbnail graph 1009 where the parameter for the number of operations has increased rapidly, the person who performs the programming ability diagnosis may want to know what operation the subject has performed. In such a case, the playback position bar 1010 may be moved so that the thumbnail playback position bar 1009a is positioned on the thumbnail graph 1009. Thereby, when the parameter of the number of operations on the thumbnail graph 1009 has increased rapidly, it is possible to quickly grasp what operation the subject has performed. Note that what parameters are to be displayed on the thumbnail graph 1009 may be selected by clicking the graph setting 1009b to display a parameter selection screen (not shown) and selecting a desired parameter. Here, only the playback position bar 1010 can be moved, but for example, only the thumbnail playback position bar 1009a or both the playback position bar 1010 and the thumbnail playback position bar 1009a can be moved. .

  FIG. 21 is a diagram illustrating a state in which a build is performed once by a subject. 21, the contents displayed in the output area 902 shown in FIG. 12 on the display unit 25 of the client terminal 2B are also displayed in the output column 1005 shown in FIG. 21 on the display unit 25 of the client terminal 2A. It should be noted that the number of builds 1008e is incremented by 1 as the build is performed once.

  FIG. 22 is a diagram illustrating a state in which a programming task is displayed in the reproduction field 1004. Specifically, according to FIG. 22, the subject Mr. A was watching the programming task when the elapsed time 1006 was 2 minutes and 7 seconds. This means that Mr. A was watching the programming problem as shown in FIG. 13 when 2 minutes and 7 seconds passed while actually programming on the client terminal 2B. And the time which was watching the programming subject is measured as the subject confirmation time 1008d mentioned above. As described above, the client terminal 2A as the programming ability diagnostic apparatus according to the present embodiment confirms the time during which the subject visually recognizes the programming task in the process of creating the test passing program based on the plurality of intermediate generation programs. It has a measurement function that measures time.

  FIG. 23 is a diagram illustrating a state when more time has elapsed than in FIG. The elapsed time 1006 indicates 30 minutes and 27 seconds, and the operation count 1008a indicates 1018 times. Further, the reproduction position bar 1010 has moved to the right from FIG. In addition, as shown in the build count 1008e and the output column 1005, the build is performed three times.

  In particular, in FIG. 23, unlike FIG. 21, a scroll bar 1004a appears in the reproduction field 1004. When the number of lines of the intermediate generation program increases, a scroll bar 1004a is automatically displayed. A person who performs a programming ability diagnosis can display a desired portion of the intermediate generation program in the reproduction column 1004 by moving the scroll bar 1004a up and down.

  FIG. 24 is a diagram illustrating a state where the last generated intermediate generation program is displayed. As shown in FIG. 24, the playback position bar 1010 is at the rightmost position, the elapsed time 1006 is 1 hour 27 minutes 39 seconds, the number of operations 1008a is 2717 times, the number of characters 1008b is 2126 characters, and the number of lines is 122 lines. The task confirmation time 1008d is 1173 seconds, the build count 1008e is 35, the execution count is 16, and the automatic test count is 0. In addition, the number of operations 1008a of 2717 matches the numerical value 2717 on the right end of the horizontal axis of the thumbnail graph 1009.

  18 to 24, the operation-based reproduction in the reproduction mode 1007 has been described. As described above, by switching the click position of the radio button 1007a in the playback mode 1007, it is possible to switch between real-time playback and operation-based playback. FIG. 25 is a diagram for explaining reproduction on a real time basis. FIG. 25 shows a state where the last generated intermediate generation program is displayed, as in FIG. 24, and the thumbnail graph 1009 shows 1 hour 27 minutes 39 seconds (= 5259 seconds) indicated by the elapsed time 1006. The numerical value 5259 indicated by the right end of the horizontal axis of FIG.

  The above is the description of the playback observation mode in the screen mode 1001. Next, the analysis mode in the screen mode 1001 will be described. FIG. 26 is a diagram showing the transition of each parameter in the analysis mode for displaying a diagram for analyzing the programming creation process. In FIG. 26, in the screen mode 1001 in the left column, the click position of the radio button is in the analysis mode.

  In the center of FIG. 26, there is provided an analysis column 1011 in which a diagram for analyzing the programming creation process is displayed. The analysis column 1011 reflects the setting contents of the parameter setting 1014 in the right column (check box check state) (partially due to the scale of the vertical axis or the relationship of the darkness of the line). Is difficult to see). In the right column of FIG. 26, the number of graphs 1012, display contents 1013, and parameter settings 1014 are displayed in order from the top. The number of graphs 1012 is for dividing and displaying 1011. In FIG. 26, the number of graphs 1012 is set to 1 × 1 in the vertical direction and is not divided (the state of division will be described later).

  In the display content 1013, each parameter transition, function life cycle, flowchart, and code duplication can be selected by radio buttons. In FIG. 26, each parameter transition is selected. In parameter setting 1014 used for each parameter transition, normalization 1015, horizontal axis value 1016, basic item 1017, and option 1018 can be set. Details of the normalization 1015 and the horizontal axis value 1016 will be described later (see FIGS. 30, 31, and 35 described later).

  As basic items 1017, the number of operations, the total number of characters, the total number of lines, the number of actual code characters, the number of actual code lines (the actual number of code lines excluding blank lines), the number of actual code words (the actual number of code words excluding space characters) ), Number of comment characters, number of comment lines, task confirmation time, task confirmation count, build count, execution count, automatic test (automatic unit test) count, build error count, build warning (warning) count, automatic test error count, etc. Can be selected by a check box whether or not to be displayed in the analysis column 1011. These parameters are not particularly limited to these, and can be increased or decreased as necessary.

  Also, regarding the number of builds, the number of executions, and the number of automatic tests, the number of times itself is small and difficult to see if displayed on the same graph as the number of operations, etc. The square item is displayed in the graph indicated by the parameter of the basic item 1017. For example, in the analysis column 1011, a square mark 1018a indicates a build timing, and a square mark 1018b indicates an execution timing. Thus, by displaying the build timing and execution timing with square marks in the graph displaying the parameter transition of the basic item 1017, the build timing and execution timing can be easily recognized.

  For example, a subject who repeats builds and executions too early may not be able to see in his head, and may be trial and error. Therefore, it is considered that the subject who repeats the build timing and execution timing at a later stage has better programming ability than the subject who repeats the build timing and execution timing at an earlier stage. If attention is paid to the number of times of the build timing and the execution timing, it is considered that the subject having a smaller number of times has better programming ability than the subject having a larger number of times. This is because the latter seems to be programming while thinking firmly.

  FIG. 27 is a diagram illustrating a function life cycle in an analysis mode for displaying a diagram for analyzing a programming creation process. As shown in the right column of FIG. 27, the click position of the radio button of the display content 1013 is in the function life cycle position. The function life cycle indicates the lifetime of a function until each function module is generated and deleted in the source code (generally, unless there is a mistake in grasping the subject matter of a programming task, etc.) The function module is not erased until the end), and is automatically drawn based on the analysis result of the timing analysis unit 33 as described above. A function means an instruction group that receives data (input data) called an argument, executes a predetermined process, and returns a result, and is generally a component of a program. In this embodiment, numbers are handled as input data. For example, a function in which a character is input as input data and the character is displayed on the screen as a processing result can be considered. The horizontal axis indicates the number of operations (the click position of the radio button with the horizontal axis value 1016 is at the position of the number of operations).

  According to the function life cycle shown in FIG. 27, Mr. A is the total of “IsWordEnd” 1021, “IsSmallAlpha” 1022, “IsBigAlpha” 1023, “IsAlNum” 1024, “CountWord” 1025, and “main” 1026 in the program. It turns out that 6 function modules are made. In particular, “main” 1026 at the bottom is a basic function (generally, a basic function that calls each subroutine) serving as a skeleton of the program, and is created in the program at an early timing. Accordingly, it can be seen that Mr. A first creates a basic structure called a main function, and then creates a program in which individual function modules are converted into subroutines within the main function.

  In addition, according to the function life cycle shown in FIG. 27, all function modules are created at the timing before the half of the number of operations (the timing of the left half in FIG. 27). Since the function module is a part of the program, it can be seen that Mr. A can see at an early timing what kind of part is necessary for the program, that is, has the power to see the configuration.

  Here, in the function life cycle drawing shown in FIG. 27, a description applicable to a definition may be recognized as a function based on a definition defined in advance. A typical method is syntax analysis by a compiler or the like. Here, the syntax analysis of the source code described using the C language will be described. A function in the C language can be defined by ““ function type specifier ”+“ function name ”+“ (”+“ argument ”+“) ”+“ {”+“ function body ”+“} ””. . The function type specifiers here are “int”, “void”, and the like. When the visualization diagram generation unit 32 or the timing analysis unit 33 recognizes a function module, it searches for a description applicable to such a definition, and if found, recognizes the description as a function. Some descriptions that do not fit this definition are not recognized as functions. In the present embodiment, syntax analysis is attempted for all intermediate generation programs. Therefore, in each of the six function modules described above, the timing at which the intermediate generation program with the smallest number of operations is generated among the intermediate generation programs that have been successfully parsed is recognized as the generation timing of the function module. For example, with regard to “IsWordEnd” 1021, it is recognized that the syntax analysis has succeeded in the intermediate generation program generated at timing 1021a, and the function module “IsWordEnd” 1021 exists in the source code. Similarly, in the intermediate generation program generated at timings 1022a, 1023a, 1024a, 1025a, and 1026a for “IsSmallAlpha” 1022, “IsBigAlpha” 1023, “IsAlNum” 1024, “CountWord” 1025, and “main” 1026, respectively. It has been recognized that the parsing has succeeded and the function modules “IsSmallAlpha” 1022, “IsBigAlpha” 1023, “IsAlNum” 1024, “CountWord” 1025, and “main” 1026 exist in the source code.

  Thus, based on the analysis result of the timing analysis unit 33, it is possible to recognize at which timing the function module is generated in the source code. However, the timing here means the timing when the “analysis was successful” by the timing analysis unit 33, and Mr. A actually noticed the necessity of the function module and “described” the function module in the source code. It does not mean timing.

  Therefore, in the present embodiment, the search unit 34 searches for the timing when the subject starts describing the function module in the program. Specifically, whether or not there is a description of the function module by the subject before the timing analyzed by the timing analysis unit 33 is searched using a plurality of intermediate generation programs. In FIG. 27, a beard 1021b protrudes to the left of the timing 1021a of “IsWordEnd” 1021. This indicates that the function module has been described by the subject at a timing before the timing 1021a based on the search result of the search unit 34. In this way, by knowing the timing immediately after the test subject has written the function module, it is possible to grasp the test subject's thought process, such as the time when the test subject notices the necessity of the function module, and this is useful for the diagnosis of programming ability. Can do. In FIG. 27, the program created by the subject is composed of a plurality of function modules, but may be composed of a single function module.

  FIG. 28 is a diagram showing a flowchart in the analysis mode for displaying a diagram for analyzing the programming creation process. The click position of the radio button of the display content 1013 in the right column is the position of the flowchart. This flowchart shows a diagram in which the processing procedure of each of the plurality of function modules is visualized by the visualization diagram generation unit 32 based on the test passing program.

  According to FIG. 28, as described above with reference to FIG. 27, the program created by Mr. A includes “IsWordEnd” 1021, “IsSmallAlpha” 1022, “IsBigAlpha” 1023, “IsAlNum” 1024, “CountWord” 1025, It can be seen that a total of six function modules of “main” 1026 are created. Further, it can be seen that three subroutines 1026b are incorporated in the processing procedure of “main” 1026 on the rightmost side. Similarly, “IsWordEnd” 1021 and “CountWord” 1025 also incorporate one or more subroutines. In this way, by confirming the flowchart shown in FIG. 28, it is possible to grasp at a glance how the subject is making the function module into a subroutine.

  Further, according to FIG. 28, it can be seen that the processing procedure 1025b in a nested state is incorporated in the processing procedure of “CountWord” 1025 on the leftmost side. Similarly, a nested processing procedure 1024 b is incorporated in the processing procedure of “IsAlNum” 1024. However, as can be seen from the nested processing procedure 1025b as compared to the nested processing procedure 1024b, a lot of branches and loops are used, which is a complicated algorithm. Therefore, Mr. A does not have a good evaluation on the diagnostic criteria that a simple algorithm with few branches and loops can be realized. Whether or not the algorithm is simple is often unknown only by looking at Mr. A's processing procedure. Therefore, a more accurate ability diagnosis can be performed by comparing the processing procedures of a plurality of subjects (see FIG. 33 and the like described later) and performing a relative evaluation (which algorithm is simple).

  In the present embodiment, the flowchart is considered as a diagram in which the processing procedure of each of the plurality of function modules is visualized. However, the present invention is not limited to this, and the program such as an HCP chart including an explanation near the symbol is used. It is also possible to use a design drawing that expresses the purpose in a hierarchical structure. Further, the visualization diagram generation unit 32 may generate a diagram in which a structure between a plurality of function modules is visualized, instead of a diagram in which processing procedures of each of the plurality of function modules are visualized. For example, it is a diagram showing a data flow between functions such as a structure diagram describing the structure of the system such as a class diagram and a DFD (data flow diagram). Further, a diagram that visualizes both the processing procedure of each of the plurality of function modules and the structure between the plurality of function modules may be generated. Furthermore, although a plurality of function modules are shown in FIG. 28, one function module may be shown. In this case, a diagram in which the processing procedure of one function module is visualized or a diagram in which the structure of one function module is visualized is generated.

  FIG. 29 is a diagram showing code duplication in an analysis mode for displaying a diagram for analyzing a programming creation process. The click position of the radio button of the display content 1013 in the right column is a code overlap position. The code duplication rate indicates the degree of code duplication among a plurality of function modules, and is analyzed by the code duplication rate analysis unit 35. A subject who has created a program with a low code duplication rate has an excellent ability to construct an efficient program structure with little waste.

  The code duplication rate 1030 shown in FIG. 29 is 0%, and there is no duplication at all. That is, when describing the source code, it is presumed that, for example, copy and paste or the like is not used at all (in the example described later with reference to FIG. 34, the code duplication rate is close to 50%). Therefore, it can be seen that Mr. A is an excellent programmer in terms of whether an efficient program structure can be constructed with little waste. In addition, if the code duplication rate is low, the number of changed portions is reduced even when a specification change occurs. In this sense, it can be said that Mr. A's program is excellent in maintainability. Any technique for analyzing the code duplication rate may be adopted. For example, the code duplication rate can be analyzed by searching for duplicate code strings in the source code that are mainly generated by copying and pasting the source code. Specifically, the source code (test passing program) is divided into tokens according to the lexical rules of the programming language, and tokens belonging to types, variables, and constants are replaced with the same token. By this replacement, for example, a set of code strings that differ only in variable names can be detected as duplicate code strings. In FIG. 29, the degree of code duplication among a plurality of function modules is shown, but the degree of code duplication within one function module may be shown.

  As described above with reference to FIGS. 26 to 29, the user can selectively display each figure on the display unit 25 only by changing the click position of the radio button of the display content 1013 through the reception unit 24. be able to. Therefore, the ability to build the program structure can be diagnosed easily and quickly from various viewpoints.

  FIG. 30 is a diagram illustrating a state in which parameter transitions of two subjects are compared. In FIG. 30, the upper part shows the parameter transition 1031a of Mr. A described above, and the lower part shows the parameter transition 1031b of Mr. B (see the target data file 1002 in the left column) newly added. Such comparison display can be realized by adjusting 2 × vertical × 1 horizontal in the number of graphs 1012 in the right column. Note that FIG. 30 is an example in which the number of graphs 1012 in the right column is adjusted to 2 vertical × 1 horizontal, but by adjusting the vertical 1 × 2 horizontal, two graphs are arranged horizontally. It is also possible.

  Here, in FIG. 30, the function of the normalization 1015 of the parameter setting 1014 is used to facilitate the comparison. Specifically, the horizontal axis column is checked in the check box of normalization 1015. As a result, the value of the right end 1031c of the horizontal axis of the parameter transition 1031a of Mr. A becomes 100, and the value of the right end 1031d of the horizontal axis of the parameter transition 1031b of Mr. B also becomes 100. FIG. 31 shows an example in which not only the horizontal axis but also the vertical axis is normalized unlike FIG. Specifically, the value of the vertical axis upper end 1031e of Mr. A's parameter transition 1031a is 100, and the value of the vertical axis upper end 1031f of Mr. B's parameter transition 1031b is 100.

  As described above, the programming capability diagnosis apparatus (client terminal 2A) according to the present embodiment has a function of normalizing the horizontal axis or the vertical axis in the diagram for analyzing the programming creation process. This normalization function can facilitate relative comparison. This is particularly convenient when you want to see the “graph shape” rather than the “absolute value” of each axis of the graph. For example, when looking at the transition of the number of characters, if you want to visualize "how many characters you wrote, how many minutes you wrote", a graph that does not use the normalization function is more convenient, while in the first half and second half, A graph using the normalization function is more convenient when you want to visualize which character has the greatest growth.

  FIG. 32 is a diagram illustrating a state in which the function life cycles of two subjects are compared. In the function life cycle of Mr. B shown at the lower side of FIG. 32, a whiskers 1032b protrude to the left of the timing 1032a of “CheckWord” 1032. This indicates that the function module “CheckWord” 1032 has been described by the subject at a timing before the timing 1032a, as described above, based on the search result of the search unit 34. In FIG. 32, comparing Mr. A's function life cycle (upper side) with Mr. B's function life cycle (lower side), Mr. A who has all the function modules necessary for the program at an early timing is It can be said that the ability to see through the structure of the program is high. On the other hand, Mr. B, who has a small number of function modules (6 for Mr. A and 3 for Mr. B), has a higher ability to simplify the algorithm. The person who diagnoses the programming ability does not need to determine the evaluation only by the function life cycle shown in FIG. 32, and may perform a comprehensive evaluation while referring to flowcharts and code duplication. If a programmer who constructs a simple algorithm is desired, Mr. B should be highly evaluated. If a programmer with high power to see the program structure is desired, Mr. A should be highly evaluated.

  FIG. 33 and FIG. 34 are diagrams showing a comparison of flowcharts and code duplications of three subjects, respectively. As shown in FIG. 33, when the flowcharts of three subjects are displayed side by side, it is possible to grasp at a glance how much the algorithm is relatively simple. The flowchart of Mr. E with nine function modules is much more complicated than Mr. B with three function modules and Mr. D with four function modules. One of the causes of such a complicated flowchart is that copy and paste is frequently used when describing source code. Therefore, when the code duplications of Mr. B, Mr. D, and Mr. E are compared, they are 0%, 0% and 48%, respectively, as shown in FIG. In other words, it can be seen that Mr. E may have copied and pasted nearly half of it. After that, in the graph showing each parameter transition, if the transition of the total number of characters is confirmed and a portion where the number of characters is rapidly increased is found, the possibility of copying and pasting increases.

  FIG. 35 is a diagram showing a state where the click position of the radio button of the horizontal axis value 1016 in the parameter setting 1014 is changed from the number of operations to time, unlike FIG. That is, in FIG. 35, the elapsed time is plotted on the horizontal axis (the value at the right end of the horizontal axis is the elapsed time 5259 seconds) unlike FIG. 26, and each parameter transition over time can be grasped at a glance. Thus, for example, when attention is paid to the number of operations 1041, it is possible to grasp how many time periods (for example, the section 1041a) when no operation is performed when the source code is described.

  FIG. 36 shows an example of a facial expression mode setting screen 1051 for changing the facial expression of a character when a time period (waiting) during which the subject is not operating anything occurs for a predetermined time. The display mode setting screen 1051 shown in FIG. 36 appears in the center as a pop-up window by clicking the character button 1006d arranged on the right side of the elapsed time 1006. According to the facial expression mode setting screen 1051, the character's facial expression becomes a favorable facial expression when continuously operated within 0.5 seconds, and if the waiting time occurs for 10 seconds, the facial expression of the character is an annoying facial expression. When the waiting time occurs for 30 seconds, the character's facial expression becomes a sleeping facial expression. Thus, the programming ability diagnostic apparatus (client terminal 2A) according to the present embodiment has a function of changing the facial expression of the character displayed on the display unit 25 based on the operation interval of the subject through the reception unit 24.

  As described above with reference to FIGS. 17 to 36, according to the programming ability diagnosis performed at the client terminal 2A, the ability to solve the programming problem, particularly the ability to construct the structure of the program, can be diagnosed easily and quickly. It is possible to prevent mis-evaluation of the program due to misunderstanding of the contents of the programming task.

  However, since these have strong qualitative aspects, there is a possibility that the diagnosis results vary depending on the person who performs the programming ability diagnosis. Even if a certain degree of variation is allowed, it is preferable that there is some objective index that gives the same result when anyone makes a diagnosis. For example, a source code in which many comments are written in pursuit of readability is not necessarily a poor quality source code. However, it is not preferable from the viewpoint of productivity if it takes too much time for one problem. Therefore, a quantitative evaluation of the subject's programming ability will be described below.

[Quantitative skill evaluation]
In the present embodiment, the productivity / quality evaluation application 174 (see FIG. 2) is used to quantitatively evaluate the programming ability of the subject. That is, the CPU 11 executes the productivity / quality evaluation application 174 and reads the log data and timing described above from the answer DB 180 to perform quantitative skill evaluation.

  The answer DB 180 of the server 1 includes basic information as time-series transition information of the source code, the time when the subject checked the grammatical error, the type of grammatical error that occurred, the number of grammatical errors that occurred, The correction information such as the check time, the type of logic error that occurred, the number of logic errors that occurred, etc. is stored. As described above with reference to FIG. 15, the time when the build button 903 or the test button 905 is clicked, and the type and number of output grammatical errors as a result are stored.

  FIG. 37 is a view showing an example of the data structure stored in the answer DB 180. As shown in FIG. FIG. 37A is a view showing an example of the data structure of the input information table. The time-series transition information of the source code, which is basic information, is stored in a format in which the source code can be restored at a specific timing in the subject's programming work. The format is not particularly limited as long as it can be restored. For example, in FIG. 37A, the input text data and the like are stored character by character together with the input time and the deletion time. By analyzing the input information table from the top, the source code at an arbitrary timing can be restored. The input information may include not only text data but also line breaks, tabs, spaces, various control codes, and the like. As another storage method, an input time for each character may be stored in association with all source codes at each input time. In this case, although the data capacity is relatively large, the source code restoration algorithm at an arbitrary timing is simple.

  In any method, since the transition information is stored for each timing when the character is input, the source code at any timing in the programming work of the subject can be restored. Actually, it is not always necessary to restore the source code at all timings, but in the calculation of the approximate curve from the time series transition data of complexity described later, the number of samplings that can maintain the predetermined accuracy is Necessary. For example, a source as an intermediate generation program at a predetermined timing that matches a predetermined condition such as a unit of a predetermined number of key inputs, a timing at which a return key is operated, or a timing at which a predetermined character is input. You may make it acquire a code.

  FIG. 37B is an exemplary diagram of the data configuration of the grammatical error table, and FIG. 37C is an exemplary diagram of the data configuration of the logical error table. Information related to the grammatical error check, which is correction information, and information related to the logical error check are stored in a grammatical error table and a logical error table, respectively, as shown in FIGS. 37B and 37C, for example. A grammar check execution time or a logic check execution time is also stored in association with a grammatical error type or a logical error type. A separate counter can be used to count the total number of occurrences of grammatical errors and logic errors. These various tables can be stored in the answer DB 180. As described above, the execution time of the grammatical error check can be stored by recognizing the timing when the build button 903 or the execution button 904 (FIG. 15) is clicked, and the execution time of the logical error check. Can be stored by recognizing the timing when the test button 905 (FIG. 15) is clicked.

  The CPU 11 of the server 1 calculates complexity transition data obtained by calculating the complexity transition in time series based on the time series transition information of the source code. Here, “complexity” is not only a concept indicating structural complexity such as many branches or many hierarchies, but also whether or not the entire source code includes useless logical structures, useless subroutines, etc. It is a concept that means an index value for evaluating whether or not the source code is structured including evaluation. Generally, at the beginning of program creation, the complexity is 0 (zero), and increases as the coding progresses. Then, it converges to a specific value when the program is completed.

  The concept of “complexity” will be described in detail below. In the present embodiment, the number of test cases necessary to completely test the created program (the number of test patterns), and the quantitative value correlated with the man-hour required to change, maintain, etc. the created program, This is called the complexity of the program. Since there is a correlation between the number of test cases and the man-hours required for program change, maintenance, etc., the concept may be represented by either one. In order to simplify the description, a case where only the number of test cases is used will be described.

  FIG. 38 is an exemplary diagram showing a specific example of the presented programming task. Here, an example in which the content is different from the programming task described above is shown. FIG. 39 is a view showing an example of a program in C language created by the subject according to the presented programming task. For calculation of complexity, various quantitative values appearing in the program can be used. The most common complexity is the number of branches in the program. As a simple way of counting, in the program shown in FIG. 39, there are six if statements (lines 13, 20, 21, 34, 36, and 38) and one while statement (line 33). There are a total of 7 branches. In general, if the number of branches is the same for all other conditions, the greater the number of branches, the greater the number of test cases for a complete test. Therefore, the number of branches 7 can be used as a quantitative value indicating complexity. .

  Similarly, even when focusing on a branch, if there are two determination conditions in one if statement as in the if statement on the 20th line, there is a way of counting that this if statement includes two branches. The judgment condition included in the return statement on the ninth line is also considered as an implicit branch and may be counted as four branches. When counting the number of branches in this way, there are 4 branches in the program shown in FIG. 39, 1 in the 13th line, 2 in the 20th line, and 1 in the 21st line. 1 in the 22nd line, 1 in the 26th line, 1 in the 33rd line, 1 in the 34th line, 2 in the 36th line, 2 in the 38th line, and the total number of branches Will be 16.

  In addition, the number of various types of variables defined in each language specification may be used as a quantitative value correlated with the number of test cases. For example, since five local variables are declared in the program shown in FIG. 39 (lines 18, 19, 30, 31, 32), the complexity of the program shown in FIG. 5 is considered. In this case, a global variable may be added, and an argument may be added as a kind of variable.

  The number of interfaces may be used as the complexity. For example, the program shown in FIG. 39 includes four functions having one argument (lines 7, 11, 24, and 28) and one function having two arguments (line 16). If it is considered that the number of arguments is the number of interfaces, the quantitative value 6 can be obtained as the number of interfaces of this program. The number of interfaces also depends on what units are considered for external interfaces. For example, the number of interfaces may be the number of public functions in a class, the number of external public functions in file units, or the like.

  For the same reason, nesting levels may be adopted. For example, in the program shown in FIG. 39, there is an if statement on the 34th line in the while statement on the 33rd line, and an if statement on the 36th line. That is, the branch is nested in three stages, and a quantitative value 3 is obtained as the maximum nesting level in the entire program. In addition, instead of simply using the maximum nesting level, the complexity is calculated as a value obtained by multiplying each quantitative value by a weighting factor corresponding to the nesting level, for example, with the idea that the branch in the branch has a strong adverse effect. Also good.

  In addition, for example, various quantitative values such as the number of states are adopted, but generally, any quantitative value correlated with the number of test cases and the man-hour required for maintenance or change can be used as the complexity. Further, a plurality of quantitative values may be combined for the purpose of increasing the calculation accuracy of the complexity. For example, assuming that the number of branches is B, the number of variables is V, the number of interfaces is I, and the nest level is N, a predetermined weighting may be applied as in f (B, V, I, N). The calculation method may be calculated collectively for the entire program, or may be recalculated as a whole for each unit such as a function. It is sufficient to determine which quantitative value is used and how it is calculated according to the calculation purpose of the required calculation accuracy and complexity.

  Further, the quantitative value indicating the complexity may be determined in consideration of the specification of the programming language to be used. For example, the program shown in FIG. 39 is written in the C language, but in the case of a language having a class concept such as C ++, JAVA (registered trademark), etc., the quantitative value for evaluating the complexity of the class structure is considered. May be. Further, the complexity is not directly calculated from the source code, but may be calculated from a design drawing such as an intermediate code or a flowchart.

  Conversely, parameters that do not correlate with the number of test cases and the man-hours required for maintenance or change cannot be used as quantitative values indicating complexity. For example, the simple number of characters and the number of lines in the source code are considered to be highly evaluated as the source code is carefully commented to facilitate post-maintenance. It is inappropriate to use it as an indicator. In the program of FIG. 39, relatively long variable names (lines 30, 31, 32) are used. However, such a length (token length) is not related to the number of test cases, and maintenance is performed. Or, in terms of the man-hours required for the change, it cannot be said that the longer (shorter) is better, so at least it cannot be used alone as the complexity. The number of function points, the number of screens, etc. are also unsuitable for use alone as complexity. In general, a quantitative value that simply represents the software scale is inappropriate for use alone as a complexity (however, when calculating the complexity, it may be calculated with a quantitative value that represents the original complexity). except).

  Further, a quantitative value indicating whether or not a specific coding rule is satisfied is often unrelated to the complexity in the first place, and at least alone, it is not used as the complexity. However, it may be used only when the coding convention itself has an effect of reducing complexity.

  The CPU 11 of the server 1 calculates the ability evaluation value of the subject based on the time series transition data of the complexity. The ability evaluation value is a value at which the ability is evaluated to be higher as the subject changes in a state where the complexity value is lower and as the subject is completed in a shorter time. That is, it sets to the value which evaluates that the test subject who has found a simple solution in a shorter time has higher ability.

The capability evaluation value can be calculated independently from the evaluation value related to the quality capability and the evaluation value related to the productivity capability. For example, complexity is an index that originally indicates quality, and subject A and subject B have similar durations, and if A's complexity is lower than B's complexity, A's quality capability is It can be evaluated that it is higher than the quality capability of B. In addition, the complexity of A and the complexity of B have changed at the same level, but if A is completed in a shorter time than B, it is evaluated that A is more productive than B be able to.

Furthermore, stored as correction information, the time when the subject generated a grammatical error, the type of error, the cumulative value of the number of grammatical errors, the time when the logical error was generated, the type of error, the cumulative value of the number of logical errors Etc., the evaluator can also correct the obtained evaluation value from the viewpoint of an error actually generated by the subject.

  In the following, a processing procedure from presenting a programming task to the subject (taking the question) and collecting data for ability evaluation from the subject's programming work will be described. FIG. 40 is a flowchart illustrating a procedure of data collection processing performed by the CPU 11 of the server 1.

  40, the CPU 11 of the server 1 displays the task name of the programming task on the display unit 25 of the client terminal 2B as described above with reference to FIG. 11 (step S11). When the operation signal of the start button 806 (see FIG. 11) indicating the start of programming is received from the client terminal 2B, the timer starts counting (step S12), and it is determined whether or not character input has been accepted (step S12). S13). Note that the character input is not limited to the input of text data, but includes the input of control codes related to editing operations such as deletion, line feed, tab, cursor movement, copy and paste, and search operation.

  When the CPU 11 determines that character input is not accepted (step S13: NO), the CPU 11 enters a character input waiting state. When CPU11 judges that the character input was received (step S13: YES), CPU11 memorize | stores in answer DB180 in the format which can decompress | restore the source code at the arbitrary timings of a test subject's programming work (step S14). . Note that all basic information may be stored, or may be stored by thinning out within a range where evaluation value calculation accuracy can be maintained.

  The CPU 11 determines whether or not a grammatical error check start instruction has been received from the subject (that is, whether or not the build button 903 has been clicked on the programming screen shown in FIG. 15) (step S15). Is determined not to be accepted (step S15: NO), the CPU 11 waits for a start instruction. When the CPU 11 determines that the start instruction has been received (step S15: YES), the CPU 11 performs a grammatical error check on the source code created by the subject when the start instruction is received, and starts the grammatical error check. The time when the instruction is accepted is stored in the answer DB 180 (step S16), and it is determined whether or not there is a grammatical error (step S17). In the case of a compiler language, whether or not there is a grammatical error may be determined based on a character string output as a compilation result by a commercially available compiler. At the same time, an executable module may be generated. Further, the answer DB 180 is not limited to storing the time when the instruction to start the grammatical error check is received, and may be the completion time of the grammatical error check, for example.

  If the CPU 11 determines that there is a grammatical error (step S17: YES), the CPU 11 adds and stores the type and number of existing grammatical errors in the answer DB 180 (step S18), and returns the process to step S13. The above process is repeated. Although not shown in the flow, even when there is no grammatical error, information indicating that there is no grammatical error is stored in the answer DB 180 in association with the completion time of the grammatical error check.

  When the CPU 11 determines that there is no grammatical error (step S17: NO), the CPU 11 skips step S18 and determines whether or not a logical error check start instruction is received from the subject (that is, the programming shown in FIG. 15). It is determined whether or not the test button 905 has been clicked on the screen (step S19). When the CPU 11 determines that the start instruction has not been received (step S19: NO), the CPU 11 enters a start instruction waiting state. When the CPU 11 determines that the start instruction has been received (step S19: YES), the CPU 11 performs an automatic unit test according to the test case on the program created by the subject at the time when the start instruction is received. The time when the error check start instruction is received is stored in the answer DB 180 (step S20), and it is determined whether or not there is a logic error (step S21). Whether or not there is a logic error may be determined based on whether or not a correct expected output can be obtained with respect to the input of the test case.

  When the CPU 11 determines that there is no logic error (step S21: NO), the CPU 11 determines that the program is completed and ends the process. When the CPU 11 determines that a logic error exists (step S21: YES), the CPU 11 adds the type and number of existing logic errors to the answer DB 180 and stores them (step S22), and returns the process to step S23. The above process is repeated. Although not shown in the flow, even if there is no logical error, information indicating that no logical error exists is stored in the answer DB 180 in association with the completion time of the logical error check. Further, in preparation for a case where the subject has an intention to perform source code shaping or the like, it may be left to the subject to determine whether to end the process or continue the programming operation.

  As described above, the flowchart shown in FIG. 40 includes a removal process in which grammatical errors and logic errors are removed. The removal process is important in ensuring the identity of external specifications. Even if the programming tasks are the same, it is not guaranteed that the external specifications of the created programs are the same. For example, if a bug remains in the program of the subject A and no bug exists in the program of the subject B, this is equivalent to a difference in external specifications, and the difference affects the complexity. Even if the complexity transitions of the subject A and the subject B are compared in this state, it cannot be determined whether the difference is due to a difference in external specifications or a difference in ability of the subject. In the flowchart shown in FIG. 40, there is no such concern because a removal process in which grammatical errors and logic errors are removed is included.

  The data collection process for quantitative skill evaluation has been described above. In FIG. 40, for convenience of explanation, a grammatical error is detected by a build and a logical error is detected by an automatic unit test. However, both may be detected by a build or both may be detected by an automatic unit test. Good. Hereinafter, a processing procedure for calculating a capability evaluation value based on data collected by the server 1 will be described. FIG. 41 shows a flowchart of the processing procedure for calculating the ability evaluation value of the CPU 11.

  First, the CPU 11 uses the time series transition information of the source code stored in the answer DB 180 to calculate the complexity in time series while sequentially restoring the source code in the process of programming work, and the time series transition of the complexity Data is created (step S31).

  The complexity is calculated using the source code as input. As a simple calculation method, for example, the number of basic paths that is a value obtained by adding 1 to the number of determination conditions (including complex conditions) in McCabe complexity may be used as the basis of calculation. It can be considered that the complexity increases as the number of basic paths increases.

  If it is necessary to improve the accuracy of the evaluation, the sources such as the types and number of variables, the number of procedures such as functions, the number of interfaces between modules, the size of the source code (for example, the number of words), etc. The calculation may be performed by adding a parameter related to the complexity of the code and performing predetermined weighting. It is also possible to use an index other than McCabe as the basis of calculation.

  The index and parameter used for calculating the complexity may be determined according to the required evaluation accuracy. For example, if a difference between a programmer close to a beginner and an excellent programmer who is regularly in charge of a high difficulty module in a company is detected, the difference in complexity is considered to be very large. Capability differences can be detected simply by applying indicators. On the other hand, when the difference in ability between the subject A and the subject B is very small (for example, about twice), the complexity calculation accuracy may be increased using more multifaceted indexes, parameters, and the like.

  FIG. 42 is an exemplary diagram of the transition data of the complexity calculated by the server 1. As shown in FIG. 42, the complexity 91 is calculated for each sampling elapsed time from the start of the time measurement by the timer. In FIG. 42, in addition to the McCabe index, the number of words of the program is also used for calculating the complexity 91.

  Returning to FIG. 41, the CPU 11 extracts an effective section used for ability evaluation from the obtained complexity transition data in order to remove the influence of the posture of the subject. “Effect of subject's posture” means, for example, subject A fully conducted automatic unit tests, source shaping, etc. and completed programming work, but subject B judged that there was no accidental logic error check during coding This happens when the programming work is completed as it is. In this example, even if subject A and subject B have the same required time and final complexity, of course, subject A must be calculated higher as an evaluation value. In order to accurately calculate the evaluation value in such a case, an effective section is extracted from the complexity transition data.

  In order to extract an effective section, the CPU 11 creates an approximate curve for the obtained transition data of complexity (step S32). As a method of creating an approximate curve, for example, a least square approximation by a quadratic curve may be combined, or a plurality of curve patterns may be stored in the answer DB 180 in advance and approximated by a curve pattern that is most approximated by pattern matching. Good.

  Although an approximate curve can be created in principle from several sampling points, it is desirable that there are several tens or more sampling points in order to ensure accuracy. The method of determining the sampling point is not particularly limited, such as a method of providing a sampling point every predetermined time, a method of using a timing when a predetermined number of characters are input as a sampling point, and the like. Further, as a method for determining the representative value at the sampling point, the above-described method for obtaining the average value of the complexity within each predetermined time or each predetermined number can be used.

  FIG. 43 is an illustration of an approximate curve calculated by the server 1. In FIG. 43, the required time is divided into 100, and the approximate curve 101 is created using the average value of the complexity 91 of each section as a representative value. As an approximation method, after determining the aspect ratio of the curve from the final complexity of the source code created by the subject and the required time, pattern matching with about 2000 curves stored in advance in the answer DB 180 is performed. . In order to simplify the processing, the curve pattern group stored in the answer DB 180 is changed to a monotonous increase after the complexity has changed for a certain period of time 0, and is unified into a pattern that converges at a certain complexity. Although pattern matching is performed here, other methods such as weighted moving average may be used.

  Returning to FIG. 41, when the approximate curve 101 is obtained, the CPU 11 extracts an effective section (step S33). FIG. 44 is an illustration of an effective section extracted by the server 1. In order to simplify the processing, the effective section 112 starts from the point where the complexity has changed from 0 to monotonically increases, and ends from the point where the monotonous increase ends. The valid section 112 can be regarded as a pure programming work part that does not include the initial design section 111 and the test section 113. Here, there is also a view that it is problematic in accuracy to exclude all of the initial design section 111 and the test section 113, so that the effective section 112 is extracted in consideration of the initial design section 111 and the test section 113 with a predetermined weight. May be.

  FIG. 45 is an exemplary diagram of the approximate curve 101 created by the server 1 when the effective section 112 needs to be corrected. The effective section 112 to be extracted is extracted as a part of the whole in FIG. However, as shown in FIG. 45, the effective section 112 may be extracted as an expanded effective section 112 '.

  As a case as shown in FIG. 45, a case where a test subject performs a logic error check during programming and a logic error is not detected by chance, a case where the test subject is in a position to neglect the test, etc. Is done. In such a case, it can be interpreted that the subject originally took more time, and it is appropriate to evaluate the ability by expanding the effective section 112.

  Returning to FIG. 41, finally, the CPU 11 performs a predetermined calculation on the extracted effective section 112 to calculate a capability evaluation value (step S34). As a simple method for calculating the ability evaluation value, for example, there is a method of obtaining an integral value for the approximate curve 101 in the effective section 112. That is, it is a method for obtaining the area surrounded by the time axis and the approximate curve 101. Such an area is considered to be the total amount of complexity that the subject has worked on during the programming operation. Since the subject with a smaller area has solved the programming problem in a shorter time and more simply, it can be evaluated that the smaller the area, the higher the ability. Therefore, the integral value (area) becomes the evaluation value as it is.

  The integral value (area) may not be calculated as an absolute value, but may be calculated as a ratio with a reference evaluation value provided separately as a reference evaluation value. For example, a known subject A solves a specific programming task, and the data of the subject A is stored in the task information storage unit 132. When the unknown subject is asked to solve the same programming task thereafter, the ability evaluation value can be calculated by the ratio with the stored “subject A data”. Thus, the method of calculating by the ratio with the ability of the known person has an advantage that it is easy for the development site to intuitively understand the meaning of the evaluation value.

  Further, in calculating the integral value (area), instead of using the approximate curve 101, raw data of complexity may be used. For example, a method of adding interval average values of transition data of complexity may be used. In this case, even if it is semantically, the calculation is merely addition, and the calculation processing load can be reduced. Further, even when the approximate curve 101 is used, if the curve is sufficiently simple, an evaluation value is calculated by simple multiplication using the time width of the effective section 112, the complexity data at a specific position, and a predetermined coefficient. May be.

  Further, as an ability evaluation from another aspect, the number of times that the complexity of a certain sampling point is smaller than the complexity of the immediately preceding sampling point in the effective section 112 may be counted as an evaluation value. Complexity increases monotonically as programming progresses smoothly. Complexity may be reduced. It can be determined that as the number of times counted increases, the initial design prospect is poor and the degree of trial and error is high.

  An evaluation value indicating the tendency of the subject can also be obtained from the extracted position, size, etc. of the effective section 112. For example, when the extracted effective section 112 is extremely located in the first half of all the transition data of the complexity, it can be determined that the tendency to start without reading the programming task well is strong. Starting development without understanding the specifications is a development attitude that tends to adversely affect quality.

  On the other hand, if the extracted effective section 112 is positioned extremely in the latter half of all the transition data of complexity, it can be determined that the operation has been completed without performing the operation check well. Such an attitude naturally has an adverse effect on quality. If the extracted effective section 112 is extremely narrow even though the total required time is very long, it may have taken too much time to understand the specifications and check the operation, and there is room for improvement in productivity. Can be determined to be large. It is easy to quantify the positional deviation of the effective section 112 and the ratio of the effective section 112 to the whole.

  The ability evaluation value can also be calculated independently on the quality ability and productivity ability of the subject. This is because the complexity transition data itself is created with the vertical axis taking the complexity deeply related to quality and the horizontal axis taking the time deeply related to productivity.

  FIG. 46 is an exemplary diagram showing transition data of complexity of four characteristic subjects A, B, C, and D. FIG. In FIG. 46, the subject A is a graph that has reached high complexity in a short time. The hand is fast but the code tends to be messy, and has a pattern of high productivity but low quality. Similarly, it can be interpreted that subject B has low productivity ability and quality ability, subject C has high productivity ability and quality ability, and subject D has low productivity ability but high quality ability.

  Quality and productivity characteristics can be easily quantified. The simplest quantification method is a method of drawing a rectangle including a curve and obtaining the area and aspect ratio of the rectangle. Based on the area and the aspect ratio, the position of the four patterns shown in FIG. 46 can be quantitatively calculated. The result of the position calculation is shown by the plot (black circle) in FIG. In obtaining the rectangle, the entire transition data of complexity may be used, but the evaluation accuracy is higher when the extraction of the effective section 112 is taken into consideration.

  In this way, by using the transition data of complexity, it is possible to calculate not only the ability evaluation value of the subject but also the quantitative value that goes into the characteristic by executing the creation of the approximate curve, the extraction of the effective interval, etc. it can. In addition to the examples described above, any evaluation index that is calculated based on the complexity and deserves evaluation of programming ability can be used as an evaluation value.

  Finally, the CPU 11 presents the correction information stored in the answer DB 180 (subject error check timing, information on generated errors, etc.) to the evaluator in the form of a list or graph. The format to be presented may be any format as long as the evaluator knows when the subject performs an error check and what kind of error occurred at that time. In addition, the presentation method may be displayed on the display unit 25 (or the display unit 15 of the server 1) of the client terminal 2A of the challenger, may be printed, or later viewed with other software or another computer. It is possible to store or transmit the data in a predetermined format.

  In the flowchart of FIG. 41, correction information (subject error check timing, information on an error that has occurred, etc.) is not used in calculating the ability evaluation value. The reason for not using the server 1 is that it is a programming task that is completed in about 30 minutes to 2 hours when the server 1 is used for a company entrance examination or the like. This is because the reproducibility is high (therefore, the reproducibility is low) and it is inappropriate as a parameter for the ability evaluation value.

  However, if the error situation was too extreme, there were, for example, excellent subjects who passed the grammatical error check and logical error check at once, and conversely, subjects who programmed while giving too many errors. In this case, error information should be considered in addition to the capability evaluation value. For this reason, the correction information is not used for calculation of the ability evaluation value, but is presented to the evaluator.

  However, if the subject's time constraints are relaxed, for example, if a statistically significant period (such as several days) is allowed in educational training in a company, this is not the case, and the correction information is used to calculate the ability evaluation value. Also good.

  In addition, although the case where a test subject was made to perform C language programming work for ability evaluation was described here, C language programming work is one example, and is not the essence of the ability evaluation value calculation here. The work content may be any work as long as the complexity can be calculated from the test passing program and the intermediate generation program created by the subject.

  According to the [quantitative skill evaluation method] described above, the intermediate generation program (log data) of the source code created by the subject is acquired from the client terminal 2B at a predetermined timing, and the timing and the complexity of the intermediate generation program are obtained. Based on this, since the skill of the subject can be quantitatively evaluated, the programming skill of the subject can be easily and objectively evaluated.

  More specifically, returning to FIG. 16 that can be visually recognized by the subject, when the chart display button 637 is clicked, for example, the display screen of the productivity / quality chart 6371 shown in FIG. 47 is displayed on the display section of the client terminal 2A. Appears at 25. The chart display screen shown in FIG. 47 corresponds to the screen shown by the example diagram shown in FIG. 46, and relatively shows the productivity ability and quality ability in the programming of the subject. The horizontal axis indicates the quality capability (indicated as quality in FIG. 46), and the vertical axis indicates the productivity capability (indicated as productivity in FIG. 46). 47, the forward direction is reversed, unlike FIG. That is, in FIG. 47, regarding the quality capability on the horizontal axis, the quality capability increases as it goes to the left, and as to the productivity capability on the vertical axis, the productivity capability increases as it goes downward. 47, the subject 2 (indicated by 2 surrounded by a square frame), the subject 3 (indicated by 3 surrounded by a square frame), the subject 5 (indicated by 5 surrounded by a square frame), the subject 6 The performance evaluation (indicated by 6 surrounded by a square frame) is performed. In the answer confirmation screen shown in FIG. 2, No. 3, no. 5, no. This is because 6 is selected.

  As shown in FIG. 47, the productivity and quality of the subject 3 are both 2.0. The productivity and quality of the subject 6 are both 5.0. This means that the subject 3 is 2.5 times more productive and the quality is about 2.5 times better than the subject 6 (that is, the lower the left of the productivity / quality chart 6371, the higher the ability). To be placed). Therefore, if the subject 6 is an existing employee and the subjects 2, 3, and 5 are applicants for the recruitment test, the subject 3 is employed because it is several times more capable than the existing employee (subject 6). Can make appropriate decisions such as As described above, in order to relatively determine the skill of the subject, data serving as a reference in the productivity / quality chart is required, so the item of the reference 632 shown in FIG. 16 is provided. Then, in the productivity / quality chart 6371 shown in FIG. 47, the subject person can easily determine the subject's skill relative to each other by putting a color in the square frame indicating the subject 6 (the person who becomes the reference). .

  In FIG. 47, a reference position adjustment box is displayed so that the position of the reference person can be freely changed. Specifically, a productivity capability reference position adjustment box 6372, a spin button 6373 for increasing / decreasing a value in this box, a quality capability reference position adjustment box 6374, and a spin button 6375 for increasing / decreasing the value in this box are displayed. The change pitch is in increments of 0.1. The default reference position is (5.0, 5.0). In order to change the reference position, the challenger may click the spin button 6373 or the spin button 6375.

  Further, in FIG. 47, a check box 6376 for selecting whether or not to draw a constant area line 6377 on the productivity / quality chart 6371 is provided. One or a plurality of constant area lines 6373 drawn in the productivity / quality chart 6371 are drawn so that productivity capacity value × quality capacity value is constant. Thereby, it can be relatively judged whether each subject's skill is comprehensively superior or inferior to the reference subject 6. Note that when the close button 6378 is clicked, the screen shown in FIG. 47 is closed.

  As described above, the productivity / quality chart described with reference to FIGS. 37 to 47 can be displayed in the analysis column 1011 by providing a check box in the parameter setting 1014 shown in FIG. 26 (not shown in FIG. 26). Specifically, when the test passing program and the intermediate generation program are downloaded to the client terminal 2 </ b> A, they may be downloaded together and stored in the storage unit 27.

  In addition, as a function of the programming ability diagnostic apparatus (client terminal 2A) according to the present embodiment, there is a diagnostic report output function. FIG. 48 is an example of a diagnostic report (first sheet), and FIG. 49 is an example of a diagnostic report (second sheet). The reports shown in FIGS. 48 and 49 include the reduced version productivity / quality chart 1071 obtained by reducing the productivity / quality chart described with reference to FIGS. 37 to 47 and the programming described with reference to FIGS. A diagram (a reduced version complexity transition graph 1072, a reduced version code duplication diagram 1073, a reduced version function life cycle diagram 1074, a reduced version flowchart 1075), which is a reduced version of each diagram for analyzing the creation process, is printed. In addition, a comment field is printed in the vicinity of each figure, and a person who performs a programming ability diagnosis can output a report after entering a comment.

  Thus, by providing a diagnostic report output function, a qualitative and quantitative diagnostic result can be provided to a subject. Note that this diagnostic report output function is such that the control unit 20 of the client terminal 2A reads the reduced version productivity / quality chart 1071 and the like from the storage unit 27 and prints out a printer (not shown in FIG. 17) including a printer or the like. ) Can be realized by transmitting a print signal. This printing unit may be a printer or the like externally connected to the client terminal 2A (via USB or the like). Alternatively, as a diagnostic report output function, data may be output in Excel (registered trademark) format.

[Main effects of the embodiment]
As described above, according to the programming capability diagnosis device (client terminal 2A) according to the first embodiment of the present invention, the processing procedure (such as the flowchart shown in FIG. 28) of each of the plurality of function modules or between the plurality of function modules. The ability to build the structure of a program, even if you are not familiar with programming or if there are a large number of subjects Diagnosis can be made easily and quickly.

  Also, by using the test button 905 in FIG. 12, a program created by misunderstanding the content of the programming task can be excluded from the target of the programming ability diagnosis. It is possible to prevent erroneous evaluation due to the creation.

  Further, the function of the timing analysis unit 33 can know the timing at which each of the plurality of function modules is generated, and thus diagnoses the ability of how quickly the program structure can be built in the head. can do. Moreover, according to the function of the search part 34, the timing when the function module was described by the test subject can be grasped. In addition, the ability of the code duplication rate analysis unit 35 to diagnose whether or not the program structure can be efficiently constructed can be diagnosed.

  Furthermore, the person who performs the programming ability diagnosis operates the reception unit 24 to display the diagram generated by the visualization diagram generation unit 32, the analysis result of the timing analysis unit 33, and the analysis result of the code duplication rate analysis unit 35. Since the display unit 25 can selectively switch, the ability to construct the program structure can be easily and quickly diagnosed from various viewpoints.

DESCRIPTION OF SYMBOLS 1 Server 2A, 2B Client terminal 20 Control part 24 Reception part 25 Display part 26 Communication part 27 Storage part 31 Playback part 32 Visualization figure generation part 33 Timing analysis part 34 Search part 35 Code duplication rate analysis part

Claims (7)

In a program creation device in which a subject who has given a programming task creates a program that solves the programming task, the subject is created by the subject, and a desired program output set by the questioner who has given the programming task is obtained. And a test passing program that is composed of one processing module or a plurality of processing modules and that has obtained the desired program output and passed the test, and the subject passes the test. A program capability diagnosis device that reads a program configuration information capable of restoring a plurality of intermediate generation programs generated in the process of creating a program, and diagnoses the programming capability of the subject,
A storage unit for storing the test passing program and the program configuration information;
Based on the plurality of intermediate generation programs restored by the program configuration information, a playback unit that plays back the creation process of the test passing program;
Based on the test passing program, a visualization diagram generation unit that generates a diagram in which a processing procedure of each of the one processing module or the plurality of processing modules or a structure between the one processing module or the plurality of processing modules is visualized When,
A programming capability diagnosis apparatus comprising: a control unit that causes a display unit to display a process for creating the test passing program reproduced by the reproduction unit and a diagram generated by the visualization diagram generation unit. Based on the plurality of intermediate generation programs restored by the program configuration information, comprising a timing analysis unit that analyzes the timing at which the one processing module or each of the plurality of processing modules is generated,
The programming ability diagnosis apparatus according to claim 1, wherein the control unit displays the analysis result of the timing analysis unit on the display unit. For each of the one processing module or the plurality of processing modules, whether or not there is a description of the processing module by the subject before the timing analyzed by the timing analysis unit is restored by the program configuration information. A search unit for searching using the plurality of intermediate generation programs,
The programming capability diagnosis apparatus according to claim 2, wherein the control unit displays a search result of the search unit on the display unit. A code duplication rate analysis unit for analyzing a code duplication rate indicating the degree of code duplication within the one processing module or between the plurality of processing modules;
The said control part displays the analysis result of the said code duplication rate analysis part on the said display part, The programming capability diagnostic apparatus in any one of Claim 1 to 3 characterized by the above-mentioned. The display unit includes a reception unit that receives a user operation,
The control unit selectively selects a diagram generated by the visualization diagram generation unit, an analysis result of the timing analysis unit, and an analysis result of the code duplication rate analysis unit based on an operation signal from the reception unit. The programming ability diagnosis apparatus according to claim 4, wherein the display is displayed on the display unit. In a program creation device in which a subject who has given a programming task creates a program that solves the programming task, the program is created by the subject, and a desired program output set by the questioner who has presented the programming task is obtained. And a test passing program that is composed of one processing module or a plurality of processing modules and that has obtained the desired program output and passed the test, and the subject passes the test. A program capability information diagnosis method for diagnosing the programming capability of the subject by reading program configuration information capable of restoring a plurality of intermediate generation programs generated in the process of creating a program,
A storage step for storing the test passing program and the program configuration information;
A reproduction step of reproducing the creation process of the test passing program based on the plurality of intermediate generation programs restored by the program configuration information;
Visualization diagram generation step of generating a diagram in which the processing procedure of each of the one processing module or the plurality of processing modules or the structure between the one processing module or the plurality of processing modules is visualized based on the test passing program When,
A method for diagnosing a programming ability, comprising: a process of creating the test passing program reproduced by the reproducing step, and a step of displaying a diagram generated by the visualized diagram generating step on a display unit. In a program creation device in which a subject who has given a programming task creates a program that solves the programming task, the program is created by the subject, and a desired program output set by the questioner who has presented the programming task is obtained. And a test passing program that is composed of one processing module or a plurality of processing modules and that has obtained the desired program output and passed the test, and the subject passes the test. A computer program for causing a computer to function as a programming ability diagnosis apparatus that reads program configuration information capable of restoring a plurality of intermediate generation programs generated in the process of creating a program and diagnoses the programming ability of the subject. A-time,
A storage unit for storing the test passing program and the program configuration information;
Based on the plurality of intermediate generation programs restored by the program configuration information, a playback unit that plays back the creation process of the test passing program;
Based on the test passing program, a visualization diagram generation unit that generates a diagram in which a processing procedure of each of the one processing module or the plurality of processing modules or a structure between the one processing module or the plurality of processing modules is visualized When,
In order to cause a computer to function as a programming ability diagnostic apparatus including a process for creating the test passing program reproduced by the reproduction unit and a control unit for displaying a diagram generated by the visualization diagram generation unit on a display unit Computer program.

Images