JP5772358B2 - Review evaluation system and method - Google Patents

Description

  The present invention relates to a review evaluation system and method for evaluating the quality of reviews from review minutes in a project for developing software and the like.

  In a software development project, the later the problem is discovered, the more cost and time it takes to deal with it. For this reason, it is possible to use upstream methods such as simulation, prototyping, test-first (write test cases before writing code), etc. Emphasis is placed on finding problems early in the process.

  The most common and useful means of finding problems early is review. The review assumes that there are always problems and defects in the specifications and programs that are the deliverables of the project. Based on the past project results, the number of problems found in the review for each development process is used as the target value. Set. Judgment is made as to whether the actual review results have reached the set target value, and additional reviews are carried out if they are insufficient.

  In addition, after the review, a number of management materials such as minutes and follow-up item management tables will be created, and the details of decisions, details to be followed, etc. will be recorded and managed as items to be pointed out. As mentioned above, reviews are an important part of a successful project.

  However, the review is often organized by the person in charge who is the project reviewer (the person who receives the review), and the person in charge may determine whether or not the review has been performed and whether or not the contents of the review are acceptable.

  In addition, setting the review target and confirming the contents of the review are primarily the responsibility of the project leader, but it is difficult to judge whether the review was appropriate because it is not possible to participate in all reviews.

  In addition, when evaluating the quality of reviews, the judgment of whether the number of cases pointed out, the level and number of reviewers (reviewers), and the time spent on reviews is appropriate is often made sensuously and empirically. .

For such a problem, for example, a technique disclosed in Patent Document 1 below is known.
That is, Patent Document 1 proposes a system that statistically analyzes review quality according to the characteristics of each project and evaluates review quality based on statistical information on the review. The statistical information evaluates review quality by statistical calculation of review speed: (review target scale) / (total review time) and defect density: (number of defects) / (review target scale).

JP 2006-178589 A

However, the invention of Patent Document 1 has problems with respect to the following points.
Problem 1: A lot of successful similar project data is required.
In the above-described conventional technology, since quality evaluation is based on statistical information, it cannot be used as a useful standard statistical index unless certain successful similar project data is collected.

Issue 2: Quality assessment according to the skill level of reviewers and reviewers has not been made.
In the above-described conventional technology, the oversight of the defect due to the review speed is prevented by the review speed, and further, it is evaluated whether the number of indications is insufficient by the defect density. However, there is no description about the skill level of reviewers and reviewers related to review quality.

  The present invention has been made in view of such points, and an object of the present invention is to enable evaluation of review quality in a project for developing software and the like.

In order to solve the above problems, the present invention provides a review evaluation system in which a client terminal and a management server are connected via a network. The management server includes a storage unit and an execution unit, and the storage unit includes a review. Review minutes file that stores information including the number of review points listed in the minutes, reviewers who participated in the review, reviewee, and time spent on the review, and information including the skill level of the reviewer and reviewer A developer information table to be stored, a quality index table to store in advance information including a reviewer's required skill level and required number of people, a review work allocation rate, and a review indication rate, which are quality index data for the review, Skill coefficient table for storing evaluation coefficients corresponding to skill levels in advance, and target projects A project size estimation table for storing information including the man-hours representing the scale and the number of program steps in advance, and a quality data storage file for storing the quality data of the review, and the execution unit includes a mail receiving unit and a review evaluation process A review minutes storage means for storing the review minutes received from the client terminal in the review minutes file, and the review minutes for transferring the review minutes to the review evaluation processing unit. The review evaluation processing unit obtains the number of man-hours spent for the review from the review minutes, and determines the number of man-hours per process man-hour from the review man-hours obtained from the review minutes and the process man-hours in the project size estimation table. Review work appropriation rate calculation means to calculate the review work appropriation rate (B) request a review pointed out the number from the review proceedings, and reviews pointed out ratio calculating means for calculating a review pointed out rate per program from the program the number of steps of the review pointed out the number obtained from the review proceedings the project scale estimation table,
(C) The number of reviewers and the number of participants who participated in the review are obtained from the review minutes, the skill level of the developer information table is referred to, the skill level of the reviewer who participated in the review is obtained, and the essential skills of the quality indicator table Reviewer satisfaction degree calculation means for obtaining a reviewer satisfaction degree from the level and the required number of persons, quality data storage file storage means for storing the quality data obtained in (a) to (c) in the quality data storage file, Quality index data comparison means for comparing the quality data obtained in (a) to (c) with the quality index data, (1) Man-hour evaluation means for evaluating the man-hours spent on the review, and (2) Number of reviews indicated (3) a skill evaluation means for evaluating the skill level and number of reviewers who participated in the review; The serial evaluation results, characterized in that it comprises an electronic mail transmission means for transmitting to a predetermined destination by e-mail.

  Further, the man-hour evaluation unit or the review indication number evaluation unit of the review evaluation processing unit according to the present invention acquires a skill level of review from the developer information table and corresponds to the skill level from the skill coefficient table. An evaluation coefficient is acquired, a quality index range corresponding to the review target product is acquired from the quality index table, and the review work allocation rate or the review indication rate is included in the quality index range obtained by correcting the quality index range with the evaluation coefficient. If there is, it is characterized by evaluating it as a reasonable quality.

Further, the skill evaluation means of the review evaluation processing unit according to the present invention evaluates that the reviewer satisfaction degree is 1 or more as an appropriate quality.
Further, the present invention provides a review evaluation method in which a client terminal and a management server are connected via a network, wherein the management server includes the number of review indications described in the review minutes, the reviewers participating in the review, the review Review minutes, including time spent on reviews, store developer information including reviewer and reviewer skill levels in advance, and provide reviewer's required skill levels and required number of people as quality index data for reviews , Quality index data including review work appropriation rate and review indication rate is stored in advance, skill coefficient information including evaluation coefficient corresponding to skill level is stored in advance, and the project includes the number of steps and program steps indicating the scale of the target project Size estimation information information is stored in advance, and quality data related to review quality is stored. Storing the review minutes received from the client terminal, and (a) obtaining the number of man-hours spent on the review from the review minutes, the review man-hours obtained from the review minutes and the project size estimation A review work appropriation rate calculation step for obtaining a review work appropriation rate per process man-hour from information process man-hours, and (b) obtaining the number of review indications from the review minutes, and the number of review indications obtained from the review minutes and the project A step of obtaining a review indication rate per program from the number of program steps of the size estimation information, and (c) obtaining reviewers and the number of participants who participated in the review from the review minutes, referring to the skill level of the developer information, and The skill level of reviewers who participated in A step of obtaining a reviewer satisfaction degree from the essential skill level and the required number of people of the quality index data, a step of storing the quality data obtained in the above (a) to (c), and a determination in the above (a) to (c) Comparing the quality data with the quality index data, (1) evaluating the man-hours spent on the review, (2) evaluating the number of reviews, (3) reviewers who participated in the review The skill level and the number of persons are evaluated, and the evaluation results (1) to (3) are transmitted to a predetermined transmission destination by e-mail.

  According to the present invention, since the review quality is evaluated based on the created minutes, the evaluation work can be reduced. In addition, since the evaluation result is notified to the review-related person, it is possible to improve the awareness of the review. As a result, problems occurring in the post-process are reduced, reworking work is reduced, and development efficiency and quality can be improved.

It is a figure which shows the structure outline | summary of the review evaluation system which concerns on embodiment of this invention. It is a figure which shows the structure of the developer information table which concerns on embodiment of this invention. It is a figure which shows the structure of the quality parameter | index 1 setting table which concerns on embodiment of this invention. It is a figure which shows the structure of the quality parameter | index 2 setting table which concerns on embodiment of this invention. It is a figure which shows the structure of the skill coefficient table which concerns on embodiment of this invention. It is a figure which shows the structure of the project scale estimation table which concerns on embodiment of this invention. It is a figure which shows the example of the review minutes concerning embodiment of this invention. It is a figure explaining the description item of the review minutes concerning embodiment of this invention. It is a figure which shows the structure of the quality data storage file which concerns on embodiment of this invention. It is a figure which shows Example 1 of the evaluation result transmission mail from the management server which concerns on embodiment of this invention. It is a figure which shows Example 2 of the evaluation result transmission mail from the management server which concerns on embodiment of this invention. It is a figure which shows Example 3 of the evaluation result transmission mail from the management server which concerns on embodiment of this invention. It is a flowchart which shows an example of the server processing procedure which manages the review minutes based on embodiment of this invention.

Hereinafter, embodiments of the present invention will be described in detail.
<Outline of review evaluation system>
FIG. 1 is a diagram showing a schematic configuration of a review evaluation system according to an embodiment of the present invention. In FIG. 1, a review evaluation system according to an embodiment of the present invention is constructed by a management server 10 that evaluates a review and a terminal 30 that is a client terminal that transmits and receives an e-mail. The terminal 30 is connected to be communicable. The network 40 can be an existing public network, LAN, WAN, or the like regardless of wired or wireless.
<Functional configuration of the management server>
In FIG. 1, the management server 10 includes an execution unit 11 and a storage unit 14, and the execution unit 110 receives an e-mail attached with review minutes from the terminal 30, and a mail reception unit 12. A review evaluation processing unit 13 for evaluating the quality of the review from the contents of the review minutes transferred from the information and the information in the various tables 16 to 20 stored in the storage unit, and transmitting the evaluation result to a predetermined destination by e-mail; The storage unit 14 includes a review minutes file 15 for storing review minutes regarding the review to be evaluated, a developer information table 16 for storing information on members related to the development project in advance, and each development process. A quality index 1 setting table 17 for storing indexes for evaluating reviews for standard workers in advance, and a review for each development process A quality index 2 setting table 17 that stores an index for evaluating a view in advance, a skill coefficient table that stores a correction coefficient corresponding to the skill level of the reviewer for the quality index 1 setting table 17 in advance, and a review target A project size estimation table 20 that stores the estimated man-hours and the number of program steps for the deliverables in advance, and a quality data storage file 21 that obtains review quality data by the review evaluation processing unit 13 and stores the obtained results. ing.

  The mail receiving unit 12 receives an e-mail attached with the review minutes from the terminal 30, stores the received review minutes in the review minutes file 15, and transfers the review minutes to the review evaluation processing unit 13.

  The review evaluation processing unit 13 obtains review quality data from the contents of the review minutes transferred from the mail receiving unit 12 and the information in the various tables 16 to 20 stored in the storage unit, and evaluates this based on the quality index. To do. As the evaluation, there are (1) the man-hour evaluation spent for the review, (2) the evaluation of the number of review points, and (3) the skill level evaluation of the reviewer who participated in the review.

(1) Evaluation of man-hours spent for review Evaluate review quality using the review work allocation rate. The number of review steps (hours) is obtained from the review time and the number of review participants described in the review minutes. Next, the man-hour (time) of the project scale estimation table 20 corresponding to the review target process described in the review minutes is extracted, and the review work allocation rate is obtained by the following formula.

Review work allocation rate = review man-hour / total process man-hour Further, the threshold value of the quality index 1 setting table 17 corresponding to the review target process is extracted, and the developer information table corresponding to the review and the review target process described in the review minutes 16 review skill levels are extracted, then the coefficient of the skill coefficient table corresponding to the extracted skill level is extracted, and the corrected threshold value (quality index data) obtained by multiplying the extracted coefficient by the threshold value is obtained. Compare the work appropriation rate (quality data) and judge that it is appropriate if it is within the range of the quality index. If the review target is not a process unit but a product unit, the process is replaced with a product and the same processing is performed.

(2) Evaluation of the number of review indications Review quality is evaluated based on the review indication rate. Calculate the total number of indications from the items of decision, indications and follow-up items described in the review minutes. Next, the number of steps (K Step) in the project scale estimation table 20 is extracted, and the review indication rate is obtained by the following formula.

Review indication rate = total number of indications / number of program steps Further, the threshold value of the quality index 1 setting table 17 corresponding to the review target process is extracted, and the developer information table corresponding to the review and the review target process described in the review minutes 16 review skill levels are extracted, then the coefficient of the skill coefficient table corresponding to the extracted skill level is extracted, and the corrected threshold value (quality index data) obtained by multiplying the extracted coefficient by the threshold value is obtained. The indication rate (quality data) is compared, and if it is within the range of the quality index, it is judged to be appropriate.

  If the review target is not a process unit but a product unit, the process is replaced with a product and the same processing is performed. When the product to be reviewed is a functional unit, the number of program steps that becomes the denominator of the review indication rate is the number of program steps that realize the function. In the case of a review of the development process, the number of program steps is the number of program steps of the entire project.

(3) Evaluation of skill level and number of reviewers participating in the review The review quality of the reviewers participating in the review is evaluated based on the reviewer satisfaction level. The reviewer skill level of the developer information table 16 corresponding to the reviewer described in the review minutes and the process to be reviewed is extracted. Next, the required skill level and the number of participants of the required skill level in the quality index 2 setting table 18 corresponding to the review target process are extracted, and the degree of reviewer satisfaction is obtained by the following formula.

Reviewer satisfaction level = (required skill level x number of required skill level holders)
/ (Required skill level x number of participants)
That is, the skill level × number of reviewers who participated in the review becomes the numerator of the formula, and the number of participants of the required skill level × essential skill level in the quality index 2 setting table 18 becomes the denominator. The obtained reviewer satisfaction degree (quality data) and “1” (quality index data) are compared, and if the reviewer satisfaction degree is “1” or more, it is determined to be appropriate.

After the above-described evaluation processing, the review evaluation processing unit 13 refers to the developer information table 16 for the email corresponding to the evaluation result, and the email addresses of reviewers, reviewers, project leaders, etc. described in the review minutes Send to.
<Hardware configuration of management server>
Although not shown, hardware resources for realizing the function as the management server 10 include, for example, a CPU, a storage device, an input / output device, various interfaces, and the like as well-known configurations. Naturally, a program for realizing the above functions is stored in the storage device. The storage unit 120 is realized by the above-described storage device, and each component in the execution unit 11 is realized by the above-described hardware resources such as the CPU, the storage device, and various interfaces, and various programs that execute a processing flow described later. The
<Client terminal functions and hardware configuration>
On the other hand, the terminal 30 as a client terminal is not shown, but a setting screen for inputting information (initial information) to the predetermined table, a communication function for transmitting and receiving, and a storage unit of the management server 10 14 is provided with a function of displaying the contents of various files and tables in 14. Although not shown, the hardware resources for realizing the functions as the terminal 30 include, for example, a CPU, a storage device, an input / output device, various interfaces, and the like as well-known configurations. A program for realizing the functions as described above is stored in the storage device of the terminal 30.

Further, the terminal 30 shares information on tables and files in the storage unit 120 of the management server 10, and information (initial information and operation information) input from the setting screen is stored in the tables and files in the management server 10. It is reflected in the information.
<Table and file structure of storage unit>
FIG. 2 is a diagram showing the configuration of the developer information table 16. FIG. 2 is created for each development project, and member skills related to the development project and transmission destination information of review evaluation results are registered in advance before system operation. Next, each item will be described.
1. Name: Register the names of the members related to the development project. The names of reviewers and reviewers, including all reviews related to development projects, are also registered.
2. Registrant type: The registration type corresponding to the name is registered. Describe the role in the development project. For example, if the registration type is PM (project manager), it is necessary to check all minutes.
3. E-mail address: Register an e-mail address corresponding to your name.
4). Phone number: Register the phone number corresponding to your name.
5. Skill level: Set for each product to be developed, system design, detailed design, program creation, and test. System design corresponds to process of requirements analysis, user interface design, system structure design, detailed design corresponds to process of program structure design, program creation corresponds to process of programming, test is program test, Compatible with integration test and system test processes. The definition of the skill level (numerical value) will be described later in the description of the skill coefficient table in FIG.

  FIG. 3 is a diagram showing the configuration of the quality index 1 setting table 17. Fig. 3 shows the upper limit of the quality index range of review work appropriation rate (%) and review indication rate (cases / K Step) among the indexes when review reviews are evaluated for each development process using reviewers as standard workers. Standard values and lower limit values are registered in advance before system operation. Of these, registration of the upper limit is not essential.

FIG. 4 is a diagram showing the configuration of the quality index 2 setting table 18. FIG. 4 is a table for evaluating the satisfaction level of reviewers. It is a table for registering in advance the required skill level and the number of participants of the required skill level for each development process before operating the system. Next, each item will be described.
1. Required skill level Register the required level of reviewers participating in the review.
2. Number of participants with required skill level Register the minimum number of reviewers who must have the required skill level or higher.

  FIG. 5 is a diagram showing the configuration of the skill coefficient table 19. The product created by the reviewer is thought to be low in quality when the skill level of the reviewer is low. Naturally, the quality of the review is low, the number of reviews indicated is higher than that of the standard worker (level standard).

Therefore, by multiplying the standard worker-based quality index range registered in the quality index 1 setting table 17 by a coefficient, an appropriate threshold value corresponding to the review skill is obtained.
In the skill coefficient table 19, names, coefficients, and outlines are registered in advance for each skill level before system operation.

  FIG. 6 is a diagram showing the configuration of the project scale estimation table 20. FIG. 6 estimates the number of pages for a product for each development process, the number of pages for programming, and the number of steps (K Step) for programming as the size of the product, and calculates the man-hour (time) from the productivity. . Finally, the total man-hours and total steps of the project are totaled.

FIG. 7 is a diagram showing an example of the review minutes registered in the review minutes file.
FIG. 8 is a diagram for explaining items described in the review minutes of FIG. Review minutes include the review agenda, development process, deliverables to be reviewed, reviewers as participants, name of reviewers, number of participants, time required for review or start time to end time, points to be pointed out, decisions to be made, Follow-up items and agenda details are described.

  FIG. 9 is a diagram showing the configuration of the quality data storage file 21. FIG. 9 shows a quality data storage file for collecting data related to review quality from the review minutes by the review evaluation processing unit 13 and storing quality evaluation data obtained by evaluating the reviews from the information in the various tables 16 to 20. 21. Next, each item will be described.

1. Total number of indications The number of indications, decisions, and follow-ups is aggregated and stored. If the content of the follow-up item in the review minutes is the same as the content of the agenda, it is counted as an item to be pointed out, and is excluded from being counted as a follow-up item.

2. Review work allocation rate The review work allocation rate is calculated using the following formula.
Review work allocation rate = review man-hour / total process man-hour The review man-hour is obtained from the review time described in the review minutes and the number of review participants. The total process man-hour is the man-hour (estimation) of the development process corresponding to the review described in the project estimation table.

3. Review indication rate The review indication rate is calculated by the following formula.
Review indication rate = total number of indications / number of program steps The number of program steps is the total number of steps described in the project estimation table 20. When the review target is a program in units of functions, the number of program steps for each function.
4). Reviewer satisfaction The reviewer satisfaction is obtained by the following formula.

Reviewer satisfaction level = (required skill level x number of required skill level holders)
/ (Required skill level x number of participants)
The required skill level x number of required skill level holders is extracted from the reviewer skill level of the developer information table corresponding to the reviewer described in the review minutes, and the required quality index 2 setting table corresponding to the review development process Calculate the required skill level x number of required skill level holders for participants with skills above skill level.

The required skill level × number of participants is the number of participants of the required skill level × required skill level in the quality index 2 setting table corresponding to the review development process.
<Example of e-mail according to the evaluation result>
FIG. 10A is a diagram illustrating an example 1 of the evaluation result transmission mail from the management server. FIG. 10A is an example when the review is properly performed.

FIG. 10B is a diagram illustrating an example 2 of the evaluation result transmission mail from the management server. FIG. 10B is an example when it is determined that the number of review indications is below the target number.
FIG. 10C is a diagram illustrating an example 3 of the evaluation result transmission mail from the management server. FIG. 10C is an example of a case where it is determined that the skill level evaluation of reviewers who participated in the review is insufficient for the target skill and the evaluation of the number of reviews indicated is below the target number. .
<Server processing procedure>
FIG. 11 is a flowchart illustrating an example of a server processing procedure for managing review minutes. FIG. 11 is a flowchart of a mail reception process that is a program process of the mail reception unit 12 and a review evaluation process that is a program process of the review evaluation processing unit 13.

  The mail reception process waits for reception of an e-mail from the terminal 30 (S11). When the e-mail is received, it is determined whether the received e-mail is a review minutes (S12), and if it is a review minutes (S12 YES). The review minutes are stored in the review minutes file, the review minutes are transferred to the review evaluation process (S13), and the process returns to S11 and waits for receiving an e-mail (S11). If it is not a review minutes (NO in S12), the process returns to S11 and waits for receiving an e-mail (S11).

  The review evaluation process waits for the transfer of the review minutes from the mail reception process (S21). When the review minutes are received (S22), the number of items to be pointed out, the number of items to be determined, and the follow-up from the information in the review minutes and various tables The number of items and the total number of pointed-out indications are obtained, and the results of calculating the review work allocation rate, the review indication rate, and the reviewer satisfaction level are stored in the quality data storage file 21 (S23).

  Next, in order to appropriately evaluate the review work allocation rate and the review indication rate, the threshold value of the quality index 1 setting table 17 corresponding to the review target process is extracted, and the review process described in the review minutes and the review target process are supported. The review skill level of the developer information table 16 is extracted, then the coefficient of the skill coefficient table corresponding to the extracted skill level is extracted, and the corrected view work allocation ratio and review by multiplying the extracted coefficient by a threshold value A threshold for the indication rate is obtained (S24).

Next, the obtained review work appropriation rate, review indication rate, reviewer satisfaction degree and respective thresholds are compared to evaluate the review quality (S25). E-mail texts are created according to the evaluation results (S26), the developer information table 16 is referred to, and the names (reviews, reviewers), project managers, and project leaders listed in the review minutes are evaluated. Send the result email.
<Supplementary explanation of the man-hour evaluation method spent for review ... Example using numerical values>
The man-hour evaluation spent for the review will be described by taking a system structure design review as an example based on the values in the tables and review minutes of FIGS. 2, 3, 5, 6, and 7.

  When evaluating the man-hours spent at the review work allocation rate, the man-hours for review (time) are obtained from the review time and the number of review participants described in the review minutes. Next, the man-hour (time) of the project scale estimation table 20 corresponding to the review target process described in the review minutes is extracted, and the review work allocation rate is obtained by the following formula.

Review work allocation rate = review man-hour / total process man-hour "40" for system structure design in the project size estimation table 20 in FIG. 6, review time "2.0" in the review minutes in FIG. 7, and "4" in the number of participants To obtain the review work allocation rate “20” (%).

Further, the threshold “lower limit 2” of the quality index 1 setting table 17 in FIG. 3 corresponding to the review target process is extracted, and the developer in FIG. 2 corresponding to the review and the review target process described in the review minutes in FIG. The review skill level “1” of the information table 16 is extracted, then the coefficient “1.6” of the skill coefficient table of FIG. 5 corresponding to the extracted skill level is extracted, and the extracted coefficient “1.6” is extracted. Is compared with the corrected threshold “lower limit 3.2” and the obtained review work allocation rate “20”, and if it is within the range of the quality index, it is determined to be appropriate. In this example, the review indication rate obtained is within the range of the quality index, so it is judged appropriate.
<Supplementary explanation of the evaluation method for the number of reviews indicated ... Example using numerical values>
The evaluation of the number of review points will be described by taking a system structure design review as an example based on the values in the table of FIG. 2, FIG. 3, FIG. 5, FIG.

  When evaluating the number of reviews pointed out by the review point rate, the total number of points to be pointed out is determined from the items of decision items, pointed items and follow-up items described in the review minutes. Next, the number of steps (K Step) in the project scale estimation table 20 is extracted, and the review indication rate is obtained by the following formula.

Review indication rate = total number of indications / number of program steps Total number of steps “50” in the project size estimation table 20 in FIG. 6 and items of decision items, indication items and follow-up items in the review minutes in FIG. 7 (not shown) The review indication rate “0.9” is obtained from the total number of indications “45” (not shown).

Further, the threshold values “lower limit 2 and upper limit 6” of the quality index 1 setting table 17 of FIG. 3 corresponding to the review target process “system structure design” are extracted, and the review “Kenzo Sato” described in the review minutes of FIG. 2 is extracted from the developer information table 16 of FIG. 2 corresponding to the process to be reviewed “system structure design (system design)”, and then the figure corresponding to the extracted skill level “1”. 5. The coefficient “1.6” in the skill coefficient table 19 of 5 is extracted, the corrected coefficient “1.6” is multiplied by the threshold “lower limit 2, upper limit 6”, and the corrected threshold “lower limit 3.2, upper limit 9. 6 ”and the obtained review indication rate“ 0.9 ”are compared, and if it is“ lower limit 3.2, upper limit 9.6 ”within the range of the quality index, it is determined to be appropriate. In this example, the review indication rate obtained is out of the range of the quality index, so it is judged inappropriate.
<Additional explanation of evaluation of skill level and number of reviewers who participated in review ... Example using numerical values>
The evaluation of the skill level and the number of reviewers who participated in the review will be described by taking a system structure design review as an example based on the values in the tables and review minutes shown in FIGS.

  When evaluating the review quality of reviewers who participated in the review based on the reviewer satisfaction level, the reviewer “Ito, Tanaka, Suzuki” described in the review minutes in FIG. 7 and the developer information of FIG. 2 corresponding to the review target process The reviewer skill levels “Ito: 3, Tanaka: 3, Suzuki: 3” in the table 16 are extracted. Next, the essential skill level “3” and the number of participants “2” of the essential skill level in the quality index 2 setting table 18 of FIG. 4 corresponding to the review target process are extracted, and the degree of reviewer satisfaction is obtained by the following formula.

Reviewer satisfaction level = (required skill level x number of required skill level holders)
/ (Required skill level x number of participants)
That is, the skill level of the reviewer who participated in the review (more than the required level) “3” × number of people “3” becomes the numerator of the formula, and the required skill level “3” × required skill level in the quality index 2 setting table 18 of FIG. The number of participants “2” becomes the denominator. The obtained reviewer satisfaction degree “1.5” is compared with “1”, and if the reviewer satisfaction degree is “1” or more, it is determined to be appropriate. In this example, since the obtained reviewer satisfaction degree is the quality index “1” or more, it is determined to be appropriate.
<Effect of Example>
According to the present embodiment described above, the quality of the review can be evaluated from the created review minutes and information of various tables. Review quality evaluation is conducted for (1) evaluation of man-hours spent on reviews, (2) evaluation of the number of review points, and (3) skill level evaluation of reviewers who participated in the review. Can be evaluated by quantity. Since evaluation indicators are associated according to the skill level of reviewers and reviewers, statistical information is created from a large amount of past successful project information, and it is appropriate without providing evaluation indicators with reference to this statistical information. Can be evaluated.

  In addition, since the evaluation results are notified to the recipients of the review by e-mail, the review quality is correctly evaluated without the project managers and managers participating in the review, and the development stakeholders are aware of the reviews. Improvements can be made. As a result, problems occurring in the post-process are reduced, reworking work is reduced, and development efficiency and quality can be improved.

DESCRIPTION OF SYMBOLS 10 Management server 11 Execution part 12 Mail receiving part 13 Review evaluation process part 14 Storage part 15 Review minutes file 16 Developer information table 17 Quality index 1 setting table 18 Quality index 2 setting table 19 Skill coefficient table 20 Project scale estimation table 21 Quality data storage file 30 Terminal 40 Network

Claims (6)

In a review evaluation system in which a client terminal and a management server are connected via a network,
The management server includes a storage unit and an execution unit,
The storage unit
Review minutes file that stores information including the number of review points listed in the review minutes, reviewers who participated in the review, review reviews, and time spent reviewing,
Developer information table that stores information including skill levels of reviewers and reviewers in advance,
Quality index table that stores in advance information including reviewer's required skill level and required number of people, review work allocation rate, review indication rate, which is quality index data of review,
A skill coefficient table for storing evaluation coefficients corresponding to skill levels in advance;
A project size estimation table for storing information including the man-hour representing the scale of the target project and the number of program steps;
A quality data storage file for storing quality data of reviews,
The execution unit is
It has an email receiver and a review evaluation processor,
The mail receiver
Review minutes storage means for storing the review minutes received from the client terminal in the review minutes file;
A review minutes transfer means for transferring the review minutes to the review evaluation processing unit;
The review evaluation processor
(A) A review work appropriation rate calculation for obtaining a review work appropriation rate per process man-hour from the review man-hours obtained from the review minutes and the process man-hours in the project size estimation table. Means,
(B) A review indication rate calculating means for obtaining the number of review indications from the review minutes, obtaining a review indication rate per program from the number of review indications obtained from the review minutes and the number of program steps of the project size estimation table;
(C) The number of reviewers and the number of participants who participated in the review are obtained from the review minutes, the skill level of the developer information table is referred to, the skill level of the reviewer who participated in the review is obtained, and the essential skills of the quality indicator table Reviewer satisfaction degree calculation means for obtaining the reviewer satisfaction degree from the level and the required number of people,
Quality data storage file storage means for storing the quality data obtained in (a) to (c) in the quality data storage file;
Quality index data comparison means for comparing the quality data obtained in (a) to (c) and the quality index data;
(1) a man-hour evaluation means for evaluating the man-hours spent on the review;
(2) A review indication number evaluation means for evaluating the number of review indications;
(3) a skill evaluation means for evaluating the skill level and number of reviewers participating in the review;
A review evaluation system comprising: an e-mail transmission unit that transmits the evaluation result to a predetermined transmission destination by e-mail. The review evaluation system according to claim 1,
The man-hour evaluation unit or the review indication number evaluation unit acquires a skill level of a reviewer from the developer information table, acquires an evaluation coefficient corresponding to the skill level from the skill coefficient table, and reviews from a quality index table Obtain a quality index range corresponding to the target deliverable, and evaluate the quality index range as appropriate if the review index appropriation rate or the review indication rate is in the quality index range corrected by the evaluation coefficient. A featured review evaluation system. The review evaluation system according to claim 1,
The said skill evaluation means evaluates the said reviewer satisfaction degree 1 or more as appropriate quality, The review evaluation system characterized by the above-mentioned. In a review evaluation method in which a client terminal and a management server are connected via a network,
The management server
Memorize the review minutes, including the number of reviews indicated in the review minutes, the reviewers who participated in the review, the reviews, and the time spent reviewing,
Developer information including skill levels of reviewers and reviewers is stored in advance,
Quality index data including the reviewer's essential skill level and number of reviewers, review work appropriation rate, review indication rate, which is the quality index data of the review, is stored in advance.
Pre-store skill coefficient information including evaluation coefficients corresponding to skill levels,
Pre-store project size estimation information information including man-hours representing the scale of the target project and the number of program steps,
Store quality data related to review quality,
Storing the review minutes received from the client terminal;
(A) A review work appropriation rate calculation for obtaining a review work appropriation rate per process man-hour from the review man-hours obtained from the review minutes and the process man-hours of the project size estimation information is obtained from the review minutes. Steps,
(B) obtaining the number of review indications from the review minutes, obtaining a review indication rate per program from the number of review indications obtained from the review minutes and the number of program steps of the project size estimation information;
(C) The number of reviewers and the number of participants who participated in the review are obtained from the review minutes, the skill level of the developer information is referred to, and the skill level of the reviewer who participated in the review is obtained.
Obtaining a reviewer satisfaction level from the required skill level and the required number of people of the quality indicator data;
Storing the quality data obtained in (a) to (c);
Comparing the quality data obtained in (a) to (c) with the quality index data;
(1) a step of evaluating the man-hours spent on the review;
(2) a step of evaluating the number of reviews indicated;
(3) a step of evaluating the skill level and number of reviewers who participated in the review;
Transmitting the evaluation results of (1) to (3) to a predetermined transmission destination by e-mail;
A review evaluation method characterized by comprising: The review evaluation method according to claim 4,
The step (1) of evaluating the man-hours spent on the review or the step of (2) evaluating the number of reviews pointed out obtains the skill level of the reviewer from the developer information, and acquires the skill level from the skill coefficient information. To obtain a quality index range corresponding to the product to be reviewed from the quality index information, and to apply the review work allocation rate to the quality index range obtained by correcting the quality index range by the evaluation coefficient, or the review A review evaluation method characterized in that if there is an indication rate, it is evaluated as a reasonable quality. The review evaluation method according to claim 4,
The step (3) of evaluating the skill level and the number of reviewers who participated in the review evaluates that the reviewer satisfaction level is 1 or more as an appropriate quality.