WO2021079496A1 - 評価装置、評価方法及びプログラム - Google Patents

評価装置、評価方法及びプログラム Download PDF

Info

Publication number
WO2021079496A1
WO2021079496A1 PCT/JP2019/041929 JP2019041929W WO2021079496A1 WO 2021079496 A1 WO2021079496 A1 WO 2021079496A1 JP 2019041929 W JP2019041929 W JP 2019041929W WO 2021079496 A1 WO2021079496 A1 WO 2021079496A1
Authority
WO
WIPO (PCT)
Prior art keywords
risk
library
evaluation
value
source code
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2019/041929
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
淳 西岡
純明 榮
和彦 磯山
佑嗣 小林
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
NEC Corp
Original Assignee
NEC Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by NEC Corp filed Critical NEC Corp
Priority to JP2021553261A priority Critical patent/JP7322963B2/ja
Priority to PCT/JP2019/041929 priority patent/WO2021079496A1/ja
Priority to US17/767,138 priority patent/US12254097B2/en
Publication of WO2021079496A1 publication Critical patent/WO2021079496A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/50—Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems
    • G06F21/57—Certifying or maintaining trusted computer platforms, e.g. secure boots or power-downs, version controls, system software checks, secure updates or assessing vulnerabilities
    • G06F21/577—Assessing vulnerabilities and evaluating computer system security
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00—Arrangements for software engineering
    • G06F8/70—Software maintenance or management
    • G06F8/77—Software metrics
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2221/00—Indexing scheme relating to security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F2221/03—Indexing scheme relating to G06F21/50, monitoring users, programs or devices to maintain the integrity of platforms
    • G06F2221/033—Test or assess software

Definitions

  • the present invention relates to an evaluation device, an evaluation method, and a program.
  • Patent Document 1 describes that it solves the difficulty of grasping the effect of change in the development process in which a dependency is generated at an unpredictable arbitrary timing in parallel development.
  • Patent Document 2 describes that it provides a risk determination program that can predict the occurrence location of a structural failure of software.
  • Patent Document 3 describes that it provides a system and a method capable of utilizing a large number of software files, which automates important aspects in the software development, maintenance, and repair life cycle.
  • Patent Document 4 describes that the load test is executed at an appropriate frequency so as not to reduce the software development efficiency and the detection accuracy of software aging.
  • Japanese Unexamined Patent Publication No. 2008-040760 Japanese Unexamined Patent Publication No. 2010-257091 Japanese Patent Application Laid-Open No. 2017-520842 International Publication No. 2015/022890
  • the library released as open source software may be used for the development of application programs.
  • the library itself has a multi-stage dependency relationship such as depending on another library (calling another library), and the complexity of the library is increasing. Therefore, it is not uncommon for the library itself to have vulnerabilities such as bugs and security holes.
  • the library is carefully verified at the time of its introduction, but after the introduction, it is often limited to a simple confirmation such as an operation check. That is, the emphasis is on the development of the application program itself, and the verification of the library after introduction is often not emphasized.
  • the source code of the side that uses the library (calling side) will be revised as the project progresses, and the usage mode of the library will also change. For example, when a library is introduced, one function only calls the library, but as the development progresses, a plurality of functions may call the library.
  • a main object of the present invention is to provide an evaluation device, an evaluation method, and a program that contribute to appropriately evaluating the risk of source code that changes over time.
  • the first library is based on the generation unit that generates the evaluation value regarding the risk of the first library described in the source code and at least the generated evaluation value.
  • the output unit which calculates the risk level of the above, calculates the risk value indicating the risk inherent in the source code based on the calculated risk level, and outputs the time-series data of the calculated risk value.
  • the first step based on the step of generating the evaluation value regarding the risk of the first library described in the source code and at least the generated evaluation value.
  • the risk level of the library is calculated, the risk value indicating the risk inherent in the source code is calculated based on the calculated risk level, and the step of outputting the time-series data of the calculated risk value is performed. Evaluation methods including are provided.
  • the computer mounted on the evaluation device is subjected to the process of generating the evaluation value regarding the risk of the first library described in the source code, and at least the generated evaluation value. Based on this, the risk level of the first library is calculated, the risk value indicating the risk inherent in the source code is calculated based on the calculated risk level, and the time-series data of the calculated risk value is output.
  • the processing to be performed and the program to execute the processing are provided.
  • an evaluation device an evaluation method, and a program that contribute to appropriately evaluating the risk of source code that changes over time are provided.
  • other effects may be produced in place of or in combination with the effect.
  • the evaluation device 100 includes a generation unit 101 and an output unit 102 (see FIG. 1).
  • the generation unit 101 generates an evaluation value regarding the risk of the first library described in the source code.
  • the output unit 102 calculates the risk level of the first library based on at least the generated evaluation value, and calculates and calculates the risk value indicating the risk inherent in the source code based on the calculated risk level. Outputs time-series data of risk values.
  • the evaluation device 100 evaluates each library described in the source code and quantifies the risk of the library as a risk level.
  • the evaluation device 100 visualizes the risk of the source code from the viewpoint of the library and displays the time-series change thereof. The user who comes into contact with the display can appropriately evaluate the risk of the source code that changes over time.
  • FIG. 2 is a diagram for explaining the evaluation device 10 according to the first embodiment.
  • the evaluation device 10 is a device that evaluates risks inherent in the source code of an application program or the like.
  • the evaluation device 10 acquires the source code to be evaluated for risk.
  • the evaluation device 10 quantifies the risk of the source code and presents the quantified risk to the user (application program administrator, etc.). At that time, the evaluation device 10 presents the user with a time-series change regarding the risk of the source code (project generated from the source code).
  • the evaluation device 10 displays the time series data of the above risks on a liquid crystal monitor or the like.
  • the evaluation device 10 may transmit the time-series data to a predetermined address or the like, or may print using a printer or the like.
  • FIG. 3 is a diagram showing an example of a processing configuration (processing module) of the evaluation device 10 according to the first embodiment.
  • the evaluation device 10 includes a source code acquisition unit 201, a library information generation unit 202, a risk information output unit 203, and a storage unit 204.
  • the source code acquisition unit 201 is a means for acquiring the source code to be subject to risk evaluation.
  • the source code acquisition unit 201 may generate GUI (Graphical User Interface) information for an administrator or the like to input (designate) the source code to be evaluated, and acquire the source code using the GUI. Good.
  • the source code acquisition unit 201 may acquire the source code via an external storage device such as a USB (Universal Serial Bus) memory, or may access an external database server or the like to acquire the source code. Good.
  • the source code acquisition unit 201 acquires the source code as shown in FIG. FIG. 4A shows an example of the source code including the main function, and FIGS. 4B and 4C show an example of the source code of the library.
  • the source code acquired by the source code acquisition unit 201 includes a README file accompanying the source code, a file generated by compiling the source code, a directory in which the source code is stored, and the like.
  • the source code acquisition unit 201 stores the acquired source code and its identification information (for example, a project name) in the storage unit 204 in association with each other.
  • the library information generation unit 202 is a means for generating information for evaluating the library described in the acquired source code (hereinafter referred to as library evaluation information).
  • the library information generation unit 202 includes a library extraction unit 211, a risk evaluation value generation unit 212, and a library influence degree calculation unit 213.
  • the library extraction unit 211 is a means for extracting the library described in the acquired source code.
  • the library extraction unit 211 scans the source code and extracts the library described in the source code. Specifically, the library extraction unit 211 confirms an area for declaring the library to be used (for example, the head area of the file in which the import declaration is described), and extracts the library described in the area.
  • the library extraction unit 211 extracts the libraries "L123" and "L345" described in the first and second lines of FIG. 4A.
  • the library extraction unit 211 extracts the libraries "L911” and “L912” described in the first and second lines of FIG. 4B.
  • the library extraction unit 211 extracts the library “L811” described in the first line of FIG. 4C.
  • the library extraction unit 211 detects the dependency between the extracted libraries. Specifically, the library extraction unit 211 detects the child library (dependent library) used by each library (called from each library), and detects the dependency (dependent relationship) between the libraries.
  • the library extraction unit 211 summarizes the dependency relationships between the detected libraries as "library configuration information". For example, in the example of FIG. 4, the library “L123” includes the libraries “L911” and “L912". Therefore, a dependency relationship is recognized between the library “L123” and the libraries “L911” and “L912". Further, the library “L911” includes the library “L811”. Therefore, there is a dependency between these libraries.
  • the library extraction unit 211 generates library configuration information as shown in FIG. 5 based on the above-detected dependency relationships between the libraries.
  • the library includes the child libraries “L911” and “L912”, and further includes the grandchild library “L811”, based on the library "L123".
  • FIG. 5 shows the case of three layers (parent, child, grandchild) as the library hierarchy, but it goes without saying that the hierarchical structure is an example. It goes without saying that the source code to be evaluated may have a structure of three or more layers.
  • the risk evaluation value generation unit 212 is a means for generating an evaluation value related to the risk of the library described in the source code.
  • the risk evaluation value generation unit 212 generates an evaluation value indicating the risk of the extracted library itself. Specifically, the risk evaluation value generation unit 212 quantifies the risk of the library alone as an evaluation value.
  • library single evaluation the risk evaluation of the library alone performed by the risk evaluation value generation unit 212 will be referred to as "library single evaluation”.
  • the risk evaluation value generation unit 212 evaluates the library alone from various viewpoints. Specifically, the risk evaluation value generation unit 212 evaluates each library based on information having different properties, and generates an evaluation value for each of the different information.
  • the risk evaluation value generation unit 212 evaluates the library using a stable index with little change over time regarding the library to be evaluated. For example, the risk evaluation value generation unit 212 calculates the evaluation value based on static information such as the directory structure of the library to be evaluated, the presence / absence of the README file, and the presence / absence of the test directory.
  • the risk evaluation value generation unit 212 gives a small evaluation value if the directory structure of the library to be evaluated is simple (for example, the number of layers is less than a predetermined value), and a large evaluation value if the directory structure is complicated. give.
  • the determination of the evaluation value is based on the finding that the risk (risk) of the corresponding library is relatively high if the directory structure is complicated.
  • the risk evaluation value generation unit 212 may give a small evaluation value if the README file or test directory exists, and may give a large evaluation value if the README file or test directory does not exist. The evaluation value is determined because it can be determined that the management and evaluation (debugging) of the corresponding library are sufficient if the README file and the test directory exist.
  • the risk evaluation value generation unit 212 calculates the evaluation value using all or part of the above items (for example, directory structure, presence / absence of README file, presence / absence of test directory).
  • static evaluation value the evaluation value determined based on static information such as the directory structure
  • the risk evaluation value generation unit 212 may evaluate the library using information that changes with time (for example, metadata about the library to be evaluated). For example, the risk evaluation value generation unit 212 calculates an evaluation value based on metadata such as "problem solving rate (issue resolution rate)", “number of commits", and "code generator (author)" related to the library to be evaluated.
  • the risk evaluation value generation unit 212 may execute threshold processing on the Issue resolution rate and the number of commits, and calculate the evaluation value according to the result. More specifically, the risk evaluation value generation unit 212 gives a small evaluation value if the Issue resolution rate or the like is equal to or more than the threshold value, and gives a large evaluation value if the Issue resolution rate or the like is smaller than the threshold value. Alternatively, the risk evaluation value generation unit 212 may determine the evaluation value according to the numerical range to which the Issue resolution rate or the like belongs. The determination of the above evaluation value is based on the finding that if the Issue resolution rate and the number of commits are larger than the predetermined values, the development of the corresponding library is continued and it is appropriate to underestimate the risk.
  • the risk evaluation value generation unit 212 may determine the evaluation value depending on whether or not the code generator corresponds to a predetermined person. For example, the risk evaluation value generation unit 212 refers to a list of people (white list prepared in advance) who have generated a large number of safe libraries with few vulnerabilities. When the list includes the generator of the library to be evaluated, the risk evaluation value generation unit 212 gives a small evaluation value to the corresponding library.
  • the risk evaluation value generation unit 212 may refer to a list of people (blacklist prepared in advance) who have generated a large number of incomplete libraries due to reports of a large number of vulnerabilities. When the list includes the generator of the library to be evaluated, the risk evaluation value generator 212 may give a large evaluation value to the corresponding library.
  • the risk evaluation value generation unit 212 calculates the evaluation value using all or part of the above items (for example, Issue resolution rate, number of commits, code creator).
  • the evaluation value determined based on dynamic information such as the Issue resolution rate
  • dynamic evaluation value is referred to as "dynamic evaluation value”.
  • the Issue resolution rate and the like are examples of metadata, and a dynamic evaluation value may be calculated using information other than the above, for example, CVE (Common Vulnerabilities and Exposures), commit size, update frequency, and the like.
  • the dynamic evaluation value may be calculated using the distance (deviation degree) between the version of the library used and the latest version (latest commit). For example, the difference between the latest version and the version of the library used may be calculated, and the dynamic evaluation value may be calculated based on the numerical range including the difference.
  • the weights may be changed between the major version and the minor version, and the dynamic evaluation value may be calculated large (calculated with high risk) when the major versions are far apart.
  • the risk evaluation value generation unit 212 may acquire the metadata published on the network and calculate the dynamic evaluation value.
  • the following reference information 1 is exemplified as an example of the above-mentioned published metadata. ⁇ Reference information 1> URL; https://developer.github.com/
  • the risk evaluation value generation unit 212 may evaluate the library using the source code of the library to be evaluated. For example, the risk evaluation value generation unit 212 calculates the evaluation value based on the number of comments described in the source code of the library to be evaluated, the complexity of the code, the name of the function described in the source code, and the like.
  • the risk evaluation value generation unit 212 may execute threshold processing on the number of comments and the complexity of the code, and determine the evaluation value according to the result. More specifically, the risk evaluation value generation unit 212 gives a small evaluation value if the number of comments or the like is equal to or greater than the threshold value, and gives a large evaluation value if the number of comments or the like is smaller than the threshold value.
  • the risk evaluation value generation unit 212 responds to whether or not the name of the function described in the source code of the library to be evaluated conforms to a predetermined rule (or how much it deviates from the predetermined rule). The evaluation value may be determined.
  • the risk evaluation value generation unit 212 calculates the evaluation value using all or part of the above items (for example, the number of comments, the code complexity, the name of the function).
  • the evaluation value determined based on the information described in the source code of the library to be evaluated, such as the code complexity, will be referred to as the "substantial evaluation value”.
  • the risk evaluation value generation unit 212 may calculate the substantive evaluation value based on the information described in the following reference information 2.
  • Reference information 2 describes the number of lines of the source code and the metrics of the cyclomatic complexity of the code. ⁇ Reference information 2> URL; https://www.techmatrix.co.jp/product/understand/function/metrics.html
  • the risk evaluation value generation unit 212 performs a single evaluation of the library to be evaluated from various viewpoints. That is, the risk evaluation value generation unit 212 executes the library unit evaluation from various viewpoints using static information such as the directory structure of the library to be evaluated, dynamic information such as metadata, and actual information based on the source code. ..
  • the above three unit evaluations are examples, and the purpose is not to limit the contents of the library unit evaluation of the risk evaluation value generation unit 212.
  • the risk evaluation value generation unit 212 may perform at least one or more evaluations out of the above three unit evaluations, and may further perform an evaluation different from the above three unit evaluations.
  • the risk evaluation value generation unit 212 executes the library unit evaluation described above for the extracted library. For example, the risk evaluation value generation unit 212 generates information as shown in FIG. 6 and stores it in the storage unit 204.
  • the risk evaluation value generation unit 212 evaluates the library in consideration of the result of the library unit evaluation related to the other library. That is, when the library includes a child library, the risk evaluation value generation unit 212 evaluates the parent library in consideration of the result of the unit evaluation of the child library (generates information for evaluating the parent library). ..
  • the library “L123” is in the relationship of the parent library
  • "L911” and “L912” are in the relationship of the child library
  • the result of the unit evaluation of the child libraries “L911” and “L912” is the parent library "L123". It will be reflected in the evaluation.
  • the risk evaluation value generation unit 212 specifies the evaluation value having the largest value (highest risk) among the evaluation values of the unit evaluations related to the child library. For example, in the example of FIG. 5, since the library "L123" includes two child libraries “L911” and "L912", the evaluation value having the largest value among the evaluation values by the single evaluation of these two child libraries is Be identified.
  • the evaluation value A13 is specified.
  • the risk evaluation value generation unit 212 treats the library (child library) corresponding to the specified evaluation value as a risk factor of the evaluation target library (parent library). In the above example, the library "L911" corresponding to the evaluation value A13 is treated as a risk factor.
  • the risk evaluation value generation unit 212 extracts the dependent library as a risk factor of the parent library if a dependency relationship is recognized between the libraries.
  • the risk evaluation value generation unit 212 collects the above unit evaluations and the risk factors and generates a “library evaluation result”. For example, the risk evaluation value generation unit 212 generates the library evaluation result as shown in FIG. 7.
  • the library influence degree calculation unit 213 is a means for calculating the influence degree indicating the influence that each library included in the source code has on the entire source code (the entire project).
  • the library influence calculation unit 213 counts the number of functions calling the library for each library included in the source code.
  • the library "L123" is called twice (7th and 8th lines) in Function A and once (10th line) in Function B, so the number of functions that call the library “L123” is It becomes "2". Since the library "L345" is called once (11th line) in Function B, the number of functions that call the library "L123" is "1".
  • the library influence degree calculation unit 213 calculates the number of functions calling each library as the "library influence degree”.
  • the function may be called recursively depending on the source code.
  • the library influence degree calculation unit 213 may give an extremely large value to the influence degree of the library.
  • the library influence degree calculation unit 213 may calculate the number in which each library is called as the "library influence degree". For example, in the above example, the library “L123” is called three times as a whole, so the library influence degree is "3", and the library "L345" is called once as a whole, so the library influence degree is "1". It becomes.
  • the library influence calculation unit 213 may calculate the library influence using either the number of functions that call each library or the number that each library is called, or uses this information.
  • the degree of library influence may be calculated.
  • the library influence calculation unit 213 calculates the average of the number of functions and the number of calls (for example, arithmetic mean, geometric mean, weighted average), and calculates the calculated average value as the library influence. May be good.
  • the library influence degree calculation unit 213 calculates the library influence degree for each library included in the source code (see FIG. 8).
  • the library information generation unit 202 combines the library evaluation result generated by the risk evaluation value generation unit 212 with the library influence degree calculated by the library influence degree calculation unit 213 to generate "library evaluation information" (see FIG. 9).
  • the risk information output unit 203 is a means for calculating the risk level of the library based on the evaluation value of the library and calculating the risk value indicating the risk inherent in the source code based on the calculated risk level. Further, the risk information output unit 203 outputs the time series data of the calculated risk value. That is, the risk information output unit 203 generates and outputs risk information regarding the source code to be evaluated based on the library evaluation information generated by the library information generation unit 202.
  • the risk information output unit 203 generates risk information for items specified by the administrator or the like.
  • the risk information output unit 203 generates a GUI for inputting the generated risk information and displays it on a liquid crystal monitor or the like.
  • the risk information output unit 203 calculates the risk level for each library included in the evaluation target source code, and sets the total value of the calculated risk levels as the risk value of the entire source code.
  • the risk information output unit 203 calculates the total of the individual evaluation values (static evaluation value, dynamic evaluation value, actual evaluation value) for each library as the risk level of each library.
  • “A11 + B11 + C11” is calculated as the risk level of the library “L123”.
  • "A12 + B12 + C12” is calculated as the risk level of the library.
  • the risk information output unit 203 calculates the total value of the evaluation values (static evaluation value, dynamic evaluation value, substance evaluation value) generated for each different information as the risk level of each library.
  • the risk information output unit 203 adds up the risk levels of each library and calculates it as the risk value of the entire application program.
  • "A11 + B11 + C11 + A12 + B12 + C12 + A13 " Is calculated as the risk value of the entire source code.
  • the risk information output unit 203 may reflect the risk factor and the degree of influence of the library when calculating the degree of risk of each library. For example, when reflecting a risk factor, the risk information output unit 203 adds the evaluation value of the risk factor to the total of the above-mentioned single evaluation values. In the above example, for the library "L123", "A11 + B11 + C11 + A13" is calculated as the risk level of the library.
  • the risk information output unit 203 multiplies the degree of library influence corresponding to the total value of the above single evaluation values. For example, for the library “L123”, “(A11 + B11 + C11) x E11” is calculated as the risk level of the library. Alternatively, the risk factor is also reflected, and for the library “L123”, “(A11 + B11 + C11 + A13) x E11” may be calculated as the risk level of the library.
  • the risk information output unit 203 stores the generated risk information (risk value) in the storage unit 204 together with the generation time. For example, time-series data (history of risk values) of risk values as shown in FIG. 10 is stored in the storage unit 204.
  • the risk information output unit 203 may display the latest risk value (current risk value) stored in the storage unit 204. Alternatively, the risk information output unit 203 may generate a graph (graph as shown in FIG. 10) showing the time transition of the risk value from the time series data stored in the storage unit 204, and display the generated graph. Good.
  • the risk information output unit 203 displays as shown in FIG. In FIG. 11, time-series data relating to the risk value of the entire source code is displayed. As shown in FIG. 11, the risk information output unit 203 calculates the total risk value calculated for the library included in the source code as the risk value of the entire source code, and the calculated risk value of the entire source code. The time series data of may be displayed.
  • the manager who comes into contact with the risk information as shown in Fig. 11 recognizes that the current risk is higher than when the project started operation.
  • the risk value is displayed in the form of a line graph in FIG. 11, the risk value may be displayed in another shape such as a bar graph.
  • the risk information output unit 203 may generate risk information regarding individual libraries. For example, the administrator uses the GUI to specify the library for which risk is to be evaluated. The risk information output unit 203 calculates the degree of risk by summing the unit evaluation values of the designated library.
  • the risk information output unit 203 displays the calculated risk level as risk information (risk value). For example, the risk information output unit 203 may display as shown in FIG. As shown in FIGS. 11 and 12, the risk information output unit 203 may generate a GUI for switching items (whole project, individual library) for displaying time-series data of risk values.
  • the risk information output unit 203 may display individual data (static evaluation value, dynamic evaluation value, actual evaluation value) related to the unit evaluation of each library. For example, the risk information output unit 203 may display as shown in FIG.
  • FIG. 13A three evaluation values (time-series data of each evaluation value) obtained by a single evaluation of the library “L123” are displayed at the same time.
  • FIG. 13B time-series data relating to the dynamic evaluation value among the three evaluation values constituting the unit evaluation of the library “L123” is displayed. Alternatively, two of the three evaluation values obtained by the unit evaluation may be selected and displayed.
  • the risk information output unit 203 may display a list of risk values of each library.
  • the risk information output unit 203 may display a library and a list of risk values of the library, as shown in FIG. 14A.
  • the risk information output unit 203 may execute threshold processing on the risk value and display a list of the results.
  • the risk information output unit 203 may display the breakdown of the risk value, or may display the breakdown as incidental information of the time series data. For example, the risk information output unit 203 may display the breakdown of the unit evaluation at the designated time as shown in FIG.
  • the risk information output unit 203 may display the dependency relationship (dependency relationship) between the libraries based on the library configuration information (see FIG. 5). For example, the risk information output unit 203 may display as shown in FIG.
  • the risk information output unit 203 may display information (for example, metadata) that is the basis of the unit evaluation.
  • the risk information output unit 203 may display information as shown in FIG.
  • the evaluation device 10 acquires the source code to be evaluated (step S101).
  • the evaluation device 10 extracts the library described in the source code (step S102).
  • the evaluation device 10 carries out a single evaluation of the extracted library (step S103).
  • the evaluation device 10 calculates the degree of influence (library influence degree) related to the extracted library (step S104).
  • the evaluation device 10 generates and outputs risk information based on the library evaluation information (step S105).
  • the risk evaluation of the source code using the evaluation device 10 is performed periodically or at a predetermined timing, and the evaluation result is stored in the storage unit 204 as history information. For example, the above risk evaluation is executed at the timing when the source code of a certain project is released.
  • the evaluation device 10 performs a single evaluation of each library described in the source code, and presents the risk change of the source code to the administrator or the like based on the evaluation.
  • the manager who comes into contact with the presented risk change can judge whether or not the risk of the project (application) is acceptable.
  • the evaluation device 10 can calculate the risk change of each library described in the source code and provide it to the administrator or the like. As a result, the administrator or the like can accurately obtain information such as a significant change in the risk of any library.
  • the evaluation device 10 evaluates the library every time the source code is acquired, it is possible to clarify the potential risk caused by the version difference of the library or the like. For example, even though the version of the library used at the start of operation is advanced, the version of the library described in the source code to be evaluated may be the same as at the start of operation. In this case, the result of the dynamic evaluation (evaluation using metadata) of the library unit evaluation by the evaluation device 10 deteriorates, and the risk value increases. The evaluation device 10 detects a situation in which such a library version upgrade is left unattended, and notifies the administrator or the like of the risk caused by the neglect.
  • the evaluation device 10 scans the source code each time the source code is acquired and calculates the degree of influence of the library included in the source code, the risk caused by the change on the side using the library is also clarified. can do.
  • the library A may be called by one function at the start of operation, and the library A may be called by three functions at the time of evaluation. In this case, even if the risk value of the library A itself has not changed, the degree of influence of the library A on the entire source code (project) has increased (three times in the above example). Since the evaluation device 10 calculates the risk value while considering the influence of the library on the entire source code, the risk value caused by the change on the side using the library can be calculated.
  • the evaluation device 10 provides risk information to an administrator or the like, and the administrator who comes into contact with the risk information determines the risk of the source code (entire source code, each library).
  • the evaluation device 10 automatically determines the risk of the source code and presents the determination result to the administrator or the like will be described.
  • FIG. 19 is a diagram showing an example of a processing configuration (processing module) of the evaluation device 10 according to the second embodiment.
  • a risk notification unit 205 is added to the evaluation device 10 according to the first embodiment.
  • the risk notification unit 205 is a means for notifying the outside that the source code contains a risk when the calculated risk value satisfies a predetermined condition. Specifically, the risk notification unit 205 notifies the administrator if a risk that needs to be notified is found in the source code.
  • the risk notification unit 205 executes threshold processing on the risk value (risk value of the application program, risk value related to the individual library) generated by the risk information output unit 203. If the risk value is equal to or higher than a predetermined threshold value, the risk notification unit 205 notifies the administrator or the like that “there is a risk in the source code”. Specifically, the risk notification unit 205 makes a display as shown in FIG.
  • the risk notification unit 205 may notify the administrator or the like to that effect. For example, the risk notification unit 205 makes a display as shown in FIG.
  • the risk notification unit 205 may calculate the risk increase rate at the start of operation and at the present time, and notify according to the calculated risk increase rate.
  • the calculation of the rate of increase may be for the entire application program or for individual libraries.
  • the risk notification unit 205 may calculate the risk increase rate for each of the evaluation values (static evaluation value, dynamic evaluation value, substantive evaluation value) that constitute the unit evaluation of the library. For example, the risk notification unit 205 notifies the administrator or the like if the dynamic evaluation value for a specific library is extremely high as compared with the start of operation. In this case, the risk notification unit 205 may display as shown in FIG. FIG. 22 shows that the dynamic evaluation value of the library “L123” is increasing and the time series data of the dynamic evaluation value including the operation start time and the current time.
  • the evaluation device 10 detects the fluctuation of the risk of the source code, and when it is necessary to notify the manager or the like of the fluctuation of the risk, it manages to that effect. Notify the person, etc. As a result, the administrator or the like does not need to comprehensively check the risk change of the library described in the source code, and can prevent overlooking a serious risk.
  • FIG. 23 is a diagram showing an example of the hardware configuration of the evaluation device 10.
  • the evaluation device 10 can be configured by an information processing device (so-called computer), and includes the configuration illustrated in FIG. 23.
  • the evaluation device 10 includes a processor 311, a memory 312, an input / output interface 313, a communication interface 314, and the like.
  • the components such as the processor 311 are connected by an internal bus or the like so that they can communicate with each other.
  • the configuration shown in FIG. 23 does not mean to limit the hardware configuration of the evaluation device 10.
  • the evaluation device 10 may include hardware (not shown), or may not include an input / output interface 313 if necessary.
  • the number of processors 311 and the like included in the evaluation device 10 is not limited to the example of FIG. 23, and for example, a plurality of processors 311 may be included in the evaluation device 10.
  • the processor 311 is a programmable device such as a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or a DSP (Digital Signal Processor). Alternatively, the processor 311 may be a device such as an FPGA (Field Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit). The processor 311 executes various programs including an operating system (OS).
  • OS operating system
  • the memory 312 is a RAM (RandomAccessMemory), a ROM (ReadOnlyMemory), an HDD (HardDiskDrive), an SSD (SolidStateDrive), or the like.
  • the memory 312 stores an OS program, an application program, and various data.
  • the input / output interface 313 is an interface of a display device or an input device (not shown).
  • the display device is, for example, a liquid crystal display or the like.
  • the input device is, for example, a device that accepts user operations such as a keyboard and a mouse.
  • the communication interface 314 is a circuit, module, or the like that communicates with another device.
  • the communication interface 314 includes a NIC (Network Interface Card) and the like.
  • the function of the evaluation device 10 is realized by various processing modules.
  • the processing module is realized, for example, by the processor 311 executing a program stored in the memory 312.
  • the program can also be recorded on a computer-readable storage medium.
  • the storage medium may be a non-transitory such as a semiconductor memory, a hard disk, a magnetic recording medium, or an optical recording medium. That is, the present invention can also be embodied as a computer program product.
  • the program can be downloaded via a network or updated using a storage medium in which the program is stored.
  • the processing module may be realized by a semiconductor chip.
  • the configuration, operation, and the like of the evaluation device 10 described in the above embodiment are examples, and are not intended to limit the configuration and the like of the device.
  • the functions of the evaluation device 10 described above may be realized by different devices.
  • the functions of the library information generation unit 202 and the risk information output unit 203 may be implemented in different devices.
  • the risk evaluation value generation unit 212 scans the source code and performs a single evaluation by "static code analysis” that analyzes the description contents.
  • “dynamic program analysis” may be executed by executing the executable file generated from the source code to be evaluated and analyzing the source code to be evaluated. For example, a test program that sets a flag each time each library is called may be prepared, and the number of times the library is called per unit time may be used as the evaluation value.
  • the risk of the source code is evaluated from the viewpoint (viewpoint) of the library described in the source code, but the risk of the source code is evaluated in consideration of the evaluation result of the source code itself. May be good.
  • the same analysis as the code analysis of the library analysis using the number of lines, complexity, etc.
  • the result may be reflected in the risk value of the entire source code.
  • the unit evaluation (static evaluation value, dynamic evaluation value, substance evaluation value) of the library is added.
  • the risk level may be calculated by a weighted average using weights given in advance for each item. For example, among the above three evaluation values, the weight may be determined so as to emphasize the dynamic evaluation value, and the risk level of the library may be calculated.
  • the machine learning method may be applied to all or part of them. For example, in the calculation of the substantive evaluation value using the source code, a large number of source codes of excellent libraries with few vulnerabilities are collected, and the collected source code is labeled with a high evaluation and prepared as teacher data. A learning model is generated using the teacher data, and by inputting the source code of the library to be evaluated into the learning model, the difference from the teacher data is output as a score. The score output by the learning model may be treated as an entity evaluation value.
  • the evaluation value of the child library is included as a risk factor in the evaluation of the parent library.
  • the evaluation value of the child library and the grandchild library may be included in the evaluation of the parent library.
  • the risk evaluation value generation unit 212 generates the library evaluation result as shown in FIG. 24.
  • the child library “L911” of the parent library “L123” is extracted as the first risk factor
  • the grandchild library “L811” is extracted as the second risk factor. Since the extraction of the risk factor can be the same as the content described above, the description thereof will be omitted.
  • the evaluation device 10 described above handles source code related to a plurality of projects. Therefore, it is possible that the projects are different but the libraries used are the same. In that case, if the same library as the library for which the unit evaluation has already been completed is used in another project (source code), the evaluation device 10 may reuse the result of the unit evaluation of the library. Good.
  • the computer By installing the evaluation program in the memory of the computer, the computer can function as an evaluation device. Further, by causing the computer to execute the evaluation program, the evaluation method can be executed by the computer.
  • [Appendix 1] A generator (101, 212) that generates an evaluation value regarding the risk of the first library described in the source code, and The risk value of the first library is calculated based on at least the generated evaluation value, the risk value indicating the risk inherent in the source code is calculated based on the calculated risk level, and the calculated risk value is calculated.
  • Output units (102, 203) that output time-series data of risk values, An evaluation device (10, 100) comprising.
  • the generation unit (101, 212) When the source code includes a second library called from the first library, the generation unit (101, 212) generates an evaluation value regarding the risk of the second library.
  • a calculation unit (213) for calculating the degree of influence indicating the influence of the first library on the entire source code is further provided.
  • [Appendix 4] The evaluation device (10, 100) according to Appendix 3, wherein the generation unit (101, 212) evaluates the first library based on information having different properties from each other and generates an evaluation value for each of the different information.
  • [Appendix 5] The evaluation device (10, 100) according to Appendix 4, wherein the output unit (102, 203) calculates the total value of the evaluation values generated for each of the different information as the risk level of the first library.
  • the output unit (102, 203) identifies the evaluation value having the largest value among the evaluation values for each of the different information generated for the second library, and the evaluation value generated for the first library.
  • [Appendix 7] The evaluation device (10,) according to Appendix 5 or 6, wherein the output unit (102, 203) calculates the degree of risk of the first library by multiplying the total value of the evaluation values by the degree of influence. 100).
  • the generation unit (101, 212) generates a static evaluation value based on the static information about the first library, and generates a dynamic evaluation value based on the dynamic information about the first library.
  • the evaluation device (10, 100) according to any one of Supplementary note 4 to 7, which generates an entity evaluation value based on the source code of the first library.
  • the output unit (102, 203) calculates the total risk value calculated for the library included in the source code as the risk value of the entire source code, and when the calculated risk value of the entire source code is used.
  • the evaluation device (10, 100) according to any one of Supplementary note 1 to 7, which displays series data.
  • the evaluation device (10, 100) according to Appendix 9, wherein the output unit (102, 203) displays time-series data of the degree of risk of the first library.
  • [Appendix 11] The evaluation device (10, 100) according to Appendix 10, wherein the output unit (102, 203) generates a GUI (Graphical User Interface) for switching an item for displaying time-series data of a risk value.
  • GUI Graphical User Interface
  • Appendix 14 On the computer (311) mounted on the evaluation device (10, 100), The process of generating the evaluation value for the risk of the first library described in the source code, The risk value of the first library is calculated based on at least the generated evaluation value, the risk value indicating the risk inherent in the source code is calculated based on the calculated risk level, and the calculated risk value is calculated. Processing to output time series data of risk value and A program that executes. Note that the form of Appendix 13 and the form of Appendix 14 can be expanded to the forms of Appendix 2 to the form of Appendix 12 in the same manner as the form of Appendix 1.

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Hardware Design (AREA)
  • Computer Security & Cryptography (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • Stored Programmes (AREA)
  • Debugging And Monitoring (AREA)
PCT/JP2019/041929 2019-10-25 2019-10-25 評価装置、評価方法及びプログラム Ceased WO2021079496A1 (ja)

Priority Applications (3)

Application Number Priority Date Filing Date Title
JP2021553261A JP7322963B2 (ja) 2019-10-25 2019-10-25 評価装置、評価方法及びプログラム
PCT/JP2019/041929 WO2021079496A1 (ja) 2019-10-25 2019-10-25 評価装置、評価方法及びプログラム
US17/767,138 US12254097B2 (en) 2019-10-25 2019-10-25 Evaluation apparatus, evaluation method, and program

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2019/041929 WO2021079496A1 (ja) 2019-10-25 2019-10-25 評価装置、評価方法及びプログラム

Publications (1)

Publication Number Publication Date
WO2021079496A1 true WO2021079496A1 (ja) 2021-04-29

Family

ID=75619722

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2019/041929 Ceased WO2021079496A1 (ja) 2019-10-25 2019-10-25 評価装置、評価方法及びプログラム

Country Status (3)

Country Link
US (1) US12254097B2 (https=)
JP (1) JP7322963B2 (https=)
WO (1) WO2021079496A1 (https=)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022269656A1 (ja) * 2021-06-21 2022-12-29 三菱電機株式会社 機械学習装置、深刻度予知装置、および機械学習方法
WO2023139822A1 (ja) * 2022-01-18 2023-07-27 三菱電機株式会社 アーキテクチャ寿命推定装置及びアーキテクチャ寿命推定方法
JP2024502587A (ja) * 2021-09-29 2024-01-22 シャンハイ トサン テクノロジー リミテッド ソフトウェアプラットフォーム用第三者ライブラリ関数の相互呼び出し方法及び相互呼び出しシステム
WO2025248642A1 (ja) * 2024-05-28 2025-12-04 Ｎｔｔ株式会社 評価装置、評価方法及び評価プログラム

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2001184441A (ja) * 1999-12-27 2001-07-06 Nec Corp 診療録装置
JP2010191922A (ja) * 2009-02-20 2010-09-02 Mitsubishi Electric Corp 履歴追跡結果出力装置及び履歴追跡結果出力プログラム及び記録媒体
US20150007330A1 (en) * 2013-06-26 2015-01-01 Sap Ag Scoring security risks of web browser extensions
US20150268948A1 (en) * 2014-03-18 2015-09-24 Henrik Plate Software dependency management through declarative constraints
US20180349614A1 (en) * 2017-05-31 2018-12-06 ShiftLeft Inc System and method for application security profiling
US20190227902A1 (en) * 2018-01-21 2019-07-25 Microsoft Technology Licensing, Llc. Time-weighted risky code prediction

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4162541B2 (ja) * 2002-06-07 2008-10-08 株式会社日立製作所 資産運用管理支援装置および資産運用管理支援方法
US7437718B2 (en) * 2003-09-05 2008-10-14 Microsoft Corporation Reviewing the security of trusted software components
JP4240504B2 (ja) 2006-08-04 2009-03-18 インターナショナル・ビジネス・マシーンズ・コーポレーション 製品開発プロセスにおける設計変更の影響度分析装置および方法
JP5201068B2 (ja) 2009-04-23 2013-06-05 富士通株式会社 危険度判定プログラム、危険度判定装置及び方法
US9858176B2 (en) 2013-08-12 2018-01-02 Nec Corporation Software aging test system, software aging test method, and program for software aging test
EP3155512A1 (en) 2014-06-13 2017-04-19 The Charles Stark Draper Laboratory, Inc. Systems and methods for software analytics
US10528741B1 (en) * 2016-07-13 2020-01-07 VCE IP Holding Company LLC Computer implemented systems and methods for assessing operational risks and mitigating operational risks associated with using a third party software component in a software application
US11481498B2 (en) * 2019-01-28 2022-10-25 Visa International Service Association Continuous vulnerability management for modern applications

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2001184441A (ja) * 1999-12-27 2001-07-06 Nec Corp 診療録装置
JP2010191922A (ja) * 2009-02-20 2010-09-02 Mitsubishi Electric Corp 履歴追跡結果出力装置及び履歴追跡結果出力プログラム及び記録媒体
US20150007330A1 (en) * 2013-06-26 2015-01-01 Sap Ag Scoring security risks of web browser extensions
US20150268948A1 (en) * 2014-03-18 2015-09-24 Henrik Plate Software dependency management through declarative constraints
US20180349614A1 (en) * 2017-05-31 2018-12-06 ShiftLeft Inc System and method for application security profiling
US20190227902A1 (en) * 2018-01-21 2019-07-25 Microsoft Technology Licensing, Llc. Time-weighted risky code prediction

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
"OSS Risk Management Tool Checkmarx CxOSA", PROMETECH SIMULATION CONFERENCE 2018, 13 December 2018 (2018-12-13), pages 1 - 2 *

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022269656A1 (ja) * 2021-06-21 2022-12-29 三菱電機株式会社 機械学習装置、深刻度予知装置、および機械学習方法
JPWO2022269656A1 (https=) * 2021-06-21 2022-12-29
JP7499965B2 (ja) 2021-06-21 2024-06-14 三菱電機株式会社 機械学習装置、深刻度予知装置、機械学習方法、および深刻度予知方法
JP2024502587A (ja) * 2021-09-29 2024-01-22 シャンハイ トサン テクノロジー リミテッド ソフトウェアプラットフォーム用第三者ライブラリ関数の相互呼び出し方法及び相互呼び出しシステム
JP7562187B2 (ja) 2021-09-29 2024-10-07 シャンハイ トサン テクノロジー リミテッド ソフトウェアプラットフォーム用第三者ライブラリ関数の相互呼び出し方法及び相互呼び出しシステム
US12547396B2 (en) 2021-09-29 2026-02-10 Shanghai Tosun Technology Ltd. Software platform-specific mutual calling method for functions of third-party program libraries, and mutual calling system
WO2023139822A1 (ja) * 2022-01-18 2023-07-27 三菱電機株式会社 アーキテクチャ寿命推定装置及びアーキテクチャ寿命推定方法
JPWO2023139822A1 (https=) * 2022-01-18 2023-07-27
JP7621522B2 (ja) 2022-01-18 2025-01-24 三菱電機株式会社 アーキテクチャ寿命推定装置及びアーキテクチャ寿命推定方法
WO2025248642A1 (ja) * 2024-05-28 2025-12-04 Ｎｔｔ株式会社 評価装置、評価方法及び評価プログラム

Also Published As

Publication number Publication date
US20220391516A1 (en) 2022-12-08
JPWO2021079496A1 (https=) 2021-04-29
US12254097B2 (en) 2025-03-18
JP7322963B2 (ja) 2023-08-08

Similar Documents

Publication Publication Date Title
US11263071B2 (en) Enabling symptom verification
US10657264B2 (en) Techniques for correlating vulnerabilities across an evolving codebase
Neuhaus et al. Predicting vulnerable software components
JP7322963B2 (ja) 評価装置、評価方法及びプログラム
Nadi et al. Mining configuration constraints: Static analyses and empirical results
Bacchelli et al. Are popular classes more defect prone?
Barbour et al. An empirical study of faults in late propagation clone genealogies
EP3618078B1 (en) System and method for controlling quality of performance of digital applications
Smidts et al. Software testing with an operational profile: OP definition
Elder et al. A survey on software vulnerability exploitability assessment
Sajnani et al. Is popularity a measure of quality? an analysis of maven components
Andrés et al. Formal passive testing of timed systems: Theory and tools
Martins et al. On the diffusion of test smells and their relationship with test code quality of java projects
Li et al. Open source software security vulnerability detection based on dynamic behavior features
US20170242663A1 (en) Software model stability metrics
Samad et al. Multiobjective test case prioritization using test case effectiveness: multicriteria scoring method
KR20140050323A (ko) 라이선스 검증 방법 및 그 장치
Mahmud et al. Acid: an api compatibility issue detector for android apps
Pilch et al. Ideas underlying the quantification of margins and uncertainties
Auguston et al. Environment behavior models for automation of testing and assessment of system safety
US8909579B2 (en) Identifying invariant candidates based on proofs
Henelius et al. Goldeneye++: A closer look into the black box
Garousi et al. Test cost-effectiveness and defect density: a case study on the android platform
Llanso et al. Estimating software vulnerability counts in the context of cyber risk assessments
Lee et al. AutoMetric: Towards Measuring Open-Source Software Quality Metrics Automatically

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19949905

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021553261

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19949905

Country of ref document: EP

Kind code of ref document: A1

WWG Wipo information: grant in national office

Ref document number: 17767138

Country of ref document: US