CN115730309A - Multi-type scanning tool source code index statistical method, storage medium and terminal - Google Patents
Multi-type scanning tool source code index statistical method, storage medium and terminal Download PDFInfo
- Publication number
- CN115730309A CN115730309A CN202211460324.4A CN202211460324A CN115730309A CN 115730309 A CN115730309 A CN 115730309A CN 202211460324 A CN202211460324 A CN 202211460324A CN 115730309 A CN115730309 A CN 115730309A
- Authority
- CN
- China
- Prior art keywords
- tool
- index
- task
- diagnosis
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Debugging And Monitoring (AREA)
Abstract
The invention relates to a method for counting source code indexes of a multi-type scanning tool, a storage medium and a terminal, comprising the following steps: the system comprises an integrated development design universal index statistical module and a diagnosis tool driving module, wherein the server is used for executing an index statistical module to perform statistics on universal index items required by a tool integration application platform; when a user triggers a diagnosis task, a task is created, and the diagnosis task is bound with a diagnosis tool driving module and a general index counting module; and the diagnostic tool driving module executes the diagnostic tool command, performs statistics on the universal index items of the codes after finishing the diagnostic tool command, and finally assembles the data into a data format meeting the requirements and sends the data format to the tool integration application platform. The implementation scheme of the invention is simple, the acquisition of the universal index data can be completed at the server side where the source code is located only by utilizing the existing professional tool, and complex interface calling is not needed; and the index acquisition function can be plugged and disconnected, and the required universal index acquisition and the like can be flexibly controlled according to the platform service requirements.
Description
Technical Field
The invention relates to the technical field of software security, in particular to a method for counting source code indexes of a multi-type scanning tool, a storage medium and a terminal.
Background
The software security field has various diagnostic tools related to code security, such as a component analysis (SCA) tool focusing on detecting software components, a Static Analysis (SAST) tool focusing on source code problem analysis, and a dynamic analysis test (DAST) tool focusing on detecting program bugs; the diversification of the tool types also promotes the production of a corresponding tool integration application platform, and integrates various tools of different types so as to meet the diagnosis requirements of users on various problems of codes; the platform can divide the diagnostic data of the user code into diagnostic indexes related to tools and some general indexes unrelated to the tools, wherein the general indexes usually comprise the size of an original file, the line number of a source code, the number of an annotation line, the complexity of the source code and the like; some problem diagnosis tools can provide some indexes, some diagnosis tools cannot provide the problems, or the provided data is not comprehensive enough, so that a method for counting the source code general indexes is needed to meet the requirements of a tool integration application platform on the display of the general indexes, secondary service development and the like.
However, the existing implementation method has the following defects: 1. the tool integration application platform obtains the universality index through the diagnosis tool, but different diagnosis tool data do not necessarily meet some key index items required by the tool integration application platform, the platform index extremely depends on the diagnosis tool, and the service expansion is not flexible enough; 2. the general indexes of the tool integration application platform come from different diagnostic tools, and the platform needs to perform data interface butt joint aiming at different tools, however, the tools are various, and the interface data formats are different, so that the maintenance cost is too high; 3. the general index data of the tool integration application platform comes from tools, and if a platform user does not use some diagnosis tools, general data items are lost; 4. and data conflict, because the universal data items come from the diagnostic tools, the statistics of some index data of the same source code are different among different diagnostic tools, and at the moment, the tool integration application platform faces the conflict data integration problem.
It is noted that the information disclosed in the above background section is only for enhancement of understanding of the background of the present disclosure and therefore may include information that does not constitute prior art that is already known to a person of ordinary skill in the art.
Disclosure of Invention
The invention aims to overcome the defects of the prior art, provides a method for counting source code indexes of a multi-type scanning tool, a storage medium and a terminal, and solves the defects of the prior art.
The purpose of the invention is realized by the following technical scheme: a multi-type scanning tool source code index statistical method comprises the following steps:
the method comprises the steps that firstly, a general index statistical module and a diagnosis tool driving module are integrated, the general index item required by a tool integration application platform is counted by the index statistical module executed on a server, command line parameters of general indexes required by various statistical services are packaged in the index statistical driving module, and scanning command line parameters related to various diagnosis tools are packaged in the diagnosis tool driving module;
step two, when a user triggers a diagnosis task, a task is created, creation parameters are transmitted in the task creating engineering, and the diagnosis task is bound with a diagnosis tool driving module and a general index counting module;
and step three, the diagnostic tool driving module executes the diagnostic tool command, executes the general index counting module after the diagnostic tool command is finished, completes counting of general index items of the codes, assembles data into a data format meeting the requirements after the general index counting module is finished, and sends the data format to the tool integration application platform.
The method comprises the following steps that when a user triggers a diagnosis task, the task is created, creation parameters are transmitted in a task creation project, and the step of binding the diagnosis task with a diagnosis tool driving module and a general index counting module specifically comprises the following steps:
the method comprises the steps that a user selects a diagnosis tool A through a tool integration application platform to perform code problem diagnosis and then triggers a diagnosis task, the tool integration application platform creates a task in a task arrangement tool through an internal service program, the task is named Job-A, and in the process of creating the task, aiming at the diagnosis tool, when the tool integration application platform creates Job-A to transmit creation parameters, service parameters required by execution of various commands are transmitted simultaneously to serve as parameters of a diagnosis tool driving module and/or a general index statistical module;
and binding a diagnosis tool driving module and a general index statistical module on the execution node of the created task Job-A in sequence according to the service sequence.
The step of simultaneously transmitting the service parameters required by the execution of various commands as the reference of the diagnostic tool driving module and/or the universal index statistical module comprises the following steps: a diagnostic tool driving module needs to transmit a source code address or a source code pulling address to be scanned by a diagnostic tool A and necessary tool connection information of the diagnostic tool A; and for the general index statistical module, a code path, a platform callback address and access token information need to be transmitted.
The diagnostic tool driver module executing the diagnostic tool commands comprises:
the diagnostic tool driving module logically connects command lines for executing the source code program scanning step in the diagnostic tool in series according to the type of the diagnostic tool, and encapsulates the diagnostic command steps needing to be called for many times in the diagnostic tool driving module;
certainly, when the initial parameters of the command line are input to the diagnostic tool driving module from the outside, the diagnostic tool driving module judges the identifier of the command of the previous step, if the identifier of the command of the previous step is a successful identifier, the next step of command is executed, otherwise, the diagnosis is finished.
The general index counting module completes counting of general index items of the codes and comprises the following steps:
the general index statistical module packages and connects different index statistical commands in series, calls initial parameters of a source code path and a platform callback address transmitted by a script program, processes each statistical index data logic through the script program, and does not influence command execution results of different index data items;
and assembling the successfully counted data into a standard format in an additional form and outputting the standard format to an appointed server path, and after all the index commands are executed, sending the statistical data to a tool integration application platform by a script program through a callback address so as to finish the collection and counting of the universal index items.
And in the triggered execution process, the task Job-A executes the Job-A arrangement task at the server end, and the task sequentially executes a diagnostic tool driving module and a general index counting module which are driven on the same service host.
A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the source code metric statistics method.
The terminal equipment comprises a memory and a processor, wherein the memory is stored with a computer program, and the processor realizes the steps of the source code index statistical method when executing the computer program.
The invention has the following advantages: a multi-type scanning tool source code index statistical method, storage medium and terminal, the acquisition of the general index is decoupled with the concrete diagnosis tool, no matter whether the diagnosis tool supports some general index data item acquisition, the tool integration application platform can accurately and standard general index items required by the statistical platform service, and can integrate other specialized index statistical tools according to the expansion requirement of the platform itself, so as to provide more dimensional code general index data for the platform user; the implementation scheme is simple, the acquisition of the universal index data can be completed at the server side where the source code is located only by utilizing the existing professional tool, and complex interface calling is not needed; the index acquisition function is pluggable, and the required universal index acquisition and the like can be flexibly controlled according to the platform service requirement; the tool integration application platform takes a professional universal index detection tool as standard data, and the problems of data difference, conflict and deficiency caused by different diagnostic tools do not exist.
Drawings
FIG. 1 is a schematic flow diagram of the process of the present invention;
py internal logic diagram of the instrument script cloc;
fig. 3 is a schematic diagram of the diagnostic procedure of the diagnostic tool a.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all the embodiments. The components of the embodiments of the present application, as generally described and illustrated in the figures herein, could be arranged and designed in a wide variety of different configurations. Thus, the detailed description of the embodiments of the present application provided below in connection with the appended drawings is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present application without making any creative effort, shall fall within the protection scope of the present application. The invention is further described below with reference to the accompanying drawings.
As shown in fig. 1, one embodiment of the present invention specifically relates to a statistical method for source code index information compatible with different types of scanning tools, which specifically includes the following steps:
and S1, according to the requirement of a universal index item required by a tool integration application platform, self-developing and designing a scripted program which is used for index statistics and is convenient to directly execute on a server, such as a Python script or a Shell script, and the like, wherein the scripted program is convenient to express and name as Cloc. The script encapsulates various command lines which can count general indexes needed by the service, wherein the command lines can be system commands, and can also be installed specialized statistical tools to execute commands and the like.
As shown in fig. 2, for example, the business requirement is to count the file size, code line, code annotation, etc. of the source code. If the file size is obtained by the Linux command du-h sourceDir, the code line, the code annotation line is obtained by the third party's statistics gadget Cloc, the execution command of which is Cloc sourceDir. The script Cloc.py is to encapsulate and connect the statistical command lines of different indexes together in a unified way, call the script Cloc.py, transmit the necessary initial parameters such as source code path, platform callback address, etc., and process the data logic of each statistical index by the Cloc.py script, and the command execution results of different index data items are not influenced mutually, i.e. whether the previous index data statistics is successful or not does not influence the statistics of the subsequent indexes, so as to realize the loose coupling of the architecture. And after all the index commands are executed, the Cloc.
When a new statistical index is needed to be added to the service, only a corresponding index statistical tool needs to be added to the service-side environment, and a calling step for a new index statistical command line is added to the Cloc. On the basis of the existing index statistical items, the circle complexity of a source code needs to be counted according to the existing business requirements, so that a command line gadget special for counting the circle complexity of the source code is additionally arranged, the script execution command of the command line gadget is lizard sourceDir, the command for adding the circle complexity is only required to be added in a related mode in the internal sequence flow of a close. Similarly, if a certain index is to be removed, simply modifying the Cloc.py script and removing the related steps of the unneeded index statistical command line; of course, flexible control of the acquisition of the universal index can be realized by dynamically selecting the required index items through the internal logic of clock.
And S2, as shown in FIG. 3, the Tool integration application platform develops and designs a diagnosis Tool driving script in advance according to different integrated diagnosis tools, and the driving script is named as Tool-A. The script encapsulates scan command line parameters associated with the diagnostic tool. For example, the Tool a needs to perform source code pulling, compiling, scanning, problem analysis, result uploading (Tool command behavior) and other ordered steps on a source code program, the drive script Tool-a.py is to connect command lines of the execution steps in series in a logic manner, the diagnostic command line steps needing to be called for many times are completely packaged inside the Tool-a.py, an external calling party only needs to transmit necessary command line initial parameters, and the Tool-a.py integrates the execution steps, so that the command line calling work of the diagnostic Tool is greatly simplified.
Because the inside of the Tool-A.py is a logic series connection of each execution step, the inside can completely determine whether to execute the next command line according to the success and failure of the command of the previous step (the success and failure judgment identification of the command is provided by the native command of the diagnostic Tool, for example, many commands are successful in outputting the 0 identification, and other non-0 numbers are failure or abnormal, so after the Tool execution command is called in the Tool-A.py script program step, only the output result is judged), and the flow control is more convenient;
because the diagnostic tool script is designed for self-development, standardization of external input commands can be achieved. No matter what type of diagnostic tool, the tool integration application platform only needs to design a set of flow and a set of standardized parameter input command lines, the standardized parameters are analyzed into the diagnostic parameters of the diagnostic tool to be executed by the internal part of the diagnostic driving script, and the difference of different diagnostic tool command lines is completely shielded for upper-layer application, so that the repeated coding and maintenance work when the tool integration application platform is connected into different diagnostic tools is greatly reduced.
The success and failure of the diagnostic tool script is determined by the command of the diagnostic tool, for example, the connection address input to the diagnostic tool is incorrect, the path of the source code file is wrong, and the like, so that the execution of the diagnostic tool command fails, the significance of the index statistical tool is lost after the failure, and the process is interrupted; the success and failure of the index item command in the general index script are determined by the index script according to the input items, and the success and failure of the whole index script depends on whether a statistical index file is generated.
And S3, after a user selects a certain diagnosis tool (such as a tool A) to diagnose a code problem by using the tool integration application platform, triggering a diagnosis task, and creating a task in a task arrangement tool (such as Jenkins) by using the tool integration application platform through an internal service program, wherein the task is named Job-A. On a script execution node (such as an Execute Shell node in Jenkins) of the created task Job-A, binding a diagnosis Tool script Tool-A.py and a general indicator script Cloc.py in sequence according to a service sequence;
and S4, in the process of creating the task in the step 3, aiming at the diagnosis tool, when the tool integration application platform creates Job-A transfer creation parameters, the tool integration application platform simultaneously transmits service parameters required by execution of various commands as the input parameters of the execution script. For example, for Tool-a.py, the source address or source pull address (such as git _ url) to be scanned by diagnostic Tool a, the necessary Tool connection information of diagnostic Tool a, etc. are transmitted; py, in order to count the code common indicators, necessary information such as the code path source _ path, the platform callback address callback _ Url, the access token, etc. is also passed, meaning other necessary information such as the callback address, the access token, and the tool necessary to perform diagnosis that needs to be passed to execute the script.
And S5, in the process of executing after triggering, job-A executes the Job-A editing task at the server end, and when the task is executed, the task is executed and configured at a script execution configuration node (for example, on a common Jenkins editing Tool, an execution Shell script command configuration node which can be selected in the construction step) which is used for editing script execution by a task editing Tool, namely, the Job-A and Job-A sequentially drive the Tool-A.py and the clock.py on the same service host to Execute.
And S6, executing a normal diagnostic Tool command by Tool-A.py, and executing a Cloc.py script after the diagnostic Tool command is executed to complete the statistics of the universal index items of the codes. Py script will be assembled into a standard and extensible Json format which is agreed and consistent with the tool integration application platform, and sent to the tool integration application platform in the form of HTTP request. And then the platform completes the subsequent service data processing.
Another embodiment of the invention relates to a computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the source code index statistical method. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, read-Only Memory (ROM), random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, and the like. It should be noted that the computer readable medium may contain content that is subject to appropriate increase or decrease as required by legislation and patent practice in jurisdictions, for example, in some jurisdictions, computer readable media does not include electrical carrier signals and telecommunications signals as is required by legislation and patent practice.
Still another embodiment of the present invention relates to a terminal device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the source code index statistical method when executing the computer program.
The foregoing is illustrative of the preferred embodiments of this invention, and it is to be understood that the invention is not limited to the precise form disclosed herein and that various other combinations, modifications, and environments may be resorted to, falling within the scope of the concept as disclosed herein, either as described above or as apparent to those skilled in the relevant art. And that modifications and variations may be effected by those skilled in the art without departing from the spirit and scope of the invention as defined by the appended claims.
Claims (8)
1. A method for counting source code indexes of multi-type scanning tools is characterized by comprising the following steps: the source code index statistical method comprises the following steps:
the method comprises the steps that firstly, a general index counting module and a diagnosis tool driving module are integrated, the index counting module is executed on a server to count general index items required by a tool integration application platform, command line parameters of general indexes required by various statistical services are packaged in the index counting driving module, and scanning command line parameters relevant to various diagnosis tools are packaged in the diagnosis tool driving module;
step two, when a user triggers a diagnosis task, a task is created, creation parameters are transmitted in the task creating engineering, and the diagnosis task is bound with a diagnosis tool driving module and a general index counting module;
and step three, the diagnostic tool driving module executes the diagnostic tool command, executes the general index counting module after the diagnostic tool command is finished, completes counting of general index items of the codes, assembles data into a data format meeting the requirements after the general index counting module is finished, and sends the data format to the tool integration application platform.
2. The method of claim 1, wherein the statistical method comprises: the method comprises the following steps that when a user triggers a diagnosis task, the task is created, creation parameters are transmitted in a task creation project, and the step of binding the diagnosis task with a diagnosis tool driving module and a general index counting module specifically comprises the following steps:
the user selects a diagnosis tool A to diagnose the code problem through a tool integration application platform and then triggers a diagnosis task, the tool integration application platform creates a task in a task arranging tool through an internal service program, the task is named Job-A, and in the process of creating the task, aiming at the diagnosis tool, when the tool integration application platform creates Job-A to transmit creation parameters, the tool integration application platform simultaneously transmits service parameters required by various commands to be executed as parameters of a diagnosis tool driving module and/or a general index statistical module;
and binding a diagnosis tool driving module and a general index statistical module on the execution node of the created task Job-A in sequence according to the service sequence.
3. The method of claim 2, wherein the statistical method comprises: the step of simultaneously transmitting the service parameters required by the execution of various commands as the reference of the diagnostic tool driving module and/or the universal index statistical module comprises the following steps: a diagnostic tool driving module needs to transmit a source code address or a source code pulling address to be scanned by a diagnostic tool A and necessary tool connection information of the diagnostic tool A; and for the general index statistical module, a code path, a platform callback address and access token information need to be transmitted.
4. The method according to claim 1, wherein the statistical method for source code indexes of multi-type scanning tools comprises: the diagnostic tool driver module executing the diagnostic tool commands comprises:
the diagnostic tool driving module logically connects command lines for executing the source code program scanning step in the diagnostic tool in series according to the type of the diagnostic tool, and encapsulates the diagnostic command steps needing to be called for many times in the diagnostic tool driving module;
certainly, when the initial parameters of the command line are input to the diagnostic tool driving module from the outside, the diagnostic tool driving module judges the identifier of the command of the previous step, if the identifier of the command of the previous step is a successful identifier, the next step of command is executed, otherwise, the diagnosis is finished.
5. The method according to claim 1, wherein the statistical method for source code indexes of multi-type scanning tools comprises: the general index counting module completes counting of general index items of the codes and comprises the following steps:
the general index statistical module packages and connects different index statistical commands in series, calls initial parameters of a source code path and a platform callback address transmitted by a script program, processes each statistical index data logic through the script program, and does not influence command execution results of different index data items;
and assembling the successfully counted data into a standard format in an additional form and outputting the standard format to an appointed server path, and after all the index commands are executed, sending the statistical data to a tool integration application platform by a script program through a callback address so as to finish the collection and counting of the universal index items.
6. The method of claim 2, wherein the statistical method comprises: and in the triggered execution process, the task Job-A executes the Job-A arrangement task at the server end, and the task sequentially executes a diagnostic tool driving module and a general index counting module which are driven on the same service host.
7. A computer-readable storage medium having stored thereon a computer program, characterized in that: the computer program when executed by a processor implements the steps of the source code index statistical method of any one of claims 1-6.
8. A terminal device comprising a memory and a processor, the memory having stored thereon a computer program, characterized in that: the processor, when executing the computer program, implements the steps of the source code index statistical method of any one of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211460324.4A CN115730309A (en) | 2022-11-17 | 2022-11-17 | Multi-type scanning tool source code index statistical method, storage medium and terminal |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211460324.4A CN115730309A (en) | 2022-11-17 | 2022-11-17 | Multi-type scanning tool source code index statistical method, storage medium and terminal |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115730309A true CN115730309A (en) | 2023-03-03 |
Family
ID=85297134
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211460324.4A Pending CN115730309A (en) | 2022-11-17 | 2022-11-17 | Multi-type scanning tool source code index statistical method, storage medium and terminal |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115730309A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116383835A (en) * | 2023-06-06 | 2023-07-04 | 北京比瓴科技有限公司 | Software vulnerability monitoring method, device, equipment and medium based on multiple security tools |
-
2022
- 2022-11-17 CN CN202211460324.4A patent/CN115730309A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116383835A (en) * | 2023-06-06 | 2023-07-04 | 北京比瓴科技有限公司 | Software vulnerability monitoring method, device, equipment and medium based on multiple security tools |
| CN116383835B (en) * | 2023-06-06 | 2023-09-19 | 北京比瓴科技有限公司 | Software vulnerability monitoring method, device, equipment and medium based on multiple security tools |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US6978440B1 (en) | System and method for developing test cases using a test object library | |
| US6421822B1 (en) | Graphical user interface for developing test cases using a test object library | |
| US7213233B1 (en) | Modeling standards validation tool for use in enterprise architecture modeling | |
| US7073093B2 (en) | Helpdesk system and method | |
| US7509536B1 (en) | Method and system for error handling | |
| CN107526676B (en) | Cross-system test method and device | |
| CN115730309A (en) | Multi-type scanning tool source code index statistical method, storage medium and terminal | |
| CN113961332A (en) | Method and device for realizing workflow engine, electronic equipment and storage medium | |
| WO2021230394A1 (en) | Software development and test automation framework | |
| CN112686019A (en) | Vehicle sensor data analysis method, device and storage medium | |
| CN114880239A (en) | Interface automation testing framework and method based on data driving | |
| CN114579194A (en) | Spring remote call-based exception handling method and system | |
| CN113434405A (en) | Method and device for determining test file, storage medium and electronic device | |
| CN114416305A (en) | Robot engine implementation method and system and electronic equipment | |
| CN114063982A (en) | Method for automatically constructing functional components based on multi-scenario application | |
| CN110633976B (en) | Virtual resource transfer method and device | |
| CN114371866A (en) | Version reconfiguration test method, device and equipment of service system | |
| KR101948927B1 (en) | Collaborative test device | |
| CN109240935A (en) | A kind of mobile phone games automated testing method, device and the storage medium of compatibility | |
| CN115840778B (en) | Visual configuration connector based on model and connection method thereof | |
| CN115033312B (en) | ElasticSearch data operation method and device | |
| JP3012366B2 (en) | Remote file access method | |
| CN114416027A (en) | Software low-code development system and method | |
| CN108536620B (en) | Bus equipment analysis method, system, device and readable storage medium | |
| CN117992065A (en) | Data stream analysis method, device, storage medium and terminal |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |