CN112333177A - Scanning report generation method and device - Google Patents
Scanning report generation method and device Download PDFInfo
- Publication number
- CN112333177A CN112333177A CN202011180579.6A CN202011180579A CN112333177A CN 112333177 A CN112333177 A CN 112333177A CN 202011180579 A CN202011180579 A CN 202011180579A CN 112333177 A CN112333177 A CN 112333177A
- Authority
- CN
- China
- Prior art keywords
- scanning
- message queue
- report
- generating
- scan
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 61
- 238000012545 processing Methods 0.000 claims abstract description 35
- 230000011218 segmentation Effects 0.000 claims description 6
- 238000012544 monitoring process Methods 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 16
- 230000008569 process Effects 0.000 description 12
- 238000005516 engineering process Methods 0.000 description 5
- 238000004891 communication Methods 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 4
- 238000009825 accumulation Methods 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 239000013307 optical fiber Substances 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000008878 coupling Effects 0.000 description 1
- 238000010168 coupling process Methods 0.000 description 1
- 238000005859 coupling reaction Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000003252 repetitive effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000004083 survival effect Effects 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1433—Vulnerability analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/50—Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems
- G06F21/57—Certifying or maintaining trusted computer platforms, e.g. secure boots or power-downs, version controls, system software checks, secure updates or assessing vulnerabilities
- G06F21/577—Assessing vulnerabilities and evaluating computer system security
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/0631—Management of faults, events, alarms or notifications using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/069—Management of faults, events, alarms or notifications using logs of notifications; Post-processing of notifications
Abstract
The disclosure relates to a scan report generation method, a scan report generation device, an electronic device and a computer readable medium. The method can be used for a vulnerability scanning device, and comprises the following steps: generating a plurality of segmented execution tasks according to the IP address to be subjected to vulnerability scanning; processing the plurality of segmented execution tasks based on the thread pool, and sequentially generating a plurality of single IP scanning records; the producer of the message queue sequentially sends each single IP scanning record in the plurality of single IP scanning records to the message queue; and the consumers of the message queue monitor the message queue to acquire the single IP scanning record so as to generate a scanning report. The scanning report generation method, the scanning report generation device, the electronic equipment and the computer readable medium can asynchronously execute the vulnerability scanning task and generate the report task, improve the processing efficiency and save the overall processing time of the vulnerability scanning task.
Description
Technical Field
The present disclosure relates to the field of computer information processing, and in particular, to a scan report generation method, an apparatus, an electronic device, and a computer-readable medium.
Background
With the rapid development of computer technology, various types of software and electronic devices are all in a hundred flowers, and the number of known bugs is rapidly increasing. Various electronic devices are not separated from various industries, however, the network information security situation is increasingly severe, and various forms of attacks and secret stealing behaviors are endless, which not only pose a serious threat to personal information and property, but also to various fields of national economy, and promote the development of the security industry.
Most businesses, organizations, and institutional offices or production require computers, video input devices, network devices, internet of things devices, and the like. The maintenance and management of the equipment needs to be performed by a certain amount of manpower. Along with the enlargement of the equipment scale, the maintenance difficulty is increased, and at the moment, the vulnerability scanning and asset management device can better manage the equipment and find potential safety problems.
The vulnerability scanning device generates vulnerability reports and an integral report according to the IP address of each device after scanning, and the reports can visually and specifically show vulnerability information, device information and other information of the devices. When the scanned assets are too many, the number of reports is also many, and the time required for report generation grows linearly with the number of scanned IPs.
The above information disclosed in this background section is only for enhancement of understanding of the background of the disclosure and therefore it may contain information that does not constitute prior art that is already known to a person of ordinary skill in the art.
Disclosure of Invention
In view of this, the present disclosure provides a scan report generation method, an apparatus, an electronic device, and a computer readable medium, which can asynchronously execute a bug scan task and generate a report task, improve processing efficiency, and save the overall processing time of the bug scan task.
Additional features and advantages of the disclosure will be set forth in the detailed description which follows, or in part will be obvious from the description, or may be learned by practice of the disclosure.
According to an aspect of the present disclosure, a method for generating a scan report is provided, which may be used for a vulnerability scanning apparatus, and the method includes: generating a plurality of segmented execution tasks according to the IP address to be subjected to vulnerability scanning; processing the plurality of segmented execution tasks based on the thread pool, and sequentially generating a plurality of single IP scanning records; the producer of the message queue sequentially sends each single IP scanning record in the plurality of single IP scanning records to the message queue; and the consumers of the message queue monitor the message queue to acquire the single IP scanning record so as to generate a scanning report.
In an exemplary embodiment of the present disclosure, generating a plurality of segment execution tasks according to an IP address to be vulnerability scanned includes: acquiring the range of an IP address to be subjected to vulnerability scanning, which is set by a user; splitting all IP addresses according to the number of the IP addresses in the range of the IP addresses to generate a plurality of IP address sets; and establishing a segmented execution task for each IP address set.
In an exemplary embodiment of the disclosure, sequentially generating a plurality of single IP scan records based on the thread pool processing the plurality of segmented execution tasks comprises: adding the plurality of segmented execution tasks to the thread pool; the thread pool calls and executes part or all of segmented execution tasks simultaneously; and after the execution of the partial or all segment execution tasks is finished, the thread pool continues to execute other segment execution tasks which are not finished.
In an exemplary embodiment of the disclosure, the processing the plurality of segmented execution tasks based on the thread pool, and the sequentially generating a plurality of single IP scan records, includes: the segmentation execution task obtains a corresponding IP address set; the segmentation execution task executes vulnerability scanning on the IP addresses in the IP address set; and after the scanning is finished, generating a plurality of single IP scanning records according to the execution result.
In an exemplary embodiment of the present disclosure, generating a plurality of single IP scan records according to an execution result includes: and after each segmented execution task is executed by the thread pool, generating a single IP scanning record according to the execution result of the segmented execution task.
In an exemplary embodiment of the disclosure, the sending, by a producer of a message queue, each of the plurality of single IP scan records to the message queue in turn includes: and after the producer of the message queue sends the single IP scanning record to the message queue, finishing the segmented execution task corresponding to the single IP scanning record in the thread pool.
In an exemplary embodiment of the present disclosure, a consumer of a message queue listens to the message queue for the single IP scan record to generate a scan report, including: the consumers of the message queue monitor the message queue to obtain the single IP scanning record; generating a single IP report according to the single IP scanning record; generating the scan report from a plurality of single IP reports.
In an exemplary embodiment of the present disclosure, generating the scan report from a plurality of single IP reports includes: the consumers of the message queue monitor the single IP report generated by the message queue; after all of the single IP reports are generated, the scan report is generated from all of the single IP reports.
In an exemplary embodiment of the present disclosure, generating the scan report from a plurality of single IP reports includes: the consumers of the message queue monitor the message queue to obtain the single IP scanning record; and when the last single IP scanning record is obtained, generating the scanning report according to all the single IP reports.
According to an aspect of the present disclosure, a scan report generating apparatus is provided, which may be used for a vulnerability scanning apparatus, the apparatus including: the task module is used for generating a plurality of segmented execution tasks according to the IP address to be subjected to vulnerability scanning; the execution module is used for processing the plurality of segmented execution tasks based on the thread pool and sequentially generating a plurality of single IP scanning records; the queue module is used for sequentially sending each single IP scanning record in the plurality of single IP scanning records to the message queue by a producer of the message queue; and the reporting module is used for monitoring the message queue by the consumer of the message queue to acquire the single IP scanning record so as to generate a scanning report.
According to an aspect of the present disclosure, an electronic device is provided, the electronic device including: one or more processors; storage means for storing one or more programs; when executed by one or more processors, cause the one or more processors to implement a method as above.
According to an aspect of the disclosure, a computer-readable medium is proposed, on which a computer program is stored, which program, when being executed by a processor, carries out the method as above.
According to the scanning report generation method, the scanning report generation device, the electronic equipment and the computer readable medium, a plurality of segmented execution tasks are generated according to the IP address to be subjected to vulnerability scanning; processing the plurality of segmented execution tasks based on the thread pool, and sequentially generating a plurality of single IP scanning records; the producer of the message queue sequentially sends each single IP scanning record in the plurality of single IP scanning records to the message queue; and the consumers of the message queue monitor the message queue to acquire the single IP scanning record so as to generate the scanning report, and can asynchronously execute the vulnerability scanning task and generate the report task, thereby improving the processing efficiency and saving the overall processing time of the vulnerability scanning task.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The above and other objects, features and advantages of the present disclosure will become more apparent by describing in detail exemplary embodiments thereof with reference to the attached drawings. The drawings described below are merely some embodiments of the present disclosure, and other drawings may be derived from those drawings by those of ordinary skill in the art without inventive effort.
Fig. 1 is a schematic diagram of a scan report generation method in the prior art.
Fig. 2 is a schematic diagram of a scan report generation method in the prior art.
Fig. 3 is a schematic diagram of a scan report generation method in the prior art.
Fig. 4 is a system block diagram illustrating a scan report generation method and apparatus according to an example embodiment.
Fig. 5 is a flow diagram illustrating a scan report generation method in accordance with an example embodiment.
Fig. 6 is a schematic diagram illustrating a scan report generation method according to another exemplary embodiment.
Fig. 7 is a schematic diagram illustrating a scan report generation method according to another exemplary embodiment.
Fig. 8 is a block diagram illustrating a scan report generation apparatus in accordance with an example embodiment.
FIG. 9 is a block diagram illustrating an electronic device in accordance with an example embodiment.
FIG. 10 is a block diagram illustrating a computer-readable medium in accordance with an example embodiment.
Detailed Description
Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art. The same reference numerals denote the same or similar parts in the drawings, and thus, a repetitive description thereof will be omitted.
Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the disclosure. One skilled in the relevant art will recognize, however, that the subject matter of the present disclosure can be practiced without one or more of the specific details, or with other methods, components, devices, steps, and so forth. In other instances, well-known methods, devices, implementations, or operations have not been shown or described in detail to avoid obscuring aspects of the disclosure.
The block diagrams shown in the figures are functional entities only and do not necessarily correspond to physically separate entities. I.e. these functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor means and/or microcontroller means.
The flow charts shown in the drawings are merely illustrative and do not necessarily include all of the contents and operations/steps, nor do they necessarily have to be performed in the order described. For example, some operations/steps may be decomposed, and some operations/steps may be combined or partially combined, so that the actual execution sequence may be changed according to the actual situation.
It will be understood that, although the terms first, second, third, etc. may be used herein to describe various components, these components should not be limited by these terms. These terms are used to distinguish one element from another. Thus, a first component discussed below may be termed a second component without departing from the teachings of the disclosed concept. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.
It is to be understood by those skilled in the art that the drawings are merely schematic representations of exemplary embodiments, and that the blocks or processes shown in the drawings are not necessarily required to practice the present disclosure and are, therefore, not intended to limit the scope of the present disclosure.
The inventor of the present disclosure finds that, in the prior art, the processing flow of the vulnerability scanning apparatus includes the following steps:
1, setting a scanning IP range by a user to create a vulnerability scanning task, then executing the task by a vulnerability scanning device, scanning all the probing IP addresses, then generating a corresponding single IP report according to the result of each IP scanning, and generating a total report after the completion. The process is shown in figure 1. If the user creates a new scanning task, wherein the number of detected active IPs is n, the average generation time of each single IP report is t1, and the total generation time of the report is t2, then the time occupied by the report is: tot ═ n × t1+ t 2.
2, when a user creates a new scanning task, only scanning the loophole, not immediately generating a report, and when the user needs to check the report, generating the report again. The process is shown in figure 2. The time taken to generate the report alone at this time is the same as in case 1.
The inventor of the present disclosure finds that, in the two modes, the whole process is synchronous, and needs to execute another generation report, at this time, the generation report is single-threaded, and the system only processes the generation report without performing other operations, which is inefficient. Furthermore, as the scan range increases (the number of probing IPs n increases), the generation time n × t1 for all single IP reports increases linearly.
And 3, setting a new vulnerability scanning task in a scanning IP range by a user, splitting according to the scanning IP number, executing the task by the vulnerability scanning device according to the split segments, simultaneously scanning a plurality of segments, namely running in a thread pool with a fixed thread number, adding all the segments into the thread pool, starting a thread of the next segment after running the thread of one segment, and fixing the running thread number. The task in each thread that scans a segment generates a single IP report within that segment immediately, and the aggregate report is generated after all segments have been executed. The process is shown in figure 3.
The inventor of the present disclosure finds that, unlike technologies 1 and 2, technology 3 scans the IPs of multiple segments simultaneously, and generates a single IP report of a segment immediately after scanning one segment, and each split segment is executed by adding to a thread pool, that is, a single IP report is generated by a single thread originally, and multiple threads can generate a single IP report now, which is more efficient than technology 1. If the user creates a new scanning task, wherein the number of detected active IPs is n, the maximum number of executing threads is m, the number of executed IPs of each thread is n/m, the average generation time of each single IP report is t1, and the total generation time of the report is t2, then the time occupied individually when generating the report is about: ttotal ≈ n/m × t1+ t2
In technique 3, the whole process is multi-threaded, but the scannable device still scans each segment and generates a single IP report synchronously, and the generation of the single IP report still needs to be processed after each thread scan. As the scan range increases (the number of probing IPs n increases), the generation time n/m x t1 for all single IP reports increases linearly while multithreading is performed to reduce the increasing coefficient to n/m.
The scanning report generation method disclosed by the invention is used for solving the problem that the background report generation time is increased linearly with the scanning IP number. The technical content of the present disclosure is described in detail below with the aid of specific embodiments.
Fig. 4 is a system block diagram illustrating a scan report generation method, apparatus, electronic device, and computer readable medium according to an example embodiment.
As shown in fig. 4, the system architecture 40 may include terminal devices 401, 402, 403, a network 404, and a vulnerability scanning apparatus 405. The network 404 is used to provide a medium of communication links between the terminal devices 401, 402, 403 and the vulnerability scanning apparatus 405. Network 404 may include various types of connections, such as wire, wireless communication links, or fiber optic cables, to name a few.
The user can use the vulnerability scanning device 405 to scan vulnerabilities existing in the terminal equipment 401, 402, 403 and generate scanning reports. The terminal devices 401, 402, 403 may have various communication client applications installed thereon, such as shopping applications, web browser applications, search applications, instant messaging tools, mailbox clients, social platform software, and the like.
The terminal devices 401, 402, 403 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop portable computers, desktop computers, and the like.
The vulnerability scanning apparatus 405 may be a physical server, or may be composed of a plurality of servers, for example, it should be noted that the scanning report generating method provided by the embodiment of the present disclosure may be executed by the vulnerability scanning apparatus 405, and accordingly, the scanning report generating apparatus may be disposed in the vulnerability scanning apparatus 405.
Fig. 5 is a flow diagram illustrating a scan report generation method in accordance with an example embodiment. The scan report generating method 50 includes at least steps S502 to S508.
As shown in fig. 5, in S502, a plurality of segment execution tasks are generated according to the IP address to be vulnerability scanned. The method specifically comprises the following steps: acquiring the range of an IP address to be subjected to vulnerability scanning, which is set by a user; splitting all IP addresses according to the number of the IP addresses in the range of the IP addresses to generate a plurality of IP address sets; and establishing a segmented execution task for each IP address set.
In S504, a plurality of single IP scan records are sequentially generated based on the thread pool processing the plurality of segment execution tasks. The plurality of segmented execution tasks may be added to the thread pool, for example; the thread pool calls and executes part or all of segmented execution tasks simultaneously; and after the execution of the partial or all segment execution tasks is finished, the thread pool continues to execute other segment execution tasks which are not finished.
Processing the plurality of segmented execution tasks based on the thread pool, and sequentially generating a plurality of single IP scanning records, wherein the method comprises the following steps: the segmentation execution task obtains a corresponding IP address set; the segmentation execution task executes vulnerability scanning on the IP addresses in the IP address set; and after the scanning is finished, generating a plurality of single IP scanning records according to the execution result.
More specifically, generating a plurality of single IP scan records according to the execution result includes: and after each segmented execution task is executed by the thread pool, generating a single IP scanning record according to the execution result of the segmented execution task.
In S506, the producer of the message queue sequentially sends each of the plurality of single IP scan records to the message queue. The segmented execution task corresponding to the single IP scan record can be ended in the thread pool after the producer of the message queue sends the single IP scan record to the message queue.
Wherein, Message Queue (Message Queue): a message queue is a container in which messages are stored. The producer puts the message data into the message queue and the consumer takes the message data out of the message queue. The message queue is mainly used for improving the system performance and peak clipping and reducing the system coupling through asynchronous processing. Currently, the more used message queues are ActiveMQ, RabbitMQ, Kafka, RockettMQ. The two operations of the missed scan device scanning task and generating the report may be decoupled from being performed asynchronously by the message queue.
In a specific embodiment, the message queue takes activeMQ as an example, and has a publish-subscribe mode topic and a peer-to-peer mode queue, and using the peer-to-peer mode queue, the producer sends a message to the message queue, and the consumer listens to the message queue to obtain the message, and consumes the message, and each message is consumed only once in the peer-to-peer mode. There may be multiple consumers listening to the same message queue.
In S508, the consumers of the message queue listen to the message queue to obtain the single IP scan record to generate a scan report. For example, a consumer of a message queue listens to the message queue to obtain the single IP scan record; generating a single IP report according to the single IP scanning record; generating the scan report from a plurality of single IP reports.
In one embodiment, generating the scan report from a plurality of single IP reports comprises: the consumers of the message queue monitor the single IP report generated by the message queue; after all of the single IP reports are generated, the scan report is generated from all of the single IP reports.
In one embodiment, generating the scan report from a plurality of single IP reports comprises: the consumers of the message queue monitor the message queue to obtain the single IP scanning record; and when the last single IP scanning record is obtained, generating the scanning report according to all the single IP reports.
According to the scanning report generation method, a plurality of segmented execution tasks are generated according to the IP address to be subjected to vulnerability scanning; processing the plurality of segmented execution tasks based on the thread pool, and sequentially generating a plurality of single IP scanning records; the producer of the message queue sequentially sends each single IP scanning record in the plurality of single IP scanning records to the message queue; and the consumers of the message queue monitor the message queue to acquire the single IP scanning record so as to generate the scanning report, and can asynchronously execute the vulnerability scanning task and generate the report task, thereby improving the processing efficiency and saving the overall processing time of the vulnerability scanning task.
It should be clearly understood that this disclosure describes how to make and use particular examples, but the principles of this disclosure are not limited to any details of these examples. Rather, these principles can be applied to many other embodiments based on the teachings of the present disclosure.
Fig. 6 and 7 are schematic diagrams illustrating a scan report generation method according to an exemplary embodiment. Fig. 6 and 7 show two specific embodiments for illustrating "a consumer of a message queue listens to the message queue to obtain the single IP scan record to generate a scan report".
In the method shown in fig. 6 and 7, a user sets a new scanning task for scanning an IP range, then splits according to the number of IPs, places the split segments into a fixed number of thread pools for execution, each segment creates a new thread, the thread executes the task for scanning the segment, after the execution is completed, a producer sends a message for recording the probing IP in the segment to a message queue, then the thread is completed, and the segment execution is completed. The unexecuted segments will be executed according to the above procedure, but the number of threads in the thread pool cannot exceed a fixed size. And meanwhile, the consumer monitors the message queue, acquires the message, reads the IP recorded in the message, generates a single IP report, judges whether all the segments of the task in the thread pool are executed completely, and generates a total report if the execution is finished.
In the embodiment shown in fig. 6, a single IP report and a total report are generated after all tasks have been performed. After each thread scans the segment, the time for the producer to send the message of the segment survival IP to the message queue is very short and can be ignored, a plurality of consumers can be set, and the consumers cannot influence each other. And the scanning time for scanning each segment is far longer than the time for generating a single IP report, namely, compared with the time for generating a report, the time for scanning the segment is more spent, the speed of adding the message by the producer cannot keep up with the speed of consuming by the consumer, and the phenomenon of message accumulation does not exist in the messages in the message queue as long as at least one consumer exists.
If the user creates a new scanning task, where the number of active IP is n, the granularity of split IP is y, that is, the number of each segmented IP is y, the average generation time of each single IP report is t1, and the total generation time of the report is t2, then the time occupied separately when generating the report is about:
ttotal ≈ y t1+ t 2. The message queue may assume that only the last message sent at this time after the task execution is complete.
In the embodiment shown in FIG. 7, when a user clicks to generate a report, the IP is split, messages are sent to the message queue, the consumer receives the messages, a single IP report is generated, and a total report is generated when the last message is consumed. If the user creates a new scanning task, wherein the number of detected active IPs is n, the average generation time of each single IP report is t1, the generation time of the total report is t2, and the number of consumers is c, then the time taken by the users to generate the reports is about:
ttotal ≈ n/c × t1+ t 2. The influence is small when the resolution granularity is small and n is large.
In the scanning report generation method disclosed by the disclosure, the two steps of the report generation step and the scanning task are independent and asynchronous, and the background can simultaneously scan the task and generate the report without interference.
In the case shown in fig. 6, when the number of scans is large, the background generates reports, the time occupied by the background is independent of the number of scans and the split granularity, and when the split granularity is constant, the total time occupied by generating reports does not fluctuate too much.
In the case shown in fig. 7, the efficiency is improved when the consumer is greater than 1 compared to other technologies, and is almost equal when 1 is equal.
In both of the above two ways, when a scan task is created and a report is generated immediately, the complexity of the individual occupation time of the background generated report is O (1)
Those skilled in the art will appreciate that all or part of the steps implementing the above embodiments are implemented as computer programs executed by a CPU. When executed by the CPU, performs the functions defined by the above-described methods provided by the present disclosure. The program may be stored in a computer readable storage medium, which may be a read-only memory, a magnetic or optical disk, or the like.
Furthermore, it should be noted that the above-mentioned figures are only schematic illustrations of the processes involved in the methods according to exemplary embodiments of the present disclosure, and are not intended to be limiting. It will be readily understood that the processes shown in the above figures are not intended to indicate or limit the chronological order of the processes. In addition, it is also readily understood that these processes may be performed synchronously or asynchronously, e.g., in multiple modules.
The following are embodiments of the disclosed apparatus that may be used to perform embodiments of the disclosed methods. For details not disclosed in the embodiments of the apparatus of the present disclosure, refer to the embodiments of the method of the present disclosure.
Fig. 8 is a block diagram illustrating a scan report generation apparatus in accordance with an example embodiment. As shown in fig. 8, the scan report generating apparatus 80 includes: a task module 802, an execution module 804, a queue module 806, and a report module 808.
The task module 802 is configured to generate a plurality of segment execution tasks according to the IP address to be subjected to vulnerability scanning;
the execution module 804 is configured to process the multiple segment execution tasks based on the thread pool, and sequentially generate multiple single IP scan records;
the queue module 806 is configured to send each of the multiple single IP scanning records to the message queue in turn by a producer of the message queue;
the reporting module 808 is configured to monitor the message queue for the single IP scan record by the consumer of the message queue to generate a scan report.
According to the scanning report generating device disclosed by the invention, a plurality of segmented execution tasks are generated according to the IP address to be subjected to vulnerability scanning; processing the plurality of segmented execution tasks based on the thread pool, and sequentially generating a plurality of single IP scanning records; the producer of the message queue sequentially sends each single IP scanning record in the plurality of single IP scanning records to the message queue; and the consumers of the message queue monitor the message queue to acquire the single IP scanning record so as to generate the scanning report, and can asynchronously execute the vulnerability scanning task and generate the report task, thereby improving the processing efficiency and saving the overall processing time of the vulnerability scanning task.
FIG. 9 is a block diagram illustrating an electronic device in accordance with an example embodiment.
An electronic device 900 according to this embodiment of the disclosure is described below with reference to fig. 9. The electronic device 900 shown in fig. 9 is only an example and should not bring any limitations to the functionality or scope of use of the embodiments of the present disclosure.
As shown in fig. 9, the electronic device 900 is embodied in the form of a general purpose computing device. Components of electronic device 900 may include, but are not limited to: at least one processing unit 910, at least one storage unit 920, a bus 930 connecting different system components (including the storage unit 920 and the processing unit 910), a display unit 940, and the like.
Wherein the storage unit stores program code that can be executed by the processing unit 910 such that the processing unit 910 performs the steps according to various exemplary embodiments of the present disclosure described in this specification. For example, the processing unit 910 may perform the steps as shown in fig. 5.
The storage unit 920 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM)9201 and/or a cache memory unit 9202, and may further include a read only memory unit (ROM) 9203.
The memory unit 920 may also include a program/utility 9204 having a set (at least one) of program modules 9205, such program modules 9205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may comprise an implementation of a network environment.
The electronic device 900 may also communicate with one or more external devices 900' (e.g., keyboard, pointing device, bluetooth device, etc.), such that a user can communicate with devices with which the electronic device 900 interacts, and/or any device (e.g., router, modem, etc.) with which the electronic device 900 can communicate with one or more other computing devices. Such communication may occur via input/output (I/O) interface 950. Also, the electronic device 900 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN) and/or a public network, such as the Internet) via the network adapter 960. The network adapter 960 may communicate with other modules of the electronic device 900 via the bus 930. It should be appreciated that although not shown, other hardware and/or software modules may be used in conjunction with the electronic device 900, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.
Colloquially, the present disclosure decouples the two operations of the missed scan device scanning task and generating the report from being performed asynchronously using the message queue. The message queue takes activeMQ as an example and has a publish-subscribe mode topic and a peer-to-peer mode queue, the producer sends messages to the message queue by using the peer-to-peer mode queue, and the consumer listens the message queue to obtain messages and consumes the messages, wherein each message is consumed only once in the peer-to-peer mode. There may be multiple consumers listening to the same message queue. In the task execution mode, a new scanning task is created by setting a scanning ip range through a user, then the scanning task is split according to the number of ip, split segments are put into thread pools with fixed number for execution, each segment can create a thread, the thread executes the task of scanning the segment, after the execution is finished, a producer sends a message for recording the probing active ip in the segment to a message queue, then the thread is finished, and the segment execution is finished. The unexecuted segments will be executed according to the above procedure, but the number of threads in the thread pool cannot exceed a fixed size. And meanwhile, the consumer monitors the message queue, acquires the message, reads the ip recorded in the message, generates a single ip report, judges whether all the segments of the task in the thread pool are executed, and generates a total report if the execution is finished. In the first case, a new scan task is created, and a single ip report and a total report are generated after the new scan task is executed. After each thread scans the segment, the time for the producer to send the live ip message of the segment to the message queue is short and can be ignored, a plurality of consumers can be set, and the consumers cannot influence each other. And the scanning time for scanning each segment is far longer than the time for generating a single IP report, namely, compared with the time for generating a report, the time for scanning the segment is more spent, the speed of adding the message by the producer cannot keep up with the speed of consuming by the consumer, and the phenomenon of message accumulation does not exist in the messages in the message queue as long as at least one consumer exists. If a user creates a new scanning task, wherein the number of detected ip is n, the granularity of split ip is y, that is, the number of each segment ip is y, the average generation time of each single ip report is t1, and the total generation time of the report is t2, then the time occupied separately when generating the report is about: ttotal ≈ y t1+ t 2. The message queue may assume that only the last message sent at this time after the task execution is complete. In the second case, the report is not generated after the task is executed, and the user clicks the generated report to generate the report in the background. When a user clicks to generate a report, the ip is split, the message is sent to a message queue, a consumer receives the message, a single ip report is generated, and a total report is generated when the last message is consumed. If the user creates a new scanning task, wherein the number of detected ip is n, the average generation time of each single ip report is t1, the total generation time of the reports is t2, and the number of consumers is c, then the time occupied by the report is about: ttotal ≈ n/c × t1+ t 2. The influence is small when the resolution granularity is small and n is large.
Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein may be implemented by software, or by software in combination with necessary hardware. Therefore, as shown in fig. 10, the technical solution according to the embodiment of the present disclosure may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (which may be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which may be a personal computer, a server, or a network device, etc.) to execute the above method according to the embodiment of the present disclosure.
The software product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
The computer readable storage medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A readable storage medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Program code for carrying out operations for the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., through the internet using an internet service provider).
The computer readable medium carries one or more programs which, when executed by a device, cause the computer readable medium to perform the functions of: generating a plurality of segmented execution tasks according to the IP address to be subjected to vulnerability scanning; processing the plurality of segmented execution tasks based on the thread pool, and sequentially generating a plurality of single IP scanning records; the producer of the message queue sequentially sends each single IP scanning record in the plurality of single IP scanning records to the message queue; and the consumers of the message queue monitor the message queue to acquire the single IP scanning record so as to generate a scanning report.
Those skilled in the art will appreciate that the modules described above may be distributed in the apparatus according to the description of the embodiments, or may be modified accordingly in one or more apparatuses unique from the embodiments. The modules of the above embodiments may be combined into one module, or further split into multiple sub-modules.
Exemplary embodiments of the present disclosure are specifically illustrated and described above. It is to be understood that the present disclosure is not limited to the precise arrangements, instrumentalities, or instrumentalities described herein; on the contrary, the disclosure is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.
Claims (10)
1. A scanning report generation method can be used for a vulnerability scanning device, and is characterized by comprising the following steps:
generating a plurality of segmented execution tasks according to the IP address to be subjected to vulnerability scanning;
processing the plurality of segmented execution tasks based on the thread pool, and sequentially generating a plurality of single IP scanning records;
the producer of the message queue sequentially sends each single IP scanning record in the plurality of single IP scanning records to the message queue;
and the consumers of the message queue monitor the message queue to acquire the single IP scanning record so as to generate a scanning report.
2. The method of claim 1, wherein generating a plurality of segmented execution tasks according to the IP address to be vulnerability scanned comprises:
acquiring the range of an IP address to be subjected to vulnerability scanning, which is set by a user;
splitting all IP addresses according to the number of the IP addresses in the range of the IP addresses to generate a plurality of IP address sets;
and establishing a segmented execution task for each IP address set.
3. The method of claim 1, wherein sequentially generating a plurality of single IP scan records based on the thread pool processing the plurality of segmented execution tasks comprises:
adding the plurality of segmented execution tasks to the thread pool;
the thread pool calls and executes part or all of segmented execution tasks simultaneously;
and after the execution of the partial or all segment execution tasks is finished, the thread pool continues to execute other segment execution tasks which are not finished.
4. The method of claim 3, wherein sequentially generating a plurality of single IP scan records based on the thread pool processing the plurality of segmented execution tasks comprises:
the segmentation execution task obtains a corresponding IP address set;
the segmentation execution task executes vulnerability scanning on the IP addresses in the IP address set;
and after the scanning is finished, generating a plurality of single IP scanning records according to the execution result.
5. The method of claim 4, wherein generating a plurality of single IP scan records based on the execution comprises:
and after each segmented execution task is executed by the thread pool, generating a single IP scanning record according to the execution result of the segmented execution task.
6. The method of claim 1, wherein the producer of the message queue sending each of the plurality of single IP scan records into the message queue in turn, comprises:
and after the producer of the message queue sends the single IP scanning record to the message queue, finishing the segmented execution task corresponding to the single IP scanning record in the thread pool.
7. The method of claim 1, wherein a consumer of a message queue listening to the message queue to obtain the single IP scan record to generate a scan report, comprises:
the consumers of the message queue monitor the message queue to obtain the single IP scanning record;
generating a single IP report according to the single IP scanning record;
generating the scan report from a plurality of single IP reports.
8. The method of claim 7, wherein generating the scan report from a plurality of single IP reports comprises:
the consumers of the message queue monitor the single IP report generated by the message queue;
after all of the single IP reports are generated, the scan report is generated from all of the single IP reports.
9. The method of claim 7, wherein generating the scan report from a plurality of single IP reports comprises:
the consumers of the message queue monitor the message queue to obtain the single IP scanning record;
and when the last single IP scanning record is obtained, generating the scanning report according to all the single IP reports.
10. A scan report generation apparatus, usable with a vulnerability scanning apparatus, comprising:
the task module is used for generating a plurality of segmented execution tasks according to the IP address to be subjected to vulnerability scanning;
the execution module is used for processing the plurality of segmented execution tasks based on the thread pool and sequentially generating a plurality of single IP scanning records;
the queue module is used for sequentially sending each single IP scanning record in the plurality of single IP scanning records to the message queue by a producer of the message queue;
and the reporting module is used for monitoring the message queue by the consumer of the message queue to acquire the single IP scanning record so as to generate a scanning report.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011180579.6A CN112333177A (en) | 2020-10-29 | 2020-10-29 | Scanning report generation method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011180579.6A CN112333177A (en) | 2020-10-29 | 2020-10-29 | Scanning report generation method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112333177A true CN112333177A (en) | 2021-02-05 |
Family
ID=74297795
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011180579.6A Pending CN112333177A (en) | 2020-10-29 | 2020-10-29 | Scanning report generation method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112333177A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113065165A (en) * | 2021-03-03 | 2021-07-02 | 腾讯科技（深圳）有限公司 | Certificate security detection method and device and storage medium |
| CN113676545A (en) * | 2021-08-25 | 2021-11-19 | 北京明朝万达科技股份有限公司 | Equipment asset scanning method, device and system |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102722417A (en) * | 2012-06-07 | 2012-10-10 | 腾讯科技（深圳）有限公司 | Distribution method and device for scan task |
| CN104811415A (en) * | 2014-01-23 | 2015-07-29 | 杭州迪普科技有限公司 | Method and device for balancing loads of distributed scanning system |
| US9800603B1 (en) * | 2014-03-31 | 2017-10-24 | Nopsec Inc. | Self-replicating distributed vulnerability management agent apparatuses, methods, and systems |
| CN109271265A (en) * | 2018-09-29 | 2019-01-25 | 平安科技（深圳）有限公司 | Request processing method, device, equipment and storage medium based on message queue |
| CN111104188A (en) * | 2019-11-11 | 2020-05-05 | 中盈优创资讯科技有限公司 | Scheduling method and device of vulnerability scanner |
| CN111353161A (en) * | 2020-03-11 | 2020-06-30 | 腾讯科技（深圳）有限公司 | Vulnerability scanning method and device |
-
2020
- 2020-10-29 CN CN202011180579.6A patent/CN112333177A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102722417A (en) * | 2012-06-07 | 2012-10-10 | 腾讯科技（深圳）有限公司 | Distribution method and device for scan task |
| CN104811415A (en) * | 2014-01-23 | 2015-07-29 | 杭州迪普科技有限公司 | Method and device for balancing loads of distributed scanning system |
| US9800603B1 (en) * | 2014-03-31 | 2017-10-24 | Nopsec Inc. | Self-replicating distributed vulnerability management agent apparatuses, methods, and systems |
| CN109271265A (en) * | 2018-09-29 | 2019-01-25 | 平安科技（深圳）有限公司 | Request processing method, device, equipment and storage medium based on message queue |
| CN111104188A (en) * | 2019-11-11 | 2020-05-05 | 中盈优创资讯科技有限公司 | Scheduling method and device of vulnerability scanner |
| CN111353161A (en) * | 2020-03-11 | 2020-06-30 | 腾讯科技（深圳）有限公司 | Vulnerability scanning method and device |
Non-Patent Citations (2)
| Title |
|---|
| 甄凯成等: "基于Netty和Kafka的物联网数据接入系统", 《计算机工程与应用》 * |
| 米昂: "基于多引擎的分布式移动APP安全扫描系统的设计", 《电子技术与软件工程》 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113065165A (en) * | 2021-03-03 | 2021-07-02 | 腾讯科技（深圳）有限公司 | Certificate security detection method and device and storage medium |
| CN113676545A (en) * | 2021-08-25 | 2021-11-19 | 北京明朝万达科技股份有限公司 | Equipment asset scanning method, device and system |
| CN113676545B (en) * | 2021-08-25 | 2024-03-19 | 北京明朝万达科技股份有限公司 | Equipment asset scanning method, device and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10771492B2 (en) | Enterprise graph method of threat detection | |
| US20180300110A1 (en) | Preserving dynamic trace purity | |
| JP2005222533A (en) | Adaptive dispatch of received messages to code using inter-positioned message modification | |
| US9870303B2 (en) | Monitoring and correlating a binary process in a distributed business transaction | |
| US20220321616A1 (en) | Intelligent screen and resource sharing during a meeting | |
| CN112333177A (en) | Scanning report generation method and device | |
| CN112347169A (en) | PHP (hypertext preprocessor) framework based user request processing method and device and electronic equipment | |
| CN111240847B (en) | Data processing method, device, medium and computing equipment | |
| US9300547B2 (en) | Modification of cloud application service levels based upon document consumption | |
| US20170147483A1 (en) | Tracking asynchronous entry points for an application | |
| CN112328564A (en) | Special resource sharing method and device and electronic equipment | |
| CN115357761A (en) | Link tracking method and device, electronic equipment and storage medium | |
| US11477215B2 (en) | Scaling a processing resource of a security information and event management system | |
| CN113205320A (en) | Service processing method and device, electronic equipment and computer readable medium | |
| CN111010453B (en) | Service request processing method, system, electronic device and computer readable medium | |
| CN116521639A (en) | Log data processing method, electronic equipment and computer readable medium | |
| US9921901B2 (en) | Alerting service desk users of business services outages | |
| CN113420302A (en) | Host vulnerability detection method and device | |
| CN112347382A (en) | Product page sharing method and device and electronic equipment | |
| CN114168607A (en) | Global serial number generation method, device, equipment, medium and product | |
| CN111950663A (en) | Display picture generation method and device and electronic equipment | |
| CN112016790A (en) | User policy distribution method and device and electronic equipment | |
| CN111178014A (en) | Method and device for processing business process | |
| CN111367889B (en) | Cross-cluster data migration method and device based on webpage interface | |
| WO2018200158A1 (en) | Mitigating absence of skill input during collaboration session |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210205 |
|
| RJ01 | Rejection of invention patent application after publication |