CN112783777B - Method and system for collecting real-time information and network traffic in android environment - Google Patents

Method and system for collecting real-time information and network traffic in android environment Download PDF

Info

Publication number
CN112783777B
CN112783777B CN202110111586.9A CN202110111586A CN112783777B CN 112783777 B CN112783777 B CN 112783777B CN 202110111586 A CN202110111586 A CN 202110111586A CN 112783777 B CN112783777 B CN 112783777B
Authority
CN
China
Prior art keywords
information
android
log
environment
application program
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110111586.9A
Other languages
Chinese (zh)
Other versions
CN112783777A (en
Inventor
陈贞翔
朱宇辉
王琳
韩冰
杨波
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
University of Jinan
Original Assignee
University of Jinan
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by University of Jinan filed Critical University of Jinan
Priority to CN202110111586.9A priority Critical patent/CN112783777B/en
Publication of CN112783777A publication Critical patent/CN112783777A/en
Application granted granted Critical
Publication of CN112783777B publication Critical patent/CN112783777B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/36Preventing errors by testing or debugging software
    • G06F11/3664Environments for testing or debugging software
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/36Preventing errors by testing or debugging software
    • G06F11/3668Software testing
    • G06F11/3672Test management
    • G06F11/3688Test management for test execution, e.g. scheduling of test suites
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/36Preventing errors by testing or debugging software
    • G06F11/3668Software testing
    • G06F11/3672Test management
    • G06F11/3692Test management for test results analysis
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02DCLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D30/00Reducing energy consumption in communication networks
    • Y02D30/50Reducing energy consumption in communication networks in wire-line communication networks, e.g. low power modes or reduced link rate

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Hardware Design (AREA)
  • Quality & Reliability (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Debugging And Monitoring (AREA)

Abstract

The invention discloses a method and a system for acquiring real-time information and network traffic in an Android environment, wherein the method and the system initialize the data acquisition environment in the Android environment where an application program to be tested runs; installing an application program to be tested in an Android environment; acquiring basic information of an Android environment; running an application program to be tested, starting to collect logs, and capturing log information; and (5) finishing acquisition, uploading log information, and storing the log information in a test result database. In the test process of acquiring the network connection behavior of the application program, the invention greatly improves the real-time performance, and can capture the information under the conditions of not contacting the original flow and not depending on the static analysis result, so that the system can more accurately track the dependence between the network connection in the Android environment and the application program or the system component to be tested in the test procedure, and the capture of the information is more convenient and feasible.

Description

Method and system for collecting real-time information and network traffic in android environment
Technical Field
The application relates to the technical field of network traffic real-time acquisition, in particular to a method and a system for acquiring real-time information and network traffic in an android environment.
Background
The statements in this section merely provide background information related to the present application and may not necessarily constitute prior art.
The software is run by using a specific software and hardware environment, and is dynamically tested and analyzed, so that the method is a key step in the software testing process. Only when the software is executed, the development team can find out the problem which can not be found by analyzing the source code in the software, and find out the compatibility problem of the software and the specific platform. Only the various runtime data of the third party application are collected for security researchers, and their specific behavior can be captured and analyzed accurately and effectively.
At present, in the field of Android application program testing, the most common method for collecting behavior information is to collect log information (i.e. locator) output by an application program through an ADB (i.e. Android Debug Bridge) tool, and then analyze the performance and behavior of the application program by using debugging symbols generated in the compiling process and a debugger. However, the two foregoing testing methods are mainly directed to application programs written and compiled by the developer themselves. For the former, since the running log of the ADB can only obtain running information from the Android level, information from the Android bottom layer component and the Linux kernel is often difficult to automatically capture in the test flow. In the latter case, the pre-compiled third party test program cannot be tested using this method, since the debug symbols must be generated during the compilation process. The traditional testing method also shows strong limitation.
For mobile applications, network behavior is often important inside, and behavior analysis for network traffic is also an important step in testing third-party Android applications. The primary task of network behavior analysis is to capture network traffic generated by Android devices. Currently, a common approach is to capture traffic at the network access point of the Android device. Because the access point cannot directly access the internal information of the Android environment, the method can only capture the flow data, and the tracking and tracing of the flow are difficult to carry out. In addition, in the current scheme, to generate summary information of network connection established by a device in a period of time, network traffic needs to be stored in its entirety. Since this scheme needs to wait for the stored content to be traversed after the capturing is finished and filter and separate the network traffic, when the key information is extracted, it not only needs to take a long time, but also lacks the capability of real-time processing. In particular, in conventional solutions, the captured and stored network traffic is the set of traffic generated by all applications and system components of the overall Android environment over a period of time, and even if each TCP flow can be separated from this set according to a network connection session, it cannot be precisely located to which component within the Android environment initiated the network connection at all, thereby adversely affecting network behavior analysis.
Disclosure of Invention
In order to solve the defects of the prior art, the application provides a method and a system for collecting real-time information and network traffic in an android environment; the Android device is controlled to perform information collection work under the condition that the Android device is connected to a computer through a USB cable, the dynamic behavior of the Android device can be acquired in a layer closer to the bottom layer of the system in the process of software testing, network connection initiated by an application program to be tested and a system component is tracked more accurately, the automation level of log collection is improved, and flexibility is improved.
In a first aspect, the present application provides a method for collecting real-time information and network traffic in an android environment;
a method for collecting real-time information and network traffic in an android environment comprises the following steps:
step (1): initializing a data acquisition environment in an Android environment in which an application program to be tested runs;
step (2): installing an application program to be tested in an Android environment;
step (3): acquiring basic information of an Android environment;
step (4): running an application program to be tested, starting to collect logs, and capturing log information;
step (5): the collection is finished, log information is uploaded and stored in a test result database;
step (6): based on the contents of the test results database, the correlation of the collected logs within the tested application is inferred.
In a second aspect, the present application provides a system for collecting real-time information and network traffic in an android environment;
a system for collecting real-time information and network traffic in an android environment, comprising:
the ADB command line driving module is used for communicating with the Android environment through an ADB protocol, and monitoring and analyzing the iptables log, a Broadcast monitoring module, an Android environment information analyzing module, a file and application program management module and a configurable task executing module. Is converted into commands that can be executed on the ADB protocol. The ADB command line driver module 100 executes all command lines that need to be executed through ADB shell connection on an Android environment, or establishes a series of long connection sessions for monitoring and collecting real-time log information;
the iptables log monitoring and analyzing module is used for operating iptables rules on an Android environment and collecting log information generated by the iptables rules; by adding the iptables rule, the iptables outputs a log message for each event for establishing network connection, and obtains endpoint information of the network connection and application program or system component information for initiating the network connection, and the network connection initiated by the application program is perceived in real time, so that the network behavior collection is simpler and more convenient;
and the Broadcast monitoring module is used for operating dumpsys commands, monitoring all Broadcast information generated in the Android system in the testing process, capturing communication information among application programs at the Android global level, and accordingly obtaining interaction relations among the application programs to be tested, the Android environment and other applications.
And the Android environment information analysis module is used for acquiring software and hardware platform information and installed software package information of the Android environment and acquiring real-time information such as a network state, a memory state, a service state and the like through operating dumpsys commands.
The file and application program management module is used for pushing the application program to be tested and auxiliary files required by the test to the Android environment and pulling output generated by the application program to be tested in a file form in the test process;
the configurable task execution module is used for executing other task scripts configured by a user;
the timer module is used for regularly calling the Broadcast monitoring module and the Android environment information analysis module, so that the Broadcast monitoring module and the Android environment information analysis module continuously refresh real-time information;
and the test result database is used for storing all files generated in the test process of the application program to be tested.
Compared with the prior art, the beneficial effects of this application are:
(1) The invention provides a technology for deploying a log output component on a firewall in an Android environment and capturing output of the log output component in real time through an ADB. In the test process of acquiring the network connection behavior of the application program, the invention greatly improves the real-time performance, and can capture the information under the conditions of not contacting the original flow and not depending on the static analysis result, so that the system can more accurately track the dependence between the network connection in the Android environment and the application program or the system component to be tested in the test procedure, and the capture of the information is more convenient and feasible.
(2) The invention provides a technology for capturing system-level broadcast events in an Android environment. The collection system broadcast is used as an effective supplement of the ADB locator, so that the mutual communication and call between the third party application program to be tested and the installed application program can be captured.
(3) The invention provides a technology for automatically capturing various information in an Android environment. The system can capture information such as network state, memory state, service state and the like according to a certain time interval, so that testers can more deeply understand interaction between an application program to be tested and an Android environment.
(4) The invention provides a system for automatically starting test and saving test results. The system can be matched with a tester, semi-automation of a test flow is realized, a developer can know various behaviors of an application program to be tested conveniently, correlation between different behaviors in time domain is deduced in all tested applications automatically, efficiency of testing and collecting logs is improved, and preliminary comprehensive audit is carried out on the behaviors of the application program.
Additional aspects of the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the application and do not constitute an undue limitation to the application.
Fig. 1 is a diagram of an overall structure according to an embodiment of the present application.
Detailed Description
It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the present application. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.
It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments in accordance with the present application. As used herein, unless the context clearly indicates otherwise, the singular forms also are intended to include the plural forms, and furthermore, it is to be understood that the terms "comprises" and "comprising" and any variations thereof are intended to cover non-exclusive inclusions, such as, for example, processes, methods, systems, products or devices that comprise a series of steps or units, are not necessarily limited to those steps or units that are expressly listed, but may include other steps or units that are not expressly listed or inherent to such processes, methods, products or devices.
Embodiments of the invention and features of the embodiments may be combined with each other without conflict.
Example 1
The embodiment provides a method for collecting real-time information and network traffic in an android environment;
a method for collecting real-time information and network traffic in an android environment comprises the following steps:
step (1): initializing a data acquisition environment in an Android environment in which an application program to be tested runs;
step (2): installing an application program to be tested in an Android environment;
step (3): acquiring basic information of an Android environment;
step (4): running an application program to be tested, starting to collect logs, and capturing log information;
step (5): the collection is finished, log information is uploaded and stored in a test result database;
step (6): based on the contents of the test results database, the correlation of the collected logs within the tested application is inferred.
As one or more embodiments, the step (1): initializing a data acquisition environment in an Android environment in which an application program to be tested runs; the method specifically comprises the following steps:
step (1-1): pushing an executable file for capturing information to an Android environment;
step (1-2): assigning executable rights to the executable file;
step (1-3): establishing connection with an Android shell environment;
step (1-4): firewall rules are created.
In order to record information of an application program for establishing a connection network in real time in an Android environment, the system utilizes a LOG (LOG) function in a firewall iptables built in a Linux kernel. When an application program or a system module in the Android environment establishes a network connection, the iptables firewall with the log function can output a piece of log information. The log information includes information such as the destination port, destination address, source port, source address, UID of the transmitting user, and time stamp of the connection. The present system uses the UID field in the log to match the application program that created the network connection.
The system in the step (1-1) sends the required executable files to the file system of the Android device by using the "adb push < source file > < target file >" through the "adb" tool in the Android SDK. Executable files include, but are not limited to, tcpdump for capturing traffic.
Illustratively, step (1-2): executable rights are given to the executable files pushed in step (1-1) using the "chmod+x < target file >" command within the Android environment.
Illustratively, steps (1-3): and establishing connection with the Android shell environment by using an adb tool in the Android SDK.
Illustratively, in the shell connection created in the step (1-4), a chain named NEWCONN_LOG is created in the nat table of iptables using the "iptables-N NEWCONN_LOG" command, and a command for writing a LOG to the Linux kernel module is created using the "iptables-A NEWCONN_LOG-j LOG- -LOG-uid- -LOG-prefix < user-specified prefix >".
As one or more embodiments, the step (3): acquiring basic information of an Android environment; the method comprises the following specific steps:
step (3-1): extracting CPU information of an Android environment in the Android shell environment;
step (3-2): extracting an SDK version and a hardware platform model of an Android environment in the Android shell environment;
step (3-3): extracting information of external equipment and a driving version of Android equipment in an Android shell environment;
step (3-4): and acquiring the packet names and UIDs corresponding to all the current installed application programs to be tested.
Illustratively, step (3-1): acquiring information in a "/proc/CPU info" file from the Android shell, and extracting information of a CPU architecture of a target Android environment from the information;
illustratively, step (3-2): and reading a "/system/build.prop" file in the Android shell to acquire attribute information related to the software and hardware platform, such as the SDK version, the hardware platform model and the like of the target Android environment.
Illustratively, step (3-3): and acquiring a directory structure under the "/sys" directory in the Android shell, and extracting information related to the system equipment and the driver of the Android equipment from the directory structure.
Illustratively, step (3-4): and acquiring the packet names and UIDs corresponding to all the current installed application programs. Running "cmd package list packages-U" in the shell connection created in step (1-3).
As one or more embodiments, the step (4): running an application program to be tested, starting to collect logs, and capturing log information; the method comprises the following specific steps:
step (4-1): activating firewall rules;
step (4-2): extracting a network state, a memory state and a service state from an execution result of the dumpsys command by using shell connection according to a set time interval, comparing the network state, the memory state and the service state with similar information obtained in the previous time, and extracting change information of the network state, the memory state and the service state;
step (4-3): monitoring a Linux kernel log by using shell connection;
step (4-4): monitoring a connection log generated by the firewall;
step (4-5): executing the executable file defined by the user by using shell connection, and starting to capture the traffic;
step (4-6): and starting the application program to be tested.
Illustratively, step (4-1): the created firewall rules are activated.
In the shell connection created in the step (1-3), using two commands of "iptables-I output-p TCP-m conntrack-ctstate NEW [ -m wner-UID-wner < UID > ] -j NEWCONN_LOG" and "iptables-I output-p UDP-m conntrack-ctstate NEW [ -m wner-UID-wner < UID > ] -j NEWCONN_LOG", the handshake messages of the newly built TCP connection and UDP connection in the system are guided to the firewall rules created in the step (1-4), so that the iptables firewall can output LOGs. If the connection information from all the application programs in the Android environment is collected, "[ -m wner-UID-wner < UID > ]" is omitted, if the network connection information established by a specific application program or system component is collected only, it cannot be omitted, and "< UID >" needs to be replaced with the UID value corresponding to the application program to be tested obtained in step (3-4).
Illustratively, step (4-2): dumpsys is monitored. Executing a dumpsys command at set time intervals using the shell connection created in step (1-3). And extracting information such as network state, memory state, service state and the like from the execution result. And comparing the network state, the memory state and the service state information obtained at this time with response information obtained by executing the dumpsys command last time, and searching out the changed attribute value.
Illustratively, step (4-3): the Linux kernel log is monitored. And (3) executing 'tail-f/proc/kmsg' by using the shell connection created in the step (1-3), and acquiring a kernel log.
Illustratively, step (4-4): the connection log generated by the system firewall is monitored. In step (1-4) and step (4-1), a series of firewall rules are added. The log information generated by these rules can be obtained in "/proc/kmsg". The connection log is filtered from the kernel message using the shell connection execution "tail-f/proc/kmsg|grep < user specified prefix >" created in step (1-3).
Illustratively, step (4-5): starting tcpdump by using the shell connection created in the step (1-3), and starting to capture traffic; and starts the executable file designated by the user and used for capturing information, and starts capturing other information which the user wants to capture. Here, the system needs to select an appropriate executable file version for the current Android environment by using the CPU architecture information acquired in step (3-1).
Illustratively, steps (4-6): starting the application program to be tested or manually executing other testing operations.
As one or more embodiments, the step (5): finishing acquisition and uploading log information; the method comprises the following specific steps:
step (5-1): stopping log recording, flow acquisition and user-defined acquisition processes;
step (5-2): acquiring a broadcast message generated within a set time in an Android environment by using shell connection; storing the test result data in a test result database;
step (5-3): obtaining a file generated by tcpdump and an executable file for capturing information from an Android environment;
step (5-4): cutting the pcap file generated by tcpdump in the step (4) according to network connection, respectively extracting data and data packets of each network connection, and respectively recombining the data and the data packets into a plurality of streams; and establishing a corresponding relation between the network connection and the application program according to the connection log collected in the step (4) and the UID and the packet name collected in the step (3-4).
Step (5-5): packaging and storing all the information collected in the steps (5-2), (5-3) and (5-4) into a database.
Step (5-4): after cutting the pcap file, a series of traffic files containing only one network connection is obtained. The system uses the source port, the destination address, the destination port and the timestamp information of the network connection to correspond to the information in the connection log, so that the corresponding relation between the network connection expressed in the connection log and the flow file obtained by segmentation can be established. Because the connection log contains UID information of the network connection initiator and each application program or system module is distributed with an independent UID in the Android environment, the system can be matched with the corresponding application program to create the network connection and correspond the network connection, the application program and the flow file obtained by segmentation.
Illustratively, step (5-2): and (3) using the shell connection created in the step (1-3) to run a dumpsys activity broadcasts history instruction, and acquiring a broadcast message generated in a period of time in the Android environment.
Illustratively, step (5-3): the file generated by tcpdump and other programs in step (4) is pulled from the Android environment by using an 'adb-s < serial number > pull < source file >' command.
Illustratively, step (5-4): cutting the pcap file collected in the step (4) according to the network connection, and establishing a corresponding relation between the network connection and the application program according to the connection log collected in the step (4) and the UID and the packet name collected in the step (3-4).
As one or more embodiments, the step (6): based on the content of the test result database, deducing the correlation of the collected logs in the tested application program; the method comprises the following specific steps:
step (6-1): dividing the Linux kernel log captured in the step (4-3) according to words and extracting keywords; keywords refer to words that can represent the type of event in the log and can describe parameters and variables of the event;
step (6-2): vectorizing the broadcast captured in step (5-2);
step (6-3): all log information obtained in the steps (6-1), (6-2), (4-2) and (4-4) are regarded as a series of events, and are placed on a time line according to time sequence; using the existing results in the test result database to perform analogy reasoning with the test results of other tested application programs, and deducing the causal relationship between the network behavior and the local behavior of the tested application program;
step (6-4): and (3) storing all the results obtained in the steps (6-1), (6-2) and (6-3) into a database.
Illustratively, step (6-1): the kernel log obtained by the test is segmented, and the key word information existing in the log is extracted by using a one-hot method in combination with log information obtained by testing other application programs;
by way of example, using existing results in the test results database, analogically reasoning with test results of other applications under test, deducing causal relationships between network behavior and local behavior of the applications under test; the method comprises the following specific steps:
screening out related events occurring within a period of time before or after the current event occurs by using a time window threshold value given by a user; inquiring the related events which occur in the same event and time threshold of other tested application programs from the database, taking intersection with the related events selected by the application of the test, calculating the association degree between the event and the related events, and analyzing the relationship between various online and offline behaviors of the tested application programs based on the association degree.
Example two
The embodiment provides a system for collecting real-time information and network traffic in an android environment;
as shown in fig. 1, a system for collecting real-time information and network traffic in an android environment includes:
the ADB command line driving module 100 is configured to communicate with the Android environment through an ADB protocol, and monitor and parse the iptables log, the Broadcast monitor module, the Android environment information parse module, the file and application management module, and the configurable task execution module. Is converted into commands that can be executed on the ADB protocol. The ADB command line driver module 100 executes all command lines that need to be executed through ADB shell connection on an Android environment, or establishes a series of long connection sessions for monitoring and collecting real-time log information;
the iptables log monitoring and analyzing module 101 is configured to operate iptables rules on an Android environment, and collect log information generated by the iptables rules; by adding the iptables rule, the iptables outputs a log message for each event for establishing network connection, and obtains endpoint information of the network connection and application program or system component information for initiating the network connection, and the network connection initiated by the application program is perceived in real time, so that the network behavior collection is simpler and more convenient;
and the Broadcast monitoring module 102 is used for operating dumpsys commands, monitoring all Broadcast information generated in the Android system in the testing process, and capturing communication information among application programs at the Android global level so as to acquire interaction relations among the application programs to be tested, the Android environment and other applications.
And the Android environment information analysis module 103 is used for acquiring software and hardware platform information and installed software package information of the Android environment, and acquiring real-time information such as a network state, a memory state, a service state and the like through operating dumpsys commands.
The file and application management module 104 is configured to push an application to be tested and an auxiliary file required for testing to an Android environment, and pull output generated by the application to be tested in a file form in a testing process;
a configurable task execution module 105 for executing other task scripts configured by a user;
the timer module 106 is configured to regularly call the Broadcast monitoring module 102 and the Android environment information analysis module 103, so that the Broadcast monitoring module 102 and the Android environment information analysis module 103 continuously refresh real-time information;
a test result database 107 for storing all files generated by the application to be tested in the test process;
a behavior correlation inference module 108, configured to infer correlation between local and network behaviors of all the tested applications.
The foregoing description is only of the preferred embodiments of the present application and is not intended to limit the same, but rather, various modifications and variations may be made by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims (6)

1. A method for collecting real-time information and network traffic in an android environment is characterized by comprising the following steps:
step (1): initializing a data acquisition environment in an Android environment in which an application program to be tested runs;
step (2): installing an application program to be tested in an Android environment;
the step (1): initializing a data acquisition environment in an Android environment in which an application program to be tested runs; the method specifically comprises the following steps:
step (1-1): pushing an executable file for capturing information to an Android environment;
step (1-2): assigning executable rights to the executable file;
step (1-3): establishing connection with an Android shell environment;
step (1-4): creating firewall rules;
step (3): acquiring basic information of an Android environment;
step (4): running an application program to be tested, starting to collect logs, and capturing log information;
the step (4): running an application program to be tested, starting to collect logs, and capturing log information; the method comprises the following specific steps:
step (4-1): activating the created firewall rules;
in the established shell connection, using two commands of "iptables-I output-p TCP-m conntrack-ctstate NEW [ -m owner-UID-owner < UID > ] -j NEWCONN_LOG" and "iptables-I output-p UDP-m conntrack-ctstate NEW [ -m owner-UID-owner < UID > ] -j NEWCONN_LOG", guiding the handshake messages of the newly established TCP connection and UDP connection in the system to the established firewall rules, so that the iptables firewall can output LOGs;
step (4-2): monitoring dumpsys: executing a dumpsys command at set time intervals by using the created shell connection; extracting a network state, a memory state and a service state from an execution result; comparing the network state, the memory state and the service state information obtained at this time with response information obtained by executing the dumpsys command last time, and searching out the changed attribute value;
step (4-3): monitoring a Linux kernel log: executing 'tail-f/proc/kmsg' by using the created shell connection, and obtaining a kernel log;
step (4-4): monitoring connection logs generated by a system firewall: the log information generated by the firewall rules is obtained in "/proc/kmsg"; filtering a connection log from the kernel message by using the created shell connection execution 'tail-f/proc/kmsg|grep < user-specified prefix >';
step (4-5): starting tcpdump by using the established shell connection, and starting to capture traffic; starting an executable file designated by a user and used for capturing information, and starting capturing other information which the user wants to capture;
step (4-6): starting an application program to be tested, or manually executing other test operations;
step (5): the collection is finished, log information is uploaded and stored in a test result database;
the step (5): finishing acquisition and uploading log information; the method comprises the following specific steps:
step (5-1): stopping log recording, flow acquisition and user-defined acquisition processes;
step (5-2): acquiring a broadcast message generated within a set time in an Android environment by using shell connection; storing the test result data in a test result database;
step (5-3): obtaining a file generated by tcpdump and an executable file for capturing information from an Android environment;
step (5-4): cutting a pcap file generated by tcpdump according to network connection, respectively extracting data and data packets connected with each network, and respectively recombining the data and the data packets into a plurality of streams; establishing a corresponding relation between network connection and an application program according to the collected connection log and the collected UID and packet name;
step (5-5): packaging and storing all the information collected in the steps (5-2), (5-3) and (5-4) into a database;
step (5-4): cutting the pcap file to obtain a series of flow files only comprising one network connection; the system uses the source port, the destination address, the destination port and the timestamp information of the network connection to correspond to the information in the connection log, namely, the corresponding relation between the network connection expressed in the connection log and the flow file obtained by segmentation is established; because the connection log contains UID information of a network connection initiator and each application program or system module is distributed with an independent UID in the Android environment, the system is matched with the corresponding application program to create the network connection, and the network connection, the application program and the flow file obtained by segmentation are corresponding;
step (6): based on the contents of the test results database, the correlation of the collected logs within the tested application is inferred.
2. The method for collecting real-time information and network traffic in an Android environment according to claim 1, wherein the system in the step (1-1) sends the required executable files to a file system of an Android device by using an adb push < source file > < target file > "through an adb tool in an Android SDK; the executable file includes tcpdump for capturing traffic;
step (1-2): using a command of chmod+x < target file >' to endow executable rights for the executable files pushed in the step (1-1) in an Android environment;
step (1-3): establishing connection with an Android shell environment by using an adb tool in the Android SDK;
in the shell connection created in the step (1-4), a chain named NEWCONN_LOG is established in the nat table of the iptables by using an "iptables-N NEWCONN_LOG" command, and a command for writing a LOG to a Linux kernel module is established by using an "iptables-A NEWCONN_LOG-j LOG-LOG-uid-LOG-prefix < user-specified prefix >".
3. The method for collecting real-time information and network traffic in an android environment as recited in claim 1, wherein said step (3) is: acquiring basic information of an Android environment; the method comprises the following specific steps:
step (3-1): extracting CPU information of an Android environment in the Android shell environment;
step (3-2): extracting an SDK version and a hardware platform model of an Android environment in the Android shell environment;
step (3-3): extracting information of external equipment and a driving version of Android equipment in an Android shell environment;
step (3-4): and acquiring the packet names and UIDs corresponding to all the current installed application programs to be tested.
4. The method for collecting real-time information and network traffic in an android environment as recited in claim 1, wherein said step (6) comprises: based on the content of the test result database, deducing the correlation of the collected logs in the tested application program; the method comprises the following specific steps:
step (6-1): dividing the captured Linux kernel log according to words and extracting keywords; keywords refer to words that can represent the type of event in the log and can describe parameters and variables of the event;
step (6-2): vectorizing the captured broadcast;
step (6-3): all log information obtained in the steps (6-1), (6-2), (4-2) and (4-4) are regarded as a series of events, and are placed on a time line according to time sequence; using the existing results in the test result database to perform analogy reasoning with the test results of other tested application programs, and deducing the causal relationship between the network behavior and the local behavior of the tested application program;
step (6-4): and (3) storing all the results obtained in the steps (6-1), (6-2) and (6-3) into a database.
5. The method for collecting real-time information and network traffic in an android environment as recited in claim 1, wherein existing results in a test result database are used to perform analog reasoning with test results of other applications under test to infer causal relationships between network behaviors and local behaviors of the applications under test; the method comprises the following specific steps:
screening out related events occurring within a period of time before or after the current event occurs by using a time window threshold value given by a user; inquiring the related events which occur in the same event and time threshold of other tested application programs from the database, taking intersection with the related events selected by the application of the test, calculating the association degree between the event and the related events, and analyzing the relationship between various online and offline behaviors of the tested application programs based on the association degree.
6. A system for collecting real-time information and network traffic in an android environment is characterized by comprising:
the ADB command line driving module is used for communicating with the Android environment through an ADB protocol, and converting task information from the iptables log monitoring and analyzing module, the Broadcast monitoring module, the Android environment information analyzing module, the file and application program management module and the configurable task executing module into commands which can be executed on the ADB protocol; the ADB command line driving module executes all command lines which need to be executed through ADB shell connection on an Android environment, or establishes a series of long connection sessions for monitoring and collecting real-time log information;
the iptables log monitoring and analyzing module is used for operating iptables rules on an Android environment and collecting log information generated by the iptables rules; by adding the iptables rule, the iptables outputs a log message for each event for establishing network connection, and obtains endpoint information of the network connection and application program or system component information for initiating the network connection, and the network connection initiated by the application program is perceived in real time, so that the network behavior collection is simpler and more convenient;
the Broadcast monitoring module is used for operating dumpsys commands, monitoring all Broadcast information generated in the Android system in the testing process, capturing communication information among application programs at the Android global level, and accordingly obtaining interaction relations among the application programs to be tested, the Android environment and other applications;
the Android environment information analysis module is used for acquiring software and hardware platform information and installed software package information of an Android environment, and acquiring real-time information of a network state, a memory state and a service state through operating dumpsys commands;
the file and application program management module is used for pushing the application program to be tested and auxiliary files required by the test to the Android environment and pulling output generated by the application program to be tested in a file form in the test process;
the configurable task execution module is used for executing other task scripts configured by a user;
the timer module is used for regularly calling the Broadcast monitoring module and the Android environment information analysis module, so that the Broadcast monitoring module and the Android environment information analysis module continuously refresh real-time information;
the test result database is used for storing all files generated in the test process of the application program to be tested;
and the behavior correlation reasoning module is used for reasoning the correlation between the local and network behaviors of all the tested application programs.
CN202110111586.9A 2021-01-27 2021-01-27 Method and system for collecting real-time information and network traffic in android environment Active CN112783777B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110111586.9A CN112783777B (en) 2021-01-27 2021-01-27 Method and system for collecting real-time information and network traffic in android environment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110111586.9A CN112783777B (en) 2021-01-27 2021-01-27 Method and system for collecting real-time information and network traffic in android environment

Publications (2)

Publication Number Publication Date
CN112783777A CN112783777A (en) 2021-05-11
CN112783777B true CN112783777B (en) 2023-08-04

Family

ID=75758069

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110111586.9A Active CN112783777B (en) 2021-01-27 2021-01-27 Method and system for collecting real-time information and network traffic in android environment

Country Status (1)

Country Link
CN (1) CN112783777B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114650168A (en) * 2022-02-14 2022-06-21 麒麟软件有限公司 Application program security testing method
CN115102884B (en) * 2022-06-23 2023-07-21 青岛联众芯云科技有限公司 Remote data flow statistics method and device for industrial personal computer application program

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106331071A (en) * 2016-08-16 2017-01-11 济南大学 Remote collection system and method for network flow of Android application
CN106330599A (en) * 2016-08-16 2017-01-11 济南大学 Multi-thread collection system and method for network flow of Android application program
CN111988239A (en) * 2020-08-21 2020-11-24 哈尔滨工业大学 Method for acquiring pure software flow for Android application

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106331071A (en) * 2016-08-16 2017-01-11 济南大学 Remote collection system and method for network flow of Android application
CN106330599A (en) * 2016-08-16 2017-01-11 济南大学 Multi-thread collection system and method for network flow of Android application program
CN111988239A (en) * 2020-08-21 2020-11-24 哈尔滨工业大学 Method for acquiring pure software flow for Android application

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
基于Android系统流量监控的安全软件设计与研究;孙少华;孙晓东;李卫;;电子设计工程(09);第136-137页 *

Also Published As

Publication number Publication date
CN112783777A (en) 2021-05-11

Similar Documents

Publication Publication Date Title
CN112783777B (en) Method and system for collecting real-time information and network traffic in android environment
CN103577324B (en) Static detection method for privacy information disclosure in mobile applications
Bento et al. Automated analysis of distributed tracing: Challenges and research directions
US20090248721A1 (en) System And Method for Stack Crawl Testing and Caching
Paakki et al. Software metrics by architectural pattern mining
CN111277643A (en) HTTP link tracking recording method and system
CN109542444B (en) JAVA application monitoring method, device, server and storage medium
CN111309609A (en) Software processing system
CN106294136B (en) The online test method and system of performance change between the concurrent program runtime
Pheng et al. Dynamic data structure analysis for Java programs
CN113836023A (en) Compiler security testing method based on system structure cross check
Johnson You can’t even ask them to push a button: Toward ubiquitous, developer-centric, empirical software engineering
Sniffen et al. Guided policy generation for application authors
CN110321130B (en) Non-repeatable compiling and positioning method based on system call log
CN115129539A (en) Log optimization method, device, equipment and storage medium
CN111158653B (en) SQL language-based integrated development and execution system for real-time computing program
EP2587380A1 (en) Runtime environment and method for non-invasive monitoring of software applications
Neginhal et al. Event views and graph reductions for understanding system level c code
Zhao et al. A microservice architecture for online mobile app optimization
EP4264857A1 (en) Industrial network-based codeless tracking analytics method and apparatus for industrial software
CN113360373A (en) Test method for full traversal of Activity page of mobile application
Zhang et al. iTES: Integrated testing and evaluation system for software vulnerability detection methods
Dalton et al. A toolkit for visualizing the runtime behavior of TinyOS applications
RU2390821C1 (en) Dynamic instrumentation technique
Lim et al. Visualizing The Intermediate Representation of Just-in-Time Compilers

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant