CN113761163A - Deep code search method, system and device based on code structure semantic information - Google Patents
Deep code search method, system and device based on code structure semantic information Download PDFInfo
- Publication number
- CN113761163A CN113761163A CN202110946937.8A CN202110946937A CN113761163A CN 113761163 A CN113761163 A CN 113761163A CN 202110946937 A CN202110946937 A CN 202110946937A CN 113761163 A CN113761163 A CN 113761163A
- Authority
- CN
- China
- Prior art keywords
- code
- code search
- deep
- search
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 99
- 238000012549 training Methods 0.000 claims abstract description 26
- 230000011218 segmentation Effects 0.000 claims abstract description 11
- 230000004927 fusion Effects 0.000 claims abstract description 10
- 239000000284 extract Substances 0.000 claims abstract description 8
- 230000006870 function Effects 0.000 claims description 22
- 230000006399 behavior Effects 0.000 claims description 15
- 238000004422 calculation algorithm Methods 0.000 claims description 14
- 238000005516 engineering process Methods 0.000 claims description 14
- 239000013598 vector Substances 0.000 claims description 11
- 238000013500 data storage Methods 0.000 claims description 10
- 230000015654 memory Effects 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 9
- 230000002688 persistence Effects 0.000 claims description 9
- 238000012360 testing method Methods 0.000 claims description 9
- 238000004458 analytical method Methods 0.000 claims description 8
- 238000005457 optimization Methods 0.000 claims description 8
- 238000007781 pre-processing Methods 0.000 claims description 8
- 238000013135 deep learning Methods 0.000 claims description 7
- 230000003993 interaction Effects 0.000 claims description 7
- 230000006403 short-term memory Effects 0.000 claims description 7
- 230000002452 interceptive effect Effects 0.000 claims description 6
- 230000007787 long-term memory Effects 0.000 claims description 6
- 230000008569 process Effects 0.000 claims description 6
- 238000007726 management method Methods 0.000 claims description 4
- 230000007246 mechanism Effects 0.000 claims description 4
- 244000035744 Hura crepitans Species 0.000 claims description 3
- 244000141353 Prunus domestica Species 0.000 claims description 3
- 238000010801 machine learning Methods 0.000 claims description 3
- 238000012544 monitoring process Methods 0.000 claims description 3
- 238000010276 construction Methods 0.000 claims description 2
- 238000007619 statistical method Methods 0.000 claims description 2
- 238000010200 validation analysis Methods 0.000 claims 2
- 230000000694 effects Effects 0.000 abstract description 6
- 238000004364 calculation method Methods 0.000 description 5
- 238000004891 communication Methods 0.000 description 5
- 238000012545 processing Methods 0.000 description 5
- 238000012795 verification Methods 0.000 description 5
- 230000018109 developmental process Effects 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 4
- 238000007405 data analysis Methods 0.000 description 3
- 238000011161 development Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000001419 dependent effect Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 241000512668 Eunectes Species 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004883 computer application Methods 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 239000010931 gold Substances 0.000 description 1
- 229910052737 gold Inorganic materials 0.000 description 1
- 230000008676 import Effects 0.000 description 1
- 238000013549 information retrieval technique Methods 0.000 description 1
- 238000011423 initialization method Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000004044 response Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/332—Query formulation
- G06F16/3329—Natural language query formulation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/279—Recognition of textual entities
- G06F40/284—Lexical analysis, e.g. tokenisation or collocates
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/30—Semantic analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/40—Transformation of program code
- G06F8/41—Compilation
- G06F8/42—Syntactic analysis
- G06F8/425—Lexical analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/40—Transformation of program code
- G06F8/41—Compilation
- G06F8/43—Checking; Contextual analysis
- G06F8/436—Semantic checking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/049—Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Artificial Intelligence (AREA)
- Mathematical Physics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Data Mining & Analysis (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- Biophysics (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Molecular Biology (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Databases & Information Systems (AREA)
- Stored Programmes (AREA)
Abstract
本发明公开了一种基于代码结构语义信息的深度代码搜索方法、系统及装置，包括从目标软件项目中提取代码文件中的方法级代码和相关注释，形成数据集；训练分词模型生成代码搜索数据集；预处理代码搜索数据集，将方法级代码解析为抽象语法树，遍历抽象语法树并提取API序列；构建深度代码搜索模型，包括代码结构信息编码模块、代码语义信息编码模块、信息融合模块、自然语言编码模块；使用预处理后的数据集，优化深度代码搜索模型；利用深度代码搜索模型，从代码搜索数据集中获得与自然语言查询对应的方法级代码。本发明的方法能够有效提升代码搜索模型对于代码结构语义信息以及自然语言描述文本的理解能力，提高代码搜索系统的搜索效果与性能。
The invention discloses a deep code search method, system and device based on code structure semantic information, including extracting method-level codes and related comments in code files from a target software project to form a data set; training a word segmentation model to generate code search data set; preprocess code search data set, parse method-level code into abstract syntax tree, traverse abstract syntax tree and extract API sequence; build deep code search model, including code structure information encoding module, code semantic information encoding module, and information fusion module , natural language coding module; use the preprocessed data set to optimize the deep code search model; use the deep code search model to obtain method-level codes corresponding to natural language queries from the code search data set. The method of the invention can effectively improve the code search model's ability to understand the code structure semantic information and the natural language description text, and improve the search effect and performance of the code search system.
Description
Technical Field
The invention belongs to the field of code search, and particularly relates to a deep code search method, a deep code search system and a deep code search device based on code structure semantic information.
Background
With the development of software development and internet technology, the size of codes accumulated by IT enterprises is larger and larger. The huge code data volume means that a large number of codes which can be repeatedly used exist, and how to utilize the existing codes and annotation texts in a massive code warehouse, so that repeated project development work of programmers is avoided, the utilization rate of the existing codes is improved, and the method is a very important problem in the field of intelligent software engineering. For a large internet company or a software company, there is an increasing demand for developing a code search system for use by an insider of an enterprise, in consideration of security, intellectual property, and the like. Some companies have developed code search systems that can be used to open source code repositories or enterprise internal code repositories, most of which are based on keyword matching and information retrieval techniques. The accuracy of these keyword matching and information retrieval technology based code search systems is often unsatisfactory, for a number of reasons including:
(1) code search systems lack comprehension capabilities for users' queries;
(2) a mismatch between semantic information expressed by the natural language query and structured information contained in the code;
(3) source code is a highly structured programming language, and natural languages that follow complex grammars are heterogeneous.
Thus, an effective natural language code search engine needs to establish a higher level semantic mapping relationship between the code and the natural language query so that the search engine can understand the semantic meaning of the natural language query and the source code to improve the accuracy of code search.
Disclosure of Invention
The invention aims to provide a deep code searching method, a system and a device based on code structure semantic information aiming at the defects of the prior art. The invention can solve the problems of different grammars of codes and natural languages and the problem of difficult semantic understanding of user query by the conventional method.
The purpose of the invention is realized by the following technical scheme: a deep code searching method based on code structure semantic information comprises the following steps:
(1) code data in the software project are obtained, and a code search data set is generated by utilizing the word segmentation model.
(2) And preprocessing the code search data.
(3) And optimizing and testing the depth code search model by utilizing the preprocessed data set.
(4) And implementing code search based on the depth code search model.
Further, the step (1) is specifically as follows: determining a Java software project range, and extracting Java methods and related annotations from Java files in the project; using the analyzed data set for training a BPE word segmentation model; and performing word segmentation processing on the data in the data set by using the trained word segmentation model to form a code search data set.
Further, in step (2), data preprocessing includes: intercepting a first segment of annotation of the Java method as a natural language annotation, and omitting the annotation of parameters and the annotation of a return value; removing Java methods that do not contain any API and duplicate Java methods; converting the Java method into an abstract syntax tree by using a code analysis tool Javaparser, extracting an API sequence from the abstract syntax tree by adopting a depth-limited traversal strategy, wherein the Java method after data centralized preprocessing comprises the abstract syntax tree and the API sequence; and (4) dividing the preprocessed data set into a training set, a verification set and a test set, and optimizing and testing the depth code search model in the step (3).
Further, in step (3): the depth code search model utilizes three attention-based long-term and short-term memory networks and comprises a network construction code structure information coding module, a code semantic information coding module and a natural language coding module.
In order to optimize parameters in the three long-term and short-term memory networks, the three modules respectively take the abstract syntax tree, the API sequence and the related comments of the Java method in the training data in the step (2) as input; then constructing an information fusion module, fusing vectors output by the code structure information coding module and the code semantic information coding module; and finally, constructing a similarity matching module, calculating cosine similarity between the fusion vector and the output vector of the natural language coding module, calculating a loss function of the depth code search model, and optimizing parameters in the three coding modules.
The stopping conditions of the optimization process are as follows: and the optimization iteration times exceed the total number of the Java methods in the training set or the effect of the model on the verification set is converged.
Further, the step (4) is specifically as follows: and taking the natural language query and the Java method as input, calculating the similarity between the two, finishing the reordering of the Java method set according to the similarity, and outputting the Java method after final ordering.
A deep code searching system based on code structure semantic information comprises an off-line end and an on-line end.
And the offline end is responsible for functions of analyzing Java files, constructing a structured data set, training a deep code search model and the like.
And the on-line end is used for interacting with the user, providing webpage search entries and search result presentation for the user, recording the behavior of the user for viewing the search results, and performing statistical analysis and subsequent further optimization on the depth code search model.
Further, the off-line end comprises a data analysis module, a data storage module and a model training module.
And the data analysis module extracts the Java method and corresponding annotations from the Java file, analyzes the Java method into an abstract syntax tree, prunes according to the depth of the abstract syntax tree, extracts the method name and method body of the Java method from the analyzed abstract syntax tree by using a method based on a regular expression, and traverses the abstract syntax tree to extract an API sequence.
And the data storage module supports large-scale calculation and storage, supports multiple calculation types, provides fine-grained authority management, sandbox protection and data monitoring functions, and stores the analyzed data in a json format for training a deep code search model.
The model training module supports various machine learning and deep learning calculation frameworks, including a streaming calculation framework, a deep learning framework and a calculation engine.
Further, the online end comprises a core business layer, a deep code search model and a data persistence module.
The core service layer has the main functions of completing a code search task requested by a user interactive interface by using a deep code search model and storing a behavior of a user for viewing a search result into a data persistence module. The core service layer uses the Springboot technology to build functions of an elastic search part, an elastic algorithm service part, a user authentication part and a user behavior analysis part. The ElasticSearch is a Lucene-dependent high-performance search engine library for storing code search data sets. The elastic algorithm service utilizes the Docker technology to serve the code search function of the deep code search model in the form of a Restful API interface. When the back-end service layer receives a natural language query request from a user interactive interface, the user authenticates and filters out an unauthorized request, the ElasticSearch searches a Java method set related to the natural language query from a code search data set for the authorized request, the code search service in the elastic algorithm service reorders the Java method set, and finally, the ordering result is sent to the user interactive interface. The behavior of the user for viewing the search result is fed back to the core service layer, records are stored in the data persistence module, and the records can be viewed and statistically analyzed through user behavior analysis.
Further, the core service layer provides a caching mechanism for the user interaction interface to relieve server stress and quickly respond to user searches. The method comprises the steps of displaying functional modules in a core service layer and the dependency relationship among the functional modules by utilizing a graphical technology to form an interactive module display interface, wherein the interface supports the function of dynamically adjusting the dependency relationship among the modules in a mode of dragging the positions of the modules by a mouse.
A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the computer program when executed implementing the method or the system.
The invention has the beneficial effects that: the invention efficiently utilizes the existing codes and annotation texts in the mass code warehouse, avoids repeated project development work, and introduces a deep code search model to improve the code utilization rate. The method comprises the steps of constructing an abstract syntax tree and an API sequence corresponding to a code segment (such as a method level code) by using code structure information, constructing a code structure information coding module, a code semantic information coding module, an information fusion module and a natural language coding module, effectively improving the comprehension capability of a code search model on code structure semantic information and a natural language description text, and effectively improving the search effect and performance of a code search system by using the code search model.
Drawings
The accompanying drawings are included to provide a better understanding of the invention and are not to be construed as unduly limiting the invention:
FIG. 1 is a flowchart of a deep code search method based on semantic information of a code structure according to an embodiment of the present invention;
FIG. 2 is a block diagram of an overall depth code search model according to an embodiment of the present invention;
FIG. 3 is an architecture diagram of a deep code search system based on semantic information of code structures according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of a computer system of a terminal device or a server for implementing the embodiment of the present invention.
Detailed Description
In order to more clearly explain the present invention, the present invention is further explained below with reference to the embodiments and the drawings. It is to be understood by persons skilled in the art that the following detailed description is illustrative and not restrictive, and is not to be taken as limiting the scope of the invention.
As shown in fig. 1, an embodiment of a depth code search method based on code structure semantic information in the present invention specifically includes the following steps:
101: code data is acquired and a code search data set is generated using a word segmentation model.
Firstly, the range of data selection needs to be determined, for example, a Java code file in an enterprise is selected, and Java methods (method-level Java code) and related annotation pairs are extracted from the Java code file as a data set. And then training a BPE (Byte-Pair Encoding) word segmentation model by using the words in the data set, and carrying out word segmentation processing on the data set by using the trained BPE model to form a code search data set.
102: the code search data set generated in step 101 is subjected to data preprocessing.
In order to achieve excellent results for the deep learning model, effective data preprocessing work needs to be performed on the data set. Intercepting a first segment of content of the Java method related annotation as a natural language annotation, and removing annotation content and the like related to Java method parameters and return values in the first segment of content; java methods that do not contain any API are removed as well as duplicate Java methods.
In order to obtain the structural semantic information in the Java method, a code analysis tool Javaparser is utilized to convert the Java method into an abstract syntax tree so as to express the structural semantic of the Java method. And traversing each part in the abstract syntax tree by adopting a depth-first traversal strategy, extracting the API in the abstract syntax tree, and generating an API sequence for expressing the key programming semantics of the Java method. Thus, the Java methods in the dataset are represented as two parts, an abstract syntax tree and an API sequence.
And finally, dividing the analyzed data set into a training set, a verification set and a test set.
103: and optimizing and testing the depth code search model by using the data set preprocessed in the step 102.
As shown in fig. 2, the depth code search model mainly includes a code encoding module, a natural language encoding module 207, and a similarity matching module 208. The code encoding module comprises a code structure information encoding module 203, a code semantic information encoding module 204 and an information fusion module 205. The code structure information encoding module 203, the code semantic information encoding module 204, and the natural language encoding module 207 are respectively formed by a Long Short-Term Memory (LSTM) network based on the attention mechanism.
In order to optimize the deep code search model, a random initialization method is adopted to initialize parameters in a long-term and short-term memory network in the code structure information coding module 203, the code semantic information coding module 204 and the natural language coding module 207 respectively, and an abstract syntax tree 201, an API sequence 202 and related annotation information (query/natural language annotation 206) analyzed in a Java method are used as input of the module code structure information coding module 203, the code semantic information coding module 204 and the natural language coding module 207 respectively and are output as vectors.
The information fusion module 205 fuses the vectors output by the code structure information encoding module 203 and the code semantic information encoding module 204 into vectors with the same size as the output of the natural language module 207, so as to better represent the structure semantic information included in the Java method.
The two vectors output by the natural language coding module 207 and the information fusion module 205 are sent to the similarity matching module 208, the similarity matching module 208 calculates the similarity between the two vectors of the natural language coding module 207 and the information fusion module 205 by using a cosine similarity algorithm, calculates a loss function of the depth code search model, and optimizes parameters of a long-term and short-term memory network in the code structure information coding module 203, the code semantic information coding module 204 and the natural language coding module 207.
In the model optimization iteration process, a training set is used as training data, a verification set is used as detection of model effect, and an iteration stopping condition is that the training data are completely used up, namely the iteration times exceed the total number of Java methods in the training set or the effect of the model on the verification set is converged. The effect of the trained deep code search model running on the test set will be the final result of evaluating the performance of the model.
104: a search system implementation based on a depth code search model.
As shown in fig. 3, an embodiment of a depth code search system based on semantic information of a code structure of the present invention mainly includes two parts, an offline end 301 and an online end 306.
The offline end 301 is divided into three parts, namely a data analysis module 302, a data storage module 303 and a model training module 304. The deep code search system of the present invention manages the code repository 305 in an enterprise using Git technology.
The data parsing module 302 mainly completes the function of parsing the Java file. The data parsing module 302 extracts Java methods and their corresponding annotations from Java files in the code repository 305 using the code parsing tool Java parser. The data parsing module 302 parses the Java method into an abstract syntax tree, prunes according to the depth of the abstract syntax tree, extracts the method name and method body of the Java method from the parsed abstract syntax tree by using a method based on a regular expression, and traverses the abstract syntax tree to extract an API sequence. As such, data parsing module 302 represents Java methods as structured data, including Java method source code, comments, method names, method bodies, abstract syntax trees, API sequences. The data storage module 303 saves all the structured data about Java methods parsed from the code repository 305 by the data parsing module 302 in json format.
The data of the data storage module 303 is all located in the server, which is a solution relying on the cloud server. The data storage module 303 supports large-scale computing and storage, can support various computing types such as SQL, MapReduce, and UDF, and provides fine-grained rights management, sandbox protection, and data monitoring functions.
The model training module 304 trains the deep code search model using the Java methods and annotation data stored in the data storage module 303. The model training module 304 is an artificial intelligence platform that can provide a one-stop deep learning solution, and supports various machine learning and deep learning computing frameworks, such as a streaming computing framework Flink, a deep learning framework tensrflow, a computing engine Spark, and the like.
The on-line end 306 mainly includes three modules, namely a core business layer 314, a deep search model 311 and a data persistence module 312. The on-line end 306 realizes interaction with the user through the user interaction interface 313, provides a web search entry and low-delay search result presentation and user behavior embedding points for the user, and records the use conditions of the search system user, such as user ID, search time, operating system, browser and version number, input natural language description, access conditions to the search results, and the like. These data are used to statistically analyze the user's code search behavior, providing training data for further model optimization.
The main task at line end 306 is to implement the functions required for the user interaction interface 313 with the core services layer 314 at the back end. The core service layer 314 is built by using a SpringBoot technology, and four functions of user authentication 307, user behavior analysis 309, elastic search308 and elastic algorithm service 310 are completed.
The bottom layer of the ElasticSearch308 implements a Lucene-dependent high-performance search engine library, which imports a code search data set, i.e., a parsed Java method and a related annotation set, from the data storage module 303. The core service layer 314 at the back end receives the natural language query request from the user interface 313, and the user authentication 307 determines whether the request has authority. When the request is authorized, the core business layer 314 of the back-end retrieves the set of Java methods associated with the natural language query from the code search dataset using the ElasticSearch 308.
Then, the elastic algorithm service 310 services the code search function in the deep search model 311 by using a Docker technology, the service takes the natural language query and the Java method as input, calculates the similarity between the two (i.e. the output of the similarity module 208 in the deep search model), completes the reordering (from large to small) of the Java method set according to the similarity, and outputs the Java method after final ordering.
The service process comprises the steps of creating a Docker container, deploying a deep search model 311 to the Docker container, optimizing the deep search model 311 by using a code search data set in an ElasticSearch308, and providing a code search function based on a Restful API interface to a core service layer 31 at the back end.
The reason why the elastic algorithm service 310 uses the Docker technology to carry the algorithm service is that Docker has particularly good environment portability and extensibility. On one hand, when the Docker technology is used, a model environment can be built locally and quickly, an algorithm model can be deployed and debugged; after the local debugging is passed, the local environment mirror image can be directly uploaded to a server for deployment. On the other hand, when the user request amount exceeds the load limit of the current Docker service, the elastic algorithm service 310 may rapidly deploy a new Docker service on a newly added server, so as to improve the response speed to the user request.
After the search is completed, the core business layer 314 at the back end sends the search result of the elastic algorithm service 310 to the user interaction interface 313, records the viewing behavior of the user on the search result on the user interaction interface 313, and stores the data in the server disk by using the data persistence module 312 for further optimizing the deep search model 311 subsequently. The data stored by the data persistence module 312 may be viewed and statistically analyzed by the user behavior analysis 309.
In addition, it has been found in practice that users often use the same natural language query multiple times in sequence when searching for a single functional method, so the core service layer 314 also provides caching to relieve server stress and quickly respond to user searches. The depth code search system can perform update iteration along with the advance of time, and in order to conveniently and flexibly adjust the logic structure of the system, the dependency relationships among the modules and the modules in the line end 306 are displayed in a background management interface by utilizing a graphical technology to form an interactive module display interface. The interface supports the function of dynamically adjusting the dependency relationship between the modules in a mode of dragging the positions of the modules by a mouse.
The experimental environment of the deep code search system based on the code structure semantic information is as follows:
operating the system: ubuntu 16.04.1LTS
CPU：Intel(R)Xeon(R)Gold 6226CPU@2.70GHz
GPU：NVIDIA GeForce RTX 2080Ti
Memory: 64GB
Hard disk: 4TB
Programming language: python 3.7
Anaconda：5.3.0
TensorFlow：2.3.0
Docker：19.03.4
As shown in fig. 4, an embodiment of a terminal device or a server of the present invention includes a removable medium 401 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like, mounted on a drive 402 as necessary, so that a computer program read out therefrom is installed into a storage section 406 as necessary. The driver 402 is also connected to the I/O interface 407 as necessary. The communication section 403 performs communication processing via a network such as the internet. The following components are connected to an output portion 405 of a line tube (CRT), a display, and the like, and a speaker and the like; a storage section 406 including a hard disk and the like; and a communication section 403 including a network interface card such as a LAN card, a modem, or the like. The communication section 403 accepts control information sent to the input section 404 from, for example, a keyboard and a mouse.
The central processing unit CPU 409 can perform various appropriate actions and processes in accordance with a program stored in the read only memory ROM 410 or a program loaded from the storage section 406 into the random access memory RAM 411. In the RAM 411, various programs and data necessary for the operation of the terminal device or the server are also stored. The CPU 409, ROM 410, and RAM 411 are connected to each other via a bus 408. An input/output I/O interface 407 is also connected to bus 408.
In particular, the processes described above with reference to the flowcharts may be implemented as computer application software according to the embodiments of the present disclosure. For example, the disclosed embodiments of the invention include a computer application comprising a computer program embodied on a computer readable medium, the computer program comprising program code for executing the system shown in FIG. 2. In the present embodiment, the computer program can be downloaded and installed from a network through the communication section 403, and/or installed from the removable medium 401. The above-described functions defined in the system of the present invention are executed when the computer program is executed by the central processing unit CPU 409.
It should be noted that the computer readable medium shown in the present invention can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a Random Access Memory (RAM), a read-only memory (ROM), a computer diskette, a hard disk, an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present invention, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The above-described embodiments are intended to illustrate rather than limit the invention, and those skilled in the art will be able to make various modifications and improvements without departing from the spirit of the invention without limiting the invention to the details of the illustrative embodiments set forth herein. Any modification and variation of the present invention within the spirit of the present invention and the scope of the claims will fall within the scope of the present invention.
Claims (10)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110946937.8A CN113761163B (en) | 2021-08-18 | 2021-08-18 | Deep code search method, system and device based on code structure semantic information |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110946937.8A CN113761163B (en) | 2021-08-18 | 2021-08-18 | Deep code search method, system and device based on code structure semantic information |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113761163A true CN113761163A (en) | 2021-12-07 |
| CN113761163B CN113761163B (en) | 2024-02-02 |
Family
ID=78790301
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110946937.8A Active CN113761163B (en) | 2021-08-18 | 2021-08-18 | Deep code search method, system and device based on code structure semantic information |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113761163B (en) |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114461194A (en) * | 2021-12-28 | 2022-05-10 | 东莞市李群自动化技术有限公司 | Auxiliary method, equipment and storage medium for code development |
| CN114896368A (en) * | 2022-05-23 | 2022-08-12 | 郑州大学产业技术研究院有限公司 | Semantic code search model construction method in large-scale candidate set and related device |
| CN115062106A (en) * | 2022-06-07 | 2022-09-16 | 华南理工大学 | Code searching method, system and medium based on function multiple graph embedding |
| CN115202732A (en) * | 2022-06-27 | 2022-10-18 | 深圳市互通创新科技有限公司 | Intelligent software development auxiliary system and use method |
| CN115577075A (en) * | 2022-10-18 | 2023-01-06 | 华中师范大学 | Deep code searching method based on relational graph convolutional network |
| CN116048454A (en) * | 2023-03-06 | 2023-05-02 | 山东师范大学 | A code rearrangement method and system based on iterative contrastive learning |
| CN116400901A (en) * | 2023-04-12 | 2023-07-07 | 上海计算机软件技术开发中心 | Python code automatic generation method and system |
| WO2023197397A1 (en) * | 2022-04-13 | 2023-10-19 | 堡垒科技有限公司 | Decentralized trusted tokenization protocol for open source software |
| CN116955719A (en) * | 2023-09-20 | 2023-10-27 | 布谷云软件技术(南京)有限公司 | Code management method and system for digital storage of chained network structure |
| CN117422002A (en) * | 2023-12-19 | 2024-01-19 | 利尔达科技集团股份有限公司 | AIGC-based embedded product generation method, system and storage medium |
| CN117573084A (en) * | 2023-08-02 | 2024-02-20 | 广东工业大学 | Code complement method based on layer-by-layer fusion abstract syntax tree |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102063488A (en) * | 2010-12-29 | 2011-05-18 | 南京航空航天大学 | Code searching method based on semantics |
| CN110716749A (en) * | 2019-09-03 | 2020-01-21 | 东南大学 | Code searching method based on function similarity matching |
| CN111159223A (en) * | 2019-12-31 | 2020-05-15 | 武汉大学 | An interactive code search method and device based on structured embedding |
| US10761839B1 (en) * | 2019-10-17 | 2020-09-01 | Globant España S.A. | Natural language search engine with a predictive writing tool for coding |
| CN112507065A (en) * | 2020-11-18 | 2021-03-16 | 电子科技大学 | Code searching method based on annotation semantic information |
| CN112800172A (en) * | 2021-02-07 | 2021-05-14 | 重庆大学 | A code search method based on two-stage attention mechanism |
-
2021
- 2021-08-18 CN CN202110946937.8A patent/CN113761163B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102063488A (en) * | 2010-12-29 | 2011-05-18 | 南京航空航天大学 | Code searching method based on semantics |
| CN110716749A (en) * | 2019-09-03 | 2020-01-21 | 东南大学 | Code searching method based on function similarity matching |
| US10761839B1 (en) * | 2019-10-17 | 2020-09-01 | Globant España S.A. | Natural language search engine with a predictive writing tool for coding |
| CN111159223A (en) * | 2019-12-31 | 2020-05-15 | 武汉大学 | An interactive code search method and device based on structured embedding |
| CN112507065A (en) * | 2020-11-18 | 2021-03-16 | 电子科技大学 | Code searching method based on annotation semantic information |
| CN112800172A (en) * | 2021-02-07 | 2021-05-14 | 重庆大学 | A code search method based on two-stage attention mechanism |
Cited By (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114461194A (en) * | 2021-12-28 | 2022-05-10 | 东莞市李群自动化技术有限公司 | Auxiliary method, equipment and storage medium for code development |
| WO2023197397A1 (en) * | 2022-04-13 | 2023-10-19 | 堡垒科技有限公司 | Decentralized trusted tokenization protocol for open source software |
| CN114896368A (en) * | 2022-05-23 | 2022-08-12 | 郑州大学产业技术研究院有限公司 | Semantic code search model construction method in large-scale candidate set and related device |
| CN115062106A (en) * | 2022-06-07 | 2022-09-16 | 华南理工大学 | Code searching method, system and medium based on function multiple graph embedding |
| CN115062106B (en) * | 2022-06-07 | 2024-11-15 | 华南理工大学 | Code search method, system and medium based on multi-graph embedding of function functions |
| CN115202732B (en) * | 2022-06-27 | 2023-08-08 | 苏州唐人数码科技有限公司 | Intelligent software development auxiliary system and application method |
| CN115202732A (en) * | 2022-06-27 | 2022-10-18 | 深圳市互通创新科技有限公司 | Intelligent software development auxiliary system and use method |
| CN115577075A (en) * | 2022-10-18 | 2023-01-06 | 华中师范大学 | Deep code searching method based on relational graph convolutional network |
| CN115577075B (en) * | 2022-10-18 | 2024-03-12 | 华中师范大学 | Depth code searching method based on relation diagram convolution network |
| CN116048454A (en) * | 2023-03-06 | 2023-05-02 | 山东师范大学 | A code rearrangement method and system based on iterative contrastive learning |
| CN116400901A (en) * | 2023-04-12 | 2023-07-07 | 上海计算机软件技术开发中心 | Python code automatic generation method and system |
| CN116400901B (en) * | 2023-04-12 | 2024-06-11 | 上海计算机软件技术开发中心 | Python code automatic generation method and system |
| CN117573084A (en) * | 2023-08-02 | 2024-02-20 | 广东工业大学 | Code complement method based on layer-by-layer fusion abstract syntax tree |
| CN117573084B (en) * | 2023-08-02 | 2024-04-12 | 广东工业大学 | A code completion method based on layer-by-layer fusion of abstract syntax trees |
| CN116955719A (en) * | 2023-09-20 | 2023-10-27 | 布谷云软件技术(南京)有限公司 | Code management method and system for digital storage of chained network structure |
| CN116955719B (en) * | 2023-09-20 | 2023-12-05 | 布谷云软件技术(南京)有限公司 | Code management method and system for digital storage of chained network structure |
| CN117422002A (en) * | 2023-12-19 | 2024-01-19 | 利尔达科技集团股份有限公司 | AIGC-based embedded product generation method, system and storage medium |
| CN117422002B (en) * | 2023-12-19 | 2024-04-19 | 利尔达科技集团股份有限公司 | AIGC-based embedded product generation method, AIGC-based embedded product generation system and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113761163B (en) | 2024-02-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113761163B (en) | Deep code search method, system and device based on code structure semantic information | |
| US12141557B2 (en) | Pruning engine | |
| CN111191002B (en) | Neural code searching method and device based on hierarchical embedding | |
| US8577823B1 (en) | Taxonomy system for enterprise data management and analysis | |
| US12007988B2 (en) | Interactive assistance for executing natural language queries to data sets | |
| US11042707B2 (en) | Conversational interface for APIs | |
| US11601453B2 (en) | Methods and systems for establishing semantic equivalence in access sequences using sentence embeddings | |
| US11726994B1 (en) | Providing query restatements for explaining natural language query results | |
| US11500865B1 (en) | Multiple stage filtering for natural language query processing pipelines | |
| US11481202B2 (en) | Transformation templates to automate aspects of computer programming | |
| CN116991990A (en) | AIGC-based program development auxiliary methods, storage media and equipment | |
| CN113836235B (en) | Data processing method based on data center and related equipment thereof | |
| CN117252261A (en) | Knowledge graph construction method, electronic equipment and storage medium | |
| EP3945431A1 (en) | Bridge from natural language processing engine to database engine | |
| US20190164061A1 (en) | Analyzing product feature requirements using machine-based learning and information retrieval | |
| CN112000929A (en) | Cross-platform data analysis method, system, equipment and readable storage medium | |
| CN116974554A (en) | Code data processing method, apparatus, computer device and storage medium | |
| Zhu et al. | A neural network architecture for program understanding inspired by human behaviors | |
| Huang et al. | BCGen: a comment generation method for bytecode | |
| CN108932225B (en) | Method and system for converting natural language requirements into semantic modeling language statements | |
| US11847117B2 (en) | Filter class for querying operations | |
| US11316807B2 (en) | Microservice deployment in multi-tenant environments | |
| Tavares et al. | How COVID-19 impacted data science: a topic retrieval and analysis from GitHub projects’ descriptions | |
| CN117294467B (en) | A SQL injection attack detection method and device based on class imbalance | |
| Gu et al. | Coral: federated query join order optimization based on deep reinforcement learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |