CN112540767A - Program code generation method, program code generation device, electronic device and storage medium - Google Patents

Program code generation method, program code generation device, electronic device and storage medium Download PDF

Info

Publication number
CN112540767A
CN112540767A CN202011563691.8A CN202011563691A CN112540767A CN 112540767 A CN112540767 A CN 112540767A CN 202011563691 A CN202011563691 A CN 202011563691A CN 112540767 A CN112540767 A CN 112540767A
Authority
CN
China
Prior art keywords
target
program code
compiling
optimization
data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202011563691.8A
Other languages
Chinese (zh)
Other versions
CN112540767B (en
Inventor
吕飞飞
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Baidu Netcom Science and Technology Co Ltd
Original Assignee
Beijing Baidu Netcom Science and Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Baidu Netcom Science and Technology Co Ltd filed Critical Beijing Baidu Netcom Science and Technology Co Ltd
Priority to CN202011563691.8A priority Critical patent/CN112540767B/en
Publication of CN112540767A publication Critical patent/CN112540767A/en
Application granted granted Critical
Publication of CN112540767B publication Critical patent/CN112540767B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00Arrangements for software engineering
    • G06F8/40Transformation of program code
    • G06F8/41Compilation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00Arrangements for software engineering
    • G06F8/40Transformation of program code
    • G06F8/51Source to source
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02DCLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Stored Programmes (AREA)

Abstract

The application discloses a program code generation method and device, electronic equipment and a storage medium, and relates to the technical field of computers, in particular to the technical field of artificial intelligence such as deep learning and cloud computing. The specific implementation scheme is as follows: acquiring a program code to be processed and a plurality of on-line performance sampling data; determining a target compiling stage and a target compiling optimization plug-in corresponding to the target compiling stage; determining an optimization algorithm and conversion parameters corresponding to the target compiling optimization plug-in; processing the plurality of on-line performance sampling data by adopting the conversion parameters to obtain a plurality of target sampling data; and processing the program code to be processed according to the corresponding optimization algorithm and the target sampling data to obtain a target program code, optimizing the program code to be processed by using the on-line performance sampling data, and generating and outputting the optimized target program code, so that the flexibility and timeliness of program compiling and optimizing can be effectively improved, and the delivery cycle of compiling to the on-line can be effectively shortened.

Description

Program code generation method, program code generation device, electronic device and storage medium
Technical Field
The application relates to the technical field of computers, in particular to the technical field of artificial intelligence such as deep learning and cloud computing, and particularly relates to a program code generation method and device, an electronic device and a storage medium.
Background
Artificial intelligence is the subject of research that makes computers simulate some human mental processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.), both at the hardware level and at the software level. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing, and the like; the artificial intelligence software technology mainly comprises a computer vision technology, a voice recognition technology, a natural language processing technology, machine learning/deep learning, a big data processing technology, a knowledge map technology and the like.
At present, in business services of various application scenarios, program codes are widely used to serve tens of thousands of machines, with continuous expansion of business, the compiling speed of the program codes gradually decreases, and the compiling running performance also decreases, so that corresponding optimization processing needs to be performed on the program codes.
Disclosure of Invention
A method, an apparatus, an electronic device, a storage medium, and a computer program product for generating a program code are provided.
According to a first aspect, there is provided a method of generating program code, comprising: acquiring a program code to be processed and a plurality of on-line performance sampling data; determining a target compiling stage and a target compiling optimization plug-in corresponding to the target compiling stage; determining an optimization algorithm and conversion parameters corresponding to the target compiling optimization plug-in; processing the plurality of on-line performance sample data using the conversion parameters to obtain a plurality of target sample data; and processing the program code to be processed according to the corresponding optimization algorithm and the target sampling data to obtain a target program code.
According to a second aspect, there is provided a program code generation apparatus comprising: the system comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring a program code to be processed and a plurality of on-line performance sampling data; the first determination module is used for determining a target compiling stage and a target compiling optimization plug-in corresponding to the target compiling stage; the second determination module is used for determining an optimization algorithm and a conversion parameter corresponding to the target compiling optimization plug-in; the conversion module is used for processing the on-line performance sampling data by adopting the conversion parameters to obtain a plurality of target sampling data; and the processing module is used for processing the program code to be processed according to the corresponding optimization algorithm and the target sampling data to obtain a target program code.
According to a third aspect, there is provided an electronic device comprising: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the program code generation method of the embodiment of the present application.
According to a fourth aspect, a non-transitory computer-readable storage medium is presented storing computer instructions for causing a computer to perform a method of generating program code disclosed in embodiments of the present application.
According to a fifth aspect, a computer program product is proposed, comprising a computer program which, when executed by a processor, implements the method for generating program code disclosed in embodiments of the present application.
It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present application, nor do they limit the scope of the present application. Other features of the present application will become apparent from the following description.
Drawings
The drawings are included to provide a better understanding of the present solution and are not intended to limit the present application. Wherein:
FIG. 1 is a schematic diagram according to a first embodiment of the present application;
FIG. 2 is a schematic diagram according to a second embodiment of the present application;
FIG. 3 is a schematic diagram of on-line CPU model distribution in the embodiment of the present application;
FIG. 4 is a schematic diagram of on-line performance sampling data acquisition in an embodiment of the present application;
FIG. 5 is a schematic diagram of a Perf tool;
FIG. 6 is a diagram illustrating format processing in an embodiment of the present application;
FIG. 7 is a schematic diagram of an optimization processing method in an embodiment of the present application;
FIG. 8 is a block diagram illustrating an integrated configuration of a program code generation apparatus according to an embodiment of the present application;
FIG. 9 is a schematic illustration according to a third embodiment of the present application;
FIG. 10 is a schematic diagram of an application in an embodiment of the present application;
FIG. 11 is a schematic illustration according to a fourth embodiment of the present application;
FIG. 12 is a schematic illustration according to a fifth embodiment of the present application;
fig. 13 is a block diagram of an electronic device for implementing a program code generation method according to an embodiment of the present application.
Detailed Description
The following description of the exemplary embodiments of the present application, taken in conjunction with the accompanying drawings, includes various details of the embodiments of the application for the understanding of the same, which are to be considered exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present application. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
Fig. 1 is a schematic diagram according to a first embodiment of the present application.
It should be noted that an execution subject of the method for generating a program code according to this embodiment is a device for generating a program code, where the device may be implemented in a software and/or hardware manner, and the device may be configured in an electronic device, and the electronic device may include, but is not limited to, a terminal, a server, and the like.
The embodiment of the application relates to the technical field of artificial intelligence such as deep learning and cloud computing.
Wherein, Artificial Intelligence (Artificial Intelligence), english is abbreviated as AI. The method is a new technical science for researching and developing theories, methods, technologies and application systems for simulating, extending and expanding human intelligence.
Deep learning is the intrinsic law and expression level of the learning sample data, and the information obtained in the learning process is very helpful for the interpretation of data such as characters, images and sounds. The final goal of deep learning is to make a machine capable of human-like analytical learning, and to recognize data such as characters, images, and sounds.
And Cloud computing (Cloud computing) refers to a technology architecture that accesses a flexibly extensible shared physical or virtual resource pool through a network, where the resources may include servers, operating systems, networks, software, applications, storage devices, and the like, and may be deployed and managed in an on-demand, self-service manner. Through the cloud computing technology, high-efficiency and strong data processing capacity can be provided for technical application and model training of artificial intelligence, block chains and the like.
As shown in fig. 1, the method for generating the program code includes:
s101: program code to be processed and a plurality of on-line performance sample data are obtained.
The program code to be optimized may be referred to as a program code to be processed, and the program code may be a binary file generated by an initial source code at any one of the execution stages, such as a compiling stage, an assembling stage, a linking stage, and the like, and the source code (also referred to as a source program) refers to an uncompiled text file written according to a certain programming language specification and is a series of human-readable computer language instructions.
An example of performing corresponding running compilation on an initial source program may be as follows, and in the embodiment of the present application, the source program is exemplified by a C + + program, which is not limited to this.
C + + program compiling flow: the method specifically comprises the following steps: source code → preprocessor → compiler → assembler → object code → Linker → executable.
Pretreatment: preprocessing corresponds to assembling a new C + + program from the preprocessed instructions. After pretreatment, an output file without macro definition, conditional compiling instruction and special symbol can be generated, and the meaning of the file is the same as that of the original file, but the content is different.
And (3) compiling: and (3) carrying out a series of lexical analysis, syntactic analysis and semantic analysis on the preprocessed files, and translating the preprocessed files into assembly code files after all instructions are confirmed to be in accordance with syntactic rules. In this step, the compiler will perform check optimization on the code to indicate syntax errors, heavy resolution errors, and other various compilation errors.
An assembly stage: the compiled assembly code is translated into a machine instruction, no complex grammar and no semantics exist, instruction optimization is not needed, and only the assembly code and the machine instruction are translated one by one according to a comparison table of the assembly instruction and the machine instruction.
And (3) a linking stage: the linker will link the various object files produced by the compiler in the previous stage to generate the final executable file.
That is, the program code to be optimized in this application may be a binary file (bin file) generated in any one of the above-mentioned runtime stages, which is not limited herein.
The above-mentioned on-line performance sampling data is compiled performance data when a corresponding Central Processing Unit (CPU) compiles a source code corresponding to a program code to be processed in an on-line production environment, for example, performance parameters, which is not limited to this.
Optionally, in some embodiments, as shown in fig. 2, fig. 2 is a schematic diagram according to a second embodiment of the present application, before acquiring the program code to be processed, further including:
the embodiment may be performed by data collection configured integrally in the generation apparatus of the program code, which is not limited thereto.
S201: and determining the number of sampling instances respectively corresponding to the CPU in the plurality of candidates.
For example, a sampling instance on a deployment line may be selected first, and then a timed sampling task may be deployed on the selected sampling instance.
Specific examples thereof include: firstly, dividing the requirement of the on-line performance sampling data size by the on-line single sampling example acquisition data size to obtain the number N of on-line sampling examples, and then selecting N examples with CPU model ratios according to the on-line whole-machine-room central processing unit CPU model distribution.
S202: a target CPU is determined from among the plurality of candidate CPUs based on the number of sampling instances.
As shown in fig. 3, fig. 3 is a schematic diagram of on-line CPU model distribution in the embodiment of the present application, and a horizontal axis indicates the number of on-line acquisition instances occupied by CPUs of corresponding models, and then, the candidate CPUs ranked N top in number of acquisition instances may be selected according to the on-line CPU model distribution and serve as target CPUs.
S203: a timed sampling task is deployed on a sampling instance of the target CPU.
The number of the sampling instances corresponding to the central processing unit CPU in the plurality of candidates is determined, the timing sampling task is deployed on the sampling instance of the target CPU, the current time can be triggered to be detected in real time, the timing sampling task is executed when the current time reaches the timing time set by the timing sampling task, the compiling performance data of the target CPU when the program code to be processed is compiled is obtained from an online production environment, and the online compiling performance data is used as the online performance sampling data.
As shown in fig. 4, fig. 4 is a schematic diagram of on-line performance sampling data acquisition in the embodiment of the present application.
In the embodiment of the application, the number of the sampling instances corresponding to the CPU in the plurality of candidates is determined, the target CPU is determined from the plurality of candidate CPUs according to the number of the sampling instances, the timing sampling task is deployed on the sampling instance of the target CPU, the plurality of on-line performance sampling data of the on-line production environment are obtained through sampling, the on-line performance sampling data can be acquired from the on-line production environment, the on-line production environment can be fully utilized, the service deployment time is avoided, the whole acquisition time is shortened, the data acquisition and the data use are decoupled, and the optimization blocking is reduced.
In some other embodiments, after the performance sampling data on the plurality of lines is obtained, the performance sampling data on the lines may be used to perform feedback type compiling optimization processing on the program to be compiled.
For example, a step of deploying an acquisition task may be performed, that is, a timing sampling task is deployed on the selected sampling instance, the compilation performance data is acquired at a regular time every day, the sampling is performed once every day, all feedback compilation optimization on the day is based on the data, so that the sampling of the compilation performance data is isolated from an offline environment, and the feedback compilation optimization effects of non-real-time performance sampling data and real-time performance sampling data are basically consistent through experimental data verification, so that the whole delivery cycle duration of the offline compilation, the test and the online is effectively reduced.
In some other embodiments, the compiling performance data obtained when the target CPU compiles the program code to be processed from the online production environment may be stored after the compiling performance data on the line is used as the online performance sampling data, for example, the binary file corresponding to the compiling performance data on the instance may be transmitted to a specified storage device, where the storage device may be a local storage device or a third-party storage device.
Alternatively, in some other embodiments, the multiple on-line Performance sampling data may be obtained directly based on a Performance Event sampling tool (Perf), and of course, the multiple on-line Performance sampling data may also be obtained in any other possible manner, which is not limited in this respect.
The principle for a Performance Event sampling tool (Perf) is illustrated as follows:
the Perf tool is a performance analysis tool built in a Linux kernel source code tree, supports performance analysis aiming at relevant performance indexes of a processor and relevant performance indexes of an operating system on the basis of performance events based on the principle of event sampling, and can be used for searching performance bottlenecks and positioning hot spot codes.
The principle of Perf is as follows: an interrupt is generated on a CPU (on each core) at a fixed time interval, and the interrupt can show which process and which function are currently in, and then a statistical value is added to the corresponding process and function, so that the fact that the CPU has a few percent of time in a certain process or a certain function can be determined. Schematic diagram as shown in fig. 5, fig. 5 is a schematic diagram of the Perf tool, and it can be seen that this is a sampled pattern, and generally the more functions that are expected to run, the greater the chance of being hit by a clock interrupt, and thus presumably the higher the CPU occupancy of that function (or process). S102: and determining a target compiling stage and a target compiling optimization plug-in corresponding to the target compiling stage.
After the program code to be processed and the plurality of on-line performance sampling data are obtained, a target compiling stage and a target compiling optimization plug-in corresponding to the target compiling stage can be determined.
Wherein, it is assumed that the optional compilation stage comprises: preprocessing, compiling stage, assembling stage, linking stage, compiling, publishing and assembling, the target compiling stage may be a compiling stage which is determined according to actual compiling optimization requirements and is expected to be optimized, and the target compiling stage is, for example: compiling stage, linking stage, compiling, releasing and assembling, and then plug-in optimization aiming at compiling optimization of different compiling stages is supported in the embodiment of the application.
For example, in the embodiment of the present application, an optimization processing module may be configured in a generation apparatus of a program code, and the optimization processing module is used to implement that a compilation optimization stage is newly added after a normal compilation stage, and an optimization parameter is configured in the compilation optimization stage, and various optimization methods for different supported target compilation stages (compilation stage, linking stage, post-compilation, release assembly) are used, so that flexible configuration according to business needs is implemented, customized opening and closing is supported, and binary files for business services are optimized by using online performance sampling data.
After the target compiling stage and the target compiling optimization plug-in corresponding to the target compiling stage are determined, the target compiling optimization plug-in (for example, an optimization plug-in supporting the compiling stage, an optimization plug-in supporting the linking stage, an optimization plug-in supporting the compiling, and an optimization plug-in supporting the publishing and assembling) may be integrated with the optimization processing module in the embodiment of the present application, so as to support a one-click optimization processing logic.
First, a corresponding principle description is performed on an optimization tool adopted in the embodiment of the present application:
(1) in the embodiment of the present application, the processing logic of the Auto fdo may be subjected to plug-in processing to obtain an optimized plug-in supporting the compiling stage.
The AutoFDO uses a sample-based performance tool perf to collect program performance information and uses this data to determine the likelihood of a jump, thereby directing the feedback compilation optimization to generate an optimized program code (e.g., an optimized binary file, bin file).
(2) Link-time optimizations (LTO), that is, an optimization plug-in supporting the link phase, in an embodiment of the present application, a processing logic of the LTO may be subjected to plug-in processing to obtain an optimization plug-in supporting the link phase.
The LTO tool can optimize the binary process of the whole program in the link process and reduce the volume of the target code. The idea of LTO is: exploring any optimization opportunities that may arise by examining the entire program after compiling the individual files; the compiler may also more aggressively detect and eliminate code and data used, and the LTO puts the compiler intermediate representation into the object file when the source file is compiled. The actual LTO phase loads all the intermediate represented code into the image of the single core, overwriting the further optimized object code.
(3) In the embodiment of the present application, a processing logic of the BOLT tool may be subjected to plug-in processing to obtain an optimized plug-in supporting compilation.
The BOLT tool is an application program dynamic optimization scheme, collects program operation data through the perf tool, rearranges the symbols of the application program by using the collected data, and improves the cache hit rate of the instructions of the central processing unit so as to achieve the purpose of improving the performance of the program.
S103: and determining an optimization algorithm and conversion parameters corresponding to the target compiling optimization plug-in.
After the target compiling optimization plug-in (e.g., an optimization plug-in supporting a compiling stage, an optimization plug-in supporting a linking stage, an optimization plug-in supporting compiling, and an optimization plug-in supporting publishing and assembling) is integrated with the optimization processing module in the embodiment of the present application, an optimization algorithm and a conversion parameter corresponding to the target compiling optimization plug-in may be determined in the optimization processing process, and then, the corresponding optimization algorithm and conversion parameter are used to assist in optimizing the program code to be processed, so that the program code obtained by the optimization processing is used as the target program code.
S104: the plurality of on-line performance sample data are processed using the conversion parameters to obtain a plurality of target sample data.
After the optimization algorithm and the conversion parameters corresponding to the target compiling optimization plug-in are determined, the conversion parameters can be adopted to process a plurality of on-line performance sampling data to obtain a plurality of target sampling data, the process can be realized by a format conversion module which is integrated and arranged in a program code generating device, the format conversion module mainly provides a function of converting a profile data format corresponding to a configuration file of the on-line performance sampling data into another specified data format, and the converted data is stored and released for subsequent use.
Assume that the inputs to the format conversion module are: bin file, profile data format; the output is target sampling data in a new data format, and the new data format may be a data format directly called by an optimization algorithm, or may also be a data format to which a binary file to be compiled can be directly added, which is not limited to this.
By adopting the conversion parameters, the plurality of on-line performance sampling data can be processed in parallel to obtain a plurality of target sampling data, for example, concurrent conversion can be performed, the data conversion efficiency can be improved, and the application function of the format conversion module can be expanded.
That is, the format conversion module can support optimization upgrade and subsequent extension, and also support concurrent conversion: and calculating the concurrency number according to the input data, and performing concurrency conversion.
For example: an example of performing format processing on the performance sample data on the line for an optimization plug-in (auto fdo) supporting the compilation stage and performing format processing on the performance sample data on the line for an optimization plug-in (BOLT tool) supporting the compilation stage is shown in fig. 6, where fig. 6 is a schematic diagram of format processing in this embodiment of the present application.
S105: and processing the program code to be processed according to the corresponding optimization algorithm and the target sampling data to obtain a target program code.
The step may be executed by an optimization processing module integrally configured in a program code generation device, that is, the optimization processing module may process the program code to be processed according to the optimization algorithm corresponding to the determined target compilation optimization plug-in (e.g., an optimization plug-in supporting a compilation stage, an optimization plug-in supporting a link stage, an optimization plug-in supporting compilation, an optimization plug-in supporting release assembly, an optimization plug-in supporting target compilation optimization, or a combination of any optimization plug-ins) and the format-converted target sample data to obtain the target program code, thereby supporting plug-in type compilation optimization, effectively expanding an application scenario of compilation optimization, and improving flexibility of program compilation optimization.
For example, referring to fig. 7 together, fig. 7 is a schematic diagram of an optimization processing method in the embodiment of the present application.
(1) Unifying the interface: the optimization method comprises the steps of inputting target sampling data and a bin file (corresponding to a program code to be processed), inputting conversion parameters, and outputting the bin file (corresponding to the program code to be processed);
(2) the function of pre-fusion merge of multiple data sources and multiple copies of data is supported;
(3) during compilation, carrying specified input data to participate in the compilation, wherein GCC AutoFDO is-failure-profile or Clang PGO is-DLLVM _ PROFDATA _ FILE;
(4) during linking: setting a link parameter to participate in link optimization, wherein GCC: -flto or Clang: -DLLVM _ ENABLE _ LTO ═ Full;
(5) after compiling: target sample data in Bolt format and bin file (corresponding to program code to be processed) are input, Bolt tool calls conversion to generate new bin file (corresponding to program code).
The program code generation device in the embodiment of the application may further integrate and configure a publishing and assembling module, after the program code is optimized integrally, the publishing and assembling module publishes the program code to the temporary product library through a publishing component (custom data format cutting processing), and enters a next pipeline stage, and a subsequent pipeline performs performance testing, publishing and online based on the optimized target program code.
As shown in fig. 8, fig. 8 is a schematic block diagram of an integrated configuration of a device for generating a program code in the embodiment of the present application, and it should be noted that, the data acquisition module, the format conversion module, the optimization processing module, and the publishing and assembling module, which are integrally configured for the device for generating a program code, may select a part of modules to be integrated into an optimization service according to an actual optimization scenario requirement, so as to maximally utilize performance and cost benefits brought by a compiling optimization technology.
In the embodiment, by acquiring a program code to be processed and a plurality of online performance sampling data, determining a target compiling stage and a target compiling optimization plug-in corresponding to the target compiling stage, determining an optimization algorithm and a conversion parameter corresponding to the target compiling optimization plug-in, processing the plurality of online performance sampling data by using the conversion parameter to obtain a plurality of target sampling data, and processing the program code to be processed according to the corresponding optimization algorithm and the target sampling data to obtain a target program code, optimizing the program code to be processed by using the online performance sampling data, and generating and outputting the optimized target program code, the flexibility and timeliness of the program compiling optimization can be effectively improved, and the delivery cycle of compiling to the online can be effectively shortened.
Fig. 9 is a schematic diagram according to a third embodiment of the present application.
As shown in fig. 9, the program code generating method includes:
s901: and deleting the program code corresponding to the target file from the target program code.
In this embodiment, the target file may be determined first, and when the program service corresponding to the executable file is published in the online production environment, the calling of the program code in the target file can be skipped, that is, when the target file is a file that does not affect the online service corresponding to the target program code, the program code corresponding to the target file may be deleted from the target program code, so as to reduce the data volume of the target program code.
S902: and forming an executable file according to the target program codes after the deletion, wherein when the program service corresponding to the executable file is issued in the online production environment, the calling of the program codes in the target file can be skipped.
Then, an executable file can be formed according to the deleted object program code, and then the object program code is directly issued and run based on the executable file.
That is, considering that files such as a lib file and a single test which are irrelevant to online release and operation of a business service may exist in compiled and produced object program codes, so that a release package may be too large, taking a mobile phone hundred degrees as an example, one file is compiled during compiling, and when the objects are operated and released online, the static lib library and the single test directory file are not called, and the lib library and the single test directory file are deleted without any influence on the online business service, in this embodiment, program codes corresponding to the object files which have no influence on the online in the release package of the object program codes can be deleted and reduced, so that the size of the release package is reduced, and the online efficiency is improved.
In this embodiment, by deleting the program code corresponding to the target file from the target program code, the executable file is formed according to the deleted target program code, wherein when the program service corresponding to the executable file is published in the online production environment, the calling of the program code in the target file can be skipped, and the deletion processing of the program code corresponding to the target file having no influence on the thread in the publishing packet of the target program code can be realized, so as to reduce the size of the publishing packet, thereby improving the online efficiency.
As shown in fig. 10, fig. 10 is a schematic view of an application in the embodiment of the present application. The on-line performance sampling data is utilized, and the compiling period, the link period and the comprehensive optimizing technology after compiling are comprehensively utilized, so that the compiling speed and the running performance are improved to the maximum extent. The performance sampling tool is arranged in the Linux kernel source code tree and is irrelevant to the compiler, so that the performance sampling tool can be applied to various compilers, and a file directory which has no influence on the online publishing operation in the publishing packet is provided, so that the data volume of the publishing packet is reduced, and the online efficiency is effectively improved.
FIG. 11 is a schematic diagram of a fourth embodiment according to the present application.
As shown in fig. 11, the program code generation apparatus 110 includes:
an obtaining module 1101, configured to obtain a program code to be processed and a plurality of on-line performance sampling data;
a first determining module 1102, configured to determine a target compilation stage and a target compilation optimization plug-in corresponding to the target compilation stage;
a second determining module 1103, configured to determine an optimization algorithm and a transformation parameter corresponding to the target compilation optimization plug-in;
a conversion module 1104, configured to process the plurality of on-line performance sample data by using the conversion parameter to obtain a plurality of target sample data; and
the processing module 1105 is configured to process the program code to be processed according to the corresponding optimization algorithm and the target sampling data to obtain a target program code.
In some embodiments of the present application, the program code generating device 120 includes: an obtaining module 1201, a first determining module 1202, a second determining module 1203, a converting module 1204, and a processing module 1205, where the method further includes:
a third determining module 1206, configured to determine the number of sampling instances corresponding to the central processing units CPU in the multiple candidates respectively;
a fourth determining module 1207, configured to determine a target CPU from the multiple candidate CPUs according to the number of sampling instances;
a deployment module 1208, configured to deploy a timed sampling task on a sampling instance of the target CPU;
the obtaining module 1201 is specifically configured to:
and if the current time reaches the timing time set by the timing sampling task, executing the timing sampling task to acquire the compiling performance data when the target CPU compiles the program code to be processed from the online production environment, and taking the compiling performance data on line as the online performance sampling data.
In some embodiments of the present application, the processing module 1205 is further configured to:
and performing feedback type compiling optimization processing on the program to be compiled by adopting on-line performance sampling data.
In some embodiments of the present application, as shown in fig. 12, further comprising:
a deleting module 1209, configured to delete a program code corresponding to the target file from the target program code;
and a generating module 1210, configured to form an executable file according to the target program code after the deletion, where when the online production environment issues a program service corresponding to the executable file, the calling of the program code in the target file can be skipped.
In some embodiments of the present application, the conversion module 1204 is specifically configured to:
and processing the plurality of on-line performance sampling data in parallel by adopting the conversion parameters to obtain a plurality of target sampling data.
In some embodiments of the present application, the target compilation optimization plug-in includes one or more of the following optimization plug-ins: the system comprises an optimization plug-in supporting a compiling stage, an optimization plug-in supporting a linking stage, an optimization plug-in supporting compiled and an optimization plug-in supporting release and assembly.
It is understood that the program code generating apparatus 120 and the program code generating apparatus 110 in fig. 12 of this embodiment, the obtaining module 1201 and the obtaining module 1101 in the above embodiment, the first determining module 1202 and the first determining module 1102 in the above embodiment, the second determining module 1203 and the second determining module 1103 in the above embodiment, the converting module 1204 and the converting module 1104 in the above embodiment, and the processing module 1205 and the processing module 1105 in the above embodiment may have the same functions and structures.
It should be noted that the explanation of the program code generation method described above is also applicable to the program code generation apparatus of the present embodiment, and is not repeated here.
In the embodiment, by acquiring a program code to be processed and a plurality of online performance sampling data, determining a target compiling stage and a target compiling optimization plug-in corresponding to the target compiling stage, determining an optimization algorithm and a conversion parameter corresponding to the target compiling optimization plug-in, processing the plurality of online performance sampling data by using the conversion parameter to obtain a plurality of target sampling data, and processing the program code to be processed according to the corresponding optimization algorithm and the target sampling data to obtain a target program code, optimizing the program code to be processed by using the online performance sampling data, and generating and outputting the optimized target program code, the flexibility and timeliness of the program compiling optimization can be effectively improved, and the delivery cycle of compiling to the online can be effectively shortened.
There is also provided, in accordance with an embodiment of the present application, an electronic device, a readable storage medium, and a computer program product.
Fig. 13 is a block diagram of an electronic device for implementing a program code generation method according to an embodiment of the present application. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present application that are described and/or claimed herein.
As shown in fig. 13, the apparatus 1300 includes a computing unit 1301 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM)1302 or a computer program loaded from a storage unit 1308 into a Random Access Memory (RAM) 1303. In the RAM 1303, various programs and data necessary for the operation of the device 1300 can also be stored. The calculation unit 1301, the ROM 1302, and the RAM 1303 are connected to each other via a bus 1304. An input/output (I/O) interface 1305 is also connected to bus 1304.
A number of components in the device 1300 connect to the I/O interface 1305, including: an input unit 1306 such as a keyboard, a mouse, or the like; an output unit 1307 such as various types of displays, speakers, and the like; storage unit 1308, such as a magnetic disk, optical disk, or the like; and a communication unit 1309 such as a network card, modem, wireless communication transceiver, etc. The communication unit 1309 allows the device 1300 to exchange information/data with other devices through a computer network such as the internet and/or various telecommunication networks.
Computing unit 1301 may be a variety of general and/or special purpose processing components with processing and computing capabilities. Some examples of computing unit 1301 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, and so forth. The calculation unit 1301 executes the respective methods and processes described above, such as the generation method of the program code. For example, in some embodiments, the generation of the program code may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as storage unit 1308. In some embodiments, some or all of the computer program may be loaded onto and/or installed onto device 1300 via ROM 1302 and/or communications unit 1309. When the computer program is loaded into the RAM 1303 and executed by the computing unit 1301, one or more steps of the generation method of the program code described above may be performed. Alternatively, in other embodiments, the computing unit 1301 may be configured in any other suitable way (e.g. by means of firmware) to perform the generation method of the program code.
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
Program code for implementing the methods of the present application may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of this application, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), the internet, and blockchain networks.
The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The Server can be a cloud Server, also called a cloud computing Server or a cloud host, and is a host product in a cloud computing service system, so as to solve the defects of high management difficulty and weak service expansibility in the traditional physical host and VPS service ("Virtual Private Server", or simply "VPS"). The server may also be a server of a distributed system, or a server incorporating a blockchain.
It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present disclosure may be executed in parallel, sequentially, or in different orders, and are not limited herein as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved.
The above-described embodiments should not be construed as limiting the scope of the present application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims (15)

1. A method of generating program code, comprising:
acquiring a program code to be processed and a plurality of on-line performance sampling data;
determining a target compiling stage and a target compiling optimization plug-in corresponding to the target compiling stage;
determining an optimization algorithm and conversion parameters corresponding to the target compiling optimization plug-in;
processing the plurality of on-line performance sample data using the conversion parameters to obtain a plurality of target sample data; and
and processing the program code to be processed according to the corresponding optimization algorithm and the target sampling data to obtain a target program code.
2. The method of claim 1, prior to said obtaining pending program code, further comprising:
determining the number of sampling instances respectively corresponding to the CPU in the plurality of candidates;
determining a target CPU from the plurality of candidate CPUs according to the number of sampling instances;
deploying a timing sampling task on a sampling instance of the target CPU;
wherein, obtain a plurality of on-line performance sample data, include:
and if the current time reaches the timing time set by the timing sampling task, executing the timing sampling task to acquire the compiling performance data when the target CPU compiles the program code to be processed from an online production environment, and taking the compiling performance data on the line as the online performance sampling data.
3. The method of claim 1, after acquiring the plurality of on-line performance sample data, further comprising:
and performing feedback type compiling optimization processing on the program to be compiled by adopting the on-line performance sampling data.
4. The method of claim 1, further comprising, after said processing the program code to be processed according to the corresponding optimization algorithm and the target sample data to obtain target program code:
deleting the program code corresponding to the target file from the target program code;
forming an executable file according to the deleted target program code;
when the program service corresponding to the executable file is issued in the online production environment, the calling of the program code in the target file can be skipped.
5. The method of claim 1, wherein said processing the plurality of on-line performance sample data using the conversion parameters to obtain a plurality of target sample data comprises:
and processing the plurality of on-line performance sampling data in parallel by adopting the conversion parameters to obtain a plurality of target sampling data.
6. The method of claim 1, the target compilation optimization plug-in, comprising one or more of the following optimization plug-ins:
the system comprises an optimization plug-in supporting a compiling stage, an optimization plug-in supporting a linking stage, an optimization plug-in supporting compiled and an optimization plug-in supporting release and assembly.
7. An apparatus for generating program code, comprising:
the system comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring a program code to be processed and a plurality of on-line performance sampling data;
the first determination module is used for determining a target compiling stage and a target compiling optimization plug-in corresponding to the target compiling stage;
the second determination module is used for determining an optimization algorithm and a conversion parameter corresponding to the target compiling optimization plug-in;
the conversion module is used for processing the on-line performance sampling data by adopting the conversion parameters to obtain a plurality of target sampling data; and
and the processing module is used for processing the program code to be processed according to the corresponding optimization algorithm and the target sampling data to obtain a target program code.
8. The apparatus of claim 7, further comprising:
the third determining module is used for determining the number of sampling instances corresponding to the central processing unit CPU in the plurality of candidates;
a fourth determining module, configured to determine a target CPU from among the plurality of candidate CPUs, according to the number of sampling instances;
the deployment module is used for deploying a timing sampling task on the sampling instance of the target CPU;
the obtaining module is specifically configured to:
and if the current time reaches the timing time set by the timing sampling task, executing the timing sampling task to acquire the compiling performance data when the target CPU compiles the program code to be processed from an online production environment, and taking the compiling performance data on the line as the online performance sampling data.
9. The apparatus of claim 7, the processing module further to:
and performing feedback type compiling optimization processing on the program to be compiled by adopting the on-line performance sampling data.
10. The apparatus of claim 7, further comprising:
the deleting module is used for deleting the program code corresponding to the target file from the target program code;
and the generation module is used for forming an executable file according to the deleted target program code, wherein when the program service corresponding to the executable file is issued in an online production environment, the calling of the program code in the target file can be skipped.
11. The apparatus of claim 7, wherein the conversion module is specifically configured to:
and processing the plurality of on-line performance sampling data in parallel by adopting the conversion parameters to obtain a plurality of target sampling data.
12. The apparatus of claim 7, the target compilation optimization plug-in, comprising one or more of the following optimization plug-ins:
the system comprises an optimization plug-in supporting a compiling stage, an optimization plug-in supporting a linking stage, an optimization plug-in supporting compiled and an optimization plug-in supporting release and assembly.
13. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-6.
14. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-6.
15. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1-6.
CN202011563691.8A 2020-12-25 2020-12-25 Program code generation method and device, electronic equipment and storage medium Active CN112540767B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011563691.8A CN112540767B (en) 2020-12-25 2020-12-25 Program code generation method and device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011563691.8A CN112540767B (en) 2020-12-25 2020-12-25 Program code generation method and device, electronic equipment and storage medium

Publications (2)

Publication Number Publication Date
CN112540767A true CN112540767A (en) 2021-03-23
CN112540767B CN112540767B (en) 2023-07-25

Family

ID=75018174

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011563691.8A Active CN112540767B (en) 2020-12-25 2020-12-25 Program code generation method and device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN112540767B (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113656042A (en) * 2021-08-24 2021-11-16 北京奇艺世纪科技有限公司 Code generation method and device, electronic equipment and storage medium
CN114924808A (en) * 2022-05-12 2022-08-19 中国电子科技集团公司第二十九研究所 SRAM type FPGA on-orbit reliable loading method based on duplicate storage program
CN117035065A (en) * 2023-10-10 2023-11-10 浙江大华技术股份有限公司 Model evaluation method and related device

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040098713A1 (en) * 2002-07-03 2004-05-20 Hajime Ogawa Compiler apparatus with flexible optimization
CN101551748A (en) * 2009-01-21 2009-10-07 北京海尔集成电路设计有限公司 Optimized compiling method
US20110055819A1 (en) * 2009-08-27 2011-03-03 International Business Machines Corporation System and Method for Optimizing Compiler Performance by Object Collocation
CN102622260A (en) * 2012-02-27 2012-08-01 中国科学院计算技术研究所 Optimization method and optimization system of on-line iteration compiling
CN105988855A (en) * 2015-02-16 2016-10-05 龙芯中科技术有限公司 Just-in-time compilation parameter optimizing method and device
CN109960507A (en) * 2017-12-14 2019-07-02 广东欧珀移动通信有限公司 Compile optimization method, device, storage medium, intelligent terminal and server
CN110780879A (en) * 2019-10-29 2020-02-11 南京星环智能科技有限公司 Decision execution method, device, equipment and medium based on intelligent compiling technology
CN110941424A (en) * 2019-11-28 2020-03-31 Oppo广东移动通信有限公司 Compiling parameter optimization method and device and electronic equipment
CN111061483A (en) * 2019-11-28 2020-04-24 Oppo广东移动通信有限公司 Compiling optimization method and device and electronic equipment
CN111949272A (en) * 2020-07-30 2020-11-17 百度在线网络技术(北京)有限公司 Compilation optimization method and device for hosted application, electronic device and readable storage medium

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040098713A1 (en) * 2002-07-03 2004-05-20 Hajime Ogawa Compiler apparatus with flexible optimization
CN101551748A (en) * 2009-01-21 2009-10-07 北京海尔集成电路设计有限公司 Optimized compiling method
US20110055819A1 (en) * 2009-08-27 2011-03-03 International Business Machines Corporation System and Method for Optimizing Compiler Performance by Object Collocation
CN102622260A (en) * 2012-02-27 2012-08-01 中国科学院计算技术研究所 Optimization method and optimization system of on-line iteration compiling
CN105988855A (en) * 2015-02-16 2016-10-05 龙芯中科技术有限公司 Just-in-time compilation parameter optimizing method and device
CN109960507A (en) * 2017-12-14 2019-07-02 广东欧珀移动通信有限公司 Compile optimization method, device, storage medium, intelligent terminal and server
CN110780879A (en) * 2019-10-29 2020-02-11 南京星环智能科技有限公司 Decision execution method, device, equipment and medium based on intelligent compiling technology
CN110941424A (en) * 2019-11-28 2020-03-31 Oppo广东移动通信有限公司 Compiling parameter optimization method and device and electronic equipment
CN111061483A (en) * 2019-11-28 2020-04-24 Oppo广东移动通信有限公司 Compiling optimization method and device and electronic equipment
CN111949272A (en) * 2020-07-30 2020-11-17 百度在线网络技术(北京)有限公司 Compilation optimization method and device for hosted application, electronic device and readable storage medium

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113656042A (en) * 2021-08-24 2021-11-16 北京奇艺世纪科技有限公司 Code generation method and device, electronic equipment and storage medium
CN113656042B (en) * 2021-08-24 2023-07-21 北京奇艺世纪科技有限公司 Code generation method and device, electronic equipment and storage medium
CN114924808A (en) * 2022-05-12 2022-08-19 中国电子科技集团公司第二十九研究所 SRAM type FPGA on-orbit reliable loading method based on duplicate storage program
CN117035065A (en) * 2023-10-10 2023-11-10 浙江大华技术股份有限公司 Model evaluation method and related device

Also Published As

Publication number Publication date
CN112540767B (en) 2023-07-25

Similar Documents

Publication Publication Date Title
CN112540767B (en) Program code generation method and device, electronic equipment and storage medium
CN113010181B (en) Deployment method and device of operators in deep learning framework and electronic equipment
CN109086215B (en) Embedded software unit test case generation method and system
CN112764755B (en) Code conversion method, device, equipment and storage medium
CN113283613B (en) Deep learning model generation method, optimization method, device, equipment and medium
JP7344259B2 (en) Pattern transformation methods, apparatus, electronic devices, computer storage media and computer program products in deep learning frameworks
CN112925587A (en) Method and apparatus for initializing applications
CN110780879A (en) Decision execution method, device, equipment and medium based on intelligent compiling technology
CN112506602A (en) Page generation method and device, electronic equipment and computer readable medium
CN111767217A (en) JS unit test case generation method and device
US20150227350A1 (en) Multi-dimensional, multi-configuration compilation phase output visualization technique
CN115809063A (en) Storage process compiling method, system, electronic equipment and storage medium
CN112633502B (en) Cross-platform execution method and device of deep learning model and electronic equipment
CN116560666B (en) AI front end unified computing method, device and medium based on multi-level code generation
CN104731705A (en) Dirty data propagation path finding method based on complex network
WO2022035476A1 (en) Representing asynchronous state machine in intermediate code
CN115563183B (en) Query method, query device and program product
WO2015014309A1 (en) Compiler and method for global-scope basic-block reordering
CN113031962B (en) Compiling method, compiling apparatus, electronic device, storage medium, and program product
EP4068141B1 (en) Method and system to enable print functionality in high-level synthesis (hls) design platforms
KR20130088285A (en) Data processing system and method of data simulation
CN114880357A (en) Source code information retrieval method and device, electronic equipment, storage medium and product
CN114327772A (en) Code injection method, device, equipment and storage medium
CN115145627A (en) Method, device, equipment and storage medium for building local operation environment of plug-in
CN114756238A (en) Dynamic compiling method, device, equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant